2024-05-20T15:52:30.357 INFO:root:teuthology version: 0.0.1.dev287+gb9e3da8 2024-05-20T15:52:30.357 DEBUG:teuthology.run:Teuthology command: teuthology --verbose --archive /home/teuthworker/archive/pdonnell-2024-05-20_15:44:59-fs-wip-pdonnell-testing-20240517.203521-squid-debug-distro-default-smithi/7715545 --name pdonnell-2024-05-20_15:44:59-fs-wip-pdonnell-testing-20240517.203521-squid-debug-distro-default-smithi --owner scheduled_pdonnell@teuthology --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/no 3-snaps/no 4-flush/yes 5-workunit/suites/iogen}} -- /home/teuthworker/archive/pdonnell-2024-05-20_15:44:59-fs-wip-pdonnell-testing-20240517.203521-squid-debug-distro-default-smithi/7715545/orig.config.yaml 2024-05-20T15:52:30.404 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-05-20T15:52:30.581 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/pdonnell-2024-05-20_15:44:59-fs-wip-pdonnell-testing-20240517.203521-squid-debug-distro-default-smithi/7715545 branch: wip-pdonnell-testing-20240517.203521-squid-debug description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/no 3-snaps/no 4-flush/yes 5-workunit/suites/iogen}} email: pdonnell@redhat.com first_in_suite: false job_id: '7715545' kernel: client: sha1: distro kdb: 1 sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: pdonnell-2024-05-20_15:44:59-fs-wip-pdonnell-testing-20240517.203521-squid-debug-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-pdonnell-testing-20240517.203521-squid-debug ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd max_mds: 3 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_export_ephemeral_random_max: 0.1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - MON_DOWN - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability sha1: d2b507c724ef6c8295ee0a35d185cdab167fd61d subvols: create: 2 subvol_options: --namespace-isolated ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported dry_run: true install: ceph: flavor: default sha1: d2b507c724ef6c8295ee0a35d185cdab167fd61d kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=crc - wsync syntax: v2 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-pdonnell-testing-20240517.203521-squid-debug sha1: d2b507c724ef6c8295ee0a35d185cdab167fd61d owner: scheduled_pdonnell@teuthology priority: 75 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 9749 sha1: d2b507c724ef6c8295ee0a35d185cdab167fd61d sleep_before_teardown: 0 subset: 272/512 suite: fs suite_branch: wip-pdonnell-testing-20240517.203521-squid-debug suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_d2b507c724ef6c8295ee0a35d185cdab167fd61d/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: d2b507c724ef6c8295ee0a35d185cdab167fd61d syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi082.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBESJq51B+K77h8dqkgs/UO9YYBNXG6w0vgQ5n5/3ye+hrbn6OPoptgneVo0ZvXZqoGgIr9ADiQ3FwKRu6VRQni0= smithi160.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBF5KYQa13XI+kzrH/7gx+3dbF26rswJouFoNaTvIyNzYAHQZNgxmLd+2NXA/9LdmUVXktAJMdLpH5U5kdpBOwmg= smithi184.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHDWmHe+6G1zSASxKUuK5kvRfg1P2A0ub0PLcvLtsgqi+Ig/n2KRpQjt7RGRRcCZaoniVWYKGV/dh5I+4ZorSBw= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate false - ceph fs subvolumegroup pin cephfs qa random 0.10 - check-counter: {} - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - workunit: clients: all: - suites/iogen.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: b9e3da87b1ec7920937cb4902b7e3fd9eda86b4a timestamp: 2024-05-20_15:44:59 tube: smithi user: pdonnell verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2698841 2024-05-20T15:52:30.581 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_d2b507c724ef6c8295ee0a35d185cdab167fd61d/qa; will attempt to use it 2024-05-20T15:52:30.582 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_d2b507c724ef6c8295ee0a35d185cdab167fd61d/qa/tasks 2024-05-20T15:52:30.582 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-05-20T15:52:30.583 INFO:teuthology.task.internal:Checking packages... 2024-05-20T15:52:30.601 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'd2b507c724ef6c8295ee0a35d185cdab167fd61d' 2024-05-20T15:52:30.601 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-05-20T15:52:30.601 INFO:teuthology.packaging:ref: None 2024-05-20T15:52:30.601 INFO:teuthology.packaging:tag: None 2024-05-20T15:52:30.601 INFO:teuthology.packaging:branch: wip-pdonnell-testing-20240517.203521-squid-debug 2024-05-20T15:52:30.601 INFO:teuthology.packaging:sha1: d2b507c724ef6c8295ee0a35d185cdab167fd61d 2024-05-20T15:52:30.601 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-pdonnell-testing-20240517.203521-squid-debug 2024-05-20T15:52:30.744 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-2744.gd2b507c7 2024-05-20T15:52:30.745 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-05-20T15:52:30.752 INFO:teuthology.task.internal:no buildpackages task found 2024-05-20T15:52:30.752 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-05-20T15:52:30.765 INFO:teuthology.task.internal:Saving configuration 2024-05-20T15:52:30.783 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-05-20T15:52:30.790 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-05-20T15:52:30.812 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi082.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-05-20_15:44:59-fs-wip-pdonnell-testing-20240517.203521-squid-debug-distro-default-smithi/7715545', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-20 15:47:18.230539', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBESJq51B+K77h8dqkgs/UO9YYBNXG6w0vgQ5n5/3ye+hrbn6OPoptgneVo0ZvXZqoGgIr9ADiQ3FwKRu6VRQni0='} 2024-05-20T15:52:30.835 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi160.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-05-20_15:44:59-fs-wip-pdonnell-testing-20240517.203521-squid-debug-distro-default-smithi/7715545', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-20 15:47:18.233603', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBF5KYQa13XI+kzrH/7gx+3dbF26rswJouFoNaTvIyNzYAHQZNgxmLd+2NXA/9LdmUVXktAJMdLpH5U5kdpBOwmg='} 2024-05-20T15:52:30.855 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi184.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-05-20_15:44:59-fs-wip-pdonnell-testing-20240517.203521-squid-debug-distro-default-smithi/7715545', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-20 15:47:18.232249', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHDWmHe+6G1zSASxKUuK5kvRfg1P2A0ub0PLcvLtsgqi+Ig/n2KRpQjt7RGRRcCZaoniVWYKGV/dh5I+4ZorSBw='} 2024-05-20T15:52:30.855 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-05-20T15:52:30.861 INFO:teuthology.task.internal:roles: ubuntu@smithi082.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-05-20T15:52:30.861 INFO:teuthology.task.internal:roles: ubuntu@smithi160.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-05-20T15:52:30.861 INFO:teuthology.task.internal:roles: ubuntu@smithi184.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-05-20T15:52:30.861 INFO:teuthology.run_tasks:Running task console_log... 2024-05-20T15:52:30.940 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fbc840b7ac0>, signals=[15]) 2024-05-20T15:52:30.940 INFO:teuthology.run_tasks:Running task internal.connect... 2024-05-20T15:52:30.946 INFO:teuthology.task.internal:Opening connections... 2024-05-20T15:52:30.946 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi082.front.sepia.ceph.com 2024-05-20T15:52:30.947 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi082.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-20T15:52:31.023 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi160.front.sepia.ceph.com 2024-05-20T15:52:31.024 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi160.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-20T15:52:31.099 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi184.front.sepia.ceph.com 2024-05-20T15:52:31.100 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi184.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-20T15:52:31.173 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-05-20T15:52:31.180 DEBUG:teuthology.orchestra.run.smithi082:> uname -m 2024-05-20T15:52:31.198 INFO:teuthology.orchestra.run.smithi082.stdout:x86_64 2024-05-20T15:52:31.198 DEBUG:teuthology.orchestra.run.smithi082:> cat /etc/os-release 2024-05-20T15:52:31.255 INFO:teuthology.orchestra.run.smithi082.stdout:NAME="CentOS Stream" 2024-05-20T15:52:31.255 INFO:teuthology.orchestra.run.smithi082.stdout:VERSION="9" 2024-05-20T15:52:31.255 INFO:teuthology.orchestra.run.smithi082.stdout:ID="centos" 2024-05-20T15:52:31.255 INFO:teuthology.orchestra.run.smithi082.stdout:ID_LIKE="rhel fedora" 2024-05-20T15:52:31.255 INFO:teuthology.orchestra.run.smithi082.stdout:VERSION_ID="9" 2024-05-20T15:52:31.255 INFO:teuthology.orchestra.run.smithi082.stdout:PLATFORM_ID="platform:el9" 2024-05-20T15:52:31.255 INFO:teuthology.orchestra.run.smithi082.stdout:PRETTY_NAME="CentOS Stream 9" 2024-05-20T15:52:31.255 INFO:teuthology.orchestra.run.smithi082.stdout:ANSI_COLOR="0;31" 2024-05-20T15:52:31.255 INFO:teuthology.orchestra.run.smithi082.stdout:LOGO="fedora-logo-icon" 2024-05-20T15:52:31.255 INFO:teuthology.orchestra.run.smithi082.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-05-20T15:52:31.255 INFO:teuthology.orchestra.run.smithi082.stdout:HOME_URL="https://centos.org/" 2024-05-20T15:52:31.255 INFO:teuthology.orchestra.run.smithi082.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-20T15:52:31.256 INFO:teuthology.orchestra.run.smithi082.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-05-20T15:52:31.256 INFO:teuthology.orchestra.run.smithi082.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-05-20T15:52:31.256 INFO:teuthology.lock.ops:Updating smithi082.front.sepia.ceph.com on lock server 2024-05-20T15:52:31.279 DEBUG:teuthology.orchestra.run.smithi160:> uname -m 2024-05-20T15:52:31.296 INFO:teuthology.orchestra.run.smithi160.stdout:x86_64 2024-05-20T15:52:31.296 DEBUG:teuthology.orchestra.run.smithi160:> cat /etc/os-release 2024-05-20T15:52:31.354 INFO:teuthology.orchestra.run.smithi160.stdout:NAME="CentOS Stream" 2024-05-20T15:52:31.354 INFO:teuthology.orchestra.run.smithi160.stdout:VERSION="9" 2024-05-20T15:52:31.354 INFO:teuthology.orchestra.run.smithi160.stdout:ID="centos" 2024-05-20T15:52:31.355 INFO:teuthology.orchestra.run.smithi160.stdout:ID_LIKE="rhel fedora" 2024-05-20T15:52:31.355 INFO:teuthology.orchestra.run.smithi160.stdout:VERSION_ID="9" 2024-05-20T15:52:31.355 INFO:teuthology.orchestra.run.smithi160.stdout:PLATFORM_ID="platform:el9" 2024-05-20T15:52:31.355 INFO:teuthology.orchestra.run.smithi160.stdout:PRETTY_NAME="CentOS Stream 9" 2024-05-20T15:52:31.355 INFO:teuthology.orchestra.run.smithi160.stdout:ANSI_COLOR="0;31" 2024-05-20T15:52:31.355 INFO:teuthology.orchestra.run.smithi160.stdout:LOGO="fedora-logo-icon" 2024-05-20T15:52:31.355 INFO:teuthology.orchestra.run.smithi160.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-05-20T15:52:31.355 INFO:teuthology.orchestra.run.smithi160.stdout:HOME_URL="https://centos.org/" 2024-05-20T15:52:31.355 INFO:teuthology.orchestra.run.smithi160.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-20T15:52:31.355 INFO:teuthology.orchestra.run.smithi160.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-05-20T15:52:31.355 INFO:teuthology.orchestra.run.smithi160.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-05-20T15:52:31.355 INFO:teuthology.lock.ops:Updating smithi160.front.sepia.ceph.com on lock server 2024-05-20T15:52:31.387 DEBUG:teuthology.orchestra.run.smithi184:> uname -m 2024-05-20T15:52:31.405 INFO:teuthology.orchestra.run.smithi184.stdout:x86_64 2024-05-20T15:52:31.405 DEBUG:teuthology.orchestra.run.smithi184:> cat /etc/os-release 2024-05-20T15:52:31.461 INFO:teuthology.orchestra.run.smithi184.stdout:NAME="CentOS Stream" 2024-05-20T15:52:31.461 INFO:teuthology.orchestra.run.smithi184.stdout:VERSION="9" 2024-05-20T15:52:31.461 INFO:teuthology.orchestra.run.smithi184.stdout:ID="centos" 2024-05-20T15:52:31.461 INFO:teuthology.orchestra.run.smithi184.stdout:ID_LIKE="rhel fedora" 2024-05-20T15:52:31.461 INFO:teuthology.orchestra.run.smithi184.stdout:VERSION_ID="9" 2024-05-20T15:52:31.462 INFO:teuthology.orchestra.run.smithi184.stdout:PLATFORM_ID="platform:el9" 2024-05-20T15:52:31.462 INFO:teuthology.orchestra.run.smithi184.stdout:PRETTY_NAME="CentOS Stream 9" 2024-05-20T15:52:31.462 INFO:teuthology.orchestra.run.smithi184.stdout:ANSI_COLOR="0;31" 2024-05-20T15:52:31.462 INFO:teuthology.orchestra.run.smithi184.stdout:LOGO="fedora-logo-icon" 2024-05-20T15:52:31.462 INFO:teuthology.orchestra.run.smithi184.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-05-20T15:52:31.462 INFO:teuthology.orchestra.run.smithi184.stdout:HOME_URL="https://centos.org/" 2024-05-20T15:52:31.462 INFO:teuthology.orchestra.run.smithi184.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-20T15:52:31.462 INFO:teuthology.orchestra.run.smithi184.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-05-20T15:52:31.462 INFO:teuthology.orchestra.run.smithi184.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-05-20T15:52:31.462 INFO:teuthology.lock.ops:Updating smithi184.front.sepia.ceph.com on lock server 2024-05-20T15:52:31.495 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-05-20T15:52:31.505 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-05-20T15:52:31.511 INFO:teuthology.task.internal:Checking for old test directory... 2024-05-20T15:52:31.511 DEBUG:teuthology.orchestra.run.smithi082:> test '!' -e /home/ubuntu/cephtest 2024-05-20T15:52:31.514 DEBUG:teuthology.orchestra.run.smithi160:> test '!' -e /home/ubuntu/cephtest 2024-05-20T15:52:31.516 DEBUG:teuthology.orchestra.run.smithi184:> test '!' -e /home/ubuntu/cephtest 2024-05-20T15:52:31.532 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-05-20T15:52:31.538 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-05-20T15:52:31.538 DEBUG:teuthology.orchestra.run.smithi082:> test -z $(ls -A /var/lib/ceph) 2024-05-20T15:52:31.571 DEBUG:teuthology.orchestra.run.smithi160:> test -z $(ls -A /var/lib/ceph) 2024-05-20T15:52:31.573 DEBUG:teuthology.orchestra.run.smithi184:> test -z $(ls -A /var/lib/ceph) 2024-05-20T15:52:31.667 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-05-20T15:52:31.737 INFO:teuthology.run_tasks:Running task kernel... 2024-05-20T15:52:31.750 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': 1, 'sha1': 'distro'} 2024-05-20T15:52:31.750 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}} 2024-05-20T15:52:31.750 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}}, timeout 300 2024-05-20T15:52:31.750 DEBUG:teuthology.orchestra.run.smithi082:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-20T15:52:31.767 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-20T15:52:31.767 DEBUG:teuthology.orchestra.run.smithi082:> uname -r 2024-05-20T15:52:31.824 INFO:teuthology.orchestra.run.smithi082.stdout:5.14.0-437.el9.x86_64 2024-05-20T15:52:31.824 INFO:teuthology.task.kernel:Running kernel on smithi082: 5.14.0-437.el9.x86_64 2024-05-20T15:52:31.824 DEBUG:teuthology.orchestra.run.smithi082:> sudo yum install -y kernel 2024-05-20T15:52:34.867 INFO:teuthology.orchestra.run.smithi082.stdout:CentOS Stream 9 - BaseOS 5.8 MB/s | 8.1 MB 00:01 2024-05-20T15:52:37.635 INFO:teuthology.orchestra.run.smithi082.stdout:CentOS Stream 9 - AppStream 11 MB/s | 19 MB 00:01 2024-05-20T15:52:42.293 INFO:teuthology.orchestra.run.smithi082.stdout:CentOS Stream 9 - CRB 6.1 MB/s | 6.2 MB 00:01 2024-05-20T15:52:43.927 INFO:teuthology.orchestra.run.smithi082.stdout:CentOS Stream 9 - Extras packages 53 kB/s | 16 kB 00:00 2024-05-20T15:52:47.354 INFO:teuthology.orchestra.run.smithi082.stdout:Extra Packages for Enterprise Linux 6.4 MB/s | 21 MB 00:03 2024-05-20T15:52:52.544 INFO:teuthology.orchestra.run.smithi082.stdout:lab-extras 12 kB/s | 1.7 kB 00:00 2024-05-20T15:52:54.500 INFO:teuthology.orchestra.run.smithi082.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-05-20T15:52:54.500 INFO:teuthology.orchestra.run.smithi082.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-05-20T15:52:54.585 INFO:teuthology.orchestra.run.smithi082.stdout:Dependencies resolved. 2024-05-20T15:52:54.591 INFO:teuthology.orchestra.run.smithi082.stdout:================================================================================ 2024-05-20T15:52:54.591 INFO:teuthology.orchestra.run.smithi082.stdout: Package Architecture Version Repository Size 2024-05-20T15:52:54.591 INFO:teuthology.orchestra.run.smithi082.stdout:================================================================================ 2024-05-20T15:52:54.591 INFO:teuthology.orchestra.run.smithi082.stdout:Installing: 2024-05-20T15:52:54.591 INFO:teuthology.orchestra.run.smithi082.stdout: kernel x86_64 5.14.0-446.el9 baseos 550 k 2024-05-20T15:52:54.591 INFO:teuthology.orchestra.run.smithi082.stdout:Installing dependencies: 2024-05-20T15:52:54.591 INFO:teuthology.orchestra.run.smithi082.stdout: kernel-core x86_64 5.14.0-446.el9 baseos 15 M 2024-05-20T15:52:54.591 INFO:teuthology.orchestra.run.smithi082.stdout: kernel-modules x86_64 5.14.0-446.el9 baseos 34 M 2024-05-20T15:52:54.591 INFO:teuthology.orchestra.run.smithi082.stdout: kernel-modules-core x86_64 5.14.0-446.el9 baseos 28 M 2024-05-20T15:52:54.591 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-05-20T15:52:54.592 INFO:teuthology.orchestra.run.smithi082.stdout:Transaction Summary 2024-05-20T15:52:54.592 INFO:teuthology.orchestra.run.smithi082.stdout:================================================================================ 2024-05-20T15:52:54.592 INFO:teuthology.orchestra.run.smithi082.stdout:Install 4 Packages 2024-05-20T15:52:54.592 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-05-20T15:52:54.592 INFO:teuthology.orchestra.run.smithi082.stdout:Total download size: 78 M 2024-05-20T15:52:54.592 INFO:teuthology.orchestra.run.smithi082.stdout:Installed size: 123 M 2024-05-20T15:52:54.592 INFO:teuthology.orchestra.run.smithi082.stdout:Downloading Packages: 2024-05-20T15:52:55.812 INFO:teuthology.orchestra.run.smithi082.stdout:(1/4): kernel-5.14.0-446.el9.x86_64.rpm 469 kB/s | 550 kB 00:01 2024-05-20T15:52:56.820 INFO:teuthology.orchestra.run.smithi082.stdout:(2/4): kernel-core-5.14.0-446.el9.x86_64.rpm 7.1 MB/s | 15 MB 00:02 2024-05-20T15:52:57.712 INFO:teuthology.orchestra.run.smithi082.stdout:(3/4): kernel-modules-5.14.0-446.el9.x86_64.rpm 11 MB/s | 34 MB 00:03 2024-05-20T15:52:58.495 INFO:teuthology.orchestra.run.smithi082.stdout:(4/4): kernel-modules-core-5.14.0-446.el9.x86_6 10 MB/s | 28 MB 00:02 2024-05-20T15:52:58.496 INFO:teuthology.orchestra.run.smithi082.stdout:-------------------------------------------------------------------------------- 2024-05-20T15:52:58.496 INFO:teuthology.orchestra.run.smithi082.stdout:Total 20 MB/s | 78 MB 00:03 2024-05-20T15:52:59.178 INFO:teuthology.orchestra.run.smithi082.stdout:Running transaction check 2024-05-20T15:52:59.525 INFO:teuthology.orchestra.run.smithi082.stdout:Transaction check succeeded. 2024-05-20T15:52:59.526 INFO:teuthology.orchestra.run.smithi082.stdout:Running transaction test 2024-05-20T15:53:00.468 INFO:teuthology.orchestra.run.smithi082.stdout:Transaction test succeeded. 2024-05-20T15:53:00.468 INFO:teuthology.orchestra.run.smithi082.stdout:Running transaction 2024-05-20T15:53:02.848 INFO:teuthology.orchestra.run.smithi082.stdout: Preparing : 1/1 2024-05-20T15:53:03.474 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : kernel-modules-core-5.14.0-446.el9.x86_64 1/4 2024-05-20T15:53:03.562 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : kernel-core-5.14.0-446.el9.x86_64 2/4 2024-05-20T15:53:06.213 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: kernel-core-5.14.0-446.el9.x86_64 2/4 2024-05-20T15:53:06.373 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : kernel-modules-5.14.0-446.el9.x86_64 3/4 2024-05-20T15:53:11.199 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: kernel-modules-5.14.0-446.el9.x86_64 3/4 2024-05-20T15:53:11.409 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : kernel-5.14.0-446.el9.x86_64 4/4 2024-05-20T15:53:16.132 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: kernel-modules-core-5.14.0-446.el9.x86_64 4/4 2024-05-20T15:53:45.597 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: kernel-core-5.14.0-446.el9.x86_64 4/4 2024-05-20T15:53:45.636 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: kernel-modules-5.14.0-446.el9.x86_64 4/4 2024-05-20T15:53:46.006 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: kernel-5.14.0-446.el9.x86_64 4/4 2024-05-20T15:53:46.006 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : kernel-5.14.0-446.el9.x86_64 1/4 2024-05-20T15:53:46.006 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : kernel-core-5.14.0-446.el9.x86_64 2/4 2024-05-20T15:53:46.006 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : kernel-modules-5.14.0-446.el9.x86_64 3/4 2024-05-20T15:53:46.817 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : kernel-modules-core-5.14.0-446.el9.x86_64 4/4 2024-05-20T15:53:46.818 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-05-20T15:53:46.818 INFO:teuthology.orchestra.run.smithi082.stdout:Installed: 2024-05-20T15:53:46.818 INFO:teuthology.orchestra.run.smithi082.stdout: kernel-5.14.0-446.el9.x86_64 2024-05-20T15:53:46.818 INFO:teuthology.orchestra.run.smithi082.stdout: kernel-core-5.14.0-446.el9.x86_64 2024-05-20T15:53:46.818 INFO:teuthology.orchestra.run.smithi082.stdout: kernel-modules-5.14.0-446.el9.x86_64 2024-05-20T15:53:46.818 INFO:teuthology.orchestra.run.smithi082.stdout: kernel-modules-core-5.14.0-446.el9.x86_64 2024-05-20T15:53:46.818 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-05-20T15:53:46.818 INFO:teuthology.orchestra.run.smithi082.stdout:Complete! 2024-05-20T15:53:47.153 DEBUG:teuthology.orchestra.run.smithi082:> rpm -q kernel | sort -rV | head -n 1 2024-05-20T15:53:47.230 INFO:teuthology.orchestra.run.smithi082.stdout:kernel-5.14.0-446.el9.x86_64 2024-05-20T15:53:47.231 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-446.el9.x86_64 2024-05-20T15:53:47.231 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-437.el9.x86_64 Expected: 5.14.0-446.el9.x86_64 2024-05-20T15:53:47.231 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-05-20T15:53:47.231 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-05-20T15:53:47.231 INFO:teuthology.task.kernel:Installing distro kernel on client.0... 2024-05-20T15:53:47.231 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi082.front.sepia.ceph.com, path=None, version=distro) 2024-05-20T15:53:47.232 DEBUG:teuthology.orchestra.run.smithi082:> rpm -q kernel | sort -rV | head -n 1 2024-05-20T15:53:47.281 INFO:teuthology.orchestra.run.smithi082.stdout:kernel-5.14.0-446.el9.x86_64 2024-05-20T15:53:47.281 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-446.el9.x86_64 2024-05-20T15:53:47.281 DEBUG:teuthology.orchestra.run.smithi082:> sudo rpm -qi grub2-tools 2024-05-20T15:53:47.311 INFO:teuthology.orchestra.run.smithi082.stdout:Name : grub2-tools 2024-05-20T15:53:47.311 INFO:teuthology.orchestra.run.smithi082.stdout:Epoch : 1 2024-05-20T15:53:47.311 INFO:teuthology.orchestra.run.smithi082.stdout:Version : 2.06 2024-05-20T15:53:47.311 INFO:teuthology.orchestra.run.smithi082.stdout:Release : 68.el9 2024-05-20T15:53:47.311 INFO:teuthology.orchestra.run.smithi082.stdout:Architecture: x86_64 2024-05-20T15:53:47.311 INFO:teuthology.orchestra.run.smithi082.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-05-20T15:53:47.311 INFO:teuthology.orchestra.run.smithi082.stdout:Group : Unspecified 2024-05-20T15:53:47.311 INFO:teuthology.orchestra.run.smithi082.stdout:Size : 8264254 2024-05-20T15:53:47.311 INFO:teuthology.orchestra.run.smithi082.stdout:License : GPLv3+ 2024-05-20T15:53:47.311 INFO:teuthology.orchestra.run.smithi082.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-05-20T15:53:47.312 INFO:teuthology.orchestra.run.smithi082.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-05-20T15:53:47.312 INFO:teuthology.orchestra.run.smithi082.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-05-20T15:53:47.312 INFO:teuthology.orchestra.run.smithi082.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-05-20T15:53:47.312 INFO:teuthology.orchestra.run.smithi082.stdout:Packager : builder@centos.org 2024-05-20T15:53:47.312 INFO:teuthology.orchestra.run.smithi082.stdout:Vendor : CentOS 2024-05-20T15:53:47.312 INFO:teuthology.orchestra.run.smithi082.stdout:URL : http://www.gnu.org/software/grub/ 2024-05-20T15:53:47.312 INFO:teuthology.orchestra.run.smithi082.stdout:Summary : Support tools for GRUB. 2024-05-20T15:53:47.312 INFO:teuthology.orchestra.run.smithi082.stdout:Description : 2024-05-20T15:53:47.312 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-05-20T15:53:47.312 INFO:teuthology.orchestra.run.smithi082.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-05-20T15:53:47.312 INFO:teuthology.orchestra.run.smithi082.stdout:customizable bootloader with modular architecture. It supports a rich 2024-05-20T15:53:47.312 INFO:teuthology.orchestra.run.smithi082.stdout:variety of kernel formats, file systems, computer architectures and 2024-05-20T15:53:47.313 INFO:teuthology.orchestra.run.smithi082.stdout:hardware devices. 2024-05-20T15:53:47.313 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-05-20T15:53:47.313 INFO:teuthology.orchestra.run.smithi082.stdout:This subpackage provides tools for support of all platforms. 2024-05-20T15:53:47.314 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-05-20T15:53:47.314 INFO:teuthology.task.kernel:Updating grub on smithi082 to boot 5.14.0-446.el9.x86_64 2024-05-20T15:53:47.314 DEBUG:teuthology.orchestra.run.smithi082:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-05-20T15:53:47.945 INFO:teuthology.orchestra.run.smithi082.stderr:Generating grub configuration file ... 2024-05-20T15:53:49.313 INFO:teuthology.orchestra.run.smithi082.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-05-20T15:53:49.330 INFO:teuthology.orchestra.run.smithi082.stderr:done 2024-05-20T15:53:49.333 DEBUG:teuthology.orchestra.run.smithi082:> mktemp 2024-05-20T15:53:49.362 INFO:teuthology.orchestra.run.smithi082.stdout:/tmp/tmp.FQZSU2ZsRR 2024-05-20T15:53:49.362 DEBUG:teuthology.orchestra.run.smithi082:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.FQZSU2ZsRR 2024-05-20T15:53:49.428 DEBUG:teuthology.orchestra.run.smithi082:> sudo chmod 0666 /tmp/tmp.FQZSU2ZsRR 2024-05-20T15:53:49.562 DEBUG:teuthology.orchestra.remote:smithi082:/tmp/tmp.FQZSU2ZsRR is 6KB 2024-05-20T15:53:49.582 DEBUG:teuthology.orchestra.run.smithi082:> rm -fr /tmp/tmp.FQZSU2ZsRR 2024-05-20T15:53:49.596 DEBUG:teuthology.orchestra.run.smithi082:> sudo /bin/ls /boot/loader/entries || true 2024-05-20T15:53:49.657 INFO:teuthology.orchestra.run.smithi082.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-05-20T15:53:49.657 INFO:teuthology.orchestra.run.smithi082.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-05-20T15:53:49.657 INFO:teuthology.orchestra.run.smithi082.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-05-20T15:53:49.657 INFO:teuthology.orchestra.run.smithi082.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-446.el9.x86_64.conf 2024-05-20T15:53:49.659 DEBUG:teuthology.orchestra.run.smithi082:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-5.14.0-446.el9.x86_64 2024-05-20T15:53:49.809 DEBUG:teuthology.orchestra.run.smithi082:> sudo shutdown -r now 2024-05-20T15:54:19.812 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-05-20T15:54:19.812 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi082.front.sepia.ceph.com' 2024-05-20T15:54:19.813 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi082.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-20T15:54:38.238 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.82 2024-05-20T15:54:47.239 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi082.front.sepia.ceph.com' 2024-05-20T15:54:47.240 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi082.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-20T15:54:50.302 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.82 2024-05-20T15:55:02.303 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi082.front.sepia.ceph.com' 2024-05-20T15:55:02.304 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi082.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-20T15:55:34.034 DEBUG:teuthology.orchestra.run.smithi082:> true 2024-05-20T15:55:34.392 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi082.front.sepia.ceph.com' 2024-05-20T15:55:34.392 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "5.14.0-446.el9.x86_64"... 2024-05-20T15:55:34.392 DEBUG:teuthology.orchestra.run.smithi082:> uname -r 2024-05-20T15:55:34.447 INFO:teuthology.orchestra.run.smithi082.stdout:5.14.0-446.el9.x86_64 2024-05-20T15:55:34.447 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-446.el9.x86_64 vs 5.14.0-446.el9.x86_64 2024-05-20T15:55:34.447 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-05-20T15:55:34.447 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-05-20T15:55:35.449 DEBUG:teuthology.parallel:result is None 2024-05-20T15:55:35.449 INFO:teuthology.run_tasks:Running task internal.base... 2024-05-20T15:55:35.458 INFO:teuthology.task.internal:Creating test directory... 2024-05-20T15:55:35.458 DEBUG:teuthology.orchestra.run.smithi082:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-20T15:55:35.461 DEBUG:teuthology.orchestra.run.smithi160:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-20T15:55:35.463 DEBUG:teuthology.orchestra.run.smithi184:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-20T15:55:35.481 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-05-20T15:55:35.487 INFO:teuthology.run_tasks:Running task internal.archive... 2024-05-20T15:55:35.496 INFO:teuthology.task.internal:Creating archive directory... 2024-05-20T15:55:35.496 DEBUG:teuthology.orchestra.run.smithi082:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-20T15:55:35.518 DEBUG:teuthology.orchestra.run.smithi160:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-20T15:55:35.521 DEBUG:teuthology.orchestra.run.smithi184:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-20T15:55:35.555 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-05-20T15:55:35.561 INFO:teuthology.task.internal:Enabling coredump saving... 2024-05-20T15:55:35.561 DEBUG:teuthology.orchestra.run.smithi160:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-20T15:55:35.602 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-20T15:55:35.602 DEBUG:teuthology.orchestra.run.smithi184:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-20T15:55:35.618 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-20T15:55:35.618 DEBUG:teuthology.orchestra.run.smithi082:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-20T15:55:35.621 DEBUG:teuthology.orchestra.run.smithi160:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-20T15:55:35.645 DEBUG:teuthology.orchestra.run.smithi184:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-20T15:55:35.674 INFO:teuthology.orchestra.run.smithi160.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-20T15:55:35.691 INFO:teuthology.orchestra.run.smithi082.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-20T15:55:35.693 INFO:teuthology.orchestra.run.smithi184.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-20T15:55:35.705 INFO:teuthology.orchestra.run.smithi160.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-20T15:55:35.713 INFO:teuthology.orchestra.run.smithi184.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-20T15:55:35.746 INFO:teuthology.orchestra.run.smithi082.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-20T15:55:35.748 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-05-20T15:55:35.755 INFO:teuthology.task.internal:Configuring sudo... 2024-05-20T15:55:35.755 DEBUG:teuthology.orchestra.run.smithi082:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-20T15:55:35.757 DEBUG:teuthology.orchestra.run.smithi160:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-20T15:55:35.760 DEBUG:teuthology.orchestra.run.smithi184:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-20T15:55:35.786 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-05-20T15:55:35.795 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-05-20T15:55:35.796 DEBUG:teuthology.orchestra.run.smithi082:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-20T15:55:35.826 DEBUG:teuthology.orchestra.run.smithi160:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-20T15:55:35.828 DEBUG:teuthology.orchestra.run.smithi184:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-20T15:55:35.844 DEBUG:teuthology.orchestra.run.smithi082:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-20T15:55:35.940 DEBUG:teuthology.orchestra.run.smithi082:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-20T15:55:35.995 DEBUG:teuthology.orchestra.run.smithi082:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-20T15:55:36.078 DEBUG:teuthology.orchestra.run.smithi082:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-20T15:55:36.119 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-05-20T15:55:36.119 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-20T15:55:36.184 DEBUG:teuthology.orchestra.run.smithi160:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-20T15:55:36.226 DEBUG:teuthology.orchestra.run.smithi160:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-20T15:55:36.279 DEBUG:teuthology.orchestra.run.smithi160:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-20T15:55:36.362 DEBUG:teuthology.orchestra.run.smithi160:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-20T15:55:36.406 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2024-05-20T15:55:36.406 DEBUG:teuthology.orchestra.run.smithi160:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-20T15:55:36.473 DEBUG:teuthology.orchestra.run.smithi184:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-20T15:55:36.518 DEBUG:teuthology.orchestra.run.smithi184:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-20T15:55:36.572 DEBUG:teuthology.orchestra.run.smithi184:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-20T15:55:36.654 DEBUG:teuthology.orchestra.run.smithi184:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-20T15:55:36.698 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-05-20T15:55:36.698 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-20T15:55:36.764 DEBUG:teuthology.orchestra.run.smithi082:> sudo service rsyslog restart 2024-05-20T15:55:36.766 DEBUG:teuthology.orchestra.run.smithi160:> sudo service rsyslog restart 2024-05-20T15:55:36.768 DEBUG:teuthology.orchestra.run.smithi184:> sudo service rsyslog restart 2024-05-20T15:55:36.808 INFO:teuthology.orchestra.run.smithi160.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-20T15:55:36.809 INFO:teuthology.orchestra.run.smithi082.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-20T15:55:36.843 INFO:teuthology.orchestra.run.smithi184.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-20T15:55:37.324 INFO:teuthology.run_tasks:Running task internal.timer... 2024-05-20T15:55:37.332 INFO:teuthology.task.internal:Starting timer... 2024-05-20T15:55:37.332 INFO:teuthology.run_tasks:Running task pcp... 2024-05-20T15:55:37.343 INFO:teuthology.run_tasks:Running task selinux... 2024-05-20T15:55:37.351 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-05-20T15:55:37.352 DEBUG:teuthology.orchestra.run.smithi082:> sudo service auditd rotate 2024-05-20T15:55:37.421 INFO:teuthology.orchestra.run.smithi082.stdout:Rotating logs: 2024-05-20T15:55:37.423 DEBUG:teuthology.orchestra.run.smithi160:> sudo service auditd rotate 2024-05-20T15:55:37.482 INFO:teuthology.orchestra.run.smithi160.stdout:Rotating logs: 2024-05-20T15:55:37.484 DEBUG:teuthology.orchestra.run.smithi184:> sudo service auditd rotate 2024-05-20T15:55:37.573 INFO:teuthology.orchestra.run.smithi184.stdout:Rotating logs: 2024-05-20T15:55:37.575 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-05-20T15:55:37.576 DEBUG:teuthology.orchestra.run.smithi082:> /usr/sbin/getenforce 2024-05-20T15:55:37.599 INFO:teuthology.orchestra.run.smithi082.stdout:Permissive 2024-05-20T15:55:37.600 DEBUG:teuthology.orchestra.run.smithi160:> /usr/sbin/getenforce 2024-05-20T15:55:37.624 INFO:teuthology.orchestra.run.smithi160.stdout:Permissive 2024-05-20T15:55:37.624 DEBUG:teuthology.orchestra.run.smithi184:> /usr/sbin/getenforce 2024-05-20T15:55:37.656 INFO:teuthology.orchestra.run.smithi184.stdout:Permissive 2024-05-20T15:55:37.656 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi082.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi160.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi184.front.sepia.ceph.com': 'permissive'} 2024-05-20T15:55:37.656 DEBUG:teuthology.orchestra.run.smithi082:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-20T15:55:37.687 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-20T15:55:37.688 DEBUG:teuthology.orchestra.run.smithi160:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-20T15:55:37.715 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-20T15:55:37.715 DEBUG:teuthology.orchestra.run.smithi184:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-20T15:55:37.743 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-20T15:55:37.744 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-05-20T15:55:37.744 DEBUG:teuthology.orchestra.run.smithi082:> sudo /usr/sbin/setenforce permissive 2024-05-20T15:55:37.772 DEBUG:teuthology.orchestra.run.smithi160:> sudo /usr/sbin/setenforce permissive 2024-05-20T15:55:37.797 DEBUG:teuthology.orchestra.run.smithi184:> sudo /usr/sbin/setenforce permissive 2024-05-20T15:55:37.833 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-05-20T15:55:37.842 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-05-20T15:55:37.854 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-05-20T15:55:37.855 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi082.front.sepia.ceph.com,smithi160.front.sepia.ceph.com,smithi184.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-05-20T16:00:11.799 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi082.front.sepia.ceph.com'), Remote(name='ubuntu@smithi160.front.sepia.ceph.com'), Remote(name='ubuntu@smithi184.front.sepia.ceph.com')] 2024-05-20T16:00:11.801 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi082.front.sepia.ceph.com' 2024-05-20T16:00:11.802 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi082.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-20T16:00:11.890 DEBUG:teuthology.orchestra.run.smithi082:> true 2024-05-20T16:00:11.930 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi082.front.sepia.ceph.com' 2024-05-20T16:00:11.930 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi160.front.sepia.ceph.com' 2024-05-20T16:00:11.930 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi160.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-20T16:00:12.007 DEBUG:teuthology.orchestra.run.smithi160:> true 2024-05-20T16:00:12.093 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi160.front.sepia.ceph.com' 2024-05-20T16:00:12.094 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi184.front.sepia.ceph.com' 2024-05-20T16:00:12.094 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi184.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-20T16:00:12.172 DEBUG:teuthology.orchestra.run.smithi184:> true 2024-05-20T16:00:12.250 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi184.front.sepia.ceph.com' 2024-05-20T16:00:12.251 INFO:teuthology.run_tasks:Running task clock... 2024-05-20T16:00:12.265 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-05-20T16:00:12.265 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-20T16:00:12.265 DEBUG:teuthology.orchestra.run.smithi082:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-20T16:00:12.268 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-20T16:00:12.268 DEBUG:teuthology.orchestra.run.smithi160:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-20T16:00:12.270 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-20T16:00:12.270 DEBUG:teuthology.orchestra.run.smithi184:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-20T16:00:12.296 INFO:teuthology.orchestra.run.smithi160.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-20T16:00:12.298 INFO:teuthology.orchestra.run.smithi082.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-20T16:00:12.309 INFO:teuthology.orchestra.run.smithi160.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-20T16:00:12.314 INFO:teuthology.orchestra.run.smithi082.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-20T16:00:12.320 INFO:teuthology.orchestra.run.smithi184.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-20T16:00:12.332 INFO:teuthology.orchestra.run.smithi160.stderr:sudo: ntpd: command not found 2024-05-20T16:00:12.335 INFO:teuthology.orchestra.run.smithi184.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-20T16:00:12.339 INFO:teuthology.orchestra.run.smithi082.stderr:sudo: ntpd: command not found 2024-05-20T16:00:12.342 INFO:teuthology.orchestra.run.smithi160.stdout:506 Cannot talk to daemon 2024-05-20T16:00:12.351 INFO:teuthology.orchestra.run.smithi082.stdout:506 Cannot talk to daemon 2024-05-20T16:00:12.355 INFO:teuthology.orchestra.run.smithi160.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-20T16:00:12.359 INFO:teuthology.orchestra.run.smithi184.stderr:sudo: ntpd: command not found 2024-05-20T16:00:12.365 INFO:teuthology.orchestra.run.smithi082.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-20T16:00:12.368 INFO:teuthology.orchestra.run.smithi160.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-20T16:00:12.370 INFO:teuthology.orchestra.run.smithi184.stdout:506 Cannot talk to daemon 2024-05-20T16:00:12.380 INFO:teuthology.orchestra.run.smithi082.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-20T16:00:12.383 INFO:teuthology.orchestra.run.smithi184.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-20T16:00:12.397 INFO:teuthology.orchestra.run.smithi184.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-20T16:00:12.424 INFO:teuthology.orchestra.run.smithi160.stderr:bash: line 1: ntpq: command not found 2024-05-20T16:00:12.427 INFO:teuthology.orchestra.run.smithi160.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-20T16:00:12.427 INFO:teuthology.orchestra.run.smithi160.stdout:=============================================================================== 2024-05-20T16:00:12.427 INFO:teuthology.orchestra.run.smithi160.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-20T16:00:12.427 INFO:teuthology.orchestra.run.smithi160.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-20T16:00:12.428 INFO:teuthology.orchestra.run.smithi160.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-20T16:00:12.428 INFO:teuthology.orchestra.run.smithi160.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-20T16:00:12.435 INFO:teuthology.orchestra.run.smithi082.stderr:bash: line 1: ntpq: command not found 2024-05-20T16:00:12.438 INFO:teuthology.orchestra.run.smithi082.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-20T16:00:12.438 INFO:teuthology.orchestra.run.smithi082.stdout:=============================================================================== 2024-05-20T16:00:12.438 INFO:teuthology.orchestra.run.smithi082.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-20T16:00:12.438 INFO:teuthology.orchestra.run.smithi082.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-20T16:00:12.438 INFO:teuthology.orchestra.run.smithi082.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-20T16:00:12.438 INFO:teuthology.orchestra.run.smithi082.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-20T16:00:12.460 INFO:teuthology.orchestra.run.smithi184.stderr:bash: line 1: ntpq: command not found 2024-05-20T16:00:12.463 INFO:teuthology.orchestra.run.smithi184.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-20T16:00:12.463 INFO:teuthology.orchestra.run.smithi184.stdout:=============================================================================== 2024-05-20T16:00:12.463 INFO:teuthology.orchestra.run.smithi184.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-20T16:00:12.463 INFO:teuthology.orchestra.run.smithi184.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-20T16:00:12.463 INFO:teuthology.orchestra.run.smithi184.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-20T16:00:12.463 INFO:teuthology.orchestra.run.smithi184.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-20T16:00:12.464 INFO:teuthology.run_tasks:Running task install... 2024-05-20T16:00:12.472 DEBUG:teuthology.task.install:project ceph 2024-05-20T16:00:12.472 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'd2b507c724ef6c8295ee0a35d185cdab167fd61d'}} 2024-05-20T16:00:12.473 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'd2b507c724ef6c8295ee0a35d185cdab167fd61d'} 2024-05-20T16:00:12.473 INFO:teuthology.task.install:Using flavor: default 2024-05-20T16:00:12.478 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-05-20T16:00:12.478 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-05-20T16:00:12.478 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd2b507c724ef6c8295ee0a35d185cdab167fd61d', 'tag': None, 'wait_for_package': False} 2024-05-20T16:00:12.479 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d2b507c724ef6c8295ee0a35d185cdab167fd61d 2024-05-20T16:00:12.481 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd2b507c724ef6c8295ee0a35d185cdab167fd61d', 'tag': None, 'wait_for_package': False} 2024-05-20T16:00:12.481 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d2b507c724ef6c8295ee0a35d185cdab167fd61d 2024-05-20T16:00:12.482 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd2b507c724ef6c8295ee0a35d185cdab167fd61d', 'tag': None, 'wait_for_package': False} 2024-05-20T16:00:12.482 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d2b507c724ef6c8295ee0a35d185cdab167fd61d 2024-05-20T16:00:12.624 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240517.203521-squid-debug/d2b507c724ef6c8295ee0a35d185cdab167fd61d/centos/9/flavors/default/ 2024-05-20T16:00:12.624 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2744.gd2b507c7 2024-05-20T16:00:12.695 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240517.203521-squid-debug/d2b507c724ef6c8295ee0a35d185cdab167fd61d/centos/9/flavors/default/ 2024-05-20T16:00:12.695 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2744.gd2b507c7 2024-05-20T16:00:12.705 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240517.203521-squid-debug/d2b507c724ef6c8295ee0a35d185cdab167fd61d/centos/9/flavors/default/ 2024-05-20T16:00:12.705 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2744.gd2b507c7 2024-05-20T16:00:12.831 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240517.203521-squid-debug/d2b507c724ef6c8295ee0a35d185cdab167fd61d/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240517.203521-squid-debug/d2b507c724ef6c8295ee0a35d185cdab167fd61d/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240517.203521-squid-debug/d2b507c724ef6c8295ee0a35d185cdab167fd61d/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-20T16:00:12.831 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2024-05-20T16:00:12.831 DEBUG:teuthology.orchestra.run.smithi160:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-20T16:00:12.856 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-05-20T16:00:12.856 DEBUG:teuthology.orchestra.run.smithi160:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/d2b507c724ef6c8295ee0a35d185cdab167fd61d/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-20T16:00:12.904 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240517.203521-squid-debug/d2b507c724ef6c8295ee0a35d185cdab167fd61d/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240517.203521-squid-debug/d2b507c724ef6c8295ee0a35d185cdab167fd61d/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240517.203521-squid-debug/d2b507c724ef6c8295ee0a35d185cdab167fd61d/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-20T16:00:12.904 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-05-20T16:00:12.904 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-20T16:00:12.920 DEBUG:teuthology.orchestra.run.smithi160:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-20T16:00:12.929 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-05-20T16:00:12.930 DEBUG:teuthology.orchestra.run.smithi184:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/d2b507c724ef6c8295ee0a35d185cdab167fd61d/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-20T16:00:12.993 DEBUG:teuthology.orchestra.run.smithi160:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-20T16:00:12.994 DEBUG:teuthology.orchestra.run.smithi184:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-20T16:00:13.004 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240517.203521-squid-debug/d2b507c724ef6c8295ee0a35d185cdab167fd61d/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240517.203521-squid-debug/d2b507c724ef6c8295ee0a35d185cdab167fd61d/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240517.203521-squid-debug/d2b507c724ef6c8295ee0a35d185cdab167fd61d/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-20T16:00:13.004 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-05-20T16:00:13.004 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-20T16:00:13.030 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-05-20T16:00:13.031 DEBUG:teuthology.orchestra.run.smithi082:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/d2b507c724ef6c8295ee0a35d185cdab167fd61d/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-20T16:00:13.056 INFO:teuthology.orchestra.run.smithi160.stdout:check_obsoletes = 1 2024-05-20T16:00:13.057 DEBUG:teuthology.orchestra.run.smithi160:> sudo yum clean all 2024-05-20T16:00:13.067 DEBUG:teuthology.orchestra.run.smithi184:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-20T16:00:13.097 DEBUG:teuthology.orchestra.run.smithi082:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-20T16:00:13.130 INFO:teuthology.orchestra.run.smithi184.stdout:check_obsoletes = 1 2024-05-20T16:00:13.131 DEBUG:teuthology.orchestra.run.smithi184:> sudo yum clean all 2024-05-20T16:00:13.172 DEBUG:teuthology.orchestra.run.smithi082:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-20T16:00:13.237 INFO:teuthology.orchestra.run.smithi082.stdout:check_obsoletes = 1 2024-05-20T16:00:13.238 DEBUG:teuthology.orchestra.run.smithi082:> sudo yum clean all 2024-05-20T16:00:13.282 INFO:teuthology.orchestra.run.smithi160.stdout:45 files removed 2024-05-20T16:00:13.309 DEBUG:teuthology.orchestra.run.smithi160:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-05-20T16:00:13.353 INFO:teuthology.orchestra.run.smithi184.stdout:45 files removed 2024-05-20T16:00:13.380 DEBUG:teuthology.orchestra.run.smithi184:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-05-20T16:00:13.474 INFO:teuthology.orchestra.run.smithi082.stdout:45 files removed 2024-05-20T16:00:13.500 DEBUG:teuthology.orchestra.run.smithi082:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-05-20T16:00:14.242 INFO:teuthology.orchestra.run.smithi160.stdout:ceph packages for x86_64 123 kB/s | 83 kB 00:00 2024-05-20T16:00:14.305 INFO:teuthology.orchestra.run.smithi184.stdout:ceph packages for x86_64 125 kB/s | 83 kB 00:00 2024-05-20T16:00:14.364 INFO:teuthology.orchestra.run.smithi082.stdout:ceph packages for x86_64 141 kB/s | 83 kB 00:00 2024-05-20T16:00:14.723 INFO:teuthology.orchestra.run.smithi082.stdout:ceph noarch packages 38 kB/s | 11 kB 00:00 2024-05-20T16:00:14.739 INFO:teuthology.orchestra.run.smithi184.stdout:ceph noarch packages 30 kB/s | 11 kB 00:00 2024-05-20T16:00:14.810 INFO:teuthology.orchestra.run.smithi160.stdout:ceph noarch packages 22 kB/s | 11 kB 00:00 2024-05-20T16:00:15.148 INFO:teuthology.orchestra.run.smithi184.stdout:ceph source packages 5.1 kB/s | 1.9 kB 00:00 2024-05-20T16:00:15.231 INFO:teuthology.orchestra.run.smithi082.stdout:ceph source packages 4.1 kB/s | 1.9 kB 00:00 2024-05-20T16:00:15.294 INFO:teuthology.orchestra.run.smithi160.stdout:ceph source packages 4.2 kB/s | 1.9 kB 00:00 2024-05-20T16:00:15.798 INFO:teuthology.orchestra.run.smithi082.stdout:CentOS Stream 9 - BaseOS 15 MB/s | 8.1 MB 00:00 2024-05-20T16:00:16.698 INFO:teuthology.orchestra.run.smithi184.stdout:CentOS Stream 9 - BaseOS 5.3 MB/s | 8.1 MB 00:01 2024-05-20T16:00:17.541 INFO:teuthology.orchestra.run.smithi082.stdout:CentOS Stream 9 - AppStream 29 MB/s | 19 MB 00:00 2024-05-20T16:00:18.069 INFO:teuthology.orchestra.run.smithi160.stdout:CentOS Stream 9 - BaseOS 2.9 MB/s | 8.1 MB 00:02 2024-05-20T16:00:18.566 INFO:teuthology.orchestra.run.smithi184.stdout:CentOS Stream 9 - AppStream 25 MB/s | 19 MB 00:00 2024-05-20T16:00:19.928 INFO:teuthology.orchestra.run.smithi160.stdout:CentOS Stream 9 - AppStream 25 MB/s | 19 MB 00:00 2024-05-20T16:00:21.525 INFO:teuthology.orchestra.run.smithi082.stdout:CentOS Stream 9 - CRB 14 MB/s | 6.2 MB 00:00 2024-05-20T16:00:22.666 INFO:teuthology.orchestra.run.smithi184.stdout:CentOS Stream 9 - CRB 12 MB/s | 6.2 MB 00:00 2024-05-20T16:00:23.142 INFO:teuthology.orchestra.run.smithi082.stdout:CentOS Stream 9 - Extras packages 37 kB/s | 16 kB 00:00 2024-05-20T16:00:24.187 INFO:teuthology.orchestra.run.smithi160.stdout:CentOS Stream 9 - CRB 8.9 MB/s | 6.2 MB 00:00 2024-05-20T16:00:24.375 INFO:teuthology.orchestra.run.smithi184.stdout:CentOS Stream 9 - Extras packages 36 kB/s | 16 kB 00:00 2024-05-20T16:00:25.188 INFO:teuthology.orchestra.run.smithi082.stdout:Extra Packages for Enterprise Linux 11 MB/s | 21 MB 00:01 2024-05-20T16:00:25.663 INFO:teuthology.orchestra.run.smithi160.stdout:CentOS Stream 9 - Extras packages 51 kB/s | 16 kB 00:00 2024-05-20T16:00:27.455 INFO:teuthology.orchestra.run.smithi184.stdout:Extra Packages for Enterprise Linux 7.2 MB/s | 21 MB 00:02 2024-05-20T16:00:30.377 INFO:teuthology.orchestra.run.smithi082.stdout:lab-extras 34 kB/s | 1.7 kB 00:00 2024-05-20T16:00:32.670 INFO:teuthology.orchestra.run.smithi082.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-20T16:00:32.671 INFO:teuthology.orchestra.run.smithi082.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-20T16:00:32.682 INFO:teuthology.orchestra.run.smithi082.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-05-20T16:00:32.683 INFO:teuthology.orchestra.run.smithi082.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-05-20T16:00:32.683 INFO:teuthology.orchestra.run.smithi082.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-05-20T16:00:32.684 INFO:teuthology.orchestra.run.smithi082.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-05-20T16:00:32.686 INFO:teuthology.orchestra.run.smithi082.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-05-20T16:00:32.688 INFO:teuthology.orchestra.run.smithi082.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-05-20T16:00:32.689 INFO:teuthology.orchestra.run.smithi082.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-05-20T16:00:32.735 INFO:teuthology.orchestra.run.smithi184.stdout:lab-extras 6.6 kB/s | 1.7 kB 00:00 2024-05-20T16:00:32.751 INFO:teuthology.orchestra.run.smithi082.stdout:Dependencies resolved. 2024-05-20T16:00:32.766 INFO:teuthology.orchestra.run.smithi082.stdout:======================================================================================= 2024-05-20T16:00:32.766 INFO:teuthology.orchestra.run.smithi082.stdout: Package Arch Version Repository Size 2024-05-20T16:00:32.766 INFO:teuthology.orchestra.run.smithi082.stdout:======================================================================================= 2024-05-20T16:00:32.766 INFO:teuthology.orchestra.run.smithi082.stdout:Installing: 2024-05-20T16:00:32.766 INFO:teuthology.orchestra.run.smithi082.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-05-20T16:00:32.766 INFO:teuthology.orchestra.run.smithi082.stdout: ceph x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 6.6 k 2024-05-20T16:00:32.766 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-base x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 5.4 M 2024-05-20T16:00:32.766 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-fuse x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 863 k 2024-05-20T16:00:32.766 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 147 k 2024-05-20T16:00:32.766 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mgr x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 1.5 M 2024-05-20T16:00:32.767 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2744.gd2b507c7.el9 ceph-noarch 142 k 2024-05-20T16:00:32.767 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2744.gd2b507c7.el9 ceph-noarch 3.5 M 2024-05-20T16:00:32.767 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2744.gd2b507c7.el9 ceph-noarch 7.4 M 2024-05-20T16:00:32.767 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mgr-rook noarch 2:19.0.0-2744.gd2b507c7.el9 ceph-noarch 49 k 2024-05-20T16:00:32.767 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-radosgw x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 9.8 M 2024-05-20T16:00:32.767 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-test x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 46 M 2024-05-20T16:00:32.767 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-volume noarch 2:19.0.0-2744.gd2b507c7.el9 ceph-noarch 264 k 2024-05-20T16:00:32.767 INFO:teuthology.orchestra.run.smithi082.stdout: cephadm noarch 2:19.0.0-2744.gd2b507c7.el9 ceph-noarch 763 k 2024-05-20T16:00:32.767 INFO:teuthology.orchestra.run.smithi082.stdout: cephfs-mirror x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 221 k 2024-05-20T16:00:32.767 INFO:teuthology.orchestra.run.smithi082.stdout: cephfs-shell x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 27 k 2024-05-20T16:00:32.767 INFO:teuthology.orchestra.run.smithi082.stdout: cephfs-top noarch 2:19.0.0-2744.gd2b507c7.el9 ceph-noarch 18 k 2024-05-20T16:00:32.767 INFO:teuthology.orchestra.run.smithi082.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-05-20T16:00:32.767 INFO:teuthology.orchestra.run.smithi082.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-05-20T16:00:32.768 INFO:teuthology.orchestra.run.smithi082.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-05-20T16:00:32.768 INFO:teuthology.orchestra.run.smithi082.stdout: libcephfs-devel x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 32 k 2024-05-20T16:00:32.768 INFO:teuthology.orchestra.run.smithi082.stdout: libcephfs2 x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 756 k 2024-05-20T16:00:32.768 INFO:teuthology.orchestra.run.smithi082.stdout: librados-devel x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 127 k 2024-05-20T16:00:32.768 INFO:teuthology.orchestra.run.smithi082.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-05-20T16:00:32.768 INFO:teuthology.orchestra.run.smithi082.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-05-20T16:00:32.768 INFO:teuthology.orchestra.run.smithi082.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-05-20T16:00:32.768 INFO:teuthology.orchestra.run.smithi082.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-05-20T16:00:32.768 INFO:teuthology.orchestra.run.smithi082.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-05-20T16:00:32.768 INFO:teuthology.orchestra.run.smithi082.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-05-20T16:00:32.768 INFO:teuthology.orchestra.run.smithi082.stdout: python3-cephfs x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 161 k 2024-05-20T16:00:32.768 INFO:teuthology.orchestra.run.smithi082.stdout: python3-rados x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 320 k 2024-05-20T16:00:32.769 INFO:teuthology.orchestra.run.smithi082.stdout: python3-rbd x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 299 k 2024-05-20T16:00:32.769 INFO:teuthology.orchestra.run.smithi082.stdout: python3-rgw x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 99 k 2024-05-20T16:00:32.769 INFO:teuthology.orchestra.run.smithi082.stdout: rbd-fuse x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 86 k 2024-05-20T16:00:32.769 INFO:teuthology.orchestra.run.smithi082.stdout: rbd-mirror x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 3.0 M 2024-05-20T16:00:32.769 INFO:teuthology.orchestra.run.smithi082.stdout: rbd-nbd x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 169 k 2024-05-20T16:00:32.769 INFO:teuthology.orchestra.run.smithi082.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-05-20T16:00:32.769 INFO:teuthology.orchestra.run.smithi082.stdout:Upgrading: 2024-05-20T16:00:32.769 INFO:teuthology.orchestra.run.smithi082.stdout: NetworkManager x86_64 1:1.47.90-1.el9 baseos 2.3 M 2024-05-20T16:00:32.769 INFO:teuthology.orchestra.run.smithi082.stdout: NetworkManager-libnm x86_64 1:1.47.90-1.el9 baseos 1.8 M 2024-05-20T16:00:32.769 INFO:teuthology.orchestra.run.smithi082.stdout: NetworkManager-team x86_64 1:1.47.90-1.el9 baseos 41 k 2024-05-20T16:00:32.769 INFO:teuthology.orchestra.run.smithi082.stdout: NetworkManager-tui x86_64 1:1.47.90-1.el9 baseos 248 k 2024-05-20T16:00:32.769 INFO:teuthology.orchestra.run.smithi082.stdout: librados2 x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 3.3 M 2024-05-20T16:00:32.769 INFO:teuthology.orchestra.run.smithi082.stdout: librbd1 x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 3.1 M 2024-05-20T16:00:32.770 INFO:teuthology.orchestra.run.smithi082.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-05-20T16:00:32.770 INFO:teuthology.orchestra.run.smithi082.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-05-20T16:00:32.770 INFO:teuthology.orchestra.run.smithi082.stdout:Installing dependencies: 2024-05-20T16:00:32.770 INFO:teuthology.orchestra.run.smithi082.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-05-20T16:00:32.770 INFO:teuthology.orchestra.run.smithi082.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-05-20T16:00:32.770 INFO:teuthology.orchestra.run.smithi082.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-05-20T16:00:32.770 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-common x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 19 M 2024-05-20T16:00:32.770 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2744.gd2b507c7.el9 ceph-noarch 32 k 2024-05-20T16:00:32.770 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mds x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 2.3 M 2024-05-20T16:00:32.770 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2744.gd2b507c7.el9 ceph-noarch 249 k 2024-05-20T16:00:32.770 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mon x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 4.9 M 2024-05-20T16:00:32.770 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-osd x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 18 M 2024-05-20T16:00:32.770 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2744.gd2b507c7.el9 ceph-noarch 16 k 2024-05-20T16:00:32.771 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-selinux x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 25 k 2024-05-20T16:00:32.771 INFO:teuthology.orchestra.run.smithi082.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-05-20T16:00:32.771 INFO:teuthology.orchestra.run.smithi082.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-05-20T16:00:32.771 INFO:teuthology.orchestra.run.smithi082.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-05-20T16:00:32.771 INFO:teuthology.orchestra.run.smithi082.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-05-20T16:00:32.771 INFO:teuthology.orchestra.run.smithi082.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-05-20T16:00:32.771 INFO:teuthology.orchestra.run.smithi082.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-05-20T16:00:32.771 INFO:teuthology.orchestra.run.smithi082.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-05-20T16:00:32.771 INFO:teuthology.orchestra.run.smithi082.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-05-20T16:00:32.771 INFO:teuthology.orchestra.run.smithi082.stdout: libcephsqlite x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 158 k 2024-05-20T16:00:32.771 INFO:teuthology.orchestra.run.smithi082.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-05-20T16:00:32.771 INFO:teuthology.orchestra.run.smithi082.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-05-20T16:00:32.772 INFO:teuthology.orchestra.run.smithi082.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-05-20T16:00:32.772 INFO:teuthology.orchestra.run.smithi082.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-05-20T16:00:32.772 INFO:teuthology.orchestra.run.smithi082.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-05-20T16:00:32.772 INFO:teuthology.orchestra.run.smithi082.stdout: libradosstriper1 x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 497 k 2024-05-20T16:00:32.772 INFO:teuthology.orchestra.run.smithi082.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-05-20T16:00:32.772 INFO:teuthology.orchestra.run.smithi082.stdout: librgw2 x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 5.0 M 2024-05-20T16:00:32.772 INFO:teuthology.orchestra.run.smithi082.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-05-20T16:00:32.772 INFO:teuthology.orchestra.run.smithi082.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-05-20T16:00:32.772 INFO:teuthology.orchestra.run.smithi082.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-05-20T16:00:32.772 INFO:teuthology.orchestra.run.smithi082.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-05-20T16:00:32.772 INFO:teuthology.orchestra.run.smithi082.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-05-20T16:00:32.772 INFO:teuthology.orchestra.run.smithi082.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-05-20T16:00:32.772 INFO:teuthology.orchestra.run.smithi082.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-05-20T16:00:32.773 INFO:teuthology.orchestra.run.smithi082.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-05-20T16:00:32.773 INFO:teuthology.orchestra.run.smithi082.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-05-20T16:00:32.773 INFO:teuthology.orchestra.run.smithi082.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-05-20T16:00:32.773 INFO:teuthology.orchestra.run.smithi082.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-05-20T16:00:32.773 INFO:teuthology.orchestra.run.smithi082.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-05-20T16:00:32.773 INFO:teuthology.orchestra.run.smithi082.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-05-20T16:00:32.773 INFO:teuthology.orchestra.run.smithi082.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-05-20T16:00:32.773 INFO:teuthology.orchestra.run.smithi082.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-05-20T16:00:32.773 INFO:teuthology.orchestra.run.smithi082.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-05-20T16:00:32.773 INFO:teuthology.orchestra.run.smithi082.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-05-20T16:00:32.773 INFO:teuthology.orchestra.run.smithi082.stdout: python3-ceph-argparse x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 45 k 2024-05-20T16:00:32.773 INFO:teuthology.orchestra.run.smithi082.stdout: python3-ceph-common x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 135 k 2024-05-20T16:00:32.773 INFO:teuthology.orchestra.run.smithi082.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-05-20T16:00:32.773 INFO:teuthology.orchestra.run.smithi082.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-05-20T16:00:32.774 INFO:teuthology.orchestra.run.smithi082.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-05-20T16:00:32.774 INFO:teuthology.orchestra.run.smithi082.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-05-20T16:00:32.774 INFO:teuthology.orchestra.run.smithi082.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-05-20T16:00:32.774 INFO:teuthology.orchestra.run.smithi082.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-05-20T16:00:32.774 INFO:teuthology.orchestra.run.smithi082.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-05-20T16:00:32.774 INFO:teuthology.orchestra.run.smithi082.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-05-20T16:00:32.774 INFO:teuthology.orchestra.run.smithi082.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-05-20T16:00:32.774 INFO:teuthology.orchestra.run.smithi082.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-05-20T16:00:32.774 INFO:teuthology.orchestra.run.smithi082.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-05-20T16:00:32.774 INFO:teuthology.orchestra.run.smithi082.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-05-20T16:00:32.774 INFO:teuthology.orchestra.run.smithi082.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-05-20T16:00:32.774 INFO:teuthology.orchestra.run.smithi082.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-05-20T16:00:32.774 INFO:teuthology.orchestra.run.smithi082.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-05-20T16:00:32.775 INFO:teuthology.orchestra.run.smithi082.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-05-20T16:00:32.775 INFO:teuthology.orchestra.run.smithi082.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-05-20T16:00:32.775 INFO:teuthology.orchestra.run.smithi082.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-05-20T16:00:32.775 INFO:teuthology.orchestra.run.smithi082.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-05-20T16:00:32.775 INFO:teuthology.orchestra.run.smithi082.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-05-20T16:00:32.775 INFO:teuthology.orchestra.run.smithi082.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-05-20T16:00:32.775 INFO:teuthology.orchestra.run.smithi082.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-05-20T16:00:32.775 INFO:teuthology.orchestra.run.smithi082.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-05-20T16:00:32.775 INFO:teuthology.orchestra.run.smithi082.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-05-20T16:00:32.775 INFO:teuthology.orchestra.run.smithi082.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-05-20T16:00:32.775 INFO:teuthology.orchestra.run.smithi082.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-05-20T16:00:32.775 INFO:teuthology.orchestra.run.smithi082.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-05-20T16:00:32.775 INFO:teuthology.orchestra.run.smithi082.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-05-20T16:00:32.775 INFO:teuthology.orchestra.run.smithi082.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-05-20T16:00:32.776 INFO:teuthology.orchestra.run.smithi082.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-05-20T16:00:32.776 INFO:teuthology.orchestra.run.smithi082.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-05-20T16:00:32.776 INFO:teuthology.orchestra.run.smithi082.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-05-20T16:00:32.776 INFO:teuthology.orchestra.run.smithi082.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-05-20T16:00:32.776 INFO:teuthology.orchestra.run.smithi082.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-05-20T16:00:32.776 INFO:teuthology.orchestra.run.smithi082.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-05-20T16:00:32.776 INFO:teuthology.orchestra.run.smithi082.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-05-20T16:00:32.776 INFO:teuthology.orchestra.run.smithi082.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-05-20T16:00:32.776 INFO:teuthology.orchestra.run.smithi082.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-05-20T16:00:32.776 INFO:teuthology.orchestra.run.smithi082.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-05-20T16:00:32.776 INFO:teuthology.orchestra.run.smithi082.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-05-20T16:00:32.776 INFO:teuthology.orchestra.run.smithi082.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-05-20T16:00:32.777 INFO:teuthology.orchestra.run.smithi082.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-05-20T16:00:32.777 INFO:teuthology.orchestra.run.smithi082.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-05-20T16:00:32.777 INFO:teuthology.orchestra.run.smithi082.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-05-20T16:00:32.777 INFO:teuthology.orchestra.run.smithi082.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-05-20T16:00:32.777 INFO:teuthology.orchestra.run.smithi082.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-05-20T16:00:32.777 INFO:teuthology.orchestra.run.smithi082.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-05-20T16:00:32.777 INFO:teuthology.orchestra.run.smithi082.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-05-20T16:00:32.777 INFO:teuthology.orchestra.run.smithi082.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-05-20T16:00:32.777 INFO:teuthology.orchestra.run.smithi082.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-05-20T16:00:32.777 INFO:teuthology.orchestra.run.smithi082.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-05-20T16:00:32.777 INFO:teuthology.orchestra.run.smithi082.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-05-20T16:00:32.777 INFO:teuthology.orchestra.run.smithi082.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-05-20T16:00:32.777 INFO:teuthology.orchestra.run.smithi082.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-05-20T16:00:32.778 INFO:teuthology.orchestra.run.smithi082.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-05-20T16:00:32.778 INFO:teuthology.orchestra.run.smithi082.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-05-20T16:00:32.778 INFO:teuthology.orchestra.run.smithi082.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-05-20T16:00:32.778 INFO:teuthology.orchestra.run.smithi082.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-05-20T16:00:32.778 INFO:teuthology.orchestra.run.smithi082.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-05-20T16:00:32.778 INFO:teuthology.orchestra.run.smithi082.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-05-20T16:00:32.778 INFO:teuthology.orchestra.run.smithi082.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-05-20T16:00:32.778 INFO:teuthology.orchestra.run.smithi082.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-05-20T16:00:32.778 INFO:teuthology.orchestra.run.smithi082.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-05-20T16:00:32.778 INFO:teuthology.orchestra.run.smithi082.stdout:Installing weak dependencies: 2024-05-20T16:00:32.778 INFO:teuthology.orchestra.run.smithi082.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-05-20T16:00:32.778 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-05-20T16:00:32.779 INFO:teuthology.orchestra.run.smithi082.stdout:Transaction Summary 2024-05-20T16:00:32.779 INFO:teuthology.orchestra.run.smithi082.stdout:======================================================================================= 2024-05-20T16:00:32.779 INFO:teuthology.orchestra.run.smithi082.stdout:Install 148 Packages 2024-05-20T16:00:32.779 INFO:teuthology.orchestra.run.smithi082.stdout:Upgrade 8 Packages 2024-05-20T16:00:32.779 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-05-20T16:00:32.779 INFO:teuthology.orchestra.run.smithi082.stdout:Total download size: 221 M 2024-05-20T16:00:32.779 INFO:teuthology.orchestra.run.smithi082.stdout:Downloading Packages: 2024-05-20T16:00:33.903 INFO:teuthology.orchestra.run.smithi082.stdout:(1/156): ceph-19.0.0-2744.gd2b507c7.el9.x86_64. 21 kB/s | 6.6 kB 00:00 2024-05-20T16:00:34.994 INFO:teuthology.orchestra.run.smithi184.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-20T16:00:34.995 INFO:teuthology.orchestra.run.smithi184.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-20T16:00:35.005 INFO:teuthology.orchestra.run.smithi184.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-05-20T16:00:35.006 INFO:teuthology.orchestra.run.smithi184.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-05-20T16:00:35.007 INFO:teuthology.orchestra.run.smithi184.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-05-20T16:00:35.008 INFO:teuthology.orchestra.run.smithi184.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-05-20T16:00:35.009 INFO:teuthology.orchestra.run.smithi184.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-05-20T16:00:35.011 INFO:teuthology.orchestra.run.smithi082.stdout:(2/156): ceph-fuse-19.0.0-2744.gd2b507c7.el9.x8 779 kB/s | 863 kB 00:01 2024-05-20T16:00:35.011 INFO:teuthology.orchestra.run.smithi184.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-05-20T16:00:35.012 INFO:teuthology.orchestra.run.smithi184.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-05-20T16:00:35.072 INFO:teuthology.orchestra.run.smithi184.stdout:Dependencies resolved. 2024-05-20T16:00:35.095 INFO:teuthology.orchestra.run.smithi184.stdout:======================================================================================= 2024-05-20T16:00:35.095 INFO:teuthology.orchestra.run.smithi184.stdout: Package Arch Version Repository Size 2024-05-20T16:00:35.095 INFO:teuthology.orchestra.run.smithi184.stdout:======================================================================================= 2024-05-20T16:00:35.095 INFO:teuthology.orchestra.run.smithi184.stdout:Installing: 2024-05-20T16:00:35.095 INFO:teuthology.orchestra.run.smithi184.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-05-20T16:00:35.095 INFO:teuthology.orchestra.run.smithi184.stdout: ceph x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 6.6 k 2024-05-20T16:00:35.095 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-base x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 5.4 M 2024-05-20T16:00:35.095 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-fuse x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 863 k 2024-05-20T16:00:35.095 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 147 k 2024-05-20T16:00:35.095 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mgr x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 1.5 M 2024-05-20T16:00:35.096 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2744.gd2b507c7.el9 ceph-noarch 142 k 2024-05-20T16:00:35.096 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2744.gd2b507c7.el9 ceph-noarch 3.5 M 2024-05-20T16:00:35.096 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2744.gd2b507c7.el9 ceph-noarch 7.4 M 2024-05-20T16:00:35.096 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mgr-rook noarch 2:19.0.0-2744.gd2b507c7.el9 ceph-noarch 49 k 2024-05-20T16:00:35.096 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-radosgw x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 9.8 M 2024-05-20T16:00:35.096 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-test x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 46 M 2024-05-20T16:00:35.096 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-volume noarch 2:19.0.0-2744.gd2b507c7.el9 ceph-noarch 264 k 2024-05-20T16:00:35.096 INFO:teuthology.orchestra.run.smithi184.stdout: cephadm noarch 2:19.0.0-2744.gd2b507c7.el9 ceph-noarch 763 k 2024-05-20T16:00:35.096 INFO:teuthology.orchestra.run.smithi184.stdout: cephfs-mirror x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 221 k 2024-05-20T16:00:35.096 INFO:teuthology.orchestra.run.smithi184.stdout: cephfs-shell x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 27 k 2024-05-20T16:00:35.096 INFO:teuthology.orchestra.run.smithi184.stdout: cephfs-top noarch 2:19.0.0-2744.gd2b507c7.el9 ceph-noarch 18 k 2024-05-20T16:00:35.096 INFO:teuthology.orchestra.run.smithi184.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-05-20T16:00:35.096 INFO:teuthology.orchestra.run.smithi184.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-05-20T16:00:35.097 INFO:teuthology.orchestra.run.smithi184.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-05-20T16:00:35.097 INFO:teuthology.orchestra.run.smithi184.stdout: libcephfs-devel x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 32 k 2024-05-20T16:00:35.097 INFO:teuthology.orchestra.run.smithi184.stdout: libcephfs2 x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 756 k 2024-05-20T16:00:35.097 INFO:teuthology.orchestra.run.smithi184.stdout: librados-devel x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 127 k 2024-05-20T16:00:35.097 INFO:teuthology.orchestra.run.smithi184.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-05-20T16:00:35.097 INFO:teuthology.orchestra.run.smithi184.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-05-20T16:00:35.097 INFO:teuthology.orchestra.run.smithi184.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-05-20T16:00:35.097 INFO:teuthology.orchestra.run.smithi184.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-05-20T16:00:35.097 INFO:teuthology.orchestra.run.smithi184.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-05-20T16:00:35.097 INFO:teuthology.orchestra.run.smithi184.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-05-20T16:00:35.097 INFO:teuthology.orchestra.run.smithi184.stdout: python3-cephfs x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 161 k 2024-05-20T16:00:35.097 INFO:teuthology.orchestra.run.smithi184.stdout: python3-rados x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 320 k 2024-05-20T16:00:35.097 INFO:teuthology.orchestra.run.smithi184.stdout: python3-rbd x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 299 k 2024-05-20T16:00:35.097 INFO:teuthology.orchestra.run.smithi184.stdout: python3-rgw x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 99 k 2024-05-20T16:00:35.098 INFO:teuthology.orchestra.run.smithi184.stdout: rbd-fuse x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 86 k 2024-05-20T16:00:35.098 INFO:teuthology.orchestra.run.smithi184.stdout: rbd-mirror x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 3.0 M 2024-05-20T16:00:35.098 INFO:teuthology.orchestra.run.smithi184.stdout: rbd-nbd x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 169 k 2024-05-20T16:00:35.098 INFO:teuthology.orchestra.run.smithi184.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-05-20T16:00:35.098 INFO:teuthology.orchestra.run.smithi184.stdout:Upgrading: 2024-05-20T16:00:35.098 INFO:teuthology.orchestra.run.smithi184.stdout: NetworkManager x86_64 1:1.47.90-1.el9 baseos 2.3 M 2024-05-20T16:00:35.098 INFO:teuthology.orchestra.run.smithi184.stdout: NetworkManager-libnm x86_64 1:1.47.90-1.el9 baseos 1.8 M 2024-05-20T16:00:35.098 INFO:teuthology.orchestra.run.smithi184.stdout: NetworkManager-team x86_64 1:1.47.90-1.el9 baseos 41 k 2024-05-20T16:00:35.098 INFO:teuthology.orchestra.run.smithi184.stdout: NetworkManager-tui x86_64 1:1.47.90-1.el9 baseos 248 k 2024-05-20T16:00:35.098 INFO:teuthology.orchestra.run.smithi184.stdout: librados2 x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 3.3 M 2024-05-20T16:00:35.098 INFO:teuthology.orchestra.run.smithi184.stdout: librbd1 x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 3.1 M 2024-05-20T16:00:35.098 INFO:teuthology.orchestra.run.smithi184.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-05-20T16:00:35.098 INFO:teuthology.orchestra.run.smithi184.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-05-20T16:00:35.099 INFO:teuthology.orchestra.run.smithi184.stdout:Installing dependencies: 2024-05-20T16:00:35.099 INFO:teuthology.orchestra.run.smithi184.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-05-20T16:00:35.099 INFO:teuthology.orchestra.run.smithi184.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-05-20T16:00:35.099 INFO:teuthology.orchestra.run.smithi184.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-05-20T16:00:35.099 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-common x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 19 M 2024-05-20T16:00:35.099 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2744.gd2b507c7.el9 ceph-noarch 32 k 2024-05-20T16:00:35.099 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mds x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 2.3 M 2024-05-20T16:00:35.099 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2744.gd2b507c7.el9 ceph-noarch 249 k 2024-05-20T16:00:35.099 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mon x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 4.9 M 2024-05-20T16:00:35.099 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-osd x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 18 M 2024-05-20T16:00:35.099 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2744.gd2b507c7.el9 ceph-noarch 16 k 2024-05-20T16:00:35.099 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-selinux x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 25 k 2024-05-20T16:00:35.099 INFO:teuthology.orchestra.run.smithi184.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-05-20T16:00:35.099 INFO:teuthology.orchestra.run.smithi184.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-05-20T16:00:35.100 INFO:teuthology.orchestra.run.smithi184.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-05-20T16:00:35.100 INFO:teuthology.orchestra.run.smithi184.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-05-20T16:00:35.100 INFO:teuthology.orchestra.run.smithi184.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-05-20T16:00:35.100 INFO:teuthology.orchestra.run.smithi184.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-05-20T16:00:35.100 INFO:teuthology.orchestra.run.smithi184.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-05-20T16:00:35.100 INFO:teuthology.orchestra.run.smithi184.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-05-20T16:00:35.100 INFO:teuthology.orchestra.run.smithi184.stdout: libcephsqlite x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 158 k 2024-05-20T16:00:35.100 INFO:teuthology.orchestra.run.smithi184.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-05-20T16:00:35.100 INFO:teuthology.orchestra.run.smithi184.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-05-20T16:00:35.100 INFO:teuthology.orchestra.run.smithi184.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-05-20T16:00:35.100 INFO:teuthology.orchestra.run.smithi184.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-05-20T16:00:35.100 INFO:teuthology.orchestra.run.smithi184.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-05-20T16:00:35.100 INFO:teuthology.orchestra.run.smithi184.stdout: libradosstriper1 x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 497 k 2024-05-20T16:00:35.101 INFO:teuthology.orchestra.run.smithi184.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-05-20T16:00:35.101 INFO:teuthology.orchestra.run.smithi184.stdout: librgw2 x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 5.0 M 2024-05-20T16:00:35.101 INFO:teuthology.orchestra.run.smithi184.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-05-20T16:00:35.101 INFO:teuthology.orchestra.run.smithi184.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-05-20T16:00:35.101 INFO:teuthology.orchestra.run.smithi184.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-05-20T16:00:35.101 INFO:teuthology.orchestra.run.smithi184.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-05-20T16:00:35.101 INFO:teuthology.orchestra.run.smithi184.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-05-20T16:00:35.101 INFO:teuthology.orchestra.run.smithi184.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-05-20T16:00:35.101 INFO:teuthology.orchestra.run.smithi184.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-05-20T16:00:35.101 INFO:teuthology.orchestra.run.smithi184.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-05-20T16:00:35.101 INFO:teuthology.orchestra.run.smithi184.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-05-20T16:00:35.101 INFO:teuthology.orchestra.run.smithi184.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-05-20T16:00:35.101 INFO:teuthology.orchestra.run.smithi184.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-05-20T16:00:35.101 INFO:teuthology.orchestra.run.smithi184.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-05-20T16:00:35.102 INFO:teuthology.orchestra.run.smithi184.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-05-20T16:00:35.102 INFO:teuthology.orchestra.run.smithi184.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-05-20T16:00:35.102 INFO:teuthology.orchestra.run.smithi184.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-05-20T16:00:35.102 INFO:teuthology.orchestra.run.smithi184.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-05-20T16:00:35.102 INFO:teuthology.orchestra.run.smithi184.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-05-20T16:00:35.102 INFO:teuthology.orchestra.run.smithi184.stdout: python3-ceph-argparse x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 45 k 2024-05-20T16:00:35.102 INFO:teuthology.orchestra.run.smithi184.stdout: python3-ceph-common x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 135 k 2024-05-20T16:00:35.102 INFO:teuthology.orchestra.run.smithi184.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-05-20T16:00:35.102 INFO:teuthology.orchestra.run.smithi184.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-05-20T16:00:35.102 INFO:teuthology.orchestra.run.smithi184.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-05-20T16:00:35.102 INFO:teuthology.orchestra.run.smithi184.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-05-20T16:00:35.102 INFO:teuthology.orchestra.run.smithi184.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-05-20T16:00:35.103 INFO:teuthology.orchestra.run.smithi184.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-05-20T16:00:35.103 INFO:teuthology.orchestra.run.smithi184.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-05-20T16:00:35.103 INFO:teuthology.orchestra.run.smithi184.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-05-20T16:00:35.103 INFO:teuthology.orchestra.run.smithi184.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-05-20T16:00:35.103 INFO:teuthology.orchestra.run.smithi184.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-05-20T16:00:35.103 INFO:teuthology.orchestra.run.smithi184.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-05-20T16:00:35.103 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-05-20T16:00:35.103 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-05-20T16:00:35.103 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-05-20T16:00:35.103 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-05-20T16:00:35.103 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-05-20T16:00:35.103 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-05-20T16:00:35.103 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-05-20T16:00:35.104 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-05-20T16:00:35.104 INFO:teuthology.orchestra.run.smithi184.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-05-20T16:00:35.104 INFO:teuthology.orchestra.run.smithi184.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-05-20T16:00:35.104 INFO:teuthology.orchestra.run.smithi184.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-05-20T16:00:35.104 INFO:teuthology.orchestra.run.smithi184.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-05-20T16:00:35.104 INFO:teuthology.orchestra.run.smithi184.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-05-20T16:00:35.104 INFO:teuthology.orchestra.run.smithi184.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-05-20T16:00:35.104 INFO:teuthology.orchestra.run.smithi184.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-05-20T16:00:35.104 INFO:teuthology.orchestra.run.smithi184.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-05-20T16:00:35.104 INFO:teuthology.orchestra.run.smithi184.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-05-20T16:00:35.104 INFO:teuthology.orchestra.run.smithi184.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-05-20T16:00:35.104 INFO:teuthology.orchestra.run.smithi184.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-05-20T16:00:35.104 INFO:teuthology.orchestra.run.smithi184.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-05-20T16:00:35.104 INFO:teuthology.orchestra.run.smithi184.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-05-20T16:00:35.105 INFO:teuthology.orchestra.run.smithi184.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-05-20T16:00:35.105 INFO:teuthology.orchestra.run.smithi184.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-05-20T16:00:35.105 INFO:teuthology.orchestra.run.smithi184.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-05-20T16:00:35.105 INFO:teuthology.orchestra.run.smithi184.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-05-20T16:00:35.105 INFO:teuthology.orchestra.run.smithi184.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-05-20T16:00:35.105 INFO:teuthology.orchestra.run.smithi184.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-05-20T16:00:35.105 INFO:teuthology.orchestra.run.smithi184.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-05-20T16:00:35.105 INFO:teuthology.orchestra.run.smithi184.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-05-20T16:00:35.105 INFO:teuthology.orchestra.run.smithi184.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-05-20T16:00:35.105 INFO:teuthology.orchestra.run.smithi184.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-05-20T16:00:35.105 INFO:teuthology.orchestra.run.smithi184.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-05-20T16:00:35.105 INFO:teuthology.orchestra.run.smithi184.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-05-20T16:00:35.105 INFO:teuthology.orchestra.run.smithi184.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-05-20T16:00:35.105 INFO:teuthology.orchestra.run.smithi184.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-05-20T16:00:35.106 INFO:teuthology.orchestra.run.smithi184.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-05-20T16:00:35.106 INFO:teuthology.orchestra.run.smithi184.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-05-20T16:00:35.106 INFO:teuthology.orchestra.run.smithi184.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-05-20T16:00:35.106 INFO:teuthology.orchestra.run.smithi184.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-05-20T16:00:35.106 INFO:teuthology.orchestra.run.smithi184.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-05-20T16:00:35.106 INFO:teuthology.orchestra.run.smithi184.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-05-20T16:00:35.106 INFO:teuthology.orchestra.run.smithi184.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-05-20T16:00:35.106 INFO:teuthology.orchestra.run.smithi184.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-05-20T16:00:35.106 INFO:teuthology.orchestra.run.smithi184.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-05-20T16:00:35.106 INFO:teuthology.orchestra.run.smithi184.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-05-20T16:00:35.106 INFO:teuthology.orchestra.run.smithi184.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-05-20T16:00:35.106 INFO:teuthology.orchestra.run.smithi184.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-05-20T16:00:35.106 INFO:teuthology.orchestra.run.smithi184.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-05-20T16:00:35.107 INFO:teuthology.orchestra.run.smithi184.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-05-20T16:00:35.107 INFO:teuthology.orchestra.run.smithi184.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-05-20T16:00:35.107 INFO:teuthology.orchestra.run.smithi184.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-05-20T16:00:35.107 INFO:teuthology.orchestra.run.smithi184.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-05-20T16:00:35.107 INFO:teuthology.orchestra.run.smithi184.stdout:Installing weak dependencies: 2024-05-20T16:00:35.107 INFO:teuthology.orchestra.run.smithi184.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-05-20T16:00:35.107 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-05-20T16:00:35.107 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction Summary 2024-05-20T16:00:35.107 INFO:teuthology.orchestra.run.smithi184.stdout:======================================================================================= 2024-05-20T16:00:35.107 INFO:teuthology.orchestra.run.smithi184.stdout:Install 148 Packages 2024-05-20T16:00:35.107 INFO:teuthology.orchestra.run.smithi184.stdout:Upgrade 8 Packages 2024-05-20T16:00:35.107 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-05-20T16:00:35.108 INFO:teuthology.orchestra.run.smithi184.stdout:Total download size: 221 M 2024-05-20T16:00:35.108 INFO:teuthology.orchestra.run.smithi184.stdout:Downloading Packages: 2024-05-20T16:00:35.261 INFO:teuthology.orchestra.run.smithi082.stdout:(3/156): ceph-immutable-object-cache-19.0.0-274 588 kB/s | 147 kB 00:00 2024-05-20T16:00:36.470 INFO:teuthology.orchestra.run.smithi184.stdout:(1/156): ceph-19.0.0-2744.gd2b507c7.el9.x86_64. 13 kB/s | 6.6 kB 00:00 2024-05-20T16:00:36.953 INFO:teuthology.orchestra.run.smithi082.stdout:(4/156): ceph-base-19.0.0-2744.gd2b507c7.el9.x8 1.6 MB/s | 5.4 MB 00:03 2024-05-20T16:00:37.187 INFO:teuthology.orchestra.run.smithi082.stdout:(5/156): ceph-mds-19.0.0-2744.gd2b507c7.el9.x86 1.2 MB/s | 2.3 MB 00:01 2024-05-20T16:00:37.520 INFO:teuthology.orchestra.run.smithi184.stdout:(2/156): ceph-fuse-19.0.0-2744.gd2b507c7.el9.x8 822 kB/s | 863 kB 00:01 2024-05-20T16:00:37.720 INFO:teuthology.orchestra.run.smithi184.stdout:(3/156): ceph-immutable-object-cache-19.0.0-274 735 kB/s | 147 kB 00:00 2024-05-20T16:00:37.847 INFO:teuthology.orchestra.run.smithi160.stdout:Extra Packages for Enterprise Linux 1.8 MB/s | 21 MB 00:12 2024-05-20T16:00:38.020 INFO:teuthology.orchestra.run.smithi082.stdout:(6/156): ceph-mgr-19.0.0-2744.gd2b507c7.el9.x86 1.4 MB/s | 1.5 MB 00:01 2024-05-20T16:00:38.921 INFO:teuthology.orchestra.run.smithi184.stdout:(4/156): ceph-mds-19.0.0-2744.gd2b507c7.el9.x86 1.9 MB/s | 2.3 MB 00:01 2024-05-20T16:00:39.479 INFO:teuthology.orchestra.run.smithi184.stdout:(5/156): ceph-mgr-19.0.0-2744.gd2b507c7.el9.x86 2.7 MB/s | 1.5 MB 00:00 2024-05-20T16:00:40.862 INFO:teuthology.orchestra.run.smithi184.stdout:(6/156): ceph-mon-19.0.0-2744.gd2b507c7.el9.x86 3.5 MB/s | 4.9 MB 00:01 2024-05-20T16:00:41.212 INFO:teuthology.orchestra.run.smithi082.stdout:(7/156): ceph-mon-19.0.0-2744.gd2b507c7.el9.x86 1.2 MB/s | 4.9 MB 00:04 2024-05-20T16:00:43.038 INFO:teuthology.orchestra.run.smithi160.stdout:lab-extras 16 kB/s | 1.7 kB 00:00 2024-05-20T16:00:43.445 INFO:teuthology.orchestra.run.smithi184.stdout:(7/156): ceph-common-19.0.0-2744.gd2b507c7.el9. 2.6 MB/s | 19 MB 00:07 2024-05-20T16:00:44.137 INFO:teuthology.orchestra.run.smithi184.stdout:(8/156): ceph-base-19.0.0-2744.gd2b507c7.el9.x8 674 kB/s | 5.4 MB 00:08 2024-05-20T16:00:44.254 INFO:teuthology.orchestra.run.smithi184.stdout:(9/156): ceph-selinux-19.0.0-2744.gd2b507c7.el9 216 kB/s | 25 kB 00:00 2024-05-20T16:00:44.821 INFO:teuthology.orchestra.run.smithi184.stdout:(10/156): ceph-osd-19.0.0-2744.gd2b507c7.el9.x8 4.5 MB/s | 18 MB 00:03 2024-05-20T16:00:44.938 INFO:teuthology.orchestra.run.smithi184.stdout:(11/156): cephfs-mirror-19.0.0-2744.gd2b507c7.e 1.9 MB/s | 221 kB 00:00 2024-05-20T16:00:45.038 INFO:teuthology.orchestra.run.smithi184.stdout:(12/156): cephfs-shell-19.0.0-2744.gd2b507c7.el 271 kB/s | 27 kB 00:00 2024-05-20T16:00:45.147 INFO:teuthology.orchestra.run.smithi184.stdout:(13/156): libcephfs-devel-19.0.0-2744.gd2b507c7 292 kB/s | 32 kB 00:00 2024-05-20T16:00:45.256 INFO:teuthology.orchestra.run.smithi160.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-20T16:00:45.257 INFO:teuthology.orchestra.run.smithi160.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-20T16:00:45.267 INFO:teuthology.orchestra.run.smithi160.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-05-20T16:00:45.268 INFO:teuthology.orchestra.run.smithi160.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-05-20T16:00:45.269 INFO:teuthology.orchestra.run.smithi160.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-05-20T16:00:45.269 INFO:teuthology.orchestra.run.smithi160.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-05-20T16:00:45.271 INFO:teuthology.orchestra.run.smithi160.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-05-20T16:00:45.273 INFO:teuthology.orchestra.run.smithi160.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-05-20T16:00:45.274 INFO:teuthology.orchestra.run.smithi160.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-05-20T16:00:45.334 INFO:teuthology.orchestra.run.smithi160.stdout:Dependencies resolved. 2024-05-20T16:00:45.351 INFO:teuthology.orchestra.run.smithi160.stdout:======================================================================================= 2024-05-20T16:00:45.351 INFO:teuthology.orchestra.run.smithi160.stdout: Package Arch Version Repository Size 2024-05-20T16:00:45.351 INFO:teuthology.orchestra.run.smithi160.stdout:======================================================================================= 2024-05-20T16:00:45.352 INFO:teuthology.orchestra.run.smithi160.stdout:Installing: 2024-05-20T16:00:45.352 INFO:teuthology.orchestra.run.smithi160.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-05-20T16:00:45.352 INFO:teuthology.orchestra.run.smithi160.stdout: ceph x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 6.6 k 2024-05-20T16:00:45.352 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-base x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 5.4 M 2024-05-20T16:00:45.352 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-fuse x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 863 k 2024-05-20T16:00:45.352 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 147 k 2024-05-20T16:00:45.352 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-mgr x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 1.5 M 2024-05-20T16:00:45.352 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2744.gd2b507c7.el9 ceph-noarch 142 k 2024-05-20T16:00:45.352 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2744.gd2b507c7.el9 ceph-noarch 3.5 M 2024-05-20T16:00:45.352 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2744.gd2b507c7.el9 ceph-noarch 7.4 M 2024-05-20T16:00:45.352 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-mgr-rook noarch 2:19.0.0-2744.gd2b507c7.el9 ceph-noarch 49 k 2024-05-20T16:00:45.352 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-radosgw x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 9.8 M 2024-05-20T16:00:45.352 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-test x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 46 M 2024-05-20T16:00:45.353 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-volume noarch 2:19.0.0-2744.gd2b507c7.el9 ceph-noarch 264 k 2024-05-20T16:00:45.353 INFO:teuthology.orchestra.run.smithi160.stdout: cephadm noarch 2:19.0.0-2744.gd2b507c7.el9 ceph-noarch 763 k 2024-05-20T16:00:45.353 INFO:teuthology.orchestra.run.smithi160.stdout: cephfs-mirror x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 221 k 2024-05-20T16:00:45.353 INFO:teuthology.orchestra.run.smithi160.stdout: cephfs-shell x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 27 k 2024-05-20T16:00:45.353 INFO:teuthology.orchestra.run.smithi160.stdout: cephfs-top noarch 2:19.0.0-2744.gd2b507c7.el9 ceph-noarch 18 k 2024-05-20T16:00:45.353 INFO:teuthology.orchestra.run.smithi160.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-05-20T16:00:45.353 INFO:teuthology.orchestra.run.smithi160.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-05-20T16:00:45.353 INFO:teuthology.orchestra.run.smithi160.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-05-20T16:00:45.353 INFO:teuthology.orchestra.run.smithi160.stdout: libcephfs-devel x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 32 k 2024-05-20T16:00:45.353 INFO:teuthology.orchestra.run.smithi160.stdout: libcephfs2 x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 756 k 2024-05-20T16:00:45.353 INFO:teuthology.orchestra.run.smithi160.stdout: librados-devel x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 127 k 2024-05-20T16:00:45.353 INFO:teuthology.orchestra.run.smithi160.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-05-20T16:00:45.353 INFO:teuthology.orchestra.run.smithi160.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-05-20T16:00:45.354 INFO:teuthology.orchestra.run.smithi160.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-05-20T16:00:45.354 INFO:teuthology.orchestra.run.smithi160.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-05-20T16:00:45.354 INFO:teuthology.orchestra.run.smithi160.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-05-20T16:00:45.354 INFO:teuthology.orchestra.run.smithi160.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-05-20T16:00:45.354 INFO:teuthology.orchestra.run.smithi160.stdout: python3-cephfs x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 161 k 2024-05-20T16:00:45.354 INFO:teuthology.orchestra.run.smithi160.stdout: python3-rados x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 320 k 2024-05-20T16:00:45.354 INFO:teuthology.orchestra.run.smithi160.stdout: python3-rbd x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 299 k 2024-05-20T16:00:45.354 INFO:teuthology.orchestra.run.smithi160.stdout: python3-rgw x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 99 k 2024-05-20T16:00:45.354 INFO:teuthology.orchestra.run.smithi160.stdout: rbd-fuse x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 86 k 2024-05-20T16:00:45.354 INFO:teuthology.orchestra.run.smithi160.stdout: rbd-mirror x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 3.0 M 2024-05-20T16:00:45.354 INFO:teuthology.orchestra.run.smithi160.stdout: rbd-nbd x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 169 k 2024-05-20T16:00:45.354 INFO:teuthology.orchestra.run.smithi160.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-05-20T16:00:45.354 INFO:teuthology.orchestra.run.smithi160.stdout:Upgrading: 2024-05-20T16:00:45.355 INFO:teuthology.orchestra.run.smithi160.stdout: NetworkManager x86_64 1:1.47.90-1.el9 baseos 2.3 M 2024-05-20T16:00:45.355 INFO:teuthology.orchestra.run.smithi160.stdout: NetworkManager-libnm x86_64 1:1.47.90-1.el9 baseos 1.8 M 2024-05-20T16:00:45.355 INFO:teuthology.orchestra.run.smithi160.stdout: NetworkManager-team x86_64 1:1.47.90-1.el9 baseos 41 k 2024-05-20T16:00:45.355 INFO:teuthology.orchestra.run.smithi160.stdout: NetworkManager-tui x86_64 1:1.47.90-1.el9 baseos 248 k 2024-05-20T16:00:45.355 INFO:teuthology.orchestra.run.smithi160.stdout: librados2 x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 3.3 M 2024-05-20T16:00:45.355 INFO:teuthology.orchestra.run.smithi160.stdout: librbd1 x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 3.1 M 2024-05-20T16:00:45.355 INFO:teuthology.orchestra.run.smithi160.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-05-20T16:00:45.355 INFO:teuthology.orchestra.run.smithi160.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-05-20T16:00:45.355 INFO:teuthology.orchestra.run.smithi160.stdout:Installing dependencies: 2024-05-20T16:00:45.355 INFO:teuthology.orchestra.run.smithi160.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-05-20T16:00:45.355 INFO:teuthology.orchestra.run.smithi160.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-05-20T16:00:45.355 INFO:teuthology.orchestra.run.smithi160.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-05-20T16:00:45.355 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-common x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 19 M 2024-05-20T16:00:45.356 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2744.gd2b507c7.el9 ceph-noarch 32 k 2024-05-20T16:00:45.356 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-mds x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 2.3 M 2024-05-20T16:00:45.356 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2744.gd2b507c7.el9 ceph-noarch 249 k 2024-05-20T16:00:45.356 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-mon x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 4.9 M 2024-05-20T16:00:45.356 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-osd x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 18 M 2024-05-20T16:00:45.356 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2744.gd2b507c7.el9 ceph-noarch 16 k 2024-05-20T16:00:45.356 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-selinux x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 25 k 2024-05-20T16:00:45.356 INFO:teuthology.orchestra.run.smithi160.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-05-20T16:00:45.356 INFO:teuthology.orchestra.run.smithi160.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-05-20T16:00:45.356 INFO:teuthology.orchestra.run.smithi160.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-05-20T16:00:45.356 INFO:teuthology.orchestra.run.smithi160.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-05-20T16:00:45.356 INFO:teuthology.orchestra.run.smithi160.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-05-20T16:00:45.356 INFO:teuthology.orchestra.run.smithi160.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-05-20T16:00:45.356 INFO:teuthology.orchestra.run.smithi160.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-05-20T16:00:45.357 INFO:teuthology.orchestra.run.smithi160.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-05-20T16:00:45.357 INFO:teuthology.orchestra.run.smithi160.stdout: libcephsqlite x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 158 k 2024-05-20T16:00:45.357 INFO:teuthology.orchestra.run.smithi160.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-05-20T16:00:45.357 INFO:teuthology.orchestra.run.smithi160.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-05-20T16:00:45.357 INFO:teuthology.orchestra.run.smithi160.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-05-20T16:00:45.357 INFO:teuthology.orchestra.run.smithi160.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-05-20T16:00:45.357 INFO:teuthology.orchestra.run.smithi160.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-05-20T16:00:45.357 INFO:teuthology.orchestra.run.smithi160.stdout: libradosstriper1 x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 497 k 2024-05-20T16:00:45.357 INFO:teuthology.orchestra.run.smithi160.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-05-20T16:00:45.357 INFO:teuthology.orchestra.run.smithi160.stdout: librgw2 x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 5.0 M 2024-05-20T16:00:45.357 INFO:teuthology.orchestra.run.smithi160.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-05-20T16:00:45.357 INFO:teuthology.orchestra.run.smithi160.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-05-20T16:00:45.357 INFO:teuthology.orchestra.run.smithi160.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-05-20T16:00:45.358 INFO:teuthology.orchestra.run.smithi160.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-05-20T16:00:45.358 INFO:teuthology.orchestra.run.smithi160.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-05-20T16:00:45.358 INFO:teuthology.orchestra.run.smithi160.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-05-20T16:00:45.358 INFO:teuthology.orchestra.run.smithi160.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-05-20T16:00:45.358 INFO:teuthology.orchestra.run.smithi160.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-05-20T16:00:45.358 INFO:teuthology.orchestra.run.smithi160.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-05-20T16:00:45.358 INFO:teuthology.orchestra.run.smithi160.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-05-20T16:00:45.358 INFO:teuthology.orchestra.run.smithi160.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-05-20T16:00:45.358 INFO:teuthology.orchestra.run.smithi160.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-05-20T16:00:45.358 INFO:teuthology.orchestra.run.smithi160.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-05-20T16:00:45.358 INFO:teuthology.orchestra.run.smithi160.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-05-20T16:00:45.358 INFO:teuthology.orchestra.run.smithi160.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-05-20T16:00:45.358 INFO:teuthology.orchestra.run.smithi160.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-05-20T16:00:45.359 INFO:teuthology.orchestra.run.smithi160.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-05-20T16:00:45.359 INFO:teuthology.orchestra.run.smithi160.stdout: python3-ceph-argparse x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 45 k 2024-05-20T16:00:45.359 INFO:teuthology.orchestra.run.smithi160.stdout: python3-ceph-common x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 135 k 2024-05-20T16:00:45.359 INFO:teuthology.orchestra.run.smithi160.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-05-20T16:00:45.359 INFO:teuthology.orchestra.run.smithi160.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-05-20T16:00:45.359 INFO:teuthology.orchestra.run.smithi160.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-05-20T16:00:45.359 INFO:teuthology.orchestra.run.smithi160.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-05-20T16:00:45.359 INFO:teuthology.orchestra.run.smithi160.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-05-20T16:00:45.359 INFO:teuthology.orchestra.run.smithi160.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-05-20T16:00:45.359 INFO:teuthology.orchestra.run.smithi160.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-05-20T16:00:45.359 INFO:teuthology.orchestra.run.smithi160.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-05-20T16:00:45.359 INFO:teuthology.orchestra.run.smithi160.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-05-20T16:00:45.359 INFO:teuthology.orchestra.run.smithi160.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-05-20T16:00:45.360 INFO:teuthology.orchestra.run.smithi160.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-05-20T16:00:45.360 INFO:teuthology.orchestra.run.smithi160.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-05-20T16:00:45.360 INFO:teuthology.orchestra.run.smithi160.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-05-20T16:00:45.360 INFO:teuthology.orchestra.run.smithi160.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-05-20T16:00:45.360 INFO:teuthology.orchestra.run.smithi160.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-05-20T16:00:45.360 INFO:teuthology.orchestra.run.smithi160.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-05-20T16:00:45.360 INFO:teuthology.orchestra.run.smithi160.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-05-20T16:00:45.360 INFO:teuthology.orchestra.run.smithi160.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-05-20T16:00:45.360 INFO:teuthology.orchestra.run.smithi160.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-05-20T16:00:45.360 INFO:teuthology.orchestra.run.smithi160.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-05-20T16:00:45.360 INFO:teuthology.orchestra.run.smithi160.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-05-20T16:00:45.360 INFO:teuthology.orchestra.run.smithi160.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-05-20T16:00:45.360 INFO:teuthology.orchestra.run.smithi160.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-05-20T16:00:45.361 INFO:teuthology.orchestra.run.smithi160.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-05-20T16:00:45.361 INFO:teuthology.orchestra.run.smithi160.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-05-20T16:00:45.361 INFO:teuthology.orchestra.run.smithi160.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-05-20T16:00:45.361 INFO:teuthology.orchestra.run.smithi160.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-05-20T16:00:45.361 INFO:teuthology.orchestra.run.smithi160.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-05-20T16:00:45.361 INFO:teuthology.orchestra.run.smithi160.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-05-20T16:00:45.361 INFO:teuthology.orchestra.run.smithi160.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-05-20T16:00:45.361 INFO:teuthology.orchestra.run.smithi160.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-05-20T16:00:45.361 INFO:teuthology.orchestra.run.smithi160.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-05-20T16:00:45.361 INFO:teuthology.orchestra.run.smithi160.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-05-20T16:00:45.361 INFO:teuthology.orchestra.run.smithi160.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-05-20T16:00:45.361 INFO:teuthology.orchestra.run.smithi160.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-05-20T16:00:45.361 INFO:teuthology.orchestra.run.smithi160.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-05-20T16:00:45.362 INFO:teuthology.orchestra.run.smithi160.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-05-20T16:00:45.362 INFO:teuthology.orchestra.run.smithi160.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-05-20T16:00:45.362 INFO:teuthology.orchestra.run.smithi160.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-05-20T16:00:45.362 INFO:teuthology.orchestra.run.smithi160.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-05-20T16:00:45.362 INFO:teuthology.orchestra.run.smithi160.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-05-20T16:00:45.362 INFO:teuthology.orchestra.run.smithi160.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-05-20T16:00:45.362 INFO:teuthology.orchestra.run.smithi160.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-05-20T16:00:45.362 INFO:teuthology.orchestra.run.smithi160.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-05-20T16:00:45.362 INFO:teuthology.orchestra.run.smithi160.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-05-20T16:00:45.362 INFO:teuthology.orchestra.run.smithi160.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-05-20T16:00:45.362 INFO:teuthology.orchestra.run.smithi160.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-05-20T16:00:45.362 INFO:teuthology.orchestra.run.smithi160.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-05-20T16:00:45.362 INFO:teuthology.orchestra.run.smithi160.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-05-20T16:00:45.362 INFO:teuthology.orchestra.run.smithi160.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-05-20T16:00:45.363 INFO:teuthology.orchestra.run.smithi160.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-05-20T16:00:45.363 INFO:teuthology.orchestra.run.smithi160.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-05-20T16:00:45.363 INFO:teuthology.orchestra.run.smithi160.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-05-20T16:00:45.363 INFO:teuthology.orchestra.run.smithi160.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-05-20T16:00:45.363 INFO:teuthology.orchestra.run.smithi160.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-05-20T16:00:45.363 INFO:teuthology.orchestra.run.smithi160.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-05-20T16:00:45.363 INFO:teuthology.orchestra.run.smithi160.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-05-20T16:00:45.363 INFO:teuthology.orchestra.run.smithi160.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-05-20T16:00:45.363 INFO:teuthology.orchestra.run.smithi160.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-05-20T16:00:45.363 INFO:teuthology.orchestra.run.smithi160.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-05-20T16:00:45.363 INFO:teuthology.orchestra.run.smithi160.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-05-20T16:00:45.363 INFO:teuthology.orchestra.run.smithi160.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-05-20T16:00:45.363 INFO:teuthology.orchestra.run.smithi160.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-05-20T16:00:45.364 INFO:teuthology.orchestra.run.smithi160.stdout:Installing weak dependencies: 2024-05-20T16:00:45.364 INFO:teuthology.orchestra.run.smithi160.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-05-20T16:00:45.364 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-20T16:00:45.364 INFO:teuthology.orchestra.run.smithi160.stdout:Transaction Summary 2024-05-20T16:00:45.364 INFO:teuthology.orchestra.run.smithi160.stdout:======================================================================================= 2024-05-20T16:00:45.364 INFO:teuthology.orchestra.run.smithi160.stdout:Install 148 Packages 2024-05-20T16:00:45.364 INFO:teuthology.orchestra.run.smithi160.stdout:Upgrade 8 Packages 2024-05-20T16:00:45.364 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-20T16:00:45.365 INFO:teuthology.orchestra.run.smithi160.stdout:Total download size: 221 M 2024-05-20T16:00:45.365 INFO:teuthology.orchestra.run.smithi160.stdout:Downloading Packages: 2024-05-20T16:00:45.372 INFO:teuthology.orchestra.run.smithi184.stdout:(14/156): libcephfs2-19.0.0-2744.gd2b507c7.el9. 3.3 MB/s | 756 kB 00:00 2024-05-20T16:00:45.481 INFO:teuthology.orchestra.run.smithi184.stdout:(15/156): libcephsqlite-19.0.0-2744.gd2b507c7.e 1.4 MB/s | 158 kB 00:00 2024-05-20T16:00:45.581 INFO:teuthology.orchestra.run.smithi184.stdout:(16/156): librados-devel-19.0.0-2744.gd2b507c7. 1.2 MB/s | 127 kB 00:00 2024-05-20T16:00:45.790 INFO:teuthology.orchestra.run.smithi184.stdout:(17/156): libradosstriper1-19.0.0-2744.gd2b507c 2.3 MB/s | 497 kB 00:00 2024-05-20T16:00:46.723 INFO:teuthology.orchestra.run.smithi160.stdout:(1/156): ceph-19.0.0-2744.gd2b507c7.el9.x86_64. 14 kB/s | 6.6 kB 00:00 2024-05-20T16:00:47.374 INFO:teuthology.orchestra.run.smithi184.stdout:(18/156): librgw2-19.0.0-2744.gd2b507c7.el9.x86 3.2 MB/s | 5.0 MB 00:01 2024-05-20T16:00:47.482 INFO:teuthology.orchestra.run.smithi184.stdout:(19/156): python3-ceph-argparse-19.0.0-2744.gd2 418 kB/s | 45 kB 00:00 2024-05-20T16:00:47.599 INFO:teuthology.orchestra.run.smithi184.stdout:(20/156): python3-ceph-common-19.0.0-2744.gd2b5 1.1 MB/s | 135 kB 00:00 2024-05-20T16:00:47.716 INFO:teuthology.orchestra.run.smithi184.stdout:(21/156): python3-cephfs-19.0.0-2744.gd2b507c7. 1.4 MB/s | 161 kB 00:00 2024-05-20T16:00:47.916 INFO:teuthology.orchestra.run.smithi184.stdout:(22/156): python3-rados-19.0.0-2744.gd2b507c7.e 1.6 MB/s | 320 kB 00:00 2024-05-20T16:00:48.116 INFO:teuthology.orchestra.run.smithi184.stdout:(23/156): python3-rbd-19.0.0-2744.gd2b507c7.el9 1.5 MB/s | 299 kB 00:00 2024-05-20T16:00:48.160 INFO:teuthology.orchestra.run.smithi082.stdout:(8/156): ceph-radosgw-19.0.0-2744.gd2b507c7.el9 1.4 MB/s | 9.8 MB 00:06 2024-05-20T16:00:48.206 INFO:teuthology.orchestra.run.smithi160.stdout:(2/156): ceph-fuse-19.0.0-2744.gd2b507c7.el9.x8 582 kB/s | 863 kB 00:01 2024-05-20T16:00:48.216 INFO:teuthology.orchestra.run.smithi184.stdout:(24/156): python3-rgw-19.0.0-2744.gd2b507c7.el9 995 kB/s | 99 kB 00:00 2024-05-20T16:00:48.252 INFO:teuthology.orchestra.run.smithi082.stdout:(9/156): ceph-selinux-19.0.0-2744.gd2b507c7.el9 275 kB/s | 25 kB 00:00 2024-05-20T16:00:48.342 INFO:teuthology.orchestra.run.smithi184.stdout:(25/156): rbd-fuse-19.0.0-2744.gd2b507c7.el9.x8 685 kB/s | 86 kB 00:00 2024-05-20T16:00:48.406 INFO:teuthology.orchestra.run.smithi160.stdout:(3/156): ceph-immutable-object-cache-19.0.0-274 736 kB/s | 147 kB 00:00 2024-05-20T16:00:49.090 INFO:teuthology.orchestra.run.smithi160.stdout:(4/156): ceph-base-19.0.0-2744.gd2b507c7.el9.x8 1.9 MB/s | 5.4 MB 00:02 2024-05-20T16:00:49.334 INFO:teuthology.orchestra.run.smithi184.stdout:(26/156): ceph-radosgw-19.0.0-2744.gd2b507c7.el 1.7 MB/s | 9.8 MB 00:05 2024-05-20T16:00:49.534 INFO:teuthology.orchestra.run.smithi184.stdout:(27/156): rbd-nbd-19.0.0-2744.gd2b507c7.el9.x86 848 kB/s | 169 kB 00:00 2024-05-20T16:00:49.643 INFO:teuthology.orchestra.run.smithi184.stdout:(28/156): ceph-grafana-dashboards-19.0.0-2744.g 293 kB/s | 32 kB 00:00 2024-05-20T16:00:49.657 INFO:teuthology.orchestra.run.smithi160.stdout:(5/156): ceph-mgr-19.0.0-2744.gd2b507c7.el9.x86 2.6 MB/s | 1.5 MB 00:00 2024-05-20T16:00:49.702 INFO:teuthology.orchestra.run.smithi184.stdout:(29/156): rbd-mirror-19.0.0-2744.gd2b507c7.el9. 2.2 MB/s | 3.0 MB 00:01 2024-05-20T16:00:49.802 INFO:teuthology.orchestra.run.smithi184.stdout:(30/156): ceph-mgr-cephadm-19.0.0-2744.gd2b507c 897 kB/s | 142 kB 00:00 2024-05-20T16:00:51.299 INFO:teuthology.orchestra.run.smithi160.stdout:(6/156): ceph-mds-19.0.0-2744.gd2b507c7.el9.x86 800 kB/s | 2.3 MB 00:02 2024-05-20T16:00:51.582 INFO:teuthology.orchestra.run.smithi160.stdout:(7/156): ceph-mon-19.0.0-2744.gd2b507c7.el9.x86 2.5 MB/s | 4.9 MB 00:01 2024-05-20T16:00:51.628 INFO:teuthology.orchestra.run.smithi082.stdout:(10/156): ceph-common-19.0.0-2744.gd2b507c7.el9 1.1 MB/s | 19 MB 00:18 2024-05-20T16:00:51.843 INFO:teuthology.orchestra.run.smithi184.stdout:(31/156): ceph-mgr-dashboard-19.0.0-2744.gd2b50 1.6 MB/s | 3.5 MB 00:02 2024-05-20T16:00:51.869 INFO:teuthology.orchestra.run.smithi082.stdout:(11/156): cephfs-mirror-19.0.0-2744.gd2b507c7.e 915 kB/s | 221 kB 00:00 2024-05-20T16:00:51.978 INFO:teuthology.orchestra.run.smithi082.stdout:(12/156): cephfs-shell-19.0.0-2744.gd2b507c7.el 250 kB/s | 27 kB 00:00 2024-05-20T16:00:52.069 INFO:teuthology.orchestra.run.smithi184.stdout:(32/156): ceph-mgr-modules-core-19.0.0-2744.gd2 1.1 MB/s | 249 kB 00:00 2024-05-20T16:00:52.069 INFO:teuthology.orchestra.run.smithi082.stdout:(13/156): libcephfs-devel-19.0.0-2744.gd2b507c7 346 kB/s | 32 kB 00:00 2024-05-20T16:00:52.169 INFO:teuthology.orchestra.run.smithi184.stdout:(33/156): ceph-mgr-rook-19.0.0-2744.gd2b507c7.e 493 kB/s | 49 kB 00:00 2024-05-20T16:00:52.278 INFO:teuthology.orchestra.run.smithi184.stdout:(34/156): ceph-prometheus-alerts-19.0.0-2744.gd 152 kB/s | 16 kB 00:00 2024-05-20T16:00:52.495 INFO:teuthology.orchestra.run.smithi184.stdout:(35/156): ceph-volume-19.0.0-2744.gd2b507c7.el9 1.2 MB/s | 264 kB 00:00 2024-05-20T16:00:52.695 INFO:teuthology.orchestra.run.smithi082.stdout:(14/156): libcephfs2-19.0.0-2744.gd2b507c7.el9. 1.2 MB/s | 756 kB 00:00 2024-05-20T16:00:52.854 INFO:teuthology.orchestra.run.smithi082.stdout:(15/156): libcephsqlite-19.0.0-2744.gd2b507c7.e 997 kB/s | 158 kB 00:00 2024-05-20T16:00:53.037 INFO:teuthology.orchestra.run.smithi082.stdout:(16/156): librados-devel-19.0.0-2744.gd2b507c7. 692 kB/s | 127 kB 00:00 2024-05-20T16:00:53.086 INFO:teuthology.orchestra.run.smithi184.stdout:(36/156): cephadm-19.0.0-2744.gd2b507c7.el9.noa 1.3 MB/s | 763 kB 00:00 2024-05-20T16:00:53.203 INFO:teuthology.orchestra.run.smithi184.stdout:(37/156): cephfs-top-19.0.0-2744.gd2b507c7.el9. 156 kB/s | 18 kB 00:00 2024-05-20T16:00:53.387 INFO:teuthology.orchestra.run.smithi184.stdout:(38/156): fuse-2.9.9-15.el9.x86_64.rpm 439 kB/s | 80 kB 00:00 2024-05-20T16:00:53.496 INFO:teuthology.orchestra.run.smithi082.stdout:(17/156): libradosstriper1-19.0.0-2744.gd2b507c 1.1 MB/s | 497 kB 00:00 2024-05-20T16:00:53.654 INFO:teuthology.orchestra.run.smithi184.stdout:(39/156): libgfortran-11.4.1-3.el9.x86_64.rpm 2.9 MB/s | 803 kB 00:00 2024-05-20T16:00:53.746 INFO:teuthology.orchestra.run.smithi184.stdout:(40/156): libquadmath-11.4.1-3.el9.x86_64.rpm 2.1 MB/s | 195 kB 00:00 2024-05-20T16:00:53.847 INFO:teuthology.orchestra.run.smithi184.stdout:(41/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.5 MB/s | 253 kB 00:00 2024-05-20T16:00:54.230 INFO:teuthology.orchestra.run.smithi184.stdout:(42/156): python3-cryptography-36.0.1-4.el9.x86 3.2 MB/s | 1.2 MB 00:00 2024-05-20T16:00:54.314 INFO:teuthology.orchestra.run.smithi184.stdout:(43/156): python3-ply-3.11-14.el9.noarch.rpm 1.3 MB/s | 106 kB 00:00 2024-05-20T16:00:54.397 INFO:teuthology.orchestra.run.smithi184.stdout:(44/156): python3-pycparser-2.20-6.el9.noarch.r 1.6 MB/s | 135 kB 00:00 2024-05-20T16:00:54.490 INFO:teuthology.orchestra.run.smithi184.stdout:(45/156): python3-requests-2.25.1-8.el9.noarch. 1.3 MB/s | 125 kB 00:00 2024-05-20T16:00:54.615 INFO:teuthology.orchestra.run.smithi184.stdout:(46/156): python3-urllib3-1.26.5-5.el9.noarch.r 1.7 MB/s | 215 kB 00:00 2024-05-20T16:00:55.082 INFO:teuthology.orchestra.run.smithi184.stdout:(47/156): automake-1.16.2-8.el9.noarch.rpm 1.4 MB/s | 689 kB 00:00 2024-05-20T16:00:55.232 INFO:teuthology.orchestra.run.smithi184.stdout:(48/156): bison-3.7.4-5.el9.x86_64.rpm 6.1 MB/s | 943 kB 00:00 2024-05-20T16:00:55.299 INFO:teuthology.orchestra.run.smithi184.stdout:(49/156): boost-program-options-1.75.0-8.el9.x8 1.6 MB/s | 107 kB 00:00 2024-05-20T16:00:55.366 INFO:teuthology.orchestra.run.smithi184.stdout:(50/156): elfutils-libelf-devel-0.190-2.el9.x86 322 kB/s | 21 kB 00:00 2024-05-20T16:00:55.441 INFO:teuthology.orchestra.run.smithi184.stdout:(51/156): flex-2.6.4-9.el9.x86_64.rpm 4.1 MB/s | 313 kB 00:00 2024-05-20T16:00:55.508 INFO:teuthology.orchestra.run.smithi184.stdout:(52/156): flexiblas-3.0.4-8.el9.x86_64.rpm 491 kB/s | 33 kB 00:00 2024-05-20T16:00:55.800 INFO:teuthology.orchestra.run.smithi184.stdout:(53/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 10 MB/s | 3.0 MB 00:00 2024-05-20T16:00:55.840 INFO:teuthology.orchestra.run.smithi160.stdout:(8/156): ceph-radosgw-19.0.0-2744.gd2b507c7.el9 2.3 MB/s | 9.8 MB 00:04 2024-05-20T16:00:55.875 INFO:teuthology.orchestra.run.smithi184.stdout:(54/156): flexiblas-openblas-openmp-3.0.4-8.el9 243 kB/s | 18 kB 00:00 2024-05-20T16:00:55.934 INFO:teuthology.orchestra.run.smithi184.stdout:(55/156): libaio-devel-0.3.111-13.el9.x86_64.rp 191 kB/s | 11 kB 00:00 2024-05-20T16:00:55.949 INFO:teuthology.orchestra.run.smithi160.stdout:(9/156): ceph-selinux-19.0.0-2744.gd2b507c7.el9 233 kB/s | 25 kB 00:00 2024-05-20T16:00:55.993 INFO:teuthology.orchestra.run.smithi184.stdout:(56/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.7 MB/s | 160 kB 00:00 2024-05-20T16:00:56.051 INFO:teuthology.orchestra.run.smithi184.stdout:(57/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 779 kB/s | 45 kB 00:00 2024-05-20T16:00:56.129 INFO:teuthology.orchestra.run.smithi082.stdout:(18/156): ceph-osd-19.0.0-2744.gd2b507c7.el9.x8 1.0 MB/s | 18 MB 00:18 2024-05-20T16:00:56.160 INFO:teuthology.orchestra.run.smithi184.stdout:(58/156): librdkafka-1.6.1-102.el9.x86_64.rpm 6.0 MB/s | 662 kB 00:00 2024-05-20T16:00:56.235 INFO:teuthology.orchestra.run.smithi184.stdout:(59/156): libtool-2.4.6-45.el9.x86_64.rpm 7.6 MB/s | 582 kB 00:00 2024-05-20T16:00:56.237 INFO:teuthology.orchestra.run.smithi082.stdout:(19/156): python3-ceph-argparse-19.0.0-2744.gd2 418 kB/s | 45 kB 00:00 2024-05-20T16:00:56.294 INFO:teuthology.orchestra.run.smithi184.stdout:(60/156): libuuid-devel-2.37.4-18.el9.x86_64.rp 435 kB/s | 25 kB 00:00 2024-05-20T16:00:56.353 INFO:teuthology.orchestra.run.smithi184.stdout:(61/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 812 kB/s | 47 kB 00:00 2024-05-20T16:00:56.419 INFO:teuthology.orchestra.run.smithi184.stdout:(62/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.3 MB/s | 292 kB 00:00 2024-05-20T16:00:56.421 INFO:teuthology.orchestra.run.smithi082.stdout:(20/156): python3-ceph-common-19.0.0-2744.gd2b5 736 kB/s | 135 kB 00:00 2024-05-20T16:00:56.488 INFO:teuthology.orchestra.run.smithi082.stdout:(21/156): librgw2-19.0.0-2744.gd2b507c7.el9.x86 1.7 MB/s | 5.0 MB 00:02 2024-05-20T16:00:56.494 INFO:teuthology.orchestra.run.smithi184.stdout:(63/156): lua-5.4.4-4.el9.x86_64.rpm 2.5 MB/s | 188 kB 00:00 2024-05-20T16:00:56.553 INFO:teuthology.orchestra.run.smithi184.stdout:(64/156): openblas-0.3.21-2.el9.x86_64.rpm 603 kB/s | 35 kB 00:00 2024-05-20T16:00:56.630 INFO:teuthology.orchestra.run.smithi082.stdout:(22/156): python3-cephfs-19.0.0-2744.gd2b507c7. 773 kB/s | 161 kB 00:00 2024-05-20T16:00:56.737 INFO:teuthology.orchestra.run.smithi184.stdout:(65/156): ceph-mgr-diskprediction-local-19.0.0- 1.1 MB/s | 7.4 MB 00:06 2024-05-20T16:00:56.922 INFO:teuthology.orchestra.run.smithi082.stdout:(23/156): python3-rados-19.0.0-2744.gd2b507c7.e 738 kB/s | 320 kB 00:00 2024-05-20T16:00:56.946 INFO:teuthology.orchestra.run.smithi184.stdout:(66/156): openblas-openmp-0.3.21-2.el9.x86_64.r 12 MB/s | 4.7 MB 00:00 2024-05-20T16:00:57.022 INFO:teuthology.orchestra.run.smithi082.stdout:(24/156): python3-rgw-19.0.0-2744.gd2b507c7.el9 995 kB/s | 99 kB 00:00 2024-05-20T16:00:57.047 INFO:teuthology.orchestra.run.smithi082.stdout:(25/156): python3-rbd-19.0.0-2744.gd2b507c7.el9 717 kB/s | 299 kB 00:00 2024-05-20T16:00:57.079 INFO:teuthology.orchestra.run.smithi184.stdout:(67/156): postgresql-13.14-1.el9.x86_64.rpm 12 MB/s | 1.6 MB 00:00 2024-05-20T16:00:57.123 INFO:teuthology.orchestra.run.smithi082.stdout:(26/156): rbd-fuse-19.0.0-2744.gd2b507c7.el9.x8 855 kB/s | 86 kB 00:00 2024-05-20T16:00:57.155 INFO:teuthology.orchestra.run.smithi184.stdout:(68/156): postgresql-contrib-13.14-1.el9.x86_64 12 MB/s | 911 kB 00:00 2024-05-20T16:00:57.222 INFO:teuthology.orchestra.run.smithi184.stdout:(69/156): postgresql-private-libs-13.14-1.el9.x 2.0 MB/s | 133 kB 00:00 2024-05-20T16:00:57.290 INFO:teuthology.orchestra.run.smithi082.stdout:(27/156): rbd-nbd-19.0.0-2744.gd2b507c7.el9.x86 1.0 MB/s | 169 kB 00:00 2024-05-20T16:00:57.390 INFO:teuthology.orchestra.run.smithi082.stdout:(28/156): ceph-grafana-dashboards-19.0.0-2744.g 317 kB/s | 32 kB 00:00 2024-05-20T16:00:57.480 INFO:teuthology.orchestra.run.smithi184.stdout:(70/156): openssl-devel-3.2.1-1.el9.x86_64.rpm 6.0 MB/s | 4.4 MB 00:00 2024-05-20T16:00:57.549 INFO:teuthology.orchestra.run.smithi082.stdout:(29/156): ceph-mgr-cephadm-19.0.0-2744.gd2b507c 899 kB/s | 142 kB 00:00 2024-05-20T16:00:57.589 INFO:teuthology.orchestra.run.smithi184.stdout:(71/156): postgresql-server-13.14-1.el9.x86_64. 16 MB/s | 5.8 MB 00:00 2024-05-20T16:00:57.639 INFO:teuthology.orchestra.run.smithi184.stdout:(72/156): protobuf-3.14.0-13.el9.x86_64.rpm 6.4 MB/s | 1.0 MB 00:00 2024-05-20T16:00:57.664 INFO:teuthology.orchestra.run.smithi184.stdout:(73/156): python3-attrs-20.3.0-7.el9.noarch.rpm 1.1 MB/s | 87 kB 00:00 2024-05-20T16:00:57.731 INFO:teuthology.orchestra.run.smithi184.stdout:(74/156): python3-devel-3.9.18-3.el9.x86_64.rpm 3.6 MB/s | 244 kB 00:00 2024-05-20T16:00:57.799 INFO:teuthology.orchestra.run.smithi184.stdout:(75/156): python3-jinja2-2.11.3-5.el9.noarch.rp 3.6 MB/s | 248 kB 00:00 2024-05-20T16:00:57.865 INFO:teuthology.orchestra.run.smithi184.stdout:(76/156): python3-jsonpatch-1.21-16.el9.noarch. 385 kB/s | 26 kB 00:00 2024-05-20T16:00:57.932 INFO:teuthology.orchestra.run.smithi184.stdout:(77/156): python3-jsonpointer-2.0-4.el9.noarch. 287 kB/s | 19 kB 00:00 2024-05-20T16:00:58.000 INFO:teuthology.orchestra.run.smithi184.stdout:(78/156): python3-mako-1.1.4-6.el9.noarch.rpm 2.5 MB/s | 172 kB 00:00 2024-05-20T16:00:58.066 INFO:teuthology.orchestra.run.smithi184.stdout:(79/156): python3-markupsafe-1.1.1-12.el9.x86_6 522 kB/s | 35 kB 00:00 2024-05-20T16:00:58.150 INFO:teuthology.orchestra.run.smithi184.stdout:(80/156): python3-babel-2.9.1-2.el9.noarch.rpm 12 MB/s | 6.0 MB 00:00 2024-05-20T16:00:58.234 INFO:teuthology.orchestra.run.smithi184.stdout:(81/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.9 MB/s | 250 kB 00:00 2024-05-20T16:00:58.301 INFO:teuthology.orchestra.run.smithi184.stdout:(82/156): python3-oauthlib-3.1.1-5.el9.noarch.r 3.3 MB/s | 222 kB 00:00 2024-05-20T16:00:58.367 INFO:teuthology.orchestra.run.smithi184.stdout:(83/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 17 MB/s | 5.1 MB 00:00 2024-05-20T16:00:58.393 INFO:teuthology.orchestra.run.smithi184.stdout:(84/156): python3-packaging-20.9-5.el9.noarch.r 838 kB/s | 77 kB 00:00 2024-05-20T16:00:58.435 INFO:teuthology.orchestra.run.smithi184.stdout:(85/156): python3-prettytable-0.7.2-27.el9.noar 631 kB/s | 42 kB 00:00 2024-05-20T16:00:58.460 INFO:teuthology.orchestra.run.smithi184.stdout:(86/156): python3-protobuf-3.14.0-13.el9.noarch 3.9 MB/s | 269 kB 00:00 2024-05-20T16:00:58.502 INFO:teuthology.orchestra.run.smithi184.stdout:(87/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.3 MB/s | 159 kB 00:00 2024-05-20T16:00:58.527 INFO:teuthology.orchestra.run.smithi184.stdout:(88/156): python3-pyasn1-modules-0.4.8-6.el9.no 4.1 MB/s | 279 kB 00:00 2024-05-20T16:00:58.561 INFO:teuthology.orchestra.run.smithi184.stdout:(89/156): python3-pytz-2021.1-5.el9.noarch.rpm 868 kB/s | 51 kB 00:00 2024-05-20T16:00:58.595 INFO:teuthology.orchestra.run.smithi184.stdout:(90/156): python3-requests-oauthlib-1.3.0-12.el 801 kB/s | 54 kB 00:00 2024-05-20T16:00:58.662 INFO:teuthology.orchestra.run.smithi184.stdout:(91/156): python3-toml-0.10.2-6.el9.noarch.rpm 624 kB/s | 42 kB 00:00 2024-05-20T16:00:58.729 INFO:teuthology.orchestra.run.smithi184.stdout:(92/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 664 kB/s | 44 kB 00:00 2024-05-20T16:00:58.804 INFO:teuthology.orchestra.run.smithi184.stdout:(93/156): qatlib-24.02.0-1.el9.x86_64.rpm 2.9 MB/s | 221 kB 00:00 2024-05-20T16:00:58.888 INFO:teuthology.orchestra.run.smithi184.stdout:(94/156): qatlib-service-24.02.0-1.el9.x86_64.r 429 kB/s | 36 kB 00:00 2024-05-20T16:00:58.963 INFO:teuthology.orchestra.run.smithi184.stdout:(95/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 632 kB/s | 47 kB 00:00 2024-05-20T16:00:59.047 INFO:teuthology.orchestra.run.smithi184.stdout:(96/156): socat-1.7.4.1-5.el9.x86_64.rpm 3.6 MB/s | 305 kB 00:00 2024-05-20T16:00:59.130 INFO:teuthology.orchestra.run.smithi184.stdout:(97/156): uuid-1.6.2-55.el9.x86_64.rpm 694 kB/s | 58 kB 00:00 2024-05-20T16:00:59.289 INFO:teuthology.orchestra.run.smithi184.stdout:(98/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 22 MB/s | 16 MB 00:00 2024-05-20T16:00:59.290 INFO:teuthology.orchestra.run.smithi082.stdout:(30/156): ceph-mgr-dashboard-19.0.0-2744.gd2b50 2.0 MB/s | 3.5 MB 00:01 2024-05-20T16:00:59.314 INFO:teuthology.orchestra.run.smithi184.stdout:(99/156): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 510 kB/s | 94 kB 00:00 2024-05-20T16:00:59.348 INFO:teuthology.orchestra.run.smithi184.stdout:(100/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.1 MB/s | 64 kB 00:00 2024-05-20T16:00:59.373 INFO:teuthology.orchestra.run.smithi184.stdout:(101/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 762 kB/s | 45 kB 00:00 2024-05-20T16:00:59.758 INFO:teuthology.orchestra.run.smithi184.stdout:(102/156): lua-devel-5.4.4-4.el9.x86_64.rpm 54 kB/s | 22 kB 00:00 2024-05-20T16:01:00.257 INFO:teuthology.orchestra.run.smithi082.stdout:(31/156): rbd-mirror-19.0.0-2744.gd2b507c7.el9. 970 kB/s | 3.0 MB 00:03 2024-05-20T16:01:00.441 INFO:teuthology.orchestra.run.smithi184.stdout:(103/156): protobuf-compiler-3.14.0-13.el9.x86_ 809 kB/s | 863 kB 00:01 2024-05-20T16:01:00.474 INFO:teuthology.orchestra.run.smithi184.stdout:(104/156): abseil-cpp-20211102.0-3.el9.x86_64.r 765 kB/s | 548 kB 00:00 2024-05-20T16:01:00.557 INFO:teuthology.orchestra.run.smithi082.stdout:(32/156): ceph-mgr-modules-core-19.0.0-2744.gd2 830 kB/s | 249 kB 00:00 2024-05-20T16:01:00.585 INFO:teuthology.orchestra.run.smithi184.stdout:(105/156): grpc-data-1.46.7-10.el9.noarch.rpm 179 kB/s | 19 kB 00:00 2024-05-20T16:01:00.674 INFO:teuthology.orchestra.run.smithi082.stdout:(33/156): ceph-mgr-rook-19.0.0-2744.gd2b507c7.e 422 kB/s | 49 kB 00:00 2024-05-20T16:01:00.783 INFO:teuthology.orchestra.run.smithi082.stdout:(34/156): ceph-prometheus-alerts-19.0.0-2744.gd 152 kB/s | 16 kB 00:00 2024-05-20T16:01:01.083 INFO:teuthology.orchestra.run.smithi184.stdout:(106/156): libarrow-9.0.0-11.el9.x86_64.rpm 8.8 MB/s | 4.4 MB 00:00 2024-05-20T16:01:01.158 INFO:teuthology.orchestra.run.smithi184.stdout:(107/156): gperftools-libs-2.9.1-2.el9.x86_64.r 430 kB/s | 309 kB 00:00 2024-05-20T16:01:01.183 INFO:teuthology.orchestra.run.smithi184.stdout:(108/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 259 kB/s | 26 kB 00:00 2024-05-20T16:01:01.284 INFO:teuthology.orchestra.run.smithi184.stdout:(109/156): liboath-2.6.7-2.el9.x86_64.rpm 387 kB/s | 49 kB 00:00 2024-05-20T16:01:01.309 INFO:teuthology.orchestra.run.smithi184.stdout:(110/156): libunwind-1.6.2-1.el9.x86_64.rpm 538 kB/s | 67 kB 00:00 2024-05-20T16:01:01.443 INFO:teuthology.orchestra.run.smithi184.stdout:(111/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 6.2 MB/s | 839 kB 00:00 2024-05-20T16:01:01.466 INFO:teuthology.orchestra.run.smithi082.stdout:(35/156): ceph-volume-19.0.0-2744.gd2b507c7.el9 386 kB/s | 264 kB 00:00 2024-05-20T16:01:01.468 INFO:teuthology.orchestra.run.smithi184.stdout:(112/156): luarocks-3.9.2-5.el9.noarch.rpm 823 kB/s | 151 kB 00:00 2024-05-20T16:01:01.568 INFO:teuthology.orchestra.run.smithi184.stdout:(113/156): python3-asyncssh-2.13.2-5.el9.noarch 4.3 MB/s | 548 kB 00:00 2024-05-20T16:01:01.602 INFO:teuthology.orchestra.run.smithi184.stdout:(114/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 325 kB/s | 43 kB 00:00 2024-05-20T16:01:01.686 INFO:teuthology.orchestra.run.smithi184.stdout:(115/156): python3-cachetools-4.2.4-1.el9.noarc 275 kB/s | 32 kB 00:00 2024-05-20T16:01:01.711 INFO:teuthology.orchestra.run.smithi184.stdout:(116/156): python3-certifi-2021.10.8-2.el9.noar 138 kB/s | 15 kB 00:00 2024-05-20T16:01:01.811 INFO:teuthology.orchestra.run.smithi184.stdout:(117/156): python3-cheroot-8.6.0-4.el9.noarch.r 1.3 MB/s | 172 kB 00:00 2024-05-20T16:01:01.916 INFO:teuthology.orchestra.run.smithi160.stdout:(10/156): ceph-common-19.0.0-2744.gd2b507c7.el9 1.2 MB/s | 19 MB 00:15 2024-05-20T16:01:01.928 INFO:teuthology.orchestra.run.smithi184.stdout:(118/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.5 MB/s | 303 kB 00:00 2024-05-20T16:01:01.954 INFO:teuthology.orchestra.run.smithi184.stdout:(119/156): python3-cherrypy-18.6.1-2.el9.noarch 1.4 MB/s | 358 kB 00:00 2024-05-20T16:01:02.046 INFO:teuthology.orchestra.run.smithi184.stdout:(120/156): python3-colorama-0.4.4-12.el9.noarch 302 kB/s | 35 kB 00:00 2024-05-20T16:01:02.082 INFO:teuthology.orchestra.run.smithi160.stdout:(11/156): cephfs-mirror-19.0.0-2744.gd2b507c7.e 1.3 MB/s | 221 kB 00:00 2024-05-20T16:01:02.088 INFO:teuthology.orchestra.run.smithi184.stdout:(121/156): python3-google-auth-2.29.0-1.el9.noa 1.5 MB/s | 206 kB 00:00 2024-05-20T16:01:02.191 INFO:teuthology.orchestra.run.smithi160.stdout:(12/156): cephfs-shell-19.0.0-2744.gd2b507c7.el 250 kB/s | 27 kB 00:00 2024-05-20T16:01:02.213 INFO:teuthology.orchestra.run.smithi184.stdout:(122/156): python3-grpcio-1.46.7-10.el9.x86_64. 12 MB/s | 2.0 MB 00:00 2024-05-20T16:01:02.238 INFO:teuthology.orchestra.run.smithi184.stdout:(123/156): python3-grpcio-tools-1.46.7-10.el9.x 962 kB/s | 144 kB 00:00 2024-05-20T16:01:02.339 INFO:teuthology.orchestra.run.smithi184.stdout:(124/156): python3-jaraco-8.2.1-3.el9.noarch.rp 85 kB/s | 11 kB 00:00 2024-05-20T16:01:02.364 INFO:teuthology.orchestra.run.smithi184.stdout:(125/156): python3-jaraco-classes-3.2.1-5.el9.n 141 kB/s | 18 kB 00:00 2024-05-20T16:01:02.374 INFO:teuthology.orchestra.run.smithi160.stdout:(13/156): libcephfs-devel-19.0.0-2744.gd2b507c7 173 kB/s | 32 kB 00:00 2024-05-20T16:01:02.439 INFO:teuthology.orchestra.run.smithi184.stdout:(126/156): python3-jaraco-collections-3.0.0-8.e 231 kB/s | 23 kB 00:00 2024-05-20T16:01:02.473 INFO:teuthology.orchestra.run.smithi184.stdout:(127/156): python3-jaraco-functools-3.5.0-2.el9 179 kB/s | 19 kB 00:00 2024-05-20T16:01:02.540 INFO:teuthology.orchestra.run.smithi184.stdout:(128/156): python3-jaraco-text-3.2.0-6.el9.noar 196 kB/s | 20 kB 00:00 2024-05-20T16:01:02.567 INFO:teuthology.orchestra.run.smithi082.stdout:(36/156): cephadm-19.0.0-2744.gd2b507c7.el9.noa 693 kB/s | 763 kB 00:01 2024-05-20T16:01:02.648 INFO:teuthology.orchestra.run.smithi184.stdout:(129/156): python3-logutils-0.3.5-21.el9.noarch 427 kB/s | 46 kB 00:00 2024-05-20T16:01:02.692 INFO:teuthology.orchestra.run.smithi082.stdout:(37/156): cephfs-top-19.0.0-2744.gd2b507c7.el9. 146 kB/s | 18 kB 00:00 2024-05-20T16:01:02.732 INFO:teuthology.orchestra.run.smithi184.stdout:(130/156): python3-kubernetes-26.1.0-2.el9.noar 3.9 MB/s | 1.0 MB 00:00 2024-05-20T16:01:02.757 INFO:teuthology.orchestra.run.smithi184.stdout:(131/156): python3-more-itertools-8.12.0-2.el9. 726 kB/s | 79 kB 00:00 2024-05-20T16:01:02.842 INFO:teuthology.orchestra.run.smithi184.stdout:(132/156): python3-natsort-7.1.1-5.el9.noarch.r 531 kB/s | 58 kB 00:00 2024-05-20T16:01:02.859 INFO:teuthology.orchestra.run.smithi082.stdout:(38/156): fuse-2.9.9-15.el9.x86_64.rpm 482 kB/s | 80 kB 00:00 2024-05-20T16:01:02.875 INFO:teuthology.orchestra.run.smithi184.stdout:(133/156): python3-pecan-1.4.2-3.el9.noarch.rpm 2.3 MB/s | 272 kB 00:00 2024-05-20T16:01:02.950 INFO:teuthology.orchestra.run.smithi184.stdout:(134/156): python3-portend-3.1.0-2.el9.noarch.r 151 kB/s | 16 kB 00:00 2024-05-20T16:01:02.975 INFO:teuthology.orchestra.run.smithi160.stdout:(14/156): libcephfs2-19.0.0-2744.gd2b507c7.el9. 1.2 MB/s | 756 kB 00:00 2024-05-20T16:01:02.983 INFO:teuthology.orchestra.run.smithi184.stdout:(135/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 827 kB/s | 90 kB 00:00 2024-05-20T16:01:03.076 INFO:teuthology.orchestra.run.smithi184.stdout:(136/156): python3-pyperclip-1.8.2-2.el9.noarch 225 kB/s | 28 kB 00:00 2024-05-20T16:01:03.076 INFO:teuthology.orchestra.run.smithi082.stdout:(39/156): libgfortran-11.4.1-3.el9.x86_64.rpm 3.6 MB/s | 803 kB 00:00 2024-05-20T16:01:03.101 INFO:teuthology.orchestra.run.smithi184.stdout:(137/156): python3-repoze-lru-0.7-16.el9.noarch 262 kB/s | 31 kB 00:00 2024-05-20T16:01:03.166 INFO:teuthology.orchestra.run.smithi160.stdout:(15/156): libcephsqlite-19.0.0-2744.gd2b507c7.e 826 kB/s | 158 kB 00:00 2024-05-20T16:01:03.193 INFO:teuthology.orchestra.run.smithi082.stdout:(40/156): ceph-mgr-diskprediction-local-19.0.0- 1.9 MB/s | 7.4 MB 00:03 2024-05-20T16:01:03.210 INFO:teuthology.orchestra.run.smithi184.stdout:(138/156): python3-routes-2.5.1-5.el9.noarch.rp 1.4 MB/s | 188 kB 00:00 2024-05-20T16:01:03.218 INFO:teuthology.orchestra.run.smithi082.stdout:(41/156): libquadmath-11.4.1-3.el9.x86_64.rpm 1.3 MB/s | 195 kB 00:00 2024-05-20T16:01:03.235 INFO:teuthology.orchestra.run.smithi184.stdout:(139/156): python3-rsa-4.9-2.el9.noarch.rpm 443 kB/s | 59 kB 00:00 2024-05-20T16:01:03.327 INFO:teuthology.orchestra.run.smithi184.stdout:(140/156): python3-tempora-5.0.0-2.el9.noarch.r 306 kB/s | 36 kB 00:00 2024-05-20T16:01:03.358 INFO:teuthology.orchestra.run.smithi160.stdout:(16/156): librados-devel-19.0.0-2744.gd2b507c7. 662 kB/s | 127 kB 00:00 2024-05-20T16:01:03.360 INFO:teuthology.orchestra.run.smithi184.stdout:(141/156): python3-typing-extensions-4.4.0-2.el 406 kB/s | 51 kB 00:00 2024-05-20T16:01:03.469 INFO:teuthology.orchestra.run.smithi184.stdout:(142/156): python3-webob-1.8.7-6.el9.noarch.rpm 1.6 MB/s | 230 kB 00:00 2024-05-20T16:01:03.485 INFO:teuthology.orchestra.run.smithi082.stdout:(42/156): python3-cryptography-36.0.1-4.el9.x86 4.6 MB/s | 1.2 MB 00:00 2024-05-20T16:01:03.486 INFO:teuthology.orchestra.run.smithi184.stdout:(143/156): python3-websocket-client-1.2.3-2.el9 716 kB/s | 90 kB 00:00 2024-05-20T16:01:03.511 INFO:teuthology.orchestra.run.smithi082.stdout:(43/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 799 kB/s | 253 kB 00:00 2024-05-20T16:01:03.561 INFO:teuthology.orchestra.run.smithi082.stdout:(44/156): python3-ply-3.11-14.el9.noarch.rpm 1.4 MB/s | 106 kB 00:00 2024-05-20T16:01:03.620 INFO:teuthology.orchestra.run.smithi184.stdout:(144/156): python3-zc-lockfile-2.0-10.el9.noarc 150 kB/s | 20 kB 00:00 2024-05-20T16:01:03.636 INFO:teuthology.orchestra.run.smithi082.stdout:(45/156): python3-pycparser-2.20-6.el9.noarch.r 1.1 MB/s | 135 kB 00:00 2024-05-20T16:01:03.645 INFO:teuthology.orchestra.run.smithi184.stdout:(145/156): python3-werkzeug-2.0.3-3.el9.1.noarc 2.4 MB/s | 427 kB 00:00 2024-05-20T16:01:03.661 INFO:teuthology.orchestra.run.smithi082.stdout:(46/156): python3-requests-2.25.1-8.el9.noarch. 1.2 MB/s | 125 kB 00:00 2024-05-20T16:01:03.742 INFO:teuthology.orchestra.run.smithi160.stdout:(17/156): libradosstriper1-19.0.0-2744.gd2b507c 1.3 MB/s | 497 kB 00:00 2024-05-20T16:01:03.745 INFO:teuthology.orchestra.run.smithi082.stdout:(47/156): python3-urllib3-1.26.5-5.el9.noarch.r 1.9 MB/s | 215 kB 00:00 2024-05-20T16:01:03.762 INFO:teuthology.orchestra.run.smithi184.stdout:(146/156): re2-20211101-3.el9.x86_64.rpm 1.3 MB/s | 192 kB 00:00 2024-05-20T16:01:03.854 INFO:teuthology.orchestra.run.smithi184.stdout:(147/156): thrift-0.15.0-2.el9.x86_64.rpm 7.6 MB/s | 1.6 MB 00:00 2024-05-20T16:01:04.745 INFO:teuthology.orchestra.run.smithi082.stdout:(48/156): automake-1.16.2-8.el9.noarch.rpm 636 kB/s | 689 kB 00:01 2024-05-20T16:01:04.795 INFO:teuthology.orchestra.run.smithi082.stdout:(49/156): bison-3.7.4-5.el9.x86_64.rpm 898 kB/s | 943 kB 00:01 2024-05-20T16:01:04.871 INFO:teuthology.orchestra.run.smithi082.stdout:(50/156): boost-program-options-1.75.0-8.el9.x8 854 kB/s | 107 kB 00:00 2024-05-20T16:01:04.913 INFO:teuthology.orchestra.run.smithi082.stdout:(51/156): elfutils-libelf-devel-0.190-2.el9.x86 184 kB/s | 21 kB 00:00 2024-05-20T16:01:05.010 INFO:teuthology.orchestra.run.smithi160.stdout:(18/156): ceph-osd-19.0.0-2744.gd2b507c7.el9.x8 1.3 MB/s | 18 MB 00:13 2024-05-20T16:01:05.030 INFO:teuthology.orchestra.run.smithi082.stdout:(52/156): flexiblas-3.0.4-8.el9.x86_64.rpm 280 kB/s | 33 kB 00:00 2024-05-20T16:01:05.126 INFO:teuthology.orchestra.run.smithi160.stdout:(19/156): python3-ceph-argparse-19.0.0-2744.gd2 388 kB/s | 45 kB 00:00 2024-05-20T16:01:05.180 INFO:teuthology.orchestra.run.smithi082.stdout:(53/156): flex-2.6.4-9.el9.x86_64.rpm 1.0 MB/s | 313 kB 00:00 2024-05-20T16:01:05.235 INFO:teuthology.orchestra.run.smithi160.stdout:(20/156): python3-ceph-common-19.0.0-2744.gd2b5 1.2 MB/s | 135 kB 00:00 2024-05-20T16:01:05.297 INFO:teuthology.orchestra.run.smithi082.stdout:(54/156): flexiblas-openblas-openmp-3.0.4-8.el9 156 kB/s | 18 kB 00:00 2024-05-20T16:01:05.352 INFO:teuthology.orchestra.run.smithi160.stdout:(21/156): python3-cephfs-19.0.0-2744.gd2b507c7. 1.4 MB/s | 161 kB 00:00 2024-05-20T16:01:05.380 INFO:teuthology.orchestra.run.smithi082.stdout:(55/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 8.5 MB/s | 3.0 MB 00:00 2024-05-20T16:01:05.414 INFO:teuthology.orchestra.run.smithi082.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 95 kB/s | 11 kB 00:00 2024-05-20T16:01:05.477 INFO:teuthology.orchestra.run.smithi160.stdout:(22/156): python3-rados-19.0.0-2744.gd2b507c7.e 2.5 MB/s | 320 kB 00:00 2024-05-20T16:01:05.514 INFO:teuthology.orchestra.run.smithi082.stdout:(57/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.2 MB/s | 160 kB 00:00 2024-05-20T16:01:05.548 INFO:teuthology.orchestra.run.smithi082.stdout:(58/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 339 kB/s | 45 kB 00:00 2024-05-20T16:01:05.594 INFO:teuthology.orchestra.run.smithi160.stdout:(23/156): python3-rbd-19.0.0-2744.gd2b507c7.el9 2.5 MB/s | 299 kB 00:00 2024-05-20T16:01:05.648 INFO:teuthology.orchestra.run.smithi082.stdout:(59/156): librdkafka-1.6.1-102.el9.x86_64.rpm 4.8 MB/s | 662 kB 00:00 2024-05-20T16:01:05.711 INFO:teuthology.orchestra.run.smithi160.stdout:(24/156): python3-rgw-19.0.0-2744.gd2b507c7.el9 851 kB/s | 99 kB 00:00 2024-05-20T16:01:05.773 INFO:teuthology.orchestra.run.smithi082.stdout:(60/156): libuuid-devel-2.37.4-18.el9.x86_64.rp 203 kB/s | 25 kB 00:00 2024-05-20T16:01:05.828 INFO:teuthology.orchestra.run.smithi160.stdout:(25/156): rbd-fuse-19.0.0-2744.gd2b507c7.el9.x8 734 kB/s | 86 kB 00:00 2024-05-20T16:01:05.890 INFO:teuthology.orchestra.run.smithi082.stdout:(61/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 407 kB/s | 47 kB 00:00 2024-05-20T16:01:05.957 INFO:teuthology.orchestra.run.smithi082.stdout:(62/156): libtool-2.4.6-45.el9.x86_64.rpm 1.4 MB/s | 582 kB 00:00 2024-05-20T16:01:06.008 INFO:teuthology.orchestra.run.smithi082.stdout:(63/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.4 MB/s | 292 kB 00:00 2024-05-20T16:01:06.075 INFO:teuthology.orchestra.run.smithi082.stdout:(64/156): lua-5.4.4-4.el9.x86_64.rpm 1.6 MB/s | 188 kB 00:00 2024-05-20T16:01:06.109 INFO:teuthology.orchestra.run.smithi082.stdout:(65/156): openblas-0.3.21-2.el9.x86_64.rpm 350 kB/s | 35 kB 00:00 2024-05-20T16:01:06.546 INFO:teuthology.orchestra.run.smithi184.stdout:(148/156): librados2-19.0.0-2744.gd2b507c7.el9. 1.2 MB/s | 3.3 MB 00:02 2024-05-20T16:01:06.762 INFO:teuthology.orchestra.run.smithi160.stdout:(26/156): rbd-mirror-19.0.0-2744.gd2b507c7.el9. 3.3 MB/s | 3.0 MB 00:00 2024-05-20T16:01:06.879 INFO:teuthology.orchestra.run.smithi160.stdout:(27/156): rbd-nbd-19.0.0-2744.gd2b507c7.el9.x86 1.4 MB/s | 169 kB 00:00 2024-05-20T16:01:06.904 INFO:teuthology.orchestra.run.smithi184.stdout:(149/156): NetworkManager-1.47.90-1.el9.x86_64. 6.5 MB/s | 2.3 MB 00:00 2024-05-20T16:01:06.987 INFO:teuthology.orchestra.run.smithi160.stdout:(28/156): ceph-grafana-dashboards-19.0.0-2744.g 293 kB/s | 32 kB 00:00 2024-05-20T16:01:06.988 INFO:teuthology.orchestra.run.smithi184.stdout:(150/156): librbd1-19.0.0-2744.gd2b507c7.el9.x8 1.0 MB/s | 3.1 MB 00:03 2024-05-20T16:01:07.105 INFO:teuthology.orchestra.run.smithi184.stdout:(151/156): NetworkManager-libnm-1.47.90-1.el9.x 9.0 MB/s | 1.8 MB 00:00 2024-05-20T16:01:07.112 INFO:teuthology.orchestra.run.smithi160.stdout:(29/156): ceph-mgr-cephadm-19.0.0-2744.gd2b507c 1.1 MB/s | 142 kB 00:00 2024-05-20T16:01:07.130 INFO:teuthology.orchestra.run.smithi184.stdout:(152/156): NetworkManager-team-1.47.90-1.el9.x8 293 kB/s | 41 kB 00:00 2024-05-20T16:01:07.372 INFO:teuthology.orchestra.run.smithi184.stdout:(153/156): NetworkManager-tui-1.47.90-1.el9.x86 928 kB/s | 248 kB 00:00 2024-05-20T16:01:07.631 INFO:teuthology.orchestra.run.smithi184.stdout:(154/156): openssl-3.2.1-1.el9.x86_64.rpm 2.7 MB/s | 1.3 MB 00:00 2024-05-20T16:01:07.748 INFO:teuthology.orchestra.run.smithi184.stdout:(155/156): openssl-libs-3.2.1-1.el9.x86_64.rpm 6.4 MB/s | 2.4 MB 00:00 2024-05-20T16:01:07.888 INFO:teuthology.orchestra.run.smithi160.stdout:(30/156): librgw2-19.0.0-2744.gd2b507c7.el9.x86 1.2 MB/s | 5.0 MB 00:04 2024-05-20T16:01:08.005 INFO:teuthology.orchestra.run.smithi160.stdout:(31/156): ceph-mgr-dashboard-19.0.0-2744.gd2b50 3.9 MB/s | 3.5 MB 00:00 2024-05-20T16:01:08.130 INFO:teuthology.orchestra.run.smithi160.stdout:(32/156): ceph-mgr-modules-core-19.0.0-2744.gd2 1.9 MB/s | 249 kB 00:00 2024-05-20T16:01:08.238 INFO:teuthology.orchestra.run.smithi160.stdout:(33/156): ceph-mgr-rook-19.0.0-2744.gd2b507c7.e 455 kB/s | 49 kB 00:00 2024-05-20T16:01:08.347 INFO:teuthology.orchestra.run.smithi160.stdout:(34/156): ceph-prometheus-alerts-19.0.0-2744.gd 152 kB/s | 16 kB 00:00 2024-05-20T16:01:08.425 INFO:teuthology.orchestra.run.smithi082.stdout:(66/156): openblas-openmp-0.3.21-2.el9.x86_64.r 2.0 MB/s | 4.7 MB 00:02 2024-05-20T16:01:08.456 INFO:teuthology.orchestra.run.smithi160.stdout:(35/156): ceph-volume-19.0.0-2744.gd2b507c7.el9 2.4 MB/s | 264 kB 00:00 2024-05-20T16:01:08.664 INFO:teuthology.orchestra.run.smithi160.stdout:(36/156): cephadm-19.0.0-2744.gd2b507c7.el9.noa 3.6 MB/s | 763 kB 00:00 2024-05-20T16:01:09.123 INFO:teuthology.orchestra.run.smithi160.stdout:(37/156): ceph-test-19.0.0-2744.gd2b507c7.el9.x 3.5 MB/s | 46 MB 00:13 2024-05-20T16:01:09.125 INFO:teuthology.orchestra.run.smithi082.stdout:(67/156): postgresql-13.14-1.el9.x86_64.rpm 2.3 MB/s | 1.6 MB 00:00 2024-05-20T16:01:09.148 INFO:teuthology.orchestra.run.smithi160.stdout:(38/156): cephfs-top-19.0.0-2744.gd2b507c7.el9. 38 kB/s | 18 kB 00:00 2024-05-20T16:01:09.357 INFO:teuthology.orchestra.run.smithi160.stdout:(39/156): fuse-2.9.9-15.el9.x86_64.rpm 344 kB/s | 80 kB 00:00 2024-05-20T16:01:09.434 INFO:teuthology.orchestra.run.smithi082.stdout:(68/156): postgresql-contrib-13.14-1.el9.x86_64 2.9 MB/s | 911 kB 00:00 2024-05-20T16:01:09.549 INFO:teuthology.orchestra.run.smithi160.stdout:(40/156): libquadmath-11.4.1-3.el9.x86_64.rpm 1.0 MB/s | 195 kB 00:00 2024-05-20T16:01:09.551 INFO:teuthology.orchestra.run.smithi082.stdout:(69/156): postgresql-private-libs-13.14-1.el9.x 1.1 MB/s | 133 kB 00:00 2024-05-20T16:01:09.624 INFO:teuthology.orchestra.run.smithi160.stdout:(41/156): libgfortran-11.4.1-3.el9.x86_64.rpm 1.6 MB/s | 803 kB 00:00 2024-05-20T16:01:09.708 INFO:teuthology.orchestra.run.smithi160.stdout:(42/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.6 MB/s | 253 kB 00:00 2024-05-20T16:01:09.783 INFO:teuthology.orchestra.run.smithi160.stdout:(43/156): python3-ply-3.11-14.el9.noarch.rpm 1.4 MB/s | 106 kB 00:00 2024-05-20T16:01:09.900 INFO:teuthology.orchestra.run.smithi160.stdout:(44/156): python3-pycparser-2.20-6.el9.noarch.r 1.1 MB/s | 135 kB 00:00 2024-05-20T16:01:09.959 INFO:teuthology.orchestra.run.smithi082.stdout:(70/156): openssl-devel-3.2.1-1.el9.x86_64.rpm 1.1 MB/s | 4.4 MB 00:03 2024-05-20T16:01:10.001 INFO:teuthology.orchestra.run.smithi160.stdout:(45/156): python3-cryptography-36.0.1-4.el9.x86 3.3 MB/s | 1.2 MB 00:00 2024-05-20T16:01:10.026 INFO:teuthology.orchestra.run.smithi160.stdout:(46/156): python3-requests-2.25.1-8.el9.noarch. 997 kB/s | 125 kB 00:00 2024-05-20T16:01:10.084 INFO:teuthology.orchestra.run.smithi160.stdout:(47/156): python3-urllib3-1.26.5-5.el9.noarch.r 2.5 MB/s | 215 kB 00:00 2024-05-20T16:01:10.318 INFO:teuthology.orchestra.run.smithi160.stdout:(48/156): automake-1.16.2-8.el9.noarch.rpm 2.3 MB/s | 689 kB 00:00 2024-05-20T16:01:10.377 INFO:teuthology.orchestra.run.smithi160.stdout:(49/156): boost-program-options-1.75.0-8.el9.x8 1.8 MB/s | 107 kB 00:00 2024-05-20T16:01:10.410 INFO:teuthology.orchestra.run.smithi160.stdout:(50/156): bison-3.7.4-5.el9.x86_64.rpm 2.8 MB/s | 943 kB 00:00 2024-05-20T16:01:10.435 INFO:teuthology.orchestra.run.smithi160.stdout:(51/156): elfutils-libelf-devel-0.190-2.el9.x86 368 kB/s | 21 kB 00:00 2024-05-20T16:01:10.469 INFO:teuthology.orchestra.run.smithi160.stdout:(52/156): flex-2.6.4-9.el9.x86_64.rpm 5.2 MB/s | 313 kB 00:00 2024-05-20T16:01:10.494 INFO:teuthology.orchestra.run.smithi160.stdout:(53/156): flexiblas-3.0.4-8.el9.x86_64.rpm 558 kB/s | 33 kB 00:00 2024-05-20T16:01:10.544 INFO:teuthology.orchestra.run.smithi160.stdout:(54/156): flexiblas-openblas-openmp-3.0.4-8.el9 365 kB/s | 18 kB 00:00 2024-05-20T16:01:10.595 INFO:teuthology.orchestra.run.smithi160.stdout:(55/156): libaio-devel-0.3.111-13.el9.x86_64.rp 223 kB/s | 11 kB 00:00 2024-05-20T16:01:10.645 INFO:teuthology.orchestra.run.smithi160.stdout:(56/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 17 MB/s | 3.0 MB 00:00 2024-05-20T16:01:10.670 INFO:teuthology.orchestra.run.smithi160.stdout:(57/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.1 MB/s | 160 kB 00:00 2024-05-20T16:01:10.704 INFO:teuthology.orchestra.run.smithi160.stdout:(58/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 775 kB/s | 45 kB 00:00 2024-05-20T16:01:10.763 INFO:teuthology.orchestra.run.smithi160.stdout:(59/156): librdkafka-1.6.1-102.el9.x86_64.rpm 7.1 MB/s | 662 kB 00:00 2024-05-20T16:01:10.796 INFO:teuthology.orchestra.run.smithi160.stdout:(60/156): libtool-2.4.6-45.el9.x86_64.rpm 6.2 MB/s | 582 kB 00:00 2024-05-20T16:01:10.821 INFO:teuthology.orchestra.run.smithi160.stdout:(61/156): libuuid-devel-2.37.4-18.el9.x86_64.rp 435 kB/s | 25 kB 00:00 2024-05-20T16:01:10.855 INFO:teuthology.orchestra.run.smithi160.stdout:(62/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 812 kB/s | 47 kB 00:00 2024-05-20T16:01:10.880 INFO:teuthology.orchestra.run.smithi160.stdout:(63/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.9 MB/s | 292 kB 00:00 2024-05-20T16:01:10.914 INFO:teuthology.orchestra.run.smithi160.stdout:(64/156): lua-5.4.4-4.el9.x86_64.rpm 3.1 MB/s | 188 kB 00:00 2024-05-20T16:01:10.939 INFO:teuthology.orchestra.run.smithi160.stdout:(65/156): openblas-0.3.21-2.el9.x86_64.rpm 601 kB/s | 35 kB 00:00 2024-05-20T16:01:11.040 INFO:teuthology.orchestra.run.smithi160.stdout:(66/156): openblas-openmp-0.3.21-2.el9.x86_64.r 38 MB/s | 4.7 MB 00:00 2024-05-20T16:01:11.132 INFO:teuthology.orchestra.run.smithi160.stdout:(67/156): openssl-devel-3.2.1-1.el9.x86_64.rpm 23 MB/s | 4.4 MB 00:00 2024-05-20T16:01:11.173 INFO:teuthology.orchestra.run.smithi160.stdout:(68/156): postgresql-13.14-1.el9.x86_64.rpm 12 MB/s | 1.6 MB 00:00 2024-05-20T16:01:11.207 INFO:teuthology.orchestra.run.smithi160.stdout:(69/156): postgresql-contrib-13.14-1.el9.x86_64 12 MB/s | 911 kB 00:00 2024-05-20T16:01:11.232 INFO:teuthology.orchestra.run.smithi160.stdout:(70/156): postgresql-private-libs-13.14-1.el9.x 2.2 MB/s | 133 kB 00:00 2024-05-20T16:01:11.308 INFO:teuthology.orchestra.run.smithi160.stdout:(71/156): protobuf-3.14.0-13.el9.x86_64.rpm 13 MB/s | 1.0 MB 00:00 2024-05-20T16:01:11.383 INFO:teuthology.orchestra.run.smithi160.stdout:(72/156): postgresql-server-13.14-1.el9.x86_64. 33 MB/s | 5.8 MB 00:00 2024-05-20T16:01:11.408 INFO:teuthology.orchestra.run.smithi160.stdout:(73/156): python3-attrs-20.3.0-7.el9.noarch.rpm 873 kB/s | 87 kB 00:00 2024-05-20T16:01:11.492 INFO:teuthology.orchestra.run.smithi160.stdout:(74/156): ceph-mgr-diskprediction-local-19.0.0- 2.0 MB/s | 7.4 MB 00:03 2024-05-20T16:01:11.497 INFO:teuthology.orchestra.run.smithi184.stdout:(156/156): ceph-test-19.0.0-2744.gd2b507c7.el9. 1.7 MB/s | 46 MB 00:27 2024-05-20T16:01:11.501 INFO:teuthology.orchestra.run.smithi184.stdout:-------------------------------------------------------------------------------- 2024-05-20T16:01:11.501 INFO:teuthology.orchestra.run.smithi184.stdout:Total 6.1 MB/s | 221 MB 00:36 2024-05-20T16:01:11.517 INFO:teuthology.orchestra.run.smithi160.stdout:(75/156): python3-devel-3.9.18-3.el9.x86_64.rpm 2.2 MB/s | 244 kB 00:00 2024-05-20T16:01:11.576 INFO:teuthology.orchestra.run.smithi160.stdout:(76/156): python3-jsonpatch-1.21-16.el9.noarch. 439 kB/s | 26 kB 00:00 2024-05-20T16:01:11.576 INFO:teuthology.orchestra.run.smithi082.stdout:(71/156): ceph-test-19.0.0-2744.gd2b507c7.el9.x 2.0 MB/s | 46 MB 00:23 2024-05-20T16:01:11.652 INFO:teuthology.orchestra.run.smithi082.stdout:(72/156): postgresql-server-13.14-1.el9.x86_64. 2.8 MB/s | 5.8 MB 00:02 2024-05-20T16:01:11.652 INFO:teuthology.orchestra.run.smithi160.stdout:(77/156): python3-babel-2.9.1-2.el9.noarch.rpm 22 MB/s | 6.0 MB 00:00 2024-05-20T16:01:11.676 INFO:teuthology.orchestra.run.smithi160.stdout:(78/156): python3-jsonpointer-2.0-4.el9.noarch. 191 kB/s | 19 kB 00:00 2024-05-20T16:01:11.710 INFO:teuthology.orchestra.run.smithi160.stdout:(79/156): python3-mako-1.1.4-6.el9.noarch.rpm 2.9 MB/s | 172 kB 00:00 2024-05-20T16:01:11.735 INFO:teuthology.orchestra.run.smithi082.stdout:(73/156): protobuf-3.14.0-13.el9.x86_64.rpm 582 kB/s | 1.0 MB 00:01 2024-05-20T16:01:11.735 INFO:teuthology.orchestra.run.smithi160.stdout:(80/156): python3-markupsafe-1.1.1-12.el9.x86_6 592 kB/s | 35 kB 00:00 2024-05-20T16:01:11.761 INFO:teuthology.orchestra.run.smithi160.stdout:(81/156): python3-jinja2-2.11.3-5.el9.noarch.rp 924 kB/s | 248 kB 00:00 2024-05-20T16:01:11.794 INFO:teuthology.orchestra.run.smithi160.stdout:(82/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 4.2 MB/s | 250 kB 00:00 2024-05-20T16:01:11.870 INFO:teuthology.orchestra.run.smithi160.stdout:(83/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 32 MB/s | 5.1 MB 00:00 2024-05-20T16:01:11.887 INFO:teuthology.orchestra.run.smithi160.stdout:(84/156): python3-packaging-20.9-5.el9.noarch.r 840 kB/s | 77 kB 00:00 2024-05-20T16:01:11.912 INFO:teuthology.orchestra.run.smithi160.stdout:(85/156): python3-oauthlib-3.1.1-5.el9.noarch.r 1.4 MB/s | 222 kB 00:00 2024-05-20T16:01:11.929 INFO:teuthology.orchestra.run.smithi160.stdout:(86/156): python3-prettytable-0.7.2-27.el9.noar 721 kB/s | 42 kB 00:00 2024-05-20T16:01:11.954 INFO:teuthology.orchestra.run.smithi160.stdout:(87/156): python3-protobuf-3.14.0-13.el9.noarch 3.9 MB/s | 269 kB 00:00 2024-05-20T16:01:11.979 INFO:teuthology.orchestra.run.smithi160.stdout:(88/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.3 MB/s | 159 kB 00:00 2024-05-20T16:01:12.005 INFO:teuthology.orchestra.run.smithi160.stdout:(89/156): python3-pyasn1-modules-0.4.8-6.el9.no 3.6 MB/s | 279 kB 00:00 2024-05-20T16:01:12.021 INFO:teuthology.orchestra.run.smithi160.stdout:(90/156): python3-pytz-2021.1-5.el9.noarch.rpm 760 kB/s | 51 kB 00:00 2024-05-20T16:01:12.038 INFO:teuthology.orchestra.run.smithi160.stdout:(91/156): python3-requests-oauthlib-1.3.0-12.el 912 kB/s | 54 kB 00:00 2024-05-20T16:01:12.072 INFO:teuthology.orchestra.run.smithi160.stdout:(92/156): python3-toml-0.10.2-6.el9.noarch.rpm 828 kB/s | 42 kB 00:00 2024-05-20T16:01:12.093 INFO:teuthology.orchestra.run.smithi082.stdout:(74/156): python3-attrs-20.3.0-7.el9.noarch.rpm 169 kB/s | 87 kB 00:00 2024-05-20T16:01:12.097 INFO:teuthology.orchestra.run.smithi160.stdout:(93/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 754 kB/s | 44 kB 00:00 2024-05-20T16:01:12.139 INFO:teuthology.orchestra.run.smithi160.stdout:(94/156): qatlib-24.02.0-1.el9.x86_64.rpm 3.2 MB/s | 221 kB 00:00 2024-05-20T16:01:12.144 INFO:teuthology.orchestra.run.smithi082.stdout:(75/156): python3-devel-3.9.18-3.el9.x86_64.rpm 598 kB/s | 244 kB 00:00 2024-05-20T16:01:12.164 INFO:teuthology.orchestra.run.smithi160.stdout:(95/156): qatlib-service-24.02.0-1.el9.x86_64.r 535 kB/s | 36 kB 00:00 2024-05-20T16:01:12.198 INFO:teuthology.orchestra.run.smithi160.stdout:(96/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 810 kB/s | 47 kB 00:00 2024-05-20T16:01:12.240 INFO:teuthology.orchestra.run.smithi160.stdout:(97/156): socat-1.7.4.1-5.el9.x86_64.rpm 4.0 MB/s | 305 kB 00:00 2024-05-20T16:01:12.261 INFO:teuthology.orchestra.run.smithi082.stdout:(76/156): python3-jsonpatch-1.21-16.el9.noarch. 219 kB/s | 26 kB 00:00 2024-05-20T16:01:12.265 INFO:teuthology.orchestra.run.smithi160.stdout:(98/156): uuid-1.6.2-55.el9.x86_64.rpm 867 kB/s | 58 kB 00:00 2024-05-20T16:01:12.395 INFO:teuthology.orchestra.run.smithi082.stdout:(77/156): python3-jsonpointer-2.0-4.el9.noarch. 143 kB/s | 19 kB 00:00 2024-05-20T16:01:12.420 INFO:teuthology.orchestra.run.smithi082.stdout:(78/156): python3-jinja2-2.11.3-5.el9.noarch.rp 761 kB/s | 248 kB 00:00 2024-05-20T16:01:12.432 INFO:teuthology.orchestra.run.smithi160.stdout:(99/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 38 MB/s | 16 MB 00:00 2024-05-20T16:01:12.457 INFO:teuthology.orchestra.run.smithi160.stdout:(100/156): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 431 kB/s | 94 kB 00:00 2024-05-20T16:01:12.483 INFO:teuthology.orchestra.run.smithi160.stdout:(101/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 293 kB/s | 64 kB 00:00 2024-05-20T16:01:12.517 INFO:teuthology.orchestra.run.smithi160.stdout:(102/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 535 kB/s | 45 kB 00:00 2024-05-20T16:01:12.545 INFO:teuthology.orchestra.run.smithi082.stdout:(79/156): python3-markupsafe-1.1.1-12.el9.x86_6 278 kB/s | 35 kB 00:00 2024-05-20T16:01:12.640 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction check 2024-05-20T16:01:12.642 INFO:teuthology.orchestra.run.smithi160.stdout:(103/156): lua-devel-5.4.4-4.el9.x86_64.rpm 121 kB/s | 22 kB 00:00 2024-05-20T16:01:12.728 INFO:teuthology.orchestra.run.smithi082.stdout:(80/156): python3-mako-1.1.4-6.el9.noarch.rpm 516 kB/s | 172 kB 00:00 2024-05-20T16:01:12.789 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction check succeeded. 2024-05-20T16:01:12.790 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction test 2024-05-20T16:01:12.901 INFO:teuthology.orchestra.run.smithi160.stdout:(104/156): protobuf-compiler-3.14.0-13.el9.x86_ 2.0 MB/s | 863 kB 00:00 2024-05-20T16:01:13.001 INFO:teuthology.orchestra.run.smithi160.stdout:(105/156): gperftools-libs-2.9.1-2.el9.x86_64.r 861 kB/s | 309 kB 00:00 2024-05-20T16:01:13.051 INFO:teuthology.orchestra.run.smithi160.stdout:(106/156): abseil-cpp-20211102.0-3.el9.x86_64.r 1.0 MB/s | 548 kB 00:00 2024-05-20T16:01:13.078 INFO:teuthology.orchestra.run.smithi160.stdout:(107/156): grpc-data-1.46.7-10.el9.noarch.rpm 111 kB/s | 19 kB 00:00 2024-05-20T16:01:13.127 INFO:teuthology.orchestra.run.smithi160.stdout:(108/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 345 kB/s | 26 kB 00:00 2024-05-20T16:01:13.162 INFO:teuthology.orchestra.run.smithi082.stdout:(81/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 578 kB/s | 250 kB 00:00 2024-05-20T16:01:13.219 INFO:teuthology.orchestra.run.smithi160.stdout:(109/156): libunwind-1.6.2-1.el9.x86_64.rpm 735 kB/s | 67 kB 00:00 2024-05-20T16:01:13.252 INFO:teuthology.orchestra.run.smithi160.stdout:(110/156): liboath-2.6.7-2.el9.x86_64.rpm 277 kB/s | 49 kB 00:00 2024-05-20T16:01:13.353 INFO:teuthology.orchestra.run.smithi160.stdout:(111/156): luarocks-3.9.2-5.el9.noarch.rpm 1.1 MB/s | 151 kB 00:00 2024-05-20T16:01:13.536 INFO:teuthology.orchestra.run.smithi160.stdout:(112/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 2.9 MB/s | 839 kB 00:00 2024-05-20T16:01:13.537 INFO:teuthology.orchestra.run.smithi082.stdout:(82/156): python3-babel-2.9.1-2.el9.noarch.rpm 3.2 MB/s | 6.0 MB 00:01 2024-05-20T16:01:13.571 INFO:teuthology.orchestra.run.smithi082.stdout:(83/156): python3-oauthlib-3.1.1-5.el9.noarch.r 544 kB/s | 222 kB 00:00 2024-05-20T16:01:13.620 INFO:teuthology.orchestra.run.smithi160.stdout:(113/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 523 kB/s | 43 kB 00:00 2024-05-20T16:01:13.661 INFO:teuthology.orchestra.run.smithi160.stdout:(114/156): python3-asyncssh-2.13.2-5.el9.noarch 1.7 MB/s | 548 kB 00:00 2024-05-20T16:01:13.681 INFO:teuthology.orchestra.run.smithi082.stdout:(84/156): python3-packaging-20.9-5.el9.noarch.r 539 kB/s | 77 kB 00:00 2024-05-20T16:01:13.687 INFO:teuthology.orchestra.run.smithi160.stdout:(115/156): python3-cachetools-4.2.4-1.el9.noarc 481 kB/s | 32 kB 00:00 2024-05-20T16:01:13.704 INFO:teuthology.orchestra.run.smithi082.stdout:(85/156): python3-prettytable-0.7.2-27.el9.noar 316 kB/s | 42 kB 00:00 2024-05-20T16:01:13.729 INFO:teuthology.orchestra.run.smithi160.stdout:(116/156): python3-certifi-2021.10.8-2.el9.noar 224 kB/s | 15 kB 00:00 2024-05-20T16:01:13.763 INFO:teuthology.orchestra.run.smithi160.stdout:(117/156): python3-cheroot-8.6.0-4.el9.noarch.r 2.2 MB/s | 172 kB 00:00 2024-05-20T16:01:13.843 INFO:teuthology.orchestra.run.smithi082.stdout:(86/156): python3-protobuf-3.14.0-13.el9.noarch 2.1 MB/s | 269 kB 00:00 2024-05-20T16:01:13.843 INFO:teuthology.orchestra.run.smithi160.stdout:(118/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 4.0 MB/s | 303 kB 00:00 2024-05-20T16:01:13.914 INFO:teuthology.orchestra.run.smithi160.stdout:(119/156): python3-colorama-0.4.4-12.el9.noarch 464 kB/s | 35 kB 00:00 2024-05-20T16:01:13.930 INFO:teuthology.orchestra.run.smithi082.stdout:(87/156): python3-pyasn1-modules-0.4.8-6.el9.no 2.2 MB/s | 279 kB 00:00 2024-05-20T16:01:13.947 INFO:teuthology.orchestra.run.smithi160.stdout:(120/156): python3-cherrypy-18.6.1-2.el9.noarch 1.6 MB/s | 358 kB 00:00 2024-05-20T16:01:13.997 INFO:teuthology.orchestra.run.smithi160.stdout:(121/156): python3-google-auth-2.29.0-1.el9.noa 2.4 MB/s | 206 kB 00:00 2024-05-20T16:01:14.022 INFO:teuthology.orchestra.run.smithi082.stdout:(88/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 500 kB/s | 159 kB 00:00 2024-05-20T16:01:14.056 INFO:teuthology.orchestra.run.smithi082.stdout:(89/156): python3-pytz-2021.1-5.el9.noarch.rpm 407 kB/s | 51 kB 00:00 2024-05-20T16:01:14.072 INFO:teuthology.orchestra.run.smithi160.stdout:(122/156): libarrow-9.0.0-11.el9.x86_64.rpm 4.1 MB/s | 4.4 MB 00:01 2024-05-20T16:01:14.097 INFO:teuthology.orchestra.run.smithi160.stdout:(123/156): python3-grpcio-tools-1.46.7-10.el9.x 1.4 MB/s | 144 kB 00:00 2024-05-20T16:01:14.156 INFO:teuthology.orchestra.run.smithi160.stdout:(124/156): python3-jaraco-8.2.1-3.el9.noarch.rp 128 kB/s | 11 kB 00:00 2024-05-20T16:01:14.157 INFO:teuthology.orchestra.run.smithi082.stdout:(90/156): python3-requests-oauthlib-1.3.0-12.el 401 kB/s | 54 kB 00:00 2024-05-20T16:01:14.198 INFO:teuthology.orchestra.run.smithi160.stdout:(125/156): python3-jaraco-classes-3.2.1-5.el9.n 176 kB/s | 18 kB 00:00 2024-05-20T16:01:14.223 INFO:teuthology.orchestra.run.smithi160.stdout:(126/156): python3-jaraco-collections-3.0.0-8.e 346 kB/s | 23 kB 00:00 2024-05-20T16:01:14.265 INFO:teuthology.orchestra.run.smithi160.stdout:(127/156): python3-jaraco-functools-3.5.0-2.el9 290 kB/s | 19 kB 00:00 2024-05-20T16:01:14.273 INFO:teuthology.orchestra.run.smithi082.stdout:(91/156): python3-toml-0.10.2-6.el9.noarch.rpm 357 kB/s | 42 kB 00:00 2024-05-20T16:01:14.291 INFO:teuthology.orchestra.run.smithi160.stdout:(128/156): python3-jaraco-text-3.2.0-6.el9.noar 293 kB/s | 20 kB 00:00 2024-05-20T16:01:14.340 INFO:teuthology.orchestra.run.smithi082.stdout:(92/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 2.8 MB/s | 5.1 MB 00:01 2024-05-20T16:01:14.358 INFO:teuthology.orchestra.run.smithi160.stdout:(129/156): python3-logutils-0.3.5-21.el9.noarch 693 kB/s | 46 kB 00:00 2024-05-20T16:01:14.399 INFO:teuthology.orchestra.run.smithi082.stdout:(93/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 352 kB/s | 44 kB 00:00 2024-05-20T16:01:14.441 INFO:teuthology.orchestra.run.smithi160.stdout:(130/156): python3-more-itertools-8.12.0-2.el9. 946 kB/s | 79 kB 00:00 2024-05-20T16:01:14.484 INFO:teuthology.orchestra.run.smithi160.stdout:(131/156): python3-kubernetes-26.1.0-2.el9.noar 4.7 MB/s | 1.0 MB 00:00 2024-05-20T16:01:14.524 INFO:teuthology.orchestra.run.smithi082.stdout:(94/156): qatlib-service-24.02.0-1.el9.x86_64.r 287 kB/s | 36 kB 00:00 2024-05-20T16:01:14.525 INFO:teuthology.orchestra.run.smithi160.stdout:(132/156): python3-natsort-7.1.1-5.el9.noarch.r 688 kB/s | 58 kB 00:00 2024-05-20T16:01:14.567 INFO:teuthology.orchestra.run.smithi160.stdout:(133/156): python3-pecan-1.4.2-3.el9.noarch.rpm 3.2 MB/s | 272 kB 00:00 2024-05-20T16:01:14.573 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction test succeeded. 2024-05-20T16:01:14.574 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction 2024-05-20T16:01:14.601 INFO:teuthology.orchestra.run.smithi160.stdout:(134/156): python3-portend-3.1.0-2.el9.noarch.r 218 kB/s | 16 kB 00:00 2024-05-20T16:01:14.641 INFO:teuthology.orchestra.run.smithi082.stdout:(95/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 407 kB/s | 47 kB 00:00 2024-05-20T16:01:14.643 INFO:teuthology.orchestra.run.smithi160.stdout:(135/156): python3-grpcio-1.46.7-10.el9.x86_64. 2.9 MB/s | 2.0 MB 00:00 2024-05-20T16:01:14.669 INFO:teuthology.orchestra.run.smithi160.stdout:(136/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 895 kB/s | 90 kB 00:00 2024-05-20T16:01:14.694 INFO:teuthology.orchestra.run.smithi160.stdout:(137/156): python3-pyperclip-1.8.2-2.el9.noarch 306 kB/s | 28 kB 00:00 2024-05-20T16:01:14.722 INFO:teuthology.orchestra.run.smithi160.stdout:(138/156): python3-repoze-lru-0.7-16.el9.noarch 407 kB/s | 31 kB 00:00 2024-05-20T16:01:14.766 INFO:teuthology.orchestra.run.smithi082.stdout:(96/156): qatlib-24.02.0-1.el9.x86_64.rpm 518 kB/s | 221 kB 00:00 2024-05-20T16:01:14.807 INFO:teuthology.orchestra.run.smithi160.stdout:(139/156): python3-routes-2.5.1-5.el9.noarch.rp 1.3 MB/s | 188 kB 00:00 2024-05-20T16:01:14.883 INFO:teuthology.orchestra.run.smithi082.stdout:(97/156): uuid-1.6.2-55.el9.x86_64.rpm 496 kB/s | 58 kB 00:00 2024-05-20T16:01:15.017 INFO:teuthology.orchestra.run.smithi160.stdout:(140/156): python3-tempora-5.0.0-2.el9.noarch.r 120 kB/s | 36 kB 00:00 2024-05-20T16:01:15.083 INFO:teuthology.orchestra.run.smithi082.stdout:(98/156): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 468 kB/s | 94 kB 00:00 2024-05-20T16:01:15.103 INFO:teuthology.orchestra.run.smithi160.stdout:(141/156): python3-rsa-4.9-2.el9.noarch.rpm 144 kB/s | 59 kB 00:00 2024-05-20T16:01:15.129 INFO:teuthology.orchestra.run.smithi160.stdout:(142/156): python3-typing-extensions-4.4.0-2.el 158 kB/s | 51 kB 00:00 2024-05-20T16:01:15.133 INFO:teuthology.orchestra.run.smithi082.stdout:(99/156): socat-1.7.4.1-5.el9.x86_64.rpm 619 kB/s | 305 kB 00:00 2024-05-20T16:01:15.170 INFO:teuthology.orchestra.run.smithi160.stdout:(143/156): python3-webob-1.8.7-6.el9.noarch.rpm 1.5 MB/s | 230 kB 00:00 2024-05-20T16:01:15.196 INFO:teuthology.orchestra.run.smithi160.stdout:(144/156): python3-websocket-client-1.2.3-2.el9 973 kB/s | 90 kB 00:00 2024-05-20T16:01:15.237 INFO:teuthology.orchestra.run.smithi160.stdout:(145/156): python3-zc-lockfile-2.0-10.el9.noarc 298 kB/s | 20 kB 00:00 2024-05-20T16:01:15.250 INFO:teuthology.orchestra.run.smithi082.stdout:(100/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 384 kB/s | 45 kB 00:00 2024-05-20T16:01:15.271 INFO:teuthology.orchestra.run.smithi160.stdout:(146/156): python3-werkzeug-2.0.3-3.el9.1.noarc 2.9 MB/s | 427 kB 00:00 2024-05-20T16:01:15.293 INFO:teuthology.orchestra.run.smithi082.stdout:(101/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 305 kB/s | 64 kB 00:00 2024-05-20T16:01:15.313 INFO:teuthology.orchestra.run.smithi160.stdout:(147/156): re2-20211101-3.el9.x86_64.rpm 1.6 MB/s | 192 kB 00:00 2024-05-20T16:01:15.568 INFO:teuthology.orchestra.run.smithi082.stdout:(102/156): lua-devel-5.4.4-4.el9.x86_64.rpm 70 kB/s | 22 kB 00:00 2024-05-20T16:01:15.613 INFO:teuthology.orchestra.run.smithi160.stdout:(148/156): thrift-0.15.0-2.el9.x86_64.rpm 4.2 MB/s | 1.6 MB 00:00 2024-05-20T16:01:16.022 INFO:teuthology.orchestra.run.smithi160.stdout:(149/156): NetworkManager-1.47.90-1.el9.x86_64. 5.7 MB/s | 2.3 MB 00:00 2024-05-20T16:01:16.118 INFO:teuthology.orchestra.run.smithi082.stdout:(103/156): abseil-cpp-20211102.0-3.el9.x86_64.r 997 kB/s | 548 kB 00:00 2024-05-20T16:01:16.289 INFO:teuthology.orchestra.run.smithi160.stdout:(150/156): NetworkManager-libnm-1.47.90-1.el9.x 6.8 MB/s | 1.8 MB 00:00 2024-05-20T16:01:16.364 INFO:teuthology.orchestra.run.smithi160.stdout:(151/156): NetworkManager-team-1.47.90-1.el9.x8 554 kB/s | 41 kB 00:00 2024-05-20T16:01:16.423 INFO:teuthology.orchestra.run.smithi160.stdout:(152/156): librados2-19.0.0-2744.gd2b507c7.el9. 2.9 MB/s | 3.3 MB 00:01 2024-05-20T16:01:16.452 INFO:teuthology.orchestra.run.smithi082.stdout:(104/156): gperftools-libs-2.9.1-2.el9.x86_64.r 927 kB/s | 309 kB 00:00 2024-05-20T16:01:16.456 INFO:teuthology.orchestra.run.smithi160.stdout:(153/156): NetworkManager-tui-1.47.90-1.el9.x86 2.6 MB/s | 248 kB 00:00 2024-05-20T16:01:16.535 INFO:teuthology.orchestra.run.smithi082.stdout:(105/156): grpc-data-1.46.7-10.el9.noarch.rpm 234 kB/s | 19 kB 00:00 2024-05-20T16:01:16.623 INFO:teuthology.orchestra.run.smithi160.stdout:(154/156): librbd1-19.0.0-2744.gd2b507c7.el9.x8 2.4 MB/s | 3.1 MB 00:01 2024-05-20T16:01:16.782 INFO:teuthology.orchestra.run.smithi160.stdout:(155/156): openssl-libs-3.2.1-1.el9.x86_64.rpm 7.4 MB/s | 2.4 MB 00:00 2024-05-20T16:01:16.924 INFO:teuthology.orchestra.run.smithi160.stdout:(156/156): openssl-3.2.1-1.el9.x86_64.rpm 2.7 MB/s | 1.3 MB 00:00 2024-05-20T16:01:16.926 INFO:teuthology.orchestra.run.smithi160.stdout:-------------------------------------------------------------------------------- 2024-05-20T16:01:16.927 INFO:teuthology.orchestra.run.smithi160.stdout:Total 7.0 MB/s | 221 MB 00:31 2024-05-20T16:01:17.161 INFO:teuthology.orchestra.run.smithi082.stdout:(106/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 5.2 MB/s | 16 MB 00:03 2024-05-20T16:01:17.194 INFO:teuthology.orchestra.run.smithi082.stdout:(107/156): protobuf-compiler-3.14.0-13.el9.x86_ 454 kB/s | 863 kB 00:01 2024-05-20T16:01:17.261 INFO:teuthology.orchestra.run.smithi082.stdout:(108/156): libarrow-9.0.0-11.el9.x86_64.rpm 6.1 MB/s | 4.4 MB 00:00 2024-05-20T16:01:17.428 INFO:teuthology.orchestra.run.smithi082.stdout:(109/156): libunwind-1.6.2-1.el9.x86_64.rpm 591 kB/s | 67 kB 00:00 2024-05-20T16:01:17.761 INFO:teuthology.orchestra.run.smithi082.stdout:(110/156): luarocks-3.9.2-5.el9.noarch.rpm 454 kB/s | 151 kB 00:00 2024-05-20T16:01:17.787 INFO:teuthology.orchestra.run.smithi082.stdout:(111/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 41 kB/s | 26 kB 00:00 2024-05-20T16:01:17.812 INFO:teuthology.orchestra.run.smithi082.stdout:(112/156): liboath-2.6.7-2.el9.x86_64.rpm 79 kB/s | 49 kB 00:00 2024-05-20T16:01:18.062 INFO:teuthology.orchestra.run.smithi160.stdout:Running transaction check 2024-05-20T16:01:18.213 INFO:teuthology.orchestra.run.smithi160.stdout:Transaction check succeeded. 2024-05-20T16:01:18.213 INFO:teuthology.orchestra.run.smithi160.stdout:Running transaction test 2024-05-20T16:01:18.487 INFO:teuthology.orchestra.run.smithi082.stdout:(113/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 1.1 MB/s | 839 kB 00:00 2024-05-20T16:01:18.846 INFO:teuthology.orchestra.run.smithi082.stdout:(114/156): python3-cachetools-4.2.4-1.el9.noarc 90 kB/s | 32 kB 00:00 2024-05-20T16:01:18.888 INFO:teuthology.orchestra.run.smithi082.stdout:(115/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 40 kB/s | 43 kB 00:01 2024-05-20T16:01:18.971 INFO:teuthology.orchestra.run.smithi082.stdout:(116/156): python3-certifi-2021.10.8-2.el9.noar 120 kB/s | 15 kB 00:00 2024-05-20T16:01:19.113 INFO:teuthology.orchestra.run.smithi082.stdout:(117/156): python3-asyncssh-2.13.2-5.el9.noarch 413 kB/s | 548 kB 00:01 2024-05-20T16:01:19.172 INFO:teuthology.orchestra.run.smithi082.stdout:(118/156): python3-cheroot-8.6.0-4.el9.noarch.r 604 kB/s | 172 kB 00:00 2024-05-20T16:01:19.347 INFO:teuthology.orchestra.run.smithi082.stdout:(119/156): python3-cherrypy-18.6.1-2.el9.noarch 954 kB/s | 358 kB 00:00 2024-05-20T16:01:19.523 INFO:teuthology.orchestra.run.smithi082.stdout:(120/156): python3-colorama-0.4.4-12.el9.noarch 101 kB/s | 35 kB 00:00 2024-05-20T16:01:19.570 INFO:teuthology.orchestra.run.smithi082.stdout:(121/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 664 kB/s | 303 kB 00:00 2024-05-20T16:01:19.598 INFO:teuthology.orchestra.run.smithi082.stdout:(122/156): python3-google-auth-2.29.0-1.el9.noa 820 kB/s | 206 kB 00:00 2024-05-20T16:01:19.682 INFO:teuthology.orchestra.run.smithi082.stdout:(123/156): python3-grpcio-tools-1.46.7-10.el9.x 1.3 MB/s | 144 kB 00:00 2024-05-20T16:01:19.815 INFO:teuthology.orchestra.run.smithi082.stdout:(124/156): python3-jaraco-8.2.1-3.el9.noarch.rp 49 kB/s | 11 kB 00:00 2024-05-20T16:01:19.840 INFO:teuthology.orchestra.run.smithi082.stdout:(125/156): python3-jaraco-classes-3.2.1-5.el9.n 111 kB/s | 18 kB 00:00 2024-05-20T16:01:19.932 INFO:teuthology.orchestra.run.smithi082.stdout:(126/156): python3-jaraco-collections-3.0.0-8.e 198 kB/s | 23 kB 00:00 2024-05-20T16:01:19.958 INFO:teuthology.orchestra.run.smithi082.stdout:(127/156): python3-jaraco-functools-3.5.0-2.el9 166 kB/s | 19 kB 00:00 2024-05-20T16:01:20.033 INFO:teuthology.orchestra.run.smithi082.stdout:(128/156): python3-grpcio-1.46.7-10.el9.x86_64. 4.0 MB/s | 2.0 MB 00:00 2024-05-20T16:01:20.058 INFO:teuthology.orchestra.run.smithi082.stdout:(129/156): python3-jaraco-text-3.2.0-6.el9.noar 156 kB/s | 20 kB 00:00 2024-05-20T16:01:20.275 INFO:teuthology.orchestra.run.smithi160.stdout:Transaction test succeeded. 2024-05-20T16:01:20.275 INFO:teuthology.orchestra.run.smithi160.stdout:Running transaction 2024-05-20T16:01:20.292 INFO:teuthology.orchestra.run.smithi082.stdout:(130/156): python3-logutils-0.3.5-21.el9.noarch 179 kB/s | 46 kB 00:00 2024-05-20T16:01:20.317 INFO:teuthology.orchestra.run.smithi082.stdout:(131/156): python3-more-itertools-8.12.0-2.el9. 305 kB/s | 79 kB 00:00 2024-05-20T16:01:20.543 INFO:teuthology.orchestra.run.smithi082.stdout:(132/156): python3-natsort-7.1.1-5.el9.noarch.r 231 kB/s | 58 kB 00:00 2024-05-20T16:01:20.576 INFO:teuthology.orchestra.run.smithi082.stdout:(133/156): python3-kubernetes-26.1.0-2.el9.noar 1.7 MB/s | 1.0 MB 00:00 2024-05-20T16:01:20.601 INFO:teuthology.orchestra.run.smithi082.stdout:(134/156): python3-pecan-1.4.2-3.el9.noarch.rpm 959 kB/s | 272 kB 00:00 2024-05-20T16:01:20.650 INFO:teuthology.orchestra.run.smithi184.stdout: Preparing : 1/1 2024-05-20T16:01:20.676 INFO:teuthology.orchestra.run.smithi082.stdout:(135/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 898 kB/s | 90 kB 00:00 2024-05-20T16:01:20.718 INFO:teuthology.orchestra.run.smithi184.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/164 2024-05-20T16:01:20.756 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-05-20T16:01:20.777 INFO:teuthology.orchestra.run.smithi082.stdout:(136/156): python3-portend-3.1.0-2.el9.noarch.r 70 kB/s | 16 kB 00:00 2024-05-20T16:01:20.802 INFO:teuthology.orchestra.run.smithi082.stdout:(137/156): python3-pyperclip-1.8.2-2.el9.noarch 141 kB/s | 28 kB 00:00 2024-05-20T16:01:20.810 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-05-20T16:01:20.827 INFO:teuthology.orchestra.run.smithi082.stdout:(138/156): python3-repoze-lru-0.7-16.el9.noarch 204 kB/s | 31 kB 00:00 2024-05-20T16:01:20.843 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-05-20T16:01:20.869 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-05-20T16:01:20.912 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-05-20T16:01:21.019 INFO:teuthology.orchestra.run.smithi082.stdout:(139/156): python3-routes-2.5.1-5.el9.noarch.rp 779 kB/s | 188 kB 00:00 2024-05-20T16:01:21.044 INFO:teuthology.orchestra.run.smithi082.stdout:(140/156): python3-tempora-5.0.0-2.el9.noarch.r 165 kB/s | 36 kB 00:00 2024-05-20T16:01:21.070 INFO:teuthology.orchestra.run.smithi082.stdout:(141/156): python3-rsa-4.9-2.el9.noarch.rpm 244 kB/s | 59 kB 00:00 2024-05-20T16:01:21.228 INFO:teuthology.orchestra.run.smithi082.stdout:(142/156): python3-typing-extensions-4.4.0-2.el 244 kB/s | 51 kB 00:00 2024-05-20T16:01:21.239 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-05-20T16:01:21.253 INFO:teuthology.orchestra.run.smithi082.stdout:(143/156): python3-webob-1.8.7-6.el9.noarch.rpm 1.1 MB/s | 230 kB 00:00 2024-05-20T16:01:21.362 INFO:teuthology.orchestra.run.smithi184.stdout: Upgrading : librados2-2:19.0.0-2744.gd2b507c7.el9.x86_64 8/164 2024-05-20T16:01:21.363 INFO:teuthology.orchestra.run.smithi082.stdout:(144/156): python3-websocket-client-1.2.3-2.el9 307 kB/s | 90 kB 00:00 2024-05-20T16:01:21.387 INFO:teuthology.orchestra.run.smithi082.stdout:(145/156): python3-zc-lockfile-2.0-10.el9.noarc 149 kB/s | 20 kB 00:00 2024-05-20T16:01:21.413 INFO:teuthology.orchestra.run.smithi082.stdout:(146/156): python3-werkzeug-2.0.3-3.el9.1.noarc 2.3 MB/s | 427 kB 00:00 2024-05-20T16:01:21.474 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: librados2-2:19.0.0-2744.gd2b507c7.el9.x86_64 8/164 2024-05-20T16:01:21.497 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : libcephfs2-2:19.0.0-2744.gd2b507c7.el9.x86_64 9/164 2024-05-20T16:01:21.530 INFO:teuthology.orchestra.run.smithi082.stdout:(147/156): re2-20211101-3.el9.x86_64.rpm 1.1 MB/s | 192 kB 00:00 2024-05-20T16:01:21.571 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: libcephfs2-2:19.0.0-2744.gd2b507c7.el9.x86_64 9/164 2024-05-20T16:01:21.601 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-rados-2:19.0.0-2744.gd2b507c7.el9.x86_64 10/164 2024-05-20T16:01:21.714 INFO:teuthology.orchestra.run.smithi082.stdout:(148/156): thrift-0.15.0-2.el9.x86_64.rpm 4.9 MB/s | 1.6 MB 00:00 2024-05-20T16:01:21.749 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/164 2024-05-20T16:01:21.799 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/164 2024-05-20T16:01:21.826 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : libcephsqlite-2:19.0.0-2744.gd2b507c7.el9.x86_64 13/164 2024-05-20T16:01:21.933 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2744.gd2b507c7.el9.x86_64 13/164 2024-05-20T16:01:21.971 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : libradosstriper1-2:19.0.0-2744.gd2b507c7.el9.x86 14/164 2024-05-20T16:01:22.025 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2744.gd2b507c7.el9.x86 14/164 2024-05-20T16:01:22.047 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-20T16:01:22.068 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-20T16:01:22.271 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-20T16:01:22.397 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-20T16:01:22.413 INFO:teuthology.orchestra.run.smithi082.stdout:(149/156): librados2-19.0.0-2744.gd2b507c7.el9. 3.3 MB/s | 3.3 MB 00:01 2024-05-20T16:01:22.558 INFO:teuthology.orchestra.run.smithi184.stdout: Upgrading : NetworkManager-libnm-1:1.47.90-1.el9.x86_64 17/164 2024-05-20T16:01:22.621 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.90-1.el9.x86_64 17/164 2024-05-20T16:01:22.721 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: NetworkManager-1:1.47.90-1.el9.x86_64 18/164 2024-05-20T16:01:22.722 INFO:teuthology.orchestra.run.smithi082.stdout:(150/156): NetworkManager-1.47.90-1.el9.x86_64. 2.3 MB/s | 2.3 MB 00:01 2024-05-20T16:01:22.797 INFO:teuthology.orchestra.run.smithi082.stdout:(151/156): NetworkManager-team-1.47.90-1.el9.x8 555 kB/s | 41 kB 00:00 2024-05-20T16:01:22.814 INFO:teuthology.orchestra.run.smithi184.stdout: Upgrading : NetworkManager-1:1.47.90-1.el9.x86_64 18/164 2024-05-20T16:01:22.906 INFO:teuthology.orchestra.run.smithi082.stdout:(152/156): NetworkManager-libnm-1.47.90-1.el9.x 3.7 MB/s | 1.8 MB 00:00 2024-05-20T16:01:22.998 INFO:teuthology.orchestra.run.smithi082.stdout:(153/156): openssl-3.2.1-1.el9.x86_64.rpm 15 MB/s | 1.3 MB 00:00 2024-05-20T16:01:23.004 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: NetworkManager-1:1.47.90-1.el9.x86_64 18/164 2024-05-20T16:01:23.111 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : re2-1:20211101-3.el9.x86_64 19/164 2024-05-20T16:01:23.123 INFO:teuthology.orchestra.run.smithi082.stdout:(154/156): openssl-libs-3.2.1-1.el9.x86_64.rpm 19 MB/s | 2.4 MB 00:00 2024-05-20T16:01:23.198 INFO:teuthology.orchestra.run.smithi082.stdout:(155/156): NetworkManager-tui-1.47.90-1.el9.x86 618 kB/s | 248 kB 00:00 2024-05-20T16:01:23.205 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-05-20T16:01:23.245 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-05-20T16:01:23.279 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-05-20T16:01:23.361 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-05-20T16:01:23.395 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-05-20T16:01:23.439 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 25/164 2024-05-20T16:01:23.471 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 26/164 2024-05-20T16:01:23.500 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : lua-5.4.4-4.el9.x86_64 27/164 2024-05-20T16:01:23.568 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 28/164 2024-05-20T16:01:23.621 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 29/164 2024-05-20T16:01:23.652 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 30/164 2024-05-20T16:01:23.688 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 31/164 2024-05-20T16:01:23.764 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 32/164 2024-05-20T16:01:23.798 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-ceph-common-2:19.0.0-2744.gd2b507c7.el9. 33/164 2024-05-20T16:01:23.842 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-ceph-argparse-2:19.0.0-2744.gd2b507c7.el 34/164 2024-05-20T16:01:23.870 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-cephfs-2:19.0.0-2744.gd2b507c7.el9.x86_6 35/164 2024-05-20T16:01:23.939 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 36/164 2024-05-20T16:01:23.985 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : luarocks-3.9.2-5.el9.noarch 37/164 2024-05-20T16:01:24.059 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 38/164 2024-05-20T16:01:24.115 INFO:teuthology.orchestra.run.smithi082.stdout:(156/156): librbd1-19.0.0-2744.gd2b507c7.el9.x8 1.2 MB/s | 3.1 MB 00:02 2024-05-20T16:01:24.121 INFO:teuthology.orchestra.run.smithi082.stdout:-------------------------------------------------------------------------------- 2024-05-20T16:01:24.122 INFO:teuthology.orchestra.run.smithi082.stdout:Total 4.3 MB/s | 221 MB 00:51 2024-05-20T16:01:24.201 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 39/164 2024-05-20T16:01:24.255 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 40/164 2024-05-20T16:01:24.783 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-rsa-4.9-2.el9.noarch 41/164 2024-05-20T16:01:24.866 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 42/164 2024-05-20T16:01:24.904 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 43/164 2024-05-20T16:01:24.947 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 44/164 2024-05-20T16:01:24.985 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : librados-devel-2:19.0.0-2744.gd2b507c7.el9.x86_6 45/164 2024-05-20T16:01:25.034 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 46/164 2024-05-20T16:01:25.188 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 47/164 2024-05-20T16:01:25.246 INFO:teuthology.orchestra.run.smithi184.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 48/164 2024-05-20T16:01:25.271 INFO:teuthology.orchestra.run.smithi082.stdout:Running transaction check 2024-05-20T16:01:25.300 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 49/164 2024-05-20T16:01:25.365 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 50/164 2024-05-20T16:01:25.405 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 51/164 2024-05-20T16:01:25.420 INFO:teuthology.orchestra.run.smithi082.stdout:Transaction check succeeded. 2024-05-20T16:01:25.420 INFO:teuthology.orchestra.run.smithi082.stdout:Running transaction test 2024-05-20T16:01:25.442 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 52/164 2024-05-20T16:01:25.489 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 53/164 2024-05-20T16:01:25.524 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 54/164 2024-05-20T16:01:25.567 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 55/164 2024-05-20T16:01:25.609 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 56/164 2024-05-20T16:01:25.747 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 57/164 2024-05-20T16:01:25.791 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 58/164 2024-05-20T16:01:25.821 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 59/164 2024-05-20T16:01:25.862 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 60/164 2024-05-20T16:01:25.957 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-05-20T16:01:26.004 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 62/164 2024-05-20T16:01:26.044 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-05-20T16:01:26.070 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 64/164 2024-05-20T16:01:26.095 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-05-20T16:01:26.168 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-05-20T16:01:26.322 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-05-20T16:01:26.352 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-05-20T16:01:26.353 INFO:teuthology.orchestra.run.smithi160.stdout: Preparing : 1/1 2024-05-20T16:01:26.380 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-05-20T16:01:26.423 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-05-20T16:01:26.427 INFO:teuthology.orchestra.run.smithi160.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/164 2024-05-20T16:01:26.462 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-05-20T16:01:26.466 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-05-20T16:01:26.493 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-05-20T16:01:26.520 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-05-20T16:01:26.523 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-05-20T16:01:26.554 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-05-20T16:01:26.558 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-05-20T16:01:26.579 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-05-20T16:01:26.591 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-05-20T16:01:26.622 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-05-20T16:01:26.633 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-05-20T16:01:26.668 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-05-20T16:01:26.746 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-05-20T16:01:26.798 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-05-20T16:01:26.829 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-05-20T16:01:26.876 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-05-20T16:01:26.956 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-05-20T16:01:26.974 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-packaging-20.9-5.el9.noarch 82/164 2024-05-20T16:01:27.022 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/164 2024-05-20T16:01:27.072 INFO:teuthology.orchestra.run.smithi160.stdout: Upgrading : librados2-2:19.0.0-2744.gd2b507c7.el9.x86_64 8/164 2024-05-20T16:01:27.179 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: librados2-2:19.0.0-2744.gd2b507c7.el9.x86_64 8/164 2024-05-20T16:01:27.206 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : libcephfs2-2:19.0.0-2744.gd2b507c7.el9.x86_64 9/164 2024-05-20T16:01:27.288 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: libcephfs2-2:19.0.0-2744.gd2b507c7.el9.x86_64 9/164 2024-05-20T16:01:27.319 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-rados-2:19.0.0-2744.gd2b507c7.el9.x86_64 10/164 2024-05-20T16:01:27.467 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/164 2024-05-20T16:01:27.524 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/164 2024-05-20T16:01:27.545 INFO:teuthology.orchestra.run.smithi082.stdout:Transaction test succeeded. 2024-05-20T16:01:27.545 INFO:teuthology.orchestra.run.smithi082.stdout:Running transaction 2024-05-20T16:01:27.552 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : libcephsqlite-2:19.0.0-2744.gd2b507c7.el9.x86_64 13/164 2024-05-20T16:01:27.656 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2744.gd2b507c7.el9.x86_64 13/164 2024-05-20T16:01:27.678 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : libradosstriper1-2:19.0.0-2744.gd2b507c7.el9.x86 14/164 2024-05-20T16:01:27.732 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2744.gd2b507c7.el9.x86 14/164 2024-05-20T16:01:27.739 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/164 2024-05-20T16:01:27.754 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-20T16:01:27.775 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-20T16:01:27.787 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/164 2024-05-20T16:01:27.835 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/164 2024-05-20T16:01:27.935 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/164 2024-05-20T16:01:27.962 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-20T16:01:27.986 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 88/164 2024-05-20T16:01:28.075 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-05-20T16:01:28.091 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-20T16:01:28.114 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-05-20T16:01:28.260 INFO:teuthology.orchestra.run.smithi160.stdout: Upgrading : NetworkManager-libnm-1:1.47.90-1.el9.x86_64 17/164 2024-05-20T16:01:28.292 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : openblas-0.3.21-2.el9.x86_64 91/164 2024-05-20T16:01:28.320 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.90-1.el9.x86_64 17/164 2024-05-20T16:01:28.320 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 92/164 2024-05-20T16:01:28.397 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-05-20T16:01:28.412 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: NetworkManager-1:1.47.90-1.el9.x86_64 18/164 2024-05-20T16:01:28.505 INFO:teuthology.orchestra.run.smithi160.stdout: Upgrading : NetworkManager-1:1.47.90-1.el9.x86_64 18/164 2024-05-20T16:01:28.697 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: NetworkManager-1:1.47.90-1.el9.x86_64 18/164 2024-05-20T16:01:28.801 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : re2-1:20211101-3.el9.x86_64 19/164 2024-05-20T16:01:28.889 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-05-20T16:01:28.927 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-05-20T16:01:28.961 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-05-20T16:01:29.026 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-05-20T16:01:29.037 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-05-20T16:01:29.069 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-05-20T16:01:29.112 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 25/164 2024-05-20T16:01:29.145 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 26/164 2024-05-20T16:01:29.166 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-05-20T16:01:29.173 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : lua-5.4.4-4.el9.x86_64 27/164 2024-05-20T16:01:29.241 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 28/164 2024-05-20T16:01:29.295 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 29/164 2024-05-20T16:01:29.325 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 30/164 2024-05-20T16:01:29.361 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 31/164 2024-05-20T16:01:29.437 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 32/164 2024-05-20T16:01:29.473 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-ceph-common-2:19.0.0-2744.gd2b507c7.el9. 33/164 2024-05-20T16:01:29.506 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-ceph-argparse-2:19.0.0-2744.gd2b507c7.el 34/164 2024-05-20T16:01:29.544 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-cephfs-2:19.0.0-2744.gd2b507c7.el9.x86_6 35/164 2024-05-20T16:01:29.613 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 36/164 2024-05-20T16:01:29.659 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : luarocks-3.9.2-5.el9.noarch 37/164 2024-05-20T16:01:29.734 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 38/164 2024-05-20T16:01:29.865 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 39/164 2024-05-20T16:01:29.919 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 40/164 2024-05-20T16:01:30.357 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-05-20T16:01:30.441 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-05-20T16:01:30.444 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-rsa-4.9-2.el9.noarch 41/164 2024-05-20T16:01:30.478 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-05-20T16:01:30.511 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 99/164 2024-05-20T16:01:30.523 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 42/164 2024-05-20T16:01:30.552 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 43/164 2024-05-20T16:01:30.595 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 44/164 2024-05-20T16:01:30.635 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : librados-devel-2:19.0.0-2744.gd2b507c7.el9.x86_6 45/164 2024-05-20T16:01:30.676 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 46/164 2024-05-20T16:01:30.824 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 47/164 2024-05-20T16:01:30.831 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-05-20T16:01:30.879 INFO:teuthology.orchestra.run.smithi160.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 48/164 2024-05-20T16:01:30.886 INFO:teuthology.orchestra.run.smithi184.stdout: Upgrading : librbd1-2:19.0.0-2744.gd2b507c7.el9.x86_64 101/164 2024-05-20T16:01:30.934 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 49/164 2024-05-20T16:01:31.007 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 50/164 2024-05-20T16:01:31.038 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 51/164 2024-05-20T16:01:31.074 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 52/164 2024-05-20T16:01:31.121 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 53/164 2024-05-20T16:01:31.157 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 54/164 2024-05-20T16:01:31.173 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: librbd1-2:19.0.0-2744.gd2b507c7.el9.x86_64 101/164 2024-05-20T16:01:31.200 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 55/164 2024-05-20T16:01:31.242 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 56/164 2024-05-20T16:01:31.354 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-rbd-2:19.0.0-2744.gd2b507c7.el9.x86_64 102/164 2024-05-20T16:01:31.378 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 57/164 2024-05-20T16:01:31.390 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-05-20T16:01:31.422 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 58/164 2024-05-20T16:01:31.453 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 59/164 2024-05-20T16:01:31.503 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 60/164 2024-05-20T16:01:31.598 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-05-20T16:01:31.644 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 62/164 2024-05-20T16:01:31.684 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-05-20T16:01:31.710 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 64/164 2024-05-20T16:01:31.735 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-05-20T16:01:31.809 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-05-20T16:01:31.869 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-05-20T16:01:31.894 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : librgw2-2:19.0.0-2744.gd2b507c7.el9.x86_64 105/164 2024-05-20T16:01:31.936 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-05-20T16:01:31.956 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: librgw2-2:19.0.0-2744.gd2b507c7.el9.x86_64 105/164 2024-05-20T16:01:31.966 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-05-20T16:01:31.987 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-rgw-2:19.0.0-2744.gd2b507c7.el9.x86_64 106/164 2024-05-20T16:01:31.994 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-05-20T16:01:32.029 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-05-20T16:01:32.068 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-05-20T16:01:32.099 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-05-20T16:01:32.129 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-05-20T16:01:32.164 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-05-20T16:01:32.197 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-05-20T16:01:32.238 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-05-20T16:01:32.273 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-05-20T16:01:32.360 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-05-20T16:01:32.411 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-05-20T16:01:32.443 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-05-20T16:01:32.489 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-05-20T16:01:32.587 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-packaging-20.9-5.el9.noarch 82/164 2024-05-20T16:01:32.636 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/164 2024-05-20T16:01:33.359 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/164 2024-05-20T16:01:33.407 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/164 2024-05-20T16:01:33.441 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/164 2024-05-20T16:01:33.539 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/164 2024-05-20T16:01:33.590 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 88/164 2024-05-20T16:01:33.601 INFO:teuthology.orchestra.run.smithi082.stdout: Preparing : 1/1 2024-05-20T16:01:33.678 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-05-20T16:01:33.685 INFO:teuthology.orchestra.run.smithi082.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/164 2024-05-20T16:01:33.725 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-05-20T16:01:33.727 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-05-20T16:01:33.792 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-05-20T16:01:33.827 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-05-20T16:01:33.853 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-05-20T16:01:33.896 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : openblas-0.3.21-2.el9.x86_64 91/164 2024-05-20T16:01:33.901 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-05-20T16:01:33.916 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 92/164 2024-05-20T16:01:33.994 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-05-20T16:01:34.227 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-common-2:19.0.0-2744.gd2b507c7.el9.x86_64 107/164 2024-05-20T16:01:34.239 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-05-20T16:01:34.277 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-common-2:19.0.0-2744.gd2b507c7.el9.x86_64 107/164 2024-05-20T16:01:34.345 INFO:teuthology.orchestra.run.smithi082.stdout: Upgrading : librados2-2:19.0.0-2744.gd2b507c7.el9.x86_64 8/164 2024-05-20T16:01:34.463 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: librados2-2:19.0.0-2744.gd2b507c7.el9.x86_64 8/164 2024-05-20T16:01:34.496 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : libcephfs2-2:19.0.0-2744.gd2b507c7.el9.x86_64 9/164 2024-05-20T16:01:34.583 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: libcephfs2-2:19.0.0-2744.gd2b507c7.el9.x86_64 9/164 2024-05-20T16:01:34.625 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-rados-2:19.0.0-2744.gd2b507c7.el9.x86_64 10/164 2024-05-20T16:01:34.639 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-05-20T16:01:34.756 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-05-20T16:01:34.795 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/164 2024-05-20T16:01:34.831 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-common-2:19.0.0-2744.gd2b507c7.el9.x86_64 107/164 2024-05-20T16:01:34.868 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/164 2024-05-20T16:01:34.872 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-base-2:19.0.0-2744.gd2b507c7.el9.x86_64 108/164 2024-05-20T16:01:34.893 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : libcephsqlite-2:19.0.0-2744.gd2b507c7.el9.x86_64 13/164 2024-05-20T16:01:34.962 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-base-2:19.0.0-2744.gd2b507c7.el9.x86_64 108/164 2024-05-20T16:01:34.962 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-05-20T16:01:34.962 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-05-20T16:01:34.962 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-05-20T16:01:34.982 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-selinux-2:19.0.0-2744.gd2b507c7.el9.x86_64 109/164 2024-05-20T16:01:34.997 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2744.gd2b507c7.el9.x86_64 13/164 2024-05-20T16:01:35.027 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : libradosstriper1-2:19.0.0-2744.gd2b507c7.el9.x86 14/164 2024-05-20T16:01:35.098 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2744.gd2b507c7.el9.x86 14/164 2024-05-20T16:01:35.120 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-20T16:01:35.138 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-20T16:01:35.288 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-20T16:01:35.469 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-20T16:01:35.608 INFO:teuthology.orchestra.run.smithi082.stdout: Upgrading : NetworkManager-libnm-1:1.47.90-1.el9.x86_64 17/164 2024-05-20T16:01:35.670 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.90-1.el9.x86_64 17/164 2024-05-20T16:01:35.777 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: NetworkManager-1:1.47.90-1.el9.x86_64 18/164 2024-05-20T16:01:35.846 INFO:teuthology.orchestra.run.smithi082.stdout: Upgrading : NetworkManager-1:1.47.90-1.el9.x86_64 18/164 2024-05-20T16:01:35.987 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-05-20T16:01:36.065 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: NetworkManager-1:1.47.90-1.el9.x86_64 18/164 2024-05-20T16:01:36.068 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-05-20T16:01:36.096 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-05-20T16:01:36.129 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 99/164 2024-05-20T16:01:36.169 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : re2-1:20211101-3.el9.x86_64 19/164 2024-05-20T16:01:36.268 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-05-20T16:01:36.310 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-05-20T16:01:36.361 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-05-20T16:01:36.441 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-05-20T16:01:36.448 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-05-20T16:01:36.484 INFO:teuthology.orchestra.run.smithi160.stdout: Upgrading : librbd1-2:19.0.0-2744.gd2b507c7.el9.x86_64 101/164 2024-05-20T16:01:36.484 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-05-20T16:01:36.528 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 25/164 2024-05-20T16:01:36.557 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: librbd1-2:19.0.0-2744.gd2b507c7.el9.x86_64 101/164 2024-05-20T16:01:36.560 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 26/164 2024-05-20T16:01:36.588 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : lua-5.4.4-4.el9.x86_64 27/164 2024-05-20T16:01:36.588 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-rbd-2:19.0.0-2744.gd2b507c7.el9.x86_64 102/164 2024-05-20T16:01:36.624 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-05-20T16:01:36.656 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 28/164 2024-05-20T16:01:36.710 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 29/164 2024-05-20T16:01:36.741 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 30/164 2024-05-20T16:01:36.776 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 31/164 2024-05-20T16:01:36.854 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 32/164 2024-05-20T16:01:36.887 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-ceph-common-2:19.0.0-2744.gd2b507c7.el9. 33/164 2024-05-20T16:01:36.930 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-ceph-argparse-2:19.0.0-2744.gd2b507c7.el 34/164 2024-05-20T16:01:36.959 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-cephfs-2:19.0.0-2744.gd2b507c7.el9.x86_6 35/164 2024-05-20T16:01:37.028 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 36/164 2024-05-20T16:01:37.074 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : luarocks-3.9.2-5.el9.noarch 37/164 2024-05-20T16:01:37.102 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-05-20T16:01:37.128 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : librgw2-2:19.0.0-2744.gd2b507c7.el9.x86_64 105/164 2024-05-20T16:01:37.148 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 38/164 2024-05-20T16:01:37.199 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: librgw2-2:19.0.0-2744.gd2b507c7.el9.x86_64 105/164 2024-05-20T16:01:37.221 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-rgw-2:19.0.0-2744.gd2b507c7.el9.x86_64 106/164 2024-05-20T16:01:37.281 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 39/164 2024-05-20T16:01:37.334 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 40/164 2024-05-20T16:01:37.864 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-rsa-4.9-2.el9.noarch 41/164 2024-05-20T16:01:37.947 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 42/164 2024-05-20T16:01:37.976 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 43/164 2024-05-20T16:01:38.019 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 44/164 2024-05-20T16:01:38.057 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : librados-devel-2:19.0.0-2744.gd2b507c7.el9.x86_6 45/164 2024-05-20T16:01:38.099 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 46/164 2024-05-20T16:01:38.246 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 47/164 2024-05-20T16:01:38.301 INFO:teuthology.orchestra.run.smithi082.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 48/164 2024-05-20T16:01:38.355 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 49/164 2024-05-20T16:01:38.428 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 50/164 2024-05-20T16:01:38.460 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 51/164 2024-05-20T16:01:38.497 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 52/164 2024-05-20T16:01:38.543 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 53/164 2024-05-20T16:01:38.579 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 54/164 2024-05-20T16:01:38.622 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 55/164 2024-05-20T16:01:38.664 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 56/164 2024-05-20T16:01:38.800 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 57/164 2024-05-20T16:01:38.853 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 58/164 2024-05-20T16:01:38.883 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 59/164 2024-05-20T16:01:38.925 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 60/164 2024-05-20T16:01:39.018 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-05-20T16:01:39.061 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 62/164 2024-05-20T16:01:39.102 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-05-20T16:01:39.135 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 64/164 2024-05-20T16:01:39.159 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-05-20T16:01:39.244 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-05-20T16:01:39.368 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-05-20T16:01:39.408 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-05-20T16:01:39.436 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-05-20T16:01:39.465 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: ceph-common-2:19.0.0-2744.gd2b507c7.el9.x86_64 107/164 2024-05-20T16:01:39.470 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-05-20T16:01:39.509 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-05-20T16:01:39.510 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : ceph-common-2:19.0.0-2744.gd2b507c7.el9.x86_64 107/164 2024-05-20T16:01:39.540 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-05-20T16:01:39.571 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-05-20T16:01:39.613 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-05-20T16:01:39.646 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-05-20T16:01:39.688 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-05-20T16:01:39.723 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-05-20T16:01:39.800 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-05-20T16:01:39.845 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-05-20T16:01:39.875 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-05-20T16:01:39.945 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-05-20T16:01:40.044 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-packaging-20.9-5.el9.noarch 82/164 2024-05-20T16:01:40.066 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: ceph-common-2:19.0.0-2744.gd2b507c7.el9.x86_64 107/164 2024-05-20T16:01:40.093 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/164 2024-05-20T16:01:40.105 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : ceph-base-2:19.0.0-2744.gd2b507c7.el9.x86_64 108/164 2024-05-20T16:01:40.181 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: ceph-base-2:19.0.0-2744.gd2b507c7.el9.x86_64 108/164 2024-05-20T16:01:40.181 INFO:teuthology.orchestra.run.smithi160.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-05-20T16:01:40.181 INFO:teuthology.orchestra.run.smithi160.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-05-20T16:01:40.182 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-20T16:01:40.214 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : ceph-selinux-2:19.0.0-2744.gd2b507c7.el9.x86_64 109/164 2024-05-20T16:01:40.806 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/164 2024-05-20T16:01:40.849 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/164 2024-05-20T16:01:40.882 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/164 2024-05-20T16:01:40.980 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/164 2024-05-20T16:01:41.032 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 88/164 2024-05-20T16:01:41.137 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-05-20T16:01:41.176 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-05-20T16:01:41.353 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : openblas-0.3.21-2.el9.x86_64 91/164 2024-05-20T16:01:41.391 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 92/164 2024-05-20T16:01:41.469 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-05-20T16:01:42.125 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-05-20T16:01:42.238 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-05-20T16:01:43.475 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-05-20T16:01:43.560 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-05-20T16:01:43.580 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-05-20T16:01:43.605 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 99/164 2024-05-20T16:01:43.916 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-05-20T16:01:43.959 INFO:teuthology.orchestra.run.smithi082.stdout: Upgrading : librbd1-2:19.0.0-2744.gd2b507c7.el9.x86_64 101/164 2024-05-20T16:01:44.050 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: librbd1-2:19.0.0-2744.gd2b507c7.el9.x86_64 101/164 2024-05-20T16:01:44.081 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-rbd-2:19.0.0-2744.gd2b507c7.el9.x86_64 102/164 2024-05-20T16:01:44.117 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-05-20T16:01:44.593 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-05-20T16:01:44.620 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : librgw2-2:19.0.0-2744.gd2b507c7.el9.x86_64 105/164 2024-05-20T16:01:44.700 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: librgw2-2:19.0.0-2744.gd2b507c7.el9.x86_64 105/164 2024-05-20T16:01:44.730 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-rgw-2:19.0.0-2744.gd2b507c7.el9.x86_64 106/164 2024-05-20T16:01:46.640 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2744.gd2b507c7.el9.x86_64 109/164 2024-05-20T16:01:46.641 INFO:teuthology.orchestra.run.smithi184.stdout:skipping the directory /sys 2024-05-20T16:01:46.641 INFO:teuthology.orchestra.run.smithi184.stdout:skipping the directory /proc 2024-05-20T16:01:46.641 INFO:teuthology.orchestra.run.smithi184.stdout:skipping the directory /mnt 2024-05-20T16:01:46.641 INFO:teuthology.orchestra.run.smithi184.stdout:skipping the directory /var/tmp 2024-05-20T16:01:46.641 INFO:teuthology.orchestra.run.smithi184.stdout:skipping the directory /home 2024-05-20T16:01:46.641 INFO:teuthology.orchestra.run.smithi184.stdout:skipping the directory /root 2024-05-20T16:01:46.641 INFO:teuthology.orchestra.run.smithi184.stdout:skipping the directory /tmp 2024-05-20T16:01:46.641 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-05-20T16:01:46.817 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-volume-2:19.0.0-2744.gd2b507c7.el9.noarch 110/164 2024-05-20T16:01:46.833 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-volume-2:19.0.0-2744.gd2b507c7.el9.noarch 110/164 2024-05-20T16:01:46.833 INFO:teuthology.orchestra.run.smithi184.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-20T16:01:46.833 INFO:teuthology.orchestra.run.smithi184.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-05-20T16:01:46.833 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-05-20T16:01:46.958 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: ceph-common-2:19.0.0-2744.gd2b507c7.el9.x86_64 107/164 2024-05-20T16:01:46.993 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-common-2:19.0.0-2744.gd2b507c7.el9.x86_64 107/164 2024-05-20T16:01:47.550 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: ceph-common-2:19.0.0-2744.gd2b507c7.el9.x86_64 107/164 2024-05-20T16:01:47.588 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-base-2:19.0.0-2744.gd2b507c7.el9.x86_64 108/164 2024-05-20T16:01:47.680 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: ceph-base-2:19.0.0-2744.gd2b507c7.el9.x86_64 108/164 2024-05-20T16:01:47.680 INFO:teuthology.orchestra.run.smithi082.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-05-20T16:01:47.680 INFO:teuthology.orchestra.run.smithi082.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-05-20T16:01:47.680 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-05-20T16:01:47.715 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-selinux-2:19.0.0-2744.gd2b507c7.el9.x86_64 109/164 2024-05-20T16:01:48.429 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-osd-2:19.0.0-2744.gd2b507c7.el9.x86_64 111/164 2024-05-20T16:01:48.454 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-osd-2:19.0.0-2744.gd2b507c7.el9.x86_64 111/164 2024-05-20T16:01:48.455 INFO:teuthology.orchestra.run.smithi184.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-20T16:01:48.455 INFO:teuthology.orchestra.run.smithi184.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-05-20T16:01:48.455 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-20T16:01:48.455 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-20T16:01:48.455 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-05-20T16:01:48.679 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-mds-2:19.0.0-2744.gd2b507c7.el9.x86_64 112/164 2024-05-20T16:01:48.702 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-mds-2:19.0.0-2744.gd2b507c7.el9.x86_64 112/164 2024-05-20T16:01:48.702 INFO:teuthology.orchestra.run.smithi184.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-20T16:01:48.702 INFO:teuthology.orchestra.run.smithi184.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-05-20T16:01:48.703 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-20T16:01:48.703 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-20T16:01:48.703 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-05-20T16:01:49.172 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-mon-2:19.0.0-2744.gd2b507c7.el9.x86_64 113/164 2024-05-20T16:01:49.194 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-mon-2:19.0.0-2744.gd2b507c7.el9.x86_64 113/164 2024-05-20T16:01:49.195 INFO:teuthology.orchestra.run.smithi184.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-20T16:01:49.195 INFO:teuthology.orchestra.run.smithi184.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-05-20T16:01:49.195 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-20T16:01:49.195 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-20T16:01:49.195 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-05-20T16:01:49.302 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-05-20T16:01:49.359 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-05-20T16:01:49.414 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-05-20T16:01:49.599 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-05-20T16:01:49.647 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-05-20T16:01:49.724 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-05-20T16:01:49.813 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-05-20T16:01:49.953 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-05-20T16:01:49.989 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-05-20T16:01:50.021 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-05-20T16:01:50.048 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : fuse-2.9.9-15.el9.x86_64 124/164 2024-05-20T16:01:50.509 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: cephadm-2:19.0.0-2744.gd2b507c7.el9.noarch 125/164 2024-05-20T16:01:50.538 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : cephadm-2:19.0.0-2744.gd2b507c7.el9.noarch 125/164 2024-05-20T16:01:50.570 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2744.gd2b507c7.e 126/164 2024-05-20T16:01:50.632 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2744.gd2b507c7. 127/164 2024-05-20T16:01:50.661 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2744.gd2b507c7.el9.noa 128/164 2024-05-20T16:01:51.565 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2744.gd2b507c7.el9.x86_64 109/164 2024-05-20T16:01:51.565 INFO:teuthology.orchestra.run.smithi160.stdout:skipping the directory /sys 2024-05-20T16:01:51.566 INFO:teuthology.orchestra.run.smithi160.stdout:skipping the directory /proc 2024-05-20T16:01:51.566 INFO:teuthology.orchestra.run.smithi160.stdout:skipping the directory /mnt 2024-05-20T16:01:51.566 INFO:teuthology.orchestra.run.smithi160.stdout:skipping the directory /var/tmp 2024-05-20T16:01:51.566 INFO:teuthology.orchestra.run.smithi160.stdout:skipping the directory /home 2024-05-20T16:01:51.566 INFO:teuthology.orchestra.run.smithi160.stdout:skipping the directory /root 2024-05-20T16:01:51.566 INFO:teuthology.orchestra.run.smithi160.stdout:skipping the directory /tmp 2024-05-20T16:01:51.566 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-20T16:01:51.687 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2744.gd2b507c7.el9.noa 128/164 2024-05-20T16:01:51.730 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2744.gd2b507c7.el9.n 129/164 2024-05-20T16:01:51.742 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : ceph-volume-2:19.0.0-2744.gd2b507c7.el9.noarch 110/164 2024-05-20T16:01:51.757 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: ceph-volume-2:19.0.0-2744.gd2b507c7.el9.noarch 110/164 2024-05-20T16:01:51.757 INFO:teuthology.orchestra.run.smithi160.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-20T16:01:51.757 INFO:teuthology.orchestra.run.smithi160.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-05-20T16:01:51.757 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-20T16:01:52.704 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2744.gd2b507c7.el9.n 129/164 2024-05-20T16:01:52.739 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2744.gd2b 130/164 2024-05-20T16:01:52.842 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2744.gd2b 130/164 2024-05-20T16:01:53.011 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2744.gd2b507c7.el 131/164 2024-05-20T16:01:53.041 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-mgr-2:19.0.0-2744.gd2b507c7.el9.x86_64 132/164 2024-05-20T16:01:53.064 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2744.gd2b507c7.el9.x86_64 132/164 2024-05-20T16:01:53.064 INFO:teuthology.orchestra.run.smithi184.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-20T16:01:53.064 INFO:teuthology.orchestra.run.smithi184.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-05-20T16:01:53.064 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-20T16:01:53.065 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-20T16:01:53.065 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-05-20T16:01:53.108 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-mgr-rook-2:19.0.0-2744.gd2b507c7.el9.noarch 133/164 2024-05-20T16:01:53.119 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2744.gd2b507c7.el9.noarch 133/164 2024-05-20T16:01:53.218 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-2:19.0.0-2744.gd2b507c7.el9.x86_64 134/164 2024-05-20T16:01:53.265 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-fuse-2:19.0.0-2744.gd2b507c7.el9.x86_64 135/164 2024-05-20T16:01:53.353 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : ceph-osd-2:19.0.0-2744.gd2b507c7.el9.x86_64 111/164 2024-05-20T16:01:53.378 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: ceph-osd-2:19.0.0-2744.gd2b507c7.el9.x86_64 111/164 2024-05-20T16:01:53.378 INFO:teuthology.orchestra.run.smithi160.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-20T16:01:53.378 INFO:teuthology.orchestra.run.smithi160.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-05-20T16:01:53.378 INFO:teuthology.orchestra.run.smithi160.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-20T16:01:53.378 INFO:teuthology.orchestra.run.smithi160.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-20T16:01:53.378 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-20T16:01:53.604 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : ceph-mds-2:19.0.0-2744.gd2b507c7.el9.x86_64 112/164 2024-05-20T16:01:53.626 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: ceph-mds-2:19.0.0-2744.gd2b507c7.el9.x86_64 112/164 2024-05-20T16:01:53.626 INFO:teuthology.orchestra.run.smithi160.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-20T16:01:53.627 INFO:teuthology.orchestra.run.smithi160.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-05-20T16:01:53.627 INFO:teuthology.orchestra.run.smithi160.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-20T16:01:53.627 INFO:teuthology.orchestra.run.smithi160.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-20T16:01:53.627 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-20T16:01:54.088 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : ceph-mon-2:19.0.0-2744.gd2b507c7.el9.x86_64 113/164 2024-05-20T16:01:54.110 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: ceph-mon-2:19.0.0-2744.gd2b507c7.el9.x86_64 113/164 2024-05-20T16:01:54.110 INFO:teuthology.orchestra.run.smithi160.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-20T16:01:54.110 INFO:teuthology.orchestra.run.smithi160.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-05-20T16:01:54.110 INFO:teuthology.orchestra.run.smithi160.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-20T16:01:54.110 INFO:teuthology.orchestra.run.smithi160.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-20T16:01:54.110 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-20T16:01:54.182 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : libtool-2.4.6-45.el9.x86_64 136/164 2024-05-20T16:01:54.218 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-05-20T16:01:54.219 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-radosgw-2:19.0.0-2744.gd2b507c7.el9.x86_64 137/164 2024-05-20T16:01:54.241 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2744.gd2b507c7.el9.x86_64 137/164 2024-05-20T16:01:54.242 INFO:teuthology.orchestra.run.smithi184.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-20T16:01:54.242 INFO:teuthology.orchestra.run.smithi184.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-05-20T16:01:54.242 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-20T16:01:54.242 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-20T16:01:54.242 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-05-20T16:01:54.274 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-05-20T16:01:54.294 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2744.gd2b50 138/164 2024-05-20T16:01:54.316 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2744.gd2b50 138/164 2024-05-20T16:01:54.316 INFO:teuthology.orchestra.run.smithi184.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-20T16:01:54.316 INFO:teuthology.orchestra.run.smithi184.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-05-20T16:01:54.316 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-05-20T16:01:54.330 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-05-20T16:01:54.362 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : cephfs-mirror-2:19.0.0-2744.gd2b507c7.el9.x86_64 139/164 2024-05-20T16:01:54.384 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2744.gd2b507c7.el9.x86_64 139/164 2024-05-20T16:01:54.384 INFO:teuthology.orchestra.run.smithi184.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-20T16:01:54.384 INFO:teuthology.orchestra.run.smithi184.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-05-20T16:01:54.384 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-20T16:01:54.384 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-20T16:01:54.385 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-05-20T16:01:54.515 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-05-20T16:01:54.572 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-05-20T16:01:54.649 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-05-20T16:01:54.704 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : rbd-mirror-2:19.0.0-2744.gd2b507c7.el9.x86_64 140/164 2024-05-20T16:01:54.726 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2744.gd2b507c7.el9.x86_64 140/164 2024-05-20T16:01:54.727 INFO:teuthology.orchestra.run.smithi184.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-20T16:01:54.727 INFO:teuthology.orchestra.run.smithi184.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-05-20T16:01:54.727 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-20T16:01:54.727 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-20T16:01:54.727 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-05-20T16:01:54.738 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-05-20T16:01:54.886 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-05-20T16:01:54.922 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-05-20T16:01:54.962 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-05-20T16:01:54.989 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : fuse-2.9.9-15.el9.x86_64 124/164 2024-05-20T16:01:55.400 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: cephadm-2:19.0.0-2744.gd2b507c7.el9.noarch 125/164 2024-05-20T16:01:55.430 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : cephadm-2:19.0.0-2744.gd2b507c7.el9.noarch 125/164 2024-05-20T16:01:55.461 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2744.gd2b507c7.e 126/164 2024-05-20T16:01:55.523 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2744.gd2b507c7. 127/164 2024-05-20T16:01:55.552 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2744.gd2b507c7.el9.noa 128/164 2024-05-20T16:01:56.590 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2744.gd2b507c7.el9.noa 128/164 2024-05-20T16:01:56.621 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2744.gd2b507c7.el9.n 129/164 2024-05-20T16:01:57.587 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2744.gd2b507c7.el9.n 129/164 2024-05-20T16:01:57.613 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2744.gd2b 130/164 2024-05-20T16:01:57.717 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2744.gd2b 130/164 2024-05-20T16:01:57.886 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2744.gd2b507c7.el 131/164 2024-05-20T16:01:57.915 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : ceph-mgr-2:19.0.0-2744.gd2b507c7.el9.x86_64 132/164 2024-05-20T16:01:57.938 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2744.gd2b507c7.el9.x86_64 132/164 2024-05-20T16:01:57.938 INFO:teuthology.orchestra.run.smithi160.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-20T16:01:57.938 INFO:teuthology.orchestra.run.smithi160.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-05-20T16:01:57.938 INFO:teuthology.orchestra.run.smithi160.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-20T16:01:57.938 INFO:teuthology.orchestra.run.smithi160.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-20T16:01:57.938 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-20T16:01:57.982 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : ceph-mgr-rook-2:19.0.0-2744.gd2b507c7.el9.noarch 133/164 2024-05-20T16:01:57.993 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2744.gd2b507c7.el9.noarch 133/164 2024-05-20T16:01:58.092 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : ceph-2:19.0.0-2744.gd2b507c7.el9.x86_64 134/164 2024-05-20T16:01:58.139 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : ceph-fuse-2:19.0.0-2744.gd2b507c7.el9.x86_64 135/164 2024-05-20T16:01:59.050 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : libtool-2.4.6-45.el9.x86_64 136/164 2024-05-20T16:01:59.085 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : ceph-radosgw-2:19.0.0-2744.gd2b507c7.el9.x86_64 137/164 2024-05-20T16:01:59.107 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2744.gd2b507c7.el9.x86_64 137/164 2024-05-20T16:01:59.107 INFO:teuthology.orchestra.run.smithi160.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-20T16:01:59.107 INFO:teuthology.orchestra.run.smithi160.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-05-20T16:01:59.107 INFO:teuthology.orchestra.run.smithi160.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-20T16:01:59.107 INFO:teuthology.orchestra.run.smithi160.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-20T16:01:59.108 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-20T16:01:59.152 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2744.gd2b50 138/164 2024-05-20T16:01:59.174 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2744.gd2b50 138/164 2024-05-20T16:01:59.174 INFO:teuthology.orchestra.run.smithi160.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-20T16:01:59.174 INFO:teuthology.orchestra.run.smithi160.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-05-20T16:01:59.174 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-20T16:01:59.219 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : cephfs-mirror-2:19.0.0-2744.gd2b507c7.el9.x86_64 139/164 2024-05-20T16:01:59.241 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2744.gd2b507c7.el9.x86_64 139/164 2024-05-20T16:01:59.242 INFO:teuthology.orchestra.run.smithi160.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-20T16:01:59.242 INFO:teuthology.orchestra.run.smithi160.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-05-20T16:01:59.242 INFO:teuthology.orchestra.run.smithi160.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-20T16:01:59.242 INFO:teuthology.orchestra.run.smithi160.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-20T16:01:59.242 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-20T16:01:59.308 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-test-2:19.0.0-2744.gd2b507c7.el9.x86_64 141/164 2024-05-20T16:01:59.349 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : rbd-fuse-2:19.0.0-2744.gd2b507c7.el9.x86_64 142/164 2024-05-20T16:01:59.389 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : rbd-nbd-2:19.0.0-2744.gd2b507c7.el9.x86_64 143/164 2024-05-20T16:01:59.403 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2744.gd2b507c7.el9.x86_64 109/164 2024-05-20T16:01:59.403 INFO:teuthology.orchestra.run.smithi082.stdout:skipping the directory /sys 2024-05-20T16:01:59.403 INFO:teuthology.orchestra.run.smithi082.stdout:skipping the directory /proc 2024-05-20T16:01:59.403 INFO:teuthology.orchestra.run.smithi082.stdout:skipping the directory /mnt 2024-05-20T16:01:59.403 INFO:teuthology.orchestra.run.smithi082.stdout:skipping the directory /var/tmp 2024-05-20T16:01:59.403 INFO:teuthology.orchestra.run.smithi082.stdout:skipping the directory /home 2024-05-20T16:01:59.403 INFO:teuthology.orchestra.run.smithi082.stdout:skipping the directory /root 2024-05-20T16:01:59.403 INFO:teuthology.orchestra.run.smithi082.stdout:skipping the directory /tmp 2024-05-20T16:01:59.403 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-05-20T16:01:59.421 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 144/164 2024-05-20T16:01:59.448 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 145/164 2024-05-20T16:01:59.553 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : rbd-mirror-2:19.0.0-2744.gd2b507c7.el9.x86_64 140/164 2024-05-20T16:01:59.559 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-volume-2:19.0.0-2744.gd2b507c7.el9.noarch 110/164 2024-05-20T16:01:59.575 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: ceph-volume-2:19.0.0-2744.gd2b507c7.el9.noarch 110/164 2024-05-20T16:01:59.575 INFO:teuthology.orchestra.run.smithi082.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-20T16:01:59.575 INFO:teuthology.orchestra.run.smithi082.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-05-20T16:01:59.575 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-05-20T16:01:59.575 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2744.gd2b507c7.el9.x86_64 140/164 2024-05-20T16:01:59.576 INFO:teuthology.orchestra.run.smithi160.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-20T16:01:59.576 INFO:teuthology.orchestra.run.smithi160.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-05-20T16:01:59.576 INFO:teuthology.orchestra.run.smithi160.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-20T16:01:59.576 INFO:teuthology.orchestra.run.smithi160.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-20T16:01:59.576 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-20T16:01:59.632 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : cephfs-shell-2:19.0.0-2744.gd2b507c7.el9.x86_64 146/164 2024-05-20T16:01:59.678 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 147/164 2024-05-20T16:01:59.708 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : libcephfs-devel-2:19.0.0-2744.gd2b507c7.el9.x86_ 148/164 2024-05-20T16:01:59.738 INFO:teuthology.orchestra.run.smithi184.stdout: Upgrading : NetworkManager-team-1:1.47.90-1.el9.x86_64 149/164 2024-05-20T16:01:59.776 INFO:teuthology.orchestra.run.smithi184.stdout: Upgrading : NetworkManager-tui-1:1.47.90-1.el9.x86_64 150/164 2024-05-20T16:02:00.386 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-20T16:02:00.432 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-20T16:02:00.451 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-20T16:02:01.162 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-osd-2:19.0.0-2744.gd2b507c7.el9.x86_64 111/164 2024-05-20T16:02:01.188 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: ceph-osd-2:19.0.0-2744.gd2b507c7.el9.x86_64 111/164 2024-05-20T16:02:01.188 INFO:teuthology.orchestra.run.smithi082.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-20T16:02:01.189 INFO:teuthology.orchestra.run.smithi082.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-05-20T16:02:01.189 INFO:teuthology.orchestra.run.smithi082.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-20T16:02:01.189 INFO:teuthology.orchestra.run.smithi082.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-20T16:02:01.189 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-05-20T16:02:01.290 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : cephfs-top-2:19.0.0-2744.gd2b507c7.el9.noarch 152/164 2024-05-20T16:02:01.386 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 153/164 2024-05-20T16:02:01.413 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-mds-2:19.0.0-2744.gd2b507c7.el9.x86_64 112/164 2024-05-20T16:02:01.427 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-05-20T16:02:01.436 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: ceph-mds-2:19.0.0-2744.gd2b507c7.el9.x86_64 112/164 2024-05-20T16:02:01.436 INFO:teuthology.orchestra.run.smithi082.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-20T16:02:01.436 INFO:teuthology.orchestra.run.smithi082.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-05-20T16:02:01.436 INFO:teuthology.orchestra.run.smithi082.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-20T16:02:01.436 INFO:teuthology.orchestra.run.smithi082.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-20T16:02:01.436 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-05-20T16:02:01.509 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-05-20T16:02:01.548 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-05-20T16:02:01.574 INFO:teuthology.orchestra.run.smithi184.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 157/164 2024-05-20T16:02:01.574 INFO:teuthology.orchestra.run.smithi184.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-20T16:02:01.658 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-20T16:02:01.699 INFO:teuthology.orchestra.run.smithi184.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 159/164 2024-05-20T16:02:01.700 INFO:teuthology.orchestra.run.smithi184.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-20T16:02:01.792 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-20T16:02:01.817 INFO:teuthology.orchestra.run.smithi184.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 161/164 2024-05-20T16:02:01.822 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-20T16:02:01.822 INFO:teuthology.orchestra.run.smithi184.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-20T16:02:01.905 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-mon-2:19.0.0-2744.gd2b507c7.el9.x86_64 113/164 2024-05-20T16:02:01.928 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: ceph-mon-2:19.0.0-2744.gd2b507c7.el9.x86_64 113/164 2024-05-20T16:02:01.928 INFO:teuthology.orchestra.run.smithi082.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-20T16:02:01.928 INFO:teuthology.orchestra.run.smithi082.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-05-20T16:02:01.928 INFO:teuthology.orchestra.run.smithi082.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-20T16:02:01.928 INFO:teuthology.orchestra.run.smithi082.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-20T16:02:01.928 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-05-20T16:02:02.010 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-20T16:02:02.011 INFO:teuthology.orchestra.run.smithi184.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-20T16:02:02.026 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-05-20T16:02:02.075 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-05-20T16:02:02.094 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-20T16:02:02.129 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-05-20T16:02:02.157 INFO:teuthology.orchestra.run.smithi184.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-20T16:02:02.311 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-05-20T16:02:02.355 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-05-20T16:02:02.432 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-05-20T16:02:02.520 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-05-20T16:02:02.661 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-05-20T16:02:02.705 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-05-20T16:02:02.745 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-05-20T16:02:02.773 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : fuse-2.9.9-15.el9.x86_64 124/164 2024-05-20T16:02:03.243 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: cephadm-2:19.0.0-2744.gd2b507c7.el9.noarch 125/164 2024-05-20T16:02:03.271 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : cephadm-2:19.0.0-2744.gd2b507c7.el9.noarch 125/164 2024-05-20T16:02:03.303 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2744.gd2b507c7.e 126/164 2024-05-20T16:02:03.364 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2744.gd2b507c7. 127/164 2024-05-20T16:02:03.395 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2744.gd2b507c7.el9.noa 128/164 2024-05-20T16:02:04.123 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : ceph-test-2:19.0.0-2744.gd2b507c7.el9.x86_64 141/164 2024-05-20T16:02:04.165 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : rbd-fuse-2:19.0.0-2744.gd2b507c7.el9.x86_64 142/164 2024-05-20T16:02:04.205 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : rbd-nbd-2:19.0.0-2744.gd2b507c7.el9.x86_64 143/164 2024-05-20T16:02:04.246 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 144/164 2024-05-20T16:02:04.274 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 145/164 2024-05-20T16:02:04.425 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2744.gd2b507c7.el9.noa 128/164 2024-05-20T16:02:04.454 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2744.gd2b507c7.el9.n 129/164 2024-05-20T16:02:04.456 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : cephfs-shell-2:19.0.0-2744.gd2b507c7.el9.x86_64 146/164 2024-05-20T16:02:04.502 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 147/164 2024-05-20T16:02:04.525 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : libcephfs-devel-2:19.0.0-2744.gd2b507c7.el9.x86_ 148/164 2024-05-20T16:02:04.554 INFO:teuthology.orchestra.run.smithi160.stdout: Upgrading : NetworkManager-team-1:1.47.90-1.el9.x86_64 149/164 2024-05-20T16:02:04.592 INFO:teuthology.orchestra.run.smithi160.stdout: Upgrading : NetworkManager-tui-1:1.47.90-1.el9.x86_64 150/164 2024-05-20T16:02:05.190 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-20T16:02:05.231 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-20T16:02:05.251 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-20T16:02:05.415 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2744.gd2b507c7.el9.n 129/164 2024-05-20T16:02:05.438 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2744.gd2b 130/164 2024-05-20T16:02:05.539 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2744.gd2b 130/164 2024-05-20T16:02:05.704 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2744.gd2b507c7.el 131/164 2024-05-20T16:02:05.748 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-mgr-2:19.0.0-2744.gd2b507c7.el9.x86_64 132/164 2024-05-20T16:02:05.772 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2744.gd2b507c7.el9.x86_64 132/164 2024-05-20T16:02:05.772 INFO:teuthology.orchestra.run.smithi082.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-20T16:02:05.772 INFO:teuthology.orchestra.run.smithi082.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-05-20T16:02:05.772 INFO:teuthology.orchestra.run.smithi082.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-20T16:02:05.773 INFO:teuthology.orchestra.run.smithi082.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-20T16:02:05.773 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-05-20T16:02:05.824 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-mgr-rook-2:19.0.0-2744.gd2b507c7.el9.noarch 133/164 2024-05-20T16:02:05.836 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2744.gd2b507c7.el9.noarch 133/164 2024-05-20T16:02:05.934 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-2:19.0.0-2744.gd2b507c7.el9.x86_64 134/164 2024-05-20T16:02:05.995 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-fuse-2:19.0.0-2744.gd2b507c7.el9.x86_64 135/164 2024-05-20T16:02:06.090 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : cephfs-top-2:19.0.0-2744.gd2b507c7.el9.noarch 152/164 2024-05-20T16:02:06.169 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 153/164 2024-05-20T16:02:06.210 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-05-20T16:02:06.291 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-05-20T16:02:06.342 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-05-20T16:02:06.365 INFO:teuthology.orchestra.run.smithi160.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 157/164 2024-05-20T16:02:06.365 INFO:teuthology.orchestra.run.smithi160.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-20T16:02:06.440 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-20T16:02:06.474 INFO:teuthology.orchestra.run.smithi160.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 159/164 2024-05-20T16:02:06.474 INFO:teuthology.orchestra.run.smithi160.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-20T16:02:06.558 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-20T16:02:06.583 INFO:teuthology.orchestra.run.smithi160.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 161/164 2024-05-20T16:02:06.587 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-20T16:02:06.588 INFO:teuthology.orchestra.run.smithi160.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-20T16:02:06.760 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-20T16:02:06.760 INFO:teuthology.orchestra.run.smithi160.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-20T16:02:06.852 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-20T16:02:06.908 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : libtool-2.4.6-45.el9.x86_64 136/164 2024-05-20T16:02:06.917 INFO:teuthology.orchestra.run.smithi160.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-20T16:02:06.934 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-radosgw-2:19.0.0-2744.gd2b507c7.el9.x86_64 137/164 2024-05-20T16:02:06.958 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2744.gd2b507c7.el9.x86_64 137/164 2024-05-20T16:02:06.958 INFO:teuthology.orchestra.run.smithi082.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-20T16:02:06.958 INFO:teuthology.orchestra.run.smithi082.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-05-20T16:02:06.958 INFO:teuthology.orchestra.run.smithi082.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-20T16:02:06.958 INFO:teuthology.orchestra.run.smithi082.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-20T16:02:06.958 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-05-20T16:02:07.002 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2744.gd2b50 138/164 2024-05-20T16:02:07.024 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2744.gd2b50 138/164 2024-05-20T16:02:07.024 INFO:teuthology.orchestra.run.smithi082.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-20T16:02:07.025 INFO:teuthology.orchestra.run.smithi082.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-05-20T16:02:07.025 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-05-20T16:02:07.069 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : cephfs-mirror-2:19.0.0-2744.gd2b507c7.el9.x86_64 139/164 2024-05-20T16:02:07.093 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2744.gd2b507c7.el9.x86_64 139/164 2024-05-20T16:02:07.093 INFO:teuthology.orchestra.run.smithi082.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-20T16:02:07.093 INFO:teuthology.orchestra.run.smithi082.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-05-20T16:02:07.093 INFO:teuthology.orchestra.run.smithi082.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-20T16:02:07.093 INFO:teuthology.orchestra.run.smithi082.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-20T16:02:07.093 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-05-20T16:02:07.403 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : rbd-mirror-2:19.0.0-2744.gd2b507c7.el9.x86_64 140/164 2024-05-20T16:02:07.426 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2744.gd2b507c7.el9.x86_64 140/164 2024-05-20T16:02:07.427 INFO:teuthology.orchestra.run.smithi082.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-20T16:02:07.427 INFO:teuthology.orchestra.run.smithi082.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-05-20T16:02:07.427 INFO:teuthology.orchestra.run.smithi082.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-20T16:02:07.427 INFO:teuthology.orchestra.run.smithi082.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-20T16:02:07.427 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-05-20T16:02:07.788 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-20T16:02:07.788 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-2:19.0.0-2744.gd2b507c7.el9.x86_64 1/164 2024-05-20T16:02:07.788 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-base-2:19.0.0-2744.gd2b507c7.el9.x86_64 2/164 2024-05-20T16:02:07.788 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-common-2:19.0.0-2744.gd2b507c7.el9.x86_64 3/164 2024-05-20T16:02:07.788 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-fuse-2:19.0.0-2744.gd2b507c7.el9.x86_64 4/164 2024-05-20T16:02:07.789 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2744.gd2b50 5/164 2024-05-20T16:02:07.789 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-mds-2:19.0.0-2744.gd2b507c7.el9.x86_64 6/164 2024-05-20T16:02:07.789 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-mgr-2:19.0.0-2744.gd2b507c7.el9.x86_64 7/164 2024-05-20T16:02:07.789 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-mon-2:19.0.0-2744.gd2b507c7.el9.x86_64 8/164 2024-05-20T16:02:07.789 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-osd-2:19.0.0-2744.gd2b507c7.el9.x86_64 9/164 2024-05-20T16:02:07.789 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-radosgw-2:19.0.0-2744.gd2b507c7.el9.x86_64 10/164 2024-05-20T16:02:07.789 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-selinux-2:19.0.0-2744.gd2b507c7.el9.x86_64 11/164 2024-05-20T16:02:07.789 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-test-2:19.0.0-2744.gd2b507c7.el9.x86_64 12/164 2024-05-20T16:02:07.789 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : cephfs-mirror-2:19.0.0-2744.gd2b507c7.el9.x86_64 13/164 2024-05-20T16:02:07.789 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : cephfs-shell-2:19.0.0-2744.gd2b507c7.el9.x86_64 14/164 2024-05-20T16:02:07.789 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libcephfs-devel-2:19.0.0-2744.gd2b507c7.el9.x86_ 15/164 2024-05-20T16:02:07.789 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libcephfs2-2:19.0.0-2744.gd2b507c7.el9.x86_64 16/164 2024-05-20T16:02:07.790 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libcephsqlite-2:19.0.0-2744.gd2b507c7.el9.x86_64 17/164 2024-05-20T16:02:07.790 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : librados-devel-2:19.0.0-2744.gd2b507c7.el9.x86_6 18/164 2024-05-20T16:02:07.790 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libradosstriper1-2:19.0.0-2744.gd2b507c7.el9.x86 19/164 2024-05-20T16:02:07.790 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : librgw2-2:19.0.0-2744.gd2b507c7.el9.x86_64 20/164 2024-05-20T16:02:07.790 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2744.gd2b507c7.el 21/164 2024-05-20T16:02:07.790 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-ceph-common-2:19.0.0-2744.gd2b507c7.el9. 22/164 2024-05-20T16:02:07.790 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-cephfs-2:19.0.0-2744.gd2b507c7.el9.x86_6 23/164 2024-05-20T16:02:07.790 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-rados-2:19.0.0-2744.gd2b507c7.el9.x86_64 24/164 2024-05-20T16:02:07.790 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-rbd-2:19.0.0-2744.gd2b507c7.el9.x86_64 25/164 2024-05-20T16:02:07.790 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-rgw-2:19.0.0-2744.gd2b507c7.el9.x86_64 26/164 2024-05-20T16:02:07.790 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : rbd-fuse-2:19.0.0-2744.gd2b507c7.el9.x86_64 27/164 2024-05-20T16:02:07.791 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : rbd-mirror-2:19.0.0-2744.gd2b507c7.el9.x86_64 28/164 2024-05-20T16:02:07.791 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : rbd-nbd-2:19.0.0-2744.gd2b507c7.el9.x86_64 29/164 2024-05-20T16:02:07.791 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2744.gd2b507c7. 30/164 2024-05-20T16:02:07.791 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2744.gd2b507c7.el9.noa 31/164 2024-05-20T16:02:07.791 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2744.gd2b507c7.el9.n 32/164 2024-05-20T16:02:07.791 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2744.gd2b 33/164 2024-05-20T16:02:07.791 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2744.gd2b507c7.el 34/164 2024-05-20T16:02:07.791 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2744.gd2b507c7.el9.noarch 35/164 2024-05-20T16:02:07.791 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2744.gd2b507c7.e 36/164 2024-05-20T16:02:07.791 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-volume-2:19.0.0-2744.gd2b507c7.el9.noarch 37/164 2024-05-20T16:02:07.792 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : cephadm-2:19.0.0-2744.gd2b507c7.el9.noarch 38/164 2024-05-20T16:02:07.792 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : cephfs-top-2:19.0.0-2744.gd2b507c7.el9.noarch 39/164 2024-05-20T16:02:07.792 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/164 2024-05-20T16:02:07.792 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/164 2024-05-20T16:02:07.792 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/164 2024-05-20T16:02:07.792 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-05-20T16:02:07.792 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-05-20T16:02:07.792 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-05-20T16:02:07.792 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-05-20T16:02:07.792 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-05-20T16:02:07.792 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/164 2024-05-20T16:02:07.792 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-05-20T16:02:07.793 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-05-20T16:02:07.793 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-05-20T16:02:07.793 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/164 2024-05-20T16:02:07.793 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-05-20T16:02:07.793 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-05-20T16:02:07.793 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-05-20T16:02:07.793 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-05-20T16:02:07.793 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-05-20T16:02:07.793 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-05-20T16:02:07.793 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-05-20T16:02:07.793 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-05-20T16:02:07.793 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/164 2024-05-20T16:02:07.793 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/164 2024-05-20T16:02:07.794 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-05-20T16:02:07.794 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-05-20T16:02:07.794 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-05-20T16:02:07.794 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/164 2024-05-20T16:02:07.794 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/164 2024-05-20T16:02:07.794 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/164 2024-05-20T16:02:07.794 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/164 2024-05-20T16:02:07.794 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/164 2024-05-20T16:02:07.794 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/164 2024-05-20T16:02:07.794 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/164 2024-05-20T16:02:07.794 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-05-20T16:02:07.794 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-05-20T16:02:07.795 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-05-20T16:02:07.795 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/164 2024-05-20T16:02:07.795 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/164 2024-05-20T16:02:07.795 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/164 2024-05-20T16:02:07.795 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/164 2024-05-20T16:02:07.795 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/164 2024-05-20T16:02:07.795 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/164 2024-05-20T16:02:07.795 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/164 2024-05-20T16:02:07.795 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/164 2024-05-20T16:02:07.795 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/164 2024-05-20T16:02:07.795 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/164 2024-05-20T16:02:07.795 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/164 2024-05-20T16:02:07.795 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/164 2024-05-20T16:02:07.795 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/164 2024-05-20T16:02:07.796 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/164 2024-05-20T16:02:07.796 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/164 2024-05-20T16:02:07.796 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/164 2024-05-20T16:02:07.796 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/164 2024-05-20T16:02:07.796 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/164 2024-05-20T16:02:07.796 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/164 2024-05-20T16:02:07.796 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/164 2024-05-20T16:02:07.796 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/164 2024-05-20T16:02:07.796 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/164 2024-05-20T16:02:07.796 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/164 2024-05-20T16:02:07.796 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/164 2024-05-20T16:02:07.796 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/164 2024-05-20T16:02:07.796 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/164 2024-05-20T16:02:07.797 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/164 2024-05-20T16:02:07.797 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/164 2024-05-20T16:02:07.797 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/164 2024-05-20T16:02:07.797 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/164 2024-05-20T16:02:07.797 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/164 2024-05-20T16:02:07.797 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/164 2024-05-20T16:02:07.797 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 108/164 2024-05-20T16:02:07.797 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 109/164 2024-05-20T16:02:07.797 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/164 2024-05-20T16:02:07.797 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/164 2024-05-20T16:02:07.797 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 112/164 2024-05-20T16:02:07.798 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 113/164 2024-05-20T16:02:07.798 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/164 2024-05-20T16:02:07.798 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/164 2024-05-20T16:02:07.798 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/164 2024-05-20T16:02:07.798 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/164 2024-05-20T16:02:07.798 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/164 2024-05-20T16:02:07.798 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/164 2024-05-20T16:02:07.798 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/164 2024-05-20T16:02:07.798 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/164 2024-05-20T16:02:07.798 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/164 2024-05-20T16:02:07.799 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-05-20T16:02:07.799 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-05-20T16:02:07.799 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-05-20T16:02:07.799 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-05-20T16:02:07.799 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-05-20T16:02:07.799 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-05-20T16:02:07.799 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-05-20T16:02:07.799 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-05-20T16:02:07.799 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-05-20T16:02:07.799 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-05-20T16:02:07.799 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-05-20T16:02:07.799 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-05-20T16:02:07.799 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-05-20T16:02:07.800 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-05-20T16:02:07.800 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-05-20T16:02:07.800 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-05-20T16:02:07.800 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-05-20T16:02:07.800 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-05-20T16:02:07.800 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-05-20T16:02:07.800 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/164 2024-05-20T16:02:07.800 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/164 2024-05-20T16:02:07.800 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-05-20T16:02:07.800 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-05-20T16:02:07.800 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-05-20T16:02:07.800 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/164 2024-05-20T16:02:07.800 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-05-20T16:02:07.801 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : librados2-2:19.0.0-2744.gd2b507c7.el9.x86_64 149/164 2024-05-20T16:02:07.801 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-05-20T16:02:07.801 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : librbd1-2:19.0.0-2744.gd2b507c7.el9.x86_64 151/164 2024-05-20T16:02:07.801 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-05-20T16:02:07.801 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : NetworkManager-1:1.47.90-1.el9.x86_64 153/164 2024-05-20T16:02:07.801 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/164 2024-05-20T16:02:07.801 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : NetworkManager-libnm-1:1.47.90-1.el9.x86_64 155/164 2024-05-20T16:02:07.801 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/164 2024-05-20T16:02:07.801 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : NetworkManager-team-1:1.47.90-1.el9.x86_64 157/164 2024-05-20T16:02:07.801 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/164 2024-05-20T16:02:07.801 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : NetworkManager-tui-1:1.47.90-1.el9.x86_64 159/164 2024-05-20T16:02:07.801 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/164 2024-05-20T16:02:07.801 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 161/164 2024-05-20T16:02:07.802 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 162/164 2024-05-20T16:02:07.802 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 163/164 2024-05-20T16:02:10.103 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-20T16:02:10.103 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-05-20T16:02:10.103 INFO:teuthology.orchestra.run.smithi184.stdout:Upgraded: 2024-05-20T16:02:10.104 INFO:teuthology.orchestra.run.smithi184.stdout: NetworkManager-1:1.47.90-1.el9.x86_64 2024-05-20T16:02:10.104 INFO:teuthology.orchestra.run.smithi184.stdout: NetworkManager-libnm-1:1.47.90-1.el9.x86_64 2024-05-20T16:02:10.104 INFO:teuthology.orchestra.run.smithi184.stdout: NetworkManager-team-1:1.47.90-1.el9.x86_64 2024-05-20T16:02:10.104 INFO:teuthology.orchestra.run.smithi184.stdout: NetworkManager-tui-1:1.47.90-1.el9.x86_64 2024-05-20T16:02:10.104 INFO:teuthology.orchestra.run.smithi184.stdout: librados2-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-20T16:02:10.104 INFO:teuthology.orchestra.run.smithi184.stdout: librbd1-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-20T16:02:10.104 INFO:teuthology.orchestra.run.smithi184.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-05-20T16:02:10.104 INFO:teuthology.orchestra.run.smithi184.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-05-20T16:02:10.104 INFO:teuthology.orchestra.run.smithi184.stdout:Installed: 2024-05-20T16:02:10.104 INFO:teuthology.orchestra.run.smithi184.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-05-20T16:02:10.104 INFO:teuthology.orchestra.run.smithi184.stdout: automake-1.16.2-8.el9.noarch 2024-05-20T16:02:10.105 INFO:teuthology.orchestra.run.smithi184.stdout: bison-3.7.4-5.el9.x86_64 2024-05-20T16:02:10.105 INFO:teuthology.orchestra.run.smithi184.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-05-20T16:02:10.105 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-20T16:02:10.105 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-base-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-20T16:02:10.105 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-common-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-20T16:02:10.105 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-fuse-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-20T16:02:10.105 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-grafana-dashboards-2:19.0.0-2744.gd2b507c7.el9.noarch 2024-05-20T16:02:10.105 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-immutable-object-cache-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-20T16:02:10.105 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mds-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-20T16:02:10.105 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mgr-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-20T16:02:10.105 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mgr-cephadm-2:19.0.0-2744.gd2b507c7.el9.noarch 2024-05-20T16:02:10.105 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mgr-dashboard-2:19.0.0-2744.gd2b507c7.el9.noarch 2024-05-20T16:02:10.106 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2744.gd2b507c7.el9.noarch 2024-05-20T16:02:10.106 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mgr-modules-core-2:19.0.0-2744.gd2b507c7.el9.noarch 2024-05-20T16:02:10.106 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mgr-rook-2:19.0.0-2744.gd2b507c7.el9.noarch 2024-05-20T16:02:10.106 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mon-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-20T16:02:10.106 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-osd-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-20T16:02:10.106 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-prometheus-alerts-2:19.0.0-2744.gd2b507c7.el9.noarch 2024-05-20T16:02:10.106 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-radosgw-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-20T16:02:10.106 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-selinux-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-20T16:02:10.106 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-test-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-20T16:02:10.106 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-volume-2:19.0.0-2744.gd2b507c7.el9.noarch 2024-05-20T16:02:10.106 INFO:teuthology.orchestra.run.smithi184.stdout: cephadm-2:19.0.0-2744.gd2b507c7.el9.noarch 2024-05-20T16:02:10.107 INFO:teuthology.orchestra.run.smithi184.stdout: cephfs-mirror-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-20T16:02:10.107 INFO:teuthology.orchestra.run.smithi184.stdout: cephfs-shell-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-20T16:02:10.107 INFO:teuthology.orchestra.run.smithi184.stdout: cephfs-top-2:19.0.0-2744.gd2b507c7.el9.noarch 2024-05-20T16:02:10.107 INFO:teuthology.orchestra.run.smithi184.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-05-20T16:02:10.107 INFO:teuthology.orchestra.run.smithi184.stdout: flex-2.6.4-9.el9.x86_64 2024-05-20T16:02:10.107 INFO:teuthology.orchestra.run.smithi184.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-05-20T16:02:10.107 INFO:teuthology.orchestra.run.smithi184.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-05-20T16:02:10.107 INFO:teuthology.orchestra.run.smithi184.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-05-20T16:02:10.107 INFO:teuthology.orchestra.run.smithi184.stdout: fuse-2.9.9-15.el9.x86_64 2024-05-20T16:02:10.107 INFO:teuthology.orchestra.run.smithi184.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-05-20T16:02:10.107 INFO:teuthology.orchestra.run.smithi184.stdout: grpc-data-1.46.7-10.el9.noarch 2024-05-20T16:02:10.107 INFO:teuthology.orchestra.run.smithi184.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-05-20T16:02:10.108 INFO:teuthology.orchestra.run.smithi184.stdout: libarrow-9.0.0-11.el9.x86_64 2024-05-20T16:02:10.108 INFO:teuthology.orchestra.run.smithi184.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-05-20T16:02:10.108 INFO:teuthology.orchestra.run.smithi184.stdout: libcephfs-devel-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-20T16:02:10.108 INFO:teuthology.orchestra.run.smithi184.stdout: libcephfs2-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-20T16:02:10.108 INFO:teuthology.orchestra.run.smithi184.stdout: libcephsqlite-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-20T16:02:10.108 INFO:teuthology.orchestra.run.smithi184.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-05-20T16:02:10.108 INFO:teuthology.orchestra.run.smithi184.stdout: liboath-2.6.7-2.el9.x86_64 2024-05-20T16:02:10.108 INFO:teuthology.orchestra.run.smithi184.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-05-20T16:02:10.108 INFO:teuthology.orchestra.run.smithi184.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-05-20T16:02:10.108 INFO:teuthology.orchestra.run.smithi184.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-05-20T16:02:10.108 INFO:teuthology.orchestra.run.smithi184.stdout: librados-devel-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-20T16:02:10.109 INFO:teuthology.orchestra.run.smithi184.stdout: libradosstriper1-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-20T16:02:10.109 INFO:teuthology.orchestra.run.smithi184.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-05-20T16:02:10.109 INFO:teuthology.orchestra.run.smithi184.stdout: librgw2-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-20T16:02:10.109 INFO:teuthology.orchestra.run.smithi184.stdout: libtool-2.4.6-45.el9.x86_64 2024-05-20T16:02:10.109 INFO:teuthology.orchestra.run.smithi184.stdout: libunwind-1.6.2-1.el9.x86_64 2024-05-20T16:02:10.109 INFO:teuthology.orchestra.run.smithi184.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-05-20T16:02:10.109 INFO:teuthology.orchestra.run.smithi184.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-05-20T16:02:10.109 INFO:teuthology.orchestra.run.smithi184.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-05-20T16:02:10.109 INFO:teuthology.orchestra.run.smithi184.stdout: lua-5.4.4-4.el9.x86_64 2024-05-20T16:02:10.109 INFO:teuthology.orchestra.run.smithi184.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-05-20T16:02:10.109 INFO:teuthology.orchestra.run.smithi184.stdout: luarocks-3.9.2-5.el9.noarch 2024-05-20T16:02:10.109 INFO:teuthology.orchestra.run.smithi184.stdout: openblas-0.3.21-2.el9.x86_64 2024-05-20T16:02:10.110 INFO:teuthology.orchestra.run.smithi184.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-05-20T16:02:10.110 INFO:teuthology.orchestra.run.smithi184.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-05-20T16:02:10.110 INFO:teuthology.orchestra.run.smithi184.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-05-20T16:02:10.110 INFO:teuthology.orchestra.run.smithi184.stdout: postgresql-13.14-1.el9.x86_64 2024-05-20T16:02:10.110 INFO:teuthology.orchestra.run.smithi184.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-05-20T16:02:10.110 INFO:teuthology.orchestra.run.smithi184.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-05-20T16:02:10.110 INFO:teuthology.orchestra.run.smithi184.stdout: postgresql-server-13.14-1.el9.x86_64 2024-05-20T16:02:10.110 INFO:teuthology.orchestra.run.smithi184.stdout: protobuf-3.14.0-13.el9.x86_64 2024-05-20T16:02:10.110 INFO:teuthology.orchestra.run.smithi184.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-05-20T16:02:10.110 INFO:teuthology.orchestra.run.smithi184.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-05-20T16:02:10.111 INFO:teuthology.orchestra.run.smithi184.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-05-20T16:02:10.111 INFO:teuthology.orchestra.run.smithi184.stdout: python3-babel-2.9.1-2.el9.noarch 2024-05-20T16:02:10.111 INFO:teuthology.orchestra.run.smithi184.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-05-20T16:02:10.111 INFO:teuthology.orchestra.run.smithi184.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-05-20T16:02:10.111 INFO:teuthology.orchestra.run.smithi184.stdout: python3-ceph-argparse-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-20T16:02:10.111 INFO:teuthology.orchestra.run.smithi184.stdout: python3-ceph-common-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-20T16:02:10.111 INFO:teuthology.orchestra.run.smithi184.stdout: python3-cephfs-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-20T16:02:10.111 INFO:teuthology.orchestra.run.smithi184.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-05-20T16:02:10.111 INFO:teuthology.orchestra.run.smithi184.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-05-20T16:02:10.111 INFO:teuthology.orchestra.run.smithi184.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-05-20T16:02:10.111 INFO:teuthology.orchestra.run.smithi184.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-05-20T16:02:10.111 INFO:teuthology.orchestra.run.smithi184.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-05-20T16:02:10.112 INFO:teuthology.orchestra.run.smithi184.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-05-20T16:02:10.112 INFO:teuthology.orchestra.run.smithi184.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-05-20T16:02:10.112 INFO:teuthology.orchestra.run.smithi184.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-05-20T16:02:10.112 INFO:teuthology.orchestra.run.smithi184.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-05-20T16:02:10.112 INFO:teuthology.orchestra.run.smithi184.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-05-20T16:02:10.112 INFO:teuthology.orchestra.run.smithi184.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-05-20T16:02:10.112 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-05-20T16:02:10.112 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-05-20T16:02:10.112 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-05-20T16:02:10.112 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-05-20T16:02:10.112 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-05-20T16:02:10.112 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-05-20T16:02:10.113 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-05-20T16:02:10.113 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-05-20T16:02:10.113 INFO:teuthology.orchestra.run.smithi184.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-05-20T16:02:10.113 INFO:teuthology.orchestra.run.smithi184.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-05-20T16:02:10.113 INFO:teuthology.orchestra.run.smithi184.stdout: python3-mako-1.1.4-6.el9.noarch 2024-05-20T16:02:10.113 INFO:teuthology.orchestra.run.smithi184.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-05-20T16:02:10.113 INFO:teuthology.orchestra.run.smithi184.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-05-20T16:02:10.113 INFO:teuthology.orchestra.run.smithi184.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-05-20T16:02:10.113 INFO:teuthology.orchestra.run.smithi184.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-05-20T16:02:10.113 INFO:teuthology.orchestra.run.smithi184.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-05-20T16:02:10.113 INFO:teuthology.orchestra.run.smithi184.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-05-20T16:02:10.114 INFO:teuthology.orchestra.run.smithi184.stdout: python3-packaging-20.9-5.el9.noarch 2024-05-20T16:02:10.114 INFO:teuthology.orchestra.run.smithi184.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-05-20T16:02:10.114 INFO:teuthology.orchestra.run.smithi184.stdout: python3-ply-3.11-14.el9.noarch 2024-05-20T16:02:10.114 INFO:teuthology.orchestra.run.smithi184.stdout: python3-portend-3.1.0-2.el9.noarch 2024-05-20T16:02:10.114 INFO:teuthology.orchestra.run.smithi184.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-05-20T16:02:10.114 INFO:teuthology.orchestra.run.smithi184.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-05-20T16:02:10.114 INFO:teuthology.orchestra.run.smithi184.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-05-20T16:02:10.114 INFO:teuthology.orchestra.run.smithi184.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-05-20T16:02:10.114 INFO:teuthology.orchestra.run.smithi184.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-05-20T16:02:10.114 INFO:teuthology.orchestra.run.smithi184.stdout: python3-pycparser-2.20-6.el9.noarch 2024-05-20T16:02:10.115 INFO:teuthology.orchestra.run.smithi184.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-05-20T16:02:10.115 INFO:teuthology.orchestra.run.smithi184.stdout: python3-pytz-2021.1-5.el9.noarch 2024-05-20T16:02:10.115 INFO:teuthology.orchestra.run.smithi184.stdout: python3-rados-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-20T16:02:10.115 INFO:teuthology.orchestra.run.smithi184.stdout: python3-rbd-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-20T16:02:10.115 INFO:teuthology.orchestra.run.smithi184.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-05-20T16:02:10.115 INFO:teuthology.orchestra.run.smithi184.stdout: python3-requests-2.25.1-8.el9.noarch 2024-05-20T16:02:10.115 INFO:teuthology.orchestra.run.smithi184.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-05-20T16:02:10.115 INFO:teuthology.orchestra.run.smithi184.stdout: python3-rgw-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-20T16:02:10.115 INFO:teuthology.orchestra.run.smithi184.stdout: python3-routes-2.5.1-5.el9.noarch 2024-05-20T16:02:10.115 INFO:teuthology.orchestra.run.smithi184.stdout: python3-rsa-4.9-2.el9.noarch 2024-05-20T16:02:10.115 INFO:teuthology.orchestra.run.smithi184.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-05-20T16:02:10.116 INFO:teuthology.orchestra.run.smithi184.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-05-20T16:02:10.116 INFO:teuthology.orchestra.run.smithi184.stdout: python3-toml-0.10.2-6.el9.noarch 2024-05-20T16:02:10.116 INFO:teuthology.orchestra.run.smithi184.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-05-20T16:02:10.116 INFO:teuthology.orchestra.run.smithi184.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-05-20T16:02:10.116 INFO:teuthology.orchestra.run.smithi184.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-05-20T16:02:10.116 INFO:teuthology.orchestra.run.smithi184.stdout: python3-webob-1.8.7-6.el9.noarch 2024-05-20T16:02:10.116 INFO:teuthology.orchestra.run.smithi184.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-05-20T16:02:10.116 INFO:teuthology.orchestra.run.smithi184.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-05-20T16:02:10.116 INFO:teuthology.orchestra.run.smithi184.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-05-20T16:02:10.116 INFO:teuthology.orchestra.run.smithi184.stdout: qatlib-24.02.0-1.el9.x86_64 2024-05-20T16:02:10.116 INFO:teuthology.orchestra.run.smithi184.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-05-20T16:02:10.116 INFO:teuthology.orchestra.run.smithi184.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-05-20T16:02:10.117 INFO:teuthology.orchestra.run.smithi184.stdout: rbd-fuse-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-20T16:02:10.117 INFO:teuthology.orchestra.run.smithi184.stdout: rbd-mirror-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-20T16:02:10.117 INFO:teuthology.orchestra.run.smithi184.stdout: rbd-nbd-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-20T16:02:10.117 INFO:teuthology.orchestra.run.smithi184.stdout: re2-1:20211101-3.el9.x86_64 2024-05-20T16:02:10.117 INFO:teuthology.orchestra.run.smithi184.stdout: socat-1.7.4.1-5.el9.x86_64 2024-05-20T16:02:10.117 INFO:teuthology.orchestra.run.smithi184.stdout: thrift-0.15.0-2.el9.x86_64 2024-05-20T16:02:10.117 INFO:teuthology.orchestra.run.smithi184.stdout: uuid-1.6.2-55.el9.x86_64 2024-05-20T16:02:10.117 INFO:teuthology.orchestra.run.smithi184.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-05-20T16:02:10.117 INFO:teuthology.orchestra.run.smithi184.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-05-20T16:02:10.118 INFO:teuthology.orchestra.run.smithi184.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-05-20T16:02:10.118 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-05-20T16:02:10.118 INFO:teuthology.orchestra.run.smithi184.stdout:Complete! 2024-05-20T16:02:10.567 DEBUG:teuthology.parallel:result is None 2024-05-20T16:02:11.997 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-test-2:19.0.0-2744.gd2b507c7.el9.x86_64 141/164 2024-05-20T16:02:12.038 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : rbd-fuse-2:19.0.0-2744.gd2b507c7.el9.x86_64 142/164 2024-05-20T16:02:12.078 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : rbd-nbd-2:19.0.0-2744.gd2b507c7.el9.x86_64 143/164 2024-05-20T16:02:12.111 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 144/164 2024-05-20T16:02:12.154 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 145/164 2024-05-20T16:02:12.345 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : cephfs-shell-2:19.0.0-2744.gd2b507c7.el9.x86_64 146/164 2024-05-20T16:02:12.393 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 147/164 2024-05-20T16:02:12.423 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : libcephfs-devel-2:19.0.0-2744.gd2b507c7.el9.x86_ 148/164 2024-05-20T16:02:12.452 INFO:teuthology.orchestra.run.smithi082.stdout: Upgrading : NetworkManager-team-1:1.47.90-1.el9.x86_64 149/164 2024-05-20T16:02:12.488 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-20T16:02:12.488 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : ceph-2:19.0.0-2744.gd2b507c7.el9.x86_64 1/164 2024-05-20T16:02:12.488 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : ceph-base-2:19.0.0-2744.gd2b507c7.el9.x86_64 2/164 2024-05-20T16:02:12.488 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : ceph-common-2:19.0.0-2744.gd2b507c7.el9.x86_64 3/164 2024-05-20T16:02:12.488 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : ceph-fuse-2:19.0.0-2744.gd2b507c7.el9.x86_64 4/164 2024-05-20T16:02:12.488 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2744.gd2b50 5/164 2024-05-20T16:02:12.488 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : ceph-mds-2:19.0.0-2744.gd2b507c7.el9.x86_64 6/164 2024-05-20T16:02:12.488 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : ceph-mgr-2:19.0.0-2744.gd2b507c7.el9.x86_64 7/164 2024-05-20T16:02:12.489 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : ceph-mon-2:19.0.0-2744.gd2b507c7.el9.x86_64 8/164 2024-05-20T16:02:12.489 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : ceph-osd-2:19.0.0-2744.gd2b507c7.el9.x86_64 9/164 2024-05-20T16:02:12.489 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : ceph-radosgw-2:19.0.0-2744.gd2b507c7.el9.x86_64 10/164 2024-05-20T16:02:12.489 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : ceph-selinux-2:19.0.0-2744.gd2b507c7.el9.x86_64 11/164 2024-05-20T16:02:12.489 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : ceph-test-2:19.0.0-2744.gd2b507c7.el9.x86_64 12/164 2024-05-20T16:02:12.489 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : cephfs-mirror-2:19.0.0-2744.gd2b507c7.el9.x86_64 13/164 2024-05-20T16:02:12.489 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : cephfs-shell-2:19.0.0-2744.gd2b507c7.el9.x86_64 14/164 2024-05-20T16:02:12.489 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : libcephfs-devel-2:19.0.0-2744.gd2b507c7.el9.x86_ 15/164 2024-05-20T16:02:12.489 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : libcephfs2-2:19.0.0-2744.gd2b507c7.el9.x86_64 16/164 2024-05-20T16:02:12.489 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : libcephsqlite-2:19.0.0-2744.gd2b507c7.el9.x86_64 17/164 2024-05-20T16:02:12.489 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : librados-devel-2:19.0.0-2744.gd2b507c7.el9.x86_6 18/164 2024-05-20T16:02:12.489 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : libradosstriper1-2:19.0.0-2744.gd2b507c7.el9.x86 19/164 2024-05-20T16:02:12.489 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : librgw2-2:19.0.0-2744.gd2b507c7.el9.x86_64 20/164 2024-05-20T16:02:12.490 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2744.gd2b507c7.el 21/164 2024-05-20T16:02:12.490 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-ceph-common-2:19.0.0-2744.gd2b507c7.el9. 22/164 2024-05-20T16:02:12.490 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-cephfs-2:19.0.0-2744.gd2b507c7.el9.x86_6 23/164 2024-05-20T16:02:12.490 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-rados-2:19.0.0-2744.gd2b507c7.el9.x86_64 24/164 2024-05-20T16:02:12.490 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-rbd-2:19.0.0-2744.gd2b507c7.el9.x86_64 25/164 2024-05-20T16:02:12.490 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-rgw-2:19.0.0-2744.gd2b507c7.el9.x86_64 26/164 2024-05-20T16:02:12.490 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : rbd-fuse-2:19.0.0-2744.gd2b507c7.el9.x86_64 27/164 2024-05-20T16:02:12.490 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : rbd-mirror-2:19.0.0-2744.gd2b507c7.el9.x86_64 28/164 2024-05-20T16:02:12.490 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : rbd-nbd-2:19.0.0-2744.gd2b507c7.el9.x86_64 29/164 2024-05-20T16:02:12.490 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2744.gd2b507c7. 30/164 2024-05-20T16:02:12.490 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2744.gd2b507c7.el9.noa 31/164 2024-05-20T16:02:12.490 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2744.gd2b507c7.el9.n 32/164 2024-05-20T16:02:12.491 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2744.gd2b 33/164 2024-05-20T16:02:12.491 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2744.gd2b507c7.el 34/164 2024-05-20T16:02:12.491 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2744.gd2b507c7.el9.noarch 35/164 2024-05-20T16:02:12.491 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2744.gd2b507c7.e 36/164 2024-05-20T16:02:12.491 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : ceph-volume-2:19.0.0-2744.gd2b507c7.el9.noarch 37/164 2024-05-20T16:02:12.491 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : cephadm-2:19.0.0-2744.gd2b507c7.el9.noarch 38/164 2024-05-20T16:02:12.491 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : cephfs-top-2:19.0.0-2744.gd2b507c7.el9.noarch 39/164 2024-05-20T16:02:12.491 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/164 2024-05-20T16:02:12.491 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/164 2024-05-20T16:02:12.491 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/164 2024-05-20T16:02:12.491 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-05-20T16:02:12.491 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-05-20T16:02:12.491 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-05-20T16:02:12.492 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-05-20T16:02:12.492 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-05-20T16:02:12.492 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/164 2024-05-20T16:02:12.492 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-05-20T16:02:12.492 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-05-20T16:02:12.492 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-05-20T16:02:12.492 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/164 2024-05-20T16:02:12.492 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-05-20T16:02:12.492 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-05-20T16:02:12.492 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-05-20T16:02:12.492 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-05-20T16:02:12.492 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-05-20T16:02:12.492 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-05-20T16:02:12.493 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-05-20T16:02:12.493 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-05-20T16:02:12.493 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/164 2024-05-20T16:02:12.493 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/164 2024-05-20T16:02:12.493 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-05-20T16:02:12.493 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-05-20T16:02:12.493 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-05-20T16:02:12.493 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/164 2024-05-20T16:02:12.493 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/164 2024-05-20T16:02:12.493 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/164 2024-05-20T16:02:12.493 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/164 2024-05-20T16:02:12.493 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/164 2024-05-20T16:02:12.493 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/164 2024-05-20T16:02:12.493 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/164 2024-05-20T16:02:12.494 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-05-20T16:02:12.494 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-05-20T16:02:12.494 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-05-20T16:02:12.494 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/164 2024-05-20T16:02:12.494 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/164 2024-05-20T16:02:12.494 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/164 2024-05-20T16:02:12.494 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/164 2024-05-20T16:02:12.494 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/164 2024-05-20T16:02:12.494 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/164 2024-05-20T16:02:12.494 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/164 2024-05-20T16:02:12.494 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/164 2024-05-20T16:02:12.494 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/164 2024-05-20T16:02:12.494 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/164 2024-05-20T16:02:12.495 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/164 2024-05-20T16:02:12.495 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/164 2024-05-20T16:02:12.495 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/164 2024-05-20T16:02:12.495 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/164 2024-05-20T16:02:12.495 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/164 2024-05-20T16:02:12.495 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/164 2024-05-20T16:02:12.495 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/164 2024-05-20T16:02:12.495 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/164 2024-05-20T16:02:12.495 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/164 2024-05-20T16:02:12.495 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/164 2024-05-20T16:02:12.495 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/164 2024-05-20T16:02:12.495 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/164 2024-05-20T16:02:12.496 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/164 2024-05-20T16:02:12.496 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/164 2024-05-20T16:02:12.496 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/164 2024-05-20T16:02:12.496 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/164 2024-05-20T16:02:12.496 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/164 2024-05-20T16:02:12.496 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/164 2024-05-20T16:02:12.496 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/164 2024-05-20T16:02:12.496 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/164 2024-05-20T16:02:12.496 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/164 2024-05-20T16:02:12.496 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/164 2024-05-20T16:02:12.496 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 108/164 2024-05-20T16:02:12.496 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 109/164 2024-05-20T16:02:12.496 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/164 2024-05-20T16:02:12.497 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/164 2024-05-20T16:02:12.497 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 112/164 2024-05-20T16:02:12.497 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 113/164 2024-05-20T16:02:12.497 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/164 2024-05-20T16:02:12.497 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/164 2024-05-20T16:02:12.497 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/164 2024-05-20T16:02:12.497 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/164 2024-05-20T16:02:12.497 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/164 2024-05-20T16:02:12.497 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/164 2024-05-20T16:02:12.497 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/164 2024-05-20T16:02:12.497 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/164 2024-05-20T16:02:12.498 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/164 2024-05-20T16:02:12.498 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-05-20T16:02:12.498 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-05-20T16:02:12.498 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-05-20T16:02:12.498 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-05-20T16:02:12.498 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-05-20T16:02:12.498 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-05-20T16:02:12.498 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-05-20T16:02:12.498 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-05-20T16:02:12.498 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-05-20T16:02:12.498 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-05-20T16:02:12.498 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-05-20T16:02:12.498 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-05-20T16:02:12.499 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-05-20T16:02:12.499 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-05-20T16:02:12.499 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-05-20T16:02:12.499 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-05-20T16:02:12.499 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-05-20T16:02:12.499 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-05-20T16:02:12.499 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-05-20T16:02:12.499 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/164 2024-05-20T16:02:12.499 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/164 2024-05-20T16:02:12.499 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-05-20T16:02:12.499 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-05-20T16:02:12.499 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-05-20T16:02:12.499 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/164 2024-05-20T16:02:12.500 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-05-20T16:02:12.500 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : librados2-2:19.0.0-2744.gd2b507c7.el9.x86_64 149/164 2024-05-20T16:02:12.500 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-05-20T16:02:12.500 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : librbd1-2:19.0.0-2744.gd2b507c7.el9.x86_64 151/164 2024-05-20T16:02:12.500 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-05-20T16:02:12.500 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : NetworkManager-1:1.47.90-1.el9.x86_64 153/164 2024-05-20T16:02:12.500 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/164 2024-05-20T16:02:12.500 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : NetworkManager-libnm-1:1.47.90-1.el9.x86_64 155/164 2024-05-20T16:02:12.500 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/164 2024-05-20T16:02:12.500 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : NetworkManager-team-1:1.47.90-1.el9.x86_64 157/164 2024-05-20T16:02:12.500 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/164 2024-05-20T16:02:12.500 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : NetworkManager-tui-1:1.47.90-1.el9.x86_64 159/164 2024-05-20T16:02:12.500 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/164 2024-05-20T16:02:12.500 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 161/164 2024-05-20T16:02:12.501 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 162/164 2024-05-20T16:02:12.501 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 163/164 2024-05-20T16:02:12.501 INFO:teuthology.orchestra.run.smithi082.stdout: Upgrading : NetworkManager-tui-1:1.47.90-1.el9.x86_64 150/164 2024-05-20T16:02:13.129 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-20T16:02:13.172 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-20T16:02:13.192 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-20T16:02:14.023 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : cephfs-top-2:19.0.0-2744.gd2b507c7.el9.noarch 152/164 2024-05-20T16:02:14.118 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 153/164 2024-05-20T16:02:14.169 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-05-20T16:02:14.272 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-05-20T16:02:14.305 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-05-20T16:02:14.331 INFO:teuthology.orchestra.run.smithi082.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 157/164 2024-05-20T16:02:14.331 INFO:teuthology.orchestra.run.smithi082.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-20T16:02:14.407 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-20T16:02:14.432 INFO:teuthology.orchestra.run.smithi082.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 159/164 2024-05-20T16:02:14.432 INFO:teuthology.orchestra.run.smithi082.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-20T16:02:14.517 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-20T16:02:14.541 INFO:teuthology.orchestra.run.smithi082.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 161/164 2024-05-20T16:02:14.546 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-20T16:02:14.546 INFO:teuthology.orchestra.run.smithi082.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-20T16:02:14.735 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-20T16:02:14.736 INFO:teuthology.orchestra.run.smithi082.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-20T16:02:14.819 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-20T16:02:14.831 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-20T16:02:14.831 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-20T16:02:14.831 INFO:teuthology.orchestra.run.smithi160.stdout:Upgraded: 2024-05-20T16:02:14.831 INFO:teuthology.orchestra.run.smithi160.stdout: NetworkManager-1:1.47.90-1.el9.x86_64 2024-05-20T16:02:14.831 INFO:teuthology.orchestra.run.smithi160.stdout: NetworkManager-libnm-1:1.47.90-1.el9.x86_64 2024-05-20T16:02:14.831 INFO:teuthology.orchestra.run.smithi160.stdout: NetworkManager-team-1:1.47.90-1.el9.x86_64 2024-05-20T16:02:14.831 INFO:teuthology.orchestra.run.smithi160.stdout: NetworkManager-tui-1:1.47.90-1.el9.x86_64 2024-05-20T16:02:14.831 INFO:teuthology.orchestra.run.smithi160.stdout: librados2-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-20T16:02:14.831 INFO:teuthology.orchestra.run.smithi160.stdout: librbd1-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-20T16:02:14.831 INFO:teuthology.orchestra.run.smithi160.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-05-20T16:02:14.832 INFO:teuthology.orchestra.run.smithi160.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-05-20T16:02:14.832 INFO:teuthology.orchestra.run.smithi160.stdout:Installed: 2024-05-20T16:02:14.832 INFO:teuthology.orchestra.run.smithi160.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-05-20T16:02:14.832 INFO:teuthology.orchestra.run.smithi160.stdout: automake-1.16.2-8.el9.noarch 2024-05-20T16:02:14.832 INFO:teuthology.orchestra.run.smithi160.stdout: bison-3.7.4-5.el9.x86_64 2024-05-20T16:02:14.832 INFO:teuthology.orchestra.run.smithi160.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-05-20T16:02:14.832 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-20T16:02:14.832 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-base-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-20T16:02:14.832 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-common-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-20T16:02:14.832 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-fuse-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-20T16:02:14.832 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-grafana-dashboards-2:19.0.0-2744.gd2b507c7.el9.noarch 2024-05-20T16:02:14.832 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-immutable-object-cache-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-20T16:02:14.832 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-mds-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-20T16:02:14.832 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-mgr-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-20T16:02:14.832 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-mgr-cephadm-2:19.0.0-2744.gd2b507c7.el9.noarch 2024-05-20T16:02:14.833 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-mgr-dashboard-2:19.0.0-2744.gd2b507c7.el9.noarch 2024-05-20T16:02:14.833 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2744.gd2b507c7.el9.noarch 2024-05-20T16:02:14.833 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-mgr-modules-core-2:19.0.0-2744.gd2b507c7.el9.noarch 2024-05-20T16:02:14.833 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-mgr-rook-2:19.0.0-2744.gd2b507c7.el9.noarch 2024-05-20T16:02:14.833 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-mon-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-20T16:02:14.833 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-osd-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-20T16:02:14.833 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-prometheus-alerts-2:19.0.0-2744.gd2b507c7.el9.noarch 2024-05-20T16:02:14.833 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-radosgw-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-20T16:02:14.833 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-selinux-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-20T16:02:14.833 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-test-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-20T16:02:14.833 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-volume-2:19.0.0-2744.gd2b507c7.el9.noarch 2024-05-20T16:02:14.833 INFO:teuthology.orchestra.run.smithi160.stdout: cephadm-2:19.0.0-2744.gd2b507c7.el9.noarch 2024-05-20T16:02:14.833 INFO:teuthology.orchestra.run.smithi160.stdout: cephfs-mirror-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-20T16:02:14.833 INFO:teuthology.orchestra.run.smithi160.stdout: cephfs-shell-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-20T16:02:14.834 INFO:teuthology.orchestra.run.smithi160.stdout: cephfs-top-2:19.0.0-2744.gd2b507c7.el9.noarch 2024-05-20T16:02:14.834 INFO:teuthology.orchestra.run.smithi160.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-05-20T16:02:14.834 INFO:teuthology.orchestra.run.smithi160.stdout: flex-2.6.4-9.el9.x86_64 2024-05-20T16:02:14.834 INFO:teuthology.orchestra.run.smithi160.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-05-20T16:02:14.834 INFO:teuthology.orchestra.run.smithi160.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-05-20T16:02:14.834 INFO:teuthology.orchestra.run.smithi160.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-05-20T16:02:14.834 INFO:teuthology.orchestra.run.smithi160.stdout: fuse-2.9.9-15.el9.x86_64 2024-05-20T16:02:14.834 INFO:teuthology.orchestra.run.smithi160.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-05-20T16:02:14.834 INFO:teuthology.orchestra.run.smithi160.stdout: grpc-data-1.46.7-10.el9.noarch 2024-05-20T16:02:14.834 INFO:teuthology.orchestra.run.smithi160.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-05-20T16:02:14.834 INFO:teuthology.orchestra.run.smithi160.stdout: libarrow-9.0.0-11.el9.x86_64 2024-05-20T16:02:14.834 INFO:teuthology.orchestra.run.smithi160.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-05-20T16:02:14.834 INFO:teuthology.orchestra.run.smithi160.stdout: libcephfs-devel-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-20T16:02:14.835 INFO:teuthology.orchestra.run.smithi160.stdout: libcephfs2-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-20T16:02:14.835 INFO:teuthology.orchestra.run.smithi160.stdout: libcephsqlite-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-20T16:02:14.835 INFO:teuthology.orchestra.run.smithi160.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-05-20T16:02:14.835 INFO:teuthology.orchestra.run.smithi160.stdout: liboath-2.6.7-2.el9.x86_64 2024-05-20T16:02:14.835 INFO:teuthology.orchestra.run.smithi160.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-05-20T16:02:14.835 INFO:teuthology.orchestra.run.smithi160.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-05-20T16:02:14.835 INFO:teuthology.orchestra.run.smithi160.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-05-20T16:02:14.835 INFO:teuthology.orchestra.run.smithi160.stdout: librados-devel-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-20T16:02:14.835 INFO:teuthology.orchestra.run.smithi160.stdout: libradosstriper1-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-20T16:02:14.835 INFO:teuthology.orchestra.run.smithi160.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-05-20T16:02:14.835 INFO:teuthology.orchestra.run.smithi160.stdout: librgw2-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-20T16:02:14.835 INFO:teuthology.orchestra.run.smithi160.stdout: libtool-2.4.6-45.el9.x86_64 2024-05-20T16:02:14.835 INFO:teuthology.orchestra.run.smithi160.stdout: libunwind-1.6.2-1.el9.x86_64 2024-05-20T16:02:14.835 INFO:teuthology.orchestra.run.smithi160.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-05-20T16:02:14.835 INFO:teuthology.orchestra.run.smithi160.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-05-20T16:02:14.836 INFO:teuthology.orchestra.run.smithi160.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-05-20T16:02:14.836 INFO:teuthology.orchestra.run.smithi160.stdout: lua-5.4.4-4.el9.x86_64 2024-05-20T16:02:14.836 INFO:teuthology.orchestra.run.smithi160.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-05-20T16:02:14.836 INFO:teuthology.orchestra.run.smithi160.stdout: luarocks-3.9.2-5.el9.noarch 2024-05-20T16:02:14.836 INFO:teuthology.orchestra.run.smithi160.stdout: openblas-0.3.21-2.el9.x86_64 2024-05-20T16:02:14.836 INFO:teuthology.orchestra.run.smithi160.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-05-20T16:02:14.836 INFO:teuthology.orchestra.run.smithi160.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-05-20T16:02:14.836 INFO:teuthology.orchestra.run.smithi160.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-05-20T16:02:14.836 INFO:teuthology.orchestra.run.smithi160.stdout: postgresql-13.14-1.el9.x86_64 2024-05-20T16:02:14.836 INFO:teuthology.orchestra.run.smithi160.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-05-20T16:02:14.836 INFO:teuthology.orchestra.run.smithi160.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-05-20T16:02:14.836 INFO:teuthology.orchestra.run.smithi160.stdout: postgresql-server-13.14-1.el9.x86_64 2024-05-20T16:02:14.836 INFO:teuthology.orchestra.run.smithi160.stdout: protobuf-3.14.0-13.el9.x86_64 2024-05-20T16:02:14.836 INFO:teuthology.orchestra.run.smithi160.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-05-20T16:02:14.837 INFO:teuthology.orchestra.run.smithi160.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-05-20T16:02:14.837 INFO:teuthology.orchestra.run.smithi160.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-05-20T16:02:14.837 INFO:teuthology.orchestra.run.smithi160.stdout: python3-babel-2.9.1-2.el9.noarch 2024-05-20T16:02:14.837 INFO:teuthology.orchestra.run.smithi160.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-05-20T16:02:14.837 INFO:teuthology.orchestra.run.smithi160.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-05-20T16:02:14.837 INFO:teuthology.orchestra.run.smithi160.stdout: python3-ceph-argparse-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-20T16:02:14.837 INFO:teuthology.orchestra.run.smithi160.stdout: python3-ceph-common-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-20T16:02:14.837 INFO:teuthology.orchestra.run.smithi160.stdout: python3-cephfs-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-20T16:02:14.837 INFO:teuthology.orchestra.run.smithi160.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-05-20T16:02:14.837 INFO:teuthology.orchestra.run.smithi160.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-05-20T16:02:14.837 INFO:teuthology.orchestra.run.smithi160.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-05-20T16:02:14.837 INFO:teuthology.orchestra.run.smithi160.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-05-20T16:02:14.838 INFO:teuthology.orchestra.run.smithi160.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-05-20T16:02:14.838 INFO:teuthology.orchestra.run.smithi160.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-05-20T16:02:14.838 INFO:teuthology.orchestra.run.smithi160.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-05-20T16:02:14.838 INFO:teuthology.orchestra.run.smithi160.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-05-20T16:02:14.838 INFO:teuthology.orchestra.run.smithi160.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-05-20T16:02:14.838 INFO:teuthology.orchestra.run.smithi160.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-05-20T16:02:14.838 INFO:teuthology.orchestra.run.smithi160.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-05-20T16:02:14.838 INFO:teuthology.orchestra.run.smithi160.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-05-20T16:02:14.838 INFO:teuthology.orchestra.run.smithi160.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-05-20T16:02:14.838 INFO:teuthology.orchestra.run.smithi160.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-05-20T16:02:14.838 INFO:teuthology.orchestra.run.smithi160.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-05-20T16:02:14.838 INFO:teuthology.orchestra.run.smithi160.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-05-20T16:02:14.839 INFO:teuthology.orchestra.run.smithi160.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-05-20T16:02:14.839 INFO:teuthology.orchestra.run.smithi160.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-05-20T16:02:14.839 INFO:teuthology.orchestra.run.smithi160.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-05-20T16:02:14.839 INFO:teuthology.orchestra.run.smithi160.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-05-20T16:02:14.839 INFO:teuthology.orchestra.run.smithi160.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-05-20T16:02:14.839 INFO:teuthology.orchestra.run.smithi160.stdout: python3-mako-1.1.4-6.el9.noarch 2024-05-20T16:02:14.839 INFO:teuthology.orchestra.run.smithi160.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-05-20T16:02:14.839 INFO:teuthology.orchestra.run.smithi160.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-05-20T16:02:14.839 INFO:teuthology.orchestra.run.smithi160.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-05-20T16:02:14.839 INFO:teuthology.orchestra.run.smithi160.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-05-20T16:02:14.839 INFO:teuthology.orchestra.run.smithi160.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-05-20T16:02:14.839 INFO:teuthology.orchestra.run.smithi160.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-05-20T16:02:14.839 INFO:teuthology.orchestra.run.smithi160.stdout: python3-packaging-20.9-5.el9.noarch 2024-05-20T16:02:14.839 INFO:teuthology.orchestra.run.smithi160.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-05-20T16:02:14.840 INFO:teuthology.orchestra.run.smithi160.stdout: python3-ply-3.11-14.el9.noarch 2024-05-20T16:02:14.840 INFO:teuthology.orchestra.run.smithi160.stdout: python3-portend-3.1.0-2.el9.noarch 2024-05-20T16:02:14.840 INFO:teuthology.orchestra.run.smithi160.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-05-20T16:02:14.840 INFO:teuthology.orchestra.run.smithi160.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-05-20T16:02:14.840 INFO:teuthology.orchestra.run.smithi160.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-05-20T16:02:14.840 INFO:teuthology.orchestra.run.smithi160.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-05-20T16:02:14.840 INFO:teuthology.orchestra.run.smithi160.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-05-20T16:02:14.840 INFO:teuthology.orchestra.run.smithi160.stdout: python3-pycparser-2.20-6.el9.noarch 2024-05-20T16:02:14.840 INFO:teuthology.orchestra.run.smithi160.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-05-20T16:02:14.840 INFO:teuthology.orchestra.run.smithi160.stdout: python3-pytz-2021.1-5.el9.noarch 2024-05-20T16:02:14.840 INFO:teuthology.orchestra.run.smithi160.stdout: python3-rados-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-20T16:02:14.840 INFO:teuthology.orchestra.run.smithi160.stdout: python3-rbd-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-20T16:02:14.840 INFO:teuthology.orchestra.run.smithi160.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-05-20T16:02:14.840 INFO:teuthology.orchestra.run.smithi160.stdout: python3-requests-2.25.1-8.el9.noarch 2024-05-20T16:02:14.841 INFO:teuthology.orchestra.run.smithi160.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-05-20T16:02:14.841 INFO:teuthology.orchestra.run.smithi160.stdout: python3-rgw-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-20T16:02:14.841 INFO:teuthology.orchestra.run.smithi160.stdout: python3-routes-2.5.1-5.el9.noarch 2024-05-20T16:02:14.841 INFO:teuthology.orchestra.run.smithi160.stdout: python3-rsa-4.9-2.el9.noarch 2024-05-20T16:02:14.841 INFO:teuthology.orchestra.run.smithi160.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-05-20T16:02:14.841 INFO:teuthology.orchestra.run.smithi160.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-05-20T16:02:14.841 INFO:teuthology.orchestra.run.smithi160.stdout: python3-toml-0.10.2-6.el9.noarch 2024-05-20T16:02:14.841 INFO:teuthology.orchestra.run.smithi160.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-05-20T16:02:14.841 INFO:teuthology.orchestra.run.smithi160.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-05-20T16:02:14.841 INFO:teuthology.orchestra.run.smithi160.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-05-20T16:02:14.841 INFO:teuthology.orchestra.run.smithi160.stdout: python3-webob-1.8.7-6.el9.noarch 2024-05-20T16:02:14.841 INFO:teuthology.orchestra.run.smithi160.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-05-20T16:02:14.841 INFO:teuthology.orchestra.run.smithi160.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-05-20T16:02:14.841 INFO:teuthology.orchestra.run.smithi160.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-05-20T16:02:14.841 INFO:teuthology.orchestra.run.smithi160.stdout: qatlib-24.02.0-1.el9.x86_64 2024-05-20T16:02:14.842 INFO:teuthology.orchestra.run.smithi160.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-05-20T16:02:14.842 INFO:teuthology.orchestra.run.smithi160.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-05-20T16:02:14.842 INFO:teuthology.orchestra.run.smithi160.stdout: rbd-fuse-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-20T16:02:14.842 INFO:teuthology.orchestra.run.smithi160.stdout: rbd-mirror-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-20T16:02:14.842 INFO:teuthology.orchestra.run.smithi160.stdout: rbd-nbd-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-20T16:02:14.842 INFO:teuthology.orchestra.run.smithi160.stdout: re2-1:20211101-3.el9.x86_64 2024-05-20T16:02:14.842 INFO:teuthology.orchestra.run.smithi160.stdout: socat-1.7.4.1-5.el9.x86_64 2024-05-20T16:02:14.842 INFO:teuthology.orchestra.run.smithi160.stdout: thrift-0.15.0-2.el9.x86_64 2024-05-20T16:02:14.842 INFO:teuthology.orchestra.run.smithi160.stdout: uuid-1.6.2-55.el9.x86_64 2024-05-20T16:02:14.842 INFO:teuthology.orchestra.run.smithi160.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-05-20T16:02:14.842 INFO:teuthology.orchestra.run.smithi160.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-05-20T16:02:14.843 INFO:teuthology.orchestra.run.smithi160.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-05-20T16:02:14.843 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-20T16:02:14.843 INFO:teuthology.orchestra.run.smithi160.stdout:Complete! 2024-05-20T16:02:14.885 INFO:teuthology.orchestra.run.smithi082.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-20T16:02:15.261 DEBUG:teuthology.parallel:result is None 2024-05-20T16:02:20.729 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-20T16:02:20.729 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-2:19.0.0-2744.gd2b507c7.el9.x86_64 1/164 2024-05-20T16:02:20.729 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-base-2:19.0.0-2744.gd2b507c7.el9.x86_64 2/164 2024-05-20T16:02:20.730 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-common-2:19.0.0-2744.gd2b507c7.el9.x86_64 3/164 2024-05-20T16:02:20.730 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-fuse-2:19.0.0-2744.gd2b507c7.el9.x86_64 4/164 2024-05-20T16:02:20.730 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2744.gd2b50 5/164 2024-05-20T16:02:20.730 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-mds-2:19.0.0-2744.gd2b507c7.el9.x86_64 6/164 2024-05-20T16:02:20.730 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-mgr-2:19.0.0-2744.gd2b507c7.el9.x86_64 7/164 2024-05-20T16:02:20.730 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-mon-2:19.0.0-2744.gd2b507c7.el9.x86_64 8/164 2024-05-20T16:02:20.730 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-osd-2:19.0.0-2744.gd2b507c7.el9.x86_64 9/164 2024-05-20T16:02:20.730 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-radosgw-2:19.0.0-2744.gd2b507c7.el9.x86_64 10/164 2024-05-20T16:02:20.730 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-selinux-2:19.0.0-2744.gd2b507c7.el9.x86_64 11/164 2024-05-20T16:02:20.730 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-test-2:19.0.0-2744.gd2b507c7.el9.x86_64 12/164 2024-05-20T16:02:20.730 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : cephfs-mirror-2:19.0.0-2744.gd2b507c7.el9.x86_64 13/164 2024-05-20T16:02:20.731 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : cephfs-shell-2:19.0.0-2744.gd2b507c7.el9.x86_64 14/164 2024-05-20T16:02:20.731 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libcephfs-devel-2:19.0.0-2744.gd2b507c7.el9.x86_ 15/164 2024-05-20T16:02:20.731 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libcephfs2-2:19.0.0-2744.gd2b507c7.el9.x86_64 16/164 2024-05-20T16:02:20.731 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libcephsqlite-2:19.0.0-2744.gd2b507c7.el9.x86_64 17/164 2024-05-20T16:02:20.731 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : librados-devel-2:19.0.0-2744.gd2b507c7.el9.x86_6 18/164 2024-05-20T16:02:20.731 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libradosstriper1-2:19.0.0-2744.gd2b507c7.el9.x86 19/164 2024-05-20T16:02:20.731 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : librgw2-2:19.0.0-2744.gd2b507c7.el9.x86_64 20/164 2024-05-20T16:02:20.731 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2744.gd2b507c7.el 21/164 2024-05-20T16:02:20.731 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-ceph-common-2:19.0.0-2744.gd2b507c7.el9. 22/164 2024-05-20T16:02:20.731 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-cephfs-2:19.0.0-2744.gd2b507c7.el9.x86_6 23/164 2024-05-20T16:02:20.731 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-rados-2:19.0.0-2744.gd2b507c7.el9.x86_64 24/164 2024-05-20T16:02:20.731 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-rbd-2:19.0.0-2744.gd2b507c7.el9.x86_64 25/164 2024-05-20T16:02:20.731 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-rgw-2:19.0.0-2744.gd2b507c7.el9.x86_64 26/164 2024-05-20T16:02:20.731 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : rbd-fuse-2:19.0.0-2744.gd2b507c7.el9.x86_64 27/164 2024-05-20T16:02:20.732 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : rbd-mirror-2:19.0.0-2744.gd2b507c7.el9.x86_64 28/164 2024-05-20T16:02:20.732 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : rbd-nbd-2:19.0.0-2744.gd2b507c7.el9.x86_64 29/164 2024-05-20T16:02:20.744 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2744.gd2b507c7. 30/164 2024-05-20T16:02:20.744 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2744.gd2b507c7.el9.noa 31/164 2024-05-20T16:02:20.744 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2744.gd2b507c7.el9.n 32/164 2024-05-20T16:02:20.744 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2744.gd2b 33/164 2024-05-20T16:02:20.744 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2744.gd2b507c7.el 34/164 2024-05-20T16:02:20.744 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2744.gd2b507c7.el9.noarch 35/164 2024-05-20T16:02:20.744 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2744.gd2b507c7.e 36/164 2024-05-20T16:02:20.744 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-volume-2:19.0.0-2744.gd2b507c7.el9.noarch 37/164 2024-05-20T16:02:20.744 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : cephadm-2:19.0.0-2744.gd2b507c7.el9.noarch 38/164 2024-05-20T16:02:20.744 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : cephfs-top-2:19.0.0-2744.gd2b507c7.el9.noarch 39/164 2024-05-20T16:02:20.745 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/164 2024-05-20T16:02:20.745 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/164 2024-05-20T16:02:20.745 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/164 2024-05-20T16:02:20.745 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-05-20T16:02:20.745 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-05-20T16:02:20.745 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-05-20T16:02:20.745 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-05-20T16:02:20.745 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-05-20T16:02:20.745 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/164 2024-05-20T16:02:20.745 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-05-20T16:02:20.745 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-05-20T16:02:20.745 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-05-20T16:02:20.745 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/164 2024-05-20T16:02:20.745 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-05-20T16:02:20.746 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-05-20T16:02:20.746 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-05-20T16:02:20.746 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-05-20T16:02:20.746 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-05-20T16:02:20.746 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-05-20T16:02:20.746 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-05-20T16:02:20.746 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-05-20T16:02:20.746 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/164 2024-05-20T16:02:20.746 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/164 2024-05-20T16:02:20.746 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-05-20T16:02:20.746 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-05-20T16:02:20.746 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-05-20T16:02:20.747 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/164 2024-05-20T16:02:20.747 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/164 2024-05-20T16:02:20.747 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/164 2024-05-20T16:02:20.747 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/164 2024-05-20T16:02:20.747 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/164 2024-05-20T16:02:20.747 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/164 2024-05-20T16:02:20.747 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/164 2024-05-20T16:02:20.747 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-05-20T16:02:20.747 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-05-20T16:02:20.747 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-05-20T16:02:20.747 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/164 2024-05-20T16:02:20.747 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/164 2024-05-20T16:02:20.747 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/164 2024-05-20T16:02:20.748 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/164 2024-05-20T16:02:20.748 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/164 2024-05-20T16:02:20.748 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/164 2024-05-20T16:02:20.748 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/164 2024-05-20T16:02:20.748 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/164 2024-05-20T16:02:20.748 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/164 2024-05-20T16:02:20.748 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/164 2024-05-20T16:02:20.748 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/164 2024-05-20T16:02:20.748 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/164 2024-05-20T16:02:20.748 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/164 2024-05-20T16:02:20.748 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/164 2024-05-20T16:02:20.748 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/164 2024-05-20T16:02:20.748 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/164 2024-05-20T16:02:20.748 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/164 2024-05-20T16:02:20.749 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/164 2024-05-20T16:02:20.749 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/164 2024-05-20T16:02:20.749 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/164 2024-05-20T16:02:20.749 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/164 2024-05-20T16:02:20.749 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/164 2024-05-20T16:02:20.749 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/164 2024-05-20T16:02:20.749 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/164 2024-05-20T16:02:20.749 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/164 2024-05-20T16:02:20.749 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/164 2024-05-20T16:02:20.749 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/164 2024-05-20T16:02:20.749 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/164 2024-05-20T16:02:20.749 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/164 2024-05-20T16:02:20.749 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/164 2024-05-20T16:02:20.749 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/164 2024-05-20T16:02:20.750 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/164 2024-05-20T16:02:20.750 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 108/164 2024-05-20T16:02:20.750 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 109/164 2024-05-20T16:02:20.750 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/164 2024-05-20T16:02:20.750 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/164 2024-05-20T16:02:20.750 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 112/164 2024-05-20T16:02:20.750 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 113/164 2024-05-20T16:02:20.750 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/164 2024-05-20T16:02:20.750 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/164 2024-05-20T16:02:20.750 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/164 2024-05-20T16:02:20.750 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/164 2024-05-20T16:02:20.750 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/164 2024-05-20T16:02:20.750 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/164 2024-05-20T16:02:20.751 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/164 2024-05-20T16:02:20.751 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/164 2024-05-20T16:02:20.751 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/164 2024-05-20T16:02:20.751 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-05-20T16:02:20.751 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-05-20T16:02:20.751 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-05-20T16:02:20.751 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-05-20T16:02:20.751 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-05-20T16:02:20.751 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-05-20T16:02:20.751 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-05-20T16:02:20.751 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-05-20T16:02:20.751 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-05-20T16:02:20.751 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-05-20T16:02:20.752 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-05-20T16:02:20.752 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-05-20T16:02:20.752 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-05-20T16:02:20.752 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-05-20T16:02:20.752 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-05-20T16:02:20.752 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-05-20T16:02:20.752 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-05-20T16:02:20.752 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-05-20T16:02:20.752 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-05-20T16:02:20.752 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/164 2024-05-20T16:02:20.752 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/164 2024-05-20T16:02:20.752 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-05-20T16:02:20.752 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-05-20T16:02:20.752 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-05-20T16:02:20.753 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/164 2024-05-20T16:02:20.753 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-05-20T16:02:20.753 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : librados2-2:19.0.0-2744.gd2b507c7.el9.x86_64 149/164 2024-05-20T16:02:20.753 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-05-20T16:02:20.753 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : librbd1-2:19.0.0-2744.gd2b507c7.el9.x86_64 151/164 2024-05-20T16:02:20.753 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-05-20T16:02:20.753 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : NetworkManager-1:1.47.90-1.el9.x86_64 153/164 2024-05-20T16:02:20.753 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/164 2024-05-20T16:02:20.753 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : NetworkManager-libnm-1:1.47.90-1.el9.x86_64 155/164 2024-05-20T16:02:20.753 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/164 2024-05-20T16:02:20.753 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : NetworkManager-team-1:1.47.90-1.el9.x86_64 157/164 2024-05-20T16:02:20.753 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/164 2024-05-20T16:02:20.754 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : NetworkManager-tui-1:1.47.90-1.el9.x86_64 159/164 2024-05-20T16:02:20.754 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/164 2024-05-20T16:02:20.754 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 161/164 2024-05-20T16:02:20.754 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 162/164 2024-05-20T16:02:20.754 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 163/164 2024-05-20T16:02:23.519 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-20T16:02:23.519 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-05-20T16:02:23.519 INFO:teuthology.orchestra.run.smithi082.stdout:Upgraded: 2024-05-20T16:02:23.519 INFO:teuthology.orchestra.run.smithi082.stdout: NetworkManager-1:1.47.90-1.el9.x86_64 2024-05-20T16:02:23.519 INFO:teuthology.orchestra.run.smithi082.stdout: NetworkManager-libnm-1:1.47.90-1.el9.x86_64 2024-05-20T16:02:23.519 INFO:teuthology.orchestra.run.smithi082.stdout: NetworkManager-team-1:1.47.90-1.el9.x86_64 2024-05-20T16:02:23.519 INFO:teuthology.orchestra.run.smithi082.stdout: NetworkManager-tui-1:1.47.90-1.el9.x86_64 2024-05-20T16:02:23.519 INFO:teuthology.orchestra.run.smithi082.stdout: librados2-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-20T16:02:23.520 INFO:teuthology.orchestra.run.smithi082.stdout: librbd1-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-20T16:02:23.520 INFO:teuthology.orchestra.run.smithi082.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-05-20T16:02:23.520 INFO:teuthology.orchestra.run.smithi082.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-05-20T16:02:23.520 INFO:teuthology.orchestra.run.smithi082.stdout:Installed: 2024-05-20T16:02:23.520 INFO:teuthology.orchestra.run.smithi082.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-05-20T16:02:23.520 INFO:teuthology.orchestra.run.smithi082.stdout: automake-1.16.2-8.el9.noarch 2024-05-20T16:02:23.520 INFO:teuthology.orchestra.run.smithi082.stdout: bison-3.7.4-5.el9.x86_64 2024-05-20T16:02:23.520 INFO:teuthology.orchestra.run.smithi082.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-05-20T16:02:23.520 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-20T16:02:23.520 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-base-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-20T16:02:23.520 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-common-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-20T16:02:23.520 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-fuse-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-20T16:02:23.520 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-grafana-dashboards-2:19.0.0-2744.gd2b507c7.el9.noarch 2024-05-20T16:02:23.521 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-immutable-object-cache-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-20T16:02:23.521 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mds-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-20T16:02:23.521 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mgr-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-20T16:02:23.521 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mgr-cephadm-2:19.0.0-2744.gd2b507c7.el9.noarch 2024-05-20T16:02:23.521 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mgr-dashboard-2:19.0.0-2744.gd2b507c7.el9.noarch 2024-05-20T16:02:23.521 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2744.gd2b507c7.el9.noarch 2024-05-20T16:02:23.521 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mgr-modules-core-2:19.0.0-2744.gd2b507c7.el9.noarch 2024-05-20T16:02:23.521 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mgr-rook-2:19.0.0-2744.gd2b507c7.el9.noarch 2024-05-20T16:02:23.521 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mon-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-20T16:02:23.521 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-osd-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-20T16:02:23.521 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-prometheus-alerts-2:19.0.0-2744.gd2b507c7.el9.noarch 2024-05-20T16:02:23.521 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-radosgw-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-20T16:02:23.521 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-selinux-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-20T16:02:23.521 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-test-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-20T16:02:23.522 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-volume-2:19.0.0-2744.gd2b507c7.el9.noarch 2024-05-20T16:02:23.522 INFO:teuthology.orchestra.run.smithi082.stdout: cephadm-2:19.0.0-2744.gd2b507c7.el9.noarch 2024-05-20T16:02:23.522 INFO:teuthology.orchestra.run.smithi082.stdout: cephfs-mirror-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-20T16:02:23.522 INFO:teuthology.orchestra.run.smithi082.stdout: cephfs-shell-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-20T16:02:23.522 INFO:teuthology.orchestra.run.smithi082.stdout: cephfs-top-2:19.0.0-2744.gd2b507c7.el9.noarch 2024-05-20T16:02:23.522 INFO:teuthology.orchestra.run.smithi082.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-05-20T16:02:23.522 INFO:teuthology.orchestra.run.smithi082.stdout: flex-2.6.4-9.el9.x86_64 2024-05-20T16:02:23.522 INFO:teuthology.orchestra.run.smithi082.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-05-20T16:02:23.522 INFO:teuthology.orchestra.run.smithi082.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-05-20T16:02:23.522 INFO:teuthology.orchestra.run.smithi082.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-05-20T16:02:23.522 INFO:teuthology.orchestra.run.smithi082.stdout: fuse-2.9.9-15.el9.x86_64 2024-05-20T16:02:23.522 INFO:teuthology.orchestra.run.smithi082.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-05-20T16:02:23.523 INFO:teuthology.orchestra.run.smithi082.stdout: grpc-data-1.46.7-10.el9.noarch 2024-05-20T16:02:23.523 INFO:teuthology.orchestra.run.smithi082.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-05-20T16:02:23.523 INFO:teuthology.orchestra.run.smithi082.stdout: libarrow-9.0.0-11.el9.x86_64 2024-05-20T16:02:23.523 INFO:teuthology.orchestra.run.smithi082.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-05-20T16:02:23.523 INFO:teuthology.orchestra.run.smithi082.stdout: libcephfs-devel-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-20T16:02:23.523 INFO:teuthology.orchestra.run.smithi082.stdout: libcephfs2-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-20T16:02:23.523 INFO:teuthology.orchestra.run.smithi082.stdout: libcephsqlite-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-20T16:02:23.523 INFO:teuthology.orchestra.run.smithi082.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-05-20T16:02:23.523 INFO:teuthology.orchestra.run.smithi082.stdout: liboath-2.6.7-2.el9.x86_64 2024-05-20T16:02:23.523 INFO:teuthology.orchestra.run.smithi082.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-05-20T16:02:23.523 INFO:teuthology.orchestra.run.smithi082.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-05-20T16:02:23.524 INFO:teuthology.orchestra.run.smithi082.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-05-20T16:02:23.524 INFO:teuthology.orchestra.run.smithi082.stdout: librados-devel-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-20T16:02:23.524 INFO:teuthology.orchestra.run.smithi082.stdout: libradosstriper1-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-20T16:02:23.524 INFO:teuthology.orchestra.run.smithi082.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-05-20T16:02:23.524 INFO:teuthology.orchestra.run.smithi082.stdout: librgw2-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-20T16:02:23.524 INFO:teuthology.orchestra.run.smithi082.stdout: libtool-2.4.6-45.el9.x86_64 2024-05-20T16:02:23.524 INFO:teuthology.orchestra.run.smithi082.stdout: libunwind-1.6.2-1.el9.x86_64 2024-05-20T16:02:23.524 INFO:teuthology.orchestra.run.smithi082.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-05-20T16:02:23.524 INFO:teuthology.orchestra.run.smithi082.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-05-20T16:02:23.524 INFO:teuthology.orchestra.run.smithi082.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-05-20T16:02:23.524 INFO:teuthology.orchestra.run.smithi082.stdout: lua-5.4.4-4.el9.x86_64 2024-05-20T16:02:23.524 INFO:teuthology.orchestra.run.smithi082.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-05-20T16:02:23.524 INFO:teuthology.orchestra.run.smithi082.stdout: luarocks-3.9.2-5.el9.noarch 2024-05-20T16:02:23.525 INFO:teuthology.orchestra.run.smithi082.stdout: openblas-0.3.21-2.el9.x86_64 2024-05-20T16:02:23.525 INFO:teuthology.orchestra.run.smithi082.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-05-20T16:02:23.525 INFO:teuthology.orchestra.run.smithi082.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-05-20T16:02:23.525 INFO:teuthology.orchestra.run.smithi082.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-05-20T16:02:23.525 INFO:teuthology.orchestra.run.smithi082.stdout: postgresql-13.14-1.el9.x86_64 2024-05-20T16:02:23.525 INFO:teuthology.orchestra.run.smithi082.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-05-20T16:02:23.525 INFO:teuthology.orchestra.run.smithi082.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-05-20T16:02:23.525 INFO:teuthology.orchestra.run.smithi082.stdout: postgresql-server-13.14-1.el9.x86_64 2024-05-20T16:02:23.525 INFO:teuthology.orchestra.run.smithi082.stdout: protobuf-3.14.0-13.el9.x86_64 2024-05-20T16:02:23.525 INFO:teuthology.orchestra.run.smithi082.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-05-20T16:02:23.525 INFO:teuthology.orchestra.run.smithi082.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-05-20T16:02:23.525 INFO:teuthology.orchestra.run.smithi082.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-05-20T16:02:23.526 INFO:teuthology.orchestra.run.smithi082.stdout: python3-babel-2.9.1-2.el9.noarch 2024-05-20T16:02:23.526 INFO:teuthology.orchestra.run.smithi082.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-05-20T16:02:23.526 INFO:teuthology.orchestra.run.smithi082.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-05-20T16:02:23.526 INFO:teuthology.orchestra.run.smithi082.stdout: python3-ceph-argparse-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-20T16:02:23.526 INFO:teuthology.orchestra.run.smithi082.stdout: python3-ceph-common-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-20T16:02:23.526 INFO:teuthology.orchestra.run.smithi082.stdout: python3-cephfs-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-20T16:02:23.526 INFO:teuthology.orchestra.run.smithi082.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-05-20T16:02:23.526 INFO:teuthology.orchestra.run.smithi082.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-05-20T16:02:23.526 INFO:teuthology.orchestra.run.smithi082.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-05-20T16:02:23.526 INFO:teuthology.orchestra.run.smithi082.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-05-20T16:02:23.526 INFO:teuthology.orchestra.run.smithi082.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-05-20T16:02:23.526 INFO:teuthology.orchestra.run.smithi082.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-05-20T16:02:23.527 INFO:teuthology.orchestra.run.smithi082.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-05-20T16:02:23.527 INFO:teuthology.orchestra.run.smithi082.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-05-20T16:02:23.527 INFO:teuthology.orchestra.run.smithi082.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-05-20T16:02:23.527 INFO:teuthology.orchestra.run.smithi082.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-05-20T16:02:23.527 INFO:teuthology.orchestra.run.smithi082.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-05-20T16:02:23.527 INFO:teuthology.orchestra.run.smithi082.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-05-20T16:02:23.527 INFO:teuthology.orchestra.run.smithi082.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-05-20T16:02:23.527 INFO:teuthology.orchestra.run.smithi082.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-05-20T16:02:23.527 INFO:teuthology.orchestra.run.smithi082.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-05-20T16:02:23.527 INFO:teuthology.orchestra.run.smithi082.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-05-20T16:02:23.527 INFO:teuthology.orchestra.run.smithi082.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-05-20T16:02:23.527 INFO:teuthology.orchestra.run.smithi082.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-05-20T16:02:23.527 INFO:teuthology.orchestra.run.smithi082.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-05-20T16:02:23.528 INFO:teuthology.orchestra.run.smithi082.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-05-20T16:02:23.528 INFO:teuthology.orchestra.run.smithi082.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-05-20T16:02:23.528 INFO:teuthology.orchestra.run.smithi082.stdout: python3-mako-1.1.4-6.el9.noarch 2024-05-20T16:02:23.528 INFO:teuthology.orchestra.run.smithi082.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-05-20T16:02:23.528 INFO:teuthology.orchestra.run.smithi082.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-05-20T16:02:23.528 INFO:teuthology.orchestra.run.smithi082.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-05-20T16:02:23.528 INFO:teuthology.orchestra.run.smithi082.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-05-20T16:02:23.528 INFO:teuthology.orchestra.run.smithi082.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-05-20T16:02:23.528 INFO:teuthology.orchestra.run.smithi082.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-05-20T16:02:23.528 INFO:teuthology.orchestra.run.smithi082.stdout: python3-packaging-20.9-5.el9.noarch 2024-05-20T16:02:23.528 INFO:teuthology.orchestra.run.smithi082.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-05-20T16:02:23.528 INFO:teuthology.orchestra.run.smithi082.stdout: python3-ply-3.11-14.el9.noarch 2024-05-20T16:02:23.529 INFO:teuthology.orchestra.run.smithi082.stdout: python3-portend-3.1.0-2.el9.noarch 2024-05-20T16:02:23.529 INFO:teuthology.orchestra.run.smithi082.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-05-20T16:02:23.529 INFO:teuthology.orchestra.run.smithi082.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-05-20T16:02:23.529 INFO:teuthology.orchestra.run.smithi082.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-05-20T16:02:23.529 INFO:teuthology.orchestra.run.smithi082.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-05-20T16:02:23.529 INFO:teuthology.orchestra.run.smithi082.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-05-20T16:02:23.529 INFO:teuthology.orchestra.run.smithi082.stdout: python3-pycparser-2.20-6.el9.noarch 2024-05-20T16:02:23.529 INFO:teuthology.orchestra.run.smithi082.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-05-20T16:02:23.529 INFO:teuthology.orchestra.run.smithi082.stdout: python3-pytz-2021.1-5.el9.noarch 2024-05-20T16:02:23.529 INFO:teuthology.orchestra.run.smithi082.stdout: python3-rados-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-20T16:02:23.530 INFO:teuthology.orchestra.run.smithi082.stdout: python3-rbd-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-20T16:02:23.530 INFO:teuthology.orchestra.run.smithi082.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-05-20T16:02:23.530 INFO:teuthology.orchestra.run.smithi082.stdout: python3-requests-2.25.1-8.el9.noarch 2024-05-20T16:02:23.530 INFO:teuthology.orchestra.run.smithi082.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-05-20T16:02:23.530 INFO:teuthology.orchestra.run.smithi082.stdout: python3-rgw-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-20T16:02:23.530 INFO:teuthology.orchestra.run.smithi082.stdout: python3-routes-2.5.1-5.el9.noarch 2024-05-20T16:02:23.530 INFO:teuthology.orchestra.run.smithi082.stdout: python3-rsa-4.9-2.el9.noarch 2024-05-20T16:02:23.530 INFO:teuthology.orchestra.run.smithi082.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-05-20T16:02:23.530 INFO:teuthology.orchestra.run.smithi082.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-05-20T16:02:23.530 INFO:teuthology.orchestra.run.smithi082.stdout: python3-toml-0.10.2-6.el9.noarch 2024-05-20T16:02:23.530 INFO:teuthology.orchestra.run.smithi082.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-05-20T16:02:23.530 INFO:teuthology.orchestra.run.smithi082.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-05-20T16:02:23.531 INFO:teuthology.orchestra.run.smithi082.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-05-20T16:02:23.531 INFO:teuthology.orchestra.run.smithi082.stdout: python3-webob-1.8.7-6.el9.noarch 2024-05-20T16:02:23.531 INFO:teuthology.orchestra.run.smithi082.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-05-20T16:02:23.531 INFO:teuthology.orchestra.run.smithi082.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-05-20T16:02:23.531 INFO:teuthology.orchestra.run.smithi082.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-05-20T16:02:23.531 INFO:teuthology.orchestra.run.smithi082.stdout: qatlib-24.02.0-1.el9.x86_64 2024-05-20T16:02:23.531 INFO:teuthology.orchestra.run.smithi082.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-05-20T16:02:23.531 INFO:teuthology.orchestra.run.smithi082.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-05-20T16:02:23.531 INFO:teuthology.orchestra.run.smithi082.stdout: rbd-fuse-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-20T16:02:23.531 INFO:teuthology.orchestra.run.smithi082.stdout: rbd-mirror-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-20T16:02:23.531 INFO:teuthology.orchestra.run.smithi082.stdout: rbd-nbd-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-20T16:02:23.531 INFO:teuthology.orchestra.run.smithi082.stdout: re2-1:20211101-3.el9.x86_64 2024-05-20T16:02:23.531 INFO:teuthology.orchestra.run.smithi082.stdout: socat-1.7.4.1-5.el9.x86_64 2024-05-20T16:02:23.531 INFO:teuthology.orchestra.run.smithi082.stdout: thrift-0.15.0-2.el9.x86_64 2024-05-20T16:02:23.532 INFO:teuthology.orchestra.run.smithi082.stdout: uuid-1.6.2-55.el9.x86_64 2024-05-20T16:02:23.532 INFO:teuthology.orchestra.run.smithi082.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-05-20T16:02:23.532 INFO:teuthology.orchestra.run.smithi082.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-05-20T16:02:23.532 INFO:teuthology.orchestra.run.smithi082.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-05-20T16:02:23.532 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-05-20T16:02:23.532 INFO:teuthology.orchestra.run.smithi082.stdout:Complete! 2024-05-20T16:02:23.928 DEBUG:teuthology.parallel:result is None 2024-05-20T16:02:23.928 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d2b507c724ef6c8295ee0a35d185cdab167fd61d 2024-05-20T16:02:24.186 DEBUG:teuthology.orchestra.run.smithi082:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-20T16:02:24.210 INFO:teuthology.orchestra.run.smithi082.stdout:19.0.0-2744.gd2b507c7.el9 2024-05-20T16:02:24.210 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2744.gd2b507c7.el9 2024-05-20T16:02:24.210 INFO:teuthology.task.install:The correct ceph version 19.0.0-2744.gd2b507c7 is installed. 2024-05-20T16:02:24.212 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d2b507c724ef6c8295ee0a35d185cdab167fd61d 2024-05-20T16:02:24.459 DEBUG:teuthology.orchestra.run.smithi160:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-20T16:02:24.485 INFO:teuthology.orchestra.run.smithi160.stdout:19.0.0-2744.gd2b507c7.el9 2024-05-20T16:02:24.485 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2744.gd2b507c7.el9 2024-05-20T16:02:24.485 INFO:teuthology.task.install:The correct ceph version 19.0.0-2744.gd2b507c7 is installed. 2024-05-20T16:02:24.487 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d2b507c724ef6c8295ee0a35d185cdab167fd61d 2024-05-20T16:02:24.743 DEBUG:teuthology.orchestra.run.smithi184:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-20T16:02:24.768 INFO:teuthology.orchestra.run.smithi184.stdout:19.0.0-2744.gd2b507c7.el9 2024-05-20T16:02:24.768 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2744.gd2b507c7.el9 2024-05-20T16:02:24.768 INFO:teuthology.task.install:The correct ceph version 19.0.0-2744.gd2b507c7 is installed. 2024-05-20T16:02:24.770 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-05-20T16:02:24.770 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-05-20T16:02:24.770 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-20T16:02:24.802 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2024-05-20T16:02:24.802 DEBUG:teuthology.orchestra.run.smithi160:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-20T16:02:24.833 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-05-20T16:02:24.833 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-20T16:02:24.862 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-05-20T16:02:24.863 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-05-20T16:02:24.863 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd of=/usr/bin/daemon-helper 2024-05-20T16:02:24.892 DEBUG:teuthology.orchestra.run.smithi082:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-20T16:02:24.959 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2024-05-20T16:02:24.959 DEBUG:teuthology.orchestra.run.smithi160:> sudo dd of=/usr/bin/daemon-helper 2024-05-20T16:02:24.987 DEBUG:teuthology.orchestra.run.smithi160:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-20T16:02:25.054 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-05-20T16:02:25.054 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd of=/usr/bin/daemon-helper 2024-05-20T16:02:25.081 DEBUG:teuthology.orchestra.run.smithi184:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-20T16:02:25.149 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-05-20T16:02:25.149 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-05-20T16:02:25.149 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-20T16:02:25.178 DEBUG:teuthology.orchestra.run.smithi082:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-20T16:02:25.245 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2024-05-20T16:02:25.245 DEBUG:teuthology.orchestra.run.smithi160:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-20T16:02:25.273 DEBUG:teuthology.orchestra.run.smithi160:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-20T16:02:25.340 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-05-20T16:02:25.340 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-20T16:02:25.368 DEBUG:teuthology.orchestra.run.smithi184:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-20T16:02:25.435 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-05-20T16:02:25.436 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-05-20T16:02:25.436 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd of=/usr/bin/stdin-killer 2024-05-20T16:02:25.463 DEBUG:teuthology.orchestra.run.smithi082:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-20T16:02:25.529 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2024-05-20T16:02:25.529 DEBUG:teuthology.orchestra.run.smithi160:> sudo dd of=/usr/bin/stdin-killer 2024-05-20T16:02:25.557 DEBUG:teuthology.orchestra.run.smithi160:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-20T16:02:25.624 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-05-20T16:02:25.624 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd of=/usr/bin/stdin-killer 2024-05-20T16:02:25.652 DEBUG:teuthology.orchestra.run.smithi184:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-20T16:02:25.721 INFO:teuthology.run_tasks:Running task cephadm... 2024-05-20T16:02:25.812 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 3, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_export_ephemeral_random_max': 0.1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability'], 'sha1': 'd2b507c724ef6c8295ee0a35d185cdab167fd61d', 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated'}} 2024-05-20T16:02:25.813 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d 2024-05-20T16:02:25.813 INFO:tasks.cephadm:Cluster fsid is 59d1600a-16c2-11ef-bc9a-c7b262605968 2024-05-20T16:02:25.813 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-05-20T16:02:25.814 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.82', 'mon.b': '172.21.15.160', 'mon.c': '172.21.15.184'} 2024-05-20T16:02:25.814 INFO:tasks.cephadm:First mon is mon.a on smithi082 2024-05-20T16:02:25.814 INFO:tasks.cephadm:First mgr is x 2024-05-20T16:02:25.814 INFO:tasks.cephadm:Normalizing hostnames... 2024-05-20T16:02:25.814 DEBUG:teuthology.orchestra.run.smithi082:> sudo hostname $(hostname -s) 2024-05-20T16:02:25.842 DEBUG:teuthology.orchestra.run.smithi160:> sudo hostname $(hostname -s) 2024-05-20T16:02:25.870 DEBUG:teuthology.orchestra.run.smithi184:> sudo hostname $(hostname -s) 2024-05-20T16:02:25.899 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-05-20T16:02:25.899 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d2b507c724ef6c8295ee0a35d185cdab167fd61d 2024-05-20T16:02:26.078 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': 'd2b507c724ef6c8295ee0a35d185cdab167fd61d', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/79503/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-2744-gd2b507c7', 'node_name': '172.21.5.39+adami09', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-2744.gd2b507c7'}, 'url': 'https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240517.203521-squid-debug/d2b507c724ef6c8295ee0a35d185cdab167fd61d/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-05-17 22:05:47.367530', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-pdonnell-testing-20240517.203521-squid-debug', 'chacra_url': 'https://3.chacra.ceph.com/repos/ceph/wip-pdonnell-testing-20240517.203521-squid-debug/d2b507c724ef6c8295ee0a35d185cdab167fd61d/centos/9/flavors/default/', 'archs': ['x86_64', 'source'], 'distro': 'centos'}] 2024-05-20T16:02:26.264 INFO:tasks.util.chacra:got chacra host 3.chacra.ceph.com, ref wip-pdonnell-testing-20240517.203521-squid-debug, sha1 d2b507c724ef6c8295ee0a35d185cdab167fd61d from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=d2b507c724ef6c8295ee0a35d185cdab167fd61d 2024-05-20T16:02:26.265 INFO:tasks.cephadm:Discovered cachra url: https://3.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240517.203521-squid-debug/d2b507c724ef6c8295ee0a35d185cdab167fd61d/centos/9/x86_64/flavors/default/cephadm 2024-05-20T16:02:26.266 DEBUG:teuthology.orchestra.run.smithi082:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240517.203521-squid-debug/d2b507c724ef6c8295ee0a35d185cdab167fd61d/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-05-20T16:02:27.286 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r--. 1 ubuntu ubuntu 782070 May 20 16:02 /home/ubuntu/cephtest/cephadm 2024-05-20T16:02:27.286 DEBUG:teuthology.orchestra.run.smithi160:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240517.203521-squid-debug/d2b507c724ef6c8295ee0a35d185cdab167fd61d/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-05-20T16:02:28.390 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r--. 1 ubuntu ubuntu 782070 May 20 16:02 /home/ubuntu/cephtest/cephadm 2024-05-20T16:02:28.391 DEBUG:teuthology.orchestra.run.smithi184:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240517.203521-squid-debug/d2b507c724ef6c8295ee0a35d185cdab167fd61d/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-05-20T16:02:29.308 INFO:teuthology.orchestra.run.smithi184.stdout:-rw-r--r--. 1 ubuntu ubuntu 782070 May 20 16:02 /home/ubuntu/cephtest/cephadm 2024-05-20T16:02:29.308 DEBUG:teuthology.orchestra.run.smithi082:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-05-20T16:02:29.328 DEBUG:teuthology.orchestra.run.smithi160:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-05-20T16:02:29.347 DEBUG:teuthology.orchestra.run.smithi184:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-05-20T16:02:29.381 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d on all hosts... 2024-05-20T16:02:29.382 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d pull 2024-05-20T16:02:29.384 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d pull 2024-05-20T16:02:29.389 DEBUG:teuthology.orchestra.run.smithi184:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d pull 2024-05-20T16:02:30.188 INFO:teuthology.orchestra.run.smithi082.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d... 2024-05-20T16:02:30.436 INFO:teuthology.orchestra.run.smithi160.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d... 2024-05-20T16:02:30.549 INFO:teuthology.orchestra.run.smithi184.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d... 2024-05-20T16:03:03.850 INFO:teuthology.orchestra.run.smithi160.stdout:{ 2024-05-20T16:03:03.851 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph_version": "ceph version 19.0.0-2744-gd2b507c7 (d2b507c724ef6c8295ee0a35d185cdab167fd61d) squid (dev)", 2024-05-20T16:03:03.851 INFO:teuthology.orchestra.run.smithi160.stdout: "image_id": "cfcc61b2d476f611479310eb042aa05f11dd5f60dbb65e3b0d7e0e11d024cdb3", 2024-05-20T16:03:03.851 INFO:teuthology.orchestra.run.smithi160.stdout: "repo_digests": [ 2024-05-20T16:03:03.851 INFO:teuthology.orchestra.run.smithi160.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708" 2024-05-20T16:03:03.851 INFO:teuthology.orchestra.run.smithi160.stdout: ] 2024-05-20T16:03:03.851 INFO:teuthology.orchestra.run.smithi160.stdout:} 2024-05-20T16:03:28.723 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-05-20T16:03:28.723 INFO:teuthology.orchestra.run.smithi184.stdout: "ceph_version": "ceph version 19.0.0-2744-gd2b507c7 (d2b507c724ef6c8295ee0a35d185cdab167fd61d) squid (dev)", 2024-05-20T16:03:28.723 INFO:teuthology.orchestra.run.smithi184.stdout: "image_id": "cfcc61b2d476f611479310eb042aa05f11dd5f60dbb65e3b0d7e0e11d024cdb3", 2024-05-20T16:03:28.723 INFO:teuthology.orchestra.run.smithi184.stdout: "repo_digests": [ 2024-05-20T16:03:28.723 INFO:teuthology.orchestra.run.smithi184.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708" 2024-05-20T16:03:28.723 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-05-20T16:03:28.723 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-05-20T16:03:37.784 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-05-20T16:03:37.784 INFO:teuthology.orchestra.run.smithi082.stdout: "ceph_version": "ceph version 19.0.0-2744-gd2b507c7 (d2b507c724ef6c8295ee0a35d185cdab167fd61d) squid (dev)", 2024-05-20T16:03:37.784 INFO:teuthology.orchestra.run.smithi082.stdout: "image_id": "cfcc61b2d476f611479310eb042aa05f11dd5f60dbb65e3b0d7e0e11d024cdb3", 2024-05-20T16:03:37.784 INFO:teuthology.orchestra.run.smithi082.stdout: "repo_digests": [ 2024-05-20T16:03:37.784 INFO:teuthology.orchestra.run.smithi082.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708" 2024-05-20T16:03:37.784 INFO:teuthology.orchestra.run.smithi082.stdout: ] 2024-05-20T16:03:37.784 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-05-20T16:03:37.809 DEBUG:teuthology.orchestra.run.smithi082:> sudo mkdir -p /etc/ceph 2024-05-20T16:03:37.869 DEBUG:teuthology.orchestra.run.smithi160:> sudo mkdir -p /etc/ceph 2024-05-20T16:03:37.902 DEBUG:teuthology.orchestra.run.smithi184:> sudo mkdir -p /etc/ceph 2024-05-20T16:03:37.935 DEBUG:teuthology.orchestra.run.smithi082:> sudo chmod 777 /etc/ceph 2024-05-20T16:03:37.960 DEBUG:teuthology.orchestra.run.smithi160:> sudo chmod 777 /etc/ceph 2024-05-20T16:03:37.987 DEBUG:teuthology.orchestra.run.smithi184:> sudo chmod 777 /etc/ceph 2024-05-20T16:03:38.015 INFO:tasks.cephadm:Writing seed config... 2024-05-20T16:03:38.016 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-05-20T16:03:38.016 INFO:tasks.cephadm: override: [client] debug client = 20 2024-05-20T16:03:38.016 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-05-20T16:03:38.016 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-05-20T16:03:38.016 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-05-20T16:03:38.016 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-05-20T16:03:38.016 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-05-20T16:03:38.016 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-05-20T16:03:38.016 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-05-20T16:03:38.016 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-05-20T16:03:38.016 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-05-20T16:03:38.016 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-05-20T16:03:38.017 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-05-20T16:03:38.017 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-05-20T16:03:38.017 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-05-20T16:03:38.017 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-05-20T16:03:38.017 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-05-20T16:03:38.017 INFO:tasks.cephadm: override: [mds] mds_export_ephemeral_random_max = 0.1 2024-05-20T16:03:38.017 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-05-20T16:03:38.017 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-05-20T16:03:38.017 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-05-20T16:03:38.017 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-05-20T16:03:38.017 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-05-20T16:03:38.017 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-05-20T16:03:38.017 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-05-20T16:03:38.017 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-05-20T16:03:38.017 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-05-20T16:03:38.018 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-05-20T16:03:38.018 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-05-20T16:03:38.018 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-05-20T16:03:38.018 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-05-20T16:03:38.018 INFO:tasks.cephadm: override: [osd] bluestore compression mode = aggressive 2024-05-20T16:03:38.018 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-05-20T16:03:38.018 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-05-20T16:03:38.018 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-05-20T16:03:38.018 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-05-20T16:03:38.018 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-05-20T16:03:38.018 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-05-20T16:03:38.018 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-05-20T16:03:38.018 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-05-20T16:03:38.018 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-05-20T16:03:38.019 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-05-20T16:03:38.019 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-05-20T16:03:38.019 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-05-20T16:03:38.019 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-05-20T16:03:38.019 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10 2024-05-20T16:03:38.020 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-05-20T16:03:38.020 DEBUG:teuthology.orchestra.run.smithi082:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-05-20T16:03:38.035 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 59d1600a-16c2-11ef-bc9a-c7b262605968 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore compression mode = aggressive bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True mds_export_ephemeral_random_max = 0.1 osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-05-20T16:03:38.035 DEBUG:teuthology.orchestra.run.smithi082:mon.a> sudo journalctl -f -n 0 -u ceph-59d1600a-16c2-11ef-bc9a-c7b262605968@mon.a.service 2024-05-20T16:03:38.078 DEBUG:teuthology.orchestra.run.smithi082:mgr.x> sudo journalctl -f -n 0 -u ceph-59d1600a-16c2-11ef-bc9a-c7b262605968@mgr.x.service 2024-05-20T16:03:38.121 INFO:tasks.cephadm:Bootstrapping... 2024-05-20T16:03:38.121 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d -v bootstrap --fsid 59d1600a-16c2-11ef-bc9a-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.82 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-05-20T16:03:38.311 INFO:teuthology.orchestra.run.smithi082.stdout:-------------------------------------------------------------------------------- 2024-05-20T16:03:38.311 INFO:teuthology.orchestra.run.smithi082.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d', '-v', 'bootstrap', '--fsid', '59d1600a-16c2-11ef-bc9a-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.82', '--skip-admin-label'] 2024-05-20T16:03:38.311 INFO:teuthology.orchestra.run.smithi082.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-05-20T16:03:38.312 INFO:teuthology.orchestra.run.smithi082.stdout:Verifying podman|docker is present... 2024-05-20T16:03:38.336 INFO:teuthology.orchestra.run.smithi082.stdout:/bin/podman: stdout 5.0.2 2024-05-20T16:03:38.336 INFO:teuthology.orchestra.run.smithi082.stdout:Verifying lvm2 is present... 2024-05-20T16:03:38.336 INFO:teuthology.orchestra.run.smithi082.stdout:Verifying time synchronization is in place... 2024-05-20T16:03:38.344 INFO:teuthology.orchestra.run.smithi082.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-05-20T16:03:38.344 INFO:teuthology.orchestra.run.smithi082.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-05-20T16:03:38.350 INFO:teuthology.orchestra.run.smithi082.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-05-20T16:03:38.350 INFO:teuthology.orchestra.run.smithi082.stdout:systemctl: stdout inactive 2024-05-20T16:03:38.357 INFO:teuthology.orchestra.run.smithi082.stdout:systemctl: stdout enabled 2024-05-20T16:03:38.364 INFO:teuthology.orchestra.run.smithi082.stdout:systemctl: stdout active 2024-05-20T16:03:38.364 INFO:teuthology.orchestra.run.smithi082.stdout:Unit chronyd.service is enabled and running 2024-05-20T16:03:38.364 INFO:teuthology.orchestra.run.smithi082.stdout:Repeating the final host check... 2024-05-20T16:03:38.386 INFO:teuthology.orchestra.run.smithi082.stdout:/bin/podman: stdout 5.0.2 2024-05-20T16:03:38.387 INFO:teuthology.orchestra.run.smithi082.stdout:podman (/bin/podman) version 5.0.2 is present 2024-05-20T16:03:38.387 INFO:teuthology.orchestra.run.smithi082.stdout:systemctl is present 2024-05-20T16:03:38.387 INFO:teuthology.orchestra.run.smithi082.stdout:lvcreate is present 2024-05-20T16:03:38.393 INFO:teuthology.orchestra.run.smithi082.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-05-20T16:03:38.394 INFO:teuthology.orchestra.run.smithi082.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-05-20T16:03:38.400 INFO:teuthology.orchestra.run.smithi082.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-05-20T16:03:38.400 INFO:teuthology.orchestra.run.smithi082.stdout:systemctl: stdout inactive 2024-05-20T16:03:38.406 INFO:teuthology.orchestra.run.smithi082.stdout:systemctl: stdout enabled 2024-05-20T16:03:38.413 INFO:teuthology.orchestra.run.smithi082.stdout:systemctl: stdout active 2024-05-20T16:03:38.413 INFO:teuthology.orchestra.run.smithi082.stdout:Unit chronyd.service is enabled and running 2024-05-20T16:03:38.413 INFO:teuthology.orchestra.run.smithi082.stdout:Host looks OK 2024-05-20T16:03:38.413 INFO:teuthology.orchestra.run.smithi082.stdout:Cluster fsid: 59d1600a-16c2-11ef-bc9a-c7b262605968 2024-05-20T16:03:38.413 INFO:teuthology.orchestra.run.smithi082.stdout:Acquiring lock 139655440065920 on /run/cephadm/59d1600a-16c2-11ef-bc9a-c7b262605968.lock 2024-05-20T16:03:38.413 INFO:teuthology.orchestra.run.smithi082.stdout:Lock 139655440065920 acquired on /run/cephadm/59d1600a-16c2-11ef-bc9a-c7b262605968.lock 2024-05-20T16:03:38.414 INFO:teuthology.orchestra.run.smithi082.stdout:Verifying IP 172.21.15.82 port 3300 ... 2024-05-20T16:03:38.414 INFO:teuthology.orchestra.run.smithi082.stdout:Verifying IP 172.21.15.82 port 6789 ... 2024-05-20T16:03:38.414 INFO:teuthology.orchestra.run.smithi082.stdout:Base mon IP(s) is [172.21.15.82:3300, 172.21.15.82:6789], mon addrv is [v2:172.21.15.82:3300,v1:172.21.15.82:6789] 2024-05-20T16:03:38.417 INFO:teuthology.orchestra.run.smithi082.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.82 metric 100 2024-05-20T16:03:38.417 INFO:teuthology.orchestra.run.smithi082.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.82 metric 100 2024-05-20T16:03:38.420 INFO:teuthology.orchestra.run.smithi082.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-05-20T16:03:38.420 INFO:teuthology.orchestra.run.smithi082.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 256 pref medium 2024-05-20T16:03:38.420 INFO:teuthology.orchestra.run.smithi082.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 1024 expires 1798sec hoplimit 64 pref medium 2024-05-20T16:03:38.423 INFO:teuthology.orchestra.run.smithi082.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-05-20T16:03:38.423 INFO:teuthology.orchestra.run.smithi082.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-05-20T16:03:38.423 INFO:teuthology.orchestra.run.smithi082.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-05-20T16:03:38.423 INFO:teuthology.orchestra.run.smithi082.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-05-20T16:03:38.423 INFO:teuthology.orchestra.run.smithi082.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:fe8f:d3ff/64 scope link 2024-05-20T16:03:38.423 INFO:teuthology.orchestra.run.smithi082.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-05-20T16:03:38.423 INFO:teuthology.orchestra.run.smithi082.stdout:Mon IP `172.21.15.82` is in CIDR network `172.21.0.0/20` 2024-05-20T16:03:38.423 INFO:teuthology.orchestra.run.smithi082.stdout:Mon IP `172.21.15.82` is in CIDR network `172.21.0.0/20` 2024-05-20T16:03:38.424 INFO:teuthology.orchestra.run.smithi082.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-05-20T16:03:38.424 INFO:teuthology.orchestra.run.smithi082.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-05-20T16:03:38.425 INFO:teuthology.orchestra.run.smithi082.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d... 2024-05-20T16:03:38.886 INFO:teuthology.orchestra.run.smithi082.stdout:/bin/podman: stdout cfcc61b2d476f611479310eb042aa05f11dd5f60dbb65e3b0d7e0e11d024cdb3 2024-05-20T16:03:38.887 INFO:teuthology.orchestra.run.smithi082.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d... 2024-05-20T16:03:38.887 INFO:teuthology.orchestra.run.smithi082.stdout:/bin/podman: stderr Getting image source signatures 2024-05-20T16:03:38.887 INFO:teuthology.orchestra.run.smithi082.stdout:/bin/podman: stderr Copying blob sha256:506d33f63dccdaac2c87f708e3d3ef98b8f2fc00666d50b72896243431ca9bae 2024-05-20T16:03:38.887 INFO:teuthology.orchestra.run.smithi082.stdout:/bin/podman: stderr Copying blob sha256:d0e808cd455827ca3108db1b0125dcee9ffc148daa1d7dde963f550bc831d501 2024-05-20T16:03:38.887 INFO:teuthology.orchestra.run.smithi082.stdout:/bin/podman: stderr Copying config sha256:cfcc61b2d476f611479310eb042aa05f11dd5f60dbb65e3b0d7e0e11d024cdb3 2024-05-20T16:03:38.887 INFO:teuthology.orchestra.run.smithi082.stdout:/bin/podman: stderr Writing manifest to image destination 2024-05-20T16:03:39.816 INFO:teuthology.orchestra.run.smithi082.stdout:ceph: stdout ceph version 19.0.0-2744-gd2b507c7 (d2b507c724ef6c8295ee0a35d185cdab167fd61d) squid (dev) 2024-05-20T16:03:39.816 INFO:teuthology.orchestra.run.smithi082.stdout:Ceph version: ceph version 19.0.0-2744-gd2b507c7 (d2b507c724ef6c8295ee0a35d185cdab167fd61d) squid (dev) 2024-05-20T16:03:39.816 INFO:teuthology.orchestra.run.smithi082.stdout:Extracting ceph user uid/gid from container image... 2024-05-20T16:03:40.779 INFO:teuthology.orchestra.run.smithi082.stdout:stat: stdout 167 167 2024-05-20T16:03:40.780 INFO:teuthology.orchestra.run.smithi082.stdout:Creating initial keys... 2024-05-20T16:03:41.652 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph-authtool: stdout AQBddEtmyEroAxAAUld+JoF9daYSyxsE1B1YMw== 2024-05-20T16:03:42.483 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph-authtool: stdout AQBddEtmGWBgNxAAsMCQO9vQUp2oyI2Gx3DQpQ== 2024-05-20T16:03:43.330 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph-authtool: stdout AQBedEtmHejFMBAAvq0zPS7xRR5OFaSPEwrj5Q== 2024-05-20T16:03:43.331 INFO:teuthology.orchestra.run.smithi082.stdout:Creating initial monmap... 2024-05-20T16:03:44.170 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-05-20T16:03:44.170 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-05-20T16:03:44.170 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 59d1600a-16c2-11ef-bc9a-c7b262605968 2024-05-20T16:03:44.170 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-05-20T16:03:44.170 INFO:teuthology.orchestra.run.smithi082.stdout:monmaptool for a [v2:172.21.15.82:3300,v1:172.21.15.82:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-05-20T16:03:44.171 INFO:teuthology.orchestra.run.smithi082.stdout:setting min_mon_release = quincy 2024-05-20T16:03:44.171 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/monmaptool: set fsid to 59d1600a-16c2-11ef-bc9a-c7b262605968 2024-05-20T16:03:44.171 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-05-20T16:03:44.171 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-05-20T16:03:44.171 INFO:teuthology.orchestra.run.smithi082.stdout:Creating mon... 2024-05-20T16:03:45.211 INFO:teuthology.orchestra.run.smithi082.stdout:create mon.a on 2024-05-20T16:03:45.410 INFO:teuthology.orchestra.run.smithi082.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-05-20T16:03:45.550 INFO:teuthology.orchestra.run.smithi082.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-05-20T16:03:45.723 INFO:teuthology.orchestra.run.smithi082.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-59d1600a-16c2-11ef-bc9a-c7b262605968.target → /etc/systemd/system/ceph-59d1600a-16c2-11ef-bc9a-c7b262605968.target. 2024-05-20T16:03:45.724 INFO:teuthology.orchestra.run.smithi082.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-59d1600a-16c2-11ef-bc9a-c7b262605968.target → /etc/systemd/system/ceph-59d1600a-16c2-11ef-bc9a-c7b262605968.target. 2024-05-20T16:03:45.954 INFO:teuthology.orchestra.run.smithi082.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-59d1600a-16c2-11ef-bc9a-c7b262605968@mon.a 2024-05-20T16:03:45.954 INFO:teuthology.orchestra.run.smithi082.stdout:systemctl: stderr Failed to reset failed state of unit ceph-59d1600a-16c2-11ef-bc9a-c7b262605968@mon.a.service: Unit ceph-59d1600a-16c2-11ef-bc9a-c7b262605968@mon.a.service not loaded. 2024-05-20T16:03:46.119 INFO:teuthology.orchestra.run.smithi082.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-59d1600a-16c2-11ef-bc9a-c7b262605968.target.wants/ceph-59d1600a-16c2-11ef-bc9a-c7b262605968@mon.a.service → /etc/systemd/system/ceph-59d1600a-16c2-11ef-bc9a-c7b262605968@.service. 2024-05-20T16:03:46.661 INFO:teuthology.orchestra.run.smithi082.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-20T16:03:46.661 INFO:teuthology.orchestra.run.smithi082.stdout:systemctl: stdout disabled 2024-05-20T16:03:46.669 INFO:teuthology.orchestra.run.smithi082.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-20T16:03:46.669 INFO:teuthology.orchestra.run.smithi082.stdout:systemctl: stdout inactive 2024-05-20T16:03:46.669 INFO:teuthology.orchestra.run.smithi082.stdout:firewalld.service is not enabled 2024-05-20T16:03:46.669 INFO:teuthology.orchestra.run.smithi082.stdout:Not possible to enable service . firewalld.service is not available 2024-05-20T16:03:46.669 INFO:teuthology.orchestra.run.smithi082.stdout:Waiting for mon to start... 2024-05-20T16:03:46.669 INFO:teuthology.orchestra.run.smithi082.stdout:Waiting for mon... 2024-05-20T16:03:47.753 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout cluster: 2024-05-20T16:03:47.754 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout id: 59d1600a-16c2-11ef-bc9a-c7b262605968 2024-05-20T16:03:47.754 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-05-20T16:03:47.754 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout 2024-05-20T16:03:47.754 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout services: 2024-05-20T16:03:47.754 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.596297s) 2024-05-20T16:03:47.754 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-05-20T16:03:47.754 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-05-20T16:03:47.754 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout 2024-05-20T16:03:47.754 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout data: 2024-05-20T16:03:47.754 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-05-20T16:03:47.754 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-05-20T16:03:47.755 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-05-20T16:03:47.755 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout pgs: 2024-05-20T16:03:47.755 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout 2024-05-20T16:03:47.755 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:47.126+0000 7f217c818700 1 Processor -- start 2024-05-20T16:03:47.755 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:47.126+0000 7f217c818700 1 -- start start 2024-05-20T16:03:47.755 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:47.126+0000 7f217c818700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f217807bca0 0x7f217810b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:03:47.755 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:47.126+0000 7f217c818700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f217810bbd0 con 0x7f217807bca0 2024-05-20T16:03:47.755 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:47.126+0000 7f217659c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f217807bca0 0x7f217810b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:03:47.755 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:47.126+0000 7f217659c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f217807bca0 0x7f217810b5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:39948/0 (socket says 172.21.15.82:39948) 2024-05-20T16:03:47.755 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:47.126+0000 7f217659c700 1 -- 172.21.15.82:0/651288989 learned_addr learned my addr 172.21.15.82:0/651288989 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:03:47.755 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:47.127+0000 7f217659c700 1 -- 172.21.15.82:0/651288989 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f217810bd50 con 0x7f217807bca0 2024-05-20T16:03:47.755 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:47.127+0000 7f217659c700 1 --2- 172.21.15.82:0/651288989 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f217807bca0 0x7f217810b5e0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f21680098a0 tx=0x7f2168009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99594da0068bb917 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:03:47.755 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:47.127+0000 7f217559a700 1 -- 172.21.15.82:0/651288989 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2168004160 con 0x7f217807bca0 2024-05-20T16:03:47.756 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:47.127+0000 7f217559a700 1 -- 172.21.15.82:0/651288989 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2168004300 con 0x7f217807bca0 2024-05-20T16:03:47.756 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:47.127+0000 7f217559a700 1 -- 172.21.15.82:0/651288989 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f21680036a0 con 0x7f217807bca0 2024-05-20T16:03:47.756 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:47.128+0000 7f217c818700 1 -- 172.21.15.82:0/651288989 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f217807bca0 msgr2=0x7f217810b5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:03:47.756 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:47.128+0000 7f217c818700 1 --2- 172.21.15.82:0/651288989 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f217807bca0 0x7f217810b5e0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f21680098a0 tx=0x7f2168009bb0 comp rx=0 tx=0).stop 2024-05-20T16:03:47.756 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:47.128+0000 7f217c818700 1 -- 172.21.15.82:0/651288989 shutdown_connections 2024-05-20T16:03:47.756 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:47.128+0000 7f217c818700 1 --2- 172.21.15.82:0/651288989 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f217807bca0 0x7f217810b5e0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:03:47.756 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:47.128+0000 7f217c818700 1 -- 172.21.15.82:0/651288989 >> 172.21.15.82:0/651288989 conn(0x7f2178103070 msgr2=0x7f21781054c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:03:47.756 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:47.128+0000 7f217c818700 1 -- 172.21.15.82:0/651288989 shutdown_connections 2024-05-20T16:03:47.756 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:47.128+0000 7f217c818700 1 -- 172.21.15.82:0/651288989 wait complete. 2024-05-20T16:03:47.756 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:47.129+0000 7f217c818700 1 Processor -- start 2024-05-20T16:03:47.756 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:47.129+0000 7f217c818700 1 -- start start 2024-05-20T16:03:47.756 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:47.129+0000 7f217c818700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f217807bca0 0x7f217819c490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:03:47.757 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:47.129+0000 7f217c818700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f217810c280 con 0x7f217807bca0 2024-05-20T16:03:47.757 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:47.130+0000 7f217659c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f217807bca0 0x7f217819c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:03:47.757 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:47.130+0000 7f217659c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f217807bca0 0x7f217819c490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:39962/0 (socket says 172.21.15.82:39962) 2024-05-20T16:03:47.757 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:47.130+0000 7f217659c700 1 -- 172.21.15.82:0/2380714311 learned_addr learned my addr 172.21.15.82:0/2380714311 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:03:47.757 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:47.130+0000 7f217659c700 1 -- 172.21.15.82:0/2380714311 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f217819c9d0 con 0x7f217807bca0 2024-05-20T16:03:47.757 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:47.130+0000 7f217659c700 1 --2- 172.21.15.82:0/2380714311 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f217807bca0 0x7f217819c490 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f2168000c00 tx=0x7f2168004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:03:47.757 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:47.130+0000 7f21677fe700 1 -- 172.21.15.82:0/2380714311 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2168003db0 con 0x7f217807bca0 2024-05-20T16:03:47.757 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:47.130+0000 7f21677fe700 1 -- 172.21.15.82:0/2380714311 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2168011630 con 0x7f217807bca0 2024-05-20T16:03:47.757 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:47.130+0000 7f21677fe700 1 -- 172.21.15.82:0/2380714311 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2168011950 con 0x7f217807bca0 2024-05-20T16:03:47.757 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:47.130+0000 7f217c818700 1 -- 172.21.15.82:0/2380714311 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f217819cc60 con 0x7f217807bca0 2024-05-20T16:03:47.757 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:47.130+0000 7f217c818700 1 -- 172.21.15.82:0/2380714311 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f217819f870 con 0x7f217807bca0 2024-05-20T16:03:47.757 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:47.130+0000 7f21677fe700 1 -- 172.21.15.82:0/2380714311 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f2168011af0 con 0x7f217807bca0 2024-05-20T16:03:47.757 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:47.131+0000 7f21677fe700 1 -- 172.21.15.82:0/2380714311 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f216801c450 con 0x7f217807bca0 2024-05-20T16:03:47.757 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:47.131+0000 7f217c818700 1 -- 172.21.15.82:0/2380714311 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2178063ca0 con 0x7f217807bca0 2024-05-20T16:03:47.758 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:47.134+0000 7f21677fe700 1 -- 172.21.15.82:0/2380714311 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f2168032020 con 0x7f217807bca0 2024-05-20T16:03:47.758 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:47.205+0000 7f217c818700 1 -- 172.21.15.82:0/2380714311 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7f217802df60 con 0x7f217807bca0 2024-05-20T16:03:47.758 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:47.206+0000 7f21677fe700 1 -- 172.21.15.82:0/2380714311 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+317 (secure 0 0 0) 0x7f2168004260 con 0x7f217807bca0 2024-05-20T16:03:47.758 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:47.207+0000 7f217c818700 1 -- 172.21.15.82:0/2380714311 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f217807bca0 msgr2=0x7f217819c490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:03:47.758 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:47.207+0000 7f217c818700 1 --2- 172.21.15.82:0/2380714311 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f217807bca0 0x7f217819c490 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f2168000c00 tx=0x7f2168004750 comp rx=0 tx=0).stop 2024-05-20T16:03:47.758 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:47.207+0000 7f217c818700 1 -- 172.21.15.82:0/2380714311 shutdown_connections 2024-05-20T16:03:47.758 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:47.207+0000 7f217c818700 1 --2- 172.21.15.82:0/2380714311 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f217807bca0 0x7f217819c490 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:03:47.758 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:47.207+0000 7f217c818700 1 -- 172.21.15.82:0/2380714311 >> 172.21.15.82:0/2380714311 conn(0x7f2178103070 msgr2=0x7f2178104d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:03:47.758 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:47.207+0000 7f217c818700 1 -- 172.21.15.82:0/2380714311 shutdown_connections 2024-05-20T16:03:47.758 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:47.207+0000 7f217c818700 1 -- 172.21.15.82:0/2380714311 wait complete. 2024-05-20T16:03:47.758 INFO:teuthology.orchestra.run.smithi082.stdout:mon is available 2024-05-20T16:03:47.758 INFO:teuthology.orchestra.run.smithi082.stdout:Assimilating anything we can from ceph.conf... 2024-05-20T16:03:48.843 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:48 smithi082 ceph-mon[31865]: from='client.? 172.21.15.82:0/38604823' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-05-20T16:03:48.854 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout 2024-05-20T16:03:48.854 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout [global] 2024-05-20T16:03:48.854 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout fsid = 59d1600a-16c2-11ef-bc9a-c7b262605968 2024-05-20T16:03:48.854 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.82:3300,v1:172.21.15.82:6789] 2024-05-20T16:03:48.854 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-05-20T16:03:48.854 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-05-20T16:03:48.854 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-05-20T16:03:48.854 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-05-20T16:03:48.854 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout 2024-05-20T16:03:48.855 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout [mgr] 2024-05-20T16:03:48.855 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-05-20T16:03:48.855 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout 2024-05-20T16:03:48.855 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout [osd] 2024-05-20T16:03:48.855 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-05-20T16:03:48.855 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-05-20T16:03:48.855 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-05-20T16:03:48.855 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-05-20T16:03:48.855 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-05-20T16:03:48.855 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:48.207+0000 7f284fab1700 1 Processor -- start 2024-05-20T16:03:48.855 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:48.208+0000 7f284fab1700 1 -- start start 2024-05-20T16:03:48.855 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:48.208+0000 7f284fab1700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2848108b20 0x7f2848108f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:03:48.855 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:48.208+0000 7f284fab1700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f2848109520 con 0x7f2848108b20 2024-05-20T16:03:48.855 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:48.208+0000 7f284d84d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2848108b20 0x7f2848108f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:03:48.856 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:48.208+0000 7f284d84d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2848108b20 0x7f2848108f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:39976/0 (socket says 172.21.15.82:39976) 2024-05-20T16:03:48.856 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:48.208+0000 7f284d84d700 1 -- 172.21.15.82:0/1795101143 learned_addr learned my addr 172.21.15.82:0/1795101143 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:03:48.856 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:48.208+0000 7f284d84d700 1 -- 172.21.15.82:0/1795101143 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2848109c00 con 0x7f2848108b20 2024-05-20T16:03:48.856 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:48.208+0000 7f284d84d700 1 --2- 172.21.15.82:0/1795101143 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2848108b20 0x7f2848108f30 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f283c0098a0 tx=0x7f283c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba13e2d576d59e46 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:03:48.856 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:48.209+0000 7f284c84b700 1 -- 172.21.15.82:0/1795101143 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f283c004160 con 0x7f2848108b20 2024-05-20T16:03:48.856 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:48.209+0000 7f284c84b700 1 -- 172.21.15.82:0/1795101143 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f283c004300 con 0x7f2848108b20 2024-05-20T16:03:48.856 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:48.209+0000 7f284c84b700 1 -- 172.21.15.82:0/1795101143 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f283c0036a0 con 0x7f2848108b20 2024-05-20T16:03:48.856 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:48.209+0000 7f284fab1700 1 -- 172.21.15.82:0/1795101143 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2848108b20 msgr2=0x7f2848108f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:03:48.856 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:48.209+0000 7f284fab1700 1 --2- 172.21.15.82:0/1795101143 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2848108b20 0x7f2848108f30 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f283c0098a0 tx=0x7f283c009bb0 comp rx=0 tx=0).stop 2024-05-20T16:03:48.856 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:48.209+0000 7f284fab1700 1 -- 172.21.15.82:0/1795101143 shutdown_connections 2024-05-20T16:03:48.856 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:48.209+0000 7f284fab1700 1 --2- 172.21.15.82:0/1795101143 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2848108b20 0x7f2848108f30 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:03:48.856 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:48.209+0000 7f284fab1700 1 -- 172.21.15.82:0/1795101143 >> 172.21.15.82:0/1795101143 conn(0x7f28481041c0 msgr2=0x7f2848106610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:03:48.857 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:48.209+0000 7f284fab1700 1 -- 172.21.15.82:0/1795101143 shutdown_connections 2024-05-20T16:03:48.857 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:48.210+0000 7f284fab1700 1 -- 172.21.15.82:0/1795101143 wait complete. 2024-05-20T16:03:48.857 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:48.210+0000 7f284fab1700 1 Processor -- start 2024-05-20T16:03:48.857 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:48.210+0000 7f284fab1700 1 -- start start 2024-05-20T16:03:48.857 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:48.210+0000 7f284fab1700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2848108b20 0x7f284819c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:03:48.857 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:48.210+0000 7f284fab1700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f284810a130 con 0x7f2848108b20 2024-05-20T16:03:48.857 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:48.211+0000 7f284d84d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2848108b20 0x7f284819c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:03:48.857 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:48.211+0000 7f284d84d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2848108b20 0x7f284819c4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:39988/0 (socket says 172.21.15.82:39988) 2024-05-20T16:03:48.857 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:48.211+0000 7f284d84d700 1 -- 172.21.15.82:0/38604823 learned_addr learned my addr 172.21.15.82:0/38604823 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:03:48.857 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:48.211+0000 7f284d84d700 1 -- 172.21.15.82:0/38604823 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f284819c9f0 con 0x7f2848108b20 2024-05-20T16:03:48.857 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:48.211+0000 7f284d84d700 1 --2- 172.21.15.82:0/38604823 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2848108b20 0x7f284819c4b0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f283c000c00 tx=0x7f283c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:03:48.857 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:48.211+0000 7f283affd700 1 -- 172.21.15.82:0/38604823 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f283c003db0 con 0x7f2848108b20 2024-05-20T16:03:48.857 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:48.211+0000 7f283affd700 1 -- 172.21.15.82:0/38604823 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f283c011630 con 0x7f2848108b20 2024-05-20T16:03:48.857 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:48.211+0000 7f283affd700 1 -- 172.21.15.82:0/38604823 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f283c011950 con 0x7f2848108b20 2024-05-20T16:03:48.857 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:48.211+0000 7f284fab1700 1 -- 172.21.15.82:0/38604823 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f284819cc80 con 0x7f2848108b20 2024-05-20T16:03:48.858 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:48.211+0000 7f284fab1700 1 -- 172.21.15.82:0/38604823 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f284819f890 con 0x7f2848108b20 2024-05-20T16:03:48.858 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:48.211+0000 7f283affd700 1 -- 172.21.15.82:0/38604823 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f283c011af0 con 0x7f2848108b20 2024-05-20T16:03:48.858 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:48.211+0000 7f283affd700 1 -- 172.21.15.82:0/38604823 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f283c01c450 con 0x7f2848108b20 2024-05-20T16:03:48.858 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:48.212+0000 7f284fab1700 1 -- 172.21.15.82:0/38604823 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2848063ca0 con 0x7f2848108b20 2024-05-20T16:03:48.858 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:48.214+0000 7f283affd700 1 -- 172.21.15.82:0/38604823 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f283c032020 con 0x7f2848108b20 2024-05-20T16:03:48.858 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:48.282+0000 7f284fab1700 1 -- 172.21.15.82:0/38604823 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f284802df60 con 0x7f2848108b20 2024-05-20T16:03:48.858 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:48.286+0000 7f283affd700 1 -- 172.21.15.82:0/38604823 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+429 (secure 0 0 0) 0x7f283c004260 con 0x7f2848108b20 2024-05-20T16:03:48.858 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:48.286+0000 7f283affd700 1 -- 172.21.15.82:0/38604823 <== mon.0 v2:172.21.15.82:3300/0 8 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f283c01cb60 con 0x7f2848108b20 2024-05-20T16:03:48.858 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:48.287+0000 7f284fab1700 1 -- 172.21.15.82:0/38604823 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2848108b20 msgr2=0x7f284819c4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:03:48.858 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:48.287+0000 7f284fab1700 1 --2- 172.21.15.82:0/38604823 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2848108b20 0x7f284819c4b0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f283c000c00 tx=0x7f283c004750 comp rx=0 tx=0).stop 2024-05-20T16:03:48.858 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:48.287+0000 7f284fab1700 1 -- 172.21.15.82:0/38604823 shutdown_connections 2024-05-20T16:03:48.858 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:48.287+0000 7f284fab1700 1 --2- 172.21.15.82:0/38604823 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2848108b20 0x7f284819c4b0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:03:48.858 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:48.287+0000 7f284fab1700 1 -- 172.21.15.82:0/38604823 >> 172.21.15.82:0/38604823 conn(0x7f28481041c0 msgr2=0x7f28481904f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:03:48.859 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:48.287+0000 7f284fab1700 1 -- 172.21.15.82:0/38604823 shutdown_connections 2024-05-20T16:03:48.859 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:48.287+0000 7f284fab1700 1 -- 172.21.15.82:0/38604823 wait complete. 2024-05-20T16:03:48.859 INFO:teuthology.orchestra.run.smithi082.stdout:Generating new minimal ceph.conf... 2024-05-20T16:03:49.925 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:49 smithi082 ceph-mon[31865]: from='client.? 172.21.15.82:0/2568993583' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:03:49.931 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:49.297+0000 7fb8359c5700 1 Processor -- start 2024-05-20T16:03:49.932 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:49.297+0000 7fb8359c5700 1 -- start start 2024-05-20T16:03:49.932 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:49.297+0000 7fb8359c5700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb830107990 0x7fb830107da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:03:49.932 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:49.297+0000 7fb8359c5700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fb830108390 con 0x7fb830107990 2024-05-20T16:03:49.932 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:49.297+0000 7fb82effd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb830107990 0x7fb830107da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:03:49.932 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:49.297+0000 7fb82effd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb830107990 0x7fb830107da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:40000/0 (socket says 172.21.15.82:40000) 2024-05-20T16:03:49.932 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:49.297+0000 7fb82effd700 1 -- 172.21.15.82:0/1828463980 learned_addr learned my addr 172.21.15.82:0/1828463980 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:03:49.932 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:49.297+0000 7fb82effd700 1 -- 172.21.15.82:0/1828463980 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb830108ac0 con 0x7fb830107990 2024-05-20T16:03:49.932 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:49.298+0000 7fb82effd700 1 --2- 172.21.15.82:0/1828463980 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb830107990 0x7fb830107da0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fb8200098a0 tx=0x7fb820009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=502081ac4bc21a27 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:03:49.933 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:49.298+0000 7fb82dffb700 1 -- 172.21.15.82:0/1828463980 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb820004160 con 0x7fb830107990 2024-05-20T16:03:49.933 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:49.298+0000 7fb82dffb700 1 -- 172.21.15.82:0/1828463980 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fb820004300 con 0x7fb830107990 2024-05-20T16:03:49.933 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:49.298+0000 7fb82dffb700 1 -- 172.21.15.82:0/1828463980 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb820011b50 con 0x7fb830107990 2024-05-20T16:03:49.933 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:49.298+0000 7fb8359c5700 1 -- 172.21.15.82:0/1828463980 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb830107990 msgr2=0x7fb830107da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:03:49.933 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:49.298+0000 7fb8359c5700 1 --2- 172.21.15.82:0/1828463980 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb830107990 0x7fb830107da0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fb8200098a0 tx=0x7fb820009bb0 comp rx=0 tx=0).stop 2024-05-20T16:03:49.933 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:49.298+0000 7fb8359c5700 1 -- 172.21.15.82:0/1828463980 shutdown_connections 2024-05-20T16:03:49.933 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:49.298+0000 7fb8359c5700 1 --2- 172.21.15.82:0/1828463980 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb830107990 0x7fb830107da0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:03:49.933 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:49.298+0000 7fb8359c5700 1 -- 172.21.15.82:0/1828463980 >> 172.21.15.82:0/1828463980 conn(0x7fb830103070 msgr2=0x7fb8301054c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:03:49.933 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:49.299+0000 7fb8359c5700 1 -- 172.21.15.82:0/1828463980 shutdown_connections 2024-05-20T16:03:49.933 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:49.299+0000 7fb8359c5700 1 -- 172.21.15.82:0/1828463980 wait complete. 2024-05-20T16:03:49.933 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:49.299+0000 7fb8359c5700 1 Processor -- start 2024-05-20T16:03:49.933 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:49.299+0000 7fb8359c5700 1 -- start start 2024-05-20T16:03:49.933 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:49.300+0000 7fb8359c5700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb830107990 0x7fb83019ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:03:49.934 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:49.300+0000 7fb8359c5700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fb83010be70 con 0x7fb830107990 2024-05-20T16:03:49.934 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:49.300+0000 7fb82effd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb830107990 0x7fb83019ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:03:49.934 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:49.300+0000 7fb82effd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb830107990 0x7fb83019ce40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:40010/0 (socket says 172.21.15.82:40010) 2024-05-20T16:03:49.934 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:49.300+0000 7fb82effd700 1 -- 172.21.15.82:0/2568993583 learned_addr learned my addr 172.21.15.82:0/2568993583 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:03:49.934 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:49.300+0000 7fb82effd700 1 -- 172.21.15.82:0/2568993583 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb83019d380 con 0x7fb830107990 2024-05-20T16:03:49.934 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:49.300+0000 7fb82effd700 1 --2- 172.21.15.82:0/2568993583 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb830107990 0x7fb83019ce40 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fb820009fb0 tx=0x7fb820011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:03:49.934 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:49.300+0000 7fb8349c3700 1 -- 172.21.15.82:0/2568993583 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb8200038b0 con 0x7fb830107990 2024-05-20T16:03:49.934 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:49.300+0000 7fb8349c3700 1 -- 172.21.15.82:0/2568993583 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fb820011e50 con 0x7fb830107990 2024-05-20T16:03:49.934 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:49.300+0000 7fb8349c3700 1 -- 172.21.15.82:0/2568993583 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb82001a430 con 0x7fb830107990 2024-05-20T16:03:49.934 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:49.300+0000 7fb8359c5700 1 -- 172.21.15.82:0/2568993583 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb83019d610 con 0x7fb830107990 2024-05-20T16:03:49.935 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:49.300+0000 7fb8359c5700 1 -- 172.21.15.82:0/2568993583 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8301a0220 con 0x7fb830107990 2024-05-20T16:03:49.935 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:49.301+0000 7fb8349c3700 1 -- 172.21.15.82:0/2568993583 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fb82001aa70 con 0x7fb830107990 2024-05-20T16:03:49.935 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:49.301+0000 7fb8349c3700 1 -- 172.21.15.82:0/2568993583 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fb820019700 con 0x7fb830107990 2024-05-20T16:03:49.935 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:49.302+0000 7fb8359c5700 1 -- 172.21.15.82:0/2568993583 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb830063ca0 con 0x7fb830107990 2024-05-20T16:03:49.935 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:49.304+0000 7fb8349c3700 1 -- 172.21.15.82:0/2568993583 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fb82002ddd0 con 0x7fb830107990 2024-05-20T16:03:49.935 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:49.371+0000 7fb8359c5700 1 -- 172.21.15.82:0/2568993583 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7fb83002df60 con 0x7fb830107990 2024-05-20T16:03:49.935 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:49.371+0000 7fb8349c3700 1 -- 172.21.15.82:0/2568993583 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+175 (secure 0 0 0) 0x7fb820032030 con 0x7fb830107990 2024-05-20T16:03:49.935 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:49.372+0000 7fb8359c5700 1 -- 172.21.15.82:0/2568993583 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb830107990 msgr2=0x7fb83019ce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:03:49.935 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:49.372+0000 7fb8359c5700 1 --2- 172.21.15.82:0/2568993583 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb830107990 0x7fb83019ce40 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fb820009fb0 tx=0x7fb820011c10 comp rx=0 tx=0).stop 2024-05-20T16:03:49.935 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:49.372+0000 7fb8359c5700 1 -- 172.21.15.82:0/2568993583 shutdown_connections 2024-05-20T16:03:49.935 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:49.372+0000 7fb8359c5700 1 --2- 172.21.15.82:0/2568993583 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb830107990 0x7fb83019ce40 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:03:49.935 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:49.372+0000 7fb8359c5700 1 -- 172.21.15.82:0/2568993583 >> 172.21.15.82:0/2568993583 conn(0x7fb830103070 msgr2=0x7fb8301933d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:03:49.935 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:49.372+0000 7fb8359c5700 1 -- 172.21.15.82:0/2568993583 shutdown_connections 2024-05-20T16:03:49.936 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:49.372+0000 7fb8359c5700 1 -- 172.21.15.82:0/2568993583 wait complete. 2024-05-20T16:03:49.936 INFO:teuthology.orchestra.run.smithi082.stdout:Restarting the monitor... 2024-05-20T16:03:50.196 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:49 smithi082 systemd[1]: Stopping Ceph mon.a for 59d1600a-16c2-11ef-bc9a-c7b262605968... 2024-05-20T16:03:50.554 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:50 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mon-a[31857]: 2024-05-20T16:03:50.194+0000 7f04c1015700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-20T16:03:50.554 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:50 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mon-a[31857]: 2024-05-20T16:03:50.194+0000 7f04c1015700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-05-20T16:03:50.554 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:50 smithi082 podman[32116]: 2024-05-20 16:03:50.328550556 +0000 UTC m=+0.240173644 container died fbb96e48caa4635340e4c15dd5533c07d2c18d077115145f90ce5c7f7c02f5ff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d, name=ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mon-a, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240514, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-05-20T16:03:50.860 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:50 smithi082 podman[32116]: 2024-05-20 16:03:50.645230673 +0000 UTC m=+0.556853753 container cleanup fbb96e48caa4635340e4c15dd5533c07d2c18d077115145f90ce5c7f7c02f5ff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d, name=ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mon-a, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240514, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c) 2024-05-20T16:03:50.860 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:50 smithi082 bash[32116]: ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mon-a 2024-05-20T16:03:50.860 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:50 smithi082 podman[32131]: 2024-05-20 16:03:50.787837409 +0000 UTC m=+0.462394341 container remove fbb96e48caa4635340e4c15dd5533c07d2c18d077115145f90ce5c7f7c02f5ff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d, name=ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mon-a, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240514, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-05-20T16:03:51.328 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:50 smithi082 systemd[1]: ceph-59d1600a-16c2-11ef-bc9a-c7b262605968@mon.a.service: Deactivated successfully. 2024-05-20T16:03:51.328 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:50 smithi082 systemd[1]: Stopped Ceph mon.a for 59d1600a-16c2-11ef-bc9a-c7b262605968. 2024-05-20T16:03:51.328 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:50 smithi082 systemd[1]: Starting Ceph mon.a for 59d1600a-16c2-11ef-bc9a-c7b262605968... 2024-05-20T16:03:51.328 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 podman[32219]: 2024-05-20 16:03:51.100130129 +0000 UTC m=+0.017131671 image pull cfcc61b2d476f611479310eb042aa05f11dd5f60dbb65e3b0d7e0e11d024cdb3 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d 2024-05-20T16:03:51.329 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 podman[32219]: 2024-05-20 16:03:51.213675075 +0000 UTC m=+0.130676603 container create 1c5b1c87352a8fe3c4399e9c039e6ffd8962413e7ab6afaaa251874aa2fc7fe3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d, name=ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mon-a, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240514, org.label-schema.vendor=CentOS) 2024-05-20T16:03:51.478 INFO:teuthology.orchestra.run.smithi082.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-05-20T16:03:51.603 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 podman[32219]: 2024-05-20 16:03:51.416048651 +0000 UTC m=+0.333050185 container init 1c5b1c87352a8fe3c4399e9c039e6ffd8962413e7ab6afaaa251874aa2fc7fe3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d, name=ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240514, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-05-20T16:03:51.603 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 podman[32219]: 2024-05-20 16:03:51.419171327 +0000 UTC m=+0.336172861 container start 1c5b1c87352a8fe3c4399e9c039e6ffd8962413e7ab6afaaa251874aa2fc7fe3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d, name=ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mon-a, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240514, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-20T16:03:51.603 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: set uid:gid to 167:167 (ceph:ceph) 2024-05-20T16:03:51.603 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: ceph version 19.0.0-2744-gd2b507c7 (d2b507c724ef6c8295ee0a35d185cdab167fd61d) squid (dev), process ceph-mon, pid 2 2024-05-20T16:03:51.603 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: pidfile_write: ignore empty --pid-file 2024-05-20T16:03:51.603 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: load: jerasure load: lrc 2024-05-20T16:03:51.603 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: rocksdb: RocksDB version: 7.9.2 2024-05-20T16:03:51.604 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: rocksdb: Git sha 0 2024-05-20T16:03:51.604 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: rocksdb: Compile date 2024-05-17 21:21:31 2024-05-20T16:03:51.604 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: rocksdb: DB SUMMARY 2024-05-20T16:03:51.604 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: rocksdb: DB Session ID: CWESX2OPB402MWCDZGPJ 2024-05-20T16:03:51.604 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: rocksdb: CURRENT file: CURRENT 2024-05-20T16:03:51.604 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: rocksdb: IDENTITY file: IDENTITY 2024-05-20T16:03:51.604 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-05-20T16:03:51.604 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-05-20T16:03:51.604 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 104854 ; 2024-05-20T16:03:51.604 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: rocksdb: Options.error_if_exists: 0 2024-05-20T16:03:51.604 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: rocksdb: Options.create_if_missing: 0 2024-05-20T16:03:51.604 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: rocksdb: Options.paranoid_checks: 1 2024-05-20T16:03:51.604 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: rocksdb: Options.flush_verify_memtable_count: 1 2024-05-20T16:03:51.605 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-05-20T16:03:51.605 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-05-20T16:03:51.605 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: rocksdb: Options.env: 0x55cdb239c4a0 2024-05-20T16:03:51.605 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: rocksdb: Options.fs: PosixFileSystem 2024-05-20T16:03:51.605 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: rocksdb: Options.info_log: 0x55cdb4721d40 2024-05-20T16:03:51.605 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: rocksdb: Options.max_file_opening_threads: 16 2024-05-20T16:03:51.605 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: rocksdb: Options.statistics: (nil) 2024-05-20T16:03:51.605 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: rocksdb: Options.use_fsync: 0 2024-05-20T16:03:51.605 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: rocksdb: Options.max_log_file_size: 0 2024-05-20T16:03:51.605 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-05-20T16:03:51.605 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: rocksdb: Options.log_file_time_to_roll: 0 2024-05-20T16:03:51.605 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: rocksdb: Options.keep_log_file_num: 1000 2024-05-20T16:03:51.605 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: rocksdb: Options.recycle_log_file_num: 0 2024-05-20T16:03:51.605 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: rocksdb: Options.allow_fallocate: 1 2024-05-20T16:03:51.606 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: rocksdb: Options.allow_mmap_reads: 0 2024-05-20T16:03:51.606 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: rocksdb: Options.allow_mmap_writes: 0 2024-05-20T16:03:51.606 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: rocksdb: Options.use_direct_reads: 0 2024-05-20T16:03:51.606 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-05-20T16:03:51.606 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: rocksdb: Options.create_missing_column_families: 0 2024-05-20T16:03:51.606 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: rocksdb: Options.db_log_dir: 2024-05-20T16:03:51.606 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: rocksdb: Options.wal_dir: 2024-05-20T16:03:51.606 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: rocksdb: Options.table_cache_numshardbits: 6 2024-05-20T16:03:51.606 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: rocksdb: Options.WAL_ttl_seconds: 0 2024-05-20T16:03:51.606 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: rocksdb: Options.WAL_size_limit_MB: 0 2024-05-20T16:03:51.606 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-05-20T16:03:51.606 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-05-20T16:03:51.606 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: rocksdb: Options.is_fd_close_on_exec: 1 2024-05-20T16:03:51.607 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: rocksdb: Options.advise_random_on_open: 1 2024-05-20T16:03:51.607 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: rocksdb: Options.db_write_buffer_size: 0 2024-05-20T16:03:51.607 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: rocksdb: Options.write_buffer_manager: 0x55cdb46f52c0 2024-05-20T16:03:51.607 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-05-20T16:03:51.607 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-05-20T16:03:51.607 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: rocksdb: Options.use_adaptive_mutex: 0 2024-05-20T16:03:51.607 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: rocksdb: Options.rate_limiter: (nil) 2024-05-20T16:03:51.607 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-05-20T16:03:51.607 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: rocksdb: Options.wal_recovery_mode: 2 2024-05-20T16:03:51.607 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: rocksdb: Options.enable_thread_tracking: 0 2024-05-20T16:03:51.607 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: rocksdb: Options.enable_pipelined_write: 0 2024-05-20T16:03:51.607 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: rocksdb: Options.unordered_write: 0 2024-05-20T16:03:51.607 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-05-20T16:03:51.607 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-05-20T16:03:51.607 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-05-20T16:03:51.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-05-20T16:03:51.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: rocksdb: Options.row_cache: None 2024-05-20T16:03:51.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: rocksdb: Options.wal_filter: None 2024-05-20T16:03:51.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-05-20T16:03:51.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: rocksdb: Options.allow_ingest_behind: 0 2024-05-20T16:03:51.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: rocksdb: Options.two_write_queues: 0 2024-05-20T16:03:51.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: rocksdb: Options.manual_wal_flush: 0 2024-05-20T16:03:51.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: rocksdb: Options.wal_compression: 0 2024-05-20T16:03:51.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: rocksdb: Options.atomic_flush: 0 2024-05-20T16:03:51.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-05-20T16:03:51.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: rocksdb: Options.persist_stats_to_disk: 0 2024-05-20T16:03:51.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: rocksdb: Options.write_dbid_to_manifest: 0 2024-05-20T16:03:51.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: rocksdb: Options.log_readahead_size: 0 2024-05-20T16:03:51.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-05-20T16:03:51.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: rocksdb: Options.best_efforts_recovery: 0 2024-05-20T16:03:51.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-05-20T16:03:51.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-05-20T16:03:51.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: rocksdb: Options.allow_data_in_errors: 0 2024-05-20T16:03:51.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: rocksdb: Options.db_host_id: __hostname__ 2024-05-20T16:03:51.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: rocksdb: Options.enforce_single_del_contracts: true 2024-05-20T16:03:51.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: rocksdb: Options.max_background_jobs: 2 2024-05-20T16:03:51.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: rocksdb: Options.max_background_compactions: -1 2024-05-20T16:03:51.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: rocksdb: Options.max_subcompactions: 1 2024-05-20T16:03:51.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-05-20T16:03:51.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-05-20T16:03:51.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: rocksdb: Options.delayed_write_rate : 16777216 2024-05-20T16:03:51.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: rocksdb: Options.max_total_wal_size: 0 2024-05-20T16:03:51.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-05-20T16:03:51.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: rocksdb: Options.stats_dump_period_sec: 600 2024-05-20T16:03:51.611 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: rocksdb: Options.stats_persist_period_sec: 600 2024-05-20T16:03:51.611 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-05-20T16:03:51.611 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: rocksdb: Options.max_open_files: -1 2024-05-20T16:03:51.611 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: rocksdb: Options.bytes_per_sync: 0 2024-05-20T16:03:51.611 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: rocksdb: Options.wal_bytes_per_sync: 0 2024-05-20T16:03:51.611 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: rocksdb: Options.strict_bytes_per_sync: 0 2024-05-20T16:03:51.611 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: rocksdb: Options.compaction_readahead_size: 0 2024-05-20T16:03:51.611 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: rocksdb: Options.max_background_flushes: -1 2024-05-20T16:03:51.611 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: rocksdb: Compression algorithms supported: 2024-05-20T16:03:51.611 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-05-20T16:03:51.611 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: rocksdb: kZSTD supported: 0 2024-05-20T16:03:51.611 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: rocksdb: kXpressCompression supported: 0 2024-05-20T16:03:51.611 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: rocksdb: kLZ4HCCompression supported: 1 2024-05-20T16:03:51.611 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: rocksdb: kZlibCompression supported: 1 2024-05-20T16:03:51.612 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: rocksdb: kSnappyCompression supported: 1 2024-05-20T16:03:51.612 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: rocksdb: kLZ4Compression supported: 1 2024-05-20T16:03:51.612 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: rocksdb: kBZip2Compression supported: 0 2024-05-20T16:03:51.612 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: rocksdb: Fast CRC32 supported: Supported on x86 2024-05-20T16:03:51.612 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: rocksdb: DMutex implementation: pthread_mutex_t 2024-05-20T16:03:51.612 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-05-20T16:03:51.612 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-05-20T16:03:51.612 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-05-20T16:03:51.612 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: rocksdb: Options.merge_operator: 2024-05-20T16:03:51.612 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: rocksdb: Options.compaction_filter: None 2024-05-20T16:03:51.612 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: rocksdb: Options.compaction_filter_factory: None 2024-05-20T16:03:51.612 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: rocksdb: Options.sst_partitioner_factory: None 2024-05-20T16:03:51.612 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: rocksdb: Options.memtable_factory: SkipListFactory 2024-05-20T16:03:51.612 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: rocksdb: Options.table_factory: BlockBasedTable 2024-05-20T16:03:51.612 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55cdb4721e40) 2024-05-20T16:03:51.612 INFO:journalctl@ceph.mon.a.smithi082.stdout: cache_index_and_filter_blocks: 1 2024-05-20T16:03:51.613 INFO:journalctl@ceph.mon.a.smithi082.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-05-20T16:03:51.613 INFO:journalctl@ceph.mon.a.smithi082.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-05-20T16:03:51.613 INFO:journalctl@ceph.mon.a.smithi082.stdout: pin_top_level_index_and_filter: 1 2024-05-20T16:03:51.613 INFO:journalctl@ceph.mon.a.smithi082.stdout: index_type: 0 2024-05-20T16:03:51.613 INFO:journalctl@ceph.mon.a.smithi082.stdout: data_block_index_type: 0 2024-05-20T16:03:51.613 INFO:journalctl@ceph.mon.a.smithi082.stdout: index_shortening: 1 2024-05-20T16:03:51.613 INFO:journalctl@ceph.mon.a.smithi082.stdout: data_block_hash_table_util_ratio: 0.750000 2024-05-20T16:03:51.613 INFO:journalctl@ceph.mon.a.smithi082.stdout: checksum: 4 2024-05-20T16:03:51.613 INFO:journalctl@ceph.mon.a.smithi082.stdout: no_block_cache: 0 2024-05-20T16:03:51.613 INFO:journalctl@ceph.mon.a.smithi082.stdout: block_cache: 0x55cdb4725090 2024-05-20T16:03:51.613 INFO:journalctl@ceph.mon.a.smithi082.stdout: block_cache_name: BinnedLRUCache 2024-05-20T16:03:51.613 INFO:journalctl@ceph.mon.a.smithi082.stdout: block_cache_options: 2024-05-20T16:03:51.613 INFO:journalctl@ceph.mon.a.smithi082.stdout: capacity : 536870912 2024-05-20T16:03:51.613 INFO:journalctl@ceph.mon.a.smithi082.stdout: num_shard_bits : 4 2024-05-20T16:03:51.614 INFO:journalctl@ceph.mon.a.smithi082.stdout: strict_capacity_limit : 0 2024-05-20T16:03:51.614 INFO:journalctl@ceph.mon.a.smithi082.stdout: high_pri_pool_ratio: 0.000 2024-05-20T16:03:51.614 INFO:journalctl@ceph.mon.a.smithi082.stdout: block_cache_compressed: (nil) 2024-05-20T16:03:51.614 INFO:journalctl@ceph.mon.a.smithi082.stdout: persistent_cache: (nil) 2024-05-20T16:03:51.614 INFO:journalctl@ceph.mon.a.smithi082.stdout: block_size: 4096 2024-05-20T16:03:51.614 INFO:journalctl@ceph.mon.a.smithi082.stdout: block_size_deviation: 10 2024-05-20T16:03:51.614 INFO:journalctl@ceph.mon.a.smithi082.stdout: block_restart_interval: 16 2024-05-20T16:03:51.614 INFO:journalctl@ceph.mon.a.smithi082.stdout: index_block_restart_interval: 1 2024-05-20T16:03:51.614 INFO:journalctl@ceph.mon.a.smithi082.stdout: metadata_block_size: 4096 2024-05-20T16:03:51.614 INFO:journalctl@ceph.mon.a.smithi082.stdout: partition_filters: 0 2024-05-20T16:03:51.614 INFO:journalctl@ceph.mon.a.smithi082.stdout: use_delta_encoding: 1 2024-05-20T16:03:51.614 INFO:journalctl@ceph.mon.a.smithi082.stdout: filter_policy: bloomfilter 2024-05-20T16:03:51.614 INFO:journalctl@ceph.mon.a.smithi082.stdout: whole_key_filtering: 1 2024-05-20T16:03:51.614 INFO:journalctl@ceph.mon.a.smithi082.stdout: verify_compression: 0 2024-05-20T16:03:51.615 INFO:journalctl@ceph.mon.a.smithi082.stdout: read_amp_bytes_per_bit: 0 2024-05-20T16:03:51.615 INFO:journalctl@ceph.mon.a.smithi082.stdout: format_version: 5 2024-05-20T16:03:51.615 INFO:journalctl@ceph.mon.a.smithi082.stdout: enable_index_compression: 1 2024-05-20T16:03:51.615 INFO:journalctl@ceph.mon.a.smithi082.stdout: block_align: 0 2024-05-20T16:03:51.615 INFO:journalctl@ceph.mon.a.smithi082.stdout: max_auto_readahead_size: 262144 2024-05-20T16:03:51.615 INFO:journalctl@ceph.mon.a.smithi082.stdout: prepopulate_block_cache: 0 2024-05-20T16:03:51.615 INFO:journalctl@ceph.mon.a.smithi082.stdout: initial_auto_readahead_size: 8192 2024-05-20T16:03:51.615 INFO:journalctl@ceph.mon.a.smithi082.stdout: num_file_reads_for_auto_readahead: 2 2024-05-20T16:03:51.615 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: rocksdb: Options.write_buffer_size: 33554432 2024-05-20T16:03:51.615 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: rocksdb: Options.max_write_buffer_number: 2 2024-05-20T16:03:51.615 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: rocksdb: Options.compression: NoCompression 2024-05-20T16:03:51.615 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: rocksdb: Options.bottommost_compression: Disabled 2024-05-20T16:03:51.615 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: rocksdb: Options.prefix_extractor: nullptr 2024-05-20T16:03:51.616 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-05-20T16:03:51.616 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: rocksdb: Options.num_levels: 7 2024-05-20T16:03:51.616 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-05-20T16:03:51.616 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-05-20T16:03:51.616 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-05-20T16:03:51.616 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-05-20T16:03:51.616 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-05-20T16:03:51.616 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-05-20T16:03:51.616 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-05-20T16:03:51.616 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-05-20T16:03:51.616 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-05-20T16:03:51.616 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-05-20T16:03:51.616 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-05-20T16:03:51.616 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-05-20T16:03:51.617 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: rocksdb: Options.compression_opts.window_bits: -14 2024-05-20T16:03:51.617 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: rocksdb: Options.compression_opts.level: 32767 2024-05-20T16:03:51.617 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: rocksdb: Options.compression_opts.strategy: 0 2024-05-20T16:03:51.617 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-05-20T16:03:51.617 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-05-20T16:03:51.617 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-05-20T16:03:51.617 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-05-20T16:03:51.617 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: rocksdb: Options.compression_opts.enabled: false 2024-05-20T16:03:51.617 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-05-20T16:03:51.617 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-05-20T16:03:51.617 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-05-20T16:03:51.617 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-05-20T16:03:51.617 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: rocksdb: Options.target_file_size_base: 67108864 2024-05-20T16:03:51.617 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: rocksdb: Options.target_file_size_multiplier: 1 2024-05-20T16:03:51.618 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-05-20T16:03:51.618 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-05-20T16:03:51.618 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-05-20T16:03:51.618 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-05-20T16:03:51.618 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-05-20T16:03:51.618 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-05-20T16:03:51.618 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-05-20T16:03:51.618 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-05-20T16:03:51.618 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-05-20T16:03:51.618 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-05-20T16:03:51.618 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-05-20T16:03:51.618 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-05-20T16:03:51.618 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-05-20T16:03:51.619 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: rocksdb: Options.arena_block_size: 1048576 2024-05-20T16:03:51.619 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-05-20T16:03:51.619 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-05-20T16:03:51.619 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: rocksdb: Options.disable_auto_compactions: 0 2024-05-20T16:03:51.619 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-05-20T16:03:51.619 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-05-20T16:03:51.619 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-05-20T16:03:51.619 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-05-20T16:03:51.619 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-05-20T16:03:51.619 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-05-20T16:03:51.619 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-05-20T16:03:51.619 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-05-20T16:03:51.619 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-05-20T16:03:51.619 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-05-20T16:03:51.619 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: rocksdb: Options.table_properties_collectors: 2024-05-20T16:03:51.620 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: rocksdb: Options.inplace_update_support: 0 2024-05-20T16:03:51.620 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: rocksdb: Options.inplace_update_num_locks: 10000 2024-05-20T16:03:51.620 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-05-20T16:03:51.620 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-05-20T16:03:51.620 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: rocksdb: Options.memtable_huge_page_size: 0 2024-05-20T16:03:51.620 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: rocksdb: Options.bloom_locality: 0 2024-05-20T16:03:51.620 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: rocksdb: Options.max_successive_merges: 0 2024-05-20T16:03:51.620 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: rocksdb: Options.optimize_filters_for_hits: 0 2024-05-20T16:03:51.620 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: rocksdb: Options.paranoid_file_checks: 0 2024-05-20T16:03:51.620 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: rocksdb: Options.force_consistency_checks: 1 2024-05-20T16:03:51.620 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: rocksdb: Options.report_bg_io_stats: 0 2024-05-20T16:03:51.620 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: rocksdb: Options.ttl: 2592000 2024-05-20T16:03:51.620 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: rocksdb: Options.periodic_compaction_seconds: 0 2024-05-20T16:03:51.620 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-05-20T16:03:51.620 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-05-20T16:03:51.620 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: rocksdb: Options.enable_blob_files: false 2024-05-20T16:03:51.621 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: rocksdb: Options.min_blob_size: 0 2024-05-20T16:03:51.621 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: rocksdb: Options.blob_file_size: 268435456 2024-05-20T16:03:51.621 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: rocksdb: Options.blob_compression_type: NoCompression 2024-05-20T16:03:51.621 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: rocksdb: Options.enable_blob_garbage_collection: false 2024-05-20T16:03:51.621 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-05-20T16:03:51.621 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-05-20T16:03:51.621 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-05-20T16:03:51.621 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: rocksdb: Options.blob_file_starting_level: 0 2024-05-20T16:03:51.621 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-05-20T16:03:51.621 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-05-20T16:03:51.621 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-05-20T16:03:51.621 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 426d530e-7e85-43bc-bddf-1180cf33c9e1 2024-05-20T16:03:51.621 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: rocksdb: EVENT_LOG_v1 {"time_micros": 1716221031458885, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-05-20T16:03:51.621 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-05-20T16:03:51.621 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: rocksdb: EVENT_LOG_v1 {"time_micros": 1716221031459991, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 99966, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 326, "table_properties": {"data_size": 97877, "index_size": 307, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 773, "raw_key_size": 14195, "raw_average_key_size": 49, "raw_value_size": 90164, "raw_average_value_size": 316, "num_data_blocks": 12, "num_entries": 285, "num_filter_entries": 285, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1716221031, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "426d530e-7e85-43bc-bddf-1180cf33c9e1", "db_session_id": "CWESX2OPB402MWCDZGPJ", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-05-20T16:03:51.622 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: rocksdb: EVENT_LOG_v1 {"time_micros": 1716221031460073, "job": 1, "event": "recovery_finished"} 2024-05-20T16:03:51.622 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-05-20T16:03:51.622 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-05-20T16:03:51.622 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55cdb470bc00 2024-05-20T16:03:51.622 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: rocksdb: DB pointer 0x55cdb94f8000 2024-05-20T16:03:51.622 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-05-20T16:03:51.622 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-05-20T16:03:51.622 INFO:journalctl@ceph.mon.a.smithi082.stdout: ** DB Stats ** 2024-05-20T16:03:51.622 INFO:journalctl@ceph.mon.a.smithi082.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-20T16:03:51.622 INFO:journalctl@ceph.mon.a.smithi082.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-05-20T16:03:51.622 INFO:journalctl@ceph.mon.a.smithi082.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-20T16:03:51.622 INFO:journalctl@ceph.mon.a.smithi082.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-20T16:03:51.622 INFO:journalctl@ceph.mon.a.smithi082.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-05-20T16:03:51.623 INFO:journalctl@ceph.mon.a.smithi082.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-20T16:03:51.623 INFO:journalctl@ceph.mon.a.smithi082.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-20T16:03:51.623 INFO:journalctl@ceph.mon.a.smithi082.stdout: 2024-05-20T16:03:51.623 INFO:journalctl@ceph.mon.a.smithi082.stdout: ** Compaction Stats [default] ** 2024-05-20T16:03:51.623 INFO:journalctl@ceph.mon.a.smithi082.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-20T16:03:51.623 INFO:journalctl@ceph.mon.a.smithi082.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-05-20T16:03:51.623 INFO:journalctl@ceph.mon.a.smithi082.stdout: L0 2/0 99.46 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 158.4 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-20T16:03:51.623 INFO:journalctl@ceph.mon.a.smithi082.stdout: Sum 2/0 99.46 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 158.4 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-20T16:03:51.623 INFO:journalctl@ceph.mon.a.smithi082.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 158.4 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-20T16:03:51.623 INFO:journalctl@ceph.mon.a.smithi082.stdout: 2024-05-20T16:03:51.623 INFO:journalctl@ceph.mon.a.smithi082.stdout: ** Compaction Stats [default] ** 2024-05-20T16:03:51.623 INFO:journalctl@ceph.mon.a.smithi082.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-20T16:03:51.623 INFO:journalctl@ceph.mon.a.smithi082.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-05-20T16:03:51.623 INFO:journalctl@ceph.mon.a.smithi082.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 158.4 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-20T16:03:51.624 INFO:journalctl@ceph.mon.a.smithi082.stdout: 2024-05-20T16:03:51.624 INFO:journalctl@ceph.mon.a.smithi082.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-05-20T16:03:51.624 INFO:journalctl@ceph.mon.a.smithi082.stdout: 2024-05-20T16:03:51.624 INFO:journalctl@ceph.mon.a.smithi082.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-20T16:03:51.624 INFO:journalctl@ceph.mon.a.smithi082.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-05-20T16:03:51.624 INFO:journalctl@ceph.mon.a.smithi082.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-05-20T16:03:51.624 INFO:journalctl@ceph.mon.a.smithi082.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-05-20T16:03:51.624 INFO:journalctl@ceph.mon.a.smithi082.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-05-20T16:03:51.624 INFO:journalctl@ceph.mon.a.smithi082.stdout: AddFile(Keys): cumulative 0, interval 0 2024-05-20T16:03:51.624 INFO:journalctl@ceph.mon.a.smithi082.stdout: Cumulative compaction: 0.00 GB write, 30.40 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-20T16:03:51.624 INFO:journalctl@ceph.mon.a.smithi082.stdout: Interval compaction: 0.00 GB write, 30.40 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-20T16:03:51.624 INFO:journalctl@ceph.mon.a.smithi082.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-05-20T16:03:51.624 INFO:journalctl@ceph.mon.a.smithi082.stdout: Block cache BinnedLRUCache@0x55cdb4725090#2 capacity: 512.00 MB usage: 1.53 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 8e-06 secs_since: 0 2024-05-20T16:03:51.624 INFO:journalctl@ceph.mon.a.smithi082.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,1.03 KB,0.000196695%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-05-20T16:03:51.624 INFO:journalctl@ceph.mon.a.smithi082.stdout: 2024-05-20T16:03:51.625 INFO:journalctl@ceph.mon.a.smithi082.stdout: ** File Read Latency Histogram By Level [default] ** 2024-05-20T16:03:51.625 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: starting mon.a rank 0 at public addrs [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] at bind addrs [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 59d1600a-16c2-11ef-bc9a-c7b262605968 2024-05-20T16:03:51.625 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: mon.a@-1(???) e1 preinit fsid 59d1600a-16c2-11ef-bc9a-c7b262605968 2024-05-20T16:03:51.625 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: mon.a@-1(???).mds e1 new map 2024-05-20T16:03:51.625 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: mon.a@-1(???).mds e1 print_map 2024-05-20T16:03:51.625 INFO:journalctl@ceph.mon.a.smithi082.stdout: e1 2024-05-20T16:03:51.625 INFO:journalctl@ceph.mon.a.smithi082.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-05-20T16:03:51.625 INFO:journalctl@ceph.mon.a.smithi082.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-05-20T16:03:51.625 INFO:journalctl@ceph.mon.a.smithi082.stdout: legacy client fscid: -1 2024-05-20T16:03:51.625 INFO:journalctl@ceph.mon.a.smithi082.stdout: 2024-05-20T16:03:51.625 INFO:journalctl@ceph.mon.a.smithi082.stdout: No filesystems configured 2024-05-20T16:03:51.625 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-05-20T16:03:51.625 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-05-20T16:03:51.625 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-05-20T16:03:51.625 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-05-20T16:03:51.626 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-05-20T16:03:51.626 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 bash[32219]: 1c5b1c87352a8fe3c4399e9c039e6ffd8962413e7ab6afaaa251874aa2fc7fe3 2024-05-20T16:03:51.626 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 systemd[1]: Started Ceph mon.a for 59d1600a-16c2-11ef-bc9a-c7b262605968. 2024-05-20T16:03:51.626 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: mon.a is new leader, mons a in quorum (ranks 0) 2024-05-20T16:03:51.626 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: monmap epoch 1 2024-05-20T16:03:51.626 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: fsid 59d1600a-16c2-11ef-bc9a-c7b262605968 2024-05-20T16:03:51.626 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: last_changed 2024-05-20T16:03:43.664950+0000 2024-05-20T16:03:51.626 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: created 2024-05-20T16:03:43.664950+0000 2024-05-20T16:03:51.626 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: min_mon_release 19 (squid) 2024-05-20T16:03:51.626 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: election_strategy: 1 2024-05-20T16:03:51.626 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: 0: [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] mon.a 2024-05-20T16:03:51.626 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: fsmap 2024-05-20T16:03:51.626 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: osdmap e1: 0 total, 0 up, 0 in 2024-05-20T16:03:51.626 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:51 smithi082 ceph-mon[32238]: mgrmap e1: no daemons active 2024-05-20T16:03:52.676 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:52.009+0000 7f5956df9700 1 Processor -- start 2024-05-20T16:03:52.676 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:52.009+0000 7f5956df9700 1 -- start start 2024-05-20T16:03:52.676 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:52.009+0000 7f5956df9700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5950108b80 0x7f5950108f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:03:52.677 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:52.009+0000 7f5956df9700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f5950109580 con 0x7f5950108b80 2024-05-20T16:03:52.677 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:52.010+0000 7f5954b95700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5950108b80 0x7f5950108f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:03:52.677 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:52.010+0000 7f5954b95700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5950108b80 0x7f5950108f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:52434/0 (socket says 172.21.15.82:52434) 2024-05-20T16:03:52.677 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:52.010+0000 7f5954b95700 1 -- 172.21.15.82:0/1295144329 learned_addr learned my addr 172.21.15.82:0/1295144329 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:03:52.677 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:52.010+0000 7f5954b95700 1 -- 172.21.15.82:0/1295144329 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5950109c10 con 0x7f5950108b80 2024-05-20T16:03:52.677 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:52.010+0000 7f5954b95700 1 --2- 172.21.15.82:0/1295144329 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5950108b80 0x7f5950108f90 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f594c0098a0 tx=0x7f594c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3437591d6a9a93e3 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:03:52.678 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:52.010+0000 7f59477fe700 1 -- 172.21.15.82:0/1295144329 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f594c004160 con 0x7f5950108b80 2024-05-20T16:03:52.678 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:52.011+0000 7f59477fe700 1 -- 172.21.15.82:0/1295144329 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f594c004300 con 0x7f5950108b80 2024-05-20T16:03:52.678 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:52.011+0000 7f59477fe700 1 -- 172.21.15.82:0/1295144329 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f594c011b50 con 0x7f5950108b80 2024-05-20T16:03:52.678 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:52.011+0000 7f5956df9700 1 -- 172.21.15.82:0/1295144329 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5950108b80 msgr2=0x7f5950108f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:03:52.678 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:52.011+0000 7f5956df9700 1 --2- 172.21.15.82:0/1295144329 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5950108b80 0x7f5950108f90 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f594c0098a0 tx=0x7f594c009bb0 comp rx=0 tx=0).stop 2024-05-20T16:03:52.678 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:52.011+0000 7f5956df9700 1 -- 172.21.15.82:0/1295144329 shutdown_connections 2024-05-20T16:03:52.679 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:52.011+0000 7f5956df9700 1 --2- 172.21.15.82:0/1295144329 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5950108b80 0x7f5950108f90 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:03:52.679 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:52.011+0000 7f5956df9700 1 -- 172.21.15.82:0/1295144329 >> 172.21.15.82:0/1295144329 conn(0x7f5950104220 msgr2=0x7f5950106670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:03:52.679 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:52.011+0000 7f5956df9700 1 -- 172.21.15.82:0/1295144329 shutdown_connections 2024-05-20T16:03:52.679 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:52.011+0000 7f5956df9700 1 -- 172.21.15.82:0/1295144329 wait complete. 2024-05-20T16:03:52.679 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:52.012+0000 7f5956df9700 1 Processor -- start 2024-05-20T16:03:52.679 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:52.012+0000 7f5956df9700 1 -- start start 2024-05-20T16:03:52.680 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:52.012+0000 7f5956df9700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5950108b80 0x7f595019ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:03:52.680 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:52.012+0000 7f5956df9700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f595010cfc0 con 0x7f5950108b80 2024-05-20T16:03:52.680 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:52.012+0000 7f5954b95700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5950108b80 0x7f595019ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:03:52.680 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:52.013+0000 7f5954b95700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5950108b80 0x7f595019ce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:52448/0 (socket says 172.21.15.82:52448) 2024-05-20T16:03:52.680 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:52.013+0000 7f5954b95700 1 -- 172.21.15.82:0/3107541762 learned_addr learned my addr 172.21.15.82:0/3107541762 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:03:52.681 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:52.013+0000 7f5954b95700 1 -- 172.21.15.82:0/3107541762 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f595019d3a0 con 0x7f5950108b80 2024-05-20T16:03:52.681 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:52.013+0000 7f5954b95700 1 --2- 172.21.15.82:0/3107541762 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5950108b80 0x7f595019ce60 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f594c009fb0 tx=0x7f594c011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:03:52.681 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:52.013+0000 7f5945ffb700 1 -- 172.21.15.82:0/3107541762 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f594c0038b0 con 0x7f5950108b80 2024-05-20T16:03:52.681 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:52.013+0000 7f5945ffb700 1 -- 172.21.15.82:0/3107541762 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f594c011e50 con 0x7f5950108b80 2024-05-20T16:03:52.681 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:52.013+0000 7f5956df9700 1 -- 172.21.15.82:0/3107541762 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f595019d630 con 0x7f5950108b80 2024-05-20T16:03:52.681 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:52.013+0000 7f5945ffb700 1 -- 172.21.15.82:0/3107541762 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f594c01a430 con 0x7f5950108b80 2024-05-20T16:03:52.682 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:52.013+0000 7f5956df9700 1 -- 172.21.15.82:0/3107541762 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59501a0240 con 0x7f5950108b80 2024-05-20T16:03:52.682 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:52.014+0000 7f5945ffb700 1 -- 172.21.15.82:0/3107541762 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f594c01aa70 con 0x7f5950108b80 2024-05-20T16:03:52.682 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:52.014+0000 7f5945ffb700 1 -- 172.21.15.82:0/3107541762 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f594c019700 con 0x7f5950108b80 2024-05-20T16:03:52.682 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:52.014+0000 7f5956df9700 1 -- 172.21.15.82:0/3107541762 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5950063ca0 con 0x7f5950108b80 2024-05-20T16:03:52.682 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:52.017+0000 7f5945ffb700 1 -- 172.21.15.82:0/3107541762 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f594c024440 con 0x7f5950108b80 2024-05-20T16:03:52.683 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:52.081+0000 7f5956df9700 1 -- 172.21.15.82:0/3107541762 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7f59501a0410 con 0x7f5950108b80 2024-05-20T16:03:52.683 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:52.084+0000 7f5945ffb700 1 -- 172.21.15.82:0/3107541762 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7f594c032030 con 0x7f5950108b80 2024-05-20T16:03:52.683 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:52.085+0000 7f5956df9700 1 -- 172.21.15.82:0/3107541762 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5950108b80 msgr2=0x7f595019ce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:03:52.683 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:52.085+0000 7f5956df9700 1 --2- 172.21.15.82:0/3107541762 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5950108b80 0x7f595019ce60 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f594c009fb0 tx=0x7f594c011c10 comp rx=0 tx=0).stop 2024-05-20T16:03:52.683 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:52.085+0000 7f5956df9700 1 -- 172.21.15.82:0/3107541762 shutdown_connections 2024-05-20T16:03:52.683 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:52.085+0000 7f5956df9700 1 --2- 172.21.15.82:0/3107541762 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5950108b80 0x7f595019ce60 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:03:52.683 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:52.085+0000 7f5956df9700 1 -- 172.21.15.82:0/3107541762 >> 172.21.15.82:0/3107541762 conn(0x7f5950104220 msgr2=0x7f5950104aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:03:52.684 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:52.085+0000 7f5956df9700 1 -- 172.21.15.82:0/3107541762 shutdown_connections 2024-05-20T16:03:52.684 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:52.085+0000 7f5956df9700 1 -- 172.21.15.82:0/3107541762 wait complete. 2024-05-20T16:03:52.684 INFO:teuthology.orchestra.run.smithi082.stdout:Wrote config to /etc/ceph/ceph.conf 2024-05-20T16:03:52.689 INFO:teuthology.orchestra.run.smithi082.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-05-20T16:03:52.690 INFO:teuthology.orchestra.run.smithi082.stdout:Creating mgr... 2024-05-20T16:03:52.690 INFO:teuthology.orchestra.run.smithi082.stdout:Verifying port 0.0.0.0:9283 ... 2024-05-20T16:03:52.691 INFO:teuthology.orchestra.run.smithi082.stdout:Verifying port 0.0.0.0:8765 ... 2024-05-20T16:03:52.945 INFO:teuthology.orchestra.run.smithi082.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-59d1600a-16c2-11ef-bc9a-c7b262605968@mgr.x 2024-05-20T16:03:52.945 INFO:teuthology.orchestra.run.smithi082.stdout:systemctl: stderr Failed to reset failed state of unit ceph-59d1600a-16c2-11ef-bc9a-c7b262605968@mgr.x.service: Unit ceph-59d1600a-16c2-11ef-bc9a-c7b262605968@mgr.x.service not loaded. 2024-05-20T16:03:53.099 INFO:teuthology.orchestra.run.smithi082.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-59d1600a-16c2-11ef-bc9a-c7b262605968.target.wants/ceph-59d1600a-16c2-11ef-bc9a-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-59d1600a-16c2-11ef-bc9a-c7b262605968@.service. 2024-05-20T16:03:53.300 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:53 smithi082 ceph-mon[32238]: from='client.? 172.21.15.82:0/3107541762' entity='client.admin' 2024-05-20T16:03:53.301 INFO:journalctl@ceph.mgr.x.smithi082.stdout:May 20 16:03:53 smithi082 systemd[1]: Starting Ceph mgr.x for 59d1600a-16c2-11ef-bc9a-c7b262605968... 2024-05-20T16:03:53.553 INFO:journalctl@ceph.mgr.x.smithi082.stdout:May 20 16:03:53 smithi082 podman[32451]: 2024-05-20 16:03:53.300565082 +0000 UTC m=+0.082372916 container create c80a162fbf8bfb0508a8b4414fd657ef0ab259d17f91d1f750cd84cb994ceb2c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d, name=ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mgr-x, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240514, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c) 2024-05-20T16:03:53.553 INFO:journalctl@ceph.mgr.x.smithi082.stdout:May 20 16:03:53 smithi082 podman[32451]: 2024-05-20 16:03:53.23532777 +0000 UTC m=+0.017135609 image pull cfcc61b2d476f611479310eb042aa05f11dd5f60dbb65e3b0d7e0e11d024cdb3 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d 2024-05-20T16:03:53.553 INFO:journalctl@ceph.mgr.x.smithi082.stdout:May 20 16:03:53 smithi082 podman[32451]: 2024-05-20 16:03:53.478078748 +0000 UTC m=+0.259886585 container init c80a162fbf8bfb0508a8b4414fd657ef0ab259d17f91d1f750cd84cb994ceb2c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d, name=ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mgr-x, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240514, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True) 2024-05-20T16:03:53.553 INFO:journalctl@ceph.mgr.x.smithi082.stdout:May 20 16:03:53 smithi082 podman[32451]: 2024-05-20 16:03:53.480548741 +0000 UTC m=+0.262356576 container start c80a162fbf8bfb0508a8b4414fd657ef0ab259d17f91d1f750cd84cb994ceb2c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d, name=ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mgr-x, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240514, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-20T16:03:53.553 INFO:journalctl@ceph.mgr.x.smithi082.stdout:May 20 16:03:53 smithi082 bash[32451]: c80a162fbf8bfb0508a8b4414fd657ef0ab259d17f91d1f750cd84cb994ceb2c 2024-05-20T16:03:53.566 INFO:teuthology.orchestra.run.smithi082.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-20T16:03:53.566 INFO:teuthology.orchestra.run.smithi082.stdout:systemctl: stdout disabled 2024-05-20T16:03:53.574 INFO:teuthology.orchestra.run.smithi082.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-20T16:03:53.574 INFO:teuthology.orchestra.run.smithi082.stdout:systemctl: stdout inactive 2024-05-20T16:03:53.574 INFO:teuthology.orchestra.run.smithi082.stdout:firewalld.service is not enabled 2024-05-20T16:03:53.574 INFO:teuthology.orchestra.run.smithi082.stdout:Not possible to enable service . firewalld.service is not available 2024-05-20T16:03:53.582 INFO:teuthology.orchestra.run.smithi082.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-20T16:03:53.582 INFO:teuthology.orchestra.run.smithi082.stdout:systemctl: stdout disabled 2024-05-20T16:03:53.590 INFO:teuthology.orchestra.run.smithi082.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-20T16:03:53.590 INFO:teuthology.orchestra.run.smithi082.stdout:systemctl: stdout inactive 2024-05-20T16:03:53.591 INFO:teuthology.orchestra.run.smithi082.stdout:firewalld.service is not enabled 2024-05-20T16:03:53.591 INFO:teuthology.orchestra.run.smithi082.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-05-20T16:03:53.591 INFO:teuthology.orchestra.run.smithi082.stdout:Waiting for mgr to start... 2024-05-20T16:03:53.591 INFO:teuthology.orchestra.run.smithi082.stdout:Waiting for mgr... 2024-05-20T16:03:53.836 INFO:journalctl@ceph.mgr.x.smithi082.stdout:May 20 16:03:53 smithi082 systemd[1]: Started Ceph mgr.x for 59d1600a-16c2-11ef-bc9a-c7b262605968. 2024-05-20T16:03:53.836 INFO:journalctl@ceph.mgr.x.smithi082.stdout:May 20 16:03:53 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mgr-x[32467]: 2024-05-20T16:03:53.647+0000 7f75f89bd200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-20T16:03:53.836 INFO:journalctl@ceph.mgr.x.smithi082.stdout:May 20 16:03:53 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mgr-x[32467]: 2024-05-20T16:03:53.835+0000 7f75f89bd200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-20T16:03:54.109 INFO:journalctl@ceph.mgr.x.smithi082.stdout:May 20 16:03:53 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mgr-x[32467]: 2024-05-20T16:03:53.939+0000 7f75f89bd200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-20T16:03:54.451 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:03:54 smithi082 ceph-mon[32238]: from='client.? 172.21.15.82:0/3686984926' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-20T16:03:54.451 INFO:journalctl@ceph.mgr.x.smithi082.stdout:May 20 16:03:54 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mgr-x[32467]: 2024-05-20T16:03:54.263+0000 7f75f89bd200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-20T16:03:54.451 INFO:journalctl@ceph.mgr.x.smithi082.stdout:May 20 16:03:54 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mgr-x[32467]: 2024-05-20T16:03:54.349+0000 7f75f89bd200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-20T16:03:54.721 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout 2024-05-20T16:03:54.723 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout { 2024-05-20T16:03:54.723 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "fsid": "59d1600a-16c2-11ef-bc9a-c7b262605968", 2024-05-20T16:03:54.724 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "health": { 2024-05-20T16:03:54.724 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-20T16:03:54.724 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-20T16:03:54.724 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-20T16:03:54.724 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout }, 2024-05-20T16:03:54.724 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-20T16:03:54.724 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-20T16:03:54.724 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout 0 2024-05-20T16:03:54.724 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout ], 2024-05-20T16:03:54.724 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-20T16:03:54.724 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "a" 2024-05-20T16:03:54.724 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout ], 2024-05-20T16:03:54.724 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-05-20T16:03:54.724 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-20T16:03:54.725 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-20T16:03:54.725 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-05-20T16:03:54.725 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-20T16:03:54.725 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout }, 2024-05-20T16:03:54.725 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-20T16:03:54.725 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-20T16:03:54.725 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-20T16:03:54.725 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-20T16:03:54.725 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-20T16:03:54.733 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-20T16:03:54.733 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-20T16:03:54.733 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-20T16:03:54.733 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout }, 2024-05-20T16:03:54.733 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-20T16:03:54.733 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-20T16:03:54.733 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-20T16:03:54.733 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-20T16:03:54.733 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-20T16:03:54.733 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-20T16:03:54.733 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-20T16:03:54.734 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-20T16:03:54.734 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-20T16:03:54.734 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout }, 2024-05-20T16:03:54.734 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-20T16:03:54.734 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-20T16:03:54.734 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-20T16:03:54.734 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-20T16:03:54.734 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout }, 2024-05-20T16:03:54.734 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-20T16:03:54.734 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "available": false, 2024-05-20T16:03:54.734 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-20T16:03:54.735 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-20T16:03:54.735 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-20T16:03:54.735 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-20T16:03:54.735 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "restful" 2024-05-20T16:03:54.735 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout ], 2024-05-20T16:03:54.735 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-20T16:03:54.735 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout }, 2024-05-20T16:03:54.735 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-20T16:03:54.735 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-20T16:03:54.735 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "modified": "2024-05-20T16:03:46.612160+0000", 2024-05-20T16:03:54.735 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-20T16:03:54.735 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout }, 2024-05-20T16:03:54.735 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-20T16:03:54.735 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout } 2024-05-20T16:03:54.735 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:54.059+0000 7fd04014e700 1 Processor -- start 2024-05-20T16:03:54.736 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:54.059+0000 7fd04014e700 1 -- start start 2024-05-20T16:03:54.736 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:54.060+0000 7fd04014e700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd03807bc80 0x7fd03810b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:03:54.736 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:54.060+0000 7fd04014e700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fd03810bbd0 con 0x7fd03807bc80 2024-05-20T16:03:54.736 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:54.060+0000 7fd03deea700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd03807bc80 0x7fd03810b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:03:54.736 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:54.060+0000 7fd03deea700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd03807bc80 0x7fd03810b600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:52472/0 (socket says 172.21.15.82:52472) 2024-05-20T16:03:54.736 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:54.060+0000 7fd03deea700 1 -- 172.21.15.82:0/1986055245 learned_addr learned my addr 172.21.15.82:0/1986055245 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:03:54.736 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:54.060+0000 7fd03deea700 1 -- 172.21.15.82:0/1986055245 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd03810bd50 con 0x7fd03807bc80 2024-05-20T16:03:54.736 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:54.061+0000 7fd03deea700 1 --2- 172.21.15.82:0/1986055245 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd03807bc80 0x7fd03810b600 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fd0340098a0 tx=0x7fd034009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aeb49ce96c839864 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:03:54.736 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:54.061+0000 7fd03cee8700 1 -- 172.21.15.82:0/1986055245 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd034004160 con 0x7fd03807bc80 2024-05-20T16:03:54.736 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:54.061+0000 7fd03cee8700 1 -- 172.21.15.82:0/1986055245 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fd034004300 con 0x7fd03807bc80 2024-05-20T16:03:54.736 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:54.061+0000 7fd03cee8700 1 -- 172.21.15.82:0/1986055245 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd034011b50 con 0x7fd03807bc80 2024-05-20T16:03:54.736 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:54.062+0000 7fd04014e700 1 -- 172.21.15.82:0/1986055245 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd03807bc80 msgr2=0x7fd03810b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:03:54.736 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:54.062+0000 7fd04014e700 1 --2- 172.21.15.82:0/1986055245 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd03807bc80 0x7fd03810b600 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fd0340098a0 tx=0x7fd034009bb0 comp rx=0 tx=0).stop 2024-05-20T16:03:54.736 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:54.062+0000 7fd04014e700 1 -- 172.21.15.82:0/1986055245 shutdown_connections 2024-05-20T16:03:54.736 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:54.062+0000 7fd04014e700 1 --2- 172.21.15.82:0/1986055245 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd03807bc80 0x7fd03810b600 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:03:54.737 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:54.062+0000 7fd04014e700 1 -- 172.21.15.82:0/1986055245 >> 172.21.15.82:0/1986055245 conn(0x7fd038103070 msgr2=0x7fd0381054c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:03:54.737 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:54.062+0000 7fd04014e700 1 -- 172.21.15.82:0/1986055245 shutdown_connections 2024-05-20T16:03:54.737 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:54.062+0000 7fd04014e700 1 -- 172.21.15.82:0/1986055245 wait complete. 2024-05-20T16:03:54.737 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:54.062+0000 7fd04014e700 1 Processor -- start 2024-05-20T16:03:54.737 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:54.063+0000 7fd04014e700 1 -- start start 2024-05-20T16:03:54.737 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:54.063+0000 7fd04014e700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd03807bc80 0x7fd038108d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:03:54.737 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:54.063+0000 7fd04014e700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fd03810ce30 con 0x7fd03807bc80 2024-05-20T16:03:54.737 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:54.063+0000 7fd03deea700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd03807bc80 0x7fd038108d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:03:54.737 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:54.063+0000 7fd03deea700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd03807bc80 0x7fd038108d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:52482/0 (socket says 172.21.15.82:52482) 2024-05-20T16:03:54.737 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:54.063+0000 7fd03deea700 1 -- 172.21.15.82:0/3686984926 learned_addr learned my addr 172.21.15.82:0/3686984926 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:03:54.738 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:54.063+0000 7fd03deea700 1 -- 172.21.15.82:0/3686984926 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0381092b0 con 0x7fd03807bc80 2024-05-20T16:03:54.738 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:54.063+0000 7fd03deea700 1 --2- 172.21.15.82:0/3686984926 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd03807bc80 0x7fd038108d70 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fd034009fb0 tx=0x7fd034011bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:03:54.738 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:54.064+0000 7fd02effd700 1 -- 172.21.15.82:0/3686984926 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd0340038b0 con 0x7fd03807bc80 2024-05-20T16:03:54.738 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:54.064+0000 7fd02effd700 1 -- 172.21.15.82:0/3686984926 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fd034011e50 con 0x7fd03807bc80 2024-05-20T16:03:54.738 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:54.064+0000 7fd04014e700 1 -- 172.21.15.82:0/3686984926 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd038107500 con 0x7fd03807bc80 2024-05-20T16:03:54.738 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:54.064+0000 7fd02effd700 1 -- 172.21.15.82:0/3686984926 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd034019420 con 0x7fd03807bc80 2024-05-20T16:03:54.738 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:54.064+0000 7fd04014e700 1 -- 172.21.15.82:0/3686984926 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0381079e0 con 0x7fd03807bc80 2024-05-20T16:03:54.738 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:54.064+0000 7fd02effd700 1 -- 172.21.15.82:0/3686984926 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fd034019a80 con 0x7fd03807bc80 2024-05-20T16:03:54.738 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:54.064+0000 7fd02effd700 1 -- 172.21.15.82:0/3686984926 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fd034024730 con 0x7fd03807bc80 2024-05-20T16:03:54.738 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:54.065+0000 7fd04014e700 1 -- 172.21.15.82:0/3686984926 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd038063ca0 con 0x7fd03807bc80 2024-05-20T16:03:54.738 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:54.067+0000 7fd02effd700 1 -- 172.21.15.82:0/3686984926 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fd034003a50 con 0x7fd03807bc80 2024-05-20T16:03:54.738 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:54.141+0000 7fd04014e700 1 -- 172.21.15.82:0/3686984926 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fd038107c60 con 0x7fd03807bc80 2024-05-20T16:03:54.738 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:54.141+0000 7fd02effd700 1 -- 172.21.15.82:0/3686984926 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7fd034032030 con 0x7fd03807bc80 2024-05-20T16:03:54.738 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:54.142+0000 7fd04014e700 1 -- 172.21.15.82:0/3686984926 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd03807bc80 msgr2=0x7fd038108d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:03:54.739 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:54.142+0000 7fd04014e700 1 --2- 172.21.15.82:0/3686984926 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd03807bc80 0x7fd038108d70 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fd034009fb0 tx=0x7fd034011bf0 comp rx=0 tx=0).stop 2024-05-20T16:03:54.739 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:54.142+0000 7fd04014e700 1 -- 172.21.15.82:0/3686984926 shutdown_connections 2024-05-20T16:03:54.739 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:54.142+0000 7fd04014e700 1 --2- 172.21.15.82:0/3686984926 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd03807bc80 0x7fd038108d70 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:03:54.739 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:54.142+0000 7fd04014e700 1 -- 172.21.15.82:0/3686984926 >> 172.21.15.82:0/3686984926 conn(0x7fd038103070 msgr2=0x7fd038104920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:03:54.739 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:54.142+0000 7fd04014e700 1 -- 172.21.15.82:0/3686984926 shutdown_connections 2024-05-20T16:03:54.739 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:03:54.142+0000 7fd04014e700 1 -- 172.21.15.82:0/3686984926 wait complete. 2024-05-20T16:03:54.739 INFO:teuthology.orchestra.run.smithi082.stdout:mgr not available, waiting (1/15)... 2024-05-20T16:03:55.860 INFO:journalctl@ceph.mgr.x.smithi082.stdout:May 20 16:03:55 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mgr-x[32467]: 2024-05-20T16:03:55.547+0000 7f75f89bd200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-20T16:03:55.860 INFO:journalctl@ceph.mgr.x.smithi082.stdout:May 20 16:03:55 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mgr-x[32467]: 2024-05-20T16:03:55.740+0000 7f75f89bd200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-20T16:03:56.197 INFO:journalctl@ceph.mgr.x.smithi082.stdout:May 20 16:03:55 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mgr-x[32467]: 2024-05-20T16:03:55.911+0000 7f75f89bd200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-20T16:03:56.197 INFO:journalctl@ceph.mgr.x.smithi082.stdout:May 20 16:03:56 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mgr-x[32467]: 2024-05-20T16:03:56.196+0000 7f75f89bd200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-20T16:03:56.490 INFO:journalctl@ceph.mgr.x.smithi082.stdout:May 20 16:03:56 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mgr-x[32467]: 2024-05-20T16:03:56.287+0000 7f75f89bd200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-20T16:03:56.490 INFO:journalctl@ceph.mgr.x.smithi082.stdout:May 20 16:03:56 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mgr-x[32467]: 2024-05-20T16:03:56.489+0000 7f75f89bd200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-20T16:03:56.857 INFO:journalctl@ceph.mgr.x.smithi082.stdout:May 20 16:03:56 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mgr-x[32467]: 2024-05-20T16:03:56.681+0000 7f75f89bd200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-20T16:03:57.609 INFO:journalctl@ceph.mgr.x.smithi082.stdout:May 20 16:03:57 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mgr-x[32467]: 2024-05-20T16:03:57.292+0000 7f75f89bd200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-20T16:03:58.053 INFO:journalctl@ceph.mgr.x.smithi082.stdout:May 20 16:03:57 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mgr-x[32467]: 2024-05-20T16:03:57.720+0000 7f75f89bd200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-20T16:03:58.053 INFO:journalctl@ceph.mgr.x.smithi082.stdout:May 20 16:03:57 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mgr-x[32467]: 2024-05-20T16:03:57.818+0000 7f75f89bd200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-20T16:03:58.053 INFO:journalctl@ceph.mgr.x.smithi082.stdout:May 20 16:03:58 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mgr-x[32467]: 2024-05-20T16:03:58.019+0000 7f75f89bd200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-20T16:03:58.860 INFO:journalctl@ceph.mgr.x.smithi082.stdout:May 20 16:03:58 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mgr-x[32467]: 2024-05-20T16:03:58.550+0000 7f75f89bd200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-20T16:03:58.860 INFO:journalctl@ceph.mgr.x.smithi082.stdout:May 20 16:03:58 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mgr-x[32467]: 2024-05-20T16:03:58.643+0000 7f75f89bd200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-20T16:03:59.240 INFO:journalctl@ceph.mgr.x.smithi082.stdout:May 20 16:03:58 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mgr-x[32467]: 2024-05-20T16:03:58.888+0000 7f75f89bd200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-20T16:03:59.240 INFO:journalctl@ceph.mgr.x.smithi082.stdout:May 20 16:03:59 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mgr-x[32467]: 2024-05-20T16:03:59.021+0000 7f75f89bd200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-20T16:03:59.240 INFO:journalctl@ceph.mgr.x.smithi082.stdout:May 20 16:03:59 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mgr-x[32467]: 2024-05-20T16:03:59.119+0000 7f75f89bd200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-20T16:03:59.240 INFO:journalctl@ceph.mgr.x.smithi082.stdout:May 20 16:03:59 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mgr-x[32467]: 2024-05-20T16:03:59.239+0000 7f75f89bd200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-20T16:03:59.859 INFO:journalctl@ceph.mgr.x.smithi082.stdout:May 20 16:03:59 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mgr-x[32467]: 2024-05-20T16:03:59.543+0000 7f75f89bd200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-20T16:04:00.292 INFO:journalctl@ceph.mgr.x.smithi082.stdout:May 20 16:04:00 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mgr-x[32467]: 2024-05-20T16:04:00.163+0000 7f75f89bd200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-20T16:04:00.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:04:00 smithi082 ceph-mon[32238]: from='client.? 172.21.15.82:0/3420475560' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-20T16:04:00.610 INFO:journalctl@ceph.mgr.x.smithi082.stdout:May 20 16:04:00 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mgr-x[32467]: 2024-05-20T16:04:00.397+0000 7f75f89bd200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-20T16:04:01.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:04:01 smithi082 ceph-mon[32238]: Activating manager daemon x 2024-05-20T16:04:01.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:04:01 smithi082 ceph-mon[32238]: mgrmap e2: x(active, starting, since 0.00374075s) 2024-05-20T16:04:01.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:04:01 smithi082 ceph-mon[32238]: from='mgr.14100 172.21.15.82:0/3554482394' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-20T16:04:01.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:04:01 smithi082 ceph-mon[32238]: from='mgr.14100 172.21.15.82:0/3554482394' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-20T16:04:01.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:04:01 smithi082 ceph-mon[32238]: from='mgr.14100 172.21.15.82:0/3554482394' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-20T16:04:01.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:04:01 smithi082 ceph-mon[32238]: from='mgr.14100 172.21.15.82:0/3554482394' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-20T16:04:01.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:04:01 smithi082 ceph-mon[32238]: from='mgr.14100 172.21.15.82:0/3554482394' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-20T16:04:01.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:04:01 smithi082 ceph-mon[32238]: Manager daemon x is now available 2024-05-20T16:04:01.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:04:01 smithi082 ceph-mon[32238]: from='mgr.14100 172.21.15.82:0/3554482394' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-20T16:04:01.611 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:04:01 smithi082 ceph-mon[32238]: from='mgr.14100 172.21.15.82:0/3554482394' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-20T16:04:01.611 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:04:01 smithi082 ceph-mon[32238]: from='mgr.14100 172.21.15.82:0/3554482394' entity='mgr.x' 2024-05-20T16:04:01.611 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:04:01 smithi082 ceph-mon[32238]: from='mgr.14100 172.21.15.82:0/3554482394' entity='mgr.x' 2024-05-20T16:04:01.611 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:04:01 smithi082 ceph-mon[32238]: from='mgr.14100 172.21.15.82:0/3554482394' entity='mgr.x' 2024-05-20T16:04:02.860 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:04:02 smithi082 ceph-mon[32238]: mgrmap e3: x(active, since 1.00986s) 2024-05-20T16:04:03.860 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:04:03 smithi082 ceph-mon[32238]: mgrmap e4: x(active, since 2s) 2024-05-20T16:04:06.101 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout 2024-05-20T16:04:06.101 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout { 2024-05-20T16:04:06.101 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "fsid": "59d1600a-16c2-11ef-bc9a-c7b262605968", 2024-05-20T16:04:06.101 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "health": { 2024-05-20T16:04:06.102 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-20T16:04:06.102 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-20T16:04:06.102 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-20T16:04:06.102 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout }, 2024-05-20T16:04:06.102 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-20T16:04:06.102 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-20T16:04:06.102 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout 0 2024-05-20T16:04:06.102 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout ], 2024-05-20T16:04:06.102 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-20T16:04:06.102 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "a" 2024-05-20T16:04:06.102 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout ], 2024-05-20T16:04:06.102 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "quorum_age": 8, 2024-05-20T16:04:06.102 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-20T16:04:06.103 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-20T16:04:06.103 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-05-20T16:04:06.103 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-20T16:04:06.103 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout }, 2024-05-20T16:04:06.103 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-20T16:04:06.103 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-20T16:04:06.103 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-20T16:04:06.103 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-20T16:04:06.103 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-20T16:04:06.111 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-20T16:04:06.111 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-20T16:04:06.111 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-20T16:04:06.111 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout }, 2024-05-20T16:04:06.111 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-20T16:04:06.111 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-20T16:04:06.111 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-20T16:04:06.111 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-20T16:04:06.111 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-20T16:04:06.111 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-20T16:04:06.112 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-20T16:04:06.112 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-20T16:04:06.112 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-20T16:04:06.112 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout }, 2024-05-20T16:04:06.112 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-20T16:04:06.112 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-20T16:04:06.112 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-20T16:04:06.112 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-20T16:04:06.112 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout }, 2024-05-20T16:04:06.112 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-20T16:04:06.112 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "available": false, 2024-05-20T16:04:06.112 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-20T16:04:06.112 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-20T16:04:06.112 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-20T16:04:06.113 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-20T16:04:06.113 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "restful" 2024-05-20T16:04:06.113 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout ], 2024-05-20T16:04:06.113 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-20T16:04:06.113 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout }, 2024-05-20T16:04:06.113 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-20T16:04:06.113 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-20T16:04:06.113 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "modified": "2024-05-20T16:03:46.612160+0000", 2024-05-20T16:04:06.113 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-20T16:04:06.113 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout }, 2024-05-20T16:04:06.113 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-20T16:04:06.113 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout } 2024-05-20T16:04:06.113 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:00.140+0000 7f5235af2700 1 Processor -- start 2024-05-20T16:04:06.114 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:00.140+0000 7f5235af2700 1 -- start start 2024-05-20T16:04:06.114 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:00.140+0000 7f5235af2700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f523007bcc0 0x7f523010b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:04:06.114 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:00.140+0000 7f5235af2700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f523010bbd0 con 0x7f523007bcc0 2024-05-20T16:04:06.114 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:00.140+0000 7f522f7fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f523007bcc0 0x7f523010b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:04:06.114 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:00.140+0000 7f522f7fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f523007bcc0 0x7f523010b5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:35936/0 (socket says 172.21.15.82:35936) 2024-05-20T16:04:06.114 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:00.140+0000 7f522f7fe700 1 -- 172.21.15.82:0/3139712234 learned_addr learned my addr 172.21.15.82:0/3139712234 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:04:06.114 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:00.141+0000 7f522f7fe700 1 -- 172.21.15.82:0/3139712234 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f523010bd50 con 0x7f523007bcc0 2024-05-20T16:04:06.114 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:00.141+0000 7f522f7fe700 1 --2- 172.21.15.82:0/3139712234 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f523007bcc0 0x7f523010b5c0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f52200098a0 tx=0x7f5220009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f1eee03d2bf47eb server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:04:06.114 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:00.141+0000 7f522e7fc700 1 -- 172.21.15.82:0/3139712234 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5220004160 con 0x7f523007bcc0 2024-05-20T16:04:06.114 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:00.141+0000 7f522e7fc700 1 -- 172.21.15.82:0/3139712234 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f5220004300 con 0x7f523007bcc0 2024-05-20T16:04:06.114 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:00.141+0000 7f522e7fc700 1 -- 172.21.15.82:0/3139712234 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5220011b50 con 0x7f523007bcc0 2024-05-20T16:04:06.114 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:00.142+0000 7f5235af2700 1 -- 172.21.15.82:0/3139712234 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f523007bcc0 msgr2=0x7f523010b5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:04:06.115 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:00.142+0000 7f5235af2700 1 --2- 172.21.15.82:0/3139712234 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f523007bcc0 0x7f523010b5c0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f52200098a0 tx=0x7f5220009bb0 comp rx=0 tx=0).stop 2024-05-20T16:04:06.115 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:00.142+0000 7f5235af2700 1 -- 172.21.15.82:0/3139712234 shutdown_connections 2024-05-20T16:04:06.115 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:00.142+0000 7f5235af2700 1 --2- 172.21.15.82:0/3139712234 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f523007bcc0 0x7f523010b5c0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:04:06.115 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:00.142+0000 7f5235af2700 1 -- 172.21.15.82:0/3139712234 >> 172.21.15.82:0/3139712234 conn(0x7f5230103070 msgr2=0x7f52301054c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:04:06.115 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:00.142+0000 7f5235af2700 1 -- 172.21.15.82:0/3139712234 shutdown_connections 2024-05-20T16:04:06.115 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:00.142+0000 7f5235af2700 1 -- 172.21.15.82:0/3139712234 wait complete. 2024-05-20T16:04:06.115 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:00.142+0000 7f5235af2700 1 Processor -- start 2024-05-20T16:04:06.115 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:00.143+0000 7f5235af2700 1 -- start start 2024-05-20T16:04:06.115 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:00.143+0000 7f5235af2700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f523007bcc0 0x7f523019cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:04:06.115 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:00.143+0000 7f5235af2700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f523010ce30 con 0x7f523007bcc0 2024-05-20T16:04:06.115 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:00.143+0000 7f522f7fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f523007bcc0 0x7f523019cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:04:06.115 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:00.143+0000 7f522f7fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f523007bcc0 0x7f523019cd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:35948/0 (socket says 172.21.15.82:35948) 2024-05-20T16:04:06.115 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:00.143+0000 7f522f7fe700 1 -- 172.21.15.82:0/3420475560 learned_addr learned my addr 172.21.15.82:0/3420475560 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:04:06.115 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:00.143+0000 7f522f7fe700 1 -- 172.21.15.82:0/3420475560 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f523019d2b0 con 0x7f523007bcc0 2024-05-20T16:04:06.115 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:00.143+0000 7f522f7fe700 1 --2- 172.21.15.82:0/3420475560 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f523007bcc0 0x7f523019cd70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f5220011850 tx=0x7f5220011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:04:06.116 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:00.144+0000 7f522cff9700 1 -- 172.21.15.82:0/3420475560 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5220003af0 con 0x7f523007bcc0 2024-05-20T16:04:06.116 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:00.144+0000 7f522cff9700 1 -- 172.21.15.82:0/3420475560 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f5220020070 con 0x7f523007bcc0 2024-05-20T16:04:06.116 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:00.144+0000 7f5235af2700 1 -- 172.21.15.82:0/3420475560 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f523019d540 con 0x7f523007bcc0 2024-05-20T16:04:06.116 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:00.144+0000 7f522cff9700 1 -- 172.21.15.82:0/3420475560 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f52200195a0 con 0x7f523007bcc0 2024-05-20T16:04:06.116 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:00.144+0000 7f5235af2700 1 -- 172.21.15.82:0/3420475560 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52301a0150 con 0x7f523007bcc0 2024-05-20T16:04:06.116 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:00.144+0000 7f522cff9700 1 -- 172.21.15.82:0/3420475560 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f5220019be0 con 0x7f523007bcc0 2024-05-20T16:04:06.116 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:00.144+0000 7f522cff9700 1 -- 172.21.15.82:0/3420475560 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f5220011e00 con 0x7f523007bcc0 2024-05-20T16:04:06.116 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:00.145+0000 7f5235af2700 1 -- 172.21.15.82:0/3420475560 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5230063ca0 con 0x7f523007bcc0 2024-05-20T16:04:06.116 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:00.147+0000 7f522cff9700 1 -- 172.21.15.82:0/3420475560 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f5220003c90 con 0x7f523007bcc0 2024-05-20T16:04:06.116 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:00.241+0000 7f5235af2700 1 -- 172.21.15.82:0/3420475560 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f52301a0320 con 0x7f523007bcc0 2024-05-20T16:04:06.116 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:00.241+0000 7f522cff9700 1 -- 172.21.15.82:0/3420475560 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f5220025460 con 0x7f523007bcc0 2024-05-20T16:04:06.116 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:00.242+0000 7f5235af2700 1 -- 172.21.15.82:0/3420475560 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f523007bcc0 msgr2=0x7f523019cd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:04:06.116 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:00.242+0000 7f5235af2700 1 --2- 172.21.15.82:0/3420475560 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f523007bcc0 0x7f523019cd70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f5220011850 tx=0x7f5220011c10 comp rx=0 tx=0).stop 2024-05-20T16:04:06.116 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:00.243+0000 7f5235af2700 1 -- 172.21.15.82:0/3420475560 shutdown_connections 2024-05-20T16:04:06.117 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:00.243+0000 7f5235af2700 1 --2- 172.21.15.82:0/3420475560 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f523007bcc0 0x7f523019cd70 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:04:06.117 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:00.243+0000 7f5235af2700 1 -- 172.21.15.82:0/3420475560 >> 172.21.15.82:0/3420475560 conn(0x7f5230103070 msgr2=0x7f5230104920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:04:06.117 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:00.243+0000 7f5235af2700 1 -- 172.21.15.82:0/3420475560 shutdown_connections 2024-05-20T16:04:06.117 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:00.243+0000 7f5235af2700 1 -- 172.21.15.82:0/3420475560 wait complete. 2024-05-20T16:04:06.117 INFO:teuthology.orchestra.run.smithi082.stdout:mgr not available, waiting (2/15)... 2024-05-20T16:04:09.110 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:04:08 smithi082 ceph-mon[32238]: from='client.? 172.21.15.82:0/134581828' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-20T16:04:09.357 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout 2024-05-20T16:04:09.358 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout { 2024-05-20T16:04:09.358 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "fsid": "59d1600a-16c2-11ef-bc9a-c7b262605968", 2024-05-20T16:04:09.358 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "health": { 2024-05-20T16:04:09.358 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-20T16:04:09.358 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-20T16:04:09.358 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-20T16:04:09.358 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout }, 2024-05-20T16:04:09.358 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-20T16:04:09.358 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-20T16:04:09.358 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout 0 2024-05-20T16:04:09.358 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout ], 2024-05-20T16:04:09.358 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-20T16:04:09.359 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "a" 2024-05-20T16:04:09.359 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout ], 2024-05-20T16:04:09.359 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "quorum_age": 17, 2024-05-20T16:04:09.359 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-20T16:04:09.359 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-20T16:04:09.359 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-05-20T16:04:09.359 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-20T16:04:09.359 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout }, 2024-05-20T16:04:09.359 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-20T16:04:09.359 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-20T16:04:09.359 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-20T16:04:09.359 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-20T16:04:09.359 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-20T16:04:09.367 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-20T16:04:09.367 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-20T16:04:09.367 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-20T16:04:09.368 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout }, 2024-05-20T16:04:09.368 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-20T16:04:09.368 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-20T16:04:09.368 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-20T16:04:09.368 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-20T16:04:09.368 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-20T16:04:09.368 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-20T16:04:09.368 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-20T16:04:09.368 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-20T16:04:09.368 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-20T16:04:09.368 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout }, 2024-05-20T16:04:09.368 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-20T16:04:09.368 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-20T16:04:09.368 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-20T16:04:09.369 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-20T16:04:09.369 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout }, 2024-05-20T16:04:09.369 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-20T16:04:09.369 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "available": true, 2024-05-20T16:04:09.369 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-20T16:04:09.369 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-20T16:04:09.369 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-20T16:04:09.369 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-20T16:04:09.369 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "restful" 2024-05-20T16:04:09.369 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout ], 2024-05-20T16:04:09.369 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-20T16:04:09.369 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout }, 2024-05-20T16:04:09.369 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-20T16:04:09.370 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-20T16:04:09.370 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "modified": "2024-05-20T16:03:46.612160+0000", 2024-05-20T16:04:09.370 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-20T16:04:09.370 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout }, 2024-05-20T16:04:09.370 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-20T16:04:09.370 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout } 2024-05-20T16:04:09.370 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:08.531+0000 7f3f5c587700 1 Processor -- start 2024-05-20T16:04:09.370 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:08.531+0000 7f3f5c587700 1 -- start start 2024-05-20T16:04:09.370 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:08.531+0000 7f3f5c587700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3f5407bcc0 0x7f3f5410b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:04:09.370 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:08.531+0000 7f3f5c587700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f3f5410bbd0 con 0x7f3f5407bcc0 2024-05-20T16:04:09.370 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:08.531+0000 7f3f5a323700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3f5407bcc0 0x7f3f5410b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:04:09.370 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:08.531+0000 7f3f5a323700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3f5407bcc0 0x7f3f5410b5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:36036/0 (socket says 172.21.15.82:36036) 2024-05-20T16:04:09.370 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:08.531+0000 7f3f5a323700 1 -- 172.21.15.82:0/3432759110 learned_addr learned my addr 172.21.15.82:0/3432759110 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:04:09.371 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:08.532+0000 7f3f5a323700 1 -- 172.21.15.82:0/3432759110 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f5410bd50 con 0x7f3f5407bcc0 2024-05-20T16:04:09.371 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:08.532+0000 7f3f5a323700 1 --2- 172.21.15.82:0/3432759110 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3f5407bcc0 0x7f3f5410b5c0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f3f500098a0 tx=0x7f3f50009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3348968e78072898 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:04:09.371 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:08.532+0000 7f3f59321700 1 -- 172.21.15.82:0/3432759110 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3f50004160 con 0x7f3f5407bcc0 2024-05-20T16:04:09.371 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:08.532+0000 7f3f59321700 1 -- 172.21.15.82:0/3432759110 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f3f50004300 con 0x7f3f5407bcc0 2024-05-20T16:04:09.371 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:08.532+0000 7f3f59321700 1 -- 172.21.15.82:0/3432759110 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3f50011b50 con 0x7f3f5407bcc0 2024-05-20T16:04:09.371 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:08.533+0000 7f3f5c587700 1 -- 172.21.15.82:0/3432759110 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3f5407bcc0 msgr2=0x7f3f5410b5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:04:09.371 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:08.533+0000 7f3f5c587700 1 --2- 172.21.15.82:0/3432759110 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3f5407bcc0 0x7f3f5410b5c0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f3f500098a0 tx=0x7f3f50009bb0 comp rx=0 tx=0).stop 2024-05-20T16:04:09.371 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:08.533+0000 7f3f5c587700 1 -- 172.21.15.82:0/3432759110 shutdown_connections 2024-05-20T16:04:09.371 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:08.533+0000 7f3f5c587700 1 --2- 172.21.15.82:0/3432759110 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3f5407bcc0 0x7f3f5410b5c0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:04:09.371 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:08.533+0000 7f3f5c587700 1 -- 172.21.15.82:0/3432759110 >> 172.21.15.82:0/3432759110 conn(0x7f3f54103070 msgr2=0x7f3f541054c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:04:09.371 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:08.533+0000 7f3f5c587700 1 -- 172.21.15.82:0/3432759110 shutdown_connections 2024-05-20T16:04:09.371 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:08.533+0000 7f3f5c587700 1 -- 172.21.15.82:0/3432759110 wait complete. 2024-05-20T16:04:09.371 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:08.534+0000 7f3f5c587700 1 Processor -- start 2024-05-20T16:04:09.371 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:08.534+0000 7f3f5c587700 1 -- start start 2024-05-20T16:04:09.372 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:08.534+0000 7f3f5c587700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3f5407bcc0 0x7f3f5419cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:04:09.372 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:08.534+0000 7f3f5c587700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f3f5410ce30 con 0x7f3f5407bcc0 2024-05-20T16:04:09.372 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:08.534+0000 7f3f5a323700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3f5407bcc0 0x7f3f5419cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:04:09.372 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:08.534+0000 7f3f5a323700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3f5407bcc0 0x7f3f5419cd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:36044/0 (socket says 172.21.15.82:36044) 2024-05-20T16:04:09.372 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:08.534+0000 7f3f5a323700 1 -- 172.21.15.82:0/134581828 learned_addr learned my addr 172.21.15.82:0/134581828 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:04:09.372 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:08.534+0000 7f3f5a323700 1 -- 172.21.15.82:0/134581828 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f5419d2b0 con 0x7f3f5407bcc0 2024-05-20T16:04:09.372 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:08.534+0000 7f3f5a323700 1 --2- 172.21.15.82:0/134581828 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3f5407bcc0 0x7f3f5419cd70 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f3f50011850 tx=0x7f3f50011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:04:09.372 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:08.535+0000 7f3f4b7fe700 1 -- 172.21.15.82:0/134581828 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3f50003af0 con 0x7f3f5407bcc0 2024-05-20T16:04:09.372 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:08.535+0000 7f3f5c587700 1 -- 172.21.15.82:0/134581828 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f5419d540 con 0x7f3f5407bcc0 2024-05-20T16:04:09.372 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:08.535+0000 7f3f4b7fe700 1 -- 172.21.15.82:0/134581828 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f3f50020070 con 0x7f3f5407bcc0 2024-05-20T16:04:09.372 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:08.535+0000 7f3f4b7fe700 1 -- 172.21.15.82:0/134581828 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3f500195a0 con 0x7f3f5407bcc0 2024-05-20T16:04:09.372 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:08.535+0000 7f3f5c587700 1 -- 172.21.15.82:0/134581828 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f541a0150 con 0x7f3f5407bcc0 2024-05-20T16:04:09.372 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:08.536+0000 7f3f4b7fe700 1 -- 172.21.15.82:0/134581828 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 4) ==== 46276+0+0 (secure 0 0 0) 0x7f3f500198a0 con 0x7f3f5407bcc0 2024-05-20T16:04:09.372 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:08.536+0000 7f3f4b7fe700 1 --2- 172.21.15.82:0/134581828 >> [v2:172.21.15.82:6800/1959459348,v1:172.21.15.82:6801/1959459348] conn(0x7f3f400390c0 0x7f3f4003b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:04:09.373 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:08.536+0000 7f3f4b7fe700 1 -- 172.21.15.82:0/134581828 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f3f50011e00 con 0x7f3f5407bcc0 2024-05-20T16:04:09.373 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:08.536+0000 7f3f5c587700 1 -- 172.21.15.82:0/134581828 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f54063ca0 con 0x7f3f5407bcc0 2024-05-20T16:04:09.373 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:08.536+0000 7f3f59b22700 1 --2- 172.21.15.82:0/134581828 >> [v2:172.21.15.82:6800/1959459348,v1:172.21.15.82:6801/1959459348] conn(0x7f3f400390c0 0x7f3f4003b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:04:09.373 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:08.536+0000 7f3f59b22700 1 --2- 172.21.15.82:0/134581828 >> [v2:172.21.15.82:6800/1959459348,v1:172.21.15.82:6801/1959459348] conn(0x7f3f400390c0 0x7f3f4003b5d0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f3f44006fd0 tx=0x7f3f44006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:04:09.373 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:08.542+0000 7f3f4b7fe700 1 -- 172.21.15.82:0/134581828 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f3f50003c90 con 0x7f3f5407bcc0 2024-05-20T16:04:09.373 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:08.828+0000 7f3f5c587700 1 -- 172.21.15.82:0/134581828 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f3f541a0720 con 0x7f3f5407bcc0 2024-05-20T16:04:09.373 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:08.829+0000 7f3f4b7fe700 1 -- 172.21.15.82:0/134581828 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f3f50023530 con 0x7f3f5407bcc0 2024-05-20T16:04:09.373 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:08.831+0000 7f3f5c587700 1 -- 172.21.15.82:0/134581828 >> [v2:172.21.15.82:6800/1959459348,v1:172.21.15.82:6801/1959459348] conn(0x7f3f400390c0 msgr2=0x7f3f4003b5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:04:09.373 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:08.831+0000 7f3f5c587700 1 --2- 172.21.15.82:0/134581828 >> [v2:172.21.15.82:6800/1959459348,v1:172.21.15.82:6801/1959459348] conn(0x7f3f400390c0 0x7f3f4003b5d0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f3f44006fd0 tx=0x7f3f44006e40 comp rx=0 tx=0).stop 2024-05-20T16:04:09.373 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:08.831+0000 7f3f5c587700 1 -- 172.21.15.82:0/134581828 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3f5407bcc0 msgr2=0x7f3f5419cd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:04:09.373 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:08.831+0000 7f3f5c587700 1 --2- 172.21.15.82:0/134581828 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3f5407bcc0 0x7f3f5419cd70 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f3f50011850 tx=0x7f3f50011c10 comp rx=0 tx=0).stop 2024-05-20T16:04:09.373 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:08.831+0000 7f3f5c587700 1 -- 172.21.15.82:0/134581828 shutdown_connections 2024-05-20T16:04:09.373 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:08.831+0000 7f3f5c587700 1 --2- 172.21.15.82:0/134581828 >> [v2:172.21.15.82:6800/1959459348,v1:172.21.15.82:6801/1959459348] conn(0x7f3f400390c0 0x7f3f4003b5d0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:04:09.373 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:08.831+0000 7f3f5c587700 1 --2- 172.21.15.82:0/134581828 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3f5407bcc0 0x7f3f5419cd70 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:04:09.374 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:08.831+0000 7f3f5c587700 1 -- 172.21.15.82:0/134581828 >> 172.21.15.82:0/134581828 conn(0x7f3f54103070 msgr2=0x7f3f54104920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:04:09.374 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:08.831+0000 7f3f5c587700 1 -- 172.21.15.82:0/134581828 shutdown_connections 2024-05-20T16:04:09.374 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:08.831+0000 7f3f5c587700 1 -- 172.21.15.82:0/134581828 wait complete. 2024-05-20T16:04:09.374 INFO:teuthology.orchestra.run.smithi082.stdout:mgr is available 2024-05-20T16:04:10.360 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:04:10 smithi082 ceph-mon[32238]: from='client.? 172.21.15.82:0/1429849811' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-05-20T16:04:10.928 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout 2024-05-20T16:04:10.935 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout [global] 2024-05-20T16:04:10.935 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout fsid = 59d1600a-16c2-11ef-bc9a-c7b262605968 2024-05-20T16:04:10.935 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-05-20T16:04:10.935 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-05-20T16:04:10.935 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-05-20T16:04:10.935 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-05-20T16:04:10.935 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout 2024-05-20T16:04:10.935 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout [mgr] 2024-05-20T16:04:10.935 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-05-20T16:04:10.935 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout 2024-05-20T16:04:10.935 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout [osd] 2024-05-20T16:04:10.935 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-05-20T16:04:10.935 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-05-20T16:04:10.936 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-05-20T16:04:10.936 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-05-20T16:04:10.936 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-05-20T16:04:10.936 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:09.783+0000 7f0bd145b700 1 Processor -- start 2024-05-20T16:04:10.936 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:09.783+0000 7f0bd145b700 1 -- start start 2024-05-20T16:04:10.936 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:09.783+0000 7f0bd145b700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0bcc108ae0 0x7f0bcc108ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:04:10.936 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:09.783+0000 7f0bd145b700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f0bcc1094e0 con 0x7f0bcc108ae0 2024-05-20T16:04:10.936 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:09.783+0000 7f0bcaffd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0bcc108ae0 0x7f0bcc108ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:04:10.936 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:09.784+0000 7f0bcaffd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0bcc108ae0 0x7f0bcc108ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:33858/0 (socket says 172.21.15.82:33858) 2024-05-20T16:04:10.936 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:09.784+0000 7f0bcaffd700 1 -- 172.21.15.82:0/2496217031 learned_addr learned my addr 172.21.15.82:0/2496217031 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:04:10.936 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:09.784+0000 7f0bcaffd700 1 -- 172.21.15.82:0/2496217031 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bcc109c10 con 0x7f0bcc108ae0 2024-05-20T16:04:10.936 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:09.784+0000 7f0bcaffd700 1 --2- 172.21.15.82:0/2496217031 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0bcc108ae0 0x7f0bcc108ef0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f0bbc0098a0 tx=0x7f0bbc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8295e815f51a6834 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:04:10.936 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:09.784+0000 7f0bc9ffb700 1 -- 172.21.15.82:0/2496217031 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0bbc004160 con 0x7f0bcc108ae0 2024-05-20T16:04:10.936 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:09.784+0000 7f0bc9ffb700 1 -- 172.21.15.82:0/2496217031 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f0bbc004300 con 0x7f0bcc108ae0 2024-05-20T16:04:10.937 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:09.784+0000 7f0bc9ffb700 1 -- 172.21.15.82:0/2496217031 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0bbc011b50 con 0x7f0bcc108ae0 2024-05-20T16:04:10.937 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:09.785+0000 7f0bd145b700 1 -- 172.21.15.82:0/2496217031 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0bcc108ae0 msgr2=0x7f0bcc108ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:04:10.937 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:09.785+0000 7f0bd145b700 1 --2- 172.21.15.82:0/2496217031 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0bcc108ae0 0x7f0bcc108ef0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f0bbc0098a0 tx=0x7f0bbc009bb0 comp rx=0 tx=0).stop 2024-05-20T16:04:10.937 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:09.785+0000 7f0bd145b700 1 -- 172.21.15.82:0/2496217031 shutdown_connections 2024-05-20T16:04:10.937 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:09.785+0000 7f0bd145b700 1 --2- 172.21.15.82:0/2496217031 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0bcc108ae0 0x7f0bcc108ef0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:04:10.937 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:09.785+0000 7f0bd145b700 1 -- 172.21.15.82:0/2496217031 >> 172.21.15.82:0/2496217031 conn(0x7f0bcc1041c0 msgr2=0x7f0bcc106610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:04:10.937 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:09.785+0000 7f0bd145b700 1 -- 172.21.15.82:0/2496217031 shutdown_connections 2024-05-20T16:04:10.937 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:09.785+0000 7f0bd145b700 1 -- 172.21.15.82:0/2496217031 wait complete. 2024-05-20T16:04:10.937 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:09.786+0000 7f0bd145b700 1 Processor -- start 2024-05-20T16:04:10.937 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:09.786+0000 7f0bd145b700 1 -- start start 2024-05-20T16:04:10.937 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:09.786+0000 7f0bd145b700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0bcc108ae0 0x7f0bcc19cee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:04:10.937 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:09.786+0000 7f0bd145b700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f0bcc10cfc0 con 0x7f0bcc108ae0 2024-05-20T16:04:10.937 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:09.786+0000 7f0bcaffd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0bcc108ae0 0x7f0bcc19cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:04:10.937 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:09.786+0000 7f0bcaffd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0bcc108ae0 0x7f0bcc19cee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:33866/0 (socket says 172.21.15.82:33866) 2024-05-20T16:04:10.938 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:09.786+0000 7f0bcaffd700 1 -- 172.21.15.82:0/1429849811 learned_addr learned my addr 172.21.15.82:0/1429849811 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:04:10.938 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:09.786+0000 7f0bcaffd700 1 -- 172.21.15.82:0/1429849811 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bcc19d420 con 0x7f0bcc108ae0 2024-05-20T16:04:10.938 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:09.787+0000 7f0bcaffd700 1 --2- 172.21.15.82:0/1429849811 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0bcc108ae0 0x7f0bcc19cee0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f0bbc0046e0 tx=0x7f0bbc011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:04:10.938 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:09.787+0000 7f0bb3fff700 1 -- 172.21.15.82:0/1429849811 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0bbc0039e0 con 0x7f0bcc108ae0 2024-05-20T16:04:10.938 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:09.787+0000 7f0bb3fff700 1 -- 172.21.15.82:0/1429849811 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f0bbc003b80 con 0x7f0bcc108ae0 2024-05-20T16:04:10.938 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:09.787+0000 7f0bd145b700 1 -- 172.21.15.82:0/1429849811 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0bcc19d6b0 con 0x7f0bcc108ae0 2024-05-20T16:04:10.938 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:09.787+0000 7f0bb3fff700 1 -- 172.21.15.82:0/1429849811 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0bbc024650 con 0x7f0bcc108ae0 2024-05-20T16:04:10.938 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:09.787+0000 7f0bd145b700 1 -- 172.21.15.82:0/1429849811 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0bcc1a02c0 con 0x7f0bcc108ae0 2024-05-20T16:04:10.938 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:09.788+0000 7f0bb3fff700 1 -- 172.21.15.82:0/1429849811 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 4) ==== 46276+0+0 (secure 0 0 0) 0x7f0bbc022020 con 0x7f0bcc108ae0 2024-05-20T16:04:10.938 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:09.788+0000 7f0bb3fff700 1 --2- 172.21.15.82:0/1429849811 >> [v2:172.21.15.82:6800/1959459348,v1:172.21.15.82:6801/1959459348] conn(0x7f0bb4039110 0x7f0bb403b620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:04:10.938 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:09.788+0000 7f0bd145b700 1 -- 172.21.15.82:0/1429849811 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0bcc063ca0 con 0x7f0bcc108ae0 2024-05-20T16:04:10.938 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:09.788+0000 7f0bb3fff700 1 -- 172.21.15.82:0/1429849811 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f0bbc011e00 con 0x7f0bcc108ae0 2024-05-20T16:04:10.938 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:09.788+0000 7f0bca7fc700 1 --2- 172.21.15.82:0/1429849811 >> [v2:172.21.15.82:6800/1959459348,v1:172.21.15.82:6801/1959459348] conn(0x7f0bb4039110 0x7f0bb403b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:04:10.939 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:09.788+0000 7f0bca7fc700 1 --2- 172.21.15.82:0/1429849811 >> [v2:172.21.15.82:6800/1959459348,v1:172.21.15.82:6801/1959459348] conn(0x7f0bb4039110 0x7f0bb403b620 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f0bc0006fd0 tx=0x7f0bc0006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:04:10.939 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:09.793+0000 7f0bb3fff700 1 -- 172.21.15.82:0/1429849811 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f0bbc003d30 con 0x7f0bcc108ae0 2024-05-20T16:04:10.939 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:09.984+0000 7f0bd145b700 1 -- 172.21.15.82:0/1429849811 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f0bcc1a1a90 con 0x7f0bcc108ae0 2024-05-20T16:04:10.939 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:09.985+0000 7f0bb3fff700 1 -- 172.21.15.82:0/1429849811 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+373 (secure 0 0 0) 0x7f0bbc04c9f0 con 0x7f0bcc108ae0 2024-05-20T16:04:10.939 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:09.987+0000 7f0bd145b700 1 -- 172.21.15.82:0/1429849811 >> [v2:172.21.15.82:6800/1959459348,v1:172.21.15.82:6801/1959459348] conn(0x7f0bb4039110 msgr2=0x7f0bb403b620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:04:10.939 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:09.987+0000 7f0bd145b700 1 --2- 172.21.15.82:0/1429849811 >> [v2:172.21.15.82:6800/1959459348,v1:172.21.15.82:6801/1959459348] conn(0x7f0bb4039110 0x7f0bb403b620 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f0bc0006fd0 tx=0x7f0bc0006e40 comp rx=0 tx=0).stop 2024-05-20T16:04:10.939 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:09.987+0000 7f0bd145b700 1 -- 172.21.15.82:0/1429849811 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0bcc108ae0 msgr2=0x7f0bcc19cee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:04:10.939 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:09.987+0000 7f0bd145b700 1 --2- 172.21.15.82:0/1429849811 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0bcc108ae0 0x7f0bcc19cee0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f0bbc0046e0 tx=0x7f0bbc011c10 comp rx=0 tx=0).stop 2024-05-20T16:04:10.939 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:09.987+0000 7f0bd145b700 1 -- 172.21.15.82:0/1429849811 shutdown_connections 2024-05-20T16:04:10.939 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:09.987+0000 7f0bd145b700 1 --2- 172.21.15.82:0/1429849811 >> [v2:172.21.15.82:6800/1959459348,v1:172.21.15.82:6801/1959459348] conn(0x7f0bb4039110 0x7f0bb403b620 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:04:10.939 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:09.987+0000 7f0bd145b700 1 --2- 172.21.15.82:0/1429849811 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0bcc108ae0 0x7f0bcc19cee0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:04:10.939 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:09.987+0000 7f0bd145b700 1 -- 172.21.15.82:0/1429849811 >> 172.21.15.82:0/1429849811 conn(0x7f0bcc1041c0 msgr2=0x7f0bcc105b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:04:10.939 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:09.988+0000 7f0bd145b700 1 -- 172.21.15.82:0/1429849811 shutdown_connections 2024-05-20T16:04:10.939 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:09.988+0000 7f0bd145b700 1 -- 172.21.15.82:0/1429849811 wait complete. 2024-05-20T16:04:10.939 INFO:teuthology.orchestra.run.smithi082.stdout:Enabling cephadm module... 2024-05-20T16:04:12.564 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:04:12 smithi082 ceph-mon[32238]: from='client.? 172.21.15.82:0/1902575948' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-05-20T16:04:12.564 INFO:journalctl@ceph.mgr.x.smithi082.stdout:May 20 16:04:12 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mgr-x[32467]: ignoring --setuser ceph since I am not root 2024-05-20T16:04:12.564 INFO:journalctl@ceph.mgr.x.smithi082.stdout:May 20 16:04:12 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mgr-x[32467]: ignoring --setgroup ceph since I am not root 2024-05-20T16:04:12.564 INFO:journalctl@ceph.mgr.x.smithi082.stdout:May 20 16:04:12 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mgr-x[32467]: 2024-05-20T16:04:12.457+0000 7efe2a45e200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-20T16:04:12.839 INFO:journalctl@ceph.mgr.x.smithi082.stdout:May 20 16:04:12 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mgr-x[32467]: 2024-05-20T16:04:12.647+0000 7efe2a45e200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-20T16:04:12.839 INFO:journalctl@ceph.mgr.x.smithi082.stdout:May 20 16:04:12 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mgr-x[32467]: 2024-05-20T16:04:12.752+0000 7efe2a45e200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-20T16:04:12.846 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:11.949+0000 7f84db1f1700 1 Processor -- start 2024-05-20T16:04:12.846 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:11.949+0000 7f84db1f1700 1 -- start start 2024-05-20T16:04:12.846 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:11.950+0000 7f84db1f1700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f84d4107990 0x7f84d4107da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:04:12.846 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:11.950+0000 7f84db1f1700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f84d4108390 con 0x7f84d4107990 2024-05-20T16:04:12.846 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:11.950+0000 7f84d8f8d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f84d4107990 0x7f84d4107da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:04:12.846 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:11.950+0000 7f84d8f8d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f84d4107990 0x7f84d4107da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:33868/0 (socket says 172.21.15.82:33868) 2024-05-20T16:04:12.846 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:11.950+0000 7f84d8f8d700 1 -- 172.21.15.82:0/678713873 learned_addr learned my addr 172.21.15.82:0/678713873 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:04:12.846 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:11.950+0000 7f84d8f8d700 1 -- 172.21.15.82:0/678713873 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84d4108ac0 con 0x7f84d4107990 2024-05-20T16:04:12.846 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:11.950+0000 7f84d8f8d700 1 --2- 172.21.15.82:0/678713873 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f84d4107990 0x7f84d4107da0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f84d0009e80 tx=0x7f84d000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7718a1e2083169f server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:04:12.846 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:11.950+0000 7f84cb7fe700 1 -- 172.21.15.82:0/678713873 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f84d0017070 con 0x7f84d4107990 2024-05-20T16:04:12.846 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:11.950+0000 7f84cb7fe700 1 -- 172.21.15.82:0/678713873 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f84d000b8c0 con 0x7f84d4107990 2024-05-20T16:04:12.847 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:11.950+0000 7f84cb7fe700 1 -- 172.21.15.82:0/678713873 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f84d00045c0 con 0x7f84d4107990 2024-05-20T16:04:12.847 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:11.951+0000 7f84db1f1700 1 -- 172.21.15.82:0/678713873 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f84d4107990 msgr2=0x7f84d4107da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:04:12.847 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:11.951+0000 7f84db1f1700 1 --2- 172.21.15.82:0/678713873 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f84d4107990 0x7f84d4107da0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f84d0009e80 tx=0x7f84d000b330 comp rx=0 tx=0).stop 2024-05-20T16:04:12.847 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:11.951+0000 7f84db1f1700 1 -- 172.21.15.82:0/678713873 shutdown_connections 2024-05-20T16:04:12.847 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:11.951+0000 7f84db1f1700 1 --2- 172.21.15.82:0/678713873 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f84d4107990 0x7f84d4107da0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:04:12.847 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:11.951+0000 7f84db1f1700 1 -- 172.21.15.82:0/678713873 >> 172.21.15.82:0/678713873 conn(0x7f84d4103070 msgr2=0x7f84d41054c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:04:12.847 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:11.951+0000 7f84db1f1700 1 -- 172.21.15.82:0/678713873 shutdown_connections 2024-05-20T16:04:12.847 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:11.951+0000 7f84db1f1700 1 -- 172.21.15.82:0/678713873 wait complete. 2024-05-20T16:04:12.847 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:11.952+0000 7f84db1f1700 1 Processor -- start 2024-05-20T16:04:12.847 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:11.952+0000 7f84db1f1700 1 -- start start 2024-05-20T16:04:12.847 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:11.952+0000 7f84db1f1700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f84d4107990 0x7f84d419ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:04:12.848 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:11.952+0000 7f84db1f1700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f84d407b140 con 0x7f84d4107990 2024-05-20T16:04:12.848 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:11.952+0000 7f84d8f8d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f84d4107990 0x7f84d419ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:04:12.848 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:11.952+0000 7f84d8f8d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f84d4107990 0x7f84d419ab90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:33870/0 (socket says 172.21.15.82:33870) 2024-05-20T16:04:12.848 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:11.952+0000 7f84d8f8d700 1 -- 172.21.15.82:0/1902575948 learned_addr learned my addr 172.21.15.82:0/1902575948 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:04:12.848 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:11.953+0000 7f84d8f8d700 1 -- 172.21.15.82:0/1902575948 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84d419b0d0 con 0x7f84d4107990 2024-05-20T16:04:12.848 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:11.953+0000 7f84d8f8d700 1 --2- 172.21.15.82:0/1902575948 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f84d4107990 0x7f84d419ab90 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f84d0009e50 tx=0x7f84d000bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:04:12.848 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:11.953+0000 7f84c9ffb700 1 -- 172.21.15.82:0/1902575948 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f84d0020020 con 0x7f84d4107990 2024-05-20T16:04:12.848 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:11.953+0000 7f84c9ffb700 1 -- 172.21.15.82:0/1902575948 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f84d0012430 con 0x7f84d4107990 2024-05-20T16:04:12.848 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:11.953+0000 7f84c9ffb700 1 -- 172.21.15.82:0/1902575948 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f84d0017040 con 0x7f84d4107990 2024-05-20T16:04:12.848 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:11.953+0000 7f84db1f1700 1 -- 172.21.15.82:0/1902575948 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84d419b360 con 0x7f84d4107990 2024-05-20T16:04:12.848 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:11.953+0000 7f84db1f1700 1 -- 172.21.15.82:0/1902575948 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84d419df70 con 0x7f84d4107990 2024-05-20T16:04:12.848 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:11.954+0000 7f84db1f1700 1 -- 172.21.15.82:0/1902575948 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84d4063ca0 con 0x7f84d4107990 2024-05-20T16:04:12.848 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:11.954+0000 7f84c9ffb700 1 -- 172.21.15.82:0/1902575948 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 4) ==== 46276+0+0 (secure 0 0 0) 0x7f84d0023020 con 0x7f84d4107990 2024-05-20T16:04:12.848 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:11.954+0000 7f84c9ffb700 1 --2- 172.21.15.82:0/1902575948 >> [v2:172.21.15.82:6800/1959459348,v1:172.21.15.82:6801/1959459348] conn(0x7f84bc03b2d0 0x7f84bc03d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:04:12.848 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:11.954+0000 7f84c9ffb700 1 -- 172.21.15.82:0/1902575948 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f84d000f050 con 0x7f84d4107990 2024-05-20T16:04:12.849 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:11.954+0000 7f84cbfff700 1 --2- 172.21.15.82:0/1902575948 >> [v2:172.21.15.82:6800/1959459348,v1:172.21.15.82:6801/1959459348] conn(0x7f84bc03b2d0 0x7f84bc03d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:04:12.849 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:11.955+0000 7f84cbfff700 1 --2- 172.21.15.82:0/1902575948 >> [v2:172.21.15.82:6800/1959459348,v1:172.21.15.82:6801/1959459348] conn(0x7f84bc03b2d0 0x7f84bc03d7e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f84c4006fd0 tx=0x7f84c4006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:04:12.849 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:11.959+0000 7f84c9ffb700 1 -- 172.21.15.82:0/1902575948 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f84d002a020 con 0x7f84d4107990 2024-05-20T16:04:12.849 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:12.208+0000 7f84db1f1700 1 -- 172.21.15.82:0/1902575948 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7f84d4193da0 con 0x7f84d4107990 2024-05-20T16:04:12.849 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:12.259+0000 7f84c9ffb700 1 -- 172.21.15.82:0/1902575948 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7f84d4193da0 con 0x7f84d4107990 2024-05-20T16:04:12.849 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:12.262+0000 7f84c9ffb700 1 -- 172.21.15.82:0/1902575948 <== mon.0 v2:172.21.15.82:3300/0 8 ==== mgrmap(e 5) ==== 46287+0+0 (secure 0 0 0) 0x7f84d00125e0 con 0x7f84d4107990 2024-05-20T16:04:12.849 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:12.263+0000 7f84db1f1700 1 -- 172.21.15.82:0/1902575948 >> [v2:172.21.15.82:6800/1959459348,v1:172.21.15.82:6801/1959459348] conn(0x7f84bc03b2d0 msgr2=0x7f84bc03d7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:04:12.849 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:12.263+0000 7f84db1f1700 1 --2- 172.21.15.82:0/1902575948 >> [v2:172.21.15.82:6800/1959459348,v1:172.21.15.82:6801/1959459348] conn(0x7f84bc03b2d0 0x7f84bc03d7e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f84c4006fd0 tx=0x7f84c4006e40 comp rx=0 tx=0).stop 2024-05-20T16:04:12.849 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:12.263+0000 7f84db1f1700 1 -- 172.21.15.82:0/1902575948 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f84d4107990 msgr2=0x7f84d419ab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:04:12.849 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:12.263+0000 7f84db1f1700 1 --2- 172.21.15.82:0/1902575948 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f84d4107990 0x7f84d419ab90 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f84d0009e50 tx=0x7f84d000bd50 comp rx=0 tx=0).stop 2024-05-20T16:04:12.849 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:12.263+0000 7f84db1f1700 1 -- 172.21.15.82:0/1902575948 shutdown_connections 2024-05-20T16:04:12.849 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:12.263+0000 7f84db1f1700 1 --2- 172.21.15.82:0/1902575948 >> [v2:172.21.15.82:6800/1959459348,v1:172.21.15.82:6801/1959459348] conn(0x7f84bc03b2d0 0x7f84bc03d7e0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:04:12.849 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:12.263+0000 7f84db1f1700 1 --2- 172.21.15.82:0/1902575948 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f84d4107990 0x7f84d419ab90 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:04:12.849 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:12.263+0000 7f84db1f1700 1 -- 172.21.15.82:0/1902575948 >> 172.21.15.82:0/1902575948 conn(0x7f84d4103070 msgr2=0x7f84d4190e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:04:12.850 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:12.263+0000 7f84db1f1700 1 -- 172.21.15.82:0/1902575948 shutdown_connections 2024-05-20T16:04:12.850 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:12.263+0000 7f84db1f1700 1 -- 172.21.15.82:0/1902575948 wait complete. 2024-05-20T16:04:13.109 INFO:journalctl@ceph.mgr.x.smithi082.stdout:May 20 16:04:13 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mgr-x[32467]: 2024-05-20T16:04:13.082+0000 7efe2a45e200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-20T16:04:13.438 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:04:13 smithi082 ceph-mon[32238]: from='client.? 172.21.15.82:0/1902575948' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-05-20T16:04:13.438 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:04:13 smithi082 ceph-mon[32238]: mgrmap e5: x(active, since 11s) 2024-05-20T16:04:13.438 INFO:journalctl@ceph.mgr.x.smithi082.stdout:May 20 16:04:13 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mgr-x[32467]: 2024-05-20T16:04:13.172+0000 7efe2a45e200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-20T16:04:14.045 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout { 2024-05-20T16:04:14.045 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-05-20T16:04:14.045 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "available": true, 2024-05-20T16:04:14.045 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-05-20T16:04:14.045 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-05-20T16:04:14.046 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout } 2024-05-20T16:04:14.046 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:13.230+0000 7f80ee8cb700 1 Processor -- start 2024-05-20T16:04:14.046 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:13.230+0000 7f80ee8cb700 1 -- start start 2024-05-20T16:04:14.046 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:13.230+0000 7f80ee8cb700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f80e807bcc0 0x7f80e810b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:04:14.046 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:13.230+0000 7f80ee8cb700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f80e810bbd0 con 0x7f80e807bcc0 2024-05-20T16:04:14.046 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:13.231+0000 7f80e7fff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f80e807bcc0 0x7f80e810b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:04:14.046 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:13.231+0000 7f80e7fff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f80e807bcc0 0x7f80e810b5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:33882/0 (socket says 172.21.15.82:33882) 2024-05-20T16:04:14.046 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:13.231+0000 7f80e7fff700 1 -- 172.21.15.82:0/3657292834 learned_addr learned my addr 172.21.15.82:0/3657292834 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:04:14.046 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:13.231+0000 7f80e7fff700 1 -- 172.21.15.82:0/3657292834 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80e810bd50 con 0x7f80e807bcc0 2024-05-20T16:04:14.046 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:13.231+0000 7f80e7fff700 1 --2- 172.21.15.82:0/3657292834 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f80e807bcc0 0x7f80e810b5c0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f80d80098a0 tx=0x7f80d8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2e31ee7ce7a19b0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:04:14.046 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:13.231+0000 7f80e77fe700 1 -- 172.21.15.82:0/3657292834 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f80d8004160 con 0x7f80e807bcc0 2024-05-20T16:04:14.046 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:13.231+0000 7f80e77fe700 1 -- 172.21.15.82:0/3657292834 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f80d8004300 con 0x7f80e807bcc0 2024-05-20T16:04:14.047 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:13.231+0000 7f80e77fe700 1 -- 172.21.15.82:0/3657292834 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f80d8011b50 con 0x7f80e807bcc0 2024-05-20T16:04:14.047 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:13.232+0000 7f80ee8cb700 1 -- 172.21.15.82:0/3657292834 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f80e807bcc0 msgr2=0x7f80e810b5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:04:14.047 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:13.232+0000 7f80ee8cb700 1 --2- 172.21.15.82:0/3657292834 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f80e807bcc0 0x7f80e810b5c0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f80d80098a0 tx=0x7f80d8009bb0 comp rx=0 tx=0).stop 2024-05-20T16:04:14.047 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:13.232+0000 7f80ee8cb700 1 -- 172.21.15.82:0/3657292834 shutdown_connections 2024-05-20T16:04:14.047 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:13.232+0000 7f80ee8cb700 1 --2- 172.21.15.82:0/3657292834 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f80e807bcc0 0x7f80e810b5c0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:04:14.047 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:13.232+0000 7f80ee8cb700 1 -- 172.21.15.82:0/3657292834 >> 172.21.15.82:0/3657292834 conn(0x7f80e8103070 msgr2=0x7f80e81054c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:04:14.047 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:13.232+0000 7f80ee8cb700 1 -- 172.21.15.82:0/3657292834 shutdown_connections 2024-05-20T16:04:14.047 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:13.232+0000 7f80ee8cb700 1 -- 172.21.15.82:0/3657292834 wait complete. 2024-05-20T16:04:14.047 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:13.233+0000 7f80ee8cb700 1 Processor -- start 2024-05-20T16:04:14.047 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:13.233+0000 7f80ee8cb700 1 -- start start 2024-05-20T16:04:14.047 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:13.233+0000 7f80ee8cb700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f80e807bcc0 0x7f80e819cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:04:14.047 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:13.233+0000 7f80ee8cb700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f80e810ce30 con 0x7f80e807bcc0 2024-05-20T16:04:14.047 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:13.233+0000 7f80e7fff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f80e807bcc0 0x7f80e819cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:04:14.047 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:13.233+0000 7f80e7fff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f80e807bcc0 0x7f80e819cd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:33884/0 (socket says 172.21.15.82:33884) 2024-05-20T16:04:14.047 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:13.233+0000 7f80e7fff700 1 -- 172.21.15.82:0/373092835 learned_addr learned my addr 172.21.15.82:0/373092835 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:04:14.048 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:13.234+0000 7f80e7fff700 1 -- 172.21.15.82:0/373092835 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80e819d2b0 con 0x7f80e807bcc0 2024-05-20T16:04:14.048 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:13.234+0000 7f80e7fff700 1 --2- 172.21.15.82:0/373092835 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f80e807bcc0 0x7f80e819cd70 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f80d8009fb0 tx=0x7f80d8011bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:04:14.048 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:13.234+0000 7f80e5ffb700 1 -- 172.21.15.82:0/373092835 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f80d80038b0 con 0x7f80e807bcc0 2024-05-20T16:04:14.048 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:13.234+0000 7f80e5ffb700 1 -- 172.21.15.82:0/373092835 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f80d8011e50 con 0x7f80e807bcc0 2024-05-20T16:04:14.048 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:13.234+0000 7f80ee8cb700 1 -- 172.21.15.82:0/373092835 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80e819d540 con 0x7f80e807bcc0 2024-05-20T16:04:14.048 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:13.234+0000 7f80e5ffb700 1 -- 172.21.15.82:0/373092835 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f80d8019550 con 0x7f80e807bcc0 2024-05-20T16:04:14.048 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:13.234+0000 7f80ee8cb700 1 -- 172.21.15.82:0/373092835 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80e81a0150 con 0x7f80e807bcc0 2024-05-20T16:04:14.048 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:13.235+0000 7f80ee8cb700 1 -- 172.21.15.82:0/373092835 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80e8063ca0 con 0x7f80e807bcc0 2024-05-20T16:04:14.048 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:13.235+0000 7f80e5ffb700 1 -- 172.21.15.82:0/373092835 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) ==== 46287+0+0 (secure 0 0 0) 0x7f80d8019800 con 0x7f80e807bcc0 2024-05-20T16:04:14.048 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:13.235+0000 7f80e5ffb700 1 --2- 172.21.15.82:0/373092835 >> [v2:172.21.15.82:6800/1959459348,v1:172.21.15.82:6801/1959459348] conn(0x7f80d4039110 0x7f80d403b620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:04:14.048 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:13.235+0000 7f80e5ffb700 1 -- 172.21.15.82:0/373092835 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f80d804d250 con 0x7f80e807bcc0 2024-05-20T16:04:14.048 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:13.235+0000 7f80dffff700 1 -- 172.21.15.82:0/373092835 >> [v2:172.21.15.82:6800/1959459348,v1:172.21.15.82:6801/1959459348] conn(0x7f80d4039110 msgr2=0x7f80d403b620 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.82:6800/1959459348 2024-05-20T16:04:14.048 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:13.235+0000 7f80dffff700 1 --2- 172.21.15.82:0/373092835 >> [v2:172.21.15.82:6800/1959459348,v1:172.21.15.82:6801/1959459348] conn(0x7f80d4039110 0x7f80d403b620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-20T16:04:14.048 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:13.241+0000 7f80e5ffb700 1 -- 172.21.15.82:0/373092835 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f80d800bc10 con 0x7f80e807bcc0 2024-05-20T16:04:14.048 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:13.436+0000 7f80dffff700 1 -- 172.21.15.82:0/373092835 >> [v2:172.21.15.82:6800/1959459348,v1:172.21.15.82:6801/1959459348] conn(0x7f80d4039110 msgr2=0x7f80d403b620 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.82:6800/1959459348 2024-05-20T16:04:14.049 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:13.436+0000 7f80dffff700 1 --2- 172.21.15.82:0/373092835 >> [v2:172.21.15.82:6800/1959459348,v1:172.21.15.82:6801/1959459348] conn(0x7f80d4039110 0x7f80d403b620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-20T16:04:14.049 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:13.500+0000 7f80ee8cb700 1 -- 172.21.15.82:0/373092835 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f80e81a0440 con 0x7f80e807bcc0 2024-05-20T16:04:14.049 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:13.501+0000 7f80e5ffb700 1 -- 172.21.15.82:0/373092835 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7f80d800fda0 con 0x7f80e807bcc0 2024-05-20T16:04:14.049 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:13.503+0000 7f80ee8cb700 1 -- 172.21.15.82:0/373092835 >> [v2:172.21.15.82:6800/1959459348,v1:172.21.15.82:6801/1959459348] conn(0x7f80d4039110 msgr2=0x7f80d403b620 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-20T16:04:14.049 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:13.503+0000 7f80ee8cb700 1 --2- 172.21.15.82:0/373092835 >> [v2:172.21.15.82:6800/1959459348,v1:172.21.15.82:6801/1959459348] conn(0x7f80d4039110 0x7f80d403b620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:04:14.049 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:13.503+0000 7f80ee8cb700 1 -- 172.21.15.82:0/373092835 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f80e807bcc0 msgr2=0x7f80e819cd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:04:14.049 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:13.503+0000 7f80ee8cb700 1 --2- 172.21.15.82:0/373092835 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f80e807bcc0 0x7f80e819cd70 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f80d8009fb0 tx=0x7f80d8011bf0 comp rx=0 tx=0).stop 2024-05-20T16:04:14.049 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:13.503+0000 7f80ee8cb700 1 -- 172.21.15.82:0/373092835 shutdown_connections 2024-05-20T16:04:14.049 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:13.503+0000 7f80ee8cb700 1 --2- 172.21.15.82:0/373092835 >> [v2:172.21.15.82:6800/1959459348,v1:172.21.15.82:6801/1959459348] conn(0x7f80d4039110 0x7f80d403b620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:04:14.049 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:13.503+0000 7f80ee8cb700 1 --2- 172.21.15.82:0/373092835 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f80e807bcc0 0x7f80e819cd70 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:04:14.049 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:13.503+0000 7f80ee8cb700 1 -- 172.21.15.82:0/373092835 >> 172.21.15.82:0/373092835 conn(0x7f80e8103070 msgr2=0x7f80e8104920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:04:14.049 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:13.503+0000 7f80ee8cb700 1 -- 172.21.15.82:0/373092835 shutdown_connections 2024-05-20T16:04:14.049 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:13.503+0000 7f80ee8cb700 1 -- 172.21.15.82:0/373092835 wait complete. 2024-05-20T16:04:14.049 INFO:teuthology.orchestra.run.smithi082.stdout:Waiting for the mgr to restart... 2024-05-20T16:04:14.049 INFO:teuthology.orchestra.run.smithi082.stdout:Waiting for mgr epoch 5... 2024-05-20T16:04:14.582 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:04:14 smithi082 ceph-mon[32238]: from='client.? 172.21.15.82:0/373092835' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-05-20T16:04:14.583 INFO:journalctl@ceph.mgr.x.smithi082.stdout:May 20 16:04:14 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mgr-x[32467]: 2024-05-20T16:04:14.402+0000 7efe2a45e200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-20T16:04:14.860 INFO:journalctl@ceph.mgr.x.smithi082.stdout:May 20 16:04:14 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mgr-x[32467]: 2024-05-20T16:04:14.581+0000 7efe2a45e200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-20T16:04:14.860 INFO:journalctl@ceph.mgr.x.smithi082.stdout:May 20 16:04:14 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mgr-x[32467]: 2024-05-20T16:04:14.755+0000 7efe2a45e200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-20T16:04:15.345 INFO:journalctl@ceph.mgr.x.smithi082.stdout:May 20 16:04:15 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mgr-x[32467]: 2024-05-20T16:04:15.048+0000 7efe2a45e200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-20T16:04:15.345 INFO:journalctl@ceph.mgr.x.smithi082.stdout:May 20 16:04:15 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mgr-x[32467]: 2024-05-20T16:04:15.142+0000 7efe2a45e200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-20T16:04:15.345 INFO:journalctl@ceph.mgr.x.smithi082.stdout:May 20 16:04:15 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mgr-x[32467]: 2024-05-20T16:04:15.344+0000 7efe2a45e200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-20T16:04:15.609 INFO:journalctl@ceph.mgr.x.smithi082.stdout:May 20 16:04:15 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mgr-x[32467]: 2024-05-20T16:04:15.540+0000 7efe2a45e200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-20T16:04:16.568 INFO:journalctl@ceph.mgr.x.smithi082.stdout:May 20 16:04:16 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mgr-x[32467]: 2024-05-20T16:04:16.138+0000 7efe2a45e200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-20T16:04:16.860 INFO:journalctl@ceph.mgr.x.smithi082.stdout:May 20 16:04:16 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mgr-x[32467]: 2024-05-20T16:04:16.567+0000 7efe2a45e200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-20T16:04:16.860 INFO:journalctl@ceph.mgr.x.smithi082.stdout:May 20 16:04:16 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mgr-x[32467]: 2024-05-20T16:04:16.665+0000 7efe2a45e200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-20T16:04:17.359 INFO:journalctl@ceph.mgr.x.smithi082.stdout:May 20 16:04:16 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mgr-x[32467]: 2024-05-20T16:04:16.869+0000 7efe2a45e200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-20T16:04:17.731 INFO:journalctl@ceph.mgr.x.smithi082.stdout:May 20 16:04:17 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mgr-x[32467]: 2024-05-20T16:04:17.392+0000 7efe2a45e200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-20T16:04:17.731 INFO:journalctl@ceph.mgr.x.smithi082.stdout:May 20 16:04:17 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mgr-x[32467]: 2024-05-20T16:04:17.485+0000 7efe2a45e200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-20T16:04:18.080 INFO:journalctl@ceph.mgr.x.smithi082.stdout:May 20 16:04:17 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mgr-x[32467]: 2024-05-20T16:04:17.730+0000 7efe2a45e200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-20T16:04:18.080 INFO:journalctl@ceph.mgr.x.smithi082.stdout:May 20 16:04:17 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mgr-x[32467]: 2024-05-20T16:04:17.866+0000 7efe2a45e200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-20T16:04:18.081 INFO:journalctl@ceph.mgr.x.smithi082.stdout:May 20 16:04:17 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mgr-x[32467]: 2024-05-20T16:04:17.962+0000 7efe2a45e200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-20T16:04:18.081 INFO:journalctl@ceph.mgr.x.smithi082.stdout:May 20 16:04:18 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mgr-x[32467]: 2024-05-20T16:04:18.079+0000 7efe2a45e200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-20T16:04:18.859 INFO:journalctl@ceph.mgr.x.smithi082.stdout:May 20 16:04:18 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mgr-x[32467]: 2024-05-20T16:04:18.379+0000 7efe2a45e200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-20T16:04:19.259 INFO:journalctl@ceph.mgr.x.smithi082.stdout:May 20 16:04:18 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mgr-x[32467]: 2024-05-20T16:04:18.977+0000 7efe2a45e200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-20T16:04:19.259 INFO:journalctl@ceph.mgr.x.smithi082.stdout:May 20 16:04:19 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mgr-x[32467]: 2024-05-20T16:04:19.205+0000 7efe2a45e200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-20T16:04:19.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:04:19 smithi082 ceph-mon[32238]: Active manager daemon x restarted 2024-05-20T16:04:19.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:04:19 smithi082 ceph-mon[32238]: Activating manager daemon x 2024-05-20T16:04:19.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:04:19 smithi082 ceph-mon[32238]: osdmap e2: 0 total, 0 up, 0 in 2024-05-20T16:04:19.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:04:19 smithi082 ceph-mon[32238]: mgrmap e6: x(active, starting, since 0.00419145s) 2024-05-20T16:04:19.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:04:19 smithi082 ceph-mon[32238]: from='mgr.14118 172.21.15.82:0/3662546898' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-20T16:04:19.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:04:19 smithi082 ceph-mon[32238]: from='mgr.14118 172.21.15.82:0/3662546898' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-20T16:04:19.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:04:19 smithi082 ceph-mon[32238]: from='mgr.14118 172.21.15.82:0/3662546898' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-20T16:04:19.611 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:04:19 smithi082 ceph-mon[32238]: from='mgr.14118 172.21.15.82:0/3662546898' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-20T16:04:19.611 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:04:19 smithi082 ceph-mon[32238]: from='mgr.14118 172.21.15.82:0/3662546898' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-20T16:04:19.611 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:04:19 smithi082 ceph-mon[32238]: Manager daemon x is now available 2024-05-20T16:04:19.611 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:04:19 smithi082 ceph-mon[32238]: from='mgr.14118 172.21.15.82:0/3662546898' entity='mgr.x' 2024-05-20T16:04:19.611 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:04:19 smithi082 ceph-mon[32238]: from='mgr.14118 172.21.15.82:0/3662546898' entity='mgr.x' 2024-05-20T16:04:19.611 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:04:19 smithi082 ceph-mon[32238]: from='mgr.14118 172.21.15.82:0/3662546898' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T16:04:19.611 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:04:19 smithi082 ceph-mon[32238]: from='mgr.14118 172.21.15.82:0/3662546898' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T16:04:20.517 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:04:20 smithi082 ceph-mon[32238]: Found migration_current of "None". Setting to last migration. 2024-05-20T16:04:20.517 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:04:20 smithi082 ceph-mon[32238]: from='mgr.14118 172.21.15.82:0/3662546898' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-20T16:04:20.518 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:04:20 smithi082 ceph-mon[32238]: from='mgr.14118 172.21.15.82:0/3662546898' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-20T16:04:20.518 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:04:20 smithi082 ceph-mon[32238]: mgrmap e7: x(active, since 1.00623s) 2024-05-20T16:04:20.814 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout { 2024-05-20T16:04:20.814 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-05-20T16:04:20.814 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "initialized": true 2024-05-20T16:04:20.814 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout } 2024-05-20T16:04:20.814 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:14.460+0000 7fc99d8d8700 1 Processor -- start 2024-05-20T16:04:20.814 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:14.460+0000 7fc99d8d8700 1 -- start start 2024-05-20T16:04:20.814 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:14.461+0000 7fc99d8d8700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc998107a20 0x7fc998107e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:04:20.814 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:14.461+0000 7fc99d8d8700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fc998108420 con 0x7fc998107a20 2024-05-20T16:04:20.814 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:14.461+0000 7fc996ffd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc998107a20 0x7fc998107e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:04:20.814 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:14.461+0000 7fc996ffd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc998107a20 0x7fc998107e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:33900/0 (socket says 172.21.15.82:33900) 2024-05-20T16:04:20.815 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:14.461+0000 7fc996ffd700 1 -- 172.21.15.82:0/3741688109 learned_addr learned my addr 172.21.15.82:0/3741688109 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:04:20.815 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:14.461+0000 7fc996ffd700 1 -- 172.21.15.82:0/3741688109 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc998108b50 con 0x7fc998107a20 2024-05-20T16:04:20.815 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:14.461+0000 7fc996ffd700 1 --2- 172.21.15.82:0/3741688109 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc998107a20 0x7fc998107e30 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fc9880098a0 tx=0x7fc988009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56f33d169a938bca server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:04:20.815 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:14.462+0000 7fc995ffb700 1 -- 172.21.15.82:0/3741688109 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc988004160 con 0x7fc998107a20 2024-05-20T16:04:20.815 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:14.462+0000 7fc995ffb700 1 -- 172.21.15.82:0/3741688109 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fc988004300 con 0x7fc998107a20 2024-05-20T16:04:20.815 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:14.462+0000 7fc995ffb700 1 -- 172.21.15.82:0/3741688109 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc988011b50 con 0x7fc998107a20 2024-05-20T16:04:20.815 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:14.462+0000 7fc99d8d8700 1 -- 172.21.15.82:0/3741688109 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc998107a20 msgr2=0x7fc998107e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:04:20.815 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:14.462+0000 7fc99d8d8700 1 --2- 172.21.15.82:0/3741688109 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc998107a20 0x7fc998107e30 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fc9880098a0 tx=0x7fc988009bb0 comp rx=0 tx=0).stop 2024-05-20T16:04:20.815 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:14.462+0000 7fc99d8d8700 1 -- 172.21.15.82:0/3741688109 shutdown_connections 2024-05-20T16:04:20.815 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:14.462+0000 7fc99d8d8700 1 --2- 172.21.15.82:0/3741688109 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc998107a20 0x7fc998107e30 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:04:20.815 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:14.462+0000 7fc99d8d8700 1 -- 172.21.15.82:0/3741688109 >> 172.21.15.82:0/3741688109 conn(0x7fc998103060 msgr2=0x7fc998105510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:04:20.815 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:14.463+0000 7fc99d8d8700 1 -- 172.21.15.82:0/3741688109 shutdown_connections 2024-05-20T16:04:20.815 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:14.463+0000 7fc99d8d8700 1 -- 172.21.15.82:0/3741688109 wait complete. 2024-05-20T16:04:20.816 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:14.463+0000 7fc99d8d8700 1 Processor -- start 2024-05-20T16:04:20.816 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:14.464+0000 7fc99d8d8700 1 -- start start 2024-05-20T16:04:20.816 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:14.464+0000 7fc99d8d8700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc998107a20 0x7fc99819bda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:04:20.816 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:14.464+0000 7fc99d8d8700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fc99810bf00 con 0x7fc998107a20 2024-05-20T16:04:20.816 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:14.464+0000 7fc996ffd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc998107a20 0x7fc99819bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:04:20.816 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:14.464+0000 7fc996ffd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc998107a20 0x7fc99819bda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:33910/0 (socket says 172.21.15.82:33910) 2024-05-20T16:04:20.816 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:14.464+0000 7fc996ffd700 1 -- 172.21.15.82:0/3383994206 learned_addr learned my addr 172.21.15.82:0/3383994206 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:04:20.816 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:14.464+0000 7fc996ffd700 1 -- 172.21.15.82:0/3383994206 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc99819c2e0 con 0x7fc998107a20 2024-05-20T16:04:20.816 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:14.465+0000 7fc996ffd700 1 --2- 172.21.15.82:0/3383994206 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc998107a20 0x7fc99819bda0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fc988009fb0 tx=0x7fc988011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:04:20.816 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:14.465+0000 7fc99c8d6700 1 -- 172.21.15.82:0/3383994206 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc9880038b0 con 0x7fc998107a20 2024-05-20T16:04:20.816 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:14.465+0000 7fc99c8d6700 1 -- 172.21.15.82:0/3383994206 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fc988011e50 con 0x7fc998107a20 2024-05-20T16:04:20.816 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:14.465+0000 7fc99d8d8700 1 -- 172.21.15.82:0/3383994206 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc99819c570 con 0x7fc998107a20 2024-05-20T16:04:20.817 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:14.465+0000 7fc99c8d6700 1 -- 172.21.15.82:0/3383994206 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc98801a430 con 0x7fc998107a20 2024-05-20T16:04:20.817 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:14.465+0000 7fc99d8d8700 1 -- 172.21.15.82:0/3383994206 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc99819f180 con 0x7fc998107a20 2024-05-20T16:04:20.817 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:14.466+0000 7fc99c8d6700 1 -- 172.21.15.82:0/3383994206 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) ==== 46287+0+0 (secure 0 0 0) 0x7fc98801a730 con 0x7fc998107a20 2024-05-20T16:04:20.817 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:14.466+0000 7fc99c8d6700 1 --2- 172.21.15.82:0/3383994206 >> [v2:172.21.15.82:6800/1959459348,v1:172.21.15.82:6801/1959459348] conn(0x7fc9800390c0 0x7fc98003b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:04:20.817 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:14.466+0000 7fc99c8d6700 1 -- 172.21.15.82:0/3383994206 --> [v2:172.21.15.82:6800/1959459348,v1:172.21.15.82:6801/1959459348] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fc98003bce0 con 0x7fc9800390c0 2024-05-20T16:04:20.817 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:14.466+0000 7fc99c8d6700 1 -- 172.21.15.82:0/3383994206 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fc98804d170 con 0x7fc998107a20 2024-05-20T16:04:20.817 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:14.466+0000 7fc9967fc700 1 -- 172.21.15.82:0/3383994206 >> [v2:172.21.15.82:6800/1959459348,v1:172.21.15.82:6801/1959459348] conn(0x7fc9800390c0 msgr2=0x7fc98003b5d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.82:6800/1959459348 2024-05-20T16:04:20.817 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:14.466+0000 7fc9967fc700 1 --2- 172.21.15.82:0/3383994206 >> [v2:172.21.15.82:6800/1959459348,v1:172.21.15.82:6801/1959459348] conn(0x7fc9800390c0 0x7fc98003b5d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-20T16:04:20.817 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:14.667+0000 7fc9967fc700 1 -- 172.21.15.82:0/3383994206 >> [v2:172.21.15.82:6800/1959459348,v1:172.21.15.82:6801/1959459348] conn(0x7fc9800390c0 msgr2=0x7fc98003b5d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.82:6800/1959459348 2024-05-20T16:04:20.817 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:14.667+0000 7fc9967fc700 1 --2- 172.21.15.82:0/3383994206 >> [v2:172.21.15.82:6800/1959459348,v1:172.21.15.82:6801/1959459348] conn(0x7fc9800390c0 0x7fc98003b5d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-20T16:04:20.817 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:15.068+0000 7fc9967fc700 1 -- 172.21.15.82:0/3383994206 >> [v2:172.21.15.82:6800/1959459348,v1:172.21.15.82:6801/1959459348] conn(0x7fc9800390c0 msgr2=0x7fc98003b5d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.82:6800/1959459348 2024-05-20T16:04:20.818 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:15.068+0000 7fc9967fc700 1 --2- 172.21.15.82:0/3383994206 >> [v2:172.21.15.82:6800/1959459348,v1:172.21.15.82:6801/1959459348] conn(0x7fc9800390c0 0x7fc98003b5d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-20T16:04:20.818 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:15.869+0000 7fc9967fc700 1 -- 172.21.15.82:0/3383994206 >> [v2:172.21.15.82:6800/1959459348,v1:172.21.15.82:6801/1959459348] conn(0x7fc9800390c0 msgr2=0x7fc98003b5d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.82:6800/1959459348 2024-05-20T16:04:20.818 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:15.869+0000 7fc9967fc700 1 --2- 172.21.15.82:0/3383994206 >> [v2:172.21.15.82:6800/1959459348,v1:172.21.15.82:6801/1959459348] conn(0x7fc9800390c0 0x7fc98003b5d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-20T16:04:20.818 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:17.470+0000 7fc9967fc700 1 -- 172.21.15.82:0/3383994206 >> [v2:172.21.15.82:6800/1959459348,v1:172.21.15.82:6801/1959459348] conn(0x7fc9800390c0 msgr2=0x7fc98003b5d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.82:6800/1959459348 2024-05-20T16:04:20.818 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:17.470+0000 7fc9967fc700 1 --2- 172.21.15.82:0/3383994206 >> [v2:172.21.15.82:6800/1959459348,v1:172.21.15.82:6801/1959459348] conn(0x7fc9800390c0 0x7fc98003b5d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-05-20T16:04:20.818 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:19.209+0000 7fc99c8d6700 1 -- 172.21.15.82:0/3383994206 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mgrmap(e 6) ==== 46054+0+0 (secure 0 0 0) 0x7fc98801aa70 con 0x7fc998107a20 2024-05-20T16:04:20.818 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:19.210+0000 7fc99c8d6700 1 -- 172.21.15.82:0/3383994206 >> [v2:172.21.15.82:6800/1959459348,v1:172.21.15.82:6801/1959459348] conn(0x7fc9800390c0 msgr2=0x7fc98003b5d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-20T16:04:20.818 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:19.210+0000 7fc99c8d6700 1 --2- 172.21.15.82:0/3383994206 >> [v2:172.21.15.82:6800/1959459348,v1:172.21.15.82:6801/1959459348] conn(0x7fc9800390c0 0x7fc98003b5d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:04:20.818 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:20.211+0000 7fc99c8d6700 1 -- 172.21.15.82:0/3383994206 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mgrmap(e 7) ==== 46181+0+0 (secure 0 0 0) 0x7fc988018350 con 0x7fc998107a20 2024-05-20T16:04:20.819 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:20.211+0000 7fc99c8d6700 1 --2- 172.21.15.82:0/3383994206 >> [v2:172.21.15.82:6800/1763049308,v1:172.21.15.82:6801/1763049308] conn(0x7fc98003ca40 0x7fc98003ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:04:20.819 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:20.211+0000 7fc99c8d6700 1 -- 172.21.15.82:0/3383994206 --> [v2:172.21.15.82:6800/1763049308,v1:172.21.15.82:6801/1763049308] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fc98003bce0 con 0x7fc98003ca40 2024-05-20T16:04:20.819 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:20.212+0000 7fc9967fc700 1 --2- 172.21.15.82:0/3383994206 >> [v2:172.21.15.82:6800/1763049308,v1:172.21.15.82:6801/1763049308] conn(0x7fc98003ca40 0x7fc98003ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:04:20.819 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:20.212+0000 7fc9967fc700 1 --2- 172.21.15.82:0/3383994206 >> [v2:172.21.15.82:6800/1763049308,v1:172.21.15.82:6801/1763049308] conn(0x7fc98003ca40 0x7fc98003ee60 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fc98c003a10 tx=0x7fc98c009290 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:04:20.819 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:20.213+0000 7fc99c8d6700 1 -- 172.21.15.82:0/3383994206 <== mgr.14118 v2:172.21.15.82:6800/1763049308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7774 (secure 0 0 0) 0x7fc98003bce0 con 0x7fc98003ca40 2024-05-20T16:04:20.819 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:20.219+0000 7fc99d8d8700 1 -- 172.21.15.82:0/3383994206 --> [v2:172.21.15.82:6800/1763049308,v1:172.21.15.82:6801/1763049308] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7fc998063ca0 con 0x7fc98003ca40 2024-05-20T16:04:20.819 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:20.220+0000 7fc99c8d6700 1 -- 172.21.15.82:0/3383994206 <== mgr.14118 v2:172.21.15.82:6800/1763049308 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7fc998063ca0 con 0x7fc98003ca40 2024-05-20T16:04:20.819 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:20.220+0000 7fc99d8d8700 1 -- 172.21.15.82:0/3383994206 >> [v2:172.21.15.82:6800/1763049308,v1:172.21.15.82:6801/1763049308] conn(0x7fc98003ca40 msgr2=0x7fc98003ee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:04:20.819 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:20.220+0000 7fc99d8d8700 1 --2- 172.21.15.82:0/3383994206 >> [v2:172.21.15.82:6800/1763049308,v1:172.21.15.82:6801/1763049308] conn(0x7fc98003ca40 0x7fc98003ee60 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fc98c003a10 tx=0x7fc98c009290 comp rx=0 tx=0).stop 2024-05-20T16:04:20.819 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:20.220+0000 7fc99d8d8700 1 -- 172.21.15.82:0/3383994206 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc998107a20 msgr2=0x7fc99819bda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:04:20.820 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:20.220+0000 7fc99d8d8700 1 --2- 172.21.15.82:0/3383994206 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc998107a20 0x7fc99819bda0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fc988009fb0 tx=0x7fc988011c10 comp rx=0 tx=0).stop 2024-05-20T16:04:20.820 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:20.220+0000 7fc99d8d8700 1 -- 172.21.15.82:0/3383994206 shutdown_connections 2024-05-20T16:04:20.820 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:20.220+0000 7fc99d8d8700 1 --2- 172.21.15.82:0/3383994206 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc998107a20 0x7fc99819bda0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:04:20.820 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:20.220+0000 7fc99d8d8700 1 --2- 172.21.15.82:0/3383994206 >> [v2:172.21.15.82:6800/1763049308,v1:172.21.15.82:6801/1763049308] conn(0x7fc98003ca40 0x7fc98003ee60 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:04:20.820 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:20.220+0000 7fc99d8d8700 1 --2- 172.21.15.82:0/3383994206 >> [v2:172.21.15.82:6800/1959459348,v1:172.21.15.82:6801/1959459348] conn(0x7fc9800390c0 0x7fc98003b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:04:20.820 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:20.220+0000 7fc99d8d8700 1 -- 172.21.15.82:0/3383994206 >> 172.21.15.82:0/3383994206 conn(0x7fc998103060 msgr2=0x7fc9981038a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:04:20.820 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:20.220+0000 7fc99d8d8700 1 -- 172.21.15.82:0/3383994206 shutdown_connections 2024-05-20T16:04:20.820 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:20.220+0000 7fc99d8d8700 1 -- 172.21.15.82:0/3383994206 wait complete. 2024-05-20T16:04:20.820 INFO:teuthology.orchestra.run.smithi082.stdout:mgr epoch 5 is available 2024-05-20T16:04:20.820 INFO:teuthology.orchestra.run.smithi082.stdout:Setting orchestrator backend to cephadm... 2024-05-20T16:04:21.498 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:04:21 smithi082 ceph-mon[32238]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-05-20T16:04:21.498 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:04:21 smithi082 ceph-mon[32238]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-05-20T16:04:21.498 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:04:21 smithi082 ceph-mon[32238]: from='mgr.14118 172.21.15.82:0/3662546898' entity='mgr.x' 2024-05-20T16:04:21.499 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:04:21 smithi082 ceph-mon[32238]: from='mgr.14118 172.21.15.82:0/3662546898' entity='mgr.x' 2024-05-20T16:04:22.032 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:21.266+0000 7fa6d0c3c700 1 Processor -- start 2024-05-20T16:04:22.032 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:21.266+0000 7fa6d0c3c700 1 -- start start 2024-05-20T16:04:22.032 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:21.267+0000 7fa6d0c3c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa6cc107990 0x7fa6cc107da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:04:22.032 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:21.267+0000 7fa6d0c3c700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fa6cc108390 con 0x7fa6cc107990 2024-05-20T16:04:22.033 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:21.267+0000 7fa6ca59c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa6cc107990 0x7fa6cc107da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:04:22.033 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:21.267+0000 7fa6ca59c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa6cc107990 0x7fa6cc107da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43232/0 (socket says 172.21.15.82:43232) 2024-05-20T16:04:22.033 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:21.267+0000 7fa6ca59c700 1 -- 172.21.15.82:0/914554743 learned_addr learned my addr 172.21.15.82:0/914554743 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:04:22.033 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:21.267+0000 7fa6ca59c700 1 -- 172.21.15.82:0/914554743 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6cc108ac0 con 0x7fa6cc107990 2024-05-20T16:04:22.033 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:21.267+0000 7fa6ca59c700 1 --2- 172.21.15.82:0/914554743 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa6cc107990 0x7fa6cc107da0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fa6c00098a0 tx=0x7fa6c0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b9de36ae7053490 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:04:22.033 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:21.268+0000 7fa6c959a700 1 -- 172.21.15.82:0/914554743 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa6c0004160 con 0x7fa6cc107990 2024-05-20T16:04:22.033 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:21.268+0000 7fa6c959a700 1 -- 172.21.15.82:0/914554743 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fa6c0004300 con 0x7fa6cc107990 2024-05-20T16:04:22.033 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:21.268+0000 7fa6c959a700 1 -- 172.21.15.82:0/914554743 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa6c0011b50 con 0x7fa6cc107990 2024-05-20T16:04:22.033 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:21.268+0000 7fa6d0c3c700 1 -- 172.21.15.82:0/914554743 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa6cc107990 msgr2=0x7fa6cc107da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:04:22.033 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:21.268+0000 7fa6d0c3c700 1 --2- 172.21.15.82:0/914554743 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa6cc107990 0x7fa6cc107da0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fa6c00098a0 tx=0x7fa6c0009bb0 comp rx=0 tx=0).stop 2024-05-20T16:04:22.033 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:21.268+0000 7fa6d0c3c700 1 -- 172.21.15.82:0/914554743 shutdown_connections 2024-05-20T16:04:22.033 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:21.268+0000 7fa6d0c3c700 1 --2- 172.21.15.82:0/914554743 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa6cc107990 0x7fa6cc107da0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:04:22.033 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:21.268+0000 7fa6d0c3c700 1 -- 172.21.15.82:0/914554743 >> 172.21.15.82:0/914554743 conn(0x7fa6cc103070 msgr2=0x7fa6cc1054c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:04:22.034 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:21.269+0000 7fa6d0c3c700 1 -- 172.21.15.82:0/914554743 shutdown_connections 2024-05-20T16:04:22.034 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:21.269+0000 7fa6d0c3c700 1 -- 172.21.15.82:0/914554743 wait complete. 2024-05-20T16:04:22.034 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:21.269+0000 7fa6d0c3c700 1 Processor -- start 2024-05-20T16:04:22.034 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:21.269+0000 7fa6d0c3c700 1 -- start start 2024-05-20T16:04:22.034 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:21.270+0000 7fa6d0c3c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa6cc107990 0x7fa6cc19bd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:04:22.034 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:21.270+0000 7fa6d0c3c700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fa6cc10be70 con 0x7fa6cc107990 2024-05-20T16:04:22.034 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:21.270+0000 7fa6ca59c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa6cc107990 0x7fa6cc19bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:04:22.034 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:21.270+0000 7fa6ca59c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa6cc107990 0x7fa6cc19bd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43248/0 (socket says 172.21.15.82:43248) 2024-05-20T16:04:22.034 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:21.270+0000 7fa6ca59c700 1 -- 172.21.15.82:0/2493090538 learned_addr learned my addr 172.21.15.82:0/2493090538 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:04:22.034 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:21.270+0000 7fa6ca59c700 1 -- 172.21.15.82:0/2493090538 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6cc19c270 con 0x7fa6cc107990 2024-05-20T16:04:22.034 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:21.270+0000 7fa6ca59c700 1 --2- 172.21.15.82:0/2493090538 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa6cc107990 0x7fa6cc19bd30 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fa6c00046e0 tx=0x7fa6c0011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:04:22.034 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:21.270+0000 7fa6bb7fe700 1 -- 172.21.15.82:0/2493090538 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa6c00039e0 con 0x7fa6cc107990 2024-05-20T16:04:22.034 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:21.270+0000 7fa6d0c3c700 1 -- 172.21.15.82:0/2493090538 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6cc19c500 con 0x7fa6cc107990 2024-05-20T16:04:22.035 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:21.270+0000 7fa6bb7fe700 1 -- 172.21.15.82:0/2493090538 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fa6c0003b80 con 0x7fa6cc107990 2024-05-20T16:04:22.035 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:21.270+0000 7fa6bb7fe700 1 -- 172.21.15.82:0/2493090538 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa6c0024690 con 0x7fa6cc107990 2024-05-20T16:04:22.035 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:21.270+0000 7fa6d0c3c700 1 -- 172.21.15.82:0/2493090538 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6cc19f100 con 0x7fa6cc107990 2024-05-20T16:04:22.035 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:21.271+0000 7fa6bb7fe700 1 -- 172.21.15.82:0/2493090538 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 7) ==== 46181+0+0 (secure 0 0 0) 0x7fa6c0022020 con 0x7fa6cc107990 2024-05-20T16:04:22.035 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:21.272+0000 7fa6bb7fe700 1 --2- 172.21.15.82:0/2493090538 >> [v2:172.21.15.82:6800/1763049308,v1:172.21.15.82:6801/1763049308] conn(0x7fa6b4038ff0 0x7fa6b403b500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:04:22.035 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:21.272+0000 7fa6d0c3c700 1 -- 172.21.15.82:0/2493090538 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6cc063ca0 con 0x7fa6cc107990 2024-05-20T16:04:22.035 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:21.272+0000 7fa6bb7fe700 1 -- 172.21.15.82:0/2493090538 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fa6c0011e00 con 0x7fa6cc107990 2024-05-20T16:04:22.035 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:21.272+0000 7fa6c9d9b700 1 --2- 172.21.15.82:0/2493090538 >> [v2:172.21.15.82:6800/1763049308,v1:172.21.15.82:6801/1763049308] conn(0x7fa6b4038ff0 0x7fa6b403b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:04:22.035 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:21.272+0000 7fa6c9d9b700 1 --2- 172.21.15.82:0/2493090538 >> [v2:172.21.15.82:6800/1763049308,v1:172.21.15.82:6801/1763049308] conn(0x7fa6b4038ff0 0x7fa6b403b500 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fa6bc006fd0 tx=0x7fa6bc006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:04:22.035 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:21.277+0000 7fa6bb7fe700 1 -- 172.21.15.82:0/2493090538 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fa6c0003d30 con 0x7fa6cc107990 2024-05-20T16:04:22.035 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:21.497+0000 7fa6d0c3c700 1 -- 172.21.15.82:0/2493090538 --> [v2:172.21.15.82:6800/1763049308,v1:172.21.15.82:6801/1763049308] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7fa6cc105060 con 0x7fa6b4038ff0 2024-05-20T16:04:22.035 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:21.506+0000 7fa6bb7fe700 1 -- 172.21.15.82:0/2493090538 <== mgr.14118 v2:172.21.15.82:6800/1763049308 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fa6cc105060 con 0x7fa6b4038ff0 2024-05-20T16:04:22.035 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:21.508+0000 7fa6d0c3c700 1 -- 172.21.15.82:0/2493090538 >> [v2:172.21.15.82:6800/1763049308,v1:172.21.15.82:6801/1763049308] conn(0x7fa6b4038ff0 msgr2=0x7fa6b403b500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:04:22.036 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:21.508+0000 7fa6d0c3c700 1 --2- 172.21.15.82:0/2493090538 >> [v2:172.21.15.82:6800/1763049308,v1:172.21.15.82:6801/1763049308] conn(0x7fa6b4038ff0 0x7fa6b403b500 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fa6bc006fd0 tx=0x7fa6bc006e40 comp rx=0 tx=0).stop 2024-05-20T16:04:22.036 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:21.508+0000 7fa6d0c3c700 1 -- 172.21.15.82:0/2493090538 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa6cc107990 msgr2=0x7fa6cc19bd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:04:22.036 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:21.508+0000 7fa6d0c3c700 1 --2- 172.21.15.82:0/2493090538 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa6cc107990 0x7fa6cc19bd30 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fa6c00046e0 tx=0x7fa6c0011c10 comp rx=0 tx=0).stop 2024-05-20T16:04:22.036 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:21.509+0000 7fa6d0c3c700 1 -- 172.21.15.82:0/2493090538 shutdown_connections 2024-05-20T16:04:22.036 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:21.509+0000 7fa6d0c3c700 1 --2- 172.21.15.82:0/2493090538 >> [v2:172.21.15.82:6800/1763049308,v1:172.21.15.82:6801/1763049308] conn(0x7fa6b4038ff0 0x7fa6b403b500 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:04:22.036 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:21.509+0000 7fa6d0c3c700 1 --2- 172.21.15.82:0/2493090538 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa6cc107990 0x7fa6cc19bd30 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:04:22.036 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:21.509+0000 7fa6d0c3c700 1 -- 172.21.15.82:0/2493090538 >> 172.21.15.82:0/2493090538 conn(0x7fa6cc103070 msgr2=0x7fa6cc104950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:04:22.036 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:21.509+0000 7fa6d0c3c700 1 -- 172.21.15.82:0/2493090538 shutdown_connections 2024-05-20T16:04:22.036 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:21.509+0000 7fa6d0c3c700 1 -- 172.21.15.82:0/2493090538 wait complete. 2024-05-20T16:04:22.860 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:04:22 smithi082 ceph-mon[32238]: [20/May/2024:16:04:21] ENGINE Bus STARTING 2024-05-20T16:04:22.860 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:04:22 smithi082 ceph-mon[32238]: from='client.14130 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-05-20T16:04:22.860 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:04:22 smithi082 ceph-mon[32238]: from='mgr.14118 172.21.15.82:0/3662546898' entity='mgr.x' 2024-05-20T16:04:22.860 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:04:22 smithi082 ceph-mon[32238]: from='mgr.14118 172.21.15.82:0/3662546898' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T16:04:22.860 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:04:22 smithi082 ceph-mon[32238]: [20/May/2024:16:04:21] ENGINE Serving on https://172.21.15.82:7150 2024-05-20T16:04:22.860 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:04:22 smithi082 ceph-mon[32238]: [20/May/2024:16:04:21] ENGINE Serving on http://172.21.15.82:8765 2024-05-20T16:04:22.860 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:04:22 smithi082 ceph-mon[32238]: [20/May/2024:16:04:21] ENGINE Bus STARTED 2024-05-20T16:04:22.860 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:04:22 smithi082 ceph-mon[32238]: from='mgr.14118 172.21.15.82:0/3662546898' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T16:04:22.860 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:04:22 smithi082 ceph-mon[32238]: mgrmap e8: x(active, since 2s) 2024-05-20T16:04:23.197 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout value unchanged 2024-05-20T16:04:23.198 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:22.420+0000 7f096e402700 1 Processor -- start 2024-05-20T16:04:23.198 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:22.420+0000 7f096e402700 1 -- start start 2024-05-20T16:04:23.198 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:22.421+0000 7f096e402700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f096807c550 0x7f096810b650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:04:23.198 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:22.421+0000 7f096e402700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f096810bc60 con 0x7f096807c550 2024-05-20T16:04:23.199 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:22.421+0000 7f0967fff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f096807c550 0x7f096810b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:04:23.199 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:22.421+0000 7f0967fff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f096807c550 0x7f096810b650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43250/0 (socket says 172.21.15.82:43250) 2024-05-20T16:04:23.199 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:22.421+0000 7f0967fff700 1 -- 172.21.15.82:0/2535815900 learned_addr learned my addr 172.21.15.82:0/2535815900 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:04:23.200 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:22.421+0000 7f0967fff700 1 -- 172.21.15.82:0/2535815900 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f096810bde0 con 0x7f096807c550 2024-05-20T16:04:23.200 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:22.421+0000 7f0967fff700 1 --2- 172.21.15.82:0/2535815900 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f096807c550 0x7f096810b650 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f09580098a0 tx=0x7f0958009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5902251dd39e7692 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:04:23.200 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:22.421+0000 7f0966ffd700 1 -- 172.21.15.82:0/2535815900 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0958004160 con 0x7f096807c550 2024-05-20T16:04:23.255 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:22.421+0000 7f0966ffd700 1 -- 172.21.15.82:0/2535815900 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f0958004300 con 0x7f096807c550 2024-05-20T16:04:23.256 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:22.421+0000 7f0966ffd700 1 -- 172.21.15.82:0/2535815900 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0958011b50 con 0x7f096807c550 2024-05-20T16:04:23.256 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:22.422+0000 7f096e402700 1 -- 172.21.15.82:0/2535815900 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f096807c550 msgr2=0x7f096810b650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:04:23.256 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:22.422+0000 7f096e402700 1 --2- 172.21.15.82:0/2535815900 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f096807c550 0x7f096810b650 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f09580098a0 tx=0x7f0958009bb0 comp rx=0 tx=0).stop 2024-05-20T16:04:23.256 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:22.422+0000 7f096e402700 1 -- 172.21.15.82:0/2535815900 shutdown_connections 2024-05-20T16:04:23.256 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:22.422+0000 7f096e402700 1 --2- 172.21.15.82:0/2535815900 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f096807c550 0x7f096810b650 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:04:23.256 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:22.422+0000 7f096e402700 1 -- 172.21.15.82:0/2535815900 >> 172.21.15.82:0/2535815900 conn(0x7f0968103060 msgr2=0x7f0968105510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:04:23.256 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:22.422+0000 7f096e402700 1 -- 172.21.15.82:0/2535815900 shutdown_connections 2024-05-20T16:04:23.256 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:22.422+0000 7f096e402700 1 -- 172.21.15.82:0/2535815900 wait complete. 2024-05-20T16:04:23.256 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:22.423+0000 7f096e402700 1 Processor -- start 2024-05-20T16:04:23.257 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:22.423+0000 7f096e402700 1 -- start start 2024-05-20T16:04:23.257 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:22.423+0000 7f096e402700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f096807c550 0x7f096819cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:04:23.257 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:22.423+0000 7f096e402700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f096810cec0 con 0x7f096807c550 2024-05-20T16:04:23.257 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:22.423+0000 7f0967fff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f096807c550 0x7f096819cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:04:23.257 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:22.423+0000 7f0967fff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f096807c550 0x7f096819cdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43262/0 (socket says 172.21.15.82:43262) 2024-05-20T16:04:23.257 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:22.423+0000 7f0967fff700 1 -- 172.21.15.82:0/1427986176 learned_addr learned my addr 172.21.15.82:0/1427986176 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:04:23.257 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:22.424+0000 7f0967fff700 1 -- 172.21.15.82:0/1427986176 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f096819d330 con 0x7f096807c550 2024-05-20T16:04:23.257 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:22.424+0000 7f0967fff700 1 --2- 172.21.15.82:0/1427986176 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f096807c550 0x7f096819cdf0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f0958011850 tx=0x7f0958011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:04:23.257 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:22.424+0000 7f09657fa700 1 -- 172.21.15.82:0/1427986176 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0958003af0 con 0x7f096807c550 2024-05-20T16:04:23.257 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:22.424+0000 7f09657fa700 1 -- 172.21.15.82:0/1427986176 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f0958020070 con 0x7f096807c550 2024-05-20T16:04:23.257 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:22.424+0000 7f096e402700 1 -- 172.21.15.82:0/1427986176 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f096819d5c0 con 0x7f096807c550 2024-05-20T16:04:23.257 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:22.424+0000 7f09657fa700 1 -- 172.21.15.82:0/1427986176 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f09580195a0 con 0x7f096807c550 2024-05-20T16:04:23.258 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:22.424+0000 7f096e402700 1 -- 172.21.15.82:0/1427986176 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09681a01d0 con 0x7f096807c550 2024-05-20T16:04:23.258 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:22.425+0000 7f09657fa700 1 -- 172.21.15.82:0/1427986176 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f0958019850 con 0x7f096807c550 2024-05-20T16:04:23.258 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:22.425+0000 7f096e402700 1 -- 172.21.15.82:0/1427986176 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0968063ca0 con 0x7f096807c550 2024-05-20T16:04:23.258 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:22.425+0000 7f09657fa700 1 --2- 172.21.15.82:0/1427986176 >> [v2:172.21.15.82:6800/1763049308,v1:172.21.15.82:6801/1763049308] conn(0x7f0954039160 0x7f095403b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:04:23.258 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:22.425+0000 7f09657fa700 1 -- 172.21.15.82:0/1427986176 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f0958011e00 con 0x7f096807c550 2024-05-20T16:04:23.258 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:22.425+0000 7f09677fe700 1 --2- 172.21.15.82:0/1427986176 >> [v2:172.21.15.82:6800/1763049308,v1:172.21.15.82:6801/1763049308] conn(0x7f0954039160 0x7f095403b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:04:23.258 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:22.426+0000 7f09677fe700 1 --2- 172.21.15.82:0/1427986176 >> [v2:172.21.15.82:6800/1763049308,v1:172.21.15.82:6801/1763049308] conn(0x7f0954039160 0x7f095403b670 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f0950006fd0 tx=0x7f0950006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:04:23.258 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:22.430+0000 7f09657fa700 1 -- 172.21.15.82:0/1427986176 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f0958003c90 con 0x7f096807c550 2024-05-20T16:04:23.258 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:22.628+0000 7f096e402700 1 -- 172.21.15.82:0/1427986176 --> [v2:172.21.15.82:6800/1763049308,v1:172.21.15.82:6801/1763049308] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7f0968103fa0 con 0x7f0954039160 2024-05-20T16:04:23.258 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:22.629+0000 7f09657fa700 1 -- 172.21.15.82:0/1427986176 <== mgr.14118 v2:172.21.15.82:6800/1763049308 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7f0968103fa0 con 0x7f0954039160 2024-05-20T16:04:23.258 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:22.631+0000 7f096e402700 1 -- 172.21.15.82:0/1427986176 >> [v2:172.21.15.82:6800/1763049308,v1:172.21.15.82:6801/1763049308] conn(0x7f0954039160 msgr2=0x7f095403b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:04:23.259 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:22.631+0000 7f096e402700 1 --2- 172.21.15.82:0/1427986176 >> [v2:172.21.15.82:6800/1763049308,v1:172.21.15.82:6801/1763049308] conn(0x7f0954039160 0x7f095403b670 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f0950006fd0 tx=0x7f0950006e40 comp rx=0 tx=0).stop 2024-05-20T16:04:23.259 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:22.631+0000 7f096e402700 1 -- 172.21.15.82:0/1427986176 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f096807c550 msgr2=0x7f096819cdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:04:23.259 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:22.631+0000 7f096e402700 1 --2- 172.21.15.82:0/1427986176 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f096807c550 0x7f096819cdf0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f0958011850 tx=0x7f0958011c10 comp rx=0 tx=0).stop 2024-05-20T16:04:23.259 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:22.631+0000 7f096e402700 1 -- 172.21.15.82:0/1427986176 shutdown_connections 2024-05-20T16:04:23.259 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:22.631+0000 7f096e402700 1 --2- 172.21.15.82:0/1427986176 >> [v2:172.21.15.82:6800/1763049308,v1:172.21.15.82:6801/1763049308] conn(0x7f0954039160 0x7f095403b670 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:04:23.259 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:22.631+0000 7f096e402700 1 --2- 172.21.15.82:0/1427986176 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f096807c550 0x7f096819cdf0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:04:23.259 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:22.631+0000 7f096e402700 1 -- 172.21.15.82:0/1427986176 >> 172.21.15.82:0/1427986176 conn(0x7f0968103060 msgr2=0x7f0968103890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:04:23.259 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:22.631+0000 7f096e402700 1 -- 172.21.15.82:0/1427986176 shutdown_connections 2024-05-20T16:04:23.259 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:22.631+0000 7f096e402700 1 -- 172.21.15.82:0/1427986176 wait complete. 2024-05-20T16:04:23.259 INFO:teuthology.orchestra.run.smithi082.stdout:Generating ssh key... 2024-05-20T16:04:23.592 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:04:23 smithi082 ceph-mon[32238]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-05-20T16:04:24.213 INFO:journalctl@ceph.mgr.x.smithi082.stdout:May 20 16:04:23 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mgr-x[32467]: Generating public/private rsa key pair. 2024-05-20T16:04:24.213 INFO:journalctl@ceph.mgr.x.smithi082.stdout:May 20 16:04:23 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mgr-x[32467]: Your identification has been saved in /tmp/tmpbkvbeohb/key. 2024-05-20T16:04:24.213 INFO:journalctl@ceph.mgr.x.smithi082.stdout:May 20 16:04:23 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mgr-x[32467]: Your public key has been saved in /tmp/tmpbkvbeohb/key.pub. 2024-05-20T16:04:24.213 INFO:journalctl@ceph.mgr.x.smithi082.stdout:May 20 16:04:23 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mgr-x[32467]: The key fingerprint is: 2024-05-20T16:04:24.213 INFO:journalctl@ceph.mgr.x.smithi082.stdout:May 20 16:04:23 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mgr-x[32467]: SHA256:dEZUFKtyhTvi/WuR89AnW4MvgRJCXbetRAtzPk9z6aY ceph-59d1600a-16c2-11ef-bc9a-c7b262605968 2024-05-20T16:04:24.213 INFO:journalctl@ceph.mgr.x.smithi082.stdout:May 20 16:04:23 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mgr-x[32467]: The key's randomart image is: 2024-05-20T16:04:24.213 INFO:journalctl@ceph.mgr.x.smithi082.stdout:May 20 16:04:23 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mgr-x[32467]: +---[RSA 3072]----+ 2024-05-20T16:04:24.213 INFO:journalctl@ceph.mgr.x.smithi082.stdout:May 20 16:04:23 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mgr-x[32467]: | ..+*+= | 2024-05-20T16:04:24.213 INFO:journalctl@ceph.mgr.x.smithi082.stdout:May 20 16:04:23 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mgr-x[32467]: | . o .B.+ .| 2024-05-20T16:04:24.213 INFO:journalctl@ceph.mgr.x.smithi082.stdout:May 20 16:04:23 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mgr-x[32467]: | . . + o* =.| 2024-05-20T16:04:24.213 INFO:journalctl@ceph.mgr.x.smithi082.stdout:May 20 16:04:23 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mgr-x[32467]: | o + +. * o| 2024-05-20T16:04:24.213 INFO:journalctl@ceph.mgr.x.smithi082.stdout:May 20 16:04:23 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mgr-x[32467]: | S * .+.+ | 2024-05-20T16:04:24.213 INFO:journalctl@ceph.mgr.x.smithi082.stdout:May 20 16:04:23 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mgr-x[32467]: | . * o=o=.o| 2024-05-20T16:04:24.214 INFO:journalctl@ceph.mgr.x.smithi082.stdout:May 20 16:04:23 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mgr-x[32467]: | . o Eo=.| 2024-05-20T16:04:24.214 INFO:journalctl@ceph.mgr.x.smithi082.stdout:May 20 16:04:23 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mgr-x[32467]: | ...o. | 2024-05-20T16:04:24.214 INFO:journalctl@ceph.mgr.x.smithi082.stdout:May 20 16:04:23 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mgr-x[32467]: | .o.. | 2024-05-20T16:04:24.214 INFO:journalctl@ceph.mgr.x.smithi082.stdout:May 20 16:04:23 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mgr-x[32467]: +----[SHA256]-----+ 2024-05-20T16:04:24.511 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:23.625+0000 7f876e981700 1 Processor -- start 2024-05-20T16:04:24.511 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:23.626+0000 7f876e981700 1 -- start start 2024-05-20T16:04:24.511 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:23.626+0000 7f876e981700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8768107990 0x7f8768107da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:04:24.511 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:23.626+0000 7f876e981700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f8768108390 con 0x7f8768107990 2024-05-20T16:04:24.511 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:23.626+0000 7f8767fff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8768107990 0x7f8768107da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:04:24.511 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:23.626+0000 7f8767fff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8768107990 0x7f8768107da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43272/0 (socket says 172.21.15.82:43272) 2024-05-20T16:04:24.512 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:23.626+0000 7f8767fff700 1 -- 172.21.15.82:0/2079848822 learned_addr learned my addr 172.21.15.82:0/2079848822 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:04:24.512 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:23.626+0000 7f8767fff700 1 -- 172.21.15.82:0/2079848822 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8768108ac0 con 0x7f8768107990 2024-05-20T16:04:24.512 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:23.626+0000 7f8767fff700 1 --2- 172.21.15.82:0/2079848822 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8768107990 0x7f8768107da0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f875800bb40 tx=0x7f875800be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcd6277f2c217ed7 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:04:24.512 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:23.627+0000 7f87677fe700 1 -- 172.21.15.82:0/2079848822 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f875800dbe0 con 0x7f8768107990 2024-05-20T16:04:24.512 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:23.627+0000 7f87677fe700 1 -- 172.21.15.82:0/2079848822 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f87580044f0 con 0x7f8768107990 2024-05-20T16:04:24.512 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:23.627+0000 7f87677fe700 1 -- 172.21.15.82:0/2079848822 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8758010470 con 0x7f8768107990 2024-05-20T16:04:24.512 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:23.627+0000 7f876e981700 1 -- 172.21.15.82:0/2079848822 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8768107990 msgr2=0x7f8768107da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:04:24.512 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:23.627+0000 7f876e981700 1 --2- 172.21.15.82:0/2079848822 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8768107990 0x7f8768107da0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f875800bb40 tx=0x7f875800be50 comp rx=0 tx=0).stop 2024-05-20T16:04:24.512 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:23.627+0000 7f876e981700 1 -- 172.21.15.82:0/2079848822 shutdown_connections 2024-05-20T16:04:24.512 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:23.627+0000 7f876e981700 1 --2- 172.21.15.82:0/2079848822 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8768107990 0x7f8768107da0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:04:24.512 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:23.627+0000 7f876e981700 1 -- 172.21.15.82:0/2079848822 >> 172.21.15.82:0/2079848822 conn(0x7f8768103070 msgr2=0x7f87681054c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:04:24.512 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:23.628+0000 7f876e981700 1 -- 172.21.15.82:0/2079848822 shutdown_connections 2024-05-20T16:04:24.513 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:23.628+0000 7f876e981700 1 -- 172.21.15.82:0/2079848822 wait complete. 2024-05-20T16:04:24.513 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:23.628+0000 7f876e981700 1 Processor -- start 2024-05-20T16:04:24.513 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:23.628+0000 7f876e981700 1 -- start start 2024-05-20T16:04:24.513 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:23.628+0000 7f876e981700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8768107990 0x7f876819abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:04:24.513 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:23.628+0000 7f876e981700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f876807b140 con 0x7f8768107990 2024-05-20T16:04:24.513 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:23.629+0000 7f8767fff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8768107990 0x7f876819abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:04:24.513 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:23.629+0000 7f8767fff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8768107990 0x7f876819abf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43274/0 (socket says 172.21.15.82:43274) 2024-05-20T16:04:24.513 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:23.629+0000 7f8767fff700 1 -- 172.21.15.82:0/3668368281 learned_addr learned my addr 172.21.15.82:0/3668368281 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:04:24.513 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:23.629+0000 7f8767fff700 1 -- 172.21.15.82:0/3668368281 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f876819b130 con 0x7f8768107990 2024-05-20T16:04:24.513 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:23.629+0000 7f8767fff700 1 --2- 172.21.15.82:0/3668368281 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8768107990 0x7f876819abf0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f875800b340 tx=0x7f875800dd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:04:24.513 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:23.629+0000 7f8765ffb700 1 -- 172.21.15.82:0/3668368281 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8758027070 con 0x7f8768107990 2024-05-20T16:04:24.513 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:23.629+0000 7f876e981700 1 -- 172.21.15.82:0/3668368281 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f876819b3c0 con 0x7f8768107990 2024-05-20T16:04:24.513 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:23.629+0000 7f8765ffb700 1 -- 172.21.15.82:0/3668368281 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f8758003dd0 con 0x7f8768107990 2024-05-20T16:04:24.513 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:23.629+0000 7f8765ffb700 1 -- 172.21.15.82:0/3668368281 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8758022a90 con 0x7f8768107990 2024-05-20T16:04:24.514 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:23.629+0000 7f876e981700 1 -- 172.21.15.82:0/3668368281 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f876819dfd0 con 0x7f8768107990 2024-05-20T16:04:24.514 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:23.630+0000 7f8765ffb700 1 -- 172.21.15.82:0/3668368281 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f875802a070 con 0x7f8768107990 2024-05-20T16:04:24.514 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:23.630+0000 7f876e981700 1 -- 172.21.15.82:0/3668368281 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8768063ca0 con 0x7f8768107990 2024-05-20T16:04:24.514 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:23.630+0000 7f8765ffb700 1 --2- 172.21.15.82:0/3668368281 >> [v2:172.21.15.82:6800/1763049308,v1:172.21.15.82:6801/1763049308] conn(0x7f8754039110 0x7f875403b620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:04:24.514 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:23.630+0000 7f8765ffb700 1 -- 172.21.15.82:0/3668368281 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f875801f070 con 0x7f8768107990 2024-05-20T16:04:24.514 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:23.630+0000 7f875ffff700 1 --2- 172.21.15.82:0/3668368281 >> [v2:172.21.15.82:6800/1763049308,v1:172.21.15.82:6801/1763049308] conn(0x7f8754039110 0x7f875403b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:04:24.514 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:23.631+0000 7f875ffff700 1 --2- 172.21.15.82:0/3668368281 >> [v2:172.21.15.82:6800/1763049308,v1:172.21.15.82:6801/1763049308] conn(0x7f8754039110 0x7f875403b620 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f8750006fd0 tx=0x7f8750006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:04:24.514 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:23.635+0000 7f8765ffb700 1 -- 172.21.15.82:0/3668368281 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f8758015070 con 0x7f8768107990 2024-05-20T16:04:24.514 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:23.832+0000 7f876e981700 1 -- 172.21.15.82:0/3668368281 --> [v2:172.21.15.82:6800/1763049308,v1:172.21.15.82:6801/1763049308] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7f8768062a80 con 0x7f8754039110 2024-05-20T16:04:24.514 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:23.935+0000 7f8765ffb700 1 -- 172.21.15.82:0/3668368281 <== mgr.14118 v2:172.21.15.82:6800/1763049308 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f8768062a80 con 0x7f8754039110 2024-05-20T16:04:24.514 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:23.937+0000 7f876e981700 1 -- 172.21.15.82:0/3668368281 >> [v2:172.21.15.82:6800/1763049308,v1:172.21.15.82:6801/1763049308] conn(0x7f8754039110 msgr2=0x7f875403b620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:04:24.514 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:23.937+0000 7f876e981700 1 --2- 172.21.15.82:0/3668368281 >> [v2:172.21.15.82:6800/1763049308,v1:172.21.15.82:6801/1763049308] conn(0x7f8754039110 0x7f875403b620 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f8750006fd0 tx=0x7f8750006e40 comp rx=0 tx=0).stop 2024-05-20T16:04:24.514 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:23.937+0000 7f876e981700 1 -- 172.21.15.82:0/3668368281 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8768107990 msgr2=0x7f876819abf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:04:24.514 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:23.937+0000 7f876e981700 1 --2- 172.21.15.82:0/3668368281 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8768107990 0x7f876819abf0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f875800b340 tx=0x7f875800dd90 comp rx=0 tx=0).stop 2024-05-20T16:04:24.515 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:23.937+0000 7f876e981700 1 -- 172.21.15.82:0/3668368281 shutdown_connections 2024-05-20T16:04:24.515 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:23.937+0000 7f876e981700 1 --2- 172.21.15.82:0/3668368281 >> [v2:172.21.15.82:6800/1763049308,v1:172.21.15.82:6801/1763049308] conn(0x7f8754039110 0x7f875403b620 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:04:24.515 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:23.937+0000 7f876e981700 1 --2- 172.21.15.82:0/3668368281 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8768107990 0x7f876819abf0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:04:24.515 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:23.937+0000 7f876e981700 1 -- 172.21.15.82:0/3668368281 >> 172.21.15.82:0/3668368281 conn(0x7f8768103070 msgr2=0x7f8768190ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:04:24.515 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:23.937+0000 7f876e981700 1 -- 172.21.15.82:0/3668368281 shutdown_connections 2024-05-20T16:04:24.515 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:23.937+0000 7f876e981700 1 -- 172.21.15.82:0/3668368281 wait complete. 2024-05-20T16:04:25.097 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:04:24 smithi082 ceph-mon[32238]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-05-20T16:04:25.097 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:04:24 smithi082 ceph-mon[32238]: Generating ssh key... 2024-05-20T16:04:25.097 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:04:24 smithi082 ceph-mon[32238]: from='mgr.14118 172.21.15.82:0/3662546898' entity='mgr.x' 2024-05-20T16:04:25.098 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:04:24 smithi082 ceph-mon[32238]: from='mgr.14118 172.21.15.82:0/3662546898' entity='mgr.x' 2024-05-20T16:04:25.627 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDGtQRYMOcyJ3OXiToYCqyh3ktsCjbrNxDoRP0B8FURq58NsWXQ1pNHE8bJk8h5sLQoImUlF8T/46ONtqlUa9ozxChtzxPjwfHcfmH2CTJLMAkV6DwN55FRtP3aVUEQ0ikGJMU16LDpMkVErBTvHS/5kTszwW75VQ6eZTeBs9fLoJ92ahxwJo898SLMMWzRsDKJZ8REkoJzocBQaztta/wDxmN34n4k+dUPYvvDmseaMD8j5RwL9JvCafs9H7W+7ho8vajoywUv7mZi4IgHUNEXQUe0uGEILbnpT16d4au3exIL4hRVUVJdAaaunQNDos6YygHfN/tY/8R+ARcbKLoT2vfeRXZlGcR0rPTUp11CUpN63Bpt6dgsusqDrBD3oXq/XQ2gIW2OgoWrgvicAhLsj1pb3i1L85kVRsUQcg+wTq3PFykRdLepp0tSGW37yoZn+fZnuWoosOvesOCV/Q9cRJKl/wgg0DkOJK4TBG7/sJ2XRrx384iTsr4/LUmzdDM= ceph-59d1600a-16c2-11ef-bc9a-c7b262605968 2024-05-20T16:04:25.627 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:24.891+0000 7f5ba4569700 1 Processor -- start 2024-05-20T16:04:25.627 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:24.891+0000 7f5ba4569700 1 -- start start 2024-05-20T16:04:25.627 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:24.891+0000 7f5ba4569700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5b9c106e10 0x7f5b9c107220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:04:25.627 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:24.891+0000 7f5ba4569700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f5b9c107810 con 0x7f5b9c106e10 2024-05-20T16:04:25.627 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:24.891+0000 7f5ba2305700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5b9c106e10 0x7f5b9c107220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:04:25.627 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:24.891+0000 7f5ba2305700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5b9c106e10 0x7f5b9c107220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43276/0 (socket says 172.21.15.82:43276) 2024-05-20T16:04:25.627 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:24.891+0000 7f5ba2305700 1 -- 172.21.15.82:0/735313825 learned_addr learned my addr 172.21.15.82:0/735313825 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:04:25.627 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:24.891+0000 7f5ba2305700 1 -- 172.21.15.82:0/735313825 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b9c107990 con 0x7f5b9c106e10 2024-05-20T16:04:25.627 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:24.892+0000 7f5ba2305700 1 --2- 172.21.15.82:0/735313825 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5b9c106e10 0x7f5b9c107220 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f5b8c0098a0 tx=0x7f5b8c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=283a22ab608c74a3 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:04:25.627 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:24.892+0000 7f5ba1303700 1 -- 172.21.15.82:0/735313825 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5b8c004160 con 0x7f5b9c106e10 2024-05-20T16:04:25.628 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:24.892+0000 7f5ba1303700 1 -- 172.21.15.82:0/735313825 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f5b8c004300 con 0x7f5b9c106e10 2024-05-20T16:04:25.628 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:24.892+0000 7f5ba1303700 1 -- 172.21.15.82:0/735313825 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5b8c011b50 con 0x7f5b9c106e10 2024-05-20T16:04:25.628 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:24.892+0000 7f5ba4569700 1 -- 172.21.15.82:0/735313825 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5b9c106e10 msgr2=0x7f5b9c107220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:04:25.628 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:24.892+0000 7f5ba4569700 1 --2- 172.21.15.82:0/735313825 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5b9c106e10 0x7f5b9c107220 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f5b8c0098a0 tx=0x7f5b8c009bb0 comp rx=0 tx=0).stop 2024-05-20T16:04:25.628 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:24.893+0000 7f5ba4569700 1 -- 172.21.15.82:0/735313825 shutdown_connections 2024-05-20T16:04:25.628 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:24.893+0000 7f5ba4569700 1 --2- 172.21.15.82:0/735313825 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5b9c106e10 0x7f5b9c107220 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:04:25.628 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:24.893+0000 7f5ba4569700 1 -- 172.21.15.82:0/735313825 >> 172.21.15.82:0/735313825 conn(0x7f5b9c1028b0 msgr2=0x7f5b9c104d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:04:25.628 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:24.893+0000 7f5ba4569700 1 -- 172.21.15.82:0/735313825 shutdown_connections 2024-05-20T16:04:25.628 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:24.893+0000 7f5ba4569700 1 -- 172.21.15.82:0/735313825 wait complete. 2024-05-20T16:04:25.628 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:24.893+0000 7f5ba4569700 1 Processor -- start 2024-05-20T16:04:25.628 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:24.893+0000 7f5ba4569700 1 -- start start 2024-05-20T16:04:25.629 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:24.894+0000 7f5ba4569700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5b9c106e10 0x7f5b9c19ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:04:25.629 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:24.894+0000 7f5ba4569700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f5b9c10cea0 con 0x7f5b9c106e10 2024-05-20T16:04:25.629 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:24.894+0000 7f5ba2305700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5b9c106e10 0x7f5b9c19ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:04:25.629 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:24.894+0000 7f5ba2305700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5b9c106e10 0x7f5b9c19ce10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43278/0 (socket says 172.21.15.82:43278) 2024-05-20T16:04:25.629 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:24.894+0000 7f5ba2305700 1 -- 172.21.15.82:0/3529299220 learned_addr learned my addr 172.21.15.82:0/3529299220 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:04:25.629 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:24.894+0000 7f5ba2305700 1 -- 172.21.15.82:0/3529299220 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b9c19d350 con 0x7f5b9c106e10 2024-05-20T16:04:25.629 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:24.894+0000 7f5ba2305700 1 --2- 172.21.15.82:0/3529299220 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5b9c106e10 0x7f5b9c19ce10 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f5b8c011850 tx=0x7f5b8c011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:04:25.629 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:24.894+0000 7f5b937fe700 1 -- 172.21.15.82:0/3529299220 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5b8c003af0 con 0x7f5b9c106e10 2024-05-20T16:04:25.629 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:24.894+0000 7f5b937fe700 1 -- 172.21.15.82:0/3529299220 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f5b8c020070 con 0x7f5b9c106e10 2024-05-20T16:04:25.629 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:24.894+0000 7f5ba4569700 1 -- 172.21.15.82:0/3529299220 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b9c19d5e0 con 0x7f5b9c106e10 2024-05-20T16:04:25.629 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:24.894+0000 7f5b937fe700 1 -- 172.21.15.82:0/3529299220 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5b8c0195a0 con 0x7f5b9c106e10 2024-05-20T16:04:25.629 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:24.894+0000 7f5ba4569700 1 -- 172.21.15.82:0/3529299220 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b9c1a01f0 con 0x7f5b9c106e10 2024-05-20T16:04:25.629 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:24.895+0000 7f5b937fe700 1 -- 172.21.15.82:0/3529299220 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f5b8c019850 con 0x7f5b9c106e10 2024-05-20T16:04:25.629 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:24.895+0000 7f5b937fe700 1 --2- 172.21.15.82:0/3529299220 >> [v2:172.21.15.82:6800/1763049308,v1:172.21.15.82:6801/1763049308] conn(0x7f5b88039160 0x7f5b8803b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:04:25.630 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:24.895+0000 7f5b937fe700 1 -- 172.21.15.82:0/3529299220 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f5b8c011e00 con 0x7f5b9c106e10 2024-05-20T16:04:25.630 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:24.895+0000 7f5ba4569700 1 -- 172.21.15.82:0/3529299220 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b9c063ca0 con 0x7f5b9c106e10 2024-05-20T16:04:25.630 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:24.896+0000 7f5ba1b04700 1 --2- 172.21.15.82:0/3529299220 >> [v2:172.21.15.82:6800/1763049308,v1:172.21.15.82:6801/1763049308] conn(0x7f5b88039160 0x7f5b8803b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:04:25.630 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:24.896+0000 7f5ba1b04700 1 --2- 172.21.15.82:0/3529299220 >> [v2:172.21.15.82:6800/1763049308,v1:172.21.15.82:6801/1763049308] conn(0x7f5b88039160 0x7f5b8803b670 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f5b98006fd0 tx=0x7f5b98006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:04:25.630 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:24.901+0000 7f5b937fe700 1 -- 172.21.15.82:0/3529299220 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f5b8c003c90 con 0x7f5b9c106e10 2024-05-20T16:04:25.630 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:25.096+0000 7f5ba4569700 1 -- 172.21.15.82:0/3529299220 --> [v2:172.21.15.82:6800/1763049308,v1:172.21.15.82:6801/1763049308] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7f5b9c1048e0 con 0x7f5b88039160 2024-05-20T16:04:25.630 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:25.097+0000 7f5b937fe700 1 -- 172.21.15.82:0/3529299220 <== mgr.14118 v2:172.21.15.82:6800/1763049308 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7f5b9c1048e0 con 0x7f5b88039160 2024-05-20T16:04:25.630 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:25.099+0000 7f5ba4569700 1 -- 172.21.15.82:0/3529299220 >> [v2:172.21.15.82:6800/1763049308,v1:172.21.15.82:6801/1763049308] conn(0x7f5b88039160 msgr2=0x7f5b8803b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:04:25.630 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:25.099+0000 7f5ba4569700 1 --2- 172.21.15.82:0/3529299220 >> [v2:172.21.15.82:6800/1763049308,v1:172.21.15.82:6801/1763049308] conn(0x7f5b88039160 0x7f5b8803b670 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f5b98006fd0 tx=0x7f5b98006e40 comp rx=0 tx=0).stop 2024-05-20T16:04:25.630 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:25.099+0000 7f5ba4569700 1 -- 172.21.15.82:0/3529299220 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5b9c106e10 msgr2=0x7f5b9c19ce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:04:25.630 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:25.099+0000 7f5ba4569700 1 --2- 172.21.15.82:0/3529299220 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5b9c106e10 0x7f5b9c19ce10 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f5b8c011850 tx=0x7f5b8c011c10 comp rx=0 tx=0).stop 2024-05-20T16:04:25.630 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:25.099+0000 7f5ba4569700 1 -- 172.21.15.82:0/3529299220 shutdown_connections 2024-05-20T16:04:25.631 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:25.099+0000 7f5ba4569700 1 --2- 172.21.15.82:0/3529299220 >> [v2:172.21.15.82:6800/1763049308,v1:172.21.15.82:6801/1763049308] conn(0x7f5b88039160 0x7f5b8803b670 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:04:25.631 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:25.099+0000 7f5ba4569700 1 --2- 172.21.15.82:0/3529299220 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5b9c106e10 0x7f5b9c19ce10 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:04:25.631 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:25.099+0000 7f5ba4569700 1 -- 172.21.15.82:0/3529299220 >> 172.21.15.82:0/3529299220 conn(0x7f5b9c1028b0 msgr2=0x7f5b9c1041d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:04:25.631 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:25.099+0000 7f5ba4569700 1 -- 172.21.15.82:0/3529299220 shutdown_connections 2024-05-20T16:04:25.631 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:25.099+0000 7f5ba4569700 1 -- 172.21.15.82:0/3529299220 wait complete. 2024-05-20T16:04:25.631 INFO:teuthology.orchestra.run.smithi082.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-05-20T16:04:25.631 INFO:teuthology.orchestra.run.smithi082.stdout:Adding key to root@localhost authorized_keys... 2024-05-20T16:04:25.631 INFO:teuthology.orchestra.run.smithi082.stdout:Adding host smithi082... 2024-05-20T16:04:26.234 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:04:25 smithi082 ceph-mon[32238]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-05-20T16:04:27.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:04:26 smithi082 ceph-mon[32238]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi082", "addr": "172.21.15.82", "target": ["mon-mgr", ""]}]: dispatch 2024-05-20T16:04:28.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:04:27 smithi082 ceph-mon[32238]: Deploying cephadm binary to smithi082 2024-05-20T16:04:30.879 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout Added host 'smithi082' with addr '172.21.15.82' 2024-05-20T16:04:30.880 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:26.025+0000 7f19a5da6700 1 Processor -- start 2024-05-20T16:04:30.880 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:26.025+0000 7f19a5da6700 1 -- start start 2024-05-20T16:04:30.880 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:26.026+0000 7f19a5da6700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f19a0107990 0x7f19a0107da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:04:30.880 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:26.026+0000 7f19a5da6700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f19a0108390 con 0x7f19a0107990 2024-05-20T16:04:30.880 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:26.026+0000 7f199f7fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f19a0107990 0x7f19a0107da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:04:30.880 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:26.026+0000 7f199f7fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f19a0107990 0x7f19a0107da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43284/0 (socket says 172.21.15.82:43284) 2024-05-20T16:04:30.880 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:26.026+0000 7f199f7fe700 1 -- 172.21.15.82:0/2011880521 learned_addr learned my addr 172.21.15.82:0/2011880521 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:04:30.880 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:26.026+0000 7f199f7fe700 1 -- 172.21.15.82:0/2011880521 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19a0108ac0 con 0x7f19a0107990 2024-05-20T16:04:30.880 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:26.026+0000 7f199f7fe700 1 --2- 172.21.15.82:0/2011880521 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f19a0107990 0x7f19a0107da0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f19900098a0 tx=0x7f1990009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd10f30cf3494b98 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:04:30.880 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:26.026+0000 7f199effd700 1 -- 172.21.15.82:0/2011880521 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1990004160 con 0x7f19a0107990 2024-05-20T16:04:30.880 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:26.026+0000 7f199effd700 1 -- 172.21.15.82:0/2011880521 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f1990004300 con 0x7f19a0107990 2024-05-20T16:04:30.880 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:26.026+0000 7f199effd700 1 -- 172.21.15.82:0/2011880521 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1990011b50 con 0x7f19a0107990 2024-05-20T16:04:30.881 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:26.027+0000 7f19a5da6700 1 -- 172.21.15.82:0/2011880521 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f19a0107990 msgr2=0x7f19a0107da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:04:30.881 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:26.027+0000 7f19a5da6700 1 --2- 172.21.15.82:0/2011880521 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f19a0107990 0x7f19a0107da0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f19900098a0 tx=0x7f1990009bb0 comp rx=0 tx=0).stop 2024-05-20T16:04:30.881 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:26.027+0000 7f19a5da6700 1 -- 172.21.15.82:0/2011880521 shutdown_connections 2024-05-20T16:04:30.881 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:26.027+0000 7f19a5da6700 1 --2- 172.21.15.82:0/2011880521 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f19a0107990 0x7f19a0107da0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:04:30.881 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:26.027+0000 7f19a5da6700 1 -- 172.21.15.82:0/2011880521 >> 172.21.15.82:0/2011880521 conn(0x7f19a0103070 msgr2=0x7f19a01054c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:04:30.881 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:26.027+0000 7f19a5da6700 1 -- 172.21.15.82:0/2011880521 shutdown_connections 2024-05-20T16:04:30.881 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:26.027+0000 7f19a5da6700 1 -- 172.21.15.82:0/2011880521 wait complete. 2024-05-20T16:04:30.881 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:26.028+0000 7f19a5da6700 1 Processor -- start 2024-05-20T16:04:30.881 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:26.028+0000 7f19a5da6700 1 -- start start 2024-05-20T16:04:30.881 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:26.028+0000 7f19a5da6700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f19a0107990 0x7f19a019abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:04:30.881 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:26.028+0000 7f19a5da6700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f19a007b140 con 0x7f19a0107990 2024-05-20T16:04:30.881 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:26.028+0000 7f199f7fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f19a0107990 0x7f19a019abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:04:30.881 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:26.028+0000 7f199f7fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f19a0107990 0x7f19a019abf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43298/0 (socket says 172.21.15.82:43298) 2024-05-20T16:04:30.881 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:26.028+0000 7f199f7fe700 1 -- 172.21.15.82:0/4196261401 learned_addr learned my addr 172.21.15.82:0/4196261401 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:04:30.882 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:26.029+0000 7f199f7fe700 1 -- 172.21.15.82:0/4196261401 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19a019b130 con 0x7f19a0107990 2024-05-20T16:04:30.882 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:26.029+0000 7f199f7fe700 1 --2- 172.21.15.82:0/4196261401 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f19a0107990 0x7f19a019abf0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f1990009fb0 tx=0x7f1990011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:04:30.882 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:26.029+0000 7f199d7fa700 1 -- 172.21.15.82:0/4196261401 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f19900038b0 con 0x7f19a0107990 2024-05-20T16:04:30.882 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:26.029+0000 7f19a5da6700 1 -- 172.21.15.82:0/4196261401 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19a019b3c0 con 0x7f19a0107990 2024-05-20T16:04:30.882 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:26.029+0000 7f199d7fa700 1 -- 172.21.15.82:0/4196261401 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f1990011e50 con 0x7f19a0107990 2024-05-20T16:04:30.882 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:26.029+0000 7f199d7fa700 1 -- 172.21.15.82:0/4196261401 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f199001a5c0 con 0x7f19a0107990 2024-05-20T16:04:30.882 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:26.029+0000 7f19a5da6700 1 -- 172.21.15.82:0/4196261401 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19a019dfd0 con 0x7f19a0107990 2024-05-20T16:04:30.882 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:26.030+0000 7f199d7fa700 1 -- 172.21.15.82:0/4196261401 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f199001a8c0 con 0x7f19a0107990 2024-05-20T16:04:30.882 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:26.030+0000 7f19a5da6700 1 -- 172.21.15.82:0/4196261401 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19a0063ca0 con 0x7f19a0107990 2024-05-20T16:04:30.882 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:26.030+0000 7f199d7fa700 1 --2- 172.21.15.82:0/4196261401 >> [v2:172.21.15.82:6800/1763049308,v1:172.21.15.82:6801/1763049308] conn(0x7f198c03b2d0 0x7f198c03d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:04:30.882 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:26.030+0000 7f199d7fa700 1 -- 172.21.15.82:0/4196261401 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f199004d4c0 con 0x7f19a0107990 2024-05-20T16:04:30.882 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:26.030+0000 7f1997fff700 1 --2- 172.21.15.82:0/4196261401 >> [v2:172.21.15.82:6800/1763049308,v1:172.21.15.82:6801/1763049308] conn(0x7f198c03b2d0 0x7f198c03d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:04:30.882 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:26.031+0000 7f1997fff700 1 --2- 172.21.15.82:0/4196261401 >> [v2:172.21.15.82:6800/1763049308,v1:172.21.15.82:6801/1763049308] conn(0x7f198c03b2d0 0x7f198c03d7e0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f1988006fd0 tx=0x7f1988006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:04:30.883 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:26.035+0000 7f199d7fa700 1 -- 172.21.15.82:0/4196261401 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f1990024900 con 0x7f19a0107990 2024-05-20T16:04:30.883 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:26.233+0000 7f19a5da6700 1 -- 172.21.15.82:0/4196261401 --> [v2:172.21.15.82:6800/1763049308,v1:172.21.15.82:6801/1763049308] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi082", "addr": "172.21.15.82", "target": ["mon-mgr", ""]}) -- 0x7f19a0062a80 con 0x7f198c03b2d0 2024-05-20T16:04:30.883 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:30.339+0000 7f199d7fa700 1 -- 172.21.15.82:0/4196261401 <== mgr.14118 v2:172.21.15.82:6800/1763049308 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7f19a0062a80 con 0x7f198c03b2d0 2024-05-20T16:04:30.883 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:30.342+0000 7f19a5da6700 1 -- 172.21.15.82:0/4196261401 >> [v2:172.21.15.82:6800/1763049308,v1:172.21.15.82:6801/1763049308] conn(0x7f198c03b2d0 msgr2=0x7f198c03d7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:04:30.883 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:30.342+0000 7f19a5da6700 1 --2- 172.21.15.82:0/4196261401 >> [v2:172.21.15.82:6800/1763049308,v1:172.21.15.82:6801/1763049308] conn(0x7f198c03b2d0 0x7f198c03d7e0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f1988006fd0 tx=0x7f1988006e40 comp rx=0 tx=0).stop 2024-05-20T16:04:30.883 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:30.342+0000 7f19a5da6700 1 -- 172.21.15.82:0/4196261401 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f19a0107990 msgr2=0x7f19a019abf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:04:30.883 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:30.342+0000 7f19a5da6700 1 --2- 172.21.15.82:0/4196261401 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f19a0107990 0x7f19a019abf0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f1990009fb0 tx=0x7f1990011c10 comp rx=0 tx=0).stop 2024-05-20T16:04:30.883 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:30.342+0000 7f19a5da6700 1 -- 172.21.15.82:0/4196261401 shutdown_connections 2024-05-20T16:04:30.883 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:30.342+0000 7f19a5da6700 1 --2- 172.21.15.82:0/4196261401 >> [v2:172.21.15.82:6800/1763049308,v1:172.21.15.82:6801/1763049308] conn(0x7f198c03b2d0 0x7f198c03d7e0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:04:30.883 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:30.342+0000 7f19a5da6700 1 --2- 172.21.15.82:0/4196261401 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f19a0107990 0x7f19a019abf0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:04:30.883 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:30.342+0000 7f19a5da6700 1 -- 172.21.15.82:0/4196261401 >> 172.21.15.82:0/4196261401 conn(0x7f19a0103070 msgr2=0x7f19a0190ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:04:30.883 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:30.342+0000 7f19a5da6700 1 -- 172.21.15.82:0/4196261401 shutdown_connections 2024-05-20T16:04:30.883 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:30.342+0000 7f19a5da6700 1 -- 172.21.15.82:0/4196261401 wait complete. 2024-05-20T16:04:30.883 INFO:teuthology.orchestra.run.smithi082.stdout:Deploying unmanaged mon service... 2024-05-20T16:04:31.523 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:04:31 smithi082 ceph-mon[32238]: from='mgr.14118 172.21.15.82:0/3662546898' entity='mgr.x' 2024-05-20T16:04:31.524 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:04:31 smithi082 ceph-mon[32238]: from='mgr.14118 172.21.15.82:0/3662546898' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T16:04:32.545 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-05-20T16:04:32.545 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:31.383+0000 7f8d16a8a700 1 Processor -- start 2024-05-20T16:04:32.545 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:31.383+0000 7f8d16a8a700 1 -- start start 2024-05-20T16:04:32.545 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:31.383+0000 7f8d16a8a700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8d10109570 0x7f8d1010b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:04:32.546 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:31.383+0000 7f8d16a8a700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f8d1007bdd0 con 0x7f8d10109570 2024-05-20T16:04:32.546 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:31.384+0000 7f8d14826700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8d10109570 0x7f8d1010b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:04:32.546 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:31.384+0000 7f8d14826700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8d10109570 0x7f8d1010b990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:57278/0 (socket says 172.21.15.82:57278) 2024-05-20T16:04:32.546 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:31.384+0000 7f8d14826700 1 -- 172.21.15.82:0/838581804 learned_addr learned my addr 172.21.15.82:0/838581804 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:04:32.546 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:31.384+0000 7f8d14826700 1 -- 172.21.15.82:0/838581804 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d1010bed0 con 0x7f8d10109570 2024-05-20T16:04:32.546 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:31.384+0000 7f8d14826700 1 --2- 172.21.15.82:0/838581804 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8d10109570 0x7f8d1010b990 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f8d04009e80 tx=0x7f8d0400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a295e62bb424e1b server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:04:32.546 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:31.384+0000 7f8d0f7fe700 1 -- 172.21.15.82:0/838581804 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8d04017070 con 0x7f8d10109570 2024-05-20T16:04:32.546 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:31.384+0000 7f8d0f7fe700 1 -- 172.21.15.82:0/838581804 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f8d0400b8c0 con 0x7f8d10109570 2024-05-20T16:04:32.546 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:31.384+0000 7f8d0f7fe700 1 -- 172.21.15.82:0/838581804 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8d040045c0 con 0x7f8d10109570 2024-05-20T16:04:32.546 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:31.385+0000 7f8d16a8a700 1 -- 172.21.15.82:0/838581804 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8d10109570 msgr2=0x7f8d1010b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:04:32.546 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:31.385+0000 7f8d16a8a700 1 --2- 172.21.15.82:0/838581804 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8d10109570 0x7f8d1010b990 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f8d04009e80 tx=0x7f8d0400b330 comp rx=0 tx=0).stop 2024-05-20T16:04:32.546 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:31.385+0000 7f8d16a8a700 1 -- 172.21.15.82:0/838581804 shutdown_connections 2024-05-20T16:04:32.546 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:31.385+0000 7f8d16a8a700 1 --2- 172.21.15.82:0/838581804 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8d10109570 0x7f8d1010b990 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:04:32.547 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:31.385+0000 7f8d16a8a700 1 -- 172.21.15.82:0/838581804 >> 172.21.15.82:0/838581804 conn(0x7f8d10103070 msgr2=0x7f8d101054c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:04:32.547 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:31.385+0000 7f8d16a8a700 1 -- 172.21.15.82:0/838581804 shutdown_connections 2024-05-20T16:04:32.547 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:31.385+0000 7f8d16a8a700 1 -- 172.21.15.82:0/838581804 wait complete. 2024-05-20T16:04:32.547 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:31.386+0000 7f8d16a8a700 1 Processor -- start 2024-05-20T16:04:32.547 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:31.386+0000 7f8d16a8a700 1 -- start start 2024-05-20T16:04:32.547 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:31.386+0000 7f8d16a8a700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8d10109570 0x7f8d1019cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:04:32.547 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:31.386+0000 7f8d16a8a700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f8d1010ce30 con 0x7f8d10109570 2024-05-20T16:04:32.547 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:31.386+0000 7f8d14826700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8d10109570 0x7f8d1019cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:04:32.547 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:31.386+0000 7f8d14826700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8d10109570 0x7f8d1019cd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:57292/0 (socket says 172.21.15.82:57292) 2024-05-20T16:04:32.547 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:31.386+0000 7f8d14826700 1 -- 172.21.15.82:0/1904074244 learned_addr learned my addr 172.21.15.82:0/1904074244 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:04:32.547 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:31.386+0000 7f8d14826700 1 -- 172.21.15.82:0/1904074244 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d1019d2a0 con 0x7f8d10109570 2024-05-20T16:04:32.547 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:31.386+0000 7f8d14826700 1 --2- 172.21.15.82:0/1904074244 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8d10109570 0x7f8d1019cd60 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f8d04009e50 tx=0x7f8d0400bd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:04:32.547 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:31.387+0000 7f8d0dffb700 1 -- 172.21.15.82:0/1904074244 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8d04020020 con 0x7f8d10109570 2024-05-20T16:04:32.548 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:31.387+0000 7f8d0dffb700 1 -- 172.21.15.82:0/1904074244 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f8d04003b50 con 0x7f8d10109570 2024-05-20T16:04:32.548 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:31.387+0000 7f8d0dffb700 1 -- 172.21.15.82:0/1904074244 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8d04017040 con 0x7f8d10109570 2024-05-20T16:04:32.548 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:31.387+0000 7f8d16a8a700 1 -- 172.21.15.82:0/1904074244 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d1019d530 con 0x7f8d10109570 2024-05-20T16:04:32.548 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:31.387+0000 7f8d16a8a700 1 -- 172.21.15.82:0/1904074244 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d101a0130 con 0x7f8d10109570 2024-05-20T16:04:32.548 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:31.388+0000 7f8d0dffb700 1 -- 172.21.15.82:0/1904074244 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f8d04023020 con 0x7f8d10109570 2024-05-20T16:04:32.548 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:31.388+0000 7f8d16a8a700 1 -- 172.21.15.82:0/1904074244 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d10063ca0 con 0x7f8d10109570 2024-05-20T16:04:32.548 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:31.388+0000 7f8d0dffb700 1 --2- 172.21.15.82:0/1904074244 >> [v2:172.21.15.82:6800/1763049308,v1:172.21.15.82:6801/1763049308] conn(0x7f8d00039110 0x7f8d0003b620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:04:32.548 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:31.388+0000 7f8d0dffb700 1 -- 172.21.15.82:0/1904074244 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f8d0400f050 con 0x7f8d10109570 2024-05-20T16:04:32.548 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:31.388+0000 7f8d0ffff700 1 --2- 172.21.15.82:0/1904074244 >> [v2:172.21.15.82:6800/1763049308,v1:172.21.15.82:6801/1763049308] conn(0x7f8d00039110 0x7f8d0003b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:04:32.548 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:31.388+0000 7f8d0ffff700 1 --2- 172.21.15.82:0/1904074244 >> [v2:172.21.15.82:6800/1763049308,v1:172.21.15.82:6801/1763049308] conn(0x7f8d00039110 0x7f8d0003b620 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f8cfc006fd0 tx=0x7f8cfc006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:04:32.548 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:31.393+0000 7f8d0dffb700 1 -- 172.21.15.82:0/1904074244 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f8d0402a020 con 0x7f8d10109570 2024-05-20T16:04:32.548 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:31.615+0000 7f8d16a8a700 1 -- 172.21.15.82:0/1904074244 --> [v2:172.21.15.82:6800/1763049308,v1:172.21.15.82:6801/1763049308] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f8d10105030 con 0x7f8d00039110 2024-05-20T16:04:32.548 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:31.621+0000 7f8d0dffb700 1 -- 172.21.15.82:0/1904074244 <== mgr.14118 v2:172.21.15.82:6800/1763049308 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f8d10105030 con 0x7f8d00039110 2024-05-20T16:04:32.548 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:31.623+0000 7f8d16a8a700 1 -- 172.21.15.82:0/1904074244 >> [v2:172.21.15.82:6800/1763049308,v1:172.21.15.82:6801/1763049308] conn(0x7f8d00039110 msgr2=0x7f8d0003b620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:04:32.549 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:31.623+0000 7f8d16a8a700 1 --2- 172.21.15.82:0/1904074244 >> [v2:172.21.15.82:6800/1763049308,v1:172.21.15.82:6801/1763049308] conn(0x7f8d00039110 0x7f8d0003b620 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f8cfc006fd0 tx=0x7f8cfc006e40 comp rx=0 tx=0).stop 2024-05-20T16:04:32.549 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:31.623+0000 7f8d16a8a700 1 -- 172.21.15.82:0/1904074244 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8d10109570 msgr2=0x7f8d1019cd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:04:32.549 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:31.623+0000 7f8d16a8a700 1 --2- 172.21.15.82:0/1904074244 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8d10109570 0x7f8d1019cd60 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f8d04009e50 tx=0x7f8d0400bd30 comp rx=0 tx=0).stop 2024-05-20T16:04:32.549 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:31.623+0000 7f8d16a8a700 1 -- 172.21.15.82:0/1904074244 shutdown_connections 2024-05-20T16:04:32.549 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:31.623+0000 7f8d16a8a700 1 --2- 172.21.15.82:0/1904074244 >> [v2:172.21.15.82:6800/1763049308,v1:172.21.15.82:6801/1763049308] conn(0x7f8d00039110 0x7f8d0003b620 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:04:32.549 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:31.623+0000 7f8d16a8a700 1 --2- 172.21.15.82:0/1904074244 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8d10109570 0x7f8d1019cd60 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:04:32.549 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:31.623+0000 7f8d16a8a700 1 -- 172.21.15.82:0/1904074244 >> 172.21.15.82:0/1904074244 conn(0x7f8d10103070 msgr2=0x7f8d10104920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:04:32.549 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:31.623+0000 7f8d16a8a700 1 -- 172.21.15.82:0/1904074244 shutdown_connections 2024-05-20T16:04:32.549 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:31.623+0000 7f8d16a8a700 1 -- 172.21.15.82:0/1904074244 wait complete. 2024-05-20T16:04:32.549 INFO:teuthology.orchestra.run.smithi082.stdout:Deploying unmanaged mgr service... 2024-05-20T16:04:32.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:04:32 smithi082 ceph-mon[32238]: Added host smithi082 2024-05-20T16:04:32.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:04:32 smithi082 ceph-mon[32238]: from='mgr.14118 172.21.15.82:0/3662546898' entity='mgr.x' 2024-05-20T16:04:33.560 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:04:33 smithi082 ceph-mon[32238]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-20T16:04:33.560 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:04:33 smithi082 ceph-mon[32238]: Saving service mon spec with placement count:5 2024-05-20T16:04:33.560 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:04:33 smithi082 ceph-mon[32238]: from='mgr.14118 172.21.15.82:0/3662546898' entity='mgr.x' 2024-05-20T16:04:34.028 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-05-20T16:04:34.029 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:33.078+0000 7feebecaf700 1 Processor -- start 2024-05-20T16:04:34.029 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:33.078+0000 7feebecaf700 1 -- start start 2024-05-20T16:04:34.029 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:33.078+0000 7feebecaf700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feeb807c510 0x7feeb810b690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:04:34.029 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:33.078+0000 7feebecaf700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7feeb810bc60 con 0x7feeb807c510 2024-05-20T16:04:34.029 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:33.078+0000 7feebca4b700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feeb807c510 0x7feeb810b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:04:34.029 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:33.078+0000 7feebca4b700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feeb807c510 0x7feeb810b690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:57302/0 (socket says 172.21.15.82:57302) 2024-05-20T16:04:34.029 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:33.078+0000 7feebca4b700 1 -- 172.21.15.82:0/2775464392 learned_addr learned my addr 172.21.15.82:0/2775464392 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:04:34.029 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:33.078+0000 7feebca4b700 1 -- 172.21.15.82:0/2775464392 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feeb810bde0 con 0x7feeb807c510 2024-05-20T16:04:34.029 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:33.079+0000 7feebca4b700 1 --2- 172.21.15.82:0/2775464392 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feeb807c510 0x7feeb810b690 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7feeac0098a0 tx=0x7feeac009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce16abfd2c939204 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:04:34.029 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:33.079+0000 7feeb77fe700 1 -- 172.21.15.82:0/2775464392 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7feeac004160 con 0x7feeb807c510 2024-05-20T16:04:34.029 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:33.079+0000 7feeb77fe700 1 -- 172.21.15.82:0/2775464392 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7feeac004300 con 0x7feeb807c510 2024-05-20T16:04:34.030 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:33.079+0000 7feeb77fe700 1 -- 172.21.15.82:0/2775464392 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7feeac011b50 con 0x7feeb807c510 2024-05-20T16:04:34.030 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:33.079+0000 7feebecaf700 1 -- 172.21.15.82:0/2775464392 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feeb807c510 msgr2=0x7feeb810b690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:04:34.030 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:33.079+0000 7feebecaf700 1 --2- 172.21.15.82:0/2775464392 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feeb807c510 0x7feeb810b690 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7feeac0098a0 tx=0x7feeac009bb0 comp rx=0 tx=0).stop 2024-05-20T16:04:34.030 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:33.079+0000 7feebecaf700 1 -- 172.21.15.82:0/2775464392 shutdown_connections 2024-05-20T16:04:34.030 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:33.079+0000 7feebecaf700 1 --2- 172.21.15.82:0/2775464392 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feeb807c510 0x7feeb810b690 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:04:34.030 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:33.079+0000 7feebecaf700 1 -- 172.21.15.82:0/2775464392 >> 172.21.15.82:0/2775464392 conn(0x7feeb8103060 msgr2=0x7feeb8105510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:04:34.030 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:33.080+0000 7feebecaf700 1 -- 172.21.15.82:0/2775464392 shutdown_connections 2024-05-20T16:04:34.030 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:33.080+0000 7feebecaf700 1 -- 172.21.15.82:0/2775464392 wait complete. 2024-05-20T16:04:34.030 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:33.080+0000 7feebecaf700 1 Processor -- start 2024-05-20T16:04:34.030 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:33.080+0000 7feebecaf700 1 -- start start 2024-05-20T16:04:34.030 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:33.080+0000 7feebecaf700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feeb807c510 0x7feeb819ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:04:34.030 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:33.080+0000 7feebecaf700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7feeb810cec0 con 0x7feeb807c510 2024-05-20T16:04:34.030 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:33.080+0000 7feebca4b700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feeb807c510 0x7feeb819ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:04:34.031 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:33.080+0000 7feebca4b700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feeb807c510 0x7feeb819ce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:57304/0 (socket says 172.21.15.82:57304) 2024-05-20T16:04:34.031 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:33.080+0000 7feebca4b700 1 -- 172.21.15.82:0/146709311 learned_addr learned my addr 172.21.15.82:0/146709311 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:04:34.031 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:33.081+0000 7feebca4b700 1 -- 172.21.15.82:0/146709311 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feeb819d340 con 0x7feeb807c510 2024-05-20T16:04:34.031 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:33.081+0000 7feebca4b700 1 --2- 172.21.15.82:0/146709311 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feeb807c510 0x7feeb819ce00 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7feeac009fb0 tx=0x7feeac011bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:04:34.031 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:33.081+0000 7feeb5ffb700 1 -- 172.21.15.82:0/146709311 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7feeac0038b0 con 0x7feeb807c510 2024-05-20T16:04:34.031 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:33.081+0000 7feeb5ffb700 1 -- 172.21.15.82:0/146709311 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7feeac011e50 con 0x7feeb807c510 2024-05-20T16:04:34.031 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:33.081+0000 7feeb5ffb700 1 -- 172.21.15.82:0/146709311 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7feeac019420 con 0x7feeb807c510 2024-05-20T16:04:34.031 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:33.081+0000 7feebecaf700 1 -- 172.21.15.82:0/146709311 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feeb819d5d0 con 0x7feeb807c510 2024-05-20T16:04:34.031 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:33.081+0000 7feebecaf700 1 -- 172.21.15.82:0/146709311 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feeb81a01e0 con 0x7feeb807c510 2024-05-20T16:04:34.031 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:33.082+0000 7feeb5ffb700 1 -- 172.21.15.82:0/146709311 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7feeac003a50 con 0x7feeb807c510 2024-05-20T16:04:34.031 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:33.082+0000 7feebecaf700 1 -- 172.21.15.82:0/146709311 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feeb8063ca0 con 0x7feeb807c510 2024-05-20T16:04:34.031 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:33.082+0000 7feeb5ffb700 1 --2- 172.21.15.82:0/146709311 >> [v2:172.21.15.82:6800/1763049308,v1:172.21.15.82:6801/1763049308] conn(0x7feea80390c0 0x7feea803b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:04:34.031 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:33.083+0000 7feeb5ffb700 1 -- 172.21.15.82:0/146709311 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7feeac04d3b0 con 0x7feeb807c510 2024-05-20T16:04:34.031 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:33.083+0000 7feeb7fff700 1 --2- 172.21.15.82:0/146709311 >> [v2:172.21.15.82:6800/1763049308,v1:172.21.15.82:6801/1763049308] conn(0x7feea80390c0 0x7feea803b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:04:34.031 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:33.083+0000 7feeb7fff700 1 --2- 172.21.15.82:0/146709311 >> [v2:172.21.15.82:6800/1763049308,v1:172.21.15.82:6801/1763049308] conn(0x7feea80390c0 0x7feea803b5d0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7feea4006fd0 tx=0x7feea4006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:04:34.032 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:33.088+0000 7feeb5ffb700 1 -- 172.21.15.82:0/146709311 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7feeac04cd00 con 0x7feeb807c510 2024-05-20T16:04:34.032 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:33.300+0000 7feebecaf700 1 -- 172.21.15.82:0/146709311 --> [v2:172.21.15.82:6800/1763049308,v1:172.21.15.82:6801/1763049308] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7feeb8103fb0 con 0x7feea80390c0 2024-05-20T16:04:34.032 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:33.305+0000 7feeb5ffb700 1 -- 172.21.15.82:0/146709311 <== mgr.14118 v2:172.21.15.82:6800/1763049308 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7feeb8103fb0 con 0x7feea80390c0 2024-05-20T16:04:34.032 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:33.307+0000 7feebecaf700 1 -- 172.21.15.82:0/146709311 >> [v2:172.21.15.82:6800/1763049308,v1:172.21.15.82:6801/1763049308] conn(0x7feea80390c0 msgr2=0x7feea803b5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:04:34.032 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:33.307+0000 7feebecaf700 1 --2- 172.21.15.82:0/146709311 >> [v2:172.21.15.82:6800/1763049308,v1:172.21.15.82:6801/1763049308] conn(0x7feea80390c0 0x7feea803b5d0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7feea4006fd0 tx=0x7feea4006e40 comp rx=0 tx=0).stop 2024-05-20T16:04:34.032 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:33.307+0000 7feebecaf700 1 -- 172.21.15.82:0/146709311 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feeb807c510 msgr2=0x7feeb819ce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:04:34.032 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:33.307+0000 7feebecaf700 1 --2- 172.21.15.82:0/146709311 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feeb807c510 0x7feeb819ce00 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7feeac009fb0 tx=0x7feeac011bf0 comp rx=0 tx=0).stop 2024-05-20T16:04:34.032 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:33.307+0000 7feebecaf700 1 -- 172.21.15.82:0/146709311 shutdown_connections 2024-05-20T16:04:34.032 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:33.307+0000 7feebecaf700 1 --2- 172.21.15.82:0/146709311 >> [v2:172.21.15.82:6800/1763049308,v1:172.21.15.82:6801/1763049308] conn(0x7feea80390c0 0x7feea803b5d0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:04:34.032 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:33.307+0000 7feebecaf700 1 --2- 172.21.15.82:0/146709311 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feeb807c510 0x7feeb819ce00 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:04:34.032 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:33.307+0000 7feebecaf700 1 -- 172.21.15.82:0/146709311 >> 172.21.15.82:0/146709311 conn(0x7feeb8103060 msgr2=0x7feeb81038a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:04:34.032 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:33.307+0000 7feebecaf700 1 -- 172.21.15.82:0/146709311 shutdown_connections 2024-05-20T16:04:34.033 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:33.307+0000 7feebecaf700 1 -- 172.21.15.82:0/146709311 wait complete. 2024-05-20T16:04:34.450 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:04:34 smithi082 ceph-mon[32238]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-20T16:04:34.450 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:04:34 smithi082 ceph-mon[32238]: Saving service mgr spec with placement count:2 2024-05-20T16:04:34.450 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:04:34 smithi082 ceph-mon[32238]: from='mgr.14118 172.21.15.82:0/3662546898' entity='mgr.x' 2024-05-20T16:04:34.450 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:04:34 smithi082 ceph-mon[32238]: from='mgr.14118 172.21.15.82:0/3662546898' entity='mgr.x' 2024-05-20T16:04:35.455 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:34.490+0000 7fb0daad1700 1 Processor -- start 2024-05-20T16:04:35.455 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:34.490+0000 7fb0daad1700 1 -- start start 2024-05-20T16:04:35.455 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:34.491+0000 7fb0daad1700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb0d4107990 0x7fb0d4107da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:04:35.455 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:34.491+0000 7fb0daad1700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fb0d4108390 con 0x7fb0d4107990 2024-05-20T16:04:35.456 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:34.491+0000 7fb0d886d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb0d4107990 0x7fb0d4107da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:04:35.456 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:34.491+0000 7fb0d886d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb0d4107990 0x7fb0d4107da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:57306/0 (socket says 172.21.15.82:57306) 2024-05-20T16:04:35.456 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:34.491+0000 7fb0d886d700 1 -- 172.21.15.82:0/2956434201 learned_addr learned my addr 172.21.15.82:0/2956434201 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:04:35.456 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:34.491+0000 7fb0d886d700 1 -- 172.21.15.82:0/2956434201 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0d4108ac0 con 0x7fb0d4107990 2024-05-20T16:04:35.456 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:34.491+0000 7fb0d886d700 1 --2- 172.21.15.82:0/2956434201 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb0d4107990 0x7fb0d4107da0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fb0c80098a0 tx=0x7fb0c8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=427f6ba76cecde14 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:04:35.456 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:34.491+0000 7fb0d37fe700 1 -- 172.21.15.82:0/2956434201 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb0c8004160 con 0x7fb0d4107990 2024-05-20T16:04:35.456 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:34.491+0000 7fb0d37fe700 1 -- 172.21.15.82:0/2956434201 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb0c8004300 con 0x7fb0d4107990 2024-05-20T16:04:35.456 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:34.491+0000 7fb0d37fe700 1 -- 172.21.15.82:0/2956434201 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb0c8011b20 con 0x7fb0d4107990 2024-05-20T16:04:35.457 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:34.492+0000 7fb0daad1700 1 -- 172.21.15.82:0/2956434201 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb0d4107990 msgr2=0x7fb0d4107da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:04:35.457 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:34.492+0000 7fb0daad1700 1 --2- 172.21.15.82:0/2956434201 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb0d4107990 0x7fb0d4107da0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fb0c80098a0 tx=0x7fb0c8009bb0 comp rx=0 tx=0).stop 2024-05-20T16:04:35.457 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:34.492+0000 7fb0daad1700 1 -- 172.21.15.82:0/2956434201 shutdown_connections 2024-05-20T16:04:35.457 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:34.492+0000 7fb0daad1700 1 --2- 172.21.15.82:0/2956434201 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb0d4107990 0x7fb0d4107da0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:04:35.457 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:34.492+0000 7fb0daad1700 1 -- 172.21.15.82:0/2956434201 >> 172.21.15.82:0/2956434201 conn(0x7fb0d4103070 msgr2=0x7fb0d41054c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:04:35.457 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:34.492+0000 7fb0daad1700 1 -- 172.21.15.82:0/2956434201 shutdown_connections 2024-05-20T16:04:35.457 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:34.492+0000 7fb0daad1700 1 -- 172.21.15.82:0/2956434201 wait complete. 2024-05-20T16:04:35.457 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:34.493+0000 7fb0daad1700 1 Processor -- start 2024-05-20T16:04:35.458 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:34.493+0000 7fb0daad1700 1 -- start start 2024-05-20T16:04:35.458 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:34.493+0000 7fb0daad1700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb0d4107990 0x7fb0d4106130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:04:35.458 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:34.493+0000 7fb0daad1700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fb0d410be70 con 0x7fb0d4107990 2024-05-20T16:04:35.458 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:34.493+0000 7fb0d886d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb0d4107990 0x7fb0d4106130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:04:35.458 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:34.493+0000 7fb0d886d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb0d4107990 0x7fb0d4106130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:57322/0 (socket says 172.21.15.82:57322) 2024-05-20T16:04:35.458 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:34.493+0000 7fb0d886d700 1 -- 172.21.15.82:0/1515658260 learned_addr learned my addr 172.21.15.82:0/1515658260 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:04:35.458 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:34.494+0000 7fb0d886d700 1 -- 172.21.15.82:0/1515658260 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0d4106670 con 0x7fb0d4107990 2024-05-20T16:04:35.458 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:34.494+0000 7fb0d886d700 1 --2- 172.21.15.82:0/1515658260 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb0d4107990 0x7fb0d4106130 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fb0c8009fb0 tx=0x7fb0c8011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:04:35.458 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:34.494+0000 7fb0d1ffb700 1 -- 172.21.15.82:0/1515658260 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb0c8003930 con 0x7fb0d4107990 2024-05-20T16:04:35.459 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:34.494+0000 7fb0d1ffb700 1 -- 172.21.15.82:0/1515658260 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb0c8003ad0 con 0x7fb0d4107990 2024-05-20T16:04:35.459 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:34.494+0000 7fb0daad1700 1 -- 172.21.15.82:0/1515658260 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0d41048c0 con 0x7fb0d4107990 2024-05-20T16:04:35.459 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:34.494+0000 7fb0d1ffb700 1 -- 172.21.15.82:0/1515658260 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb0c8019670 con 0x7fb0d4107990 2024-05-20T16:04:35.459 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:34.494+0000 7fb0daad1700 1 -- 172.21.15.82:0/1515658260 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0d4104d20 con 0x7fb0d4107990 2024-05-20T16:04:35.459 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:34.495+0000 7fb0daad1700 1 -- 172.21.15.82:0/1515658260 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0d4063ca0 con 0x7fb0d4107990 2024-05-20T16:04:35.459 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:34.495+0000 7fb0d1ffb700 1 -- 172.21.15.82:0/1515658260 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fb0c8019920 con 0x7fb0d4107990 2024-05-20T16:04:35.459 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:34.496+0000 7fb0d1ffb700 1 --2- 172.21.15.82:0/1515658260 >> [v2:172.21.15.82:6800/1763049308,v1:172.21.15.82:6801/1763049308] conn(0x7fb0c4039110 0x7fb0c403b620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:04:35.459 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:34.496+0000 7fb0d1ffb700 1 -- 172.21.15.82:0/1515658260 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fb0c804d630 con 0x7fb0d4107990 2024-05-20T16:04:35.459 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:34.496+0000 7fb0d3fff700 1 --2- 172.21.15.82:0/1515658260 >> [v2:172.21.15.82:6800/1763049308,v1:172.21.15.82:6801/1763049308] conn(0x7fb0c4039110 0x7fb0c403b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:04:35.460 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:34.496+0000 7fb0d3fff700 1 --2- 172.21.15.82:0/1515658260 >> [v2:172.21.15.82:6800/1763049308,v1:172.21.15.82:6801/1763049308] conn(0x7fb0c4039110 0x7fb0c403b620 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fb0c0006fd0 tx=0x7fb0c0006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:04:35.460 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:34.501+0000 7fb0d1ffb700 1 -- 172.21.15.82:0/1515658260 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fb0c8011e20 con 0x7fb0d4107990 2024-05-20T16:04:35.460 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:34.721+0000 7fb0daad1700 1 -- 172.21.15.82:0/1515658260 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7fb0d4195e70 con 0x7fb0d4107990 2024-05-20T16:04:35.460 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:34.727+0000 7fb0d1ffb700 1 -- 172.21.15.82:0/1515658260 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) ==== 142+0+0 (secure 0 0 0) 0x7fb0c8019d10 con 0x7fb0d4107990 2024-05-20T16:04:35.460 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:34.729+0000 7fb0daad1700 1 -- 172.21.15.82:0/1515658260 >> [v2:172.21.15.82:6800/1763049308,v1:172.21.15.82:6801/1763049308] conn(0x7fb0c4039110 msgr2=0x7fb0c403b620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:04:35.460 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:34.729+0000 7fb0daad1700 1 --2- 172.21.15.82:0/1515658260 >> [v2:172.21.15.82:6800/1763049308,v1:172.21.15.82:6801/1763049308] conn(0x7fb0c4039110 0x7fb0c403b620 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fb0c0006fd0 tx=0x7fb0c0006e40 comp rx=0 tx=0).stop 2024-05-20T16:04:35.460 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:34.729+0000 7fb0daad1700 1 -- 172.21.15.82:0/1515658260 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb0d4107990 msgr2=0x7fb0d4106130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:04:35.460 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:34.729+0000 7fb0daad1700 1 --2- 172.21.15.82:0/1515658260 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb0d4107990 0x7fb0d4106130 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fb0c8009fb0 tx=0x7fb0c8011c10 comp rx=0 tx=0).stop 2024-05-20T16:04:35.461 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:34.729+0000 7fb0daad1700 1 -- 172.21.15.82:0/1515658260 shutdown_connections 2024-05-20T16:04:35.461 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:34.729+0000 7fb0daad1700 1 --2- 172.21.15.82:0/1515658260 >> [v2:172.21.15.82:6800/1763049308,v1:172.21.15.82:6801/1763049308] conn(0x7fb0c4039110 0x7fb0c403b620 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:04:35.461 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:34.729+0000 7fb0daad1700 1 --2- 172.21.15.82:0/1515658260 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb0d4107990 0x7fb0d4106130 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:04:35.461 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:34.729+0000 7fb0daad1700 1 -- 172.21.15.82:0/1515658260 >> 172.21.15.82:0/1515658260 conn(0x7fb0d4103070 msgr2=0x7fb0d4193380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:04:35.461 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:34.729+0000 7fb0daad1700 1 -- 172.21.15.82:0/1515658260 shutdown_connections 2024-05-20T16:04:35.461 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:34.729+0000 7fb0daad1700 1 -- 172.21.15.82:0/1515658260 wait complete. 2024-05-20T16:04:35.984 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:04:35 smithi082 ceph-mon[32238]: from='client.? 172.21.15.82:0/1515658260' entity='client.admin' 2024-05-20T16:04:36.853 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:36.103+0000 7f7b4f628700 1 Processor -- start 2024-05-20T16:04:36.853 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:36.104+0000 7f7b4f628700 1 -- start start 2024-05-20T16:04:36.853 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:36.104+0000 7f7b4f628700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7b481079b0 0x7f7b48107dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:04:36.853 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:36.104+0000 7f7b4f628700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f7b481083b0 con 0x7f7b481079b0 2024-05-20T16:04:36.854 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:36.104+0000 7f7b4d3c4700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7b481079b0 0x7f7b48107dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:04:36.854 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:36.104+0000 7f7b4d3c4700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7b481079b0 0x7f7b48107dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:57330/0 (socket says 172.21.15.82:57330) 2024-05-20T16:04:36.854 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:36.104+0000 7f7b4d3c4700 1 -- 172.21.15.82:0/2177960070 learned_addr learned my addr 172.21.15.82:0/2177960070 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:04:36.854 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:36.104+0000 7f7b4d3c4700 1 -- 172.21.15.82:0/2177960070 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b48108ae0 con 0x7f7b481079b0 2024-05-20T16:04:36.854 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:36.105+0000 7f7b4d3c4700 1 --2- 172.21.15.82:0/2177960070 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7b481079b0 0x7f7b48107dc0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f7b440098a0 tx=0x7f7b44009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b184c1903f890bc server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:04:36.854 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:36.105+0000 7f7b3bfff700 1 -- 172.21.15.82:0/2177960070 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7b44004160 con 0x7f7b481079b0 2024-05-20T16:04:36.854 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:36.105+0000 7f7b3bfff700 1 -- 172.21.15.82:0/2177960070 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b44004300 con 0x7f7b481079b0 2024-05-20T16:04:36.854 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:36.105+0000 7f7b3bfff700 1 -- 172.21.15.82:0/2177960070 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7b44011b20 con 0x7f7b481079b0 2024-05-20T16:04:36.854 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:36.105+0000 7f7b4f628700 1 -- 172.21.15.82:0/2177960070 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7b481079b0 msgr2=0x7f7b48107dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:04:36.854 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:36.106+0000 7f7b4f628700 1 --2- 172.21.15.82:0/2177960070 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7b481079b0 0x7f7b48107dc0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f7b440098a0 tx=0x7f7b44009bb0 comp rx=0 tx=0).stop 2024-05-20T16:04:36.854 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:36.106+0000 7f7b4f628700 1 -- 172.21.15.82:0/2177960070 shutdown_connections 2024-05-20T16:04:36.855 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:36.106+0000 7f7b4f628700 1 --2- 172.21.15.82:0/2177960070 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7b481079b0 0x7f7b48107dc0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:04:36.855 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:36.106+0000 7f7b4f628700 1 -- 172.21.15.82:0/2177960070 >> 172.21.15.82:0/2177960070 conn(0x7f7b48103050 msgr2=0x7f7b481054a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:04:36.855 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:36.106+0000 7f7b4f628700 1 -- 172.21.15.82:0/2177960070 shutdown_connections 2024-05-20T16:04:36.855 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:36.106+0000 7f7b4f628700 1 -- 172.21.15.82:0/2177960070 wait complete. 2024-05-20T16:04:36.855 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:36.106+0000 7f7b4f628700 1 Processor -- start 2024-05-20T16:04:36.855 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:36.106+0000 7f7b4f628700 1 -- start start 2024-05-20T16:04:36.855 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:36.107+0000 7f7b4f628700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7b481079b0 0x7f7b4819bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:04:36.855 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:36.107+0000 7f7b4f628700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f7b4810be90 con 0x7f7b481079b0 2024-05-20T16:04:36.855 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:36.107+0000 7f7b4d3c4700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7b481079b0 0x7f7b4819bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:04:36.855 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:36.107+0000 7f7b4d3c4700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7b481079b0 0x7f7b4819bd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:57332/0 (socket says 172.21.15.82:57332) 2024-05-20T16:04:36.855 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:36.107+0000 7f7b4d3c4700 1 -- 172.21.15.82:0/2332569445 learned_addr learned my addr 172.21.15.82:0/2332569445 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:04:36.855 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:36.107+0000 7f7b4d3c4700 1 -- 172.21.15.82:0/2332569445 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b4819c250 con 0x7f7b481079b0 2024-05-20T16:04:36.855 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:36.107+0000 7f7b4d3c4700 1 --2- 172.21.15.82:0/2332569445 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7b481079b0 0x7f7b4819bd10 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f7b44011ad0 tx=0x7f7b44011bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:04:36.855 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:36.107+0000 7f7b3a7fc700 1 -- 172.21.15.82:0/2332569445 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7b44003a40 con 0x7f7b481079b0 2024-05-20T16:04:36.856 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:36.107+0000 7f7b3a7fc700 1 -- 172.21.15.82:0/2332569445 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b44003be0 con 0x7f7b481079b0 2024-05-20T16:04:36.856 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:36.107+0000 7f7b4f628700 1 -- 172.21.15.82:0/2332569445 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b4819c4e0 con 0x7f7b481079b0 2024-05-20T16:04:36.856 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:36.107+0000 7f7b3a7fc700 1 -- 172.21.15.82:0/2332569445 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7b44024740 con 0x7f7b481079b0 2024-05-20T16:04:36.856 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:36.107+0000 7f7b4f628700 1 -- 172.21.15.82:0/2332569445 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b4819f0e0 con 0x7f7b481079b0 2024-05-20T16:04:36.856 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:36.108+0000 7f7b3a7fc700 1 -- 172.21.15.82:0/2332569445 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f7b440248e0 con 0x7f7b481079b0 2024-05-20T16:04:36.856 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:36.108+0000 7f7b3a7fc700 1 --2- 172.21.15.82:0/2332569445 >> [v2:172.21.15.82:6800/1763049308,v1:172.21.15.82:6801/1763049308] conn(0x7f7b34039160 0x7f7b3403b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:04:36.856 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:36.108+0000 7f7b3a7fc700 1 -- 172.21.15.82:0/2332569445 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f7b4404e2b0 con 0x7f7b481079b0 2024-05-20T16:04:36.856 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:36.108+0000 7f7b4f628700 1 -- 172.21.15.82:0/2332569445 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b48063ca0 con 0x7f7b481079b0 2024-05-20T16:04:36.856 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:36.108+0000 7f7b4cbc3700 1 --2- 172.21.15.82:0/2332569445 >> [v2:172.21.15.82:6800/1763049308,v1:172.21.15.82:6801/1763049308] conn(0x7f7b34039160 0x7f7b3403b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:04:36.856 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:36.109+0000 7f7b4cbc3700 1 --2- 172.21.15.82:0/2332569445 >> [v2:172.21.15.82:6800/1763049308,v1:172.21.15.82:6801/1763049308] conn(0x7f7b34039160 0x7f7b3403b670 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f7b3c006fd0 tx=0x7f7b3c006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:04:36.856 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:36.114+0000 7f7b3a7fc700 1 -- 172.21.15.82:0/2332569445 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f7b440172b0 con 0x7f7b481079b0 2024-05-20T16:04:36.856 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:36.328+0000 7f7b4f628700 1 -- 172.21.15.82:0/2332569445 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7f7b4819f2b0 con 0x7f7b481079b0 2024-05-20T16:04:36.856 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:36.333+0000 7f7b3a7fc700 1 -- 172.21.15.82:0/2332569445 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7f7b440094f0 con 0x7f7b481079b0 2024-05-20T16:04:36.857 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:36.335+0000 7f7b4f628700 1 -- 172.21.15.82:0/2332569445 >> [v2:172.21.15.82:6800/1763049308,v1:172.21.15.82:6801/1763049308] conn(0x7f7b34039160 msgr2=0x7f7b3403b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:04:36.857 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:36.335+0000 7f7b4f628700 1 --2- 172.21.15.82:0/2332569445 >> [v2:172.21.15.82:6800/1763049308,v1:172.21.15.82:6801/1763049308] conn(0x7f7b34039160 0x7f7b3403b670 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f7b3c006fd0 tx=0x7f7b3c006e40 comp rx=0 tx=0).stop 2024-05-20T16:04:36.857 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:36.335+0000 7f7b4f628700 1 -- 172.21.15.82:0/2332569445 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7b481079b0 msgr2=0x7f7b4819bd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:04:36.857 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:36.335+0000 7f7b4f628700 1 --2- 172.21.15.82:0/2332569445 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7b481079b0 0x7f7b4819bd10 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f7b44011ad0 tx=0x7f7b44011bb0 comp rx=0 tx=0).stop 2024-05-20T16:04:36.857 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:36.335+0000 7f7b4f628700 1 -- 172.21.15.82:0/2332569445 shutdown_connections 2024-05-20T16:04:36.857 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:36.335+0000 7f7b4f628700 1 --2- 172.21.15.82:0/2332569445 >> [v2:172.21.15.82:6800/1763049308,v1:172.21.15.82:6801/1763049308] conn(0x7f7b34039160 0x7f7b3403b670 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:04:36.857 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:36.335+0000 7f7b4f628700 1 --2- 172.21.15.82:0/2332569445 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7b481079b0 0x7f7b4819bd10 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:04:36.857 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:36.335+0000 7f7b4f628700 1 -- 172.21.15.82:0/2332569445 >> 172.21.15.82:0/2332569445 conn(0x7f7b48103050 msgr2=0x7f7b48103830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:04:36.857 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:36.336+0000 7f7b4f628700 1 -- 172.21.15.82:0/2332569445 shutdown_connections 2024-05-20T16:04:36.857 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:36.336+0000 7f7b4f628700 1 -- 172.21.15.82:0/2332569445 wait complete. 2024-05-20T16:04:36.857 INFO:teuthology.orchestra.run.smithi082.stdout:Enabling the dashboard module... 2024-05-20T16:04:37.244 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:04:37 smithi082 ceph-mon[32238]: from='mgr.14118 172.21.15.82:0/3662546898' entity='mgr.x' 2024-05-20T16:04:37.244 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:04:37 smithi082 ceph-mon[32238]: from='client.? 172.21.15.82:0/2332569445' entity='client.admin' 2024-05-20T16:04:38.452 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:04:38 smithi082 ceph-mon[32238]: from='client.? 172.21.15.82:0/1867483540' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-05-20T16:04:38.452 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:04:38 smithi082 ceph-mon[32238]: from='mgr.14118 172.21.15.82:0/3662546898' entity='mgr.x' 2024-05-20T16:04:38.452 INFO:journalctl@ceph.mgr.x.smithi082.stdout:May 20 16:04:38 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mgr-x[32467]: ignoring --setuser ceph since I am not root 2024-05-20T16:04:38.452 INFO:journalctl@ceph.mgr.x.smithi082.stdout:May 20 16:04:38 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mgr-x[32467]: ignoring --setgroup ceph since I am not root 2024-05-20T16:04:38.452 INFO:journalctl@ceph.mgr.x.smithi082.stdout:May 20 16:04:38 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mgr-x[32467]: 2024-05-20T16:04:38.367+0000 7f1acb2ee200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-20T16:04:38.769 INFO:journalctl@ceph.mgr.x.smithi082.stdout:May 20 16:04:38 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mgr-x[32467]: 2024-05-20T16:04:38.559+0000 7f1acb2ee200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-20T16:04:38.770 INFO:journalctl@ceph.mgr.x.smithi082.stdout:May 20 16:04:38 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mgr-x[32467]: 2024-05-20T16:04:38.662+0000 7f1acb2ee200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-20T16:04:38.777 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:37.243+0000 7f1862f24700 1 Processor -- start 2024-05-20T16:04:38.777 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:37.243+0000 7f1862f24700 1 -- start start 2024-05-20T16:04:38.777 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:37.243+0000 7f1862f24700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f185c108f10 0x7f185c109320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:04:38.777 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:37.243+0000 7f1862f24700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f185c109910 con 0x7f185c108f10 2024-05-20T16:04:38.777 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:37.244+0000 7f1860cc0700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f185c108f10 0x7f185c109320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:04:38.777 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:37.244+0000 7f1860cc0700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f185c108f10 0x7f185c109320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:57344/0 (socket says 172.21.15.82:57344) 2024-05-20T16:04:38.777 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:37.244+0000 7f1860cc0700 1 -- 172.21.15.82:0/4028443437 learned_addr learned my addr 172.21.15.82:0/4028443437 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:04:38.778 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:37.244+0000 7f1860cc0700 1 -- 172.21.15.82:0/4028443437 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f185c107c30 con 0x7f185c108f10 2024-05-20T16:04:38.778 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:37.244+0000 7f1860cc0700 1 --2- 172.21.15.82:0/4028443437 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f185c108f10 0x7f185c109320 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f18580098a0 tx=0x7f1858009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8fa534b70fe70d8 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:04:38.778 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:37.244+0000 7f18537fe700 1 -- 172.21.15.82:0/4028443437 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1858004160 con 0x7f185c108f10 2024-05-20T16:04:38.778 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:37.244+0000 7f18537fe700 1 -- 172.21.15.82:0/4028443437 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1858004300 con 0x7f185c108f10 2024-05-20T16:04:38.778 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:37.244+0000 7f18537fe700 1 -- 172.21.15.82:0/4028443437 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1858011b20 con 0x7f185c108f10 2024-05-20T16:04:38.778 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:37.245+0000 7f1862f24700 1 -- 172.21.15.82:0/4028443437 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f185c108f10 msgr2=0x7f185c109320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:04:38.778 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:37.245+0000 7f1862f24700 1 --2- 172.21.15.82:0/4028443437 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f185c108f10 0x7f185c109320 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f18580098a0 tx=0x7f1858009bb0 comp rx=0 tx=0).stop 2024-05-20T16:04:38.778 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:37.245+0000 7f1862f24700 1 -- 172.21.15.82:0/4028443437 shutdown_connections 2024-05-20T16:04:38.778 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:37.245+0000 7f1862f24700 1 --2- 172.21.15.82:0/4028443437 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f185c108f10 0x7f185c109320 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:04:38.778 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:37.245+0000 7f1862f24700 1 -- 172.21.15.82:0/4028443437 >> 172.21.15.82:0/4028443437 conn(0x7f185c1028b0 msgr2=0x7f185c104d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:04:38.778 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:37.245+0000 7f1862f24700 1 -- 172.21.15.82:0/4028443437 shutdown_connections 2024-05-20T16:04:38.778 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:37.245+0000 7f1862f24700 1 -- 172.21.15.82:0/4028443437 wait complete. 2024-05-20T16:04:38.778 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:37.246+0000 7f1862f24700 1 Processor -- start 2024-05-20T16:04:38.779 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:37.246+0000 7f1862f24700 1 -- start start 2024-05-20T16:04:38.779 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:37.246+0000 7f1862f24700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f185c108f10 0x7f185c19ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:04:38.779 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:37.246+0000 7f1862f24700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f185c106b20 con 0x7f185c108f10 2024-05-20T16:04:38.779 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:37.246+0000 7f1860cc0700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f185c108f10 0x7f185c19ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:04:38.779 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:37.246+0000 7f1860cc0700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f185c108f10 0x7f185c19ab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:57354/0 (socket says 172.21.15.82:57354) 2024-05-20T16:04:38.779 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:37.246+0000 7f1860cc0700 1 -- 172.21.15.82:0/1867483540 learned_addr learned my addr 172.21.15.82:0/1867483540 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:04:38.779 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:37.246+0000 7f1860cc0700 1 -- 172.21.15.82:0/1867483540 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f185c19b0b0 con 0x7f185c108f10 2024-05-20T16:04:38.779 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:37.247+0000 7f1860cc0700 1 --2- 172.21.15.82:0/1867483540 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f185c108f10 0x7f185c19ab70 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f1858011ad0 tx=0x7f1858011bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:04:38.779 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:37.247+0000 7f1851ffb700 1 -- 172.21.15.82:0/1867483540 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1858003a40 con 0x7f185c108f10 2024-05-20T16:04:38.779 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:37.247+0000 7f1851ffb700 1 -- 172.21.15.82:0/1867483540 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1858003be0 con 0x7f185c108f10 2024-05-20T16:04:38.779 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:37.247+0000 7f1862f24700 1 -- 172.21.15.82:0/1867483540 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f185c19b340 con 0x7f185c108f10 2024-05-20T16:04:38.779 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:37.247+0000 7f1851ffb700 1 -- 172.21.15.82:0/1867483540 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1858024740 con 0x7f185c108f10 2024-05-20T16:04:38.779 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:37.247+0000 7f1862f24700 1 -- 172.21.15.82:0/1867483540 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f185c19df50 con 0x7f185c108f10 2024-05-20T16:04:38.780 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:37.248+0000 7f1851ffb700 1 -- 172.21.15.82:0/1867483540 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f18580248e0 con 0x7f185c108f10 2024-05-20T16:04:38.780 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:37.248+0000 7f1862f24700 1 -- 172.21.15.82:0/1867483540 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f185c19b7a0 con 0x7f185c108f10 2024-05-20T16:04:38.780 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:37.248+0000 7f1851ffb700 1 --2- 172.21.15.82:0/1867483540 >> [v2:172.21.15.82:6800/1763049308,v1:172.21.15.82:6801/1763049308] conn(0x7f184403b3c0 0x7f184403d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:04:38.780 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:37.248+0000 7f1851ffb700 1 -- 172.21.15.82:0/1867483540 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f185804d280 con 0x7f185c108f10 2024-05-20T16:04:38.780 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:37.248+0000 7f1853fff700 1 --2- 172.21.15.82:0/1867483540 >> [v2:172.21.15.82:6800/1763049308,v1:172.21.15.82:6801/1763049308] conn(0x7f184403b3c0 0x7f184403d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:04:38.780 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:37.249+0000 7f1853fff700 1 --2- 172.21.15.82:0/1867483540 >> [v2:172.21.15.82:6800/1763049308,v1:172.21.15.82:6801/1763049308] conn(0x7f184403b3c0 0x7f184403d8d0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f184c006fd0 tx=0x7f184c006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:04:38.780 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:37.253+0000 7f1851ffb700 1 -- 172.21.15.82:0/1867483540 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f18580094f0 con 0x7f185c108f10 2024-05-20T16:04:38.780 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:37.515+0000 7f1862f24700 1 -- 172.21.15.82:0/1867483540 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7f185c193d80 con 0x7f185c108f10 2024-05-20T16:04:38.780 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:38.128+0000 7f1851ffb700 1 -- 172.21.15.82:0/1867483540 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7f185c193d80 con 0x7f185c108f10 2024-05-20T16:04:38.780 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:38.130+0000 7f1851ffb700 1 -- 172.21.15.82:0/1867483540 <== mon.0 v2:172.21.15.82:3300/0 8 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7f185802d5d0 con 0x7f185c108f10 2024-05-20T16:04:38.780 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:38.131+0000 7f1862f24700 1 -- 172.21.15.82:0/1867483540 >> [v2:172.21.15.82:6800/1763049308,v1:172.21.15.82:6801/1763049308] conn(0x7f184403b3c0 msgr2=0x7f184403d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:04:38.780 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:38.131+0000 7f1862f24700 1 --2- 172.21.15.82:0/1867483540 >> [v2:172.21.15.82:6800/1763049308,v1:172.21.15.82:6801/1763049308] conn(0x7f184403b3c0 0x7f184403d8d0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f184c006fd0 tx=0x7f184c006e40 comp rx=0 tx=0).stop 2024-05-20T16:04:38.780 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:38.131+0000 7f1862f24700 1 -- 172.21.15.82:0/1867483540 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f185c108f10 msgr2=0x7f185c19ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:04:38.781 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:38.131+0000 7f1862f24700 1 --2- 172.21.15.82:0/1867483540 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f185c108f10 0x7f185c19ab70 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f1858011ad0 tx=0x7f1858011bb0 comp rx=0 tx=0).stop 2024-05-20T16:04:38.781 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:38.131+0000 7f1862f24700 1 -- 172.21.15.82:0/1867483540 shutdown_connections 2024-05-20T16:04:38.781 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:38.131+0000 7f1862f24700 1 --2- 172.21.15.82:0/1867483540 >> [v2:172.21.15.82:6800/1763049308,v1:172.21.15.82:6801/1763049308] conn(0x7f184403b3c0 0x7f184403d8d0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:04:38.781 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:38.131+0000 7f1862f24700 1 --2- 172.21.15.82:0/1867483540 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f185c108f10 0x7f185c19ab70 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:04:38.781 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:38.131+0000 7f1862f24700 1 -- 172.21.15.82:0/1867483540 >> 172.21.15.82:0/1867483540 conn(0x7f185c1028b0 msgr2=0x7f185c190e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:04:38.781 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:38.131+0000 7f1862f24700 1 -- 172.21.15.82:0/1867483540 shutdown_connections 2024-05-20T16:04:38.781 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:38.131+0000 7f1862f24700 1 -- 172.21.15.82:0/1867483540 wait complete. 2024-05-20T16:04:39.069 INFO:journalctl@ceph.mgr.x.smithi082.stdout:May 20 16:04:38 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mgr-x[32467]: 2024-05-20T16:04:38.976+0000 7f1acb2ee200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-20T16:04:39.069 INFO:journalctl@ceph.mgr.x.smithi082.stdout:May 20 16:04:39 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mgr-x[32467]: 2024-05-20T16:04:39.068+0000 7f1acb2ee200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-20T16:04:39.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:04:39 smithi082 ceph-mon[32238]: from='client.? 172.21.15.82:0/1867483540' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-05-20T16:04:39.360 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:04:39 smithi082 ceph-mon[32238]: mgrmap e9: x(active, since 18s) 2024-05-20T16:04:40.069 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout { 2024-05-20T16:04:40.069 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-05-20T16:04:40.069 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "available": true, 2024-05-20T16:04:40.069 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-05-20T16:04:40.069 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-05-20T16:04:40.069 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout } 2024-05-20T16:04:40.069 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:39.201+0000 7f418ecdb700 1 Processor -- start 2024-05-20T16:04:40.070 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:39.201+0000 7f418ecdb700 1 -- start start 2024-05-20T16:04:40.070 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:39.201+0000 7f418ecdb700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f418807bcc0 0x7f418810b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:04:40.070 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:39.201+0000 7f418ecdb700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f418810bbd0 con 0x7f418807bcc0 2024-05-20T16:04:40.070 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:39.202+0000 7f418ca77700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f418807bcc0 0x7f418810b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:04:40.070 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:39.202+0000 7f418ca77700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f418807bcc0 0x7f418810b5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:57376/0 (socket says 172.21.15.82:57376) 2024-05-20T16:04:40.070 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:39.202+0000 7f418ca77700 1 -- 172.21.15.82:0/1457242208 learned_addr learned my addr 172.21.15.82:0/1457242208 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:04:40.070 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:39.202+0000 7f418ca77700 1 -- 172.21.15.82:0/1457242208 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f418810bd50 con 0x7f418807bcc0 2024-05-20T16:04:40.070 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:39.202+0000 7f418ca77700 1 --2- 172.21.15.82:0/1457242208 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f418807bcc0 0x7f418810b5c0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f417c00f960 tx=0x7f417c00fc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8959711a78bd9997 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:04:40.070 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:39.202+0000 7f41877fe700 1 -- 172.21.15.82:0/1457242208 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f417c01c070 con 0x7f418807bcc0 2024-05-20T16:04:40.070 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:39.202+0000 7f41877fe700 1 -- 172.21.15.82:0/1457242208 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f417c004510 con 0x7f418807bcc0 2024-05-20T16:04:40.070 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:39.202+0000 7f41877fe700 1 -- 172.21.15.82:0/1457242208 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f417c003d10 con 0x7f418807bcc0 2024-05-20T16:04:40.070 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:39.203+0000 7f418ecdb700 1 -- 172.21.15.82:0/1457242208 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f418807bcc0 msgr2=0x7f418810b5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:04:40.071 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:39.203+0000 7f418ecdb700 1 --2- 172.21.15.82:0/1457242208 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f418807bcc0 0x7f418810b5c0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f417c00f960 tx=0x7f417c00fc70 comp rx=0 tx=0).stop 2024-05-20T16:04:40.071 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:39.203+0000 7f418ecdb700 1 -- 172.21.15.82:0/1457242208 shutdown_connections 2024-05-20T16:04:40.071 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:39.203+0000 7f418ecdb700 1 --2- 172.21.15.82:0/1457242208 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f418807bcc0 0x7f418810b5c0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:04:40.071 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:39.203+0000 7f418ecdb700 1 -- 172.21.15.82:0/1457242208 >> 172.21.15.82:0/1457242208 conn(0x7f4188103070 msgr2=0x7f41881054c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:04:40.071 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:39.203+0000 7f418ecdb700 1 -- 172.21.15.82:0/1457242208 shutdown_connections 2024-05-20T16:04:40.071 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:39.203+0000 7f418ecdb700 1 -- 172.21.15.82:0/1457242208 wait complete. 2024-05-20T16:04:40.071 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:39.204+0000 7f418ecdb700 1 Processor -- start 2024-05-20T16:04:40.071 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:39.204+0000 7f418ecdb700 1 -- start start 2024-05-20T16:04:40.071 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:39.204+0000 7f418ecdb700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f418807bcc0 0x7f418819ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:04:40.071 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:39.204+0000 7f418ecdb700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f4188108230 con 0x7f418807bcc0 2024-05-20T16:04:40.071 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:39.204+0000 7f418ca77700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f418807bcc0 0x7f418819ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:04:40.071 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:39.204+0000 7f418ca77700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f418807bcc0 0x7f418819ab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:57390/0 (socket says 172.21.15.82:57390) 2024-05-20T16:04:40.071 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:39.204+0000 7f418ca77700 1 -- 172.21.15.82:0/3775194216 learned_addr learned my addr 172.21.15.82:0/3775194216 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:04:40.072 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:39.204+0000 7f418ca77700 1 -- 172.21.15.82:0/3775194216 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f418819b0b0 con 0x7f418807bcc0 2024-05-20T16:04:40.072 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:39.205+0000 7f418ca77700 1 --2- 172.21.15.82:0/3775194216 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f418807bcc0 0x7f418819ab70 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f417c0046e0 tx=0x7f417c0042a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:04:40.072 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:39.205+0000 7f4185ffb700 1 -- 172.21.15.82:0/3775194216 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f417c025070 con 0x7f418807bcc0 2024-05-20T16:04:40.072 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:39.205+0000 7f4185ffb700 1 -- 172.21.15.82:0/3775194216 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f417c0036a0 con 0x7f418807bcc0 2024-05-20T16:04:40.072 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:39.205+0000 7f418ecdb700 1 -- 172.21.15.82:0/3775194216 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f418819b340 con 0x7f418807bcc0 2024-05-20T16:04:40.072 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:39.205+0000 7f4185ffb700 1 -- 172.21.15.82:0/3775194216 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f417c01c040 con 0x7f418807bcc0 2024-05-20T16:04:40.072 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:39.205+0000 7f418ecdb700 1 -- 172.21.15.82:0/3775194216 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f418819df50 con 0x7f418807bcc0 2024-05-20T16:04:40.072 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:39.206+0000 7f4185ffb700 1 -- 172.21.15.82:0/3775194216 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7f417c028070 con 0x7f418807bcc0 2024-05-20T16:04:40.072 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:39.206+0000 7f4185ffb700 1 --2- 172.21.15.82:0/3775194216 >> [v2:172.21.15.82:6800/1763049308,v1:172.21.15.82:6801/1763049308] conn(0x7f4178039170 0x7f417803b680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:04:40.072 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:39.206+0000 7f4185ffb700 1 -- 172.21.15.82:0/3775194216 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f417c014070 con 0x7f418807bcc0 2024-05-20T16:04:40.072 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:39.206+0000 7f4187fff700 1 -- 172.21.15.82:0/3775194216 >> [v2:172.21.15.82:6800/1763049308,v1:172.21.15.82:6801/1763049308] conn(0x7f4178039170 msgr2=0x7f417803b680 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.82:6800/1763049308 2024-05-20T16:04:40.072 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:39.206+0000 7f4187fff700 1 --2- 172.21.15.82:0/3775194216 >> [v2:172.21.15.82:6800/1763049308,v1:172.21.15.82:6801/1763049308] conn(0x7f4178039170 0x7f417803b680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-20T16:04:40.072 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:39.206+0000 7f418ecdb700 1 -- 172.21.15.82:0/3775194216 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4188063ca0 con 0x7f418807bcc0 2024-05-20T16:04:40.072 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:39.211+0000 7f4185ffb700 1 -- 172.21.15.82:0/3775194216 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f417c0131d0 con 0x7f418807bcc0 2024-05-20T16:04:40.072 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:39.407+0000 7f4187fff700 1 -- 172.21.15.82:0/3775194216 >> [v2:172.21.15.82:6800/1763049308,v1:172.21.15.82:6801/1763049308] conn(0x7f4178039170 msgr2=0x7f417803b680 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.82:6800/1763049308 2024-05-20T16:04:40.073 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:39.407+0000 7f4187fff700 1 --2- 172.21.15.82:0/3775194216 >> [v2:172.21.15.82:6800/1763049308,v1:172.21.15.82:6801/1763049308] conn(0x7f4178039170 0x7f417803b680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-20T16:04:40.073 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:39.479+0000 7f418ecdb700 1 -- 172.21.15.82:0/3775194216 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f418802de20 con 0x7f418807bcc0 2024-05-20T16:04:40.073 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:39.480+0000 7f4185ffb700 1 -- 172.21.15.82:0/3775194216 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7f417c025d00 con 0x7f418807bcc0 2024-05-20T16:04:40.073 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:39.484+0000 7f418ecdb700 1 -- 172.21.15.82:0/3775194216 >> [v2:172.21.15.82:6800/1763049308,v1:172.21.15.82:6801/1763049308] conn(0x7f4178039170 msgr2=0x7f417803b680 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-20T16:04:40.073 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:39.484+0000 7f418ecdb700 1 --2- 172.21.15.82:0/3775194216 >> [v2:172.21.15.82:6800/1763049308,v1:172.21.15.82:6801/1763049308] conn(0x7f4178039170 0x7f417803b680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:04:40.073 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:39.484+0000 7f418ecdb700 1 -- 172.21.15.82:0/3775194216 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f418807bcc0 msgr2=0x7f418819ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:04:40.073 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:39.484+0000 7f418ecdb700 1 --2- 172.21.15.82:0/3775194216 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f418807bcc0 0x7f418819ab70 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f417c0046e0 tx=0x7f417c0042a0 comp rx=0 tx=0).stop 2024-05-20T16:04:40.073 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:39.484+0000 7f418ecdb700 1 -- 172.21.15.82:0/3775194216 shutdown_connections 2024-05-20T16:04:40.073 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:39.484+0000 7f418ecdb700 1 --2- 172.21.15.82:0/3775194216 >> [v2:172.21.15.82:6800/1763049308,v1:172.21.15.82:6801/1763049308] conn(0x7f4178039170 0x7f417803b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:04:40.073 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:39.484+0000 7f418ecdb700 1 --2- 172.21.15.82:0/3775194216 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f418807bcc0 0x7f418819ab70 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:04:40.073 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:39.484+0000 7f418ecdb700 1 -- 172.21.15.82:0/3775194216 >> 172.21.15.82:0/3775194216 conn(0x7f4188103070 msgr2=0x7f4188105100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:04:40.073 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:39.484+0000 7f418ecdb700 1 -- 172.21.15.82:0/3775194216 shutdown_connections 2024-05-20T16:04:40.073 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:39.485+0000 7f418ecdb700 1 -- 172.21.15.82:0/3775194216 wait complete. 2024-05-20T16:04:40.073 INFO:teuthology.orchestra.run.smithi082.stdout:Waiting for the mgr to restart... 2024-05-20T16:04:40.074 INFO:teuthology.orchestra.run.smithi082.stdout:Waiting for mgr epoch 9... 2024-05-20T16:04:40.405 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:04:40 smithi082 ceph-mon[32238]: from='client.? 172.21.15.82:0/3775194216' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-05-20T16:04:40.406 INFO:journalctl@ceph.mgr.x.smithi082.stdout:May 20 16:04:40 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mgr-x[32467]: 2024-05-20T16:04:40.310+0000 7f1acb2ee200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-20T16:04:40.673 INFO:journalctl@ceph.mgr.x.smithi082.stdout:May 20 16:04:40 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mgr-x[32467]: 2024-05-20T16:04:40.500+0000 7f1acb2ee200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-20T16:04:40.996 INFO:journalctl@ceph.mgr.x.smithi082.stdout:May 20 16:04:40 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mgr-x[32467]: 2024-05-20T16:04:40.682+0000 7f1acb2ee200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-20T16:04:41.304 INFO:journalctl@ceph.mgr.x.smithi082.stdout:May 20 16:04:40 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mgr-x[32467]: 2024-05-20T16:04:40.995+0000 7f1acb2ee200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-20T16:04:41.305 INFO:journalctl@ceph.mgr.x.smithi082.stdout:May 20 16:04:41 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mgr-x[32467]: 2024-05-20T16:04:41.093+0000 7f1acb2ee200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-20T16:04:41.305 INFO:journalctl@ceph.mgr.x.smithi082.stdout:May 20 16:04:41 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mgr-x[32467]: 2024-05-20T16:04:41.304+0000 7f1acb2ee200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-20T16:04:41.567 INFO:journalctl@ceph.mgr.x.smithi082.stdout:May 20 16:04:41 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mgr-x[32467]: 2024-05-20T16:04:41.504+0000 7f1acb2ee200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-20T16:04:42.360 INFO:journalctl@ceph.mgr.x.smithi082.stdout:May 20 16:04:42 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mgr-x[32467]: 2024-05-20T16:04:42.113+0000 7f1acb2ee200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-20T16:04:42.839 INFO:journalctl@ceph.mgr.x.smithi082.stdout:May 20 16:04:42 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mgr-x[32467]: 2024-05-20T16:04:42.537+0000 7f1acb2ee200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-20T16:04:42.839 INFO:journalctl@ceph.mgr.x.smithi082.stdout:May 20 16:04:42 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mgr-x[32467]: 2024-05-20T16:04:42.635+0000 7f1acb2ee200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-20T16:04:42.840 INFO:journalctl@ceph.mgr.x.smithi082.stdout:May 20 16:04:42 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mgr-x[32467]: 2024-05-20T16:04:42.838+0000 7f1acb2ee200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-20T16:04:43.610 INFO:journalctl@ceph.mgr.x.smithi082.stdout:May 20 16:04:43 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mgr-x[32467]: 2024-05-20T16:04:43.359+0000 7f1acb2ee200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-20T16:04:43.610 INFO:journalctl@ceph.mgr.x.smithi082.stdout:May 20 16:04:43 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mgr-x[32467]: 2024-05-20T16:04:43.451+0000 7f1acb2ee200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-20T16:04:43.926 INFO:journalctl@ceph.mgr.x.smithi082.stdout:May 20 16:04:43 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mgr-x[32467]: 2024-05-20T16:04:43.695+0000 7f1acb2ee200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-20T16:04:43.926 INFO:journalctl@ceph.mgr.x.smithi082.stdout:May 20 16:04:43 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mgr-x[32467]: 2024-05-20T16:04:43.829+0000 7f1acb2ee200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-20T16:04:43.927 INFO:journalctl@ceph.mgr.x.smithi082.stdout:May 20 16:04:43 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mgr-x[32467]: 2024-05-20T16:04:43.925+0000 7f1acb2ee200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-20T16:04:44.341 INFO:journalctl@ceph.mgr.x.smithi082.stdout:May 20 16:04:44 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mgr-x[32467]: 2024-05-20T16:04:44.043+0000 7f1acb2ee200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-20T16:04:44.341 INFO:journalctl@ceph.mgr.x.smithi082.stdout:May 20 16:04:44 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mgr-x[32467]: 2024-05-20T16:04:44.340+0000 7f1acb2ee200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-20T16:04:45.213 INFO:journalctl@ceph.mgr.x.smithi082.stdout:May 20 16:04:44 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mgr-x[32467]: 2024-05-20T16:04:44.942+0000 7f1acb2ee200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-20T16:04:45.213 INFO:journalctl@ceph.mgr.x.smithi082.stdout:May 20 16:04:45 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mgr-x[32467]: 2024-05-20T16:04:45.159+0000 7f1acb2ee200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-20T16:04:45.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:04:45 smithi082 ceph-mon[32238]: Active manager daemon x restarted 2024-05-20T16:04:45.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:04:45 smithi082 ceph-mon[32238]: Activating manager daemon x 2024-05-20T16:04:45.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:04:45 smithi082 ceph-mon[32238]: osdmap e3: 0 total, 0 up, 0 in 2024-05-20T16:04:45.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:04:45 smithi082 ceph-mon[32238]: mgrmap e10: x(active, starting, since 0.00367811s) 2024-05-20T16:04:45.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:04:45 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-20T16:04:45.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:04:45 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-20T16:04:45.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:04:45 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-20T16:04:45.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:04:45 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-20T16:04:45.611 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:04:45 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-20T16:04:45.611 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:04:45 smithi082 ceph-mon[32238]: Manager daemon x is now available 2024-05-20T16:04:45.611 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:04:45 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T16:04:45.611 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:04:45 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-20T16:04:46.360 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:04:46 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-20T16:04:46.360 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:04:46 smithi082 ceph-mon[32238]: mgrmap e11: x(active, since 1.00638s) 2024-05-20T16:04:46.968 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout { 2024-05-20T16:04:46.968 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-05-20T16:04:46.968 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "initialized": true 2024-05-20T16:04:46.968 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout } 2024-05-20T16:04:46.968 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:40.614+0000 7fb45f540700 1 Processor -- start 2024-05-20T16:04:46.968 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:40.614+0000 7fb45f540700 1 -- start start 2024-05-20T16:04:46.968 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:40.614+0000 7fb45f540700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb45810a000 0x7fb45810a410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:04:46.968 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:40.614+0000 7fb45f540700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fb45810a9e0 con 0x7fb45810a000 2024-05-20T16:04:46.968 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:40.614+0000 7fb45d2dc700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb45810a000 0x7fb45810a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:04:46.968 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:40.614+0000 7fb45d2dc700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb45810a000 0x7fb45810a410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:54354/0 (socket says 172.21.15.82:54354) 2024-05-20T16:04:46.968 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:40.614+0000 7fb45d2dc700 1 -- 172.21.15.82:0/1940209614 learned_addr learned my addr 172.21.15.82:0/1940209614 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:04:46.968 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:40.615+0000 7fb45d2dc700 1 -- 172.21.15.82:0/1940209614 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb45810ab60 con 0x7fb45810a000 2024-05-20T16:04:46.969 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:40.615+0000 7fb45d2dc700 1 --2- 172.21.15.82:0/1940209614 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb45810a000 0x7fb45810a410 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fb4540098a0 tx=0x7fb454009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87c0902bc77245dd server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:04:46.969 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:40.615+0000 7fb44ffff700 1 -- 172.21.15.82:0/1940209614 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb454004160 con 0x7fb45810a000 2024-05-20T16:04:46.969 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:40.615+0000 7fb44ffff700 1 -- 172.21.15.82:0/1940209614 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb454004300 con 0x7fb45810a000 2024-05-20T16:04:46.969 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:40.615+0000 7fb44ffff700 1 -- 172.21.15.82:0/1940209614 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb454011b20 con 0x7fb45810a000 2024-05-20T16:04:46.969 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:40.616+0000 7fb45f540700 1 -- 172.21.15.82:0/1940209614 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb45810a000 msgr2=0x7fb45810a410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:04:46.969 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:40.616+0000 7fb45f540700 1 --2- 172.21.15.82:0/1940209614 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb45810a000 0x7fb45810a410 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fb4540098a0 tx=0x7fb454009bb0 comp rx=0 tx=0).stop 2024-05-20T16:04:46.969 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:40.616+0000 7fb45f540700 1 -- 172.21.15.82:0/1940209614 shutdown_connections 2024-05-20T16:04:46.969 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:40.616+0000 7fb45f540700 1 --2- 172.21.15.82:0/1940209614 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb45810a000 0x7fb45810a410 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:04:46.969 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:40.616+0000 7fb45f540700 1 -- 172.21.15.82:0/1940209614 >> 172.21.15.82:0/1940209614 conn(0x7fb4581027a0 msgr2=0x7fb458104bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:04:46.969 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:40.616+0000 7fb45f540700 1 -- 172.21.15.82:0/1940209614 shutdown_connections 2024-05-20T16:04:46.969 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:40.616+0000 7fb45f540700 1 -- 172.21.15.82:0/1940209614 wait complete. 2024-05-20T16:04:46.969 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:40.617+0000 7fb45f540700 1 Processor -- start 2024-05-20T16:04:46.969 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:40.617+0000 7fb45f540700 1 -- start start 2024-05-20T16:04:46.969 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:40.618+0000 7fb45f540700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb45810a000 0x7fb45819cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:04:46.970 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:40.618+0000 7fb45f540700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fb45810cfa0 con 0x7fb45810a000 2024-05-20T16:04:46.970 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:40.618+0000 7fb45d2dc700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb45810a000 0x7fb45819cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:04:46.970 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:40.618+0000 7fb45d2dc700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb45810a000 0x7fb45819cd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:54362/0 (socket says 172.21.15.82:54362) 2024-05-20T16:04:46.970 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:40.618+0000 7fb45d2dc700 1 -- 172.21.15.82:0/3772625688 learned_addr learned my addr 172.21.15.82:0/3772625688 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:04:46.970 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:40.618+0000 7fb45d2dc700 1 -- 172.21.15.82:0/3772625688 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb45819d270 con 0x7fb45810a000 2024-05-20T16:04:46.970 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:40.618+0000 7fb45d2dc700 1 --2- 172.21.15.82:0/3772625688 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb45810a000 0x7fb45819cd30 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fb454011740 tx=0x7fb454011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:04:46.970 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:40.618+0000 7fb44e7fc700 1 -- 172.21.15.82:0/3772625688 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb454003a40 con 0x7fb45810a000 2024-05-20T16:04:46.970 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:40.618+0000 7fb44e7fc700 1 -- 172.21.15.82:0/3772625688 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb454003be0 con 0x7fb45810a000 2024-05-20T16:04:46.970 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:40.618+0000 7fb44e7fc700 1 -- 172.21.15.82:0/3772625688 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb45401a720 con 0x7fb45810a000 2024-05-20T16:04:46.970 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:40.619+0000 7fb45f540700 1 -- 172.21.15.82:0/3772625688 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb45819d500 con 0x7fb45810a000 2024-05-20T16:04:46.970 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:40.619+0000 7fb45f540700 1 -- 172.21.15.82:0/3772625688 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4581a0100 con 0x7fb45810a000 2024-05-20T16:04:46.970 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:40.620+0000 7fb44e7fc700 1 -- 172.21.15.82:0/3772625688 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7fb45401a9d0 con 0x7fb45810a000 2024-05-20T16:04:46.970 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:40.620+0000 7fb44e7fc700 1 --2- 172.21.15.82:0/3772625688 >> [v2:172.21.15.82:6800/1763049308,v1:172.21.15.82:6801/1763049308] conn(0x7fb444039170 0x7fb44403b680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:04:46.970 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:40.620+0000 7fb44e7fc700 1 -- 172.21.15.82:0/3772625688 --> [v2:172.21.15.82:6800/1763049308,v1:172.21.15.82:6801/1763049308] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fb44403bd90 con 0x7fb444039170 2024-05-20T16:04:46.971 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:40.620+0000 7fb44e7fc700 1 -- 172.21.15.82:0/3772625688 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fb45404d680 con 0x7fb45810a000 2024-05-20T16:04:46.971 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:40.620+0000 7fb45cadb700 1 -- 172.21.15.82:0/3772625688 >> [v2:172.21.15.82:6800/1763049308,v1:172.21.15.82:6801/1763049308] conn(0x7fb444039170 msgr2=0x7fb44403b680 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.82:6800/1763049308 2024-05-20T16:04:46.971 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:40.620+0000 7fb45cadb700 1 --2- 172.21.15.82:0/3772625688 >> [v2:172.21.15.82:6800/1763049308,v1:172.21.15.82:6801/1763049308] conn(0x7fb444039170 0x7fb44403b680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-20T16:04:46.972 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:40.821+0000 7fb45cadb700 1 -- 172.21.15.82:0/3772625688 >> [v2:172.21.15.82:6800/1763049308,v1:172.21.15.82:6801/1763049308] conn(0x7fb444039170 msgr2=0x7fb44403b680 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.82:6800/1763049308 2024-05-20T16:04:46.972 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:40.821+0000 7fb45cadb700 1 --2- 172.21.15.82:0/3772625688 >> [v2:172.21.15.82:6800/1763049308,v1:172.21.15.82:6801/1763049308] conn(0x7fb444039170 0x7fb44403b680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-20T16:04:46.972 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:41.222+0000 7fb45cadb700 1 -- 172.21.15.82:0/3772625688 >> [v2:172.21.15.82:6800/1763049308,v1:172.21.15.82:6801/1763049308] conn(0x7fb444039170 msgr2=0x7fb44403b680 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.82:6800/1763049308 2024-05-20T16:04:46.972 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:41.222+0000 7fb45cadb700 1 --2- 172.21.15.82:0/3772625688 >> [v2:172.21.15.82:6800/1763049308,v1:172.21.15.82:6801/1763049308] conn(0x7fb444039170 0x7fb44403b680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-20T16:04:46.972 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:42.023+0000 7fb45cadb700 1 -- 172.21.15.82:0/3772625688 >> [v2:172.21.15.82:6800/1763049308,v1:172.21.15.82:6801/1763049308] conn(0x7fb444039170 msgr2=0x7fb44403b680 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.82:6800/1763049308 2024-05-20T16:04:46.972 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:42.023+0000 7fb45cadb700 1 --2- 172.21.15.82:0/3772625688 >> [v2:172.21.15.82:6800/1763049308,v1:172.21.15.82:6801/1763049308] conn(0x7fb444039170 0x7fb44403b680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-20T16:04:46.972 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:43.624+0000 7fb45cadb700 1 -- 172.21.15.82:0/3772625688 >> [v2:172.21.15.82:6800/1763049308,v1:172.21.15.82:6801/1763049308] conn(0x7fb444039170 msgr2=0x7fb44403b680 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.82:6800/1763049308 2024-05-20T16:04:46.972 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:43.624+0000 7fb45cadb700 1 --2- 172.21.15.82:0/3772625688 >> [v2:172.21.15.82:6800/1763049308,v1:172.21.15.82:6801/1763049308] conn(0x7fb444039170 0x7fb44403b680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-05-20T16:04:46.973 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:45.163+0000 7fb44e7fc700 1 -- 172.21.15.82:0/3772625688 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mgrmap(e 10) ==== 46067+0+0 (secure 0 0 0) 0x7fb45402d910 con 0x7fb45810a000 2024-05-20T16:04:46.973 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:45.163+0000 7fb44e7fc700 1 -- 172.21.15.82:0/3772625688 >> [v2:172.21.15.82:6800/1763049308,v1:172.21.15.82:6801/1763049308] conn(0x7fb444039170 msgr2=0x7fb44403b680 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-20T16:04:46.973 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:45.163+0000 7fb44e7fc700 1 --2- 172.21.15.82:0/3772625688 >> [v2:172.21.15.82:6800/1763049308,v1:172.21.15.82:6801/1763049308] conn(0x7fb444039170 0x7fb44403b680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:04:46.973 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:46.164+0000 7fb44e7fc700 1 -- 172.21.15.82:0/3772625688 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mgrmap(e 11) ==== 46194+0+0 (secure 0 0 0) 0x7fb45404f280 con 0x7fb45810a000 2024-05-20T16:04:46.973 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:46.165+0000 7fb44e7fc700 1 --2- 172.21.15.82:0/3772625688 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fb45404d210 0x7fb44403ec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:04:46.973 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:46.165+0000 7fb44e7fc700 1 -- 172.21.15.82:0/3772625688 --> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fb454025e70 con 0x7fb45404d210 2024-05-20T16:04:46.973 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:46.165+0000 7fb45cadb700 1 --2- 172.21.15.82:0/3772625688 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fb45404d210 0x7fb44403ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:04:46.973 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:46.166+0000 7fb45cadb700 1 --2- 172.21.15.82:0/3772625688 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fb45404d210 0x7fb44403ec90 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fb448003a10 tx=0x7fb448009270 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:04:46.973 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:46.166+0000 7fb44e7fc700 1 -- 172.21.15.82:0/3772625688 <== mgr.14150 v2:172.21.15.82:6800/379530823 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7774 (secure 0 0 0) 0x7fb454025e70 con 0x7fb45404d210 2024-05-20T16:04:46.973 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:46.173+0000 7fb45f540700 1 -- 172.21.15.82:0/3772625688 --> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7fb45819d960 con 0x7fb45404d210 2024-05-20T16:04:46.973 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:46.173+0000 7fb44e7fc700 1 -- 172.21.15.82:0/3772625688 <== mgr.14150 v2:172.21.15.82:6800/379530823 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7fb45819d960 con 0x7fb45404d210 2024-05-20T16:04:46.973 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:46.173+0000 7fb45f540700 1 -- 172.21.15.82:0/3772625688 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fb45404d210 msgr2=0x7fb44403ec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:04:46.973 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:46.173+0000 7fb45f540700 1 --2- 172.21.15.82:0/3772625688 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fb45404d210 0x7fb44403ec90 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fb448003a10 tx=0x7fb448009270 comp rx=0 tx=0).stop 2024-05-20T16:04:46.973 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:46.173+0000 7fb45f540700 1 -- 172.21.15.82:0/3772625688 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb45810a000 msgr2=0x7fb45819cd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:04:46.974 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:46.173+0000 7fb45f540700 1 --2- 172.21.15.82:0/3772625688 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb45810a000 0x7fb45819cd30 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fb454011740 tx=0x7fb454011ba0 comp rx=0 tx=0).stop 2024-05-20T16:04:46.974 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:46.173+0000 7fb45f540700 1 -- 172.21.15.82:0/3772625688 shutdown_connections 2024-05-20T16:04:46.974 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:46.173+0000 7fb45f540700 1 --2- 172.21.15.82:0/3772625688 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fb45404d210 0x7fb44403ec90 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:04:46.974 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:46.173+0000 7fb45f540700 1 --2- 172.21.15.82:0/3772625688 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb45810a000 0x7fb45819cd30 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:04:46.974 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:46.173+0000 7fb45f540700 1 --2- 172.21.15.82:0/3772625688 >> [v2:172.21.15.82:6800/1763049308,v1:172.21.15.82:6801/1763049308] conn(0x7fb444039170 0x7fb44403b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:04:46.974 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:46.173+0000 7fb45f540700 1 -- 172.21.15.82:0/3772625688 >> 172.21.15.82:0/3772625688 conn(0x7fb4581027a0 msgr2=0x7fb458102ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:04:46.974 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:46.174+0000 7fb45f540700 1 -- 172.21.15.82:0/3772625688 shutdown_connections 2024-05-20T16:04:46.974 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:46.174+0000 7fb45f540700 1 -- 172.21.15.82:0/3772625688 wait complete. 2024-05-20T16:04:46.974 INFO:teuthology.orchestra.run.smithi082.stdout:mgr epoch 9 is available 2024-05-20T16:04:46.974 INFO:teuthology.orchestra.run.smithi082.stdout:Generating a dashboard self-signed certificate... 2024-05-20T16:04:47.335 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:04:47 smithi082 ceph-mon[32238]: [20/May/2024:16:04:45] ENGINE Bus STARTING 2024-05-20T16:04:47.335 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:04:47 smithi082 ceph-mon[32238]: [20/May/2024:16:04:45] ENGINE Serving on http://172.21.15.82:8765 2024-05-20T16:04:47.336 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:04:47 smithi082 ceph-mon[32238]: [20/May/2024:16:04:45] ENGINE Serving on https://172.21.15.82:7150 2024-05-20T16:04:47.336 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:04:47 smithi082 ceph-mon[32238]: [20/May/2024:16:04:45] ENGINE Bus STARTED 2024-05-20T16:04:47.336 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:04:47 smithi082 ceph-mon[32238]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-05-20T16:04:47.336 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:04:47 smithi082 ceph-mon[32238]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-05-20T16:04:47.336 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:04:47 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:04:48.354 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-05-20T16:04:48.355 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:47.448+0000 7f7609442700 1 Processor -- start 2024-05-20T16:04:48.355 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:47.448+0000 7f7609442700 1 -- start start 2024-05-20T16:04:48.355 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:47.449+0000 7f7609442700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7604109570 0x7f760410b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:04:48.355 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:47.449+0000 7f7609442700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f760407bdd0 con 0x7f7604109570 2024-05-20T16:04:48.355 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:47.449+0000 7f7602ffd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7604109570 0x7f760410b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:04:48.355 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:47.449+0000 7f7602ffd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7604109570 0x7f760410b990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:54454/0 (socket says 172.21.15.82:54454) 2024-05-20T16:04:48.355 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:47.449+0000 7f7602ffd700 1 -- 172.21.15.82:0/1304820844 learned_addr learned my addr 172.21.15.82:0/1304820844 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:04:48.355 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:47.449+0000 7f7602ffd700 1 -- 172.21.15.82:0/1304820844 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f760410bed0 con 0x7f7604109570 2024-05-20T16:04:48.355 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:47.450+0000 7f7602ffd700 1 --2- 172.21.15.82:0/1304820844 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7604109570 0x7f760410b990 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f75f40098a0 tx=0x7f75f4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68a3dd49199c05ae server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:04:48.355 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:47.450+0000 7f7601ffb700 1 -- 172.21.15.82:0/1304820844 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f75f4004160 con 0x7f7604109570 2024-05-20T16:04:48.355 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:47.450+0000 7f7601ffb700 1 -- 172.21.15.82:0/1304820844 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f75f4004300 con 0x7f7604109570 2024-05-20T16:04:48.356 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:47.450+0000 7f7601ffb700 1 -- 172.21.15.82:0/1304820844 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f75f4011b20 con 0x7f7604109570 2024-05-20T16:04:48.356 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:47.451+0000 7f7609442700 1 -- 172.21.15.82:0/1304820844 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7604109570 msgr2=0x7f760410b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:04:48.356 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:47.451+0000 7f7609442700 1 --2- 172.21.15.82:0/1304820844 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7604109570 0x7f760410b990 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f75f40098a0 tx=0x7f75f4009bb0 comp rx=0 tx=0).stop 2024-05-20T16:04:48.356 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:47.451+0000 7f7609442700 1 -- 172.21.15.82:0/1304820844 shutdown_connections 2024-05-20T16:04:48.356 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:47.451+0000 7f7609442700 1 --2- 172.21.15.82:0/1304820844 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7604109570 0x7f760410b990 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:04:48.356 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:47.451+0000 7f7609442700 1 -- 172.21.15.82:0/1304820844 >> 172.21.15.82:0/1304820844 conn(0x7f7604103070 msgr2=0x7f76041054c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:04:48.356 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:47.451+0000 7f7609442700 1 -- 172.21.15.82:0/1304820844 shutdown_connections 2024-05-20T16:04:48.356 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:47.451+0000 7f7609442700 1 -- 172.21.15.82:0/1304820844 wait complete. 2024-05-20T16:04:48.356 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:47.451+0000 7f7609442700 1 Processor -- start 2024-05-20T16:04:48.356 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:47.452+0000 7f7609442700 1 -- start start 2024-05-20T16:04:48.356 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:47.452+0000 7f7609442700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7604109570 0x7f7604108cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:04:48.356 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:47.452+0000 7f7609442700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f760410ce70 con 0x7f7604109570 2024-05-20T16:04:48.356 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:47.452+0000 7f7602ffd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7604109570 0x7f7604108cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:04:48.357 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:47.452+0000 7f7602ffd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7604109570 0x7f7604108cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:54466/0 (socket says 172.21.15.82:54466) 2024-05-20T16:04:48.357 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:47.452+0000 7f7602ffd700 1 -- 172.21.15.82:0/3037714412 learned_addr learned my addr 172.21.15.82:0/3037714412 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:04:48.357 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:47.452+0000 7f7602ffd700 1 -- 172.21.15.82:0/3037714412 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7604109230 con 0x7f7604109570 2024-05-20T16:04:48.357 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:47.452+0000 7f7602ffd700 1 --2- 172.21.15.82:0/3037714412 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7604109570 0x7f7604108cf0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f75f4000c00 tx=0x7f75f4011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:04:48.357 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:47.453+0000 7f75ebfff700 1 -- 172.21.15.82:0/3037714412 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f75f4003b60 con 0x7f7604109570 2024-05-20T16:04:48.357 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:47.453+0000 7f7609442700 1 -- 172.21.15.82:0/3037714412 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7604107480 con 0x7f7604109570 2024-05-20T16:04:48.357 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:47.453+0000 7f75ebfff700 1 -- 172.21.15.82:0/3037714412 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f75f401a910 con 0x7f7604109570 2024-05-20T16:04:48.357 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:47.453+0000 7f75ebfff700 1 -- 172.21.15.82:0/3037714412 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f75f4024930 con 0x7f7604109570 2024-05-20T16:04:48.357 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:47.453+0000 7f7609442700 1 -- 172.21.15.82:0/3037714412 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76041078e0 con 0x7f7604109570 2024-05-20T16:04:48.357 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:47.454+0000 7f75ebfff700 1 -- 172.21.15.82:0/3037714412 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 11) ==== 46194+0+0 (secure 0 0 0) 0x7f75f4022020 con 0x7f7604109570 2024-05-20T16:04:48.357 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:47.454+0000 7f7609442700 1 -- 172.21.15.82:0/3037714412 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7604063ca0 con 0x7f7604109570 2024-05-20T16:04:48.357 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:47.454+0000 7f75ebfff700 1 --2- 172.21.15.82:0/3037714412 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f75ec039040 0x7f75ec03b550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:04:48.357 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:47.454+0000 7f75ebfff700 1 -- 172.21.15.82:0/3037714412 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f75f404d240 con 0x7f7604109570 2024-05-20T16:04:48.357 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:47.454+0000 7f76027fc700 1 --2- 172.21.15.82:0/3037714412 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f75ec039040 0x7f75ec03b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:04:48.358 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:47.454+0000 7f76027fc700 1 --2- 172.21.15.82:0/3037714412 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f75ec039040 0x7f75ec03b550 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f75f8006fd0 tx=0x7f75f8006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:04:48.358 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:47.459+0000 7f75ebfff700 1 -- 172.21.15.82:0/3037714412 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f75f400b5a0 con 0x7f7604109570 2024-05-20T16:04:48.358 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:47.671+0000 7f7609442700 1 -- 172.21.15.82:0/3037714412 --> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7f7604108010 con 0x7f75ec039040 2024-05-20T16:04:48.358 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:47.674+0000 7f75ebfff700 1 -- 172.21.15.82:0/3037714412 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mgrmap(e 12) ==== 46300+0+0 (secure 0 0 0) 0x7f75f401a430 con 0x7f7604109570 2024-05-20T16:04:48.358 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:47.804+0000 7f75ebfff700 1 -- 172.21.15.82:0/3037714412 <== mgr.14150 v2:172.21.15.82:6800/379530823 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7f7604108010 con 0x7f75ec039040 2024-05-20T16:04:48.358 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:47.807+0000 7f7609442700 1 -- 172.21.15.82:0/3037714412 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f75ec039040 msgr2=0x7f75ec03b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:04:48.358 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:47.807+0000 7f7609442700 1 --2- 172.21.15.82:0/3037714412 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f75ec039040 0x7f75ec03b550 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f75f8006fd0 tx=0x7f75f8006e40 comp rx=0 tx=0).stop 2024-05-20T16:04:48.358 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:47.807+0000 7f7609442700 1 -- 172.21.15.82:0/3037714412 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7604109570 msgr2=0x7f7604108cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:04:48.358 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:47.807+0000 7f7609442700 1 --2- 172.21.15.82:0/3037714412 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7604109570 0x7f7604108cf0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f75f4000c00 tx=0x7f75f4011c10 comp rx=0 tx=0).stop 2024-05-20T16:04:48.358 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:47.807+0000 7f7609442700 1 -- 172.21.15.82:0/3037714412 shutdown_connections 2024-05-20T16:04:48.358 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:47.807+0000 7f7609442700 1 --2- 172.21.15.82:0/3037714412 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f75ec039040 0x7f75ec03b550 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:04:48.358 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:47.807+0000 7f7609442700 1 --2- 172.21.15.82:0/3037714412 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7604109570 0x7f7604108cf0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:04:48.358 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:47.807+0000 7f7609442700 1 -- 172.21.15.82:0/3037714412 >> 172.21.15.82:0/3037714412 conn(0x7f7604103070 msgr2=0x7f76041048f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:04:48.358 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:47.807+0000 7f7609442700 1 -- 172.21.15.82:0/3037714412 shutdown_connections 2024-05-20T16:04:48.359 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:47.807+0000 7f7609442700 1 -- 172.21.15.82:0/3037714412 wait complete. 2024-05-20T16:04:48.359 INFO:teuthology.orchestra.run.smithi082.stdout:Creating initial admin user... 2024-05-20T16:04:48.886 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:04:48 smithi082 ceph-mon[32238]: from='client.14162 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-05-20T16:04:48.886 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:04:48 smithi082 ceph-mon[32238]: mgrmap e12: x(active, since 2s) 2024-05-20T16:04:48.886 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:04:48 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:04:48.886 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:04:48 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:04:48.886 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:04:48 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:04:49.865 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$fmUjg3RGmIVizAdsbglzOu/dnLVKRK6lpSLkzG0KzWKHnczVS9N6m", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1716221089, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-05-20T16:04:49.865 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:48.742+0000 7f8278462700 1 Processor -- start 2024-05-20T16:04:49.865 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:48.742+0000 7f8278462700 1 -- start start 2024-05-20T16:04:49.865 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:48.743+0000 7f8278462700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f827010a000 0x7f827010a410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:04:49.865 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:48.743+0000 7f8278462700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f827010a9e0 con 0x7f827010a000 2024-05-20T16:04:49.865 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:48.743+0000 7f82761fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f827010a000 0x7f827010a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:04:49.865 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:48.743+0000 7f82761fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f827010a000 0x7f827010a410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:54478/0 (socket says 172.21.15.82:54478) 2024-05-20T16:04:49.865 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:48.743+0000 7f82761fe700 1 -- 172.21.15.82:0/2054258228 learned_addr learned my addr 172.21.15.82:0/2054258228 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:04:49.865 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:48.743+0000 7f82761fe700 1 -- 172.21.15.82:0/2054258228 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f827010ab60 con 0x7f827010a000 2024-05-20T16:04:49.865 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:48.743+0000 7f82761fe700 1 --2- 172.21.15.82:0/2054258228 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f827010a000 0x7f827010a410 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f826c0098a0 tx=0x7f826c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e529d008a3b533c server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:04:49.866 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:48.743+0000 7f82751fc700 1 -- 172.21.15.82:0/2054258228 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f826c004160 con 0x7f827010a000 2024-05-20T16:04:49.866 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:48.744+0000 7f82751fc700 1 -- 172.21.15.82:0/2054258228 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f826c004300 con 0x7f827010a000 2024-05-20T16:04:49.866 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:48.744+0000 7f82751fc700 1 -- 172.21.15.82:0/2054258228 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f826c011b20 con 0x7f827010a000 2024-05-20T16:04:49.866 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:48.744+0000 7f8278462700 1 -- 172.21.15.82:0/2054258228 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f827010a000 msgr2=0x7f827010a410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:04:49.866 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:48.744+0000 7f8278462700 1 --2- 172.21.15.82:0/2054258228 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f827010a000 0x7f827010a410 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f826c0098a0 tx=0x7f826c009bb0 comp rx=0 tx=0).stop 2024-05-20T16:04:49.866 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:48.745+0000 7f8278462700 1 -- 172.21.15.82:0/2054258228 shutdown_connections 2024-05-20T16:04:49.866 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:48.745+0000 7f8278462700 1 --2- 172.21.15.82:0/2054258228 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f827010a000 0x7f827010a410 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:04:49.866 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:48.745+0000 7f8278462700 1 -- 172.21.15.82:0/2054258228 >> 172.21.15.82:0/2054258228 conn(0x7f82701027a0 msgr2=0x7f8270104bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:04:49.866 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:48.745+0000 7f8278462700 1 -- 172.21.15.82:0/2054258228 shutdown_connections 2024-05-20T16:04:49.866 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:48.745+0000 7f8278462700 1 -- 172.21.15.82:0/2054258228 wait complete. 2024-05-20T16:04:49.866 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:48.745+0000 7f8278462700 1 Processor -- start 2024-05-20T16:04:49.866 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:48.745+0000 7f8278462700 1 -- start start 2024-05-20T16:04:49.866 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:48.745+0000 7f8278462700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f827010a000 0x7f827019cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:04:49.867 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:48.746+0000 7f8278462700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f827010cfa0 con 0x7f827010a000 2024-05-20T16:04:49.867 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:48.746+0000 7f82761fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f827010a000 0x7f827019cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:04:49.867 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:48.746+0000 7f82761fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f827010a000 0x7f827019cd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:54488/0 (socket says 172.21.15.82:54488) 2024-05-20T16:04:49.867 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:48.746+0000 7f82761fe700 1 -- 172.21.15.82:0/3567536248 learned_addr learned my addr 172.21.15.82:0/3567536248 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:04:49.867 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:48.746+0000 7f82761fe700 1 -- 172.21.15.82:0/3567536248 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f827019d290 con 0x7f827010a000 2024-05-20T16:04:49.867 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:48.746+0000 7f82761fe700 1 --2- 172.21.15.82:0/3567536248 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f827010a000 0x7f827019cd50 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f826c000c00 tx=0x7f826c011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:04:49.867 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:48.746+0000 7f82677fe700 1 -- 172.21.15.82:0/3567536248 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f826c003b60 con 0x7f827010a000 2024-05-20T16:04:49.867 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:48.747+0000 7f82677fe700 1 -- 172.21.15.82:0/3567536248 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f826c01a910 con 0x7f827010a000 2024-05-20T16:04:49.867 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:48.747+0000 7f8278462700 1 -- 172.21.15.82:0/3567536248 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f827019d520 con 0x7f827010a000 2024-05-20T16:04:49.867 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:48.747+0000 7f82677fe700 1 -- 172.21.15.82:0/3567536248 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f826c024930 con 0x7f827010a000 2024-05-20T16:04:49.867 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:48.747+0000 7f8278462700 1 -- 172.21.15.82:0/3567536248 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82701a0120 con 0x7f827010a000 2024-05-20T16:04:49.867 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:48.748+0000 7f82657fa700 1 -- 172.21.15.82:0/3567536248 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8258005330 con 0x7f827010a000 2024-05-20T16:04:49.867 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:48.749+0000 7f82677fe700 1 -- 172.21.15.82:0/3567536248 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 12) ==== 46300+0+0 (secure 0 0 0) 0x7f826c022020 con 0x7f827010a000 2024-05-20T16:04:49.867 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:48.749+0000 7f82677fe700 1 --2- 172.21.15.82:0/3567536248 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f825c039160 0x7f825c03b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:04:49.868 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:48.749+0000 7f82677fe700 1 -- 172.21.15.82:0/3567536248 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f826c04d840 con 0x7f827010a000 2024-05-20T16:04:49.868 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:48.749+0000 7f82759fd700 1 --2- 172.21.15.82:0/3567536248 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f825c039160 0x7f825c03b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:04:49.868 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:48.750+0000 7f82759fd700 1 --2- 172.21.15.82:0/3567536248 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f825c039160 0x7f825c03b670 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f8260006fd0 tx=0x7f8260006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:04:49.868 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:48.754+0000 7f82677fe700 1 -- 172.21.15.82:0/3567536248 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f826c02ddb0 con 0x7f827010a000 2024-05-20T16:04:49.868 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:48.977+0000 7f82657fa700 1 -- 172.21.15.82:0/3567536248 --> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7f82580059b0 con 0x7f825c039160 2024-05-20T16:04:49.868 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:49.203+0000 7f82677fe700 1 -- 172.21.15.82:0/3567536248 <== mgr.14150 v2:172.21.15.82:6800/379530823 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7f826000a040 con 0x7f825c039160 2024-05-20T16:04:49.868 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:49.205+0000 7f82657fa700 1 -- 172.21.15.82:0/3567536248 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f825c039160 msgr2=0x7f825c03b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:04:49.868 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:49.205+0000 7f82657fa700 1 --2- 172.21.15.82:0/3567536248 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f825c039160 0x7f825c03b670 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f8260006fd0 tx=0x7f8260006e40 comp rx=0 tx=0).stop 2024-05-20T16:04:49.868 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:49.205+0000 7f82657fa700 1 -- 172.21.15.82:0/3567536248 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f827010a000 msgr2=0x7f827019cd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:04:49.868 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:49.205+0000 7f82657fa700 1 --2- 172.21.15.82:0/3567536248 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f827010a000 0x7f827019cd50 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f826c000c00 tx=0x7f826c011c10 comp rx=0 tx=0).stop 2024-05-20T16:04:49.868 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:49.205+0000 7f82657fa700 1 -- 172.21.15.82:0/3567536248 shutdown_connections 2024-05-20T16:04:49.868 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:49.205+0000 7f82657fa700 1 --2- 172.21.15.82:0/3567536248 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f825c039160 0x7f825c03b670 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:04:49.868 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:49.205+0000 7f82657fa700 1 --2- 172.21.15.82:0/3567536248 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f827010a000 0x7f827019cd50 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:04:49.869 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:49.205+0000 7f82657fa700 1 -- 172.21.15.82:0/3567536248 >> 172.21.15.82:0/3567536248 conn(0x7f82701027a0 msgr2=0x7f8270102ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:04:49.869 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:49.206+0000 7f82657fa700 1 -- 172.21.15.82:0/3567536248 shutdown_connections 2024-05-20T16:04:49.869 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:49.206+0000 7f82657fa700 1 -- 172.21.15.82:0/3567536248 wait complete. 2024-05-20T16:04:49.869 INFO:teuthology.orchestra.run.smithi082.stdout:Fetching dashboard port number... 2024-05-20T16:04:50.311 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:04:50 smithi082 ceph-mon[32238]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-20T16:04:50.311 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:04:50 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:04:51.360 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:04:51 smithi082 ceph-mon[32238]: from='client.? 172.21.15.82:0/2640011788' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-05-20T16:04:51.415 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout 8443 2024-05-20T16:04:51.415 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:50.603+0000 7fd486881700 1 Processor -- start 2024-05-20T16:04:51.415 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:50.603+0000 7fd486881700 1 -- start start 2024-05-20T16:04:51.415 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:50.604+0000 7fd486881700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd480107a00 0x7fd480107e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:04:51.416 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:50.604+0000 7fd486881700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fd480108400 con 0x7fd480107a00 2024-05-20T16:04:51.416 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:50.604+0000 7fd47ffff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd480107a00 0x7fd480107e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:04:51.416 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:50.604+0000 7fd47ffff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd480107a00 0x7fd480107e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43642/0 (socket says 172.21.15.82:43642) 2024-05-20T16:04:51.416 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:50.604+0000 7fd47ffff700 1 -- 172.21.15.82:0/1124356196 learned_addr learned my addr 172.21.15.82:0/1124356196 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:04:51.416 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:50.604+0000 7fd47ffff700 1 -- 172.21.15.82:0/1124356196 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd480108b30 con 0x7fd480107a00 2024-05-20T16:04:51.416 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:50.604+0000 7fd47ffff700 1 --2- 172.21.15.82:0/1124356196 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd480107a00 0x7fd480107e10 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fd4700098a0 tx=0x7fd470009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63ae70bb941e3731 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:04:51.416 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:50.605+0000 7fd47effd700 1 -- 172.21.15.82:0/1124356196 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd470004160 con 0x7fd480107a00 2024-05-20T16:04:51.416 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:50.605+0000 7fd47effd700 1 -- 172.21.15.82:0/1124356196 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd470004300 con 0x7fd480107a00 2024-05-20T16:04:51.416 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:50.605+0000 7fd47effd700 1 -- 172.21.15.82:0/1124356196 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd470011b20 con 0x7fd480107a00 2024-05-20T16:04:51.416 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:50.605+0000 7fd486881700 1 -- 172.21.15.82:0/1124356196 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd480107a00 msgr2=0x7fd480107e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:04:51.416 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:50.605+0000 7fd486881700 1 --2- 172.21.15.82:0/1124356196 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd480107a00 0x7fd480107e10 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fd4700098a0 tx=0x7fd470009bb0 comp rx=0 tx=0).stop 2024-05-20T16:04:51.416 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:50.605+0000 7fd486881700 1 -- 172.21.15.82:0/1124356196 shutdown_connections 2024-05-20T16:04:51.416 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:50.605+0000 7fd486881700 1 --2- 172.21.15.82:0/1124356196 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd480107a00 0x7fd480107e10 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:04:51.416 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:50.605+0000 7fd486881700 1 -- 172.21.15.82:0/1124356196 >> 172.21.15.82:0/1124356196 conn(0x7fd480103060 msgr2=0x7fd4801054f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:04:51.417 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:50.606+0000 7fd486881700 1 -- 172.21.15.82:0/1124356196 shutdown_connections 2024-05-20T16:04:51.417 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:50.606+0000 7fd486881700 1 -- 172.21.15.82:0/1124356196 wait complete. 2024-05-20T16:04:51.417 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:50.606+0000 7fd486881700 1 Processor -- start 2024-05-20T16:04:51.417 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:50.606+0000 7fd486881700 1 -- start start 2024-05-20T16:04:51.417 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:50.607+0000 7fd486881700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd480107a00 0x7fd48019bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:04:51.417 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:50.607+0000 7fd486881700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fd48010bee0 con 0x7fd480107a00 2024-05-20T16:04:51.417 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:50.607+0000 7fd47ffff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd480107a00 0x7fd48019bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:04:51.417 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:50.607+0000 7fd47ffff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd480107a00 0x7fd48019bd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43656/0 (socket says 172.21.15.82:43656) 2024-05-20T16:04:51.417 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:50.607+0000 7fd47ffff700 1 -- 172.21.15.82:0/2640011788 learned_addr learned my addr 172.21.15.82:0/2640011788 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:04:51.417 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:50.607+0000 7fd47ffff700 1 -- 172.21.15.82:0/2640011788 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd48019c290 con 0x7fd480107a00 2024-05-20T16:04:51.417 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:50.607+0000 7fd47ffff700 1 --2- 172.21.15.82:0/2640011788 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd480107a00 0x7fd48019bd50 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fd470009fb0 tx=0x7fd470011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:04:51.417 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:50.607+0000 7fd47d7fa700 1 -- 172.21.15.82:0/2640011788 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd470003930 con 0x7fd480107a00 2024-05-20T16:04:51.417 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:50.607+0000 7fd47d7fa700 1 -- 172.21.15.82:0/2640011788 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd470003ad0 con 0x7fd480107a00 2024-05-20T16:04:51.417 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:50.607+0000 7fd486881700 1 -- 172.21.15.82:0/2640011788 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd48019c520 con 0x7fd480107a00 2024-05-20T16:04:51.417 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:50.607+0000 7fd47d7fa700 1 -- 172.21.15.82:0/2640011788 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd470019670 con 0x7fd480107a00 2024-05-20T16:04:51.418 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:50.607+0000 7fd486881700 1 -- 172.21.15.82:0/2640011788 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd48019f120 con 0x7fd480107a00 2024-05-20T16:04:51.418 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:50.608+0000 7fd47d7fa700 1 -- 172.21.15.82:0/2640011788 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 12) ==== 46300+0+0 (secure 0 0 0) 0x7fd470019970 con 0x7fd480107a00 2024-05-20T16:04:51.418 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:50.608+0000 7fd486881700 1 -- 172.21.15.82:0/2640011788 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd480063ca0 con 0x7fd480107a00 2024-05-20T16:04:51.418 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:50.608+0000 7fd47d7fa700 1 --2- 172.21.15.82:0/2640011788 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fd468039110 0x7fd46803b620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:04:51.418 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:50.609+0000 7fd47d7fa700 1 -- 172.21.15.82:0/2640011788 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fd47004d760 con 0x7fd480107a00 2024-05-20T16:04:51.418 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:50.609+0000 7fd47f7fe700 1 --2- 172.21.15.82:0/2640011788 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fd468039110 0x7fd46803b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:04:51.418 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:50.609+0000 7fd47f7fe700 1 --2- 172.21.15.82:0/2640011788 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fd468039110 0x7fd46803b620 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fd474006fd0 tx=0x7fd474006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:04:51.418 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:50.614+0000 7fd47d7fa700 1 -- 172.21.15.82:0/2640011788 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fd470011e20 con 0x7fd480107a00 2024-05-20T16:04:51.418 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:50.811+0000 7fd486881700 1 -- 172.21.15.82:0/2640011788 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7fd48019f2f0 con 0x7fd480107a00 2024-05-20T16:04:51.418 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:50.812+0000 7fd47d7fa700 1 -- 172.21.15.82:0/2640011788 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7fd470019d60 con 0x7fd480107a00 2024-05-20T16:04:51.418 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:50.814+0000 7fd486881700 1 -- 172.21.15.82:0/2640011788 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fd468039110 msgr2=0x7fd46803b620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:04:51.418 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:50.814+0000 7fd486881700 1 --2- 172.21.15.82:0/2640011788 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fd468039110 0x7fd46803b620 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fd474006fd0 tx=0x7fd474006e40 comp rx=0 tx=0).stop 2024-05-20T16:04:51.418 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:50.814+0000 7fd486881700 1 -- 172.21.15.82:0/2640011788 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd480107a00 msgr2=0x7fd48019bd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:04:51.419 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:50.814+0000 7fd486881700 1 --2- 172.21.15.82:0/2640011788 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd480107a00 0x7fd48019bd50 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fd470009fb0 tx=0x7fd470011c10 comp rx=0 tx=0).stop 2024-05-20T16:04:51.419 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:50.814+0000 7fd486881700 1 -- 172.21.15.82:0/2640011788 shutdown_connections 2024-05-20T16:04:51.419 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:50.814+0000 7fd486881700 1 --2- 172.21.15.82:0/2640011788 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fd468039110 0x7fd46803b620 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:04:51.419 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:50.814+0000 7fd486881700 1 --2- 172.21.15.82:0/2640011788 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd480107a00 0x7fd48019bd50 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:04:51.419 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:50.814+0000 7fd486881700 1 -- 172.21.15.82:0/2640011788 >> 172.21.15.82:0/2640011788 conn(0x7fd480103060 msgr2=0x7fd480103850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:04:51.419 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:50.814+0000 7fd486881700 1 -- 172.21.15.82:0/2640011788 shutdown_connections 2024-05-20T16:04:51.419 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:50.814+0000 7fd486881700 1 -- 172.21.15.82:0/2640011788 wait complete. 2024-05-20T16:04:51.422 INFO:teuthology.orchestra.run.smithi082.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-20T16:04:51.423 INFO:teuthology.orchestra.run.smithi082.stdout:systemctl: stdout disabled 2024-05-20T16:04:51.430 INFO:teuthology.orchestra.run.smithi082.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-20T16:04:51.430 INFO:teuthology.orchestra.run.smithi082.stdout:systemctl: stdout inactive 2024-05-20T16:04:51.430 INFO:teuthology.orchestra.run.smithi082.stdout:firewalld.service is not enabled 2024-05-20T16:04:51.430 INFO:teuthology.orchestra.run.smithi082.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-05-20T16:04:51.431 INFO:teuthology.orchestra.run.smithi082.stdout:Ceph Dashboard is now available at: 2024-05-20T16:04:51.432 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-05-20T16:04:51.432 INFO:teuthology.orchestra.run.smithi082.stdout: URL: https://smithi082.front.sepia.ceph.com:8443/ 2024-05-20T16:04:51.432 INFO:teuthology.orchestra.run.smithi082.stdout: User: admin 2024-05-20T16:04:51.432 INFO:teuthology.orchestra.run.smithi082.stdout: Password: r8ihmeuu4q 2024-05-20T16:04:51.432 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-05-20T16:04:51.432 INFO:teuthology.orchestra.run.smithi082.stdout:Saving cluster configuration to /var/lib/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/config directory 2024-05-20T16:04:51.433 INFO:teuthology.orchestra.run.smithi082.stdout:Enabling autotune for osd_memory_target 2024-05-20T16:04:52.429 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:04:52 smithi082 ceph-mon[32238]: mgrmap e13: x(active, since 6s) 2024-05-20T16:04:52.664 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:51.913+0000 7f55af572700 1 Processor -- start 2024-05-20T16:04:52.664 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:51.913+0000 7f55af572700 1 -- start start 2024-05-20T16:04:52.664 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:51.914+0000 7f55af572700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f55a807bcc0 0x7f55a810b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:04:52.664 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:51.914+0000 7f55af572700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f55a810bbd0 con 0x7f55a807bcc0 2024-05-20T16:04:52.664 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:51.914+0000 7f55ad30e700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f55a807bcc0 0x7f55a810b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:04:52.664 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:51.914+0000 7f55ad30e700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f55a807bcc0 0x7f55a810b5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43660/0 (socket says 172.21.15.82:43660) 2024-05-20T16:04:52.664 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:51.914+0000 7f55ad30e700 1 -- 172.21.15.82:0/1612994548 learned_addr learned my addr 172.21.15.82:0/1612994548 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:04:52.664 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:51.914+0000 7f55ad30e700 1 -- 172.21.15.82:0/1612994548 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55a810bd50 con 0x7f55a807bcc0 2024-05-20T16:04:52.664 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:51.914+0000 7f55ad30e700 1 --2- 172.21.15.82:0/1612994548 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f55a807bcc0 0x7f55a810b5c0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f55a4009e80 tx=0x7f55a400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77e8de745327ae6b server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:04:52.664 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:51.914+0000 7f559ffff700 1 -- 172.21.15.82:0/1612994548 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f55a4017070 con 0x7f55a807bcc0 2024-05-20T16:04:52.664 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:51.914+0000 7f559ffff700 1 -- 172.21.15.82:0/1612994548 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f55a400b8c0 con 0x7f55a807bcc0 2024-05-20T16:04:52.665 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:51.914+0000 7f559ffff700 1 -- 172.21.15.82:0/1612994548 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f55a40045c0 con 0x7f55a807bcc0 2024-05-20T16:04:52.665 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:51.915+0000 7f55af572700 1 -- 172.21.15.82:0/1612994548 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f55a807bcc0 msgr2=0x7f55a810b5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:04:52.665 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:51.915+0000 7f55af572700 1 --2- 172.21.15.82:0/1612994548 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f55a807bcc0 0x7f55a810b5c0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f55a4009e80 tx=0x7f55a400b330 comp rx=0 tx=0).stop 2024-05-20T16:04:52.665 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:51.915+0000 7f55af572700 1 -- 172.21.15.82:0/1612994548 shutdown_connections 2024-05-20T16:04:52.665 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:51.915+0000 7f55af572700 1 --2- 172.21.15.82:0/1612994548 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f55a807bcc0 0x7f55a810b5c0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:04:52.665 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:51.915+0000 7f55af572700 1 -- 172.21.15.82:0/1612994548 >> 172.21.15.82:0/1612994548 conn(0x7f55a8103070 msgr2=0x7f55a81054c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:04:52.665 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:51.915+0000 7f55af572700 1 -- 172.21.15.82:0/1612994548 shutdown_connections 2024-05-20T16:04:52.665 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:51.915+0000 7f55af572700 1 -- 172.21.15.82:0/1612994548 wait complete. 2024-05-20T16:04:52.665 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:51.916+0000 7f55af572700 1 Processor -- start 2024-05-20T16:04:52.665 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:51.916+0000 7f55af572700 1 -- start start 2024-05-20T16:04:52.665 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:51.916+0000 7f55af572700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f55a807bcc0 0x7f55a819cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:04:52.665 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:51.916+0000 7f55af572700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f55a810ce70 con 0x7f55a807bcc0 2024-05-20T16:04:52.666 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:51.916+0000 7f55ad30e700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f55a807bcc0 0x7f55a819cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:04:52.666 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:51.916+0000 7f55ad30e700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f55a807bcc0 0x7f55a819cd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43670/0 (socket says 172.21.15.82:43670) 2024-05-20T16:04:52.666 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:51.916+0000 7f55ad30e700 1 -- 172.21.15.82:0/1351113711 learned_addr learned my addr 172.21.15.82:0/1351113711 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:04:52.666 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:51.917+0000 7f55ad30e700 1 -- 172.21.15.82:0/1351113711 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55a819d2c0 con 0x7f55a807bcc0 2024-05-20T16:04:52.666 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:51.917+0000 7f55ad30e700 1 --2- 172.21.15.82:0/1351113711 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f55a807bcc0 0x7f55a819cd80 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f55a4003fa0 tx=0x7f55a4004080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:04:52.666 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:51.917+0000 7f559e7fc700 1 -- 172.21.15.82:0/1351113711 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f55a4020020 con 0x7f55a807bcc0 2024-05-20T16:04:52.666 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:51.917+0000 7f559e7fc700 1 -- 172.21.15.82:0/1351113711 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f55a400bba0 con 0x7f55a807bcc0 2024-05-20T16:04:52.666 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:51.917+0000 7f55af572700 1 -- 172.21.15.82:0/1351113711 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55a819d550 con 0x7f55a807bcc0 2024-05-20T16:04:52.666 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:51.917+0000 7f559e7fc700 1 -- 172.21.15.82:0/1351113711 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f55a4017040 con 0x7f55a807bcc0 2024-05-20T16:04:52.666 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:51.917+0000 7f55af572700 1 -- 172.21.15.82:0/1351113711 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55a81a0160 con 0x7f55a807bcc0 2024-05-20T16:04:52.666 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:51.918+0000 7f55af572700 1 -- 172.21.15.82:0/1351113711 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55a8063ca0 con 0x7f55a807bcc0 2024-05-20T16:04:52.666 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:51.918+0000 7f559e7fc700 1 -- 172.21.15.82:0/1351113711 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f55a4023020 con 0x7f55a807bcc0 2024-05-20T16:04:52.666 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:51.918+0000 7f559e7fc700 1 --2- 172.21.15.82:0/1351113711 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f55940391b0 0x7f559403b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:04:52.667 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:51.918+0000 7f559e7fc700 1 -- 172.21.15.82:0/1351113711 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f55a400f050 con 0x7f55a807bcc0 2024-05-20T16:04:52.667 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:51.918+0000 7f55acb0d700 1 --2- 172.21.15.82:0/1351113711 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f55940391b0 0x7f559403b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:04:52.667 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:51.919+0000 7f55acb0d700 1 --2- 172.21.15.82:0/1351113711 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f55940391b0 0x7f559403b6c0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f5598006fd0 tx=0x7f5598006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:04:52.667 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:51.924+0000 7f559e7fc700 1 -- 172.21.15.82:0/1351113711 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f55a402a020 con 0x7f55a807bcc0 2024-05-20T16:04:52.667 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:52.120+0000 7f55af572700 1 -- 172.21.15.82:0/1351113711 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) -- 0x7f55a81a0380 con 0x7f55a807bcc0 2024-05-20T16:04:52.667 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:52.120+0000 7f559e7fc700 1 -- 172.21.15.82:0/1351113711 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) ==== 127+0+0 (secure 0 0 0) 0x7f55a401bde0 con 0x7f55a807bcc0 2024-05-20T16:04:52.667 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:52.122+0000 7f55af572700 1 -- 172.21.15.82:0/1351113711 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f55940391b0 msgr2=0x7f559403b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:04:52.667 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:52.122+0000 7f55af572700 1 --2- 172.21.15.82:0/1351113711 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f55940391b0 0x7f559403b6c0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f5598006fd0 tx=0x7f5598006e40 comp rx=0 tx=0).stop 2024-05-20T16:04:52.667 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:52.122+0000 7f55af572700 1 -- 172.21.15.82:0/1351113711 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f55a807bcc0 msgr2=0x7f55a819cd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:04:52.667 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:52.122+0000 7f55af572700 1 --2- 172.21.15.82:0/1351113711 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f55a807bcc0 0x7f55a819cd80 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f55a4003fa0 tx=0x7f55a4004080 comp rx=0 tx=0).stop 2024-05-20T16:04:52.667 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:52.122+0000 7f55af572700 1 -- 172.21.15.82:0/1351113711 shutdown_connections 2024-05-20T16:04:52.667 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:52.122+0000 7f55af572700 1 --2- 172.21.15.82:0/1351113711 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f55940391b0 0x7f559403b6c0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:04:52.668 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:52.122+0000 7f55af572700 1 --2- 172.21.15.82:0/1351113711 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f55a807bcc0 0x7f55a819cd80 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:04:52.668 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:52.122+0000 7f55af572700 1 -- 172.21.15.82:0/1351113711 >> 172.21.15.82:0/1351113711 conn(0x7f55a8103070 msgr2=0x7f55a81048e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:04:52.668 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:52.122+0000 7f55af572700 1 -- 172.21.15.82:0/1351113711 shutdown_connections 2024-05-20T16:04:52.668 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:52.122+0000 7f55af572700 1 -- 172.21.15.82:0/1351113711 wait complete. 2024-05-20T16:04:53.979 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:53.091+0000 7efe98fc6700 1 Processor -- start 2024-05-20T16:04:54.046 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:53.091+0000 7efe98fc6700 1 -- start start 2024-05-20T16:04:54.046 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:53.091+0000 7efe98fc6700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efe9407c550 0x7efe9410b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:04:54.046 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:53.091+0000 7efe98fc6700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7efe9410bbe0 con 0x7efe9407c550 2024-05-20T16:04:54.046 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:53.092+0000 7efe9259c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efe9407c550 0x7efe9410b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:04:54.046 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:53.092+0000 7efe9259c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efe9407c550 0x7efe9410b5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43686/0 (socket says 172.21.15.82:43686) 2024-05-20T16:04:54.046 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:53.092+0000 7efe9259c700 1 -- 172.21.15.82:0/748535679 learned_addr learned my addr 172.21.15.82:0/748535679 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:04:54.046 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:53.092+0000 7efe9259c700 1 -- 172.21.15.82:0/748535679 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe9410bd60 con 0x7efe9407c550 2024-05-20T16:04:54.047 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:53.092+0000 7efe9259c700 1 --2- 172.21.15.82:0/748535679 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efe9407c550 0x7efe9410b5d0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7efe840098a0 tx=0x7efe84009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c049422a120bf3de server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:04:54.047 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:53.092+0000 7efe91d9b700 1 -- 172.21.15.82:0/748535679 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efe84004160 con 0x7efe9407c550 2024-05-20T16:04:54.047 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:53.092+0000 7efe91d9b700 1 -- 172.21.15.82:0/748535679 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe84004300 con 0x7efe9407c550 2024-05-20T16:04:54.047 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:53.092+0000 7efe91d9b700 1 -- 172.21.15.82:0/748535679 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efe84011b20 con 0x7efe9407c550 2024-05-20T16:04:54.047 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:53.093+0000 7efe98fc6700 1 -- 172.21.15.82:0/748535679 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efe9407c550 msgr2=0x7efe9410b5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:04:54.047 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:53.093+0000 7efe98fc6700 1 --2- 172.21.15.82:0/748535679 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efe9407c550 0x7efe9410b5d0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7efe840098a0 tx=0x7efe84009bb0 comp rx=0 tx=0).stop 2024-05-20T16:04:54.047 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:53.093+0000 7efe98fc6700 1 -- 172.21.15.82:0/748535679 shutdown_connections 2024-05-20T16:04:54.047 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:53.093+0000 7efe98fc6700 1 --2- 172.21.15.82:0/748535679 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efe9407c550 0x7efe9410b5d0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:04:54.047 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:53.093+0000 7efe98fc6700 1 -- 172.21.15.82:0/748535679 >> 172.21.15.82:0/748535679 conn(0x7efe941030a0 msgr2=0x7efe941054f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:04:54.047 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:53.093+0000 7efe98fc6700 1 -- 172.21.15.82:0/748535679 shutdown_connections 2024-05-20T16:04:54.047 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:53.093+0000 7efe98fc6700 1 -- 172.21.15.82:0/748535679 wait complete. 2024-05-20T16:04:54.048 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:53.094+0000 7efe98fc6700 1 Processor -- start 2024-05-20T16:04:54.048 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:53.094+0000 7efe98fc6700 1 -- start start 2024-05-20T16:04:54.048 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:53.094+0000 7efe98fc6700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efe9407c550 0x7efe94108d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:04:54.048 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:53.094+0000 7efe98fc6700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7efe9410ce80 con 0x7efe9407c550 2024-05-20T16:04:54.048 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:53.094+0000 7efe9259c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efe9407c550 0x7efe94108d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:04:54.048 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:53.094+0000 7efe9259c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efe9407c550 0x7efe94108d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43694/0 (socket says 172.21.15.82:43694) 2024-05-20T16:04:54.048 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:53.094+0000 7efe9259c700 1 -- 172.21.15.82:0/321408834 learned_addr learned my addr 172.21.15.82:0/321408834 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:04:54.048 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:53.095+0000 7efe9259c700 1 -- 172.21.15.82:0/321408834 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe94109280 con 0x7efe9407c550 2024-05-20T16:04:54.048 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:53.095+0000 7efe9259c700 1 --2- 172.21.15.82:0/321408834 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efe9407c550 0x7efe94108d40 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7efe84000c00 tx=0x7efe84011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:04:54.048 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:53.095+0000 7efe8b7fe700 1 -- 172.21.15.82:0/321408834 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efe84003b60 con 0x7efe9407c550 2024-05-20T16:04:54.048 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:53.095+0000 7efe98fc6700 1 -- 172.21.15.82:0/321408834 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe941074d0 con 0x7efe9407c550 2024-05-20T16:04:54.049 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:53.095+0000 7efe8b7fe700 1 -- 172.21.15.82:0/321408834 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe8401a910 con 0x7efe9407c550 2024-05-20T16:04:54.049 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:53.095+0000 7efe8b7fe700 1 -- 172.21.15.82:0/321408834 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efe84024930 con 0x7efe9407c550 2024-05-20T16:04:54.049 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:53.095+0000 7efe98fc6700 1 -- 172.21.15.82:0/321408834 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe94107930 con 0x7efe9407c550 2024-05-20T16:04:54.049 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:53.096+0000 7efe98fc6700 1 -- 172.21.15.82:0/321408834 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe94063ca0 con 0x7efe9407c550 2024-05-20T16:04:54.049 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:53.096+0000 7efe8b7fe700 1 -- 172.21.15.82:0/321408834 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7efe84022020 con 0x7efe9407c550 2024-05-20T16:04:54.049 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:53.096+0000 7efe8b7fe700 1 --2- 172.21.15.82:0/321408834 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7efe80039200 0x7efe8003b710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:04:54.049 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:53.096+0000 7efe8b7fe700 1 -- 172.21.15.82:0/321408834 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7efe8404d4b0 con 0x7efe9407c550 2024-05-20T16:04:54.049 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:53.096+0000 7efe8bfff700 1 --2- 172.21.15.82:0/321408834 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7efe80039200 0x7efe8003b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:04:54.049 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:53.097+0000 7efe8bfff700 1 --2- 172.21.15.82:0/321408834 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7efe80039200 0x7efe8003b710 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7efe7c006fd0 tx=0x7efe7c006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:04:54.049 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:53.101+0000 7efe8b7fe700 1 -- 172.21.15.82:0/321408834 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7efe8400b3e0 con 0x7efe9407c550 2024-05-20T16:04:54.049 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:53.408+0000 7efe98fc6700 1 -- 172.21.15.82:0/321408834 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7efe94108020 con 0x7efe9407c550 2024-05-20T16:04:54.050 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:53.411+0000 7efe8b7fe700 1 -- 172.21.15.82:0/321408834 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) ==== 153+0+0 (secure 0 0 0) 0x7efe840172b0 con 0x7efe9407c550 2024-05-20T16:04:54.050 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:53.413+0000 7efe98fc6700 1 -- 172.21.15.82:0/321408834 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7efe80039200 msgr2=0x7efe8003b710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:04:54.050 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:53.413+0000 7efe98fc6700 1 --2- 172.21.15.82:0/321408834 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7efe80039200 0x7efe8003b710 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7efe7c006fd0 tx=0x7efe7c006e40 comp rx=0 tx=0).stop 2024-05-20T16:04:54.050 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:53.413+0000 7efe98fc6700 1 -- 172.21.15.82:0/321408834 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efe9407c550 msgr2=0x7efe94108d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:04:54.050 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:53.413+0000 7efe98fc6700 1 --2- 172.21.15.82:0/321408834 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efe9407c550 0x7efe94108d40 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7efe84000c00 tx=0x7efe84011c10 comp rx=0 tx=0).stop 2024-05-20T16:04:54.050 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:53.413+0000 7efe98fc6700 1 -- 172.21.15.82:0/321408834 shutdown_connections 2024-05-20T16:04:54.050 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:53.413+0000 7efe98fc6700 1 --2- 172.21.15.82:0/321408834 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7efe80039200 0x7efe8003b710 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:04:54.050 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:53.413+0000 7efe98fc6700 1 --2- 172.21.15.82:0/321408834 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efe9407c550 0x7efe94108d40 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:04:54.050 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:53.413+0000 7efe98fc6700 1 -- 172.21.15.82:0/321408834 >> 172.21.15.82:0/321408834 conn(0x7efe941030a0 msgr2=0x7efe94104960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:04:54.050 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:53.414+0000 7efe98fc6700 1 -- 172.21.15.82:0/321408834 shutdown_connections 2024-05-20T16:04:54.050 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-05-20T16:04:53.414+0000 7efe98fc6700 1 -- 172.21.15.82:0/321408834 wait complete. 2024-05-20T16:04:54.051 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-05-20T16:04:54.051 INFO:teuthology.orchestra.run.smithi082.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-05-20T16:04:54.051 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-05-20T16:04:54.051 INFO:teuthology.orchestra.run.smithi082.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 59d1600a-16c2-11ef-bc9a-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-05-20T16:04:54.051 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-05-20T16:04:54.051 INFO:teuthology.orchestra.run.smithi082.stdout:Or, if you are only running a single cluster on this host: 2024-05-20T16:04:54.051 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-05-20T16:04:54.051 INFO:teuthology.orchestra.run.smithi082.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-05-20T16:04:54.051 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-05-20T16:04:54.051 INFO:teuthology.orchestra.run.smithi082.stdout:Please consider enabling telemetry to help improve Ceph: 2024-05-20T16:04:54.051 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-05-20T16:04:54.051 INFO:teuthology.orchestra.run.smithi082.stdout: ceph telemetry on 2024-05-20T16:04:54.052 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-05-20T16:04:54.052 INFO:teuthology.orchestra.run.smithi082.stdout:For more information see: 2024-05-20T16:04:54.052 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-05-20T16:04:54.052 INFO:teuthology.orchestra.run.smithi082.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-05-20T16:04:54.052 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-05-20T16:04:54.052 INFO:teuthology.orchestra.run.smithi082.stdout:Bootstrap complete. 2024-05-20T16:04:54.053 INFO:tasks.cephadm:Fetching config... 2024-05-20T16:04:54.053 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-05-20T16:04:54.054 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-05-20T16:04:54.070 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-05-20T16:04:54.070 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-05-20T16:04:54.070 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-05-20T16:04:54.126 INFO:tasks.cephadm:Fetching mon keyring... 2024-05-20T16:04:54.126 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-05-20T16:04:54.126 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd if=/var/lib/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/mon.a/keyring of=/dev/stdout 2024-05-20T16:04:54.192 INFO:tasks.cephadm:Fetching pub ssh key... 2024-05-20T16:04:54.192 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-05-20T16:04:54.192 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-05-20T16:04:54.247 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-05-20T16:04:54.248 DEBUG:teuthology.orchestra.run.smithi082:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDGtQRYMOcyJ3OXiToYCqyh3ktsCjbrNxDoRP0B8FURq58NsWXQ1pNHE8bJk8h5sLQoImUlF8T/46ONtqlUa9ozxChtzxPjwfHcfmH2CTJLMAkV6DwN55FRtP3aVUEQ0ikGJMU16LDpMkVErBTvHS/5kTszwW75VQ6eZTeBs9fLoJ92ahxwJo898SLMMWzRsDKJZ8REkoJzocBQaztta/wDxmN34n4k+dUPYvvDmseaMD8j5RwL9JvCafs9H7W+7ho8vajoywUv7mZi4IgHUNEXQUe0uGEILbnpT16d4au3exIL4hRVUVJdAaaunQNDos6YygHfN/tY/8R+ARcbKLoT2vfeRXZlGcR0rPTUp11CUpN63Bpt6dgsusqDrBD3oXq/XQ2gIW2OgoWrgvicAhLsj1pb3i1L85kVRsUQcg+wTq3PFykRdLepp0tSGW37yoZn+fZnuWoosOvesOCV/Q9cRJKl/wgg0DkOJK4TBG7/sJ2XRrx384iTsr4/LUmzdDM= ceph-59d1600a-16c2-11ef-bc9a-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-05-20T16:04:54.321 INFO:teuthology.orchestra.run.smithi082.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDGtQRYMOcyJ3OXiToYCqyh3ktsCjbrNxDoRP0B8FURq58NsWXQ1pNHE8bJk8h5sLQoImUlF8T/46ONtqlUa9ozxChtzxPjwfHcfmH2CTJLMAkV6DwN55FRtP3aVUEQ0ikGJMU16LDpMkVErBTvHS/5kTszwW75VQ6eZTeBs9fLoJ92ahxwJo898SLMMWzRsDKJZ8REkoJzocBQaztta/wDxmN34n4k+dUPYvvDmseaMD8j5RwL9JvCafs9H7W+7ho8vajoywUv7mZi4IgHUNEXQUe0uGEILbnpT16d4au3exIL4hRVUVJdAaaunQNDos6YygHfN/tY/8R+ARcbKLoT2vfeRXZlGcR0rPTUp11CUpN63Bpt6dgsusqDrBD3oXq/XQ2gIW2OgoWrgvicAhLsj1pb3i1L85kVRsUQcg+wTq3PFykRdLepp0tSGW37yoZn+fZnuWoosOvesOCV/Q9cRJKl/wgg0DkOJK4TBG7/sJ2XRrx384iTsr4/LUmzdDM= ceph-59d1600a-16c2-11ef-bc9a-c7b262605968 2024-05-20T16:04:54.333 DEBUG:teuthology.orchestra.run.smithi160:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDGtQRYMOcyJ3OXiToYCqyh3ktsCjbrNxDoRP0B8FURq58NsWXQ1pNHE8bJk8h5sLQoImUlF8T/46ONtqlUa9ozxChtzxPjwfHcfmH2CTJLMAkV6DwN55FRtP3aVUEQ0ikGJMU16LDpMkVErBTvHS/5kTszwW75VQ6eZTeBs9fLoJ92ahxwJo898SLMMWzRsDKJZ8REkoJzocBQaztta/wDxmN34n4k+dUPYvvDmseaMD8j5RwL9JvCafs9H7W+7ho8vajoywUv7mZi4IgHUNEXQUe0uGEILbnpT16d4au3exIL4hRVUVJdAaaunQNDos6YygHfN/tY/8R+ARcbKLoT2vfeRXZlGcR0rPTUp11CUpN63Bpt6dgsusqDrBD3oXq/XQ2gIW2OgoWrgvicAhLsj1pb3i1L85kVRsUQcg+wTq3PFykRdLepp0tSGW37yoZn+fZnuWoosOvesOCV/Q9cRJKl/wgg0DkOJK4TBG7/sJ2XRrx384iTsr4/LUmzdDM= ceph-59d1600a-16c2-11ef-bc9a-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-05-20T16:04:54.374 INFO:teuthology.orchestra.run.smithi160.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDGtQRYMOcyJ3OXiToYCqyh3ktsCjbrNxDoRP0B8FURq58NsWXQ1pNHE8bJk8h5sLQoImUlF8T/46ONtqlUa9ozxChtzxPjwfHcfmH2CTJLMAkV6DwN55FRtP3aVUEQ0ikGJMU16LDpMkVErBTvHS/5kTszwW75VQ6eZTeBs9fLoJ92ahxwJo898SLMMWzRsDKJZ8REkoJzocBQaztta/wDxmN34n4k+dUPYvvDmseaMD8j5RwL9JvCafs9H7W+7ho8vajoywUv7mZi4IgHUNEXQUe0uGEILbnpT16d4au3exIL4hRVUVJdAaaunQNDos6YygHfN/tY/8R+ARcbKLoT2vfeRXZlGcR0rPTUp11CUpN63Bpt6dgsusqDrBD3oXq/XQ2gIW2OgoWrgvicAhLsj1pb3i1L85kVRsUQcg+wTq3PFykRdLepp0tSGW37yoZn+fZnuWoosOvesOCV/Q9cRJKl/wgg0DkOJK4TBG7/sJ2XRrx384iTsr4/LUmzdDM= ceph-59d1600a-16c2-11ef-bc9a-c7b262605968 2024-05-20T16:04:54.386 DEBUG:teuthology.orchestra.run.smithi184:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDGtQRYMOcyJ3OXiToYCqyh3ktsCjbrNxDoRP0B8FURq58NsWXQ1pNHE8bJk8h5sLQoImUlF8T/46ONtqlUa9ozxChtzxPjwfHcfmH2CTJLMAkV6DwN55FRtP3aVUEQ0ikGJMU16LDpMkVErBTvHS/5kTszwW75VQ6eZTeBs9fLoJ92ahxwJo898SLMMWzRsDKJZ8REkoJzocBQaztta/wDxmN34n4k+dUPYvvDmseaMD8j5RwL9JvCafs9H7W+7ho8vajoywUv7mZi4IgHUNEXQUe0uGEILbnpT16d4au3exIL4hRVUVJdAaaunQNDos6YygHfN/tY/8R+ARcbKLoT2vfeRXZlGcR0rPTUp11CUpN63Bpt6dgsusqDrBD3oXq/XQ2gIW2OgoWrgvicAhLsj1pb3i1L85kVRsUQcg+wTq3PFykRdLepp0tSGW37yoZn+fZnuWoosOvesOCV/Q9cRJKl/wgg0DkOJK4TBG7/sJ2XRrx384iTsr4/LUmzdDM= ceph-59d1600a-16c2-11ef-bc9a-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-05-20T16:04:54.425 INFO:teuthology.orchestra.run.smithi184.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDGtQRYMOcyJ3OXiToYCqyh3ktsCjbrNxDoRP0B8FURq58NsWXQ1pNHE8bJk8h5sLQoImUlF8T/46ONtqlUa9ozxChtzxPjwfHcfmH2CTJLMAkV6DwN55FRtP3aVUEQ0ikGJMU16LDpMkVErBTvHS/5kTszwW75VQ6eZTeBs9fLoJ92ahxwJo898SLMMWzRsDKJZ8REkoJzocBQaztta/wDxmN34n4k+dUPYvvDmseaMD8j5RwL9JvCafs9H7W+7ho8vajoywUv7mZi4IgHUNEXQUe0uGEILbnpT16d4au3exIL4hRVUVJdAaaunQNDos6YygHfN/tY/8R+ARcbKLoT2vfeRXZlGcR0rPTUp11CUpN63Bpt6dgsusqDrBD3oXq/XQ2gIW2OgoWrgvicAhLsj1pb3i1L85kVRsUQcg+wTq3PFykRdLepp0tSGW37yoZn+fZnuWoosOvesOCV/Q9cRJKl/wgg0DkOJK4TBG7/sJ2XRrx384iTsr4/LUmzdDM= ceph-59d1600a-16c2-11ef-bc9a-c7b262605968 2024-05-20T16:04:54.437 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 59d1600a-16c2-11ef-bc9a-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-05-20T16:04:54.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:04:54 smithi082 ceph-mon[32238]: from='client.? 172.21.15.82:0/321408834' entity='client.admin' 2024-05-20T16:04:54.681 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T16:04:56.886 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:04:56.885+0000 7fb2c09cc700 1 -- 172.21.15.82:0/2872932967 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb2bc101f20 msgr2=0x7fb2bc102330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:04:56.886 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:04:56.885+0000 7fb2c09cc700 1 --2- 172.21.15.82:0/2872932967 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb2bc101f20 0x7fb2bc102330 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fb2b0009910 tx=0x7fb2b0009c20 comp rx=0 tx=0).stop 2024-05-20T16:04:56.886 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:04:56.885+0000 7fb2c09cc700 1 -- 172.21.15.82:0/2872932967 shutdown_connections 2024-05-20T16:04:56.886 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:04:56.885+0000 7fb2c09cc700 1 --2- 172.21.15.82:0/2872932967 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb2bc101f20 0x7fb2bc102330 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:04:56.886 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:04:56.885+0000 7fb2c09cc700 1 -- 172.21.15.82:0/2872932967 >> 172.21.15.82:0/2872932967 conn(0x7fb2bc0fc710 msgr2=0x7fb2bc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:04:56.886 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:04:56.885+0000 7fb2c09cc700 1 -- 172.21.15.82:0/2872932967 shutdown_connections 2024-05-20T16:04:56.886 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:04:56.885+0000 7fb2c09cc700 1 -- 172.21.15.82:0/2872932967 wait complete. 2024-05-20T16:04:56.887 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:04:56.886+0000 7fb2c09cc700 1 Processor -- start 2024-05-20T16:04:56.887 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:04:56.886+0000 7fb2c09cc700 1 -- start start 2024-05-20T16:04:56.887 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:04:56.886+0000 7fb2c09cc700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb2bc101f20 0x7fb2bc198650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:04:56.888 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:04:56.886+0000 7fb2c09cc700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fb2bc107a60 con 0x7fb2bc101f20 2024-05-20T16:04:56.888 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:04:56.886+0000 7fb2ba59c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb2bc101f20 0x7fb2bc198650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:04:56.888 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:04:56.886+0000 7fb2ba59c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb2bc101f20 0x7fb2bc198650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43718/0 (socket says 172.21.15.82:43718) 2024-05-20T16:04:56.888 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:04:56.886+0000 7fb2ba59c700 1 -- 172.21.15.82:0/1376828723 learned_addr learned my addr 172.21.15.82:0/1376828723 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:04:56.888 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:04:56.886+0000 7fb2ba59c700 1 -- 172.21.15.82:0/1376828723 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2bc198b90 con 0x7fb2bc101f20 2024-05-20T16:04:56.888 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:04:56.886+0000 7fb2ba59c700 1 --2- 172.21.15.82:0/1376828723 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb2bc101f20 0x7fb2bc198650 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fb2b000b5c0 tx=0x7fb2b0005050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:04:56.888 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:04:56.887+0000 7fb2ab7fe700 1 -- 172.21.15.82:0/1376828723 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb2b0027070 con 0x7fb2bc101f20 2024-05-20T16:04:56.888 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:04:56.887+0000 7fb2c09cc700 1 -- 172.21.15.82:0/1376828723 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2bc198e20 con 0x7fb2bc101f20 2024-05-20T16:04:56.888 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:04:56.887+0000 7fb2ab7fe700 1 -- 172.21.15.82:0/1376828723 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb2b0022470 con 0x7fb2bc101f20 2024-05-20T16:04:56.888 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:04:56.887+0000 7fb2ab7fe700 1 -- 172.21.15.82:0/1376828723 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb2b000f5d0 con 0x7fb2bc101f20 2024-05-20T16:04:56.888 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:04:56.887+0000 7fb2c09cc700 1 -- 172.21.15.82:0/1376828723 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2bc19ba30 con 0x7fb2bc101f20 2024-05-20T16:04:56.889 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:04:56.888+0000 7fb2ab7fe700 1 -- 172.21.15.82:0/1376828723 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fb2b002a070 con 0x7fb2bc101f20 2024-05-20T16:04:56.889 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:04:56.888+0000 7fb2ab7fe700 1 --2- 172.21.15.82:0/1376828723 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fb2a40390f0 0x7fb2a403b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:04:56.889 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:04:56.888+0000 7fb2ab7fe700 1 -- 172.21.15.82:0/1376828723 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fb2b001e030 con 0x7fb2bc101f20 2024-05-20T16:04:56.889 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:04:56.888+0000 7fb2b9d9b700 1 --2- 172.21.15.82:0/1376828723 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fb2a40390f0 0x7fb2a403b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:04:56.889 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:04:56.888+0000 7fb2c09cc700 1 -- 172.21.15.82:0/1376828723 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2bc1081e0 con 0x7fb2bc101f20 2024-05-20T16:04:56.889 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:04:56.888+0000 7fb2b9d9b700 1 --2- 172.21.15.82:0/1376828723 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fb2a40390f0 0x7fb2a403b600 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fb2ac006fd0 tx=0x7fb2ac006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:04:56.894 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:04:56.894+0000 7fb2ab7fe700 1 -- 172.21.15.82:0/1376828723 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fb2b0031020 con 0x7fb2bc101f20 2024-05-20T16:04:57.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:04:57.096+0000 7fb2c09cc700 1 -- 172.21.15.82:0/1376828723 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7fb2bc02dd00 con 0x7fb2bc101f20 2024-05-20T16:04:57.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:04:57.100+0000 7fb2ab7fe700 1 -- 172.21.15.82:0/1376828723 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7fb2b0013070 con 0x7fb2bc101f20 2024-05-20T16:04:57.103 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:04:57.102+0000 7fb2c09cc700 1 -- 172.21.15.82:0/1376828723 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fb2a40390f0 msgr2=0x7fb2a403b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:04:57.103 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:04:57.102+0000 7fb2c09cc700 1 --2- 172.21.15.82:0/1376828723 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fb2a40390f0 0x7fb2a403b600 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fb2ac006fd0 tx=0x7fb2ac006e40 comp rx=0 tx=0).stop 2024-05-20T16:04:57.103 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:04:57.102+0000 7fb2c09cc700 1 -- 172.21.15.82:0/1376828723 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb2bc101f20 msgr2=0x7fb2bc198650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:04:57.103 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:04:57.102+0000 7fb2c09cc700 1 --2- 172.21.15.82:0/1376828723 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb2bc101f20 0x7fb2bc198650 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fb2b000b5c0 tx=0x7fb2b0005050 comp rx=0 tx=0).stop 2024-05-20T16:04:57.103 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:04:57.102+0000 7fb2c09cc700 1 -- 172.21.15.82:0/1376828723 shutdown_connections 2024-05-20T16:04:57.103 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:04:57.102+0000 7fb2c09cc700 1 --2- 172.21.15.82:0/1376828723 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fb2a40390f0 0x7fb2a403b600 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:04:57.103 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:04:57.102+0000 7fb2c09cc700 1 --2- 172.21.15.82:0/1376828723 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb2bc101f20 0x7fb2bc198650 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:04:57.103 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:04:57.102+0000 7fb2c09cc700 1 -- 172.21.15.82:0/1376828723 >> 172.21.15.82:0/1376828723 conn(0x7fb2bc0fc710 msgr2=0x7fb2bc1011c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:04:57.103 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:04:57.102+0000 7fb2c09cc700 1 -- 172.21.15.82:0/1376828723 shutdown_connections 2024-05-20T16:04:57.103 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:04:57.102+0000 7fb2c09cc700 1 -- 172.21.15.82:0/1376828723 wait complete. 2024-05-20T16:04:57.232 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:04:57 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:04:57.232 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:04:57 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:04:57.232 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:04:57 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi082", "name": "osd_memory_target"}]: dispatch 2024-05-20T16:04:57.232 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:04:57 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:04:57.232 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:04:57 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T16:04:57.232 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:04:57 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:04:57.879 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-05-20T16:04:57.879 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 59d1600a-16c2-11ef-bc9a-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-05-20T16:04:58.110 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:04:58 smithi082 ceph-mon[32238]: from='client.? 172.21.15.82:0/1376828723' entity='client.admin' 2024-05-20T16:04:58.110 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:04:58 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T16:04:58.110 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:04:58 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:04:58.558 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T16:04:59.965 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:04:59.963+0000 7fce20f89700 1 -- 172.21.15.82:0/3332375898 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fce1c101f60 msgr2=0x7fce1c102370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:04:59.965 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:04:59.963+0000 7fce20f89700 1 --2- 172.21.15.82:0/3332375898 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fce1c101f60 0x7fce1c102370 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fce0c009910 tx=0x7fce0c009c20 comp rx=0 tx=0).stop 2024-05-20T16:04:59.965 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:04:59.963+0000 7fce20f89700 1 -- 172.21.15.82:0/3332375898 shutdown_connections 2024-05-20T16:04:59.965 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:04:59.963+0000 7fce20f89700 1 --2- 172.21.15.82:0/3332375898 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fce1c101f60 0x7fce1c102370 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:04:59.965 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:04:59.963+0000 7fce20f89700 1 -- 172.21.15.82:0/3332375898 >> 172.21.15.82:0/3332375898 conn(0x7fce1c0fc730 msgr2=0x7fce1c0fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:04:59.966 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:04:59.963+0000 7fce20f89700 1 -- 172.21.15.82:0/3332375898 shutdown_connections 2024-05-20T16:04:59.966 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:04:59.963+0000 7fce20f89700 1 -- 172.21.15.82:0/3332375898 wait complete. 2024-05-20T16:04:59.966 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:04:59.963+0000 7fce20f89700 1 Processor -- start 2024-05-20T16:04:59.966 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:04:59.964+0000 7fce20f89700 1 -- start start 2024-05-20T16:04:59.966 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:04:59.964+0000 7fce20f89700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fce1c101f60 0x7fce1c1975a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:04:59.966 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:04:59.964+0000 7fce20f89700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fce1c107aa0 con 0x7fce1c101f60 2024-05-20T16:04:59.966 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:04:59.964+0000 7fce1a59c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fce1c101f60 0x7fce1c1975a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:04:59.966 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:04:59.964+0000 7fce1a59c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fce1c101f60 0x7fce1c1975a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44690/0 (socket says 172.21.15.82:44690) 2024-05-20T16:04:59.966 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:04:59.964+0000 7fce1a59c700 1 -- 172.21.15.82:0/3375935645 learned_addr learned my addr 172.21.15.82:0/3375935645 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:04:59.966 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:04:59.964+0000 7fce1a59c700 1 -- 172.21.15.82:0/3375935645 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce1c197ae0 con 0x7fce1c101f60 2024-05-20T16:04:59.966 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:04:59.964+0000 7fce1a59c700 1 --2- 172.21.15.82:0/3375935645 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fce1c101f60 0x7fce1c1975a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fce0c00b5c0 tx=0x7fce0c005050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:04:59.966 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:04:59.964+0000 7fce0b7fe700 1 -- 172.21.15.82:0/3375935645 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fce0c027070 con 0x7fce1c101f60 2024-05-20T16:04:59.967 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:04:59.964+0000 7fce0b7fe700 1 -- 172.21.15.82:0/3375935645 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fce0c022470 con 0x7fce1c101f60 2024-05-20T16:04:59.967 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:04:59.964+0000 7fce20f89700 1 -- 172.21.15.82:0/3375935645 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce1c197d70 con 0x7fce1c101f60 2024-05-20T16:04:59.967 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:04:59.964+0000 7fce0b7fe700 1 -- 172.21.15.82:0/3375935645 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fce0c00f5d0 con 0x7fce1c101f60 2024-05-20T16:04:59.967 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:04:59.965+0000 7fce20f89700 1 -- 172.21.15.82:0/3375935645 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce1c19a980 con 0x7fce1c101f60 2024-05-20T16:04:59.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:04:59.965+0000 7fce20f89700 1 -- 172.21.15.82:0/3375935645 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce1c108220 con 0x7fce1c101f60 2024-05-20T16:04:59.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:04:59.966+0000 7fce0b7fe700 1 -- 172.21.15.82:0/3375935645 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fce0c02a070 con 0x7fce1c101f60 2024-05-20T16:04:59.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:04:59.966+0000 7fce0b7fe700 1 --2- 172.21.15.82:0/3375935645 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fce040390f0 0x7fce0403b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:04:59.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:04:59.966+0000 7fce0b7fe700 1 -- 172.21.15.82:0/3375935645 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fce0c01e030 con 0x7fce1c101f60 2024-05-20T16:04:59.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:04:59.966+0000 7fce19d9b700 1 --2- 172.21.15.82:0/3375935645 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fce040390f0 0x7fce0403b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:04:59.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:04:59.966+0000 7fce19d9b700 1 --2- 172.21.15.82:0/3375935645 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fce040390f0 0x7fce0403b600 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fce10006fd0 tx=0x7fce10006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:04:59.971 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:04:59.971+0000 7fce0b7fe700 1 -- 172.21.15.82:0/3375935645 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fce0c013070 con 0x7fce1c101f60 2024-05-20T16:05:00.179 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:00.178+0000 7fce20f89700 1 -- 172.21.15.82:0/3375935645 --> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7fce1c02dd70 con 0x7fce040390f0 2024-05-20T16:05:00.182 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:00.182+0000 7fce0b7fe700 1 -- 172.21.15.82:0/3375935645 <== mgr.14150 v2:172.21.15.82:6800/379530823 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fce1c02dd70 con 0x7fce040390f0 2024-05-20T16:05:00.185 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:00.184+0000 7fce20f89700 1 -- 172.21.15.82:0/3375935645 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fce040390f0 msgr2=0x7fce0403b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:05:00.185 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:00.184+0000 7fce20f89700 1 --2- 172.21.15.82:0/3375935645 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fce040390f0 0x7fce0403b600 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fce10006fd0 tx=0x7fce10006e40 comp rx=0 tx=0).stop 2024-05-20T16:05:00.185 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:00.184+0000 7fce20f89700 1 -- 172.21.15.82:0/3375935645 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fce1c101f60 msgr2=0x7fce1c1975a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:05:00.185 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:00.184+0000 7fce20f89700 1 --2- 172.21.15.82:0/3375935645 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fce1c101f60 0x7fce1c1975a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fce0c00b5c0 tx=0x7fce0c005050 comp rx=0 tx=0).stop 2024-05-20T16:05:00.185 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:00.184+0000 7fce20f89700 1 -- 172.21.15.82:0/3375935645 shutdown_connections 2024-05-20T16:05:00.185 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:00.184+0000 7fce20f89700 1 --2- 172.21.15.82:0/3375935645 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fce040390f0 0x7fce0403b600 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:05:00.185 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:00.184+0000 7fce20f89700 1 --2- 172.21.15.82:0/3375935645 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fce1c101f60 0x7fce1c1975a0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:05:00.185 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:00.184+0000 7fce20f89700 1 -- 172.21.15.82:0/3375935645 >> 172.21.15.82:0/3375935645 conn(0x7fce1c0fc730 msgr2=0x7fce1c0fd3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:05:00.186 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:00.184+0000 7fce20f89700 1 -- 172.21.15.82:0/3375935645 shutdown_connections 2024-05-20T16:05:00.186 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:00.184+0000 7fce20f89700 1 -- 172.21.15.82:0/3375935645 wait complete. 2024-05-20T16:05:00.873 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi160 2024-05-20T16:05:00.873 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2024-05-20T16:05:00.873 DEBUG:teuthology.orchestra.run.smithi160:> dd of=/etc/ceph/ceph.conf 2024-05-20T16:05:00.891 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2024-05-20T16:05:00.891 DEBUG:teuthology.orchestra.run.smithi160:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-20T16:05:00.948 INFO:tasks.cephadm:Adding host smithi160 to orchestrator... 2024-05-20T16:05:00.948 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 59d1600a-16c2-11ef-bc9a-c7b262605968 -- ceph orch host add smithi160 2024-05-20T16:05:01.191 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T16:05:01.312 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:01 smithi082 ceph-mon[32238]: from='client.14174 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-05-20T16:05:01.313 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:01 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:05:01.313 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:01 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T16:05:01.313 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:01 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:05:01.313 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:01 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T16:05:01.313 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:01 smithi082 ceph-mon[32238]: Updating smithi082:/etc/ceph/ceph.conf 2024-05-20T16:05:01.313 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:01 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:05:01.314 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:01 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:05:01.314 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:01 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:05:02.268 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:02 smithi082 ceph-mon[32238]: Updating smithi082:/var/lib/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-20T16:05:02.269 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:02 smithi082 ceph-mon[32238]: Updating smithi082:/etc/ceph/ceph.client.admin.keyring 2024-05-20T16:05:02.269 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:02 smithi082 ceph-mon[32238]: Updating smithi082:/var/lib/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/config/ceph.client.admin.keyring 2024-05-20T16:05:02.843 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:02.841+0000 7f8f1832a700 1 -- 172.21.15.82:0/536753514 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8f101089f0 msgr2=0x7f8f10108e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:05:02.843 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:02.841+0000 7f8f1832a700 1 --2- 172.21.15.82:0/536753514 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8f101089f0 0x7f8f10108e00 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f8f0c009910 tx=0x7f8f0c009c20 comp rx=0 tx=0).stop 2024-05-20T16:05:02.843 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:02.841+0000 7f8f1832a700 1 -- 172.21.15.82:0/536753514 shutdown_connections 2024-05-20T16:05:02.843 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:02.841+0000 7f8f1832a700 1 --2- 172.21.15.82:0/536753514 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8f101089f0 0x7f8f10108e00 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:05:02.843 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:02.841+0000 7f8f1832a700 1 -- 172.21.15.82:0/536753514 >> 172.21.15.82:0/536753514 conn(0x7f8f100fe4e0 msgr2=0x7f8f10100930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:05:02.843 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:02.841+0000 7f8f1832a700 1 -- 172.21.15.82:0/536753514 shutdown_connections 2024-05-20T16:05:02.843 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:02.841+0000 7f8f1832a700 1 -- 172.21.15.82:0/536753514 wait complete. 2024-05-20T16:05:02.844 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:02.841+0000 7f8f1832a700 1 Processor -- start 2024-05-20T16:05:02.844 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:02.841+0000 7f8f1832a700 1 -- start start 2024-05-20T16:05:02.844 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:02.842+0000 7f8f1832a700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8f101089f0 0x7f8f1019e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:05:02.844 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:02.842+0000 7f8f1832a700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f8f1010d090 con 0x7f8f101089f0 2024-05-20T16:05:02.844 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:02.842+0000 7f8f160c6700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8f101089f0 0x7f8f1019e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:05:02.844 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:02.842+0000 7f8f160c6700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8f101089f0 0x7f8f1019e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44704/0 (socket says 172.21.15.82:44704) 2024-05-20T16:05:02.844 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:02.842+0000 7f8f160c6700 1 -- 172.21.15.82:0/486947050 learned_addr learned my addr 172.21.15.82:0/486947050 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:05:02.844 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:02.842+0000 7f8f160c6700 1 -- 172.21.15.82:0/486947050 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f1019eac0 con 0x7f8f101089f0 2024-05-20T16:05:02.844 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:02.842+0000 7f8f160c6700 1 --2- 172.21.15.82:0/486947050 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8f101089f0 0x7f8f1019e580 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f8f0c004f80 tx=0x7f8f0c005060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:05:02.844 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:02.842+0000 7f8f077fe700 1 -- 172.21.15.82:0/486947050 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8f0c026070 con 0x7f8f101089f0 2024-05-20T16:05:02.844 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:02.842+0000 7f8f077fe700 1 -- 172.21.15.82:0/486947050 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f0c00b6f0 con 0x7f8f101089f0 2024-05-20T16:05:02.844 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:02.842+0000 7f8f1832a700 1 -- 172.21.15.82:0/486947050 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f1019edb0 con 0x7f8f101089f0 2024-05-20T16:05:02.844 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:02.842+0000 7f8f077fe700 1 -- 172.21.15.82:0/486947050 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8f0c003910 con 0x7f8f101089f0 2024-05-20T16:05:02.844 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:02.842+0000 7f8f1832a700 1 -- 172.21.15.82:0/486947050 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f10198930 con 0x7f8f101089f0 2024-05-20T16:05:02.845 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:02.843+0000 7f8f1832a700 1 -- 172.21.15.82:0/486947050 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f10104100 con 0x7f8f101089f0 2024-05-20T16:05:02.845 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:02.843+0000 7f8f077fe700 1 -- 172.21.15.82:0/486947050 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f8f0c029070 con 0x7f8f101089f0 2024-05-20T16:05:02.846 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:02.844+0000 7f8f077fe700 1 --2- 172.21.15.82:0/486947050 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f8efc0390f0 0x7f8efc03b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:05:02.846 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:02.844+0000 7f8f077fe700 1 -- 172.21.15.82:0/486947050 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f8f0c01e070 con 0x7f8f101089f0 2024-05-20T16:05:02.846 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:02.844+0000 7f8f158c5700 1 --2- 172.21.15.82:0/486947050 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f8efc0390f0 0x7f8efc03b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:05:02.846 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:02.844+0000 7f8f158c5700 1 --2- 172.21.15.82:0/486947050 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f8efc0390f0 0x7f8efc03b600 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f8f00006fd0 tx=0x7f8f00006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:05:02.849 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:02.849+0000 7f8f077fe700 1 -- 172.21.15.82:0/486947050 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f8f0c013070 con 0x7f8f101089f0 2024-05-20T16:05:03.050 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:03.049+0000 7f8f1832a700 1 -- 172.21.15.82:0/486947050 --> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi160", "target": ["mon-mgr", ""]}) -- 0x7f8f10199c50 con 0x7f8efc0390f0 2024-05-20T16:05:03.360 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:03 smithi082 ceph-mon[32238]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi160", "target": ["mon-mgr", ""]}]: dispatch 2024-05-20T16:05:04.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:04 smithi082 ceph-mon[32238]: Deploying cephadm binary to smithi160 2024-05-20T16:05:06.431 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:06 smithi082 ceph-mon[32238]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-20T16:05:07.268 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:07.267+0000 7f8f077fe700 1 -- 172.21.15.82:0/486947050 <== mgr.14150 v2:172.21.15.82:6800/379530823 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f8f10199c50 con 0x7f8efc0390f0 2024-05-20T16:05:07.268 INFO:teuthology.orchestra.run.smithi082.stdout:Added host 'smithi160' with addr '172.21.15.160' 2024-05-20T16:05:07.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:07.270+0000 7f8f1832a700 1 -- 172.21.15.82:0/486947050 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f8efc0390f0 msgr2=0x7f8efc03b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:05:07.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:07.270+0000 7f8f1832a700 1 --2- 172.21.15.82:0/486947050 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f8efc0390f0 0x7f8efc03b600 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f8f00006fd0 tx=0x7f8f00006e40 comp rx=0 tx=0).stop 2024-05-20T16:05:07.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:07.270+0000 7f8f1832a700 1 -- 172.21.15.82:0/486947050 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8f101089f0 msgr2=0x7f8f1019e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:05:07.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:07.270+0000 7f8f1832a700 1 --2- 172.21.15.82:0/486947050 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8f101089f0 0x7f8f1019e580 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f8f0c004f80 tx=0x7f8f0c005060 comp rx=0 tx=0).stop 2024-05-20T16:05:07.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:07.270+0000 7f8f1832a700 1 -- 172.21.15.82:0/486947050 shutdown_connections 2024-05-20T16:05:07.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:07.270+0000 7f8f1832a700 1 --2- 172.21.15.82:0/486947050 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f8efc0390f0 0x7f8efc03b600 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:05:07.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:07.270+0000 7f8f1832a700 1 --2- 172.21.15.82:0/486947050 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8f101089f0 0x7f8f1019e580 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:05:07.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:07.270+0000 7f8f1832a700 1 -- 172.21.15.82:0/486947050 >> 172.21.15.82:0/486947050 conn(0x7f8f100fe4e0 msgr2=0x7f8f100ff410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:05:07.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:07.270+0000 7f8f1832a700 1 -- 172.21.15.82:0/486947050 shutdown_connections 2024-05-20T16:05:07.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:07.270+0000 7f8f1832a700 1 -- 172.21.15.82:0/486947050 wait complete. 2024-05-20T16:05:07.803 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 59d1600a-16c2-11ef-bc9a-c7b262605968 -- ceph orch host ls --format=json 2024-05-20T16:05:08.040 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T16:05:08.362 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:08 smithi082 ceph-mon[32238]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-20T16:05:08.362 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:08 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:05:08.362 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:08 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T16:05:08.362 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:08 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:05:09.360 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:09 smithi082 ceph-mon[32238]: Added host smithi160 2024-05-20T16:05:09.360 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:09 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:05:09.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:09.631+0000 7ff7f5f0e700 1 -- 172.21.15.82:0/2791133586 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff7f0101150 msgr2=0x7ff7f0101560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:05:09.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:09.631+0000 7ff7f5f0e700 1 --2- 172.21.15.82:0/2791133586 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff7f0101150 0x7ff7f0101560 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7ff7d8009910 tx=0x7ff7d8009c20 comp rx=0 tx=0).stop 2024-05-20T16:05:09.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:09.631+0000 7ff7f5f0e700 1 -- 172.21.15.82:0/2791133586 shutdown_connections 2024-05-20T16:05:09.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:09.631+0000 7ff7f5f0e700 1 --2- 172.21.15.82:0/2791133586 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff7f0101150 0x7ff7f0101560 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:05:09.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:09.631+0000 7ff7f5f0e700 1 -- 172.21.15.82:0/2791133586 >> 172.21.15.82:0/2791133586 conn(0x7ff7f00fccb0 msgr2=0x7ff7f00ff0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:05:09.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:09.632+0000 7ff7f5f0e700 1 -- 172.21.15.82:0/2791133586 shutdown_connections 2024-05-20T16:05:09.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:09.632+0000 7ff7f5f0e700 1 -- 172.21.15.82:0/2791133586 wait complete. 2024-05-20T16:05:09.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:09.632+0000 7ff7f5f0e700 1 Processor -- start 2024-05-20T16:05:09.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:09.633+0000 7ff7f5f0e700 1 -- start start 2024-05-20T16:05:09.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:09.633+0000 7ff7f5f0e700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff7f0101150 0x7ff7f019e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:05:09.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:09.633+0000 7ff7f5f0e700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7ff7f010d080 con 0x7ff7f0101150 2024-05-20T16:05:09.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:09.633+0000 7ff7ef7fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff7f0101150 0x7ff7f019e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:05:09.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:09.633+0000 7ff7ef7fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff7f0101150 0x7ff7f019e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:46390/0 (socket says 172.21.15.82:46390) 2024-05-20T16:05:09.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:09.633+0000 7ff7ef7fe700 1 -- 172.21.15.82:0/2158814954 learned_addr learned my addr 172.21.15.82:0/2158814954 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:05:09.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:09.633+0000 7ff7ef7fe700 1 -- 172.21.15.82:0/2158814954 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7f019eac0 con 0x7ff7f0101150 2024-05-20T16:05:09.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:09.634+0000 7ff7ef7fe700 1 --2- 172.21.15.82:0/2158814954 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff7f0101150 0x7ff7f019e580 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7ff7d8005850 tx=0x7ff7d8005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:05:09.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:09.634+0000 7ff7ecff9700 1 -- 172.21.15.82:0/2158814954 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff7d8026070 con 0x7ff7f0101150 2024-05-20T16:05:09.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:09.634+0000 7ff7ecff9700 1 -- 172.21.15.82:0/2158814954 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff7d800bbd0 con 0x7ff7f0101150 2024-05-20T16:05:09.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:09.634+0000 7ff7f5f0e700 1 -- 172.21.15.82:0/2158814954 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7f019edb0 con 0x7ff7f0101150 2024-05-20T16:05:09.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:09.634+0000 7ff7ecff9700 1 -- 172.21.15.82:0/2158814954 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff7d8003970 con 0x7ff7f0101150 2024-05-20T16:05:09.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:09.634+0000 7ff7f5f0e700 1 -- 172.21.15.82:0/2158814954 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7f01988e0 con 0x7ff7f0101150 2024-05-20T16:05:09.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:09.635+0000 7ff7ecff9700 1 -- 172.21.15.82:0/2158814954 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7ff7d8029070 con 0x7ff7f0101150 2024-05-20T16:05:09.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:09.635+0000 7ff7f5f0e700 1 -- 172.21.15.82:0/2158814954 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7f01041e0 con 0x7ff7f0101150 2024-05-20T16:05:09.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:09.635+0000 7ff7ecff9700 1 --2- 172.21.15.82:0/2158814954 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7ff7dc039150 0x7ff7dc03b660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:05:09.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:09.635+0000 7ff7ecff9700 1 -- 172.21.15.82:0/2158814954 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7ff7d801e070 con 0x7ff7f0101150 2024-05-20T16:05:09.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:09.635+0000 7ff7eeffd700 1 --2- 172.21.15.82:0/2158814954 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7ff7dc039150 0x7ff7dc03b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:05:09.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:09.636+0000 7ff7eeffd700 1 --2- 172.21.15.82:0/2158814954 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7ff7dc039150 0x7ff7dc03b660 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7ff7e0006fd0 tx=0x7ff7e0006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:05:09.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:09.640+0000 7ff7ecff9700 1 -- 172.21.15.82:0/2158814954 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7ff7d8013070 con 0x7ff7f0101150 2024-05-20T16:05:09.852 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:09.851+0000 7ff7f5f0e700 1 -- 172.21.15.82:0/2158814954 --> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff7f0199c80 con 0x7ff7dc039150 2024-05-20T16:05:09.852 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:09.852+0000 7ff7ecff9700 1 -- 172.21.15.82:0/2158814954 <== mgr.14150 v2:172.21.15.82:6800/379530823 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+160 (secure 0 0 0) 0x7ff7f0199c80 con 0x7ff7dc039150 2024-05-20T16:05:09.853 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-05-20T16:05:09.853 INFO:teuthology.orchestra.run.smithi082.stdout:[{"addr": "172.21.15.82", "hostname": "smithi082", "labels": [], "status": ""}, {"addr": "172.21.15.160", "hostname": "smithi160", "labels": [], "status": ""}] 2024-05-20T16:05:09.855 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:09.854+0000 7ff7f5f0e700 1 -- 172.21.15.82:0/2158814954 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7ff7dc039150 msgr2=0x7ff7dc03b660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:05:09.855 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:09.854+0000 7ff7f5f0e700 1 --2- 172.21.15.82:0/2158814954 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7ff7dc039150 0x7ff7dc03b660 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7ff7e0006fd0 tx=0x7ff7e0006e40 comp rx=0 tx=0).stop 2024-05-20T16:05:09.855 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:09.854+0000 7ff7f5f0e700 1 -- 172.21.15.82:0/2158814954 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff7f0101150 msgr2=0x7ff7f019e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:05:09.855 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:09.854+0000 7ff7f5f0e700 1 --2- 172.21.15.82:0/2158814954 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff7f0101150 0x7ff7f019e580 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7ff7d8005850 tx=0x7ff7d8005030 comp rx=0 tx=0).stop 2024-05-20T16:05:09.855 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:09.854+0000 7ff7f5f0e700 1 -- 172.21.15.82:0/2158814954 shutdown_connections 2024-05-20T16:05:09.855 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:09.854+0000 7ff7f5f0e700 1 --2- 172.21.15.82:0/2158814954 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7ff7dc039150 0x7ff7dc03b660 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:05:09.855 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:09.854+0000 7ff7f5f0e700 1 --2- 172.21.15.82:0/2158814954 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff7f0101150 0x7ff7f019e580 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:05:09.855 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:09.854+0000 7ff7f5f0e700 1 -- 172.21.15.82:0/2158814954 >> 172.21.15.82:0/2158814954 conn(0x7ff7f00fccb0 msgr2=0x7ff7f0108770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:05:09.855 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:09.854+0000 7ff7f5f0e700 1 -- 172.21.15.82:0/2158814954 shutdown_connections 2024-05-20T16:05:09.855 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:09.854+0000 7ff7f5f0e700 1 -- 172.21.15.82:0/2158814954 wait complete. 2024-05-20T16:05:10.445 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:10 smithi082 ceph-mon[32238]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-20T16:05:10.445 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:10 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:05:10.472 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi184 2024-05-20T16:05:10.473 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-05-20T16:05:10.473 DEBUG:teuthology.orchestra.run.smithi184:> dd of=/etc/ceph/ceph.conf 2024-05-20T16:05:10.491 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-05-20T16:05:10.491 DEBUG:teuthology.orchestra.run.smithi184:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-20T16:05:10.549 INFO:tasks.cephadm:Adding host smithi184 to orchestrator... 2024-05-20T16:05:10.550 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 59d1600a-16c2-11ef-bc9a-c7b262605968 -- ceph orch host add smithi184 2024-05-20T16:05:10.787 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T16:05:11.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:11 smithi082 ceph-mon[32238]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-20T16:05:12.119 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:12.117+0000 7f250e884700 1 -- 172.21.15.82:0/3948531645 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f25081089e0 msgr2=0x7f2508108df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:05:12.119 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:12.117+0000 7f250e884700 1 --2- 172.21.15.82:0/3948531645 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f25081089e0 0x7f2508108df0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f24f8009910 tx=0x7f24f8009c20 comp rx=0 tx=0).stop 2024-05-20T16:05:12.119 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:12.117+0000 7f250e884700 1 -- 172.21.15.82:0/3948531645 shutdown_connections 2024-05-20T16:05:12.119 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:12.117+0000 7f250e884700 1 --2- 172.21.15.82:0/3948531645 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f25081089e0 0x7f2508108df0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:05:12.120 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:12.117+0000 7f250e884700 1 -- 172.21.15.82:0/3948531645 >> 172.21.15.82:0/3948531645 conn(0x7f25080fe530 msgr2=0x7f2508100980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:05:12.120 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:12.117+0000 7f250e884700 1 -- 172.21.15.82:0/3948531645 shutdown_connections 2024-05-20T16:05:12.120 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:12.117+0000 7f250e884700 1 -- 172.21.15.82:0/3948531645 wait complete. 2024-05-20T16:05:12.120 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:12.118+0000 7f250e884700 1 Processor -- start 2024-05-20T16:05:12.120 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:12.118+0000 7f250e884700 1 -- start start 2024-05-20T16:05:12.120 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:12.118+0000 7f250e884700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f25081089e0 0x7f250819e550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:05:12.120 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:12.118+0000 7f250e884700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f250810d080 con 0x7f25081089e0 2024-05-20T16:05:12.120 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:12.119+0000 7f2507fff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f25081089e0 0x7f250819e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:05:12.120 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:12.119+0000 7f2507fff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f25081089e0 0x7f250819e550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:46408/0 (socket says 172.21.15.82:46408) 2024-05-20T16:05:12.120 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:12.119+0000 7f2507fff700 1 -- 172.21.15.82:0/1707439065 learned_addr learned my addr 172.21.15.82:0/1707439065 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:05:12.122 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:12.119+0000 7f2507fff700 1 -- 172.21.15.82:0/1707439065 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f250819ea90 con 0x7f25081089e0 2024-05-20T16:05:12.122 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:12.119+0000 7f2507fff700 1 --2- 172.21.15.82:0/1707439065 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f25081089e0 0x7f250819e550 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f24f8004f80 tx=0x7f24f8005060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:05:12.122 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:12.119+0000 7f25057fa700 1 -- 172.21.15.82:0/1707439065 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f24f8026070 con 0x7f25081089e0 2024-05-20T16:05:12.122 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:12.119+0000 7f250e884700 1 -- 172.21.15.82:0/1707439065 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f250819ed80 con 0x7f25081089e0 2024-05-20T16:05:12.122 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:12.119+0000 7f25057fa700 1 -- 172.21.15.82:0/1707439065 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f24f800b6f0 con 0x7f25081089e0 2024-05-20T16:05:12.122 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:12.119+0000 7f25057fa700 1 -- 172.21.15.82:0/1707439065 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f24f8003910 con 0x7f25081089e0 2024-05-20T16:05:12.122 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:12.119+0000 7f250e884700 1 -- 172.21.15.82:0/1707439065 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25081988b0 con 0x7f25081089e0 2024-05-20T16:05:12.123 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:12.120+0000 7f25057fa700 1 -- 172.21.15.82:0/1707439065 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f24f8029070 con 0x7f25081089e0 2024-05-20T16:05:12.123 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:12.120+0000 7f250e884700 1 -- 172.21.15.82:0/1707439065 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25081040f0 con 0x7f25081089e0 2024-05-20T16:05:12.123 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:12.120+0000 7f25057fa700 1 --2- 172.21.15.82:0/1707439065 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f24f00390f0 0x7f24f003b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:05:12.123 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:12.120+0000 7f25057fa700 1 -- 172.21.15.82:0/1707439065 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f24f801e070 con 0x7f25081089e0 2024-05-20T16:05:12.123 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:12.120+0000 7f25077fe700 1 --2- 172.21.15.82:0/1707439065 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f24f00390f0 0x7f24f003b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:05:12.123 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:12.121+0000 7f25077fe700 1 --2- 172.21.15.82:0/1707439065 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f24f00390f0 0x7f24f003b600 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f24fc006fd0 tx=0x7f24fc006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:05:12.126 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:12.126+0000 7f25057fa700 1 -- 172.21.15.82:0/1707439065 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f24f8013070 con 0x7f25081089e0 2024-05-20T16:05:12.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:12.324+0000 7f250e884700 1 -- 172.21.15.82:0/1707439065 --> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi184", "target": ["mon-mgr", ""]}) -- 0x7f2508199c20 con 0x7f24f00390f0 2024-05-20T16:05:12.325 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:12 smithi082 ceph-mon[32238]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-20T16:05:14.360 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:14 smithi082 ceph-mon[32238]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi184", "target": ["mon-mgr", ""]}]: dispatch 2024-05-20T16:05:14.360 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:14 smithi082 ceph-mon[32238]: Deploying cephadm binary to smithi184 2024-05-20T16:05:14.360 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:14 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:05:14.360 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:14 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:05:14.360 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:14 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:05:14.360 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:14 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:05:14.360 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:14 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi160", "name": "osd_memory_target"}]: dispatch 2024-05-20T16:05:14.360 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:14 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:05:14.360 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:14 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T16:05:14.360 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:14 smithi082 ceph-mon[32238]: Updating smithi160:/etc/ceph/ceph.conf 2024-05-20T16:05:14.361 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:14 smithi082 ceph-mon[32238]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-20T16:05:15.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:15 smithi082 ceph-mon[32238]: Updating smithi160:/var/lib/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-20T16:05:16.360 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:16 smithi082 ceph-mon[32238]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-20T16:05:16.402 INFO:teuthology.orchestra.run.smithi082.stdout:Added host 'smithi184' with addr '172.21.15.184' 2024-05-20T16:05:16.403 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:16.401+0000 7f25057fa700 1 -- 172.21.15.82:0/1707439065 <== mgr.14150 v2:172.21.15.82:6800/379530823 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f2508199c20 con 0x7f24f00390f0 2024-05-20T16:05:16.404 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:16.404+0000 7f250e884700 1 -- 172.21.15.82:0/1707439065 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f24f00390f0 msgr2=0x7f24f003b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:05:16.405 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:16.404+0000 7f250e884700 1 --2- 172.21.15.82:0/1707439065 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f24f00390f0 0x7f24f003b600 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f24fc006fd0 tx=0x7f24fc006e40 comp rx=0 tx=0).stop 2024-05-20T16:05:16.405 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:16.404+0000 7f250e884700 1 -- 172.21.15.82:0/1707439065 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f25081089e0 msgr2=0x7f250819e550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:05:16.405 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:16.404+0000 7f250e884700 1 --2- 172.21.15.82:0/1707439065 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f25081089e0 0x7f250819e550 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f24f8004f80 tx=0x7f24f8005060 comp rx=0 tx=0).stop 2024-05-20T16:05:16.405 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:16.404+0000 7f250e884700 1 -- 172.21.15.82:0/1707439065 shutdown_connections 2024-05-20T16:05:16.405 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:16.404+0000 7f250e884700 1 --2- 172.21.15.82:0/1707439065 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f24f00390f0 0x7f24f003b600 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:05:16.405 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:16.404+0000 7f250e884700 1 --2- 172.21.15.82:0/1707439065 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f25081089e0 0x7f250819e550 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:05:16.405 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:16.404+0000 7f250e884700 1 -- 172.21.15.82:0/1707439065 >> 172.21.15.82:0/1707439065 conn(0x7f25080fe530 msgr2=0x7f25080ff480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:05:16.405 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:16.404+0000 7f250e884700 1 -- 172.21.15.82:0/1707439065 shutdown_connections 2024-05-20T16:05:16.405 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:16.404+0000 7f250e884700 1 -- 172.21.15.82:0/1707439065 wait complete. 2024-05-20T16:05:17.006 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 59d1600a-16c2-11ef-bc9a-c7b262605968 -- ceph orch host ls --format=json 2024-05-20T16:05:17.240 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T16:05:17.555 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:17 smithi082 ceph-mon[32238]: Updating smithi160:/etc/ceph/ceph.client.admin.keyring 2024-05-20T16:05:17.555 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:17 smithi082 ceph-mon[32238]: Updating smithi160:/var/lib/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/config/ceph.client.admin.keyring 2024-05-20T16:05:17.555 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:17 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:05:17.555 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:17 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:05:17.555 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:17 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:05:17.555 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:17 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:05:17.556 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:17 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T16:05:17.556 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:17 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:05:18.360 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:18 smithi082 ceph-mon[32238]: Added host smithi184 2024-05-20T16:05:18.360 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:18 smithi082 ceph-mon[32238]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-20T16:05:18.360 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:18 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:05:18.767 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:18.766+0000 7fc2b4be7700 1 -- 172.21.15.82:0/1375435000 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc2b0101990 msgr2=0x7fc2b0101da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:05:18.767 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:18.766+0000 7fc2b4be7700 1 --2- 172.21.15.82:0/1375435000 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc2b0101990 0x7fc2b0101da0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fc2a4009910 tx=0x7fc2a4009c20 comp rx=0 tx=0).stop 2024-05-20T16:05:18.767 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:18.766+0000 7fc2b4be7700 1 -- 172.21.15.82:0/1375435000 shutdown_connections 2024-05-20T16:05:18.767 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:18.766+0000 7fc2b4be7700 1 --2- 172.21.15.82:0/1375435000 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc2b0101990 0x7fc2b0101da0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:05:18.767 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:18.766+0000 7fc2b4be7700 1 -- 172.21.15.82:0/1375435000 >> 172.21.15.82:0/1375435000 conn(0x7fc2b00fd3f0 msgr2=0x7fc2b00ff840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:05:18.767 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:18.766+0000 7fc2b4be7700 1 -- 172.21.15.82:0/1375435000 shutdown_connections 2024-05-20T16:05:18.767 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:18.766+0000 7fc2b4be7700 1 -- 172.21.15.82:0/1375435000 wait complete. 2024-05-20T16:05:18.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:18.767+0000 7fc2b4be7700 1 Processor -- start 2024-05-20T16:05:18.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:18.767+0000 7fc2b4be7700 1 -- start start 2024-05-20T16:05:18.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:18.767+0000 7fc2b4be7700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc2b0101990 0x7fc2b019eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:05:18.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:18.767+0000 7fc2b4be7700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fc2b01136f0 con 0x7fc2b0101990 2024-05-20T16:05:18.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:18.767+0000 7fc2ae59c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc2b0101990 0x7fc2b019eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:05:18.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:18.767+0000 7fc2ae59c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc2b0101990 0x7fc2b019eed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:46430/0 (socket says 172.21.15.82:46430) 2024-05-20T16:05:18.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:18.767+0000 7fc2ae59c700 1 -- 172.21.15.82:0/3388128350 learned_addr learned my addr 172.21.15.82:0/3388128350 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:05:18.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:18.767+0000 7fc2ae59c700 1 -- 172.21.15.82:0/3388128350 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2b019f410 con 0x7fc2b0101990 2024-05-20T16:05:18.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:18.768+0000 7fc2ae59c700 1 --2- 172.21.15.82:0/3388128350 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc2b0101990 0x7fc2b019eed0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fc2a4006010 tx=0x7fc2a4005050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:05:18.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:18.768+0000 7fc29f7fe700 1 -- 172.21.15.82:0/3388128350 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc2a4026070 con 0x7fc2b0101990 2024-05-20T16:05:18.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:18.768+0000 7fc2b4be7700 1 -- 172.21.15.82:0/3388128350 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2b019f6a0 con 0x7fc2b0101990 2024-05-20T16:05:18.770 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:18.768+0000 7fc29f7fe700 1 -- 172.21.15.82:0/3388128350 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc2a400bc30 con 0x7fc2b0101990 2024-05-20T16:05:18.770 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:18.768+0000 7fc29f7fe700 1 -- 172.21.15.82:0/3388128350 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc2a4003ab0 con 0x7fc2b0101990 2024-05-20T16:05:18.770 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:18.768+0000 7fc2b4be7700 1 -- 172.21.15.82:0/3388128350 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2b019fb00 con 0x7fc2b0101990 2024-05-20T16:05:18.770 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:18.769+0000 7fc29f7fe700 1 -- 172.21.15.82:0/3388128350 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fc2a4029070 con 0x7fc2b0101990 2024-05-20T16:05:18.771 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:18.769+0000 7fc29f7fe700 1 --2- 172.21.15.82:0/3388128350 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fc2980390f0 0x7fc29803b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:05:18.771 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:18.769+0000 7fc29f7fe700 1 -- 172.21.15.82:0/3388128350 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fc2a401e070 con 0x7fc2b0101990 2024-05-20T16:05:18.771 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:18.769+0000 7fc2b4be7700 1 -- 172.21.15.82:0/3388128350 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2b006b2a0 con 0x7fc2b0101990 2024-05-20T16:05:18.771 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:18.769+0000 7fc2add9b700 1 --2- 172.21.15.82:0/3388128350 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fc2980390f0 0x7fc29803b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:05:18.771 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:18.770+0000 7fc2add9b700 1 --2- 172.21.15.82:0/3388128350 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fc2980390f0 0x7fc29803b600 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fc2a0006fd0 tx=0x7fc2a0006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:05:18.775 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:18.774+0000 7fc29f7fe700 1 -- 172.21.15.82:0/3388128350 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fc2a4030080 con 0x7fc2b0101990 2024-05-20T16:05:18.975 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:18.975+0000 7fc2b4be7700 1 -- 172.21.15.82:0/3388128350 --> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc2b01a0230 con 0x7fc2980390f0 2024-05-20T16:05:18.976 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:18.976+0000 7fc29f7fe700 1 -- 172.21.15.82:0/3388128350 <== mgr.14150 v2:172.21.15.82:6800/379530823 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+240 (secure 0 0 0) 0x7fc2b01a0230 con 0x7fc2980390f0 2024-05-20T16:05:18.976 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-05-20T16:05:18.977 INFO:teuthology.orchestra.run.smithi082.stdout:[{"addr": "172.21.15.82", "hostname": "smithi082", "labels": [], "status": ""}, {"addr": "172.21.15.160", "hostname": "smithi160", "labels": [], "status": ""}, {"addr": "172.21.15.184", "hostname": "smithi184", "labels": [], "status": ""}] 2024-05-20T16:05:18.979 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:18.978+0000 7fc2b4be7700 1 -- 172.21.15.82:0/3388128350 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fc2980390f0 msgr2=0x7fc29803b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:05:18.979 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:18.978+0000 7fc2b4be7700 1 --2- 172.21.15.82:0/3388128350 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fc2980390f0 0x7fc29803b600 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fc2a0006fd0 tx=0x7fc2a0006e40 comp rx=0 tx=0).stop 2024-05-20T16:05:18.979 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:18.978+0000 7fc2b4be7700 1 -- 172.21.15.82:0/3388128350 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc2b0101990 msgr2=0x7fc2b019eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:05:18.979 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:18.978+0000 7fc2b4be7700 1 --2- 172.21.15.82:0/3388128350 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc2b0101990 0x7fc2b019eed0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fc2a4006010 tx=0x7fc2a4005050 comp rx=0 tx=0).stop 2024-05-20T16:05:18.979 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:18.978+0000 7fc2b4be7700 1 -- 172.21.15.82:0/3388128350 shutdown_connections 2024-05-20T16:05:18.979 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:18.978+0000 7fc2b4be7700 1 --2- 172.21.15.82:0/3388128350 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fc2980390f0 0x7fc29803b600 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:05:18.979 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:18.978+0000 7fc2b4be7700 1 --2- 172.21.15.82:0/3388128350 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc2b0101990 0x7fc2b019eed0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:05:18.979 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:18.978+0000 7fc2b4be7700 1 -- 172.21.15.82:0/3388128350 >> 172.21.15.82:0/3388128350 conn(0x7fc2b00fd3f0 msgr2=0x7fc2b00fe380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:05:18.979 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:18.978+0000 7fc2b4be7700 1 -- 172.21.15.82:0/3388128350 shutdown_connections 2024-05-20T16:05:18.979 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:18.978+0000 7fc2b4be7700 1 -- 172.21.15.82:0/3388128350 wait complete. 2024-05-20T16:05:19.525 INFO:tasks.cephadm:Setting crush tunables to default 2024-05-20T16:05:19.526 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 59d1600a-16c2-11ef-bc9a-c7b262605968 -- ceph osd crush tunables default 2024-05-20T16:05:19.759 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T16:05:20.210 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:20 smithi082 ceph-mon[32238]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-20T16:05:20.210 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:20 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:05:20.210 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:20 smithi082 ceph-mon[32238]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-20T16:05:21.239 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:21.238+0000 7fb7623f1700 1 -- 172.21.15.82:0/1224483317 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb75c1089f0 msgr2=0x7fb75c108e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:05:21.239 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:21.238+0000 7fb7623f1700 1 --2- 172.21.15.82:0/1224483317 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb75c1089f0 0x7fb75c108e00 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fb74c009910 tx=0x7fb74c009c20 comp rx=0 tx=0).stop 2024-05-20T16:05:21.239 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:21.238+0000 7fb7623f1700 1 -- 172.21.15.82:0/1224483317 shutdown_connections 2024-05-20T16:05:21.239 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:21.238+0000 7fb7623f1700 1 --2- 172.21.15.82:0/1224483317 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb75c1089f0 0x7fb75c108e00 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:05:21.239 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:21.238+0000 7fb7623f1700 1 -- 172.21.15.82:0/1224483317 >> 172.21.15.82:0/1224483317 conn(0x7fb75c0fe4e0 msgr2=0x7fb75c100930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:05:21.239 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:21.238+0000 7fb7623f1700 1 -- 172.21.15.82:0/1224483317 shutdown_connections 2024-05-20T16:05:21.239 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:21.238+0000 7fb7623f1700 1 -- 172.21.15.82:0/1224483317 wait complete. 2024-05-20T16:05:21.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:21.239+0000 7fb7623f1700 1 Processor -- start 2024-05-20T16:05:21.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:21.239+0000 7fb7623f1700 1 -- start start 2024-05-20T16:05:21.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:21.239+0000 7fb7623f1700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb75c1089f0 0x7fb75c19e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:05:21.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:21.239+0000 7fb7623f1700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fb75c10d090 con 0x7fb75c1089f0 2024-05-20T16:05:21.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:21.239+0000 7fb75bfff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb75c1089f0 0x7fb75c19e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:05:21.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:21.239+0000 7fb75bfff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb75c1089f0 0x7fb75c19e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41908/0 (socket says 172.21.15.82:41908) 2024-05-20T16:05:21.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:21.239+0000 7fb75bfff700 1 -- 172.21.15.82:0/917057685 learned_addr learned my addr 172.21.15.82:0/917057685 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:05:21.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:21.240+0000 7fb75bfff700 1 -- 172.21.15.82:0/917057685 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb75c19eac0 con 0x7fb75c1089f0 2024-05-20T16:05:21.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:21.240+0000 7fb75bfff700 1 --2- 172.21.15.82:0/917057685 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb75c1089f0 0x7fb75c19e580 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fb74c004f80 tx=0x7fb74c005060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:05:21.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:21.240+0000 7fb759ffb700 1 -- 172.21.15.82:0/917057685 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb74c026070 con 0x7fb75c1089f0 2024-05-20T16:05:21.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:21.240+0000 7fb759ffb700 1 -- 172.21.15.82:0/917057685 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb74c00b6f0 con 0x7fb75c1089f0 2024-05-20T16:05:21.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:21.240+0000 7fb7623f1700 1 -- 172.21.15.82:0/917057685 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb75c19edb0 con 0x7fb75c1089f0 2024-05-20T16:05:21.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:21.240+0000 7fb759ffb700 1 -- 172.21.15.82:0/917057685 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb74c003910 con 0x7fb75c1089f0 2024-05-20T16:05:21.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:21.240+0000 7fb7623f1700 1 -- 172.21.15.82:0/917057685 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb75c198930 con 0x7fb75c1089f0 2024-05-20T16:05:21.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:21.241+0000 7fb759ffb700 1 -- 172.21.15.82:0/917057685 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fb74c029070 con 0x7fb75c1089f0 2024-05-20T16:05:21.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:21.241+0000 7fb7623f1700 1 -- 172.21.15.82:0/917057685 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb75c104100 con 0x7fb75c1089f0 2024-05-20T16:05:21.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:21.241+0000 7fb759ffb700 1 --2- 172.21.15.82:0/917057685 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fb748039150 0x7fb74803b660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:05:21.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:21.241+0000 7fb759ffb700 1 -- 172.21.15.82:0/917057685 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fb74c01e070 con 0x7fb75c1089f0 2024-05-20T16:05:21.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:21.241+0000 7fb753fff700 1 --2- 172.21.15.82:0/917057685 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fb748039150 0x7fb74803b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:05:21.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:21.242+0000 7fb753fff700 1 --2- 172.21.15.82:0/917057685 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fb748039150 0x7fb74803b660 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fb744006fd0 tx=0x7fb744006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:05:21.247 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:21.246+0000 7fb759ffb700 1 -- 172.21.15.82:0/917057685 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fb74c013070 con 0x7fb75c1089f0 2024-05-20T16:05:21.451 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:21.450+0000 7fb7623f1700 1 -- 172.21.15.82:0/917057685 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7fb75c19efe0 con 0x7fb75c1089f0 2024-05-20T16:05:22.223 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:22.223+0000 7fb759ffb700 1 -- 172.21.15.82:0/917057685 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7fb74c030090 con 0x7fb75c1089f0 2024-05-20T16:05:22.226 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:22.225+0000 7fb7623f1700 1 -- 172.21.15.82:0/917057685 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fb748039150 msgr2=0x7fb74803b660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:05:22.227 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:22.225+0000 7fb7623f1700 1 --2- 172.21.15.82:0/917057685 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fb748039150 0x7fb74803b660 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fb744006fd0 tx=0x7fb744006e40 comp rx=0 tx=0).stop 2024-05-20T16:05:22.227 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:22.225+0000 7fb7623f1700 1 -- 172.21.15.82:0/917057685 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb75c1089f0 msgr2=0x7fb75c19e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:05:22.227 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:22.225+0000 7fb7623f1700 1 --2- 172.21.15.82:0/917057685 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb75c1089f0 0x7fb75c19e580 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fb74c004f80 tx=0x7fb74c005060 comp rx=0 tx=0).stop 2024-05-20T16:05:22.227 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:22.225+0000 7fb7623f1700 1 -- 172.21.15.82:0/917057685 shutdown_connections 2024-05-20T16:05:22.227 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:22.225+0000 7fb7623f1700 1 --2- 172.21.15.82:0/917057685 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fb748039150 0x7fb74803b660 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:05:22.227 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:22.225+0000 7fb7623f1700 1 --2- 172.21.15.82:0/917057685 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb75c1089f0 0x7fb75c19e580 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:05:22.227 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:22.225+0000 7fb7623f1700 1 -- 172.21.15.82:0/917057685 >> 172.21.15.82:0/917057685 conn(0x7fb75c0fe4e0 msgr2=0x7fb75c0ff410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:05:22.227 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:22.226+0000 7fb7623f1700 1 -- 172.21.15.82:0/917057685 shutdown_connections 2024-05-20T16:05:22.227 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:22.226+0000 7fb7623f1700 1 -- 172.21.15.82:0/917057685 wait complete. 2024-05-20T16:05:22.228 INFO:teuthology.orchestra.run.smithi082.stderr:adjusted tunables profile to default 2024-05-20T16:05:22.544 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:22 smithi082 ceph-mon[32238]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-20T16:05:22.544 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:22 smithi082 ceph-mon[32238]: from='client.? 172.21.15.82:0/917057685' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-05-20T16:05:22.544 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:22 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:05:22.544 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:22 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:05:22.544 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:22 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:05:22.545 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:22 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:05:22.545 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:22 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi184", "name": "osd_memory_target"}]: dispatch 2024-05-20T16:05:22.545 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:22 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:05:22.545 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:22 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T16:05:22.863 INFO:tasks.cephadm:Adding mon.a on smithi082 2024-05-20T16:05:22.863 INFO:tasks.cephadm:Adding mon.b on smithi160 2024-05-20T16:05:22.863 INFO:tasks.cephadm:Adding mon.c on smithi184 2024-05-20T16:05:22.864 DEBUG:teuthology.orchestra.run.smithi184:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 59d1600a-16c2-11ef-bc9a-c7b262605968 -- ceph orch apply mon '3;smithi082:172.21.15.82=a;smithi160:172.21.15.160=b;smithi184:172.21.15.184=c' 2024-05-20T16:05:23.091 INFO:teuthology.orchestra.run.smithi184.stderr:Inferring config /var/lib/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-20T16:05:23.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:23 smithi082 ceph-mon[32238]: Updating smithi184:/etc/ceph/ceph.conf 2024-05-20T16:05:23.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:23 smithi082 ceph-mon[32238]: from='client.? 172.21.15.82:0/917057685' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-05-20T16:05:23.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:23 smithi082 ceph-mon[32238]: osdmap e4: 0 total, 0 up, 0 in 2024-05-20T16:05:23.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:23 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:05:23.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:23 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:05:23.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:23 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:05:24.600 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:24.599+0000 7f37c62b1700 1 -- 172.21.15.184:0/1289367777 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f37c0107910 msgr2=0x7f37c0107d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:05:24.600 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:24.599+0000 7f37c62b1700 1 --2- 172.21.15.184:0/1289367777 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f37c0107910 0x7f37c0107d20 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f37b0009910 tx=0x7f37b0009c20 comp rx=0 tx=0).stop 2024-05-20T16:05:24.601 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:24.599+0000 7f37c62b1700 1 -- 172.21.15.184:0/1289367777 shutdown_connections 2024-05-20T16:05:24.601 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:24.599+0000 7f37c62b1700 1 --2- 172.21.15.184:0/1289367777 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f37c0107910 0x7f37c0107d20 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:05:24.601 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:24.599+0000 7f37c62b1700 1 -- 172.21.15.184:0/1289367777 >> 172.21.15.184:0/1289367777 conn(0x7f37c0077430 msgr2=0x7f37c0077890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:05:24.601 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:24.599+0000 7f37c62b1700 1 -- 172.21.15.184:0/1289367777 shutdown_connections 2024-05-20T16:05:24.601 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:24.599+0000 7f37c62b1700 1 -- 172.21.15.184:0/1289367777 wait complete. 2024-05-20T16:05:24.601 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:24.600+0000 7f37c62b1700 1 Processor -- start 2024-05-20T16:05:24.601 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:24.600+0000 7f37c62b1700 1 -- start start 2024-05-20T16:05:24.602 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:24.600+0000 7f37c62b1700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f37c0107910 0x7f37c019e550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:05:24.602 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:24.600+0000 7f37c62b1700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f37c010bfb0 con 0x7f37c0107910 2024-05-20T16:05:24.602 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:24.600+0000 7f37bffff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f37c0107910 0x7f37c019e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:05:24.602 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:24.600+0000 7f37bffff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f37c0107910 0x7f37c019e550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.184:41128/0 (socket says 172.21.15.184:41128) 2024-05-20T16:05:24.603 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:24.600+0000 7f37bffff700 1 -- 172.21.15.184:0/2649777755 learned_addr learned my addr 172.21.15.184:0/2649777755 (peer_addr_for_me v2:172.21.15.184:0/0) 2024-05-20T16:05:24.603 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:24.601+0000 7f37bffff700 1 -- 172.21.15.184:0/2649777755 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37c019ea90 con 0x7f37c0107910 2024-05-20T16:05:24.603 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:24.601+0000 7f37bffff700 1 --2- 172.21.15.184:0/2649777755 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f37c0107910 0x7f37c019e550 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f37b0004f80 tx=0x7f37b0005060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:05:24.603 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:24.601+0000 7f37bd7fa700 1 -- 172.21.15.184:0/2649777755 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f37b0026070 con 0x7f37c0107910 2024-05-20T16:05:24.603 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:24.601+0000 7f37c62b1700 1 -- 172.21.15.184:0/2649777755 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37c019ed80 con 0x7f37c0107910 2024-05-20T16:05:24.603 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:24.601+0000 7f37bd7fa700 1 -- 172.21.15.184:0/2649777755 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f37b000b6f0 con 0x7f37c0107910 2024-05-20T16:05:24.603 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:24.601+0000 7f37bd7fa700 1 -- 172.21.15.184:0/2649777755 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f37b0003910 con 0x7f37c0107910 2024-05-20T16:05:24.603 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:24.601+0000 7f37c62b1700 1 -- 172.21.15.184:0/2649777755 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37c01988b0 con 0x7f37c0107910 2024-05-20T16:05:24.604 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:24.602+0000 7f37c62b1700 1 -- 172.21.15.184:0/2649777755 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37c0103020 con 0x7f37c0107910 2024-05-20T16:05:24.604 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:24.602+0000 7f37bd7fa700 1 -- 172.21.15.184:0/2649777755 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f37b0029070 con 0x7f37c0107910 2024-05-20T16:05:24.604 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:24.602+0000 7f37bd7fa700 1 --2- 172.21.15.184:0/2649777755 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f37a80390f0 0x7f37a803b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:05:24.604 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:24.603+0000 7f37bd7fa700 1 -- 172.21.15.184:0/2649777755 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f37b001e070 con 0x7f37c0107910 2024-05-20T16:05:24.604 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:24.603+0000 7f37bf7fe700 1 --2- 172.21.15.184:0/2649777755 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f37a80390f0 0x7f37a803b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:05:24.604 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:24.603+0000 7f37bf7fe700 1 --2- 172.21.15.184:0/2649777755 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f37a80390f0 0x7f37a803b600 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f37b4006fd0 tx=0x7f37b4006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:05:24.609 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:24.608+0000 7f37bd7fa700 1 -- 172.21.15.184:0/2649777755 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f37b0013070 con 0x7f37c0107910 2024-05-20T16:05:24.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:24 smithi082 ceph-mon[32238]: Updating smithi184:/var/lib/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-20T16:05:24.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:24 smithi082 ceph-mon[32238]: Updating smithi184:/etc/ceph/ceph.client.admin.keyring 2024-05-20T16:05:24.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:24 smithi082 ceph-mon[32238]: Updating smithi184:/var/lib/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/config/ceph.client.admin.keyring 2024-05-20T16:05:24.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:24 smithi082 ceph-mon[32238]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-20T16:05:24.815 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:24.814+0000 7f37c62b1700 1 -- 172.21.15.184:0/2649777755 --> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi082:172.21.15.82=a;smithi160:172.21.15.160=b;smithi184:172.21.15.184=c", "target": ["mon-mgr", ""]}) -- 0x7f37c0199cb0 con 0x7f37a80390f0 2024-05-20T16:05:24.822 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:24.822+0000 7f37bd7fa700 1 -- 172.21.15.184:0/2649777755 <== mgr.14150 v2:172.21.15.82:6800/379530823 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f37c0199cb0 con 0x7f37a80390f0 2024-05-20T16:05:24.822 INFO:teuthology.orchestra.run.smithi184.stdout:Scheduled mon update... 2024-05-20T16:05:24.825 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:24.824+0000 7f37c62b1700 1 -- 172.21.15.184:0/2649777755 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f37a80390f0 msgr2=0x7f37a803b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:05:24.825 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:24.824+0000 7f37c62b1700 1 --2- 172.21.15.184:0/2649777755 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f37a80390f0 0x7f37a803b600 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f37b4006fd0 tx=0x7f37b4006e40 comp rx=0 tx=0).stop 2024-05-20T16:05:24.825 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:24.824+0000 7f37c62b1700 1 -- 172.21.15.184:0/2649777755 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f37c0107910 msgr2=0x7f37c019e550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:05:24.825 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:24.824+0000 7f37c62b1700 1 --2- 172.21.15.184:0/2649777755 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f37c0107910 0x7f37c019e550 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f37b0004f80 tx=0x7f37b0005060 comp rx=0 tx=0).stop 2024-05-20T16:05:24.825 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:24.824+0000 7f37c62b1700 1 -- 172.21.15.184:0/2649777755 shutdown_connections 2024-05-20T16:05:24.825 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:24.824+0000 7f37c62b1700 1 --2- 172.21.15.184:0/2649777755 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f37a80390f0 0x7f37a803b600 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:05:24.825 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:24.824+0000 7f37c62b1700 1 --2- 172.21.15.184:0/2649777755 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f37c0107910 0x7f37c019e550 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:05:24.825 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:24.824+0000 7f37c62b1700 1 -- 172.21.15.184:0/2649777755 >> 172.21.15.184:0/2649777755 conn(0x7f37c0077430 msgr2=0x7f37c0069c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:05:24.825 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:24.824+0000 7f37c62b1700 1 -- 172.21.15.184:0/2649777755 shutdown_connections 2024-05-20T16:05:24.825 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:24.824+0000 7f37c62b1700 1 -- 172.21.15.184:0/2649777755 wait complete. 2024-05-20T16:05:25.459 DEBUG:teuthology.orchestra.run.smithi160:mon.b> sudo journalctl -f -n 0 -u ceph-59d1600a-16c2-11ef-bc9a-c7b262605968@mon.b.service 2024-05-20T16:05:25.461 DEBUG:teuthology.orchestra.run.smithi184:mon.c> sudo journalctl -f -n 0 -u ceph-59d1600a-16c2-11ef-bc9a-c7b262605968@mon.c.service 2024-05-20T16:05:25.464 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-05-20T16:05:25.464 DEBUG:teuthology.orchestra.run.smithi184:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 59d1600a-16c2-11ef-bc9a-c7b262605968 -- ceph mon dump -f json 2024-05-20T16:05:26.110 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:25 smithi082 ceph-mon[32238]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi082:172.21.15.82=a;smithi160:172.21.15.160=b;smithi184:172.21.15.184=c", "target": ["mon-mgr", ""]}]: dispatch 2024-05-20T16:05:26.110 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:25 smithi082 ceph-mon[32238]: Saving service mon spec with placement smithi082:172.21.15.82=a;smithi160:172.21.15.160=b;smithi184:172.21.15.184=c;count:3 2024-05-20T16:05:26.110 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:25 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:05:26.110 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:25 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T16:05:26.110 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:25 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:05:26.110 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:25 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T16:05:26.110 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:25 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:05:26.110 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:25 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-20T16:05:26.110 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:25 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:05:26.111 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:25 smithi082 ceph-mon[32238]: Deploying daemon mon.c on smithi184 2024-05-20T16:05:26.111 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:25 smithi082 ceph-mon[32238]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-20T16:05:27.187 INFO:teuthology.orchestra.run.smithi184.stderr:Inferring config /var/lib/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-20T16:05:28.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:28 smithi082 ceph-mon[32238]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-20T16:05:29.309 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:29.307+0000 7f9c0a3c9700 1 -- 172.21.15.184:0/755235236 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9c041012e0 msgr2=0x7f9c041016f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:05:29.309 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:29.307+0000 7f9c0a3c9700 1 --2- 172.21.15.184:0/755235236 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9c041012e0 0x7f9c041016f0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f9bf8009910 tx=0x7f9bf8009c20 comp rx=0 tx=0).stop 2024-05-20T16:05:29.309 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:29.307+0000 7f9c0a3c9700 1 -- 172.21.15.184:0/755235236 shutdown_connections 2024-05-20T16:05:29.309 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:29.307+0000 7f9c0a3c9700 1 --2- 172.21.15.184:0/755235236 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9c041012e0 0x7f9c041016f0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:05:29.309 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:29.307+0000 7f9c0a3c9700 1 -- 172.21.15.184:0/755235236 >> 172.21.15.184:0/755235236 conn(0x7f9c04076bb0 msgr2=0x7f9c04076ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:05:29.309 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:29.307+0000 7f9c0a3c9700 1 -- 172.21.15.184:0/755235236 shutdown_connections 2024-05-20T16:05:29.309 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:29.307+0000 7f9c0a3c9700 1 -- 172.21.15.184:0/755235236 wait complete. 2024-05-20T16:05:29.309 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:29.308+0000 7f9c0a3c9700 1 Processor -- start 2024-05-20T16:05:29.309 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:29.308+0000 7f9c0a3c9700 1 -- start start 2024-05-20T16:05:29.310 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:29.308+0000 7f9c0a3c9700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9c041012e0 0x7f9c041977b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:05:29.310 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:29.308+0000 7f9c0a3c9700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f9c0410b720 con 0x7f9c041012e0 2024-05-20T16:05:29.310 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:29.309+0000 7f9c03fff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9c041012e0 0x7f9c041977b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:05:29.310 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:29.309+0000 7f9c03fff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9c041012e0 0x7f9c041977b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.184:41152/0 (socket says 172.21.15.184:41152) 2024-05-20T16:05:29.310 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:29.309+0000 7f9c03fff700 1 -- 172.21.15.184:0/638210720 learned_addr learned my addr 172.21.15.184:0/638210720 (peer_addr_for_me v2:172.21.15.184:0/0) 2024-05-20T16:05:29.310 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:29.309+0000 7f9c03fff700 1 -- 172.21.15.184:0/638210720 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c04197cf0 con 0x7f9c041012e0 2024-05-20T16:05:29.310 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:29.309+0000 7f9c03fff700 1 --2- 172.21.15.184:0/638210720 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9c041012e0 0x7f9c041977b0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f9bf8005850 tx=0x7f9bf8005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:05:29.311 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:29.309+0000 7f9c017fa700 1 -- 172.21.15.184:0/638210720 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9bf8026070 con 0x7f9c041012e0 2024-05-20T16:05:29.311 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:29.309+0000 7f9c0a3c9700 1 -- 172.21.15.184:0/638210720 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c04197fe0 con 0x7f9c041012e0 2024-05-20T16:05:29.312 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:29.310+0000 7f9c017fa700 1 -- 172.21.15.184:0/638210720 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9bf800bbd0 con 0x7f9c041012e0 2024-05-20T16:05:29.312 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:29.310+0000 7f9c0a3c9700 1 -- 172.21.15.184:0/638210720 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c0419bfe0 con 0x7f9c041012e0 2024-05-20T16:05:29.312 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:29.310+0000 7f9c017fa700 1 -- 172.21.15.184:0/638210720 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9bf8030060 con 0x7f9c041012e0 2024-05-20T16:05:29.312 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:29.310+0000 7f9c017fa700 1 -- 172.21.15.184:0/638210720 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f9bf8029070 con 0x7f9c041012e0 2024-05-20T16:05:29.312 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:29.311+0000 7f9c017fa700 1 --2- 172.21.15.184:0/638210720 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f9bec0390f0 0x7f9bec03b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:05:29.312 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:29.311+0000 7f9c0a3c9700 1 -- 172.21.15.184:0/638210720 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c04102790 con 0x7f9c041012e0 2024-05-20T16:05:29.312 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:29.311+0000 7f9c017fa700 1 -- 172.21.15.184:0/638210720 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f9bf801e070 con 0x7f9c041012e0 2024-05-20T16:05:29.312 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:29.311+0000 7f9c037fe700 1 --2- 172.21.15.184:0/638210720 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f9bec0390f0 0x7f9bec03b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:05:29.312 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:29.311+0000 7f9c037fe700 1 --2- 172.21.15.184:0/638210720 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f9bec0390f0 0x7f9bec03b600 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f9bf4006fd0 tx=0x7f9bf4006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:05:29.317 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:29.317+0000 7f9c017fa700 1 -- 172.21.15.184:0/638210720 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f9bf8013070 con 0x7f9c041012e0 2024-05-20T16:05:29.601 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:29.600+0000 7f9c0a3c9700 1 -- 172.21.15.184:0/638210720 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f9c04063ca0 con 0x7f9c041012e0 2024-05-20T16:05:29.601 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:29.601+0000 7f9c017fa700 1 -- 172.21.15.184:0/638210720 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+745 (secure 0 0 0) 0x7f9bf802ac70 con 0x7f9c041012e0 2024-05-20T16:05:29.601 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-05-20T16:05:29.601 INFO:teuthology.orchestra.run.smithi184.stdout:{"epoch":1,"fsid":"59d1600a-16c2-11ef-bc9a-c7b262605968","modified":"2024-05-20T16:03:43.664950Z","created":"2024-05-20T16:03:43.664950Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:3300","nonce":0},{"type":"v1","addr":"172.21.15.82:6789","nonce":0}]},"addr":"172.21.15.82:6789/0","public_addr":"172.21.15.82:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-05-20T16:05:29.603 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:29.603+0000 7f9c0a3c9700 1 -- 172.21.15.184:0/638210720 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f9bec0390f0 msgr2=0x7f9bec03b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:05:29.603 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:29.603+0000 7f9c0a3c9700 1 --2- 172.21.15.184:0/638210720 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f9bec0390f0 0x7f9bec03b600 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f9bf4006fd0 tx=0x7f9bf4006e40 comp rx=0 tx=0).stop 2024-05-20T16:05:29.604 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:29.603+0000 7f9c0a3c9700 1 -- 172.21.15.184:0/638210720 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9c041012e0 msgr2=0x7f9c041977b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:05:29.604 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:29.603+0000 7f9c0a3c9700 1 --2- 172.21.15.184:0/638210720 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9c041012e0 0x7f9c041977b0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f9bf8005850 tx=0x7f9bf8005030 comp rx=0 tx=0).stop 2024-05-20T16:05:29.604 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:29.603+0000 7f9c0a3c9700 1 -- 172.21.15.184:0/638210720 shutdown_connections 2024-05-20T16:05:29.604 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:29.603+0000 7f9c0a3c9700 1 --2- 172.21.15.184:0/638210720 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f9bec0390f0 0x7f9bec03b600 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:05:29.604 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:29.603+0000 7f9c0a3c9700 1 --2- 172.21.15.184:0/638210720 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9c041012e0 0x7f9c041977b0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:05:29.604 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:29.603+0000 7f9c0a3c9700 1 -- 172.21.15.184:0/638210720 >> 172.21.15.184:0/638210720 conn(0x7f9c04076bb0 msgr2=0x7f9c040ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:05:29.604 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:29.604+0000 7f9c0a3c9700 1 -- 172.21.15.184:0/638210720 shutdown_connections 2024-05-20T16:05:29.604 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:29.604+0000 7f9c0a3c9700 1 -- 172.21.15.184:0/638210720 wait complete. 2024-05-20T16:05:29.604 INFO:teuthology.orchestra.run.smithi184.stderr:dumped monmap epoch 1 2024-05-20T16:05:30.076 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:05:29 smithi184 systemd[1]: Started Ceph mon.c for 59d1600a-16c2-11ef-bc9a-c7b262605968. 2024-05-20T16:05:31.438 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-05-20T16:05:31.438 DEBUG:teuthology.orchestra.run.smithi184:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 59d1600a-16c2-11ef-bc9a-c7b262605968 -- ceph mon dump -f json 2024-05-20T16:05:31.682 INFO:teuthology.orchestra.run.smithi184.stderr:Inferring config /var/lib/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/mon.c/config 2024-05-20T16:05:35.360 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:34 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-20T16:05:35.360 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:34 smithi082 ceph-mon[32238]: mon.a calling monitor election 2024-05-20T16:05:35.360 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:34 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-20T16:05:35.360 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:34 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-20T16:05:35.360 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:34 smithi082 ceph-mon[32238]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-20T16:05:35.360 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:34 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-20T16:05:35.360 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:34 smithi082 ceph-mon[32238]: mon.c calling monitor election 2024-05-20T16:05:35.360 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:34 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-20T16:05:35.360 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:34 smithi082 ceph-mon[32238]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-20T16:05:35.361 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:34 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-20T16:05:35.361 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:34 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-20T16:05:35.361 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:34 smithi082 ceph-mon[32238]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-05-20T16:05:35.361 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:34 smithi082 ceph-mon[32238]: monmap epoch 2 2024-05-20T16:05:35.361 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:34 smithi082 ceph-mon[32238]: fsid 59d1600a-16c2-11ef-bc9a-c7b262605968 2024-05-20T16:05:35.361 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:34 smithi082 ceph-mon[32238]: last_changed 2024-05-20T16:05:29.893233+0000 2024-05-20T16:05:35.361 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:34 smithi082 ceph-mon[32238]: created 2024-05-20T16:03:43.664950+0000 2024-05-20T16:05:35.361 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:34 smithi082 ceph-mon[32238]: min_mon_release 19 (squid) 2024-05-20T16:05:35.361 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:34 smithi082 ceph-mon[32238]: election_strategy: 1 2024-05-20T16:05:35.361 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:34 smithi082 ceph-mon[32238]: 0: [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] mon.a 2024-05-20T16:05:35.361 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:34 smithi082 ceph-mon[32238]: 1: [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] mon.c 2024-05-20T16:05:35.361 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:34 smithi082 ceph-mon[32238]: fsmap 2024-05-20T16:05:35.361 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:34 smithi082 ceph-mon[32238]: osdmap e4: 0 total, 0 up, 0 in 2024-05-20T16:05:35.361 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:34 smithi082 ceph-mon[32238]: mgrmap e13: x(active, since 49s) 2024-05-20T16:05:35.361 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:34 smithi082 ceph-mon[32238]: overall HEALTH_OK 2024-05-20T16:05:35.362 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:34 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:05:35.362 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:34 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:05:35.362 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:34 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:05:35.362 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:34 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-20T16:05:35.362 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:34 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:05:35.386 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:05:34 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-20T16:05:35.386 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:05:34 smithi184 ceph-mon[36209]: mon.a calling monitor election 2024-05-20T16:05:35.386 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:05:34 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-20T16:05:35.387 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:05:34 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-20T16:05:35.387 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:05:34 smithi184 ceph-mon[36209]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-20T16:05:35.387 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:05:34 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-20T16:05:35.387 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:05:34 smithi184 ceph-mon[36209]: mon.c calling monitor election 2024-05-20T16:05:35.387 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:05:34 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-20T16:05:35.387 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:05:34 smithi184 ceph-mon[36209]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-20T16:05:35.387 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:05:34 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-20T16:05:35.387 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:05:34 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-20T16:05:35.387 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:05:34 smithi184 ceph-mon[36209]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-05-20T16:05:35.387 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:05:34 smithi184 ceph-mon[36209]: monmap epoch 2 2024-05-20T16:05:35.387 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:05:34 smithi184 ceph-mon[36209]: fsid 59d1600a-16c2-11ef-bc9a-c7b262605968 2024-05-20T16:05:35.387 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:05:34 smithi184 ceph-mon[36209]: last_changed 2024-05-20T16:05:29.893233+0000 2024-05-20T16:05:35.387 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:05:34 smithi184 ceph-mon[36209]: created 2024-05-20T16:03:43.664950+0000 2024-05-20T16:05:35.387 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:05:34 smithi184 ceph-mon[36209]: min_mon_release 19 (squid) 2024-05-20T16:05:35.387 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:05:34 smithi184 ceph-mon[36209]: election_strategy: 1 2024-05-20T16:05:35.388 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:05:34 smithi184 ceph-mon[36209]: 0: [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] mon.a 2024-05-20T16:05:35.388 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:05:34 smithi184 ceph-mon[36209]: 1: [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] mon.c 2024-05-20T16:05:35.388 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:05:34 smithi184 ceph-mon[36209]: fsmap 2024-05-20T16:05:35.388 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:05:34 smithi184 ceph-mon[36209]: osdmap e4: 0 total, 0 up, 0 in 2024-05-20T16:05:35.388 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:05:34 smithi184 ceph-mon[36209]: mgrmap e13: x(active, since 49s) 2024-05-20T16:05:35.388 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:05:34 smithi184 ceph-mon[36209]: overall HEALTH_OK 2024-05-20T16:05:35.388 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:05:34 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:05:35.388 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:05:34 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:05:35.388 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:05:34 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:05:35.388 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:05:34 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-20T16:05:35.388 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:05:34 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:05:36.023 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:36.021+0000 7f5a22afd700 1 -- 172.21.15.184:0/1277350453 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5a0c0035c0 msgr2=0x7f5a0c005aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:05:36.023 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:36.021+0000 7f5a22afd700 1 --2- 172.21.15.184:0/1277350453 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5a0c0035c0 0x7f5a0c005aa0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f5a1c1029f0 tx=0x7f5a1000feb0 comp rx=0 tx=0).stop 2024-05-20T16:05:36.023 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:36.022+0000 7f5a22afd700 1 -- 172.21.15.184:0/1277350453 shutdown_connections 2024-05-20T16:05:36.023 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:36.022+0000 7f5a22afd700 1 --2- 172.21.15.184:0/1277350453 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5a0c0035c0 0x7f5a0c005aa0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:05:36.024 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:36.022+0000 7f5a22afd700 1 -- 172.21.15.184:0/1277350453 >> 172.21.15.184:0/1277350453 conn(0x7f5a1c0fe620 msgr2=0x7f5a1c100a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:05:36.024 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:36.022+0000 7f5a22afd700 1 -- 172.21.15.184:0/1277350453 shutdown_connections 2024-05-20T16:05:36.024 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:36.022+0000 7f5a22afd700 1 -- 172.21.15.184:0/1277350453 wait complete. 2024-05-20T16:05:36.024 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:36.023+0000 7f5a22afd700 1 Processor -- start 2024-05-20T16:05:36.024 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:36.023+0000 7f5a22afd700 1 -- start start 2024-05-20T16:05:36.025 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:36.023+0000 7f5a22afd700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5a0c0035c0 0x7f5a1c19e660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:05:36.025 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:36.023+0000 7f5a22afd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5a1c19eba0 0x7f5a1c198880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:05:36.026 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:36.023+0000 7f5a22afd700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f5a1c10abf0 con 0x7f5a1c19eba0 2024-05-20T16:05:36.026 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:36.023+0000 7f5a22afd700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f5a1c10aa70 con 0x7f5a0c0035c0 2024-05-20T16:05:36.026 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:36.023+0000 7f5a20899700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5a0c0035c0 0x7f5a1c19e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:05:36.026 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:36.023+0000 7f5a20899700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5a0c0035c0 0x7f5a1c19e660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.184:50472/0 (socket says 172.21.15.184:50472) 2024-05-20T16:05:36.026 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:36.023+0000 7f5a20899700 1 -- 172.21.15.184:0/608549852 learned_addr learned my addr 172.21.15.184:0/608549852 (peer_addr_for_me v2:172.21.15.184:0/0) 2024-05-20T16:05:36.026 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:36.023+0000 7f5a1bfff700 1 --2- 172.21.15.184:0/608549852 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5a1c19eba0 0x7f5a1c198880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:05:36.026 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:36.023+0000 7f5a20899700 1 -- 172.21.15.184:0/608549852 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5a0c0035c0 msgr2=0x7f5a1c19e660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-05-20T16:05:36.026 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:36.023+0000 7f5a20899700 1 -- 172.21.15.184:0/608549852 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5a0c0035c0 msgr2=0x7f5a1c19e660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-20T16:05:36.026 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:36.023+0000 7f5a20899700 1 --2- 172.21.15.184:0/608549852 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5a0c0035c0 0x7f5a1c19e660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-20T16:05:36.026 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:36.023+0000 7f5a20899700 1 --2- 172.21.15.184:0/608549852 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5a0c0035c0 0x7f5a1c19e660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-20T16:05:36.026 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:36.024+0000 7f5a1bfff700 1 -- 172.21.15.184:0/608549852 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5a0c0035c0 msgr2=0x7f5a1c19e660 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-20T16:05:36.026 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:36.024+0000 7f5a1bfff700 1 --2- 172.21.15.184:0/608549852 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5a0c0035c0 0x7f5a1c19e660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:05:36.027 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:36.024+0000 7f5a1bfff700 1 -- 172.21.15.184:0/608549852 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a1c19f170 con 0x7f5a1c19eba0 2024-05-20T16:05:36.027 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:36.024+0000 7f5a1bfff700 1 --2- 172.21.15.184:0/608549852 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5a1c19eba0 0x7f5a1c198880 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f5a0800b4d0 tx=0x7f5a0800b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:05:36.027 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:36.024+0000 7f5a19ffb700 1 -- 172.21.15.184:0/608549852 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f5a08011630 con 0x7f5a1c19eba0 2024-05-20T16:05:36.027 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:36.024+0000 7f5a22afd700 1 -- 172.21.15.184:0/608549852 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a1c198e20 con 0x7f5a1c19eba0 2024-05-20T16:05:36.027 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:36.024+0000 7f5a19ffb700 1 -- 172.21.15.184:0/608549852 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5a08011cb0 con 0x7f5a1c19eba0 2024-05-20T16:05:36.027 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:36.024+0000 7f5a19ffb700 1 -- 172.21.15.184:0/608549852 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f5a08010e60 con 0x7f5a1c19eba0 2024-05-20T16:05:36.027 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:36.025+0000 7f5a22afd700 1 -- 172.21.15.184:0/608549852 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a1c1993b0 con 0x7f5a1c19eba0 2024-05-20T16:05:36.028 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:36.025+0000 7f5a19ffb700 1 -- 172.21.15.184:0/608549852 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f5a08018020 con 0x7f5a1c19eba0 2024-05-20T16:05:36.028 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:36.026+0000 7f5a19ffb700 1 --2- 172.21.15.184:0/608549852 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f5a00038e90 0x7f5a0003b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:05:36.028 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:36.026+0000 7f5a19ffb700 1 -- 172.21.15.184:0/608549852 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f5a0804cb80 con 0x7f5a1c19eba0 2024-05-20T16:05:36.028 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:36.026+0000 7f5a20899700 1 --2- 172.21.15.184:0/608549852 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f5a00038e90 0x7f5a0003b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:05:36.028 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:36.026+0000 7f5a22afd700 1 -- 172.21.15.184:0/608549852 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a1c1996c0 con 0x7f5a1c19eba0 2024-05-20T16:05:36.028 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:36.027+0000 7f5a20899700 1 --2- 172.21.15.184:0/608549852 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f5a00038e90 0x7f5a0003b3a0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f5a1000fa60 tx=0x7f5a10005e60 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:05:36.033 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:36.032+0000 7f5a19ffb700 1 -- 172.21.15.184:0/608549852 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f5a0801f020 con 0x7f5a1c19eba0 2024-05-20T16:05:36.320 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:36.319+0000 7f5a22afd700 1 -- 172.21.15.184:0/608549852 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f5a1c199db0 con 0x7f5a1c19eba0 2024-05-20T16:05:36.320 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:05:35 smithi184 ceph-mon[36209]: Deploying daemon mon.b on smithi160 2024-05-20T16:05:36.320 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:05:35 smithi184 ceph-mon[36209]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-20T16:05:36.320 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:05:35 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-20T16:05:36.321 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:36.320+0000 7f5a19ffb700 1 -- 172.21.15.184:0/608549852 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1015 (secure 0 0 0) 0x7f5a08018310 con 0x7f5a1c19eba0 2024-05-20T16:05:36.321 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-05-20T16:05:36.321 INFO:teuthology.orchestra.run.smithi184.stdout:{"epoch":2,"fsid":"59d1600a-16c2-11ef-bc9a-c7b262605968","modified":"2024-05-20T16:05:29.893233Z","created":"2024-05-20T16:03:43.664950Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:3300","nonce":0},{"type":"v1","addr":"172.21.15.82:6789","nonce":0}]},"addr":"172.21.15.82:6789/0","public_addr":"172.21.15.82:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:3300","nonce":0},{"type":"v1","addr":"172.21.15.184:6789","nonce":0}]},"addr":"172.21.15.184:6789/0","public_addr":"172.21.15.184:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-05-20T16:05:36.323 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:36.322+0000 7f5a22afd700 1 -- 172.21.15.184:0/608549852 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f5a00038e90 msgr2=0x7f5a0003b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:05:36.323 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:36.322+0000 7f5a22afd700 1 --2- 172.21.15.184:0/608549852 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f5a00038e90 0x7f5a0003b3a0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f5a1000fa60 tx=0x7f5a10005e60 comp rx=0 tx=0).stop 2024-05-20T16:05:36.323 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:36.322+0000 7f5a22afd700 1 -- 172.21.15.184:0/608549852 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5a1c19eba0 msgr2=0x7f5a1c198880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:05:36.323 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:36.322+0000 7f5a22afd700 1 --2- 172.21.15.184:0/608549852 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5a1c19eba0 0x7f5a1c198880 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f5a0800b4d0 tx=0x7f5a0800b890 comp rx=0 tx=0).stop 2024-05-20T16:05:36.323 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:36.322+0000 7f5a22afd700 1 -- 172.21.15.184:0/608549852 shutdown_connections 2024-05-20T16:05:36.323 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:36.322+0000 7f5a22afd700 1 --2- 172.21.15.184:0/608549852 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f5a00038e90 0x7f5a0003b3a0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:05:36.323 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:36.322+0000 7f5a22afd700 1 --2- 172.21.15.184:0/608549852 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5a0c0035c0 0x7f5a1c19e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:05:36.323 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:36.322+0000 7f5a22afd700 1 --2- 172.21.15.184:0/608549852 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5a1c19eba0 0x7f5a1c198880 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:05:36.323 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:36.322+0000 7f5a22afd700 1 -- 172.21.15.184:0/608549852 >> 172.21.15.184:0/608549852 conn(0x7f5a1c0fe620 msgr2=0x7f5a1c105ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:05:36.323 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:36.322+0000 7f5a22afd700 1 -- 172.21.15.184:0/608549852 shutdown_connections 2024-05-20T16:05:36.323 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:36.322+0000 7f5a22afd700 1 -- 172.21.15.184:0/608549852 wait complete. 2024-05-20T16:05:36.324 INFO:teuthology.orchestra.run.smithi184.stderr:dumped monmap epoch 2 2024-05-20T16:05:36.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:35 smithi082 ceph-mon[32238]: Deploying daemon mon.b on smithi160 2024-05-20T16:05:36.360 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:35 smithi082 ceph-mon[32238]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-20T16:05:36.360 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:35 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-20T16:05:37.136 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:05:36 smithi184 ceph-mon[36209]: from='client.? 172.21.15.184:0/608549852' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-20T16:05:37.360 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:36 smithi082 ceph-mon[32238]: from='client.? 172.21.15.184:0/608549852' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-20T16:05:37.360 INFO:journalctl@ceph.mgr.x.smithi082.stdout:May 20 16:05:36 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mgr-x[32467]: 2024-05-20T16:05:36.892+0000 7f1a88571700 -1 mgr.server handle_report got status from non-daemon mon.c 2024-05-20T16:05:37.886 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-05-20T16:05:37.886 DEBUG:teuthology.orchestra.run.smithi184:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 59d1600a-16c2-11ef-bc9a-c7b262605968 -- ceph mon dump -f json 2024-05-20T16:05:38.130 INFO:teuthology.orchestra.run.smithi184.stderr:Inferring config /var/lib/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/mon.c/config 2024-05-20T16:05:38.243 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:05:37 smithi184 ceph-mon[36209]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-20T16:05:38.261 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:37 smithi082 ceph-mon[32238]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-20T16:05:38.911 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: rocksdb: DB SUMMARY 2024-05-20T16:05:38.911 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: rocksdb: DB Session ID: 3FK4VVAC7POLB7SEMN3P 2024-05-20T16:05:38.911 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: rocksdb: CURRENT file: CURRENT 2024-05-20T16:05:38.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: rocksdb: IDENTITY file: IDENTITY 2024-05-20T16:05:38.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-05-20T16:05:38.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2024-05-20T16:05:38.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000004.log size: 636 ; 2024-05-20T16:05:38.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: rocksdb: Options.error_if_exists: 0 2024-05-20T16:05:38.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: rocksdb: Options.create_if_missing: 0 2024-05-20T16:05:38.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: rocksdb: Options.paranoid_checks: 1 2024-05-20T16:05:38.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: rocksdb: Options.flush_verify_memtable_count: 1 2024-05-20T16:05:38.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-05-20T16:05:38.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-05-20T16:05:38.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: rocksdb: Options.env: 0x55727b51f4a0 2024-05-20T16:05:38.913 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: rocksdb: Options.fs: PosixFileSystem 2024-05-20T16:05:38.913 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: rocksdb: Options.info_log: 0x55727ccfbd40 2024-05-20T16:05:39.314 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: rocksdb: Options.max_file_opening_threads: 16 2024-05-20T16:05:39.314 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: rocksdb: Options.statistics: (nil) 2024-05-20T16:05:39.314 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: rocksdb: Options.use_fsync: 0 2024-05-20T16:05:39.314 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: rocksdb: Options.max_log_file_size: 0 2024-05-20T16:05:39.314 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-05-20T16:05:39.314 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: rocksdb: Options.log_file_time_to_roll: 0 2024-05-20T16:05:39.314 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: rocksdb: Options.keep_log_file_num: 1000 2024-05-20T16:05:39.314 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: rocksdb: Options.recycle_log_file_num: 0 2024-05-20T16:05:39.314 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: rocksdb: Options.allow_fallocate: 1 2024-05-20T16:05:39.314 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: rocksdb: Options.allow_mmap_reads: 0 2024-05-20T16:05:39.315 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: rocksdb: Options.allow_mmap_writes: 0 2024-05-20T16:05:39.315 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: rocksdb: Options.use_direct_reads: 0 2024-05-20T16:05:39.315 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-05-20T16:05:39.315 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: rocksdb: Options.create_missing_column_families: 0 2024-05-20T16:05:39.315 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: rocksdb: Options.db_log_dir: 2024-05-20T16:05:39.315 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: rocksdb: Options.wal_dir: 2024-05-20T16:05:39.315 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: rocksdb: Options.table_cache_numshardbits: 6 2024-05-20T16:05:39.315 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: rocksdb: Options.WAL_ttl_seconds: 0 2024-05-20T16:05:39.315 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: rocksdb: Options.WAL_size_limit_MB: 0 2024-05-20T16:05:39.315 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-05-20T16:05:39.315 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-05-20T16:05:39.315 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: rocksdb: Options.is_fd_close_on_exec: 1 2024-05-20T16:05:39.315 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: rocksdb: Options.advise_random_on_open: 1 2024-05-20T16:05:39.315 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: rocksdb: Options.db_write_buffer_size: 0 2024-05-20T16:05:39.316 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: rocksdb: Options.write_buffer_manager: 0x55727cccf2c0 2024-05-20T16:05:39.316 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-05-20T16:05:39.316 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-05-20T16:05:39.316 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: rocksdb: Options.use_adaptive_mutex: 0 2024-05-20T16:05:39.316 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: rocksdb: Options.rate_limiter: (nil) 2024-05-20T16:05:39.316 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-05-20T16:05:39.316 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: rocksdb: Options.wal_recovery_mode: 2 2024-05-20T16:05:39.316 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: rocksdb: Options.enable_thread_tracking: 0 2024-05-20T16:05:39.316 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: rocksdb: Options.enable_pipelined_write: 0 2024-05-20T16:05:39.316 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: rocksdb: Options.unordered_write: 0 2024-05-20T16:05:39.316 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-05-20T16:05:39.316 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-05-20T16:05:39.316 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-05-20T16:05:39.316 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-05-20T16:05:39.316 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: rocksdb: Options.row_cache: None 2024-05-20T16:05:39.317 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: rocksdb: Options.wal_filter: None 2024-05-20T16:05:39.317 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-05-20T16:05:39.317 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: rocksdb: Options.allow_ingest_behind: 0 2024-05-20T16:05:39.317 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: rocksdb: Options.two_write_queues: 0 2024-05-20T16:05:39.317 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: rocksdb: Options.manual_wal_flush: 0 2024-05-20T16:05:39.317 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: rocksdb: Options.wal_compression: 0 2024-05-20T16:05:39.317 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: rocksdb: Options.atomic_flush: 0 2024-05-20T16:05:39.317 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-05-20T16:05:39.317 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: rocksdb: Options.persist_stats_to_disk: 0 2024-05-20T16:05:39.317 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: rocksdb: Options.write_dbid_to_manifest: 0 2024-05-20T16:05:39.317 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: rocksdb: Options.log_readahead_size: 0 2024-05-20T16:05:39.317 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-05-20T16:05:39.317 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: rocksdb: Options.best_efforts_recovery: 0 2024-05-20T16:05:39.317 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-05-20T16:05:39.317 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-05-20T16:05:39.318 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: rocksdb: Options.allow_data_in_errors: 0 2024-05-20T16:05:39.318 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: rocksdb: Options.db_host_id: __hostname__ 2024-05-20T16:05:39.318 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: rocksdb: Options.enforce_single_del_contracts: true 2024-05-20T16:05:39.318 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: rocksdb: Options.max_background_jobs: 2 2024-05-20T16:05:39.318 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: rocksdb: Options.max_background_compactions: -1 2024-05-20T16:05:39.318 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: rocksdb: Options.max_subcompactions: 1 2024-05-20T16:05:39.318 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-05-20T16:05:39.318 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-05-20T16:05:39.318 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: rocksdb: Options.delayed_write_rate : 16777216 2024-05-20T16:05:39.318 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: rocksdb: Options.max_total_wal_size: 0 2024-05-20T16:05:39.318 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-05-20T16:05:39.318 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: rocksdb: Options.stats_dump_period_sec: 600 2024-05-20T16:05:39.318 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: rocksdb: Options.stats_persist_period_sec: 600 2024-05-20T16:05:39.318 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-05-20T16:05:39.319 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: rocksdb: Options.max_open_files: -1 2024-05-20T16:05:39.319 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: rocksdb: Options.bytes_per_sync: 0 2024-05-20T16:05:39.319 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: rocksdb: Options.wal_bytes_per_sync: 0 2024-05-20T16:05:39.319 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: rocksdb: Options.strict_bytes_per_sync: 0 2024-05-20T16:05:39.319 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: rocksdb: Options.compaction_readahead_size: 0 2024-05-20T16:05:39.319 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: rocksdb: Options.max_background_flushes: -1 2024-05-20T16:05:39.319 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: rocksdb: Compression algorithms supported: 2024-05-20T16:05:39.319 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-05-20T16:05:39.319 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: rocksdb: kZSTD supported: 0 2024-05-20T16:05:39.319 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: rocksdb: kXpressCompression supported: 0 2024-05-20T16:05:39.319 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: rocksdb: kLZ4HCCompression supported: 1 2024-05-20T16:05:39.319 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: rocksdb: kZlibCompression supported: 1 2024-05-20T16:05:39.319 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: rocksdb: kSnappyCompression supported: 1 2024-05-20T16:05:39.319 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: rocksdb: kLZ4Compression supported: 1 2024-05-20T16:05:39.320 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: rocksdb: kBZip2Compression supported: 0 2024-05-20T16:05:39.320 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: rocksdb: Fast CRC32 supported: Supported on x86 2024-05-20T16:05:39.320 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: rocksdb: DMutex implementation: pthread_mutex_t 2024-05-20T16:05:39.320 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 2024-05-20T16:05:39.320 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-05-20T16:05:39.320 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-05-20T16:05:39.320 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: rocksdb: Options.merge_operator: 2024-05-20T16:05:39.320 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: rocksdb: Options.compaction_filter: None 2024-05-20T16:05:39.320 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: rocksdb: Options.compaction_filter_factory: None 2024-05-20T16:05:39.320 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: rocksdb: Options.sst_partitioner_factory: None 2024-05-20T16:05:39.320 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: rocksdb: Options.memtable_factory: SkipListFactory 2024-05-20T16:05:39.320 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: rocksdb: Options.table_factory: BlockBasedTable 2024-05-20T16:05:39.320 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55727ccfbe40) 2024-05-20T16:05:39.320 INFO:journalctl@ceph.mon.b.smithi160.stdout: cache_index_and_filter_blocks: 1 2024-05-20T16:05:39.321 INFO:journalctl@ceph.mon.b.smithi160.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-05-20T16:05:39.321 INFO:journalctl@ceph.mon.b.smithi160.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-05-20T16:05:39.321 INFO:journalctl@ceph.mon.b.smithi160.stdout: pin_top_level_index_and_filter: 1 2024-05-20T16:05:39.321 INFO:journalctl@ceph.mon.b.smithi160.stdout: index_type: 0 2024-05-20T16:05:39.321 INFO:journalctl@ceph.mon.b.smithi160.stdout: data_block_index_type: 0 2024-05-20T16:05:39.321 INFO:journalctl@ceph.mon.b.smithi160.stdout: index_shortening: 1 2024-05-20T16:05:39.321 INFO:journalctl@ceph.mon.b.smithi160.stdout: data_block_hash_table_util_ratio: 0.750000 2024-05-20T16:05:39.321 INFO:journalctl@ceph.mon.b.smithi160.stdout: checksum: 4 2024-05-20T16:05:39.321 INFO:journalctl@ceph.mon.b.smithi160.stdout: no_block_cache: 0 2024-05-20T16:05:39.321 INFO:journalctl@ceph.mon.b.smithi160.stdout: block_cache: 0x55727ccff090 2024-05-20T16:05:39.321 INFO:journalctl@ceph.mon.b.smithi160.stdout: block_cache_name: BinnedLRUCache 2024-05-20T16:05:39.321 INFO:journalctl@ceph.mon.b.smithi160.stdout: block_cache_options: 2024-05-20T16:05:39.321 INFO:journalctl@ceph.mon.b.smithi160.stdout: capacity : 536870912 2024-05-20T16:05:39.321 INFO:journalctl@ceph.mon.b.smithi160.stdout: num_shard_bits : 4 2024-05-20T16:05:39.322 INFO:journalctl@ceph.mon.b.smithi160.stdout: strict_capacity_limit : 0 2024-05-20T16:05:39.322 INFO:journalctl@ceph.mon.b.smithi160.stdout: high_pri_pool_ratio: 0.000 2024-05-20T16:05:39.322 INFO:journalctl@ceph.mon.b.smithi160.stdout: block_cache_compressed: (nil) 2024-05-20T16:05:39.322 INFO:journalctl@ceph.mon.b.smithi160.stdout: persistent_cache: (nil) 2024-05-20T16:05:39.322 INFO:journalctl@ceph.mon.b.smithi160.stdout: block_size: 4096 2024-05-20T16:05:39.322 INFO:journalctl@ceph.mon.b.smithi160.stdout: block_size_deviation: 10 2024-05-20T16:05:39.322 INFO:journalctl@ceph.mon.b.smithi160.stdout: block_restart_interval: 16 2024-05-20T16:05:39.322 INFO:journalctl@ceph.mon.b.smithi160.stdout: index_block_restart_interval: 1 2024-05-20T16:05:39.322 INFO:journalctl@ceph.mon.b.smithi160.stdout: metadata_block_size: 4096 2024-05-20T16:05:39.322 INFO:journalctl@ceph.mon.b.smithi160.stdout: partition_filters: 0 2024-05-20T16:05:39.322 INFO:journalctl@ceph.mon.b.smithi160.stdout: use_delta_encoding: 1 2024-05-20T16:05:39.322 INFO:journalctl@ceph.mon.b.smithi160.stdout: filter_policy: bloomfilter 2024-05-20T16:05:39.322 INFO:journalctl@ceph.mon.b.smithi160.stdout: whole_key_filtering: 1 2024-05-20T16:05:39.323 INFO:journalctl@ceph.mon.b.smithi160.stdout: verify_compression: 0 2024-05-20T16:05:39.323 INFO:journalctl@ceph.mon.b.smithi160.stdout: read_amp_bytes_per_bit: 0 2024-05-20T16:05:39.323 INFO:journalctl@ceph.mon.b.smithi160.stdout: format_version: 5 2024-05-20T16:05:39.323 INFO:journalctl@ceph.mon.b.smithi160.stdout: enable_index_compression: 1 2024-05-20T16:05:39.323 INFO:journalctl@ceph.mon.b.smithi160.stdout: block_align: 0 2024-05-20T16:05:39.323 INFO:journalctl@ceph.mon.b.smithi160.stdout: max_auto_readahead_size: 262144 2024-05-20T16:05:39.323 INFO:journalctl@ceph.mon.b.smithi160.stdout: prepopulate_block_cache: 0 2024-05-20T16:05:39.324 INFO:journalctl@ceph.mon.b.smithi160.stdout: initial_auto_readahead_size: 8192 2024-05-20T16:05:39.324 INFO:journalctl@ceph.mon.b.smithi160.stdout: num_file_reads_for_auto_readahead: 2 2024-05-20T16:05:39.324 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: rocksdb: Options.write_buffer_size: 33554432 2024-05-20T16:05:39.324 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: rocksdb: Options.max_write_buffer_number: 2 2024-05-20T16:05:39.324 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: rocksdb: Options.compression: NoCompression 2024-05-20T16:05:39.324 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: rocksdb: Options.bottommost_compression: Disabled 2024-05-20T16:05:39.324 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: rocksdb: Options.prefix_extractor: nullptr 2024-05-20T16:05:39.325 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-05-20T16:05:39.325 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: rocksdb: Options.num_levels: 7 2024-05-20T16:05:39.325 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-05-20T16:05:39.325 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-05-20T16:05:39.325 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-05-20T16:05:39.325 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-05-20T16:05:39.325 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-05-20T16:05:39.325 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-05-20T16:05:39.325 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-05-20T16:05:39.325 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-05-20T16:05:39.325 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-05-20T16:05:39.325 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-05-20T16:05:39.325 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-05-20T16:05:39.325 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-05-20T16:05:39.325 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: rocksdb: Options.compression_opts.window_bits: -14 2024-05-20T16:05:39.326 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: rocksdb: Options.compression_opts.level: 32767 2024-05-20T16:05:39.326 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: rocksdb: Options.compression_opts.strategy: 0 2024-05-20T16:05:39.326 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-05-20T16:05:39.326 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-05-20T16:05:39.326 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-05-20T16:05:39.326 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-05-20T16:05:39.326 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: rocksdb: Options.compression_opts.enabled: false 2024-05-20T16:05:39.326 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-05-20T16:05:39.326 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-05-20T16:05:39.326 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-05-20T16:05:39.326 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-05-20T16:05:39.326 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: rocksdb: Options.target_file_size_base: 67108864 2024-05-20T16:05:39.326 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: rocksdb: Options.target_file_size_multiplier: 1 2024-05-20T16:05:39.326 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-05-20T16:05:39.327 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-05-20T16:05:39.327 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-05-20T16:05:39.327 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-05-20T16:05:39.327 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-05-20T16:05:39.327 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-05-20T16:05:39.327 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-05-20T16:05:39.327 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-05-20T16:05:39.327 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-05-20T16:05:39.327 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-05-20T16:05:39.327 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-05-20T16:05:39.327 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-05-20T16:05:39.327 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-05-20T16:05:39.327 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: rocksdb: Options.arena_block_size: 1048576 2024-05-20T16:05:39.327 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-05-20T16:05:39.328 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-05-20T16:05:39.328 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: rocksdb: Options.disable_auto_compactions: 0 2024-05-20T16:05:39.328 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-05-20T16:05:39.328 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-05-20T16:05:39.328 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-05-20T16:05:39.328 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-05-20T16:05:39.328 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-05-20T16:05:39.328 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-05-20T16:05:39.328 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-05-20T16:05:39.328 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-05-20T16:05:39.328 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-05-20T16:05:39.328 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-05-20T16:05:39.328 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: rocksdb: Options.table_properties_collectors: 2024-05-20T16:05:39.328 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: rocksdb: Options.inplace_update_support: 0 2024-05-20T16:05:39.328 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: rocksdb: Options.inplace_update_num_locks: 10000 2024-05-20T16:05:39.328 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-05-20T16:05:39.328 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-05-20T16:05:39.329 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: rocksdb: Options.memtable_huge_page_size: 0 2024-05-20T16:05:39.329 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: rocksdb: Options.bloom_locality: 0 2024-05-20T16:05:39.329 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: rocksdb: Options.max_successive_merges: 0 2024-05-20T16:05:39.329 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: rocksdb: Options.optimize_filters_for_hits: 0 2024-05-20T16:05:39.329 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: rocksdb: Options.paranoid_file_checks: 0 2024-05-20T16:05:39.329 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: rocksdb: Options.force_consistency_checks: 1 2024-05-20T16:05:39.329 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: rocksdb: Options.report_bg_io_stats: 0 2024-05-20T16:05:39.329 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: rocksdb: Options.ttl: 2592000 2024-05-20T16:05:39.329 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: rocksdb: Options.periodic_compaction_seconds: 0 2024-05-20T16:05:39.329 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-05-20T16:05:39.329 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-05-20T16:05:39.329 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: rocksdb: Options.enable_blob_files: false 2024-05-20T16:05:39.329 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: rocksdb: Options.min_blob_size: 0 2024-05-20T16:05:39.329 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: rocksdb: Options.blob_file_size: 268435456 2024-05-20T16:05:39.329 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: rocksdb: Options.blob_compression_type: NoCompression 2024-05-20T16:05:39.330 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: rocksdb: Options.enable_blob_garbage_collection: false 2024-05-20T16:05:39.330 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-05-20T16:05:39.330 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-05-20T16:05:39.330 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-05-20T16:05:39.330 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: rocksdb: Options.blob_file_starting_level: 0 2024-05-20T16:05:39.330 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-05-20T16:05:39.330 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-05-20T16:05:39.330 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-05-20T16:05:39.330 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 5fd071d3-c7ba-4c27-8ef2-0e0ed48e5c9c 2024-05-20T16:05:39.330 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: rocksdb: EVENT_LOG_v1 {"time_micros": 1716221138912325, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-05-20T16:05:39.330 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-05-20T16:05:39.330 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: rocksdb: EVENT_LOG_v1 {"time_micros": 1716221138912783, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1747, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1716221138, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "5fd071d3-c7ba-4c27-8ef2-0e0ed48e5c9c", "db_session_id": "3FK4VVAC7POLB7SEMN3P", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-05-20T16:05:39.330 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: rocksdb: EVENT_LOG_v1 {"time_micros": 1716221138912871, "job": 1, "event": "recovery_finished"} 2024-05-20T16:05:39.331 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-05-20T16:05:39.331 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-05-20T16:05:39.331 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55727cce5c00 2024-05-20T16:05:39.331 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: rocksdb: DB pointer 0x557281ad2000 2024-05-20T16:05:39.331 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-05-20T16:05:39.331 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-05-20T16:05:39.331 INFO:journalctl@ceph.mon.b.smithi160.stdout: ** DB Stats ** 2024-05-20T16:05:39.331 INFO:journalctl@ceph.mon.b.smithi160.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-20T16:05:39.331 INFO:journalctl@ceph.mon.b.smithi160.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-05-20T16:05:39.331 INFO:journalctl@ceph.mon.b.smithi160.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-20T16:05:39.331 INFO:journalctl@ceph.mon.b.smithi160.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-20T16:05:39.331 INFO:journalctl@ceph.mon.b.smithi160.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-05-20T16:05:39.331 INFO:journalctl@ceph.mon.b.smithi160.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-20T16:05:39.331 INFO:journalctl@ceph.mon.b.smithi160.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-20T16:05:39.331 INFO:journalctl@ceph.mon.b.smithi160.stdout: 2024-05-20T16:05:39.331 INFO:journalctl@ceph.mon.b.smithi160.stdout: ** Compaction Stats [default] ** 2024-05-20T16:05:39.332 INFO:journalctl@ceph.mon.b.smithi160.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-20T16:05:39.332 INFO:journalctl@ceph.mon.b.smithi160.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-05-20T16:05:39.332 INFO:journalctl@ceph.mon.b.smithi160.stdout: L0 1/0 1.71 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.9 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-05-20T16:05:39.332 INFO:journalctl@ceph.mon.b.smithi160.stdout: Sum 1/0 1.71 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.9 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-05-20T16:05:39.332 INFO:journalctl@ceph.mon.b.smithi160.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.9 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-05-20T16:05:39.332 INFO:journalctl@ceph.mon.b.smithi160.stdout: 2024-05-20T16:05:39.332 INFO:journalctl@ceph.mon.b.smithi160.stdout: ** Compaction Stats [default] ** 2024-05-20T16:05:39.332 INFO:journalctl@ceph.mon.b.smithi160.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-20T16:05:39.332 INFO:journalctl@ceph.mon.b.smithi160.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-05-20T16:05:39.332 INFO:journalctl@ceph.mon.b.smithi160.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 3.9 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-05-20T16:05:39.332 INFO:journalctl@ceph.mon.b.smithi160.stdout: 2024-05-20T16:05:39.332 INFO:journalctl@ceph.mon.b.smithi160.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-05-20T16:05:39.332 INFO:journalctl@ceph.mon.b.smithi160.stdout: 2024-05-20T16:05:39.332 INFO:journalctl@ceph.mon.b.smithi160.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-20T16:05:39.332 INFO:journalctl@ceph.mon.b.smithi160.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-05-20T16:05:39.333 INFO:journalctl@ceph.mon.b.smithi160.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-05-20T16:05:39.333 INFO:journalctl@ceph.mon.b.smithi160.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-05-20T16:05:39.333 INFO:journalctl@ceph.mon.b.smithi160.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-05-20T16:05:39.333 INFO:journalctl@ceph.mon.b.smithi160.stdout: AddFile(Keys): cumulative 0, interval 0 2024-05-20T16:05:39.333 INFO:journalctl@ceph.mon.b.smithi160.stdout: Cumulative compaction: 0.00 GB write, 0.62 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-20T16:05:39.333 INFO:journalctl@ceph.mon.b.smithi160.stdout: Interval compaction: 0.00 GB write, 0.62 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-20T16:05:39.333 INFO:journalctl@ceph.mon.b.smithi160.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-05-20T16:05:39.333 INFO:journalctl@ceph.mon.b.smithi160.stdout: Block cache BinnedLRUCache@0x55727ccff090#2 capacity: 512.00 MB usage: 0.23 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 8e-06 secs_since: 0 2024-05-20T16:05:39.333 INFO:journalctl@ceph.mon.b.smithi160.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.12 KB,2.23517e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-05-20T16:05:39.333 INFO:journalctl@ceph.mon.b.smithi160.stdout: 2024-05-20T16:05:39.333 INFO:journalctl@ceph.mon.b.smithi160.stdout: ** File Read Latency Histogram By Level [default] ** 2024-05-20T16:05:39.333 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: mon.b does not exist in monmap, will attempt to join an existing cluster 2024-05-20T16:05:39.333 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: using public_addr v2:172.21.15.160:0/0 -> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] 2024-05-20T16:05:39.333 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: starting mon.b rank -1 at public addrs [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] at bind addrs [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid 59d1600a-16c2-11ef-bc9a-c7b262605968 2024-05-20T16:05:39.333 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: mon.b@-1(???) e0 preinit fsid 59d1600a-16c2-11ef-bc9a-c7b262605968 2024-05-20T16:05:39.334 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: mon.b@-1(synchronizing).mds e1 new map 2024-05-20T16:05:39.334 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: mon.b@-1(synchronizing).mds e1 print_map 2024-05-20T16:05:39.334 INFO:journalctl@ceph.mon.b.smithi160.stdout: e1 2024-05-20T16:05:39.334 INFO:journalctl@ceph.mon.b.smithi160.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-05-20T16:05:39.334 INFO:journalctl@ceph.mon.b.smithi160.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-05-20T16:05:39.334 INFO:journalctl@ceph.mon.b.smithi160.stdout: legacy client fscid: -1 2024-05-20T16:05:39.334 INFO:journalctl@ceph.mon.b.smithi160.stdout: 2024-05-20T16:05:39.334 INFO:journalctl@ceph.mon.b.smithi160.stdout: No filesystems configured 2024-05-20T16:05:39.334 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-05-20T16:05:39.334 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-05-20T16:05:39.334 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-05-20T16:05:39.334 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-05-20T16:05:39.334 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-05-20T16:05:39.334 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-05-20T16:05:39.335 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-05-20T16:05:39.335 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-20T16:05:39.335 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-20T16:05:39.335 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-20T16:05:39.335 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: Added host smithi160 2024-05-20T16:05:39.335 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:05:39.335 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-20T16:05:39.335 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:05:39.335 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-20T16:05:39.335 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-20T16:05:39.335 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi184", "target": ["mon-mgr", ""]}]: dispatch 2024-05-20T16:05:39.335 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: Deploying cephadm binary to smithi184 2024-05-20T16:05:39.335 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:05:39.335 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:05:39.335 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:05:39.336 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:05:39.336 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi160", "name": "osd_memory_target"}]: dispatch 2024-05-20T16:05:39.336 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:05:39.336 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T16:05:39.336 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: Updating smithi160:/etc/ceph/ceph.conf 2024-05-20T16:05:39.336 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-20T16:05:39.336 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: Updating smithi160:/var/lib/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-20T16:05:39.336 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-20T16:05:39.336 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: Updating smithi160:/etc/ceph/ceph.client.admin.keyring 2024-05-20T16:05:39.336 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: Updating smithi160:/var/lib/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/config/ceph.client.admin.keyring 2024-05-20T16:05:39.336 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:05:39.336 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:05:39.336 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:05:39.336 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:05:39.336 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T16:05:39.336 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:05:39.337 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: Added host smithi184 2024-05-20T16:05:39.337 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-20T16:05:39.337 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:05:39.337 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-20T16:05:39.337 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:05:39.337 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-20T16:05:39.337 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-20T16:05:39.337 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: from='client.? 172.21.15.82:0/917057685' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-05-20T16:05:39.337 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:05:39.337 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:05:39.337 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:05:39.337 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:05:39.337 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi184", "name": "osd_memory_target"}]: dispatch 2024-05-20T16:05:39.337 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:05:39.337 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T16:05:39.338 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: Updating smithi184:/etc/ceph/ceph.conf 2024-05-20T16:05:39.338 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: from='client.? 172.21.15.82:0/917057685' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-05-20T16:05:39.338 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: osdmap e4: 0 total, 0 up, 0 in 2024-05-20T16:05:39.338 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:05:39.338 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:05:39.338 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:05:39.338 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: Updating smithi184:/var/lib/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-20T16:05:39.338 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: Updating smithi184:/etc/ceph/ceph.client.admin.keyring 2024-05-20T16:05:39.338 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: Updating smithi184:/var/lib/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/config/ceph.client.admin.keyring 2024-05-20T16:05:39.338 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-20T16:05:39.338 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi082:172.21.15.82=a;smithi160:172.21.15.160=b;smithi184:172.21.15.184=c", "target": ["mon-mgr", ""]}]: dispatch 2024-05-20T16:05:39.338 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: Saving service mon spec with placement smithi082:172.21.15.82=a;smithi160:172.21.15.160=b;smithi184:172.21.15.184=c;count:3 2024-05-20T16:05:39.338 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:05:39.338 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T16:05:39.339 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:05:39.339 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T16:05:39.339 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:05:39.339 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-20T16:05:39.339 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:05:39.339 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: Deploying daemon mon.c on smithi184 2024-05-20T16:05:39.339 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-20T16:05:39.339 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-20T16:05:39.339 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-20T16:05:39.339 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: mon.a calling monitor election 2024-05-20T16:05:39.339 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-20T16:05:39.339 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-20T16:05:39.339 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-20T16:05:39.339 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-20T16:05:39.339 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: mon.c calling monitor election 2024-05-20T16:05:39.340 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-20T16:05:39.340 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-20T16:05:39.340 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-20T16:05:39.340 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-20T16:05:39.340 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-05-20T16:05:39.340 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: monmap epoch 2 2024-05-20T16:05:39.340 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: fsid 59d1600a-16c2-11ef-bc9a-c7b262605968 2024-05-20T16:05:39.340 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: last_changed 2024-05-20T16:05:29.893233+0000 2024-05-20T16:05:39.340 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: created 2024-05-20T16:03:43.664950+0000 2024-05-20T16:05:39.340 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: min_mon_release 19 (squid) 2024-05-20T16:05:39.340 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: election_strategy: 1 2024-05-20T16:05:39.340 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: 0: [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] mon.a 2024-05-20T16:05:39.340 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: 1: [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] mon.c 2024-05-20T16:05:39.340 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: fsmap 2024-05-20T16:05:39.340 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: osdmap e4: 0 total, 0 up, 0 in 2024-05-20T16:05:39.340 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: mgrmap e13: x(active, since 49s) 2024-05-20T16:05:39.340 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: overall HEALTH_OK 2024-05-20T16:05:39.341 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:05:39.341 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:05:39.341 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:05:39.341 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-20T16:05:39.341 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:05:39.341 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: Deploying daemon mon.b on smithi160 2024-05-20T16:05:39.341 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-20T16:05:39.341 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-20T16:05:39.341 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: from='client.? 172.21.15.184:0/608549852' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-20T16:05:39.341 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-20T16:05:39.341 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 ceph-mon[36347]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-05-20T16:05:39.341 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 bash[36319]: 8de30f4d82e8fbfbceeda87df920c160c2b31856d9c59f76e06928e7f3a0e1fd 2024-05-20T16:05:39.341 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:38 smithi160 systemd[1]: Started Ceph mon.b for 59d1600a-16c2-11ef-bc9a-c7b262605968. 2024-05-20T16:05:43.997 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:43.996+0000 7f8f39e7f700 1 -- 172.21.15.184:0/4164381671 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8f200057d0 msgr2=0x7f8f20007cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:05:43.998 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:43.996+0000 7f8f39e7f700 1 --2- 172.21.15.184:0/4164381671 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8f200057d0 0x7f8f20007cb0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f8f3406a2f0 tx=0x7f8f2400fe60 comp rx=0 tx=0).stop 2024-05-20T16:05:43.998 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:43.997+0000 7f8f39e7f700 1 -- 172.21.15.184:0/4164381671 shutdown_connections 2024-05-20T16:05:43.998 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:43.997+0000 7f8f39e7f700 1 --2- 172.21.15.184:0/4164381671 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8f200057d0 0x7f8f20007cb0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:05:43.998 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:43.997+0000 7f8f39e7f700 1 -- 172.21.15.184:0/4164381671 >> 172.21.15.184:0/4164381671 conn(0x7f8f34076c90 msgr2=0x7f8f340770f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:05:43.998 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:43.997+0000 7f8f39e7f700 1 -- 172.21.15.184:0/4164381671 shutdown_connections 2024-05-20T16:05:43.998 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:43.997+0000 7f8f39e7f700 1 -- 172.21.15.184:0/4164381671 wait complete. 2024-05-20T16:05:43.999 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:43.997+0000 7f8f39e7f700 1 Processor -- start 2024-05-20T16:05:43.999 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:43.998+0000 7f8f39e7f700 1 -- start start 2024-05-20T16:05:44.000 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:43.998+0000 7f8f39e7f700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8f200057d0 0x7f8f3419c610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:05:44.000 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:43.998+0000 7f8f39e7f700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8f3419cb50 0x7f8f34196850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:05:44.000 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:43.998+0000 7f8f39e7f700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8f34196d90 0x7f8f341971a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:05:44.000 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:43.998+0000 7f8f39e7f700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f8f34108a40 con 0x7f8f34196d90 2024-05-20T16:05:44.000 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:43.998+0000 7f8f39e7f700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f8f341088c0 con 0x7f8f3419cb50 2024-05-20T16:05:44.000 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:43.998+0000 7f8f39e7f700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f8f34108bc0 con 0x7f8f200057d0 2024-05-20T16:05:44.000 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:43.998+0000 7f8f337fe700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8f200057d0 0x7f8f3419c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:05:44.000 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:43.998+0000 7f8f337fe700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8f200057d0 0x7f8f3419c610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.184:42432/0 (socket says 172.21.15.184:42432) 2024-05-20T16:05:44.000 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:43.998+0000 7f8f337fe700 1 -- 172.21.15.184:0/2130797470 learned_addr learned my addr 172.21.15.184:0/2130797470 (peer_addr_for_me v2:172.21.15.184:0/0) 2024-05-20T16:05:44.000 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:43.999+0000 7f8f337fe700 1 -- 172.21.15.184:0/2130797470 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8f200057d0 msgr2=0x7f8f3419c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-05-20T16:05:44.000 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:43.999+0000 7f8f33fff700 1 --2- 172.21.15.184:0/2130797470 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8f34196d90 0x7f8f341971a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:05:44.000 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:43.999+0000 7f8f337fe700 1 -- 172.21.15.184:0/2130797470 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8f200057d0 msgr2=0x7f8f3419c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-20T16:05:44.001 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:43.999+0000 7f8f337fe700 1 --2- 172.21.15.184:0/2130797470 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8f200057d0 0x7f8f3419c610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-20T16:05:44.001 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:43.999+0000 7f8f337fe700 1 --2- 172.21.15.184:0/2130797470 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8f200057d0 0x7f8f3419c610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-20T16:05:44.001 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:43.999+0000 7f8f2affd700 1 --2- 172.21.15.184:0/2130797470 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8f3419cb50 0x7f8f34196850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:05:44.001 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:43.999+0000 7f8f2affd700 1 -- 172.21.15.184:0/2130797470 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8f3419cb50 msgr2=0x7f8f34196850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-05-20T16:05:44.001 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:43.999+0000 7f8f2affd700 1 -- 172.21.15.184:0/2130797470 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8f3419cb50 msgr2=0x7f8f34196850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-20T16:05:44.001 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:43.999+0000 7f8f2affd700 1 --2- 172.21.15.184:0/2130797470 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8f3419cb50 0x7f8f34196850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-20T16:05:44.001 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:43.999+0000 7f8f2affd700 1 --2- 172.21.15.184:0/2130797470 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8f3419cb50 0x7f8f34196850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-20T16:05:44.001 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:43.999+0000 7f8f33fff700 1 -- 172.21.15.184:0/2130797470 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8f200057d0 msgr2=0x7f8f3419c610 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-20T16:05:44.001 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:43.999+0000 7f8f33fff700 1 --2- 172.21.15.184:0/2130797470 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8f200057d0 0x7f8f3419c610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:05:44.002 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:43.999+0000 7f8f33fff700 1 -- 172.21.15.184:0/2130797470 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8f3419cb50 msgr2=0x7f8f34196850 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-20T16:05:44.002 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:43.999+0000 7f8f33fff700 1 --2- 172.21.15.184:0/2130797470 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8f3419cb50 0x7f8f34196850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:05:44.002 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:43.999+0000 7f8f33fff700 1 -- 172.21.15.184:0/2130797470 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f341a3470 con 0x7f8f34196d90 2024-05-20T16:05:44.002 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:43.999+0000 7f8f33fff700 1 --2- 172.21.15.184:0/2130797470 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8f34196d90 0x7f8f341971a0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f8f2c00bd50 tx=0x7f8f2c00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:05:44.004 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:44.003+0000 7f8f317fa700 1 -- 172.21.15.184:0/2130797470 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f2c019070 con 0x7f8f34196d90 2024-05-20T16:05:44.004 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:44.004+0000 7f8f317fa700 1 -- 172.21.15.184:0/2130797470 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f2c00cd10 con 0x7f8f34196d90 2024-05-20T16:05:44.005 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:44.004+0000 7f8f317fa700 1 -- 172.21.15.184:0/2130797470 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f2c0127e0 con 0x7f8f34196d90 2024-05-20T16:05:44.005 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:44.004+0000 7f8f39e7f700 1 -- 172.21.15.184:0/2130797470 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f341a3760 con 0x7f8f34196d90 2024-05-20T16:05:44.005 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:44.004+0000 7f8f39e7f700 1 -- 172.21.15.184:0/2130797470 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f341a3c70 con 0x7f8f34196d90 2024-05-20T16:05:44.006 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:44.005+0000 7f8f39e7f700 1 -- 172.21.15.184:0/2130797470 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f34103170 con 0x7f8f34196d90 2024-05-20T16:05:44.007 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:44.006+0000 7f8f317fa700 1 -- 172.21.15.184:0/2130797470 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f8f2c01c070 con 0x7f8f34196d90 2024-05-20T16:05:44.007 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:44.007+0000 7f8f317fa700 1 --2- 172.21.15.184:0/2130797470 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f8f14039580 0x7f8f1403ba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:05:44.008 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:44.007+0000 7f8f317fa700 1 -- 172.21.15.184:0/2130797470 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f8f2c04c060 con 0x7f8f34196d90 2024-05-20T16:05:44.008 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:44.007+0000 7f8f337fe700 1 --2- 172.21.15.184:0/2130797470 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f8f14039580 0x7f8f1403ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:05:44.008 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:44.007+0000 7f8f337fe700 1 --2- 172.21.15.184:0/2130797470 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f8f14039580 0x7f8f1403ba90 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f8f2400fa10 tx=0x7f8f24014040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:05:44.013 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:44.012+0000 7f8f317fa700 1 -- 172.21.15.184:0/2130797470 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f8f2c019210 con 0x7f8f34196d90 2024-05-20T16:05:44.299 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:44.297+0000 7f8f39e7f700 1 -- 172.21.15.184:0/2130797470 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f8f3402dc60 con 0x7f8f34196d90 2024-05-20T16:05:44.299 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:44.298+0000 7f8f317fa700 1 -- 172.21.15.184:0/2130797470 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1283 (secure 0 0 0) 0x7f8f2c012980 con 0x7f8f34196d90 2024-05-20T16:05:44.299 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-05-20T16:05:44.299 INFO:teuthology.orchestra.run.smithi184.stdout:{"epoch":3,"fsid":"59d1600a-16c2-11ef-bc9a-c7b262605968","modified":"2024-05-20T16:05:38.939658Z","created":"2024-05-20T16:03:43.664950Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:3300","nonce":0},{"type":"v1","addr":"172.21.15.82:6789","nonce":0}]},"addr":"172.21.15.82:6789/0","public_addr":"172.21.15.82:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:3300","nonce":0},{"type":"v1","addr":"172.21.15.184:6789","nonce":0}]},"addr":"172.21.15.184:6789/0","public_addr":"172.21.15.184:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:3300","nonce":0},{"type":"v1","addr":"172.21.15.160:6789","nonce":0}]},"addr":"172.21.15.160:6789/0","public_addr":"172.21.15.160:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-05-20T16:05:44.300 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:05:44 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-20T16:05:44.300 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:05:44 smithi184 ceph-mon[36209]: mon.a calling monitor election 2024-05-20T16:05:44.300 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:05:44 smithi184 ceph-mon[36209]: mon.c calling monitor election 2024-05-20T16:05:44.300 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:05:44 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-20T16:05:44.300 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:05:44 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-20T16:05:44.300 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:05:44 smithi184 ceph-mon[36209]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-20T16:05:44.300 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:05:44 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-20T16:05:44.300 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:05:44 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-20T16:05:44.300 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:05:44 smithi184 ceph-mon[36209]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-20T16:05:44.301 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:05:44 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-20T16:05:44.301 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:05:44 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-20T16:05:44.301 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:05:44 smithi184 ceph-mon[36209]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-20T16:05:44.301 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:05:44 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-20T16:05:44.301 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:05:44 smithi184 ceph-mon[36209]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-05-20T16:05:44.301 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:05:44 smithi184 ceph-mon[36209]: monmap epoch 3 2024-05-20T16:05:44.301 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:05:44 smithi184 ceph-mon[36209]: fsid 59d1600a-16c2-11ef-bc9a-c7b262605968 2024-05-20T16:05:44.301 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:05:44 smithi184 ceph-mon[36209]: last_changed 2024-05-20T16:05:38.939658+0000 2024-05-20T16:05:44.301 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:05:44 smithi184 ceph-mon[36209]: created 2024-05-20T16:03:43.664950+0000 2024-05-20T16:05:44.301 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:05:44 smithi184 ceph-mon[36209]: min_mon_release 19 (squid) 2024-05-20T16:05:44.301 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:05:44 smithi184 ceph-mon[36209]: election_strategy: 1 2024-05-20T16:05:44.301 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:05:44 smithi184 ceph-mon[36209]: 0: [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] mon.a 2024-05-20T16:05:44.302 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:05:44 smithi184 ceph-mon[36209]: 1: [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] mon.c 2024-05-20T16:05:44.302 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:05:44 smithi184 ceph-mon[36209]: 2: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.b 2024-05-20T16:05:44.302 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:05:44 smithi184 ceph-mon[36209]: fsmap 2024-05-20T16:05:44.302 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:05:44 smithi184 ceph-mon[36209]: osdmap e4: 0 total, 0 up, 0 in 2024-05-20T16:05:44.302 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:05:44 smithi184 ceph-mon[36209]: mgrmap e13: x(active, since 58s) 2024-05-20T16:05:44.302 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:05:44 smithi184 ceph-mon[36209]: overall HEALTH_OK 2024-05-20T16:05:44.302 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:05:44 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:05:44.302 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:05:44 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:05:44.302 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:05:44 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:05:44.302 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:05:44 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:05:44.303 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:44.301+0000 7f8f39e7f700 1 -- 172.21.15.184:0/2130797470 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f8f14039580 msgr2=0x7f8f1403ba90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:05:44.303 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:44.301+0000 7f8f39e7f700 1 --2- 172.21.15.184:0/2130797470 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f8f14039580 0x7f8f1403ba90 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f8f2400fa10 tx=0x7f8f24014040 comp rx=0 tx=0).stop 2024-05-20T16:05:44.303 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:44.301+0000 7f8f39e7f700 1 -- 172.21.15.184:0/2130797470 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8f34196d90 msgr2=0x7f8f341971a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:05:44.303 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:44.301+0000 7f8f39e7f700 1 --2- 172.21.15.184:0/2130797470 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8f34196d90 0x7f8f341971a0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f8f2c00bd50 tx=0x7f8f2c00be30 comp rx=0 tx=0).stop 2024-05-20T16:05:44.304 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:44.301+0000 7f8f39e7f700 1 -- 172.21.15.184:0/2130797470 shutdown_connections 2024-05-20T16:05:44.304 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:44.301+0000 7f8f39e7f700 1 --2- 172.21.15.184:0/2130797470 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f8f14039580 0x7f8f1403ba90 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:05:44.304 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:44.301+0000 7f8f39e7f700 1 --2- 172.21.15.184:0/2130797470 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8f34196d90 0x7f8f341971a0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:05:44.304 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:44.301+0000 7f8f39e7f700 1 --2- 172.21.15.184:0/2130797470 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8f200057d0 0x7f8f3419c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:05:44.304 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:44.301+0000 7f8f39e7f700 1 --2- 172.21.15.184:0/2130797470 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8f3419cb50 0x7f8f34196850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:05:44.304 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:44.301+0000 7f8f39e7f700 1 -- 172.21.15.184:0/2130797470 >> 172.21.15.184:0/2130797470 conn(0x7f8f34076c90 msgr2=0x7f8f340fefe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:05:44.304 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:44.301+0000 7f8f39e7f700 1 -- 172.21.15.184:0/2130797470 shutdown_connections 2024-05-20T16:05:44.304 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:44.302+0000 7f8f39e7f700 1 -- 172.21.15.184:0/2130797470 wait complete. 2024-05-20T16:05:44.304 INFO:teuthology.orchestra.run.smithi184.stderr:dumped monmap epoch 3 2024-05-20T16:05:44.360 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:44 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-20T16:05:44.360 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:44 smithi082 ceph-mon[32238]: mon.a calling monitor election 2024-05-20T16:05:44.360 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:44 smithi082 ceph-mon[32238]: mon.c calling monitor election 2024-05-20T16:05:44.360 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:44 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-20T16:05:44.360 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:44 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-20T16:05:44.360 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:44 smithi082 ceph-mon[32238]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-20T16:05:44.360 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:44 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-20T16:05:44.360 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:44 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-20T16:05:44.360 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:44 smithi082 ceph-mon[32238]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-20T16:05:44.361 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:44 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-20T16:05:44.361 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:44 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-20T16:05:44.361 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:44 smithi082 ceph-mon[32238]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-20T16:05:44.361 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:44 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-20T16:05:44.361 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:44 smithi082 ceph-mon[32238]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-05-20T16:05:44.361 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:44 smithi082 ceph-mon[32238]: monmap epoch 3 2024-05-20T16:05:44.361 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:44 smithi082 ceph-mon[32238]: fsid 59d1600a-16c2-11ef-bc9a-c7b262605968 2024-05-20T16:05:44.361 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:44 smithi082 ceph-mon[32238]: last_changed 2024-05-20T16:05:38.939658+0000 2024-05-20T16:05:44.361 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:44 smithi082 ceph-mon[32238]: created 2024-05-20T16:03:43.664950+0000 2024-05-20T16:05:44.361 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:44 smithi082 ceph-mon[32238]: min_mon_release 19 (squid) 2024-05-20T16:05:44.361 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:44 smithi082 ceph-mon[32238]: election_strategy: 1 2024-05-20T16:05:44.361 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:44 smithi082 ceph-mon[32238]: 0: [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] mon.a 2024-05-20T16:05:44.361 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:44 smithi082 ceph-mon[32238]: 1: [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] mon.c 2024-05-20T16:05:44.361 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:44 smithi082 ceph-mon[32238]: 2: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.b 2024-05-20T16:05:44.362 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:44 smithi082 ceph-mon[32238]: fsmap 2024-05-20T16:05:44.362 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:44 smithi082 ceph-mon[32238]: osdmap e4: 0 total, 0 up, 0 in 2024-05-20T16:05:44.362 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:44 smithi082 ceph-mon[32238]: mgrmap e13: x(active, since 58s) 2024-05-20T16:05:44.362 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:44 smithi082 ceph-mon[32238]: overall HEALTH_OK 2024-05-20T16:05:44.362 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:44 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:05:44.362 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:44 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:05:44.362 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:44 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:05:44.362 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:44 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:05:44.880 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-05-20T16:05:44.881 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 59d1600a-16c2-11ef-bc9a-c7b262605968 -- ceph config generate-minimal-conf 2024-05-20T16:05:45.129 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T16:05:45.136 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:05:45 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T16:05:45.136 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:05:45 smithi184 ceph-mon[36209]: from='client.? 172.21.15.184:0/2130797470' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-20T16:05:45.136 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:05:45 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-20T16:05:45.251 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:45 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T16:05:45.252 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:45 smithi082 ceph-mon[32238]: from='client.? 172.21.15.184:0/2130797470' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-20T16:05:45.252 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:45 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-20T16:05:46.313 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:45 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-20T16:05:46.313 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:45 smithi160 ceph-mon[36347]: mon.a calling monitor election 2024-05-20T16:05:46.313 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:45 smithi160 ceph-mon[36347]: mon.c calling monitor election 2024-05-20T16:05:46.313 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:45 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-20T16:05:46.314 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:45 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-20T16:05:46.314 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:45 smithi160 ceph-mon[36347]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-20T16:05:46.314 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:45 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-20T16:05:46.314 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:45 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-20T16:05:46.314 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:45 smithi160 ceph-mon[36347]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-20T16:05:46.314 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:45 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-20T16:05:46.314 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:45 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-20T16:05:46.314 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:45 smithi160 ceph-mon[36347]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-20T16:05:46.314 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:45 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-20T16:05:46.314 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:45 smithi160 ceph-mon[36347]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-05-20T16:05:46.314 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:45 smithi160 ceph-mon[36347]: monmap epoch 3 2024-05-20T16:05:46.315 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:45 smithi160 ceph-mon[36347]: fsid 59d1600a-16c2-11ef-bc9a-c7b262605968 2024-05-20T16:05:46.315 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:45 smithi160 ceph-mon[36347]: last_changed 2024-05-20T16:05:38.939658+0000 2024-05-20T16:05:46.315 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:45 smithi160 ceph-mon[36347]: created 2024-05-20T16:03:43.664950+0000 2024-05-20T16:05:46.315 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:45 smithi160 ceph-mon[36347]: min_mon_release 19 (squid) 2024-05-20T16:05:46.315 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:45 smithi160 ceph-mon[36347]: election_strategy: 1 2024-05-20T16:05:46.315 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:45 smithi160 ceph-mon[36347]: 0: [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] mon.a 2024-05-20T16:05:46.315 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:45 smithi160 ceph-mon[36347]: 1: [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] mon.c 2024-05-20T16:05:46.315 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:45 smithi160 ceph-mon[36347]: 2: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.b 2024-05-20T16:05:46.315 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:45 smithi160 ceph-mon[36347]: fsmap 2024-05-20T16:05:46.315 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:45 smithi160 ceph-mon[36347]: osdmap e4: 0 total, 0 up, 0 in 2024-05-20T16:05:46.315 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:45 smithi160 ceph-mon[36347]: mgrmap e13: x(active, since 58s) 2024-05-20T16:05:46.315 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:45 smithi160 ceph-mon[36347]: overall HEALTH_OK 2024-05-20T16:05:46.315 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:45 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:05:46.315 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:45 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:05:46.316 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:45 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:05:46.316 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:45 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:05:46.316 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:45 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T16:05:46.316 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:45 smithi160 ceph-mon[36347]: from='client.? 172.21.15.184:0/2130797470' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-20T16:05:46.316 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:45 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-20T16:05:46.711 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:46.710+0000 7f1b70d43700 1 -- 172.21.15.82:0/3717465601 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1b6c1078e0 msgr2=0x7f1b6c107cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:05:46.711 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:46.710+0000 7f1b70d43700 1 --2- 172.21.15.82:0/3717465601 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1b6c1078e0 0x7f1b6c107cf0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f1b60009910 tx=0x7f1b60009c20 comp rx=0 tx=0).stop 2024-05-20T16:05:46.711 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:46.710+0000 7f1b70d43700 1 -- 172.21.15.82:0/3717465601 shutdown_connections 2024-05-20T16:05:46.711 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:46.710+0000 7f1b70d43700 1 --2- 172.21.15.82:0/3717465601 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1b6c1078e0 0x7f1b6c107cf0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:05:46.711 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:46.710+0000 7f1b70d43700 1 -- 172.21.15.82:0/3717465601 >> 172.21.15.82:0/3717465601 conn(0x7f1b6c0fd3f0 msgr2=0x7f1b6c0ff840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:05:46.711 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:46.710+0000 7f1b70d43700 1 -- 172.21.15.82:0/3717465601 shutdown_connections 2024-05-20T16:05:46.711 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:46.710+0000 7f1b70d43700 1 -- 172.21.15.82:0/3717465601 wait complete. 2024-05-20T16:05:46.712 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:46 smithi082 ceph-mon[32238]: mon.b calling monitor election 2024-05-20T16:05:46.712 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:46 smithi082 ceph-mon[32238]: Updating smithi082:/etc/ceph/ceph.conf 2024-05-20T16:05:46.712 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:46 smithi082 ceph-mon[32238]: Updating smithi160:/etc/ceph/ceph.conf 2024-05-20T16:05:46.712 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:46 smithi082 ceph-mon[32238]: Updating smithi184:/etc/ceph/ceph.conf 2024-05-20T16:05:46.712 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:46 smithi082 ceph-mon[32238]: Updating smithi184:/var/lib/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-20T16:05:46.712 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:46 smithi082 ceph-mon[32238]: Updating smithi160:/var/lib/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-20T16:05:46.712 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:46 smithi082 ceph-mon[32238]: Updating smithi082:/var/lib/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-20T16:05:46.713 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:46 smithi082 ceph-mon[32238]: mon.b calling monitor election 2024-05-20T16:05:46.713 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:46 smithi082 ceph-mon[32238]: mon.c calling monitor election 2024-05-20T16:05:46.713 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:46 smithi082 ceph-mon[32238]: mon.a calling monitor election 2024-05-20T16:05:46.713 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:46 smithi082 ceph-mon[32238]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-05-20T16:05:46.713 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:46 smithi082 ceph-mon[32238]: monmap epoch 3 2024-05-20T16:05:46.713 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:46 smithi082 ceph-mon[32238]: fsid 59d1600a-16c2-11ef-bc9a-c7b262605968 2024-05-20T16:05:46.713 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:46 smithi082 ceph-mon[32238]: last_changed 2024-05-20T16:05:38.939658+0000 2024-05-20T16:05:46.713 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:46 smithi082 ceph-mon[32238]: created 2024-05-20T16:03:43.664950+0000 2024-05-20T16:05:46.713 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:46 smithi082 ceph-mon[32238]: min_mon_release 19 (squid) 2024-05-20T16:05:46.713 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:46 smithi082 ceph-mon[32238]: election_strategy: 1 2024-05-20T16:05:46.713 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:46 smithi082 ceph-mon[32238]: 0: [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] mon.a 2024-05-20T16:05:46.713 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:46 smithi082 ceph-mon[32238]: 1: [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] mon.c 2024-05-20T16:05:46.714 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:46 smithi082 ceph-mon[32238]: 2: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.b 2024-05-20T16:05:46.714 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:46 smithi082 ceph-mon[32238]: fsmap 2024-05-20T16:05:46.714 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:46 smithi082 ceph-mon[32238]: osdmap e4: 0 total, 0 up, 0 in 2024-05-20T16:05:46.714 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:46 smithi082 ceph-mon[32238]: mgrmap e13: x(active, since 60s) 2024-05-20T16:05:46.714 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:46 smithi082 ceph-mon[32238]: overall HEALTH_OK 2024-05-20T16:05:46.714 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:46 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:05:46.714 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:46 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:05:46.714 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:46 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:05:46.714 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:46 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:05:46.714 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:46 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:05:46.714 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:46 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:05:46.715 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:46 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:05:46.715 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:46 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:05:46.715 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:46 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:05:46.715 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:46 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:05:46.715 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:46 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:05:46.715 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:46 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-20T16:05:46.715 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:46 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-20T16:05:46.715 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:46 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:05:46.717 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:46.711+0000 7f1b70d43700 1 Processor -- start 2024-05-20T16:05:46.717 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:46.711+0000 7f1b70d43700 1 -- start start 2024-05-20T16:05:46.717 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:46.711+0000 7f1b70d43700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1b6c1078e0 0x7f1b6c103480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:05:46.718 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:46.711+0000 7f1b70d43700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1b6c1039c0 0x7f1b6c103dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:05:46.718 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:46.711+0000 7f1b70d43700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1b6c1a4d50 0x7f1b6c1a7170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:05:46.718 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:46.711+0000 7f1b70d43700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f1b6c10bf40 con 0x7f1b6c1078e0 2024-05-20T16:05:46.718 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:46.711+0000 7f1b70d43700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f1b6c10bdc0 con 0x7f1b6c1039c0 2024-05-20T16:05:46.718 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:46.711+0000 7f1b70d43700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f1b6c10c0c0 con 0x7f1b6c1a4d50 2024-05-20T16:05:46.718 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:46.712+0000 7f1b6a59c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1b6c1078e0 0x7f1b6c103480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:05:46.718 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:46.712+0000 7f1b6a59c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1b6c1078e0 0x7f1b6c103480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:48220/0 (socket says 172.21.15.82:48220) 2024-05-20T16:05:46.718 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:46.712+0000 7f1b6a59c700 1 -- 172.21.15.82:0/3911078019 learned_addr learned my addr 172.21.15.82:0/3911078019 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:05:46.718 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:46.712+0000 7f1b69d9b700 1 --2- 172.21.15.82:0/3911078019 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1b6c1039c0 0x7f1b6c103dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:05:46.718 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:46.712+0000 7f1b6ad9d700 1 --2- 172.21.15.82:0/3911078019 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1b6c1a4d50 0x7f1b6c1a7170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:05:46.718 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:46.712+0000 7f1b6a59c700 1 -- 172.21.15.82:0/3911078019 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1b6c1a4d50 msgr2=0x7f1b6c1a7170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:05:46.718 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:46.712+0000 7f1b6a59c700 1 --2- 172.21.15.82:0/3911078019 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1b6c1a4d50 0x7f1b6c1a7170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:05:46.718 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:46.712+0000 7f1b6a59c700 1 -- 172.21.15.82:0/3911078019 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1b6c1039c0 msgr2=0x7f1b6c103dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:05:46.719 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:46.712+0000 7f1b6a59c700 1 --2- 172.21.15.82:0/3911078019 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1b6c1039c0 0x7f1b6c103dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:05:46.719 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:46.712+0000 7f1b6a59c700 1 -- 172.21.15.82:0/3911078019 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b6c1a77f0 con 0x7f1b6c1078e0 2024-05-20T16:05:46.719 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:46.712+0000 7f1b6a59c700 1 --2- 172.21.15.82:0/3911078019 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1b6c1078e0 0x7f1b6c103480 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f1b60005850 tx=0x7f1b6000bdc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:05:46.719 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:46.712+0000 7f1b5b7fe700 1 -- 172.21.15.82:0/3911078019 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b60027070 con 0x7f1b6c1078e0 2024-05-20T16:05:46.719 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:46.712+0000 7f1b5b7fe700 1 -- 172.21.15.82:0/3911078019 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1b600050d0 con 0x7f1b6c1078e0 2024-05-20T16:05:46.719 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:46.712+0000 7f1b5b7fe700 1 -- 172.21.15.82:0/3911078019 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b60022d50 con 0x7f1b6c1078e0 2024-05-20T16:05:46.719 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:46.713+0000 7f1b70d43700 1 -- 172.21.15.82:0/3911078019 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b6c1a7ae0 con 0x7f1b6c1078e0 2024-05-20T16:05:46.719 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:46.713+0000 7f1b70d43700 1 -- 172.21.15.82:0/3911078019 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b6c1a8070 con 0x7f1b6c1078e0 2024-05-20T16:05:46.719 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:46.714+0000 7f1b70d43700 1 -- 172.21.15.82:0/3911078019 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b6c102ff0 con 0x7f1b6c1078e0 2024-05-20T16:05:46.719 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:46.714+0000 7f1b5b7fe700 1 -- 172.21.15.82:0/3911078019 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f1b6002a070 con 0x7f1b6c1078e0 2024-05-20T16:05:46.719 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:46.714+0000 7f1b5b7fe700 1 --2- 172.21.15.82:0/3911078019 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f1b54039580 0x7f1b5403ba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:05:46.719 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:46.714+0000 7f1b5b7fe700 1 -- 172.21.15.82:0/3911078019 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f1b6004d150 con 0x7f1b6c1078e0 2024-05-20T16:05:46.719 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:46.714+0000 7f1b69d9b700 1 --2- 172.21.15.82:0/3911078019 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f1b54039580 0x7f1b5403ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:05:46.719 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:46.715+0000 7f1b69d9b700 1 --2- 172.21.15.82:0/3911078019 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f1b54039580 0x7f1b5403ba90 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f1b6c06baa0 tx=0x7f1b5c009210 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:05:46.724 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:46.724+0000 7f1b5b7fe700 1 -- 172.21.15.82:0/3911078019 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f1b60013070 con 0x7f1b6c1078e0 2024-05-20T16:05:46.813 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:46 smithi160 ceph-mon[36347]: mon.b calling monitor election 2024-05-20T16:05:46.813 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:46 smithi160 ceph-mon[36347]: Updating smithi082:/etc/ceph/ceph.conf 2024-05-20T16:05:46.813 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:46 smithi160 ceph-mon[36347]: Updating smithi160:/etc/ceph/ceph.conf 2024-05-20T16:05:46.813 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:46 smithi160 ceph-mon[36347]: Updating smithi184:/etc/ceph/ceph.conf 2024-05-20T16:05:46.813 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:46 smithi160 ceph-mon[36347]: Updating smithi184:/var/lib/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-20T16:05:46.813 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:46 smithi160 ceph-mon[36347]: Updating smithi160:/var/lib/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-20T16:05:46.814 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:46 smithi160 ceph-mon[36347]: Updating smithi082:/var/lib/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-20T16:05:46.814 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:46 smithi160 ceph-mon[36347]: mon.b calling monitor election 2024-05-20T16:05:46.814 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:46 smithi160 ceph-mon[36347]: mon.c calling monitor election 2024-05-20T16:05:46.814 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:46 smithi160 ceph-mon[36347]: mon.a calling monitor election 2024-05-20T16:05:46.814 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:46 smithi160 ceph-mon[36347]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-05-20T16:05:46.814 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:46 smithi160 ceph-mon[36347]: monmap epoch 3 2024-05-20T16:05:46.814 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:46 smithi160 ceph-mon[36347]: fsid 59d1600a-16c2-11ef-bc9a-c7b262605968 2024-05-20T16:05:46.814 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:46 smithi160 ceph-mon[36347]: last_changed 2024-05-20T16:05:38.939658+0000 2024-05-20T16:05:46.814 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:46 smithi160 ceph-mon[36347]: created 2024-05-20T16:03:43.664950+0000 2024-05-20T16:05:46.814 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:46 smithi160 ceph-mon[36347]: min_mon_release 19 (squid) 2024-05-20T16:05:46.814 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:46 smithi160 ceph-mon[36347]: election_strategy: 1 2024-05-20T16:05:46.814 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:46 smithi160 ceph-mon[36347]: 0: [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] mon.a 2024-05-20T16:05:46.815 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:46 smithi160 ceph-mon[36347]: 1: [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] mon.c 2024-05-20T16:05:46.815 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:46 smithi160 ceph-mon[36347]: 2: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.b 2024-05-20T16:05:46.815 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:46 smithi160 ceph-mon[36347]: fsmap 2024-05-20T16:05:46.815 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:46 smithi160 ceph-mon[36347]: osdmap e4: 0 total, 0 up, 0 in 2024-05-20T16:05:46.815 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:46 smithi160 ceph-mon[36347]: mgrmap e13: x(active, since 60s) 2024-05-20T16:05:46.815 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:46 smithi160 ceph-mon[36347]: overall HEALTH_OK 2024-05-20T16:05:46.815 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:46 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:05:46.815 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:46 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:05:46.815 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:46 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:05:46.815 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:46 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:05:46.815 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:46 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:05:46.815 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:46 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:05:46.815 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:46 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:05:46.815 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:46 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:05:46.815 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:46 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:05:46.816 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:46 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:05:46.816 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:46 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:05:46.816 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:46 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-20T16:05:46.816 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:46 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-20T16:05:46.816 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:46 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:05:46.887 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:05:46 smithi184 ceph-mon[36209]: mon.b calling monitor election 2024-05-20T16:05:46.887 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:05:46 smithi184 ceph-mon[36209]: Updating smithi082:/etc/ceph/ceph.conf 2024-05-20T16:05:46.887 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:05:46 smithi184 ceph-mon[36209]: Updating smithi160:/etc/ceph/ceph.conf 2024-05-20T16:05:46.887 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:05:46 smithi184 ceph-mon[36209]: Updating smithi184:/etc/ceph/ceph.conf 2024-05-20T16:05:46.887 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:05:46 smithi184 ceph-mon[36209]: Updating smithi184:/var/lib/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-20T16:05:46.887 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:05:46 smithi184 ceph-mon[36209]: Updating smithi160:/var/lib/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-20T16:05:46.887 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:05:46 smithi184 ceph-mon[36209]: Updating smithi082:/var/lib/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-20T16:05:46.887 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:05:46 smithi184 ceph-mon[36209]: mon.b calling monitor election 2024-05-20T16:05:46.887 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:05:46 smithi184 ceph-mon[36209]: mon.c calling monitor election 2024-05-20T16:05:46.887 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:05:46 smithi184 ceph-mon[36209]: mon.a calling monitor election 2024-05-20T16:05:46.887 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:05:46 smithi184 ceph-mon[36209]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-05-20T16:05:46.887 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:05:46 smithi184 ceph-mon[36209]: monmap epoch 3 2024-05-20T16:05:46.888 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:05:46 smithi184 ceph-mon[36209]: fsid 59d1600a-16c2-11ef-bc9a-c7b262605968 2024-05-20T16:05:46.888 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:05:46 smithi184 ceph-mon[36209]: last_changed 2024-05-20T16:05:38.939658+0000 2024-05-20T16:05:46.888 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:05:46 smithi184 ceph-mon[36209]: created 2024-05-20T16:03:43.664950+0000 2024-05-20T16:05:46.888 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:05:46 smithi184 ceph-mon[36209]: min_mon_release 19 (squid) 2024-05-20T16:05:46.888 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:05:46 smithi184 ceph-mon[36209]: election_strategy: 1 2024-05-20T16:05:46.888 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:05:46 smithi184 ceph-mon[36209]: 0: [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] mon.a 2024-05-20T16:05:46.888 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:05:46 smithi184 ceph-mon[36209]: 1: [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] mon.c 2024-05-20T16:05:46.888 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:05:46 smithi184 ceph-mon[36209]: 2: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.b 2024-05-20T16:05:46.888 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:05:46 smithi184 ceph-mon[36209]: fsmap 2024-05-20T16:05:46.888 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:05:46 smithi184 ceph-mon[36209]: osdmap e4: 0 total, 0 up, 0 in 2024-05-20T16:05:46.888 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:05:46 smithi184 ceph-mon[36209]: mgrmap e13: x(active, since 60s) 2024-05-20T16:05:46.888 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:05:46 smithi184 ceph-mon[36209]: overall HEALTH_OK 2024-05-20T16:05:46.888 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:05:46 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:05:46.888 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:05:46 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:05:46.888 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:05:46 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:05:46.889 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:05:46 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:05:46.889 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:05:46 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:05:46.889 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:05:46 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:05:46.889 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:05:46 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:05:46.889 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:05:46 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:05:46.889 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:05:46 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:05:46.889 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:05:46 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:05:46.889 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:05:46 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:05:46.889 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:05:46 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-20T16:05:46.889 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:05:46 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-20T16:05:46.889 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:05:46 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:05:46.932 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:46.931+0000 7f1b70d43700 1 -- 172.21.15.82:0/3911078019 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f1b6c063ca0 con 0x7f1b6c1078e0 2024-05-20T16:05:46.932 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:46.932+0000 7f1b5b7fe700 1 -- 172.21.15.82:0/3911078019 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+275 (secure 0 0 0) 0x7f1b6c063ca0 con 0x7f1b6c1078e0 2024-05-20T16:05:46.932 INFO:teuthology.orchestra.run.smithi082.stdout:# minimal ceph.conf for 59d1600a-16c2-11ef-bc9a-c7b262605968 2024-05-20T16:05:46.932 INFO:teuthology.orchestra.run.smithi082.stdout:[global] 2024-05-20T16:05:46.933 INFO:teuthology.orchestra.run.smithi082.stdout: fsid = 59d1600a-16c2-11ef-bc9a-c7b262605968 2024-05-20T16:05:46.933 INFO:teuthology.orchestra.run.smithi082.stdout: mon_host = [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] 2024-05-20T16:05:46.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:46.934+0000 7f1b70d43700 1 -- 172.21.15.82:0/3911078019 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f1b54039580 msgr2=0x7f1b5403ba90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:05:46.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:46.934+0000 7f1b70d43700 1 --2- 172.21.15.82:0/3911078019 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f1b54039580 0x7f1b5403ba90 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f1b6c06baa0 tx=0x7f1b5c009210 comp rx=0 tx=0).stop 2024-05-20T16:05:46.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:46.934+0000 7f1b70d43700 1 -- 172.21.15.82:0/3911078019 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1b6c1078e0 msgr2=0x7f1b6c103480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:05:46.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:46.934+0000 7f1b70d43700 1 --2- 172.21.15.82:0/3911078019 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1b6c1078e0 0x7f1b6c103480 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f1b60005850 tx=0x7f1b6000bdc0 comp rx=0 tx=0).stop 2024-05-20T16:05:46.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:46.934+0000 7f1b70d43700 1 -- 172.21.15.82:0/3911078019 shutdown_connections 2024-05-20T16:05:46.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:46.934+0000 7f1b70d43700 1 --2- 172.21.15.82:0/3911078019 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f1b54039580 0x7f1b5403ba90 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:05:46.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:46.934+0000 7f1b70d43700 1 --2- 172.21.15.82:0/3911078019 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1b6c1a4d50 0x7f1b6c1a7170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:05:46.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:46.934+0000 7f1b70d43700 1 --2- 172.21.15.82:0/3911078019 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1b6c1078e0 0x7f1b6c103480 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:05:46.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:46.934+0000 7f1b70d43700 1 --2- 172.21.15.82:0/3911078019 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1b6c1039c0 0x7f1b6c103dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:05:46.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:46.934+0000 7f1b70d43700 1 -- 172.21.15.82:0/3911078019 >> 172.21.15.82:0/3911078019 conn(0x7f1b6c0fd3f0 msgr2=0x7f1b6c0fe3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:05:46.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:46.934+0000 7f1b70d43700 1 -- 172.21.15.82:0/3911078019 shutdown_connections 2024-05-20T16:05:46.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:46.934+0000 7f1b70d43700 1 -- 172.21.15.82:0/3911078019 wait complete. 2024-05-20T16:05:47.795 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:47 smithi082 ceph-mon[32238]: Reconfiguring mon.a (unknown last config time)... 2024-05-20T16:05:47.795 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:47 smithi082 ceph-mon[32238]: Reconfiguring daemon mon.a on smithi082 2024-05-20T16:05:47.795 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:47 smithi082 ceph-mon[32238]: from='client.? 172.21.15.82:0/3911078019' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:05:47.795 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:47 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-20T16:05:47.795 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:47 smithi082 ceph-mon[32238]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-20T16:05:47.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:47 smithi160 ceph-mon[36347]: Reconfiguring mon.a (unknown last config time)... 2024-05-20T16:05:47.813 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:47 smithi160 ceph-mon[36347]: Reconfiguring daemon mon.a on smithi082 2024-05-20T16:05:47.813 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:47 smithi160 ceph-mon[36347]: from='client.? 172.21.15.82:0/3911078019' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:05:47.813 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:47 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-20T16:05:47.813 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:47 smithi160 ceph-mon[36347]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-20T16:05:47.821 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-05-20T16:05:47.822 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-05-20T16:05:47.822 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd of=/etc/ceph/ceph.conf 2024-05-20T16:05:47.850 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-05-20T16:05:47.850 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-20T16:05:47.886 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:05:47 smithi184 ceph-mon[36209]: Reconfiguring mon.a (unknown last config time)... 2024-05-20T16:05:47.886 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:05:47 smithi184 ceph-mon[36209]: Reconfiguring daemon mon.a on smithi082 2024-05-20T16:05:47.886 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:05:47 smithi184 ceph-mon[36209]: from='client.? 172.21.15.82:0/3911078019' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:05:47.886 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:05:47 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-20T16:05:47.886 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:05:47 smithi184 ceph-mon[36209]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-20T16:05:47.915 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2024-05-20T16:05:47.915 DEBUG:teuthology.orchestra.run.smithi160:> sudo dd of=/etc/ceph/ceph.conf 2024-05-20T16:05:47.947 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2024-05-20T16:05:47.948 DEBUG:teuthology.orchestra.run.smithi160:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-20T16:05:48.015 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-05-20T16:05:48.016 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd of=/etc/ceph/ceph.conf 2024-05-20T16:05:48.047 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-05-20T16:05:48.047 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-20T16:05:48.116 INFO:tasks.cephadm:Adding mgr.x on smithi082 2024-05-20T16:05:48.116 INFO:tasks.cephadm:Adding mgr.y on smithi160 2024-05-20T16:05:48.116 INFO:tasks.cephadm:Adding mgr.z on smithi184 2024-05-20T16:05:48.116 DEBUG:teuthology.orchestra.run.smithi184:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 59d1600a-16c2-11ef-bc9a-c7b262605968 -- ceph orch apply mgr '3;smithi082=x;smithi160=y;smithi184=z' 2024-05-20T16:05:48.387 INFO:teuthology.orchestra.run.smithi184.stderr:Inferring config /var/lib/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/mon.c/config 2024-05-20T16:05:49.528 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:49 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:05:49.528 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:49 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:05:49.528 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:49 smithi160 ceph-mon[36347]: Reconfiguring mon.b (monmap changed)... 2024-05-20T16:05:49.528 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:49 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-20T16:05:49.528 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:49 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-20T16:05:49.528 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:49 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:05:49.528 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:49 smithi160 ceph-mon[36347]: Reconfiguring daemon mon.b on smithi160 2024-05-20T16:05:49.528 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:49 smithi160 ceph-mon[36347]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-20T16:05:49.531 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:05:49 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:05:49.531 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:05:49 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:05:49.531 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:05:49 smithi184 ceph-mon[36209]: Reconfiguring mon.b (monmap changed)... 2024-05-20T16:05:49.531 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:05:49 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-20T16:05:49.531 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:05:49 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-20T16:05:49.531 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:05:49 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:05:49.531 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:05:49 smithi184 ceph-mon[36209]: Reconfiguring daemon mon.b on smithi160 2024-05-20T16:05:49.532 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:05:49 smithi184 ceph-mon[36209]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-20T16:05:49.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:49 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:05:49.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:49 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:05:49.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:49 smithi082 ceph-mon[32238]: Reconfiguring mon.b (monmap changed)... 2024-05-20T16:05:49.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:49 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-20T16:05:49.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:49 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-20T16:05:49.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:49 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:05:49.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:49 smithi082 ceph-mon[32238]: Reconfiguring daemon mon.b on smithi160 2024-05-20T16:05:49.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:49 smithi082 ceph-mon[32238]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-20T16:05:49.805 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:49.804+0000 7f9ec089c700 1 -- 172.21.15.184:0/1603820734 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9ebc101a70 msgr2=0x7f9ebc101e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:05:49.805 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:49.804+0000 7f9ec089c700 1 --2- 172.21.15.184:0/1603820734 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9ebc101a70 0x7f9ebc101e80 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f9eac009910 tx=0x7f9eac009c20 comp rx=0 tx=0).stop 2024-05-20T16:05:49.805 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:49.804+0000 7f9ec089c700 1 -- 172.21.15.184:0/1603820734 shutdown_connections 2024-05-20T16:05:49.806 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:49.804+0000 7f9ec089c700 1 --2- 172.21.15.184:0/1603820734 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9ebc101a70 0x7f9ebc101e80 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:05:49.806 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:49.804+0000 7f9ec089c700 1 -- 172.21.15.184:0/1603820734 >> 172.21.15.184:0/1603820734 conn(0x7f9ebc0fd0d0 msgr2=0x7f9ebc0ff560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:05:49.806 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:49.804+0000 7f9ec089c700 1 -- 172.21.15.184:0/1603820734 shutdown_connections 2024-05-20T16:05:49.806 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:49.804+0000 7f9ec089c700 1 -- 172.21.15.184:0/1603820734 wait complete. 2024-05-20T16:05:49.808 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:49.805+0000 7f9ec089c700 1 Processor -- start 2024-05-20T16:05:49.808 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:49.805+0000 7f9ec089c700 1 -- start start 2024-05-20T16:05:49.808 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:49.805+0000 7f9ec089c700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9ebc101a70 0x7f9ebc06b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:05:49.809 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:49.805+0000 7f9ec089c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9ebc19b640 0x7f9ebc19da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:05:49.809 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:49.805+0000 7f9ec089c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9ebc19dfa0 0x7f9ebc1a03e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:05:49.809 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:49.805+0000 7f9ec089c700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f9ebc107b40 con 0x7f9ebc19b640 2024-05-20T16:05:49.809 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:49.805+0000 7f9ec089c700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f9ebc1079c0 con 0x7f9ebc19dfa0 2024-05-20T16:05:49.809 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:49.805+0000 7f9ec089c700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f9ebc107cc0 con 0x7f9ebc101a70 2024-05-20T16:05:49.809 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:49.805+0000 7f9eba59c700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9ebc101a70 0x7f9ebc06b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:05:49.809 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:49.805+0000 7f9eba59c700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9ebc101a70 0x7f9ebc06b5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.184:42442/0 (socket says 172.21.15.184:42442) 2024-05-20T16:05:49.809 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:49.805+0000 7f9eba59c700 1 -- 172.21.15.184:0/264279755 learned_addr learned my addr 172.21.15.184:0/264279755 (peer_addr_for_me v2:172.21.15.184:0/0) 2024-05-20T16:05:49.809 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:49.805+0000 7f9eb9d9b700 1 --2- 172.21.15.184:0/264279755 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9ebc19b640 0x7f9ebc19da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:05:49.809 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:49.805+0000 7f9ebad9d700 1 --2- 172.21.15.184:0/264279755 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9ebc19dfa0 0x7f9ebc1a03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:05:49.809 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:49.806+0000 7f9eba59c700 1 -- 172.21.15.184:0/264279755 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9ebc19dfa0 msgr2=0x7f9ebc1a03e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:05:49.810 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:49.806+0000 7f9eba59c700 1 --2- 172.21.15.184:0/264279755 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9ebc19dfa0 0x7f9ebc1a03e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:05:49.810 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:49.806+0000 7f9eba59c700 1 -- 172.21.15.184:0/264279755 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9ebc19b640 msgr2=0x7f9ebc19da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:05:49.810 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:49.806+0000 7f9eba59c700 1 --2- 172.21.15.184:0/264279755 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9ebc19b640 0x7f9ebc19da60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:05:49.810 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:49.806+0000 7f9eba59c700 1 -- 172.21.15.184:0/264279755 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ebc1a0c10 con 0x7f9ebc101a70 2024-05-20T16:05:49.810 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:49.806+0000 7f9eba59c700 1 --2- 172.21.15.184:0/264279755 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9ebc101a70 0x7f9ebc06b5c0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f9eac004f80 tx=0x7f9eac004fb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:05:49.810 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:49.806+0000 7f9eab7fe700 1 -- 172.21.15.184:0/264279755 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9eac026070 con 0x7f9ebc101a70 2024-05-20T16:05:49.810 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:49.806+0000 7f9ec089c700 1 -- 172.21.15.184:0/264279755 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ebc1a0ea0 con 0x7f9ebc101a70 2024-05-20T16:05:49.810 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:49.806+0000 7f9eab7fe700 1 -- 172.21.15.184:0/264279755 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9eac00b820 con 0x7f9ebc101a70 2024-05-20T16:05:49.810 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:49.806+0000 7f9eab7fe700 1 -- 172.21.15.184:0/264279755 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9eac0039d0 con 0x7f9ebc101a70 2024-05-20T16:05:49.810 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:49.806+0000 7f9ec089c700 1 -- 172.21.15.184:0/264279755 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ebc1a1370 con 0x7f9ebc101a70 2024-05-20T16:05:49.810 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:49.807+0000 7f9eab7fe700 1 -- 172.21.15.184:0/264279755 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f9eac029070 con 0x7f9ebc101a70 2024-05-20T16:05:49.810 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:49.807+0000 7f9ec089c700 1 -- 172.21.15.184:0/264279755 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ebc1a1570 con 0x7f9ebc101a70 2024-05-20T16:05:49.811 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:49.807+0000 7f9eab7fe700 1 --2- 172.21.15.184:0/264279755 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f9ea4039580 0x7f9ea403ba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:05:49.811 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:49.807+0000 7f9eab7fe700 1 -- 172.21.15.184:0/264279755 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f9eac04cf00 con 0x7f9ebc101a70 2024-05-20T16:05:49.811 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:49.808+0000 7f9eb9d9b700 1 --2- 172.21.15.184:0/264279755 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f9ea4039580 0x7f9ea403ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:05:49.811 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:49.808+0000 7f9eb9d9b700 1 --2- 172.21.15.184:0/264279755 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f9ea4039580 0x7f9ea403ba90 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f9ebc06af70 tx=0x7f9eb0009210 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:05:49.813 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:49.813+0000 7f9eab7fe700 1 -- 172.21.15.184:0/264279755 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f9eac013070 con 0x7f9ebc101a70 2024-05-20T16:05:50.027 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:50.026+0000 7f9ec089c700 1 -- 172.21.15.184:0/264279755 --> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi082=x;smithi160=y;smithi184=z", "target": ["mon-mgr", ""]}) -- 0x7f9ebc0fee80 con 0x7f9ea4039580 2024-05-20T16:05:50.034 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:50.033+0000 7f9eab7fe700 1 -- 172.21.15.184:0/264279755 <== mgr.14150 v2:172.21.15.82:6800/379530823 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f9ebc0fee80 con 0x7f9ea4039580 2024-05-20T16:05:50.034 INFO:teuthology.orchestra.run.smithi184.stdout:Scheduled mgr update... 2024-05-20T16:05:50.036 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:50.035+0000 7f9ec089c700 1 -- 172.21.15.184:0/264279755 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f9ea4039580 msgr2=0x7f9ea403ba90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:05:50.036 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:50.035+0000 7f9ec089c700 1 --2- 172.21.15.184:0/264279755 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f9ea4039580 0x7f9ea403ba90 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f9ebc06af70 tx=0x7f9eb0009210 comp rx=0 tx=0).stop 2024-05-20T16:05:50.036 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:50.035+0000 7f9ec089c700 1 -- 172.21.15.184:0/264279755 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9ebc101a70 msgr2=0x7f9ebc06b5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:05:50.036 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:50.035+0000 7f9ec089c700 1 --2- 172.21.15.184:0/264279755 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9ebc101a70 0x7f9ebc06b5c0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f9eac004f80 tx=0x7f9eac004fb0 comp rx=0 tx=0).stop 2024-05-20T16:05:50.036 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:50.035+0000 7f9ec089c700 1 -- 172.21.15.184:0/264279755 shutdown_connections 2024-05-20T16:05:50.036 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:50.035+0000 7f9ec089c700 1 --2- 172.21.15.184:0/264279755 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f9ea4039580 0x7f9ea403ba90 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:05:50.036 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:50.035+0000 7f9ec089c700 1 --2- 172.21.15.184:0/264279755 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9ebc19dfa0 0x7f9ebc1a03e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:05:50.036 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:50.035+0000 7f9ec089c700 1 --2- 172.21.15.184:0/264279755 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9ebc101a70 0x7f9ebc06b5c0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:05:50.037 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:50.035+0000 7f9ec089c700 1 --2- 172.21.15.184:0/264279755 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9ebc19b640 0x7f9ebc19da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:05:50.037 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:50.035+0000 7f9ec089c700 1 -- 172.21.15.184:0/264279755 >> 172.21.15.184:0/264279755 conn(0x7f9ebc0fd0d0 msgr2=0x7f9ebc0fe770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:05:50.037 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:50.035+0000 7f9ec089c700 1 -- 172.21.15.184:0/264279755 shutdown_connections 2024-05-20T16:05:50.037 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:05:50.035+0000 7f9ec089c700 1 -- 172.21.15.184:0/264279755 wait complete. 2024-05-20T16:05:50.638 DEBUG:teuthology.orchestra.run.smithi160:mgr.y> sudo journalctl -f -n 0 -u ceph-59d1600a-16c2-11ef-bc9a-c7b262605968@mgr.y.service 2024-05-20T16:05:50.641 DEBUG:teuthology.orchestra.run.smithi184:mgr.z> sudo journalctl -f -n 0 -u ceph-59d1600a-16c2-11ef-bc9a-c7b262605968@mgr.z.service 2024-05-20T16:05:50.643 INFO:tasks.cephadm:Deploying OSDs... 2024-05-20T16:05:50.644 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-05-20T16:05:50.644 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/scratch_devs of=/dev/stdout 2024-05-20T16:05:50.664 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-20T16:05:50.664 DEBUG:teuthology.orchestra.run.smithi082:> stat /dev/vg_nvme/lv_1 2024-05-20T16:05:50.723 INFO:teuthology.orchestra.run.smithi082.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-20T16:05:50.723 INFO:teuthology.orchestra.run.smithi082.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-20T16:05:50.723 INFO:teuthology.orchestra.run.smithi082.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-05-20T16:05:50.723 INFO:teuthology.orchestra.run.smithi082.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-20T16:05:50.723 INFO:teuthology.orchestra.run.smithi082.stdout:Context: system_u:object_r:device_t:s0 2024-05-20T16:05:50.723 INFO:teuthology.orchestra.run.smithi082.stdout:Access: 2024-05-20 16:04:57.725137147 +0000 2024-05-20T16:05:50.724 INFO:teuthology.orchestra.run.smithi082.stdout:Modify: 2024-05-20 15:59:15.994664456 +0000 2024-05-20T16:05:50.724 INFO:teuthology.orchestra.run.smithi082.stdout:Change: 2024-05-20 15:59:15.994664456 +0000 2024-05-20T16:05:50.724 INFO:teuthology.orchestra.run.smithi082.stdout: Birth: 2024-05-20 15:59:15.994664456 +0000 2024-05-20T16:05:50.724 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-20T16:05:50.791 INFO:teuthology.orchestra.run.smithi082.stderr:1+0 records in 2024-05-20T16:05:50.791 INFO:teuthology.orchestra.run.smithi082.stderr:1+0 records out 2024-05-20T16:05:50.791 INFO:teuthology.orchestra.run.smithi082.stderr:512 bytes copied, 0.000212727 s, 2.4 MB/s 2024-05-20T16:05:50.792 DEBUG:teuthology.orchestra.run.smithi082:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-20T16:05:50.850 DEBUG:teuthology.orchestra.run.smithi082:> stat /dev/vg_nvme/lv_2 2024-05-20T16:05:50.908 INFO:teuthology.orchestra.run.smithi082.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-20T16:05:50.908 INFO:teuthology.orchestra.run.smithi082.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-20T16:05:50.908 INFO:teuthology.orchestra.run.smithi082.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-05-20T16:05:50.908 INFO:teuthology.orchestra.run.smithi082.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-20T16:05:50.909 INFO:teuthology.orchestra.run.smithi082.stdout:Context: system_u:object_r:device_t:s0 2024-05-20T16:05:50.909 INFO:teuthology.orchestra.run.smithi082.stdout:Access: 2024-05-20 16:04:57.759136504 +0000 2024-05-20T16:05:50.909 INFO:teuthology.orchestra.run.smithi082.stdout:Modify: 2024-05-20 15:59:16.319658041 +0000 2024-05-20T16:05:50.909 INFO:teuthology.orchestra.run.smithi082.stdout:Change: 2024-05-20 15:59:16.319658041 +0000 2024-05-20T16:05:50.909 INFO:teuthology.orchestra.run.smithi082.stdout: Birth: 2024-05-20 15:59:16.319658041 +0000 2024-05-20T16:05:50.909 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-20T16:05:50.973 INFO:teuthology.orchestra.run.smithi082.stderr:1+0 records in 2024-05-20T16:05:50.973 INFO:teuthology.orchestra.run.smithi082.stderr:1+0 records out 2024-05-20T16:05:50.973 INFO:teuthology.orchestra.run.smithi082.stderr:512 bytes copied, 0.000219424 s, 2.3 MB/s 2024-05-20T16:05:50.975 DEBUG:teuthology.orchestra.run.smithi082:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-20T16:05:51.032 DEBUG:teuthology.orchestra.run.smithi082:> stat /dev/vg_nvme/lv_3 2024-05-20T16:05:51.063 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:50 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:05:51.063 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:50 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:05:51.063 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:50 smithi160 ceph-mon[36347]: Reconfiguring mon.c (monmap changed)... 2024-05-20T16:05:51.063 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:50 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-20T16:05:51.063 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:50 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-20T16:05:51.063 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:50 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:05:51.063 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:50 smithi160 ceph-mon[36347]: Reconfiguring daemon mon.c on smithi184 2024-05-20T16:05:51.063 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:50 smithi160 ceph-mon[36347]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi082=x;smithi160=y;smithi184=z", "target": ["mon-mgr", ""]}]: dispatch 2024-05-20T16:05:51.063 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:50 smithi160 ceph-mon[36347]: Saving service mgr spec with placement smithi082=x;smithi160=y;smithi184=z;count:3 2024-05-20T16:05:51.063 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:50 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:05:51.065 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:05:50 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:05:51.065 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:05:50 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:05:51.065 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:05:50 smithi184 ceph-mon[36209]: Reconfiguring mon.c (monmap changed)... 2024-05-20T16:05:51.066 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:05:50 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-20T16:05:51.066 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:05:50 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-20T16:05:51.066 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:05:50 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:05:51.066 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:05:50 smithi184 ceph-mon[36209]: Reconfiguring daemon mon.c on smithi184 2024-05-20T16:05:51.066 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:05:50 smithi184 ceph-mon[36209]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi082=x;smithi160=y;smithi184=z", "target": ["mon-mgr", ""]}]: dispatch 2024-05-20T16:05:51.066 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:05:50 smithi184 ceph-mon[36209]: Saving service mgr spec with placement smithi082=x;smithi160=y;smithi184=z;count:3 2024-05-20T16:05:51.066 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:05:50 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:05:51.089 INFO:teuthology.orchestra.run.smithi082.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-20T16:05:51.089 INFO:teuthology.orchestra.run.smithi082.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-20T16:05:51.089 INFO:teuthology.orchestra.run.smithi082.stdout:Device: 5h/5d Inode: 705 Links: 1 2024-05-20T16:05:51.090 INFO:teuthology.orchestra.run.smithi082.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-20T16:05:51.090 INFO:teuthology.orchestra.run.smithi082.stdout:Context: system_u:object_r:device_t:s0 2024-05-20T16:05:51.090 INFO:teuthology.orchestra.run.smithi082.stdout:Access: 2024-05-20 16:04:57.790135919 +0000 2024-05-20T16:05:51.090 INFO:teuthology.orchestra.run.smithi082.stdout:Modify: 2024-05-20 15:59:16.651651488 +0000 2024-05-20T16:05:51.090 INFO:teuthology.orchestra.run.smithi082.stdout:Change: 2024-05-20 15:59:16.651651488 +0000 2024-05-20T16:05:51.090 INFO:teuthology.orchestra.run.smithi082.stdout: Birth: 2024-05-20 15:59:16.651651488 +0000 2024-05-20T16:05:51.090 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-20T16:05:51.110 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:50 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:05:51.110 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:50 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:05:51.110 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:50 smithi082 ceph-mon[32238]: Reconfiguring mon.c (monmap changed)... 2024-05-20T16:05:51.110 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:50 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-20T16:05:51.110 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:50 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-20T16:05:51.110 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:50 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:05:51.110 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:50 smithi082 ceph-mon[32238]: Reconfiguring daemon mon.c on smithi184 2024-05-20T16:05:51.110 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:50 smithi082 ceph-mon[32238]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi082=x;smithi160=y;smithi184=z", "target": ["mon-mgr", ""]}]: dispatch 2024-05-20T16:05:51.110 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:50 smithi082 ceph-mon[32238]: Saving service mgr spec with placement smithi082=x;smithi160=y;smithi184=z;count:3 2024-05-20T16:05:51.110 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:50 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:05:51.133 INFO:teuthology.orchestra.run.smithi082.stderr:1+0 records in 2024-05-20T16:05:51.133 INFO:teuthology.orchestra.run.smithi082.stderr:1+0 records out 2024-05-20T16:05:51.134 INFO:teuthology.orchestra.run.smithi082.stderr:512 bytes copied, 0.000190643 s, 2.7 MB/s 2024-05-20T16:05:51.135 DEBUG:teuthology.orchestra.run.smithi082:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-20T16:05:51.192 DEBUG:teuthology.orchestra.run.smithi082:> stat /dev/vg_nvme/lv_4 2024-05-20T16:05:51.248 INFO:teuthology.orchestra.run.smithi082.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-20T16:05:51.248 INFO:teuthology.orchestra.run.smithi082.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-20T16:05:51.248 INFO:teuthology.orchestra.run.smithi082.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-05-20T16:05:51.248 INFO:teuthology.orchestra.run.smithi082.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-20T16:05:51.248 INFO:teuthology.orchestra.run.smithi082.stdout:Context: system_u:object_r:device_t:s0 2024-05-20T16:05:51.249 INFO:teuthology.orchestra.run.smithi082.stdout:Access: 2024-05-20 16:04:57.822135314 +0000 2024-05-20T16:05:51.249 INFO:teuthology.orchestra.run.smithi082.stdout:Modify: 2024-05-20 15:59:16.988644836 +0000 2024-05-20T16:05:51.249 INFO:teuthology.orchestra.run.smithi082.stdout:Change: 2024-05-20 15:59:16.988644836 +0000 2024-05-20T16:05:51.249 INFO:teuthology.orchestra.run.smithi082.stdout: Birth: 2024-05-20 15:59:16.988644836 +0000 2024-05-20T16:05:51.249 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-20T16:05:51.312 INFO:teuthology.orchestra.run.smithi082.stderr:1+0 records in 2024-05-20T16:05:51.312 INFO:teuthology.orchestra.run.smithi082.stderr:1+0 records out 2024-05-20T16:05:51.312 INFO:teuthology.orchestra.run.smithi082.stderr:512 bytes copied, 0.000216487 s, 2.4 MB/s 2024-05-20T16:05:51.314 DEBUG:teuthology.orchestra.run.smithi082:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-20T16:05:51.370 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2024-05-20T16:05:51.370 DEBUG:teuthology.orchestra.run.smithi160:> dd if=/scratch_devs of=/dev/stdout 2024-05-20T16:05:51.389 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-20T16:05:51.389 DEBUG:teuthology.orchestra.run.smithi160:> stat /dev/vg_nvme/lv_1 2024-05-20T16:05:51.447 INFO:teuthology.orchestra.run.smithi160.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-20T16:05:51.447 INFO:teuthology.orchestra.run.smithi160.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-20T16:05:51.447 INFO:teuthology.orchestra.run.smithi160.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-05-20T16:05:51.447 INFO:teuthology.orchestra.run.smithi160.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-20T16:05:51.447 INFO:teuthology.orchestra.run.smithi160.stdout:Context: system_u:object_r:device_t:s0 2024-05-20T16:05:51.447 INFO:teuthology.orchestra.run.smithi160.stdout:Access: 2024-05-20 16:05:38.369590803 +0000 2024-05-20T16:05:51.447 INFO:teuthology.orchestra.run.smithi160.stdout:Modify: 2024-05-20 15:59:29.141835646 +0000 2024-05-20T16:05:51.448 INFO:teuthology.orchestra.run.smithi160.stdout:Change: 2024-05-20 15:59:29.141835646 +0000 2024-05-20T16:05:51.448 INFO:teuthology.orchestra.run.smithi160.stdout: Birth: 2024-05-20 15:59:29.141835646 +0000 2024-05-20T16:05:51.448 DEBUG:teuthology.orchestra.run.smithi160:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-20T16:05:51.513 INFO:teuthology.orchestra.run.smithi160.stderr:1+0 records in 2024-05-20T16:05:51.514 INFO:teuthology.orchestra.run.smithi160.stderr:1+0 records out 2024-05-20T16:05:51.514 INFO:teuthology.orchestra.run.smithi160.stderr:512 bytes copied, 0.000348431 s, 1.5 MB/s 2024-05-20T16:05:51.515 DEBUG:teuthology.orchestra.run.smithi160:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-20T16:05:51.573 DEBUG:teuthology.orchestra.run.smithi160:> stat /dev/vg_nvme/lv_2 2024-05-20T16:05:51.629 INFO:teuthology.orchestra.run.smithi160.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-20T16:05:51.629 INFO:teuthology.orchestra.run.smithi160.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-20T16:05:51.629 INFO:teuthology.orchestra.run.smithi160.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-05-20T16:05:51.630 INFO:teuthology.orchestra.run.smithi160.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-20T16:05:51.630 INFO:teuthology.orchestra.run.smithi160.stdout:Context: system_u:object_r:device_t:s0 2024-05-20T16:05:51.630 INFO:teuthology.orchestra.run.smithi160.stdout:Access: 2024-05-20 16:05:38.370590788 +0000 2024-05-20T16:05:51.630 INFO:teuthology.orchestra.run.smithi160.stdout:Modify: 2024-05-20 15:59:29.481830532 +0000 2024-05-20T16:05:51.630 INFO:teuthology.orchestra.run.smithi160.stdout:Change: 2024-05-20 15:59:29.481830532 +0000 2024-05-20T16:05:51.630 INFO:teuthology.orchestra.run.smithi160.stdout: Birth: 2024-05-20 15:59:29.481830532 +0000 2024-05-20T16:05:51.630 DEBUG:teuthology.orchestra.run.smithi160:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-20T16:05:51.695 INFO:teuthology.orchestra.run.smithi160.stderr:1+0 records in 2024-05-20T16:05:51.695 INFO:teuthology.orchestra.run.smithi160.stderr:1+0 records out 2024-05-20T16:05:51.695 INFO:teuthology.orchestra.run.smithi160.stderr:512 bytes copied, 0.000219308 s, 2.3 MB/s 2024-05-20T16:05:51.696 DEBUG:teuthology.orchestra.run.smithi160:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-20T16:05:51.754 DEBUG:teuthology.orchestra.run.smithi160:> stat /dev/vg_nvme/lv_3 2024-05-20T16:05:51.811 INFO:teuthology.orchestra.run.smithi160.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-20T16:05:51.811 INFO:teuthology.orchestra.run.smithi160.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-20T16:05:51.811 INFO:teuthology.orchestra.run.smithi160.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-05-20T16:05:51.811 INFO:teuthology.orchestra.run.smithi160.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-20T16:05:51.811 INFO:teuthology.orchestra.run.smithi160.stdout:Context: system_u:object_r:device_t:s0 2024-05-20T16:05:51.811 INFO:teuthology.orchestra.run.smithi160.stdout:Access: 2024-05-20 16:05:38.370590788 +0000 2024-05-20T16:05:51.811 INFO:teuthology.orchestra.run.smithi160.stdout:Modify: 2024-05-20 15:59:29.808825613 +0000 2024-05-20T16:05:51.811 INFO:teuthology.orchestra.run.smithi160.stdout:Change: 2024-05-20 15:59:29.808825613 +0000 2024-05-20T16:05:51.811 INFO:teuthology.orchestra.run.smithi160.stdout: Birth: 2024-05-20 15:59:29.808825613 +0000 2024-05-20T16:05:51.812 DEBUG:teuthology.orchestra.run.smithi160:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-20T16:05:51.875 INFO:teuthology.orchestra.run.smithi160.stderr:1+0 records in 2024-05-20T16:05:51.875 INFO:teuthology.orchestra.run.smithi160.stderr:1+0 records out 2024-05-20T16:05:51.876 INFO:teuthology.orchestra.run.smithi160.stderr:512 bytes copied, 0.000221735 s, 2.3 MB/s 2024-05-20T16:05:51.877 DEBUG:teuthology.orchestra.run.smithi160:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-20T16:05:51.934 DEBUG:teuthology.orchestra.run.smithi160:> stat /dev/vg_nvme/lv_4 2024-05-20T16:05:51.989 INFO:teuthology.orchestra.run.smithi160.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-20T16:05:51.990 INFO:teuthology.orchestra.run.smithi160.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-20T16:05:51.990 INFO:teuthology.orchestra.run.smithi160.stdout:Device: 5h/5d Inode: 721 Links: 1 2024-05-20T16:05:51.990 INFO:teuthology.orchestra.run.smithi160.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-20T16:05:51.990 INFO:teuthology.orchestra.run.smithi160.stdout:Context: system_u:object_r:device_t:s0 2024-05-20T16:05:51.990 INFO:teuthology.orchestra.run.smithi160.stdout:Access: 2024-05-20 16:05:38.370590788 +0000 2024-05-20T16:05:51.990 INFO:teuthology.orchestra.run.smithi160.stdout:Modify: 2024-05-20 15:59:30.156820378 +0000 2024-05-20T16:05:51.990 INFO:teuthology.orchestra.run.smithi160.stdout:Change: 2024-05-20 15:59:30.156820378 +0000 2024-05-20T16:05:51.990 INFO:teuthology.orchestra.run.smithi160.stdout: Birth: 2024-05-20 15:59:30.156820378 +0000 2024-05-20T16:05:51.991 DEBUG:teuthology.orchestra.run.smithi160:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-20T16:05:52.053 INFO:teuthology.orchestra.run.smithi160.stderr:1+0 records in 2024-05-20T16:05:52.053 INFO:teuthology.orchestra.run.smithi160.stderr:1+0 records out 2024-05-20T16:05:52.053 INFO:teuthology.orchestra.run.smithi160.stderr:512 bytes copied, 0.000198649 s, 2.6 MB/s 2024-05-20T16:05:52.054 DEBUG:teuthology.orchestra.run.smithi160:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-20T16:05:52.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:51 smithi082 ceph-mon[32238]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-20T16:05:52.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:51 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:05:52.110 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:51 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:05:52.110 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:51 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T16:05:52.110 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-05-20T16:05:52.111 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/scratch_devs of=/dev/stdout 2024-05-20T16:05:52.127 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-20T16:05:52.127 DEBUG:teuthology.orchestra.run.smithi184:> stat /dev/vg_nvme/lv_1 2024-05-20T16:05:52.136 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:05:51 smithi184 ceph-mon[36209]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-20T16:05:52.136 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:05:51 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:05:52.136 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:05:51 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:05:52.136 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:05:51 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T16:05:52.150 INFO:teuthology.orchestra.run.smithi184.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-20T16:05:52.151 INFO:teuthology.orchestra.run.smithi184.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-20T16:05:52.151 INFO:teuthology.orchestra.run.smithi184.stdout:Device: 5h/5d Inode: 664 Links: 1 2024-05-20T16:05:52.151 INFO:teuthology.orchestra.run.smithi184.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-20T16:05:52.151 INFO:teuthology.orchestra.run.smithi184.stdout:Context: system_u:object_r:device_t:s0 2024-05-20T16:05:52.151 INFO:teuthology.orchestra.run.smithi184.stdout:Access: 2024-05-20 16:05:28.466104649 +0000 2024-05-20T16:05:52.151 INFO:teuthology.orchestra.run.smithi184.stdout:Modify: 2024-05-20 15:59:29.886042618 +0000 2024-05-20T16:05:52.151 INFO:teuthology.orchestra.run.smithi184.stdout:Change: 2024-05-20 15:59:29.886042618 +0000 2024-05-20T16:05:52.151 INFO:teuthology.orchestra.run.smithi184.stdout: Birth: 2024-05-20 15:59:29.886042618 +0000 2024-05-20T16:05:52.151 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-20T16:05:52.214 INFO:teuthology.orchestra.run.smithi184.stderr:1+0 records in 2024-05-20T16:05:52.214 INFO:teuthology.orchestra.run.smithi184.stderr:1+0 records out 2024-05-20T16:05:52.214 INFO:teuthology.orchestra.run.smithi184.stderr:512 bytes copied, 0.000270366 s, 1.9 MB/s 2024-05-20T16:05:52.215 DEBUG:teuthology.orchestra.run.smithi184:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-20T16:05:52.271 DEBUG:teuthology.orchestra.run.smithi184:> stat /dev/vg_nvme/lv_2 2024-05-20T16:05:52.312 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:51 smithi160 ceph-mon[36347]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-20T16:05:52.313 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:51 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:05:52.313 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:51 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:05:52.313 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:51 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T16:05:52.326 INFO:teuthology.orchestra.run.smithi184.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-20T16:05:52.327 INFO:teuthology.orchestra.run.smithi184.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-20T16:05:52.327 INFO:teuthology.orchestra.run.smithi184.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-05-20T16:05:52.327 INFO:teuthology.orchestra.run.smithi184.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-20T16:05:52.327 INFO:teuthology.orchestra.run.smithi184.stdout:Context: system_u:object_r:device_t:s0 2024-05-20T16:05:52.327 INFO:teuthology.orchestra.run.smithi184.stdout:Access: 2024-05-20 16:05:28.467104629 +0000 2024-05-20T16:05:52.327 INFO:teuthology.orchestra.run.smithi184.stdout:Modify: 2024-05-20 15:59:30.254035178 +0000 2024-05-20T16:05:52.327 INFO:teuthology.orchestra.run.smithi184.stdout:Change: 2024-05-20 15:59:30.254035178 +0000 2024-05-20T16:05:52.327 INFO:teuthology.orchestra.run.smithi184.stdout: Birth: 2024-05-20 15:59:30.254035178 +0000 2024-05-20T16:05:52.327 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-20T16:05:52.389 INFO:teuthology.orchestra.run.smithi184.stderr:1+0 records in 2024-05-20T16:05:52.389 INFO:teuthology.orchestra.run.smithi184.stderr:1+0 records out 2024-05-20T16:05:52.389 INFO:teuthology.orchestra.run.smithi184.stderr:512 bytes copied, 0.000201965 s, 2.5 MB/s 2024-05-20T16:05:52.390 DEBUG:teuthology.orchestra.run.smithi184:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-20T16:05:52.446 DEBUG:teuthology.orchestra.run.smithi184:> stat /dev/vg_nvme/lv_3 2024-05-20T16:05:52.502 INFO:teuthology.orchestra.run.smithi184.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-20T16:05:52.502 INFO:teuthology.orchestra.run.smithi184.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-20T16:05:52.502 INFO:teuthology.orchestra.run.smithi184.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-05-20T16:05:52.502 INFO:teuthology.orchestra.run.smithi184.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-20T16:05:52.502 INFO:teuthology.orchestra.run.smithi184.stdout:Context: system_u:object_r:device_t:s0 2024-05-20T16:05:52.503 INFO:teuthology.orchestra.run.smithi184.stdout:Access: 2024-05-20 16:05:28.467104629 +0000 2024-05-20T16:05:52.503 INFO:teuthology.orchestra.run.smithi184.stdout:Modify: 2024-05-20 15:59:30.589028406 +0000 2024-05-20T16:05:52.503 INFO:teuthology.orchestra.run.smithi184.stdout:Change: 2024-05-20 15:59:30.589028406 +0000 2024-05-20T16:05:52.503 INFO:teuthology.orchestra.run.smithi184.stdout: Birth: 2024-05-20 15:59:30.589028406 +0000 2024-05-20T16:05:52.503 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-20T16:05:52.565 INFO:teuthology.orchestra.run.smithi184.stderr:1+0 records in 2024-05-20T16:05:52.566 INFO:teuthology.orchestra.run.smithi184.stderr:1+0 records out 2024-05-20T16:05:52.566 INFO:teuthology.orchestra.run.smithi184.stderr:512 bytes copied, 0.0001882 s, 2.7 MB/s 2024-05-20T16:05:52.567 DEBUG:teuthology.orchestra.run.smithi184:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-20T16:05:52.623 DEBUG:teuthology.orchestra.run.smithi184:> stat /dev/vg_nvme/lv_4 2024-05-20T16:05:52.678 INFO:teuthology.orchestra.run.smithi184.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-20T16:05:52.678 INFO:teuthology.orchestra.run.smithi184.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-20T16:05:52.678 INFO:teuthology.orchestra.run.smithi184.stdout:Device: 5h/5d Inode: 721 Links: 1 2024-05-20T16:05:52.678 INFO:teuthology.orchestra.run.smithi184.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-20T16:05:52.678 INFO:teuthology.orchestra.run.smithi184.stdout:Context: system_u:object_r:device_t:s0 2024-05-20T16:05:52.679 INFO:teuthology.orchestra.run.smithi184.stdout:Access: 2024-05-20 16:05:28.467104629 +0000 2024-05-20T16:05:52.679 INFO:teuthology.orchestra.run.smithi184.stdout:Modify: 2024-05-20 15:59:30.920021714 +0000 2024-05-20T16:05:52.679 INFO:teuthology.orchestra.run.smithi184.stdout:Change: 2024-05-20 15:59:30.920021714 +0000 2024-05-20T16:05:52.679 INFO:teuthology.orchestra.run.smithi184.stdout: Birth: 2024-05-20 15:59:30.920021714 +0000 2024-05-20T16:05:52.679 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-20T16:05:52.742 INFO:teuthology.orchestra.run.smithi184.stderr:1+0 records in 2024-05-20T16:05:52.742 INFO:teuthology.orchestra.run.smithi184.stderr:1+0 records out 2024-05-20T16:05:52.742 INFO:teuthology.orchestra.run.smithi184.stderr:512 bytes copied, 0.000213645 s, 2.4 MB/s 2024-05-20T16:05:52.743 DEBUG:teuthology.orchestra.run.smithi184:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-20T16:05:52.798 INFO:tasks.cephadm:Deploying osd.0 on smithi082 with /dev/vg_nvme/lv_4... 2024-05-20T16:05:52.799 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 59d1600a-16c2-11ef-bc9a-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-05-20T16:05:53.041 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T16:05:53.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:52 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:05:53.110 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:52 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T16:05:53.110 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:52 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:05:53.110 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:52 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-20T16:05:53.110 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:52 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-20T16:05:53.110 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:52 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-20T16:05:53.110 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:52 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:05:53.121 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:05:52 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:05:53.122 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:05:52 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T16:05:53.122 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:05:52 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:05:53.122 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:05:52 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-20T16:05:53.122 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:05:52 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-20T16:05:53.122 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:05:52 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-20T16:05:53.122 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:05:52 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:05:53.313 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:52 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:05:53.313 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:52 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T16:05:53.313 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:52 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:05:53.313 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:52 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-20T16:05:53.313 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:52 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-20T16:05:53.313 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:52 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-20T16:05:53.313 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:52 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:05:53.985 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:05:53 smithi184 ceph-mon[36209]: Deploying daemon mgr.z on smithi184 2024-05-20T16:05:53.985 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:05:53 smithi184 ceph-mon[36209]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-20T16:05:54.099 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:53 smithi082 ceph-mon[32238]: Deploying daemon mgr.z on smithi184 2024-05-20T16:05:54.099 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:53 smithi082 ceph-mon[32238]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-20T16:05:54.313 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:53 smithi160 ceph-mon[36347]: Deploying daemon mgr.z on smithi184 2024-05-20T16:05:54.313 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:53 smithi160 ceph-mon[36347]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-20T16:05:54.367 INFO:journalctl@ceph.mgr.z.smithi184.stdout:May 20 16:05:54 smithi184 systemd[1]: Starting Ceph mgr.z for 59d1600a-16c2-11ef-bc9a-c7b262605968... 2024-05-20T16:05:54.625 INFO:journalctl@ceph.mgr.z.smithi184.stdout:May 20 16:05:54 smithi184 podman[38027]: 2024-05-20 16:05:54.269302476 +0000 UTC m=+0.016999021 image pull cfcc61b2d476f611479310eb042aa05f11dd5f60dbb65e3b0d7e0e11d024cdb3 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708 2024-05-20T16:05:54.626 INFO:journalctl@ceph.mgr.z.smithi184.stdout:May 20 16:05:54 smithi184 podman[38027]: 2024-05-20 16:05:54.377577531 +0000 UTC m=+0.125274065 container create f9abb8907d231d6494badea5a296b0b8ff52ee42226fb7ace2c02ac70fcd2b4b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mgr-z, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240514, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD) 2024-05-20T16:05:54.626 INFO:journalctl@ceph.mgr.z.smithi184.stdout:May 20 16:05:54 smithi184 podman[38027]: 2024-05-20 16:05:54.588507705 +0000 UTC m=+0.336204225 container init f9abb8907d231d6494badea5a296b0b8ff52ee42226fb7ace2c02ac70fcd2b4b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mgr-z, org.label-schema.build-date=20240514, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, org.label-schema.schema-version=1.0, ceph=True) 2024-05-20T16:05:54.626 INFO:journalctl@ceph.mgr.z.smithi184.stdout:May 20 16:05:54 smithi184 podman[38027]: 2024-05-20 16:05:54.591378708 +0000 UTC m=+0.339075240 container start f9abb8907d231d6494badea5a296b0b8ff52ee42226fb7ace2c02ac70fcd2b4b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mgr-z, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240514, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-05-20T16:05:54.769 INFO:teuthology.orchestra.run.smithi082.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-05-20T16:05:54.770 INFO:teuthology.orchestra.run.smithi082.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-05-20T16:05:54.770 INFO:teuthology.orchestra.run.smithi082.stderr: stderr: 10+0 records in 2024-05-20T16:05:54.770 INFO:teuthology.orchestra.run.smithi082.stderr:10+0 records out 2024-05-20T16:05:54.770 INFO:teuthology.orchestra.run.smithi082.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0135639 s, 773 MB/s 2024-05-20T16:05:54.770 INFO:teuthology.orchestra.run.smithi082.stderr:--> Zapping successful for: 2024-05-20T16:05:54.886 INFO:journalctl@ceph.mgr.z.smithi184.stdout:May 20 16:05:54 smithi184 ceph-mgr[38046]: -- 172.21.15.184:0/95307022 <== mon.1 v2:172.21.15.184:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5646bd5185a0 con 0x5646bd656900 2024-05-20T16:05:54.886 INFO:journalctl@ceph.mgr.z.smithi184.stdout:May 20 16:05:54 smithi184 bash[38027]: f9abb8907d231d6494badea5a296b0b8ff52ee42226fb7ace2c02ac70fcd2b4b 2024-05-20T16:05:54.886 INFO:journalctl@ceph.mgr.z.smithi184.stdout:May 20 16:05:54 smithi184 systemd[1]: Started Ceph mgr.z for 59d1600a-16c2-11ef-bc9a-c7b262605968. 2024-05-20T16:05:54.886 INFO:journalctl@ceph.mgr.z.smithi184.stdout:May 20 16:05:54 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mgr-z[38042]: 2024-05-20T16:05:54.756+0000 7ff88a8db200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-20T16:05:55.349 INFO:journalctl@ceph.mgr.z.smithi184.stdout:May 20 16:05:54 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mgr-z[38042]: 2024-05-20T16:05:54.937+0000 7ff88a8db200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-20T16:05:55.349 INFO:journalctl@ceph.mgr.z.smithi184.stdout:May 20 16:05:55 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mgr-z[38042]: 2024-05-20T16:05:55.039+0000 7ff88a8db200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-20T16:05:55.349 INFO:journalctl@ceph.mgr.z.smithi184.stdout:May 20 16:05:55 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mgr-z[38042]: 2024-05-20T16:05:55.348+0000 7ff88a8db200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-20T16:05:55.361 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 59d1600a-16c2-11ef-bc9a-c7b262605968 -- ceph orch daemon add osd smithi082:vg_nvme/lv_4 2024-05-20T16:05:55.593 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T16:05:55.636 INFO:journalctl@ceph.mgr.z.smithi184.stdout:May 20 16:05:55 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mgr-z[38042]: 2024-05-20T16:05:55.432+0000 7ff88a8db200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-20T16:05:55.995 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:55 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:05:55.996 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:55 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:05:55.996 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:55 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:05:55.996 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:55 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-20T16:05:55.996 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:55 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-20T16:05:55.996 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:55 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-20T16:05:55.996 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:55 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:05:55.996 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:55 smithi082 ceph-mon[32238]: Deploying daemon mgr.y on smithi160 2024-05-20T16:05:55.996 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:55 smithi082 ceph-mon[32238]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-20T16:05:56.040 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:55 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:05:56.040 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:55 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:05:56.040 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:55 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:05:56.040 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:55 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-20T16:05:56.041 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:55 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-20T16:05:56.041 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:55 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-20T16:05:56.041 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:55 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:05:56.041 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:55 smithi160 ceph-mon[36347]: Deploying daemon mgr.y on smithi160 2024-05-20T16:05:56.041 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:55 smithi160 ceph-mon[36347]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-20T16:05:56.136 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:05:55 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:05:56.136 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:05:55 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:05:56.136 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:05:55 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:05:56.136 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:05:55 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-20T16:05:56.136 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:05:55 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-20T16:05:56.136 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:05:55 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-20T16:05:56.137 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:05:55 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:05:56.137 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:05:55 smithi184 ceph-mon[36209]: Deploying daemon mgr.y on smithi160 2024-05-20T16:05:56.137 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:05:55 smithi184 ceph-mon[36209]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-20T16:05:56.886 INFO:journalctl@ceph.mgr.z.smithi184.stdout:May 20 16:05:56 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mgr-z[38042]: 2024-05-20T16:05:56.621+0000 7ff88a8db200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-20T16:05:56.887 INFO:journalctl@ceph.mgr.z.smithi184.stdout:May 20 16:05:56 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mgr-z[38042]: 2024-05-20T16:05:56.794+0000 7ff88a8db200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-20T16:05:56.989 INFO:journalctl@ceph.mgr.y.smithi160.stdout:May 20 16:05:56 smithi160 systemd[1]: Starting Ceph mgr.y for 59d1600a-16c2-11ef-bc9a-c7b262605968... 2024-05-20T16:05:57.148 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:57.147+0000 7f8a1b950700 1 -- 172.21.15.82:0/2299924784 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8a14104830 msgr2=0x7f8a14104c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:05:57.149 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:57.147+0000 7f8a1b950700 1 --2- 172.21.15.82:0/2299924784 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8a14104830 0x7f8a14104c40 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f8a10009990 tx=0x7f8a10009ca0 comp rx=0 tx=0).stop 2024-05-20T16:05:57.149 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:57.147+0000 7f8a1b950700 1 -- 172.21.15.82:0/2299924784 shutdown_connections 2024-05-20T16:05:57.149 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:57.147+0000 7f8a1b950700 1 --2- 172.21.15.82:0/2299924784 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8a14103860 0x7f8a1410b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:05:57.149 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:57.147+0000 7f8a1b950700 1 --2- 172.21.15.82:0/2299924784 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8a14104830 0x7f8a14104c40 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:05:57.149 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:57.148+0000 7f8a1b950700 1 --2- 172.21.15.82:0/2299924784 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8a14102de0 0x7f8a141031f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:05:57.149 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:57.148+0000 7f8a1b950700 1 -- 172.21.15.82:0/2299924784 >> 172.21.15.82:0/2299924784 conn(0x7f8a140fe850 msgr2=0x7f8a14100ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:05:57.149 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:57.148+0000 7f8a1b950700 1 -- 172.21.15.82:0/2299924784 shutdown_connections 2024-05-20T16:05:57.149 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:57.148+0000 7f8a1b950700 1 -- 172.21.15.82:0/2299924784 wait complete. 2024-05-20T16:05:57.151 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:57.148+0000 7f8a1b950700 1 Processor -- start 2024-05-20T16:05:57.152 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:57.148+0000 7f8a1b950700 1 -- start start 2024-05-20T16:05:57.152 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:57.149+0000 7f8a1b950700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8a14102de0 0x7f8a14199980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:05:57.152 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:57.149+0000 7f8a1b950700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8a14103860 0x7f8a14199ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:05:57.152 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:57.149+0000 7f8a1b950700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8a14104830 0x7f8a141a0e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:05:57.152 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:57.149+0000 7f8a1b950700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f8a1410eea0 con 0x7f8a14102de0 2024-05-20T16:05:57.152 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:57.149+0000 7f8a1b950700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f8a1410ed20 con 0x7f8a14104830 2024-05-20T16:05:57.152 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:57.149+0000 7f8a1b950700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f8a1410f020 con 0x7f8a14103860 2024-05-20T16:05:57.152 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:57.149+0000 7f8a196ec700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8a14102de0 0x7f8a14199980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:05:57.152 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:57.149+0000 7f8a196ec700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8a14102de0 0x7f8a14199980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:35828/0 (socket says 172.21.15.82:35828) 2024-05-20T16:05:57.152 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:57.149+0000 7f8a196ec700 1 -- 172.21.15.82:0/676497591 learned_addr learned my addr 172.21.15.82:0/676497591 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:05:57.152 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:57.149+0000 7f8a18eeb700 1 --2- 172.21.15.82:0/676497591 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8a14103860 0x7f8a14199ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:05:57.153 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:57.149+0000 7f8a19eed700 1 --2- 172.21.15.82:0/676497591 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8a14104830 0x7f8a141a0e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:05:57.153 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:57.149+0000 7f8a196ec700 1 -- 172.21.15.82:0/676497591 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8a14103860 msgr2=0x7f8a14199ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:05:57.153 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:57.149+0000 7f8a196ec700 1 --2- 172.21.15.82:0/676497591 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8a14103860 0x7f8a14199ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:05:57.153 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:57.149+0000 7f8a196ec700 1 -- 172.21.15.82:0/676497591 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8a14104830 msgr2=0x7f8a141a0e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:05:57.153 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:57.149+0000 7f8a196ec700 1 --2- 172.21.15.82:0/676497591 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8a14104830 0x7f8a141a0e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:05:57.153 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:57.149+0000 7f8a196ec700 1 -- 172.21.15.82:0/676497591 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a141a14e0 con 0x7f8a14102de0 2024-05-20T16:05:57.153 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:57.149+0000 7f8a196ec700 1 --2- 172.21.15.82:0/676497591 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8a14102de0 0x7f8a14199980 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f8a10000c00 tx=0x7f8a100057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:05:57.153 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:57.150+0000 7f8a0a7fc700 1 -- 172.21.15.82:0/676497591 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a10027070 con 0x7f8a14102de0 2024-05-20T16:05:57.153 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:57.150+0000 7f8a0a7fc700 1 -- 172.21.15.82:0/676497591 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a10004150 con 0x7f8a14102de0 2024-05-20T16:05:57.153 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:57.150+0000 7f8a0a7fc700 1 -- 172.21.15.82:0/676497591 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a10022e30 con 0x7f8a14102de0 2024-05-20T16:05:57.153 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:57.150+0000 7f8a1b950700 1 -- 172.21.15.82:0/676497591 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a14075980 con 0x7f8a14102de0 2024-05-20T16:05:57.153 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:57.150+0000 7f8a1b950700 1 -- 172.21.15.82:0/676497591 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a14075eb0 con 0x7f8a14102de0 2024-05-20T16:05:57.153 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:57.151+0000 7f8a0a7fc700 1 -- 172.21.15.82:0/676497591 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f8a1002a070 con 0x7f8a14102de0 2024-05-20T16:05:57.153 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:57.151+0000 7f8a1b950700 1 -- 172.21.15.82:0/676497591 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a14050170 con 0x7f8a14102de0 2024-05-20T16:05:57.154 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:57.151+0000 7f8a0a7fc700 1 --2- 172.21.15.82:0/676497591 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f8a00039330 0x7f8a0003b840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:05:57.154 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:57.151+0000 7f8a0a7fc700 1 -- 172.21.15.82:0/676497591 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f8a1004cfa0 con 0x7f8a14102de0 2024-05-20T16:05:57.154 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:57.151+0000 7f8a18eeb700 1 --2- 172.21.15.82:0/676497591 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f8a00039330 0x7f8a0003b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:05:57.154 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:57.151+0000 7f8a18eeb700 1 --2- 172.21.15.82:0/676497591 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f8a00039330 0x7f8a0003b840 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f8a1419aa30 tx=0x7f8a040073d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:05:57.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:57.156+0000 7f8a0a7fc700 1 -- 172.21.15.82:0/676497591 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f8a10013070 con 0x7f8a14102de0 2024-05-20T16:05:57.245 INFO:journalctl@ceph.mgr.y.smithi160.stdout:May 20 16:05:56 smithi160 podman[37703]: 2024-05-20 16:05:56.891416126 +0000 UTC m=+0.017205428 image pull cfcc61b2d476f611479310eb042aa05f11dd5f60dbb65e3b0d7e0e11d024cdb3 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708 2024-05-20T16:05:57.246 INFO:journalctl@ceph.mgr.y.smithi160.stdout:May 20 16:05:56 smithi160 podman[37703]: 2024-05-20 16:05:56.992721215 +0000 UTC m=+0.118510508 container create 83c00ac57e95195b99534a88fe54e5775f8ff54dd24962ca5b457e02c4dd89a7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mgr-y, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240514, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD) 2024-05-20T16:05:57.246 INFO:journalctl@ceph.mgr.y.smithi160.stdout:May 20 16:05:57 smithi160 podman[37703]: 2024-05-20 16:05:57.170253075 +0000 UTC m=+0.296042369 container init 83c00ac57e95195b99534a88fe54e5775f8ff54dd24962ca5b457e02c4dd89a7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mgr-y, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240514) 2024-05-20T16:05:57.246 INFO:journalctl@ceph.mgr.y.smithi160.stdout:May 20 16:05:57 smithi160 podman[37703]: 2024-05-20 16:05:57.173578154 +0000 UTC m=+0.299367448 container start 83c00ac57e95195b99534a88fe54e5775f8ff54dd24962ca5b457e02c4dd89a7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mgr-y, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240514, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True) 2024-05-20T16:05:57.246 INFO:journalctl@ceph.mgr.y.smithi160.stdout:May 20 16:05:57 smithi160 ceph-mgr[37723]: -- 172.21.15.160:0/1781316736 <== mon.2 v2:172.21.15.160:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5644e9e825a0 con 0x5644e9fc0000 2024-05-20T16:05:57.246 INFO:journalctl@ceph.mgr.y.smithi160.stdout:May 20 16:05:57 smithi160 bash[37703]: 83c00ac57e95195b99534a88fe54e5775f8ff54dd24962ca5b457e02c4dd89a7 2024-05-20T16:05:57.277 INFO:journalctl@ceph.mgr.z.smithi184.stdout:May 20 16:05:56 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mgr-z[38042]: 2024-05-20T16:05:56.971+0000 7ff88a8db200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-20T16:05:57.277 INFO:journalctl@ceph.mgr.z.smithi184.stdout:May 20 16:05:57 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mgr-z[38042]: 2024-05-20T16:05:57.276+0000 7ff88a8db200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-20T16:05:57.366 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:05:57.366+0000 7f8a1b950700 1 -- 172.21.15.82:0/676497591 --> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi082:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f8a1402dfd0 con 0x7f8a00039330 2024-05-20T16:05:57.518 INFO:journalctl@ceph.mgr.y.smithi160.stdout:May 20 16:05:57 smithi160 systemd[1]: Started Ceph mgr.y for 59d1600a-16c2-11ef-bc9a-c7b262605968. 2024-05-20T16:05:57.518 INFO:journalctl@ceph.mgr.y.smithi160.stdout:May 20 16:05:57 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mgr-y[37718]: 2024-05-20T16:05:57.353+0000 7fc514a7e200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-20T16:05:57.584 INFO:journalctl@ceph.mgr.z.smithi184.stdout:May 20 16:05:57 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mgr-z[38042]: 2024-05-20T16:05:57.371+0000 7ff88a8db200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-20T16:05:57.584 INFO:journalctl@ceph.mgr.z.smithi184.stdout:May 20 16:05:57 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mgr-z[38042]: 2024-05-20T16:05:57.583+0000 7ff88a8db200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-20T16:05:57.813 INFO:journalctl@ceph.mgr.y.smithi160.stdout:May 20 16:05:57 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mgr-y[37718]: 2024-05-20T16:05:57.552+0000 7fc514a7e200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-20T16:05:57.813 INFO:journalctl@ceph.mgr.y.smithi160.stdout:May 20 16:05:57 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mgr-y[37718]: 2024-05-20T16:05:57.664+0000 7fc514a7e200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-20T16:05:57.886 INFO:journalctl@ceph.mgr.z.smithi184.stdout:May 20 16:05:57 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mgr-z[38042]: 2024-05-20T16:05:57.788+0000 7ff88a8db200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-20T16:05:58.313 INFO:journalctl@ceph.mgr.y.smithi160.stdout:May 20 16:05:58 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mgr-y[37718]: 2024-05-20T16:05:58.001+0000 7fc514a7e200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-20T16:05:58.313 INFO:journalctl@ceph.mgr.y.smithi160.stdout:May 20 16:05:58 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mgr-y[37718]: 2024-05-20T16:05:58.102+0000 7fc514a7e200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-20T16:05:58.313 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:58 smithi160 ceph-mon[36347]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-20T16:05:58.313 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:58 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:05:58.313 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:58 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:05:58.313 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:58 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-20T16:05:58.313 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:58 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-20T16:05:58.313 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:58 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:05:58.314 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:58 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:05:58.314 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:58 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:05:58.314 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:58 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T16:05:58.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:58 smithi082 ceph-mon[32238]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-20T16:05:58.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:58 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:05:58.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:58 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:05:58.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:58 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-20T16:05:58.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:58 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-20T16:05:58.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:58 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:05:58.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:58 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:05:58.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:58 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:05:58.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:58 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T16:05:58.636 INFO:journalctl@ceph.mgr.z.smithi184.stdout:May 20 16:05:58 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mgr-z[38042]: 2024-05-20T16:05:58.432+0000 7ff88a8db200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-20T16:05:58.637 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:05:58 smithi184 ceph-mon[36209]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-20T16:05:58.637 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:05:58 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:05:58.637 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:05:58 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:05:58.637 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:05:58 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-20T16:05:58.637 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:05:58 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-20T16:05:58.637 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:05:58 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:05:58.637 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:05:58 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:05:58.637 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:05:58 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:05:58.637 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:05:58 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T16:05:59.136 INFO:journalctl@ceph.mgr.z.smithi184.stdout:May 20 16:05:58 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mgr-z[38042]: 2024-05-20T16:05:58.872+0000 7ff88a8db200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-20T16:05:59.136 INFO:journalctl@ceph.mgr.z.smithi184.stdout:May 20 16:05:58 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mgr-z[38042]: 2024-05-20T16:05:58.971+0000 7ff88a8db200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-20T16:05:59.508 INFO:journalctl@ceph.mgr.y.smithi160.stdout:May 20 16:05:59 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mgr-y[37718]: 2024-05-20T16:05:59.342+0000 7fc514a7e200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-20T16:05:59.509 INFO:journalctl@ceph.mgr.y.smithi160.stdout:May 20 16:05:59 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mgr-y[37718]: 2024-05-20T16:05:59.508+0000 7fc514a7e200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-20T16:05:59.509 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:59 smithi160 ceph-mon[36347]: from='client.14214 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi082:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-20T16:05:59.509 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:59 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:05:59.509 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:59 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:05:59.509 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:59 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:05:59.509 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:59 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:05:59.509 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:59 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:05:59.509 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:59 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T16:05:59.509 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:59 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:05:59.509 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:59 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-20T16:05:59.509 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:59 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-20T16:05:59.509 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:05:59 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:05:59.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:59 smithi082 ceph-mon[32238]: from='client.14214 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi082:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-20T16:05:59.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:59 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:05:59.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:59 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:05:59.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:59 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:05:59.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:59 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:05:59.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:59 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:05:59.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:59 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T16:05:59.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:59 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:05:59.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:59 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-20T16:05:59.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:59 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-20T16:05:59.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:05:59 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:05:59.636 INFO:journalctl@ceph.mgr.z.smithi184.stdout:May 20 16:05:59 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mgr-z[38042]: 2024-05-20T16:05:59.172+0000 7ff88a8db200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-20T16:05:59.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:05:59 smithi184 ceph-mon[36209]: from='client.14214 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi082:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-20T16:05:59.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:05:59 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:05:59.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:05:59 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:05:59.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:05:59 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:05:59.637 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:05:59 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:05:59.637 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:05:59 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:05:59.637 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:05:59 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T16:05:59.637 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:05:59 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:05:59.637 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:05:59 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-20T16:05:59.637 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:05:59 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-20T16:05:59.637 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:05:59 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:05:59.812 INFO:journalctl@ceph.mgr.y.smithi160.stdout:May 20 16:05:59 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mgr-y[37718]: 2024-05-20T16:05:59.677+0000 7fc514a7e200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-20T16:06:00.031 INFO:journalctl@ceph.mgr.z.smithi184.stdout:May 20 16:05:59 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mgr-z[38042]: 2024-05-20T16:05:59.696+0000 7ff88a8db200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-20T16:06:00.031 INFO:journalctl@ceph.mgr.z.smithi184.stdout:May 20 16:05:59 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mgr-z[38042]: 2024-05-20T16:05:59.788+0000 7ff88a8db200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-20T16:06:00.234 INFO:journalctl@ceph.mgr.y.smithi160.stdout:May 20 16:05:59 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mgr-y[37718]: 2024-05-20T16:05:59.962+0000 7fc514a7e200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-20T16:06:00.235 INFO:journalctl@ceph.mgr.y.smithi160.stdout:May 20 16:06:00 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mgr-y[37718]: 2024-05-20T16:06:00.054+0000 7fc514a7e200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-20T16:06:00.379 INFO:journalctl@ceph.mgr.z.smithi184.stdout:May 20 16:06:00 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mgr-z[38042]: 2024-05-20T16:06:00.030+0000 7ff88a8db200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-20T16:06:00.379 INFO:journalctl@ceph.mgr.z.smithi184.stdout:May 20 16:06:00 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mgr-z[38042]: 2024-05-20T16:06:00.165+0000 7ff88a8db200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-20T16:06:00.380 INFO:journalctl@ceph.mgr.z.smithi184.stdout:May 20 16:06:00 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mgr-z[38042]: 2024-05-20T16:06:00.263+0000 7ff88a8db200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-20T16:06:00.380 INFO:journalctl@ceph.mgr.z.smithi184.stdout:May 20 16:06:00 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mgr-z[38042]: 2024-05-20T16:06:00.378+0000 7ff88a8db200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-20T16:06:00.380 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:06:00 smithi184 ceph-mon[36209]: Reconfiguring mgr.x (unknown last config time)... 2024-05-20T16:06:00.380 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:06:00 smithi184 ceph-mon[36209]: Reconfiguring daemon mgr.x on smithi082 2024-05-20T16:06:00.380 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:06:00 smithi184 ceph-mon[36209]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-20T16:06:00.380 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:06:00 smithi184 ceph-mon[36209]: from='client.? 172.21.15.82:0/1619746401' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bae842a9-03ec-490e-b56d-f7957235cdc8"}]: dispatch 2024-05-20T16:06:00.381 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:06:00 smithi184 ceph-mon[36209]: from='client.? 172.21.15.82:0/1619746401' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bae842a9-03ec-490e-b56d-f7957235cdc8"}]': finished 2024-05-20T16:06:00.381 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:06:00 smithi184 ceph-mon[36209]: osdmap e5: 1 total, 0 up, 1 in 2024-05-20T16:06:00.381 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:06:00 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-20T16:06:00.381 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:06:00 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:06:00.563 INFO:journalctl@ceph.mgr.y.smithi160.stdout:May 20 16:06:00 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mgr-y[37718]: 2024-05-20T16:06:00.264+0000 7fc514a7e200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-20T16:06:00.563 INFO:journalctl@ceph.mgr.y.smithi160.stdout:May 20 16:06:00 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mgr-y[37718]: 2024-05-20T16:06:00.453+0000 7fc514a7e200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-20T16:06:00.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:06:00 smithi160 ceph-mon[36347]: Reconfiguring mgr.x (unknown last config time)... 2024-05-20T16:06:00.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:06:00 smithi160 ceph-mon[36347]: Reconfiguring daemon mgr.x on smithi082 2024-05-20T16:06:00.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:06:00 smithi160 ceph-mon[36347]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-20T16:06:00.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:06:00 smithi160 ceph-mon[36347]: from='client.? 172.21.15.82:0/1619746401' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bae842a9-03ec-490e-b56d-f7957235cdc8"}]: dispatch 2024-05-20T16:06:00.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:06:00 smithi160 ceph-mon[36347]: from='client.? 172.21.15.82:0/1619746401' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bae842a9-03ec-490e-b56d-f7957235cdc8"}]': finished 2024-05-20T16:06:00.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:06:00 smithi160 ceph-mon[36347]: osdmap e5: 1 total, 0 up, 1 in 2024-05-20T16:06:00.564 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:06:00 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-20T16:06:00.564 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:06:00 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:06:00.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:06:00 smithi082 ceph-mon[32238]: Reconfiguring mgr.x (unknown last config time)... 2024-05-20T16:06:00.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:06:00 smithi082 ceph-mon[32238]: Reconfiguring daemon mgr.x on smithi082 2024-05-20T16:06:00.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:06:00 smithi082 ceph-mon[32238]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-20T16:06:00.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:06:00 smithi082 ceph-mon[32238]: from='client.? 172.21.15.82:0/1619746401' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bae842a9-03ec-490e-b56d-f7957235cdc8"}]: dispatch 2024-05-20T16:06:00.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:06:00 smithi082 ceph-mon[32238]: from='client.? 172.21.15.82:0/1619746401' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bae842a9-03ec-490e-b56d-f7957235cdc8"}]': finished 2024-05-20T16:06:00.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:06:00 smithi082 ceph-mon[32238]: osdmap e5: 1 total, 0 up, 1 in 2024-05-20T16:06:00.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:06:00 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-20T16:06:00.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:06:00 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:06:01.136 INFO:journalctl@ceph.mgr.z.smithi184.stdout:May 20 16:06:00 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mgr-z[38042]: 2024-05-20T16:06:00.682+0000 7ff88a8db200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-20T16:06:01.313 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:06:01 smithi160 ceph-mon[36347]: from='client.? 172.21.15.82:0/2324444408' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-20T16:06:01.313 INFO:journalctl@ceph.mgr.y.smithi160.stdout:May 20 16:06:01 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mgr-y[37718]: 2024-05-20T16:06:01.051+0000 7fc514a7e200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-20T16:06:01.495 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:06:01 smithi184 ceph-mon[36209]: from='client.? 172.21.15.82:0/2324444408' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-20T16:06:01.496 INFO:journalctl@ceph.mgr.z.smithi184.stdout:May 20 16:06:01 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mgr-z[38042]: 2024-05-20T16:06:01.279+0000 7ff88a8db200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-20T16:06:01.496 INFO:journalctl@ceph.mgr.z.smithi184.stdout:May 20 16:06:01 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mgr-z[38042]: 2024-05-20T16:06:01.494+0000 7ff88a8db200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-20T16:06:01.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:06:01 smithi082 ceph-mon[32238]: from='client.? 172.21.15.82:0/2324444408' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-20T16:06:01.769 INFO:journalctl@ceph.mgr.y.smithi160.stdout:May 20 16:06:01 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mgr-y[37718]: 2024-05-20T16:06:01.472+0000 7fc514a7e200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-20T16:06:01.770 INFO:journalctl@ceph.mgr.y.smithi160.stdout:May 20 16:06:01 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mgr-y[37718]: 2024-05-20T16:06:01.568+0000 7fc514a7e200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-20T16:06:01.770 INFO:journalctl@ceph.mgr.y.smithi160.stdout:May 20 16:06:01 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mgr-y[37718]: 2024-05-20T16:06:01.769+0000 7fc514a7e200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-20T16:06:02.249 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:06:02.248+0000 7f8a0a7fc700 1 -- 172.21.15.82:0/676497591 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mgrmap(e 14) ==== 92060+0+0 (secure 0 0 0) 0x7f8a1000fa20 con 0x7f8a14102de0 2024-05-20T16:06:02.562 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:06:02 smithi160 ceph-mon[36347]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-20T16:06:02.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:06:02 smithi160 ceph-mon[36347]: Standby manager daemon z started 2024-05-20T16:06:02.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:06:02 smithi160 ceph-mon[36347]: from='mgr.? 172.21.15.184:0/1142734438' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-20T16:06:02.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:06:02 smithi160 ceph-mon[36347]: from='mgr.? 172.21.15.184:0/1142734438' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-20T16:06:02.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:06:02 smithi160 ceph-mon[36347]: from='mgr.? 172.21.15.184:0/1142734438' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-20T16:06:02.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:06:02 smithi160 ceph-mon[36347]: from='mgr.? 172.21.15.184:0/1142734438' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-20T16:06:02.563 INFO:journalctl@ceph.mgr.y.smithi160.stdout:May 20 16:06:02 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mgr-y[37718]: 2024-05-20T16:06:02.305+0000 7fc514a7e200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-20T16:06:02.563 INFO:journalctl@ceph.mgr.y.smithi160.stdout:May 20 16:06:02 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mgr-y[37718]: 2024-05-20T16:06:02.396+0000 7fc514a7e200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-20T16:06:02.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:06:02 smithi082 ceph-mon[32238]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-20T16:06:02.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:06:02 smithi082 ceph-mon[32238]: Standby manager daemon z started 2024-05-20T16:06:02.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:06:02 smithi082 ceph-mon[32238]: from='mgr.? 172.21.15.184:0/1142734438' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-20T16:06:02.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:06:02 smithi082 ceph-mon[32238]: from='mgr.? 172.21.15.184:0/1142734438' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-20T16:06:02.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:06:02 smithi082 ceph-mon[32238]: from='mgr.? 172.21.15.184:0/1142734438' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-20T16:06:02.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:06:02 smithi082 ceph-mon[32238]: from='mgr.? 172.21.15.184:0/1142734438' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-20T16:06:02.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:06:02 smithi184 ceph-mon[36209]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-20T16:06:02.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:06:02 smithi184 ceph-mon[36209]: Standby manager daemon z started 2024-05-20T16:06:02.637 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:06:02 smithi184 ceph-mon[36209]: from='mgr.? 172.21.15.184:0/1142734438' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-20T16:06:02.637 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:06:02 smithi184 ceph-mon[36209]: from='mgr.? 172.21.15.184:0/1142734438' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-20T16:06:02.637 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:06:02 smithi184 ceph-mon[36209]: from='mgr.? 172.21.15.184:0/1142734438' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-20T16:06:02.637 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:06:02 smithi184 ceph-mon[36209]: from='mgr.? 172.21.15.184:0/1142734438' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-20T16:06:02.979 INFO:journalctl@ceph.mgr.y.smithi160.stdout:May 20 16:06:02 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mgr-y[37718]: 2024-05-20T16:06:02.637+0000 7fc514a7e200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-20T16:06:02.979 INFO:journalctl@ceph.mgr.y.smithi160.stdout:May 20 16:06:02 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mgr-y[37718]: 2024-05-20T16:06:02.769+0000 7fc514a7e200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-20T16:06:02.979 INFO:journalctl@ceph.mgr.y.smithi160.stdout:May 20 16:06:02 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mgr-y[37718]: 2024-05-20T16:06:02.863+0000 7fc514a7e200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-20T16:06:02.979 INFO:journalctl@ceph.mgr.y.smithi160.stdout:May 20 16:06:02 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mgr-y[37718]: 2024-05-20T16:06:02.978+0000 7fc514a7e200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-20T16:06:03.562 INFO:journalctl@ceph.mgr.y.smithi160.stdout:May 20 16:06:03 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mgr-y[37718]: 2024-05-20T16:06:03.280+0000 7fc514a7e200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-20T16:06:03.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:06:03 smithi160 ceph-mon[36347]: mgrmap e14: x(active, since 77s), standbys: z 2024-05-20T16:06:03.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:06:03 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-20T16:06:03.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:06:03 smithi082 ceph-mon[32238]: mgrmap e14: x(active, since 77s), standbys: z 2024-05-20T16:06:03.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:06:03 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-20T16:06:03.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:06:03 smithi184 ceph-mon[36209]: mgrmap e14: x(active, since 77s), standbys: z 2024-05-20T16:06:03.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:06:03 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-20T16:06:04.252 INFO:journalctl@ceph.mgr.y.smithi160.stdout:May 20 16:06:03 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mgr-y[37718]: 2024-05-20T16:06:03.874+0000 7fc514a7e200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-20T16:06:04.252 INFO:journalctl@ceph.mgr.y.smithi160.stdout:May 20 16:06:04 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mgr-y[37718]: 2024-05-20T16:06:04.089+0000 7fc514a7e200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-20T16:06:04.263 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:06:04.261+0000 7f8a0a7fc700 1 -- 172.21.15.82:0/676497591 <== mon.0 v2:172.21.15.82:3300/0 8 ==== mgrmap(e 15) ==== 137777+0+0 (secure 0 0 0) 0x7f8a100163a0 con 0x7f8a14102de0 2024-05-20T16:06:04.333 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:06:04 smithi082 ceph-mon[32238]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-20T16:06:04.333 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:06:04 smithi082 ceph-mon[32238]: Standby manager daemon y started 2024-05-20T16:06:04.333 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:06:04 smithi082 ceph-mon[32238]: from='mgr.? 172.21.15.160:0/4230261430' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-20T16:06:04.333 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:06:04 smithi082 ceph-mon[32238]: from='mgr.? 172.21.15.160:0/4230261430' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-20T16:06:04.333 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:06:04 smithi082 ceph-mon[32238]: from='mgr.? 172.21.15.160:0/4230261430' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-20T16:06:04.333 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:06:04 smithi082 ceph-mon[32238]: from='mgr.? 172.21.15.160:0/4230261430' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-20T16:06:04.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:06:04 smithi160 ceph-mon[36347]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-20T16:06:04.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:06:04 smithi160 ceph-mon[36347]: Standby manager daemon y started 2024-05-20T16:06:04.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:06:04 smithi160 ceph-mon[36347]: from='mgr.? 172.21.15.160:0/4230261430' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-20T16:06:04.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:06:04 smithi160 ceph-mon[36347]: from='mgr.? 172.21.15.160:0/4230261430' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-20T16:06:04.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:06:04 smithi160 ceph-mon[36347]: from='mgr.? 172.21.15.160:0/4230261430' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-20T16:06:04.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:06:04 smithi160 ceph-mon[36347]: from='mgr.? 172.21.15.160:0/4230261430' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-20T16:06:04.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:06:04 smithi184 ceph-mon[36209]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-20T16:06:04.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:06:04 smithi184 ceph-mon[36209]: Standby manager daemon y started 2024-05-20T16:06:04.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:06:04 smithi184 ceph-mon[36209]: from='mgr.? 172.21.15.160:0/4230261430' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-20T16:06:04.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:06:04 smithi184 ceph-mon[36209]: from='mgr.? 172.21.15.160:0/4230261430' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-20T16:06:04.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:06:04 smithi184 ceph-mon[36209]: from='mgr.? 172.21.15.160:0/4230261430' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-20T16:06:04.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:06:04 smithi184 ceph-mon[36209]: from='mgr.? 172.21.15.160:0/4230261430' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-20T16:06:05.373 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:06:05 smithi082 ceph-mon[32238]: mgrmap e15: x(active, since 79s), standbys: z, y 2024-05-20T16:06:05.373 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:06:05 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-20T16:06:05.373 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:06:05 smithi082 ceph-mon[32238]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-20T16:06:05.565 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:06:05 smithi160 ceph-mon[36347]: mgrmap e15: x(active, since 79s), standbys: z, y 2024-05-20T16:06:05.565 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:06:05 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-20T16:06:05.565 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:06:05 smithi160 ceph-mon[36347]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-20T16:06:05.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:06:05 smithi184 ceph-mon[36209]: mgrmap e15: x(active, since 79s), standbys: z, y 2024-05-20T16:06:05.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:06:05 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-20T16:06:05.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:06:05 smithi184 ceph-mon[36209]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-20T16:06:06.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:06:06 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:06:06.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:06:06 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:06:06.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:06:06 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T16:06:06.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:06:06 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:06:06.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:06:06 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T16:06:06.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:06:06 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:06:06.813 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:06:06 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:06:06.813 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:06:06 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:06:06.813 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:06:06 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T16:06:06.813 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:06:06 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:06:06.813 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:06:06 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T16:06:06.813 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:06:06 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:06:06.886 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:06:06 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:06:06.886 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:06:06 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:06:06.886 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:06:06 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T16:06:06.886 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:06:06 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:06:06.887 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:06:06 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T16:06:06.887 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:06:06 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:06:07.602 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:06:07 smithi082 ceph-mon[32238]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-20T16:06:07.802 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:06:07 smithi160 ceph-mon[36347]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-20T16:06:07.886 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:06:07 smithi184 ceph-mon[36209]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-20T16:06:08.538 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:06:08 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-05-20T16:06:08.538 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:06:08 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:06:08.538 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:06:08 smithi082 ceph-mon[32238]: Deploying daemon osd.0 on smithi082 2024-05-20T16:06:08.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:06:08 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-05-20T16:06:08.813 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:06:08 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:06:08.813 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:06:08 smithi160 ceph-mon[36347]: Deploying daemon osd.0 on smithi082 2024-05-20T16:06:08.886 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:06:08 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-05-20T16:06:08.886 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:06:08 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:06:08.887 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:06:08 smithi184 ceph-mon[36209]: Deploying daemon osd.0 on smithi082 2024-05-20T16:06:09.813 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:06:09 smithi160 ceph-mon[36347]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-20T16:06:09.825 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:06:09 smithi082 ceph-mon[32238]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-20T16:06:09.886 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:06:09 smithi184 ceph-mon[36209]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-20T16:06:12.398 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:06:12 smithi082 ceph-mon[32238]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-20T16:06:12.562 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:06:12 smithi160 ceph-mon[36347]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-20T16:06:12.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:06:12 smithi184 ceph-mon[36209]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-20T16:06:14.314 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:06:13 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:06:14.314 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:06:13 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T16:06:14.314 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:06:13 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:06:14.314 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:06:13 smithi160 ceph-mon[36347]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-20T16:06:14.360 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:06:13 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:06:14.360 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:06:13 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T16:06:14.360 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:06:13 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:06:14.360 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:06:13 smithi082 ceph-mon[32238]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-20T16:06:14.386 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:06:13 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:06:14.386 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:06:13 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T16:06:14.386 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:06:13 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:06:14.386 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:06:13 smithi184 ceph-mon[36209]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-20T16:06:15.970 INFO:teuthology.orchestra.run.smithi082.stdout:Created osd(s) 0 on host 'smithi082' 2024-05-20T16:06:15.970 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:06:15.969+0000 7f8a0a7fc700 1 -- 172.21.15.82:0/676497591 <== mgr.14150 v2:172.21.15.82:6800/379530823 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f8a1402dfd0 con 0x7f8a00039330 2024-05-20T16:06:15.974 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:06:15.973+0000 7f8a1b950700 1 -- 172.21.15.82:0/676497591 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f8a00039330 msgr2=0x7f8a0003b840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:06:15.974 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:06:15.973+0000 7f8a1b950700 1 --2- 172.21.15.82:0/676497591 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f8a00039330 0x7f8a0003b840 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f8a1419aa30 tx=0x7f8a040073d0 comp rx=0 tx=0).stop 2024-05-20T16:06:15.974 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:06:15.973+0000 7f8a1b950700 1 -- 172.21.15.82:0/676497591 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8a14102de0 msgr2=0x7f8a14199980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:06:15.974 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:06:15.973+0000 7f8a1b950700 1 --2- 172.21.15.82:0/676497591 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8a14102de0 0x7f8a14199980 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f8a10000c00 tx=0x7f8a100057c0 comp rx=0 tx=0).stop 2024-05-20T16:06:15.974 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:06:15.973+0000 7f8a1b950700 1 -- 172.21.15.82:0/676497591 shutdown_connections 2024-05-20T16:06:15.974 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:06:15.973+0000 7f8a1b950700 1 --2- 172.21.15.82:0/676497591 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f8a00039330 0x7f8a0003b840 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:06:15.974 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:06:15.973+0000 7f8a1b950700 1 --2- 172.21.15.82:0/676497591 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8a14104830 0x7f8a141a0e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:06:15.974 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:06:15.973+0000 7f8a1b950700 1 --2- 172.21.15.82:0/676497591 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8a14102de0 0x7f8a14199980 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:06:15.974 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:06:15.973+0000 7f8a1b950700 1 --2- 172.21.15.82:0/676497591 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8a14103860 0x7f8a14199ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:06:15.974 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:06:15.973+0000 7f8a1b950700 1 -- 172.21.15.82:0/676497591 >> 172.21.15.82:0/676497591 conn(0x7f8a140fe850 msgr2=0x7f8a14107550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:06:15.975 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:06:15.973+0000 7f8a1b950700 1 -- 172.21.15.82:0/676497591 shutdown_connections 2024-05-20T16:06:15.975 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:06:15.973+0000 7f8a1b950700 1 -- 172.21.15.82:0/676497591 wait complete. 2024-05-20T16:06:16.554 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:06:16 smithi082 ceph-mon[32238]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-20T16:06:16.554 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:06:16 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:06:16.554 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:06:16 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:06:16.554 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:06:16 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:06:16.554 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:06:16 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:06:16.562 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:06:16 smithi160 ceph-mon[36347]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-20T16:06:16.562 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:06:16 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:06:16.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:06:16 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:06:16.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:06:16 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:06:16.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:06:16 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:06:16.584 DEBUG:teuthology.orchestra.run.smithi082:osd.0> sudo journalctl -f -n 0 -u ceph-59d1600a-16c2-11ef-bc9a-c7b262605968@osd.0.service 2024-05-20T16:06:16.586 INFO:tasks.cephadm:Deploying osd.1 on smithi160 with /dev/vg_nvme/lv_4... 2024-05-20T16:06:16.587 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 59d1600a-16c2-11ef-bc9a-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-05-20T16:06:16.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:06:16 smithi184 ceph-mon[36209]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-20T16:06:16.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:06:16 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:06:16.637 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:06:16 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:06:16.637 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:06:16 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:06:16.637 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:06:16 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:06:16.827 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/mon.b/config 2024-05-20T16:06:17.091 INFO:journalctl@ceph.osd.0.smithi082.stdout:May 20 16:06:16 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-0[46979]: 2024-05-20T16:06:16.831+0000 7f2b875cb7c0 -1 osd.0 0 log_to_monitors true 2024-05-20T16:06:17.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:06:17 smithi082 ceph-mon[32238]: from='osd.0 [v2:172.21.15.82:6802/1484330167,v1:172.21.15.82:6803/1484330167]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-20T16:06:17.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:06:17 smithi160 ceph-mon[36347]: from='osd.0 [v2:172.21.15.82:6802/1484330167,v1:172.21.15.82:6803/1484330167]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-20T16:06:17.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:06:17 smithi184 ceph-mon[36209]: from='osd.0 [v2:172.21.15.82:6802/1484330167,v1:172.21.15.82:6803/1484330167]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-20T16:06:18.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:06:18 smithi160 ceph-mon[36347]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-20T16:06:18.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:06:18 smithi160 ceph-mon[36347]: from='osd.0 [v2:172.21.15.82:6802/1484330167,v1:172.21.15.82:6803/1484330167]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-05-20T16:06:18.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:06:18 smithi160 ceph-mon[36347]: osdmap e6: 1 total, 0 up, 1 in 2024-05-20T16:06:18.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:06:18 smithi160 ceph-mon[36347]: from='osd.0 [v2:172.21.15.82:6802/1484330167,v1:172.21.15.82:6803/1484330167]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]: dispatch 2024-05-20T16:06:18.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:06:18 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-20T16:06:18.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:06:18 smithi082 ceph-mon[32238]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-20T16:06:18.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:06:18 smithi082 ceph-mon[32238]: from='osd.0 [v2:172.21.15.82:6802/1484330167,v1:172.21.15.82:6803/1484330167]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-05-20T16:06:18.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:06:18 smithi082 ceph-mon[32238]: osdmap e6: 1 total, 0 up, 1 in 2024-05-20T16:06:18.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:06:18 smithi082 ceph-mon[32238]: from='osd.0 [v2:172.21.15.82:6802/1484330167,v1:172.21.15.82:6803/1484330167]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]: dispatch 2024-05-20T16:06:18.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:06:18 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-20T16:06:18.610 INFO:journalctl@ceph.osd.0.smithi082.stdout:May 20 16:06:18 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-0[46979]: 2024-05-20T16:06:18.248+0000 7f2b7cc2f700 -1 osd.0 0 waiting for initial osdmap 2024-05-20T16:06:18.611 INFO:journalctl@ceph.osd.0.smithi082.stdout:May 20 16:06:18 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-0[46979]: 2024-05-20T16:06:18.252+0000 7f2b789e4700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-20T16:06:18.637 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:06:18 smithi184 ceph-mon[36209]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-20T16:06:18.637 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:06:18 smithi184 ceph-mon[36209]: from='osd.0 [v2:172.21.15.82:6802/1484330167,v1:172.21.15.82:6803/1484330167]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-05-20T16:06:18.637 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:06:18 smithi184 ceph-mon[36209]: osdmap e6: 1 total, 0 up, 1 in 2024-05-20T16:06:18.637 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:06:18 smithi184 ceph-mon[36209]: from='osd.0 [v2:172.21.15.82:6802/1484330167,v1:172.21.15.82:6803/1484330167]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]: dispatch 2024-05-20T16:06:18.637 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:06:18 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-20T16:06:18.693 INFO:teuthology.orchestra.run.smithi160.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-05-20T16:06:18.693 INFO:teuthology.orchestra.run.smithi160.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-05-20T16:06:18.693 INFO:teuthology.orchestra.run.smithi160.stderr: stderr: 10+0 records in 2024-05-20T16:06:18.693 INFO:teuthology.orchestra.run.smithi160.stderr:10+0 records out 2024-05-20T16:06:18.693 INFO:teuthology.orchestra.run.smithi160.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0130772 s, 802 MB/s 2024-05-20T16:06:18.693 INFO:teuthology.orchestra.run.smithi160.stderr:--> Zapping successful for: 2024-05-20T16:06:19.263 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 59d1600a-16c2-11ef-bc9a-c7b262605968 -- ceph orch daemon add osd smithi160:vg_nvme/lv_4 2024-05-20T16:06:19.285 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:06:19 smithi160 ceph-mon[36347]: from='osd.0 [v2:172.21.15.82:6802/1484330167,v1:172.21.15.82:6803/1484330167]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]': finished 2024-05-20T16:06:19.285 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:06:19 smithi160 ceph-mon[36347]: osdmap e7: 1 total, 0 up, 1 in 2024-05-20T16:06:19.285 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:06:19 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-20T16:06:19.285 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:06:19 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-20T16:06:19.501 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:06:19 smithi082 ceph-mon[32238]: from='osd.0 [v2:172.21.15.82:6802/1484330167,v1:172.21.15.82:6803/1484330167]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]': finished 2024-05-20T16:06:19.501 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:06:19 smithi082 ceph-mon[32238]: osdmap e7: 1 total, 0 up, 1 in 2024-05-20T16:06:19.502 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:06:19 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-20T16:06:19.502 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:06:19 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-20T16:06:19.502 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/mon.b/config 2024-05-20T16:06:19.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:06:19 smithi184 ceph-mon[36209]: from='osd.0 [v2:172.21.15.82:6802/1484330167,v1:172.21.15.82:6803/1484330167]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]': finished 2024-05-20T16:06:19.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:06:19 smithi184 ceph-mon[36209]: osdmap e7: 1 total, 0 up, 1 in 2024-05-20T16:06:19.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:06:19 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-20T16:06:19.637 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:06:19 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-20T16:06:20.502 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:06:20 smithi160 ceph-mon[36347]: purged_snaps scrub starts 2024-05-20T16:06:20.503 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:06:20 smithi160 ceph-mon[36347]: purged_snaps scrub ok 2024-05-20T16:06:20.503 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:06:20 smithi160 ceph-mon[36347]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-20T16:06:20.503 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:06:20 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-20T16:06:20.503 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:06:20 smithi160 ceph-mon[36347]: osd.0 [v2:172.21.15.82:6802/1484330167,v1:172.21.15.82:6803/1484330167] boot 2024-05-20T16:06:20.503 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:06:20 smithi160 ceph-mon[36347]: osdmap e8: 1 total, 1 up, 1 in 2024-05-20T16:06:20.503 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:06:20 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-20T16:06:20.503 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:06:20 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:06:20.503 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:06:20 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:06:20.503 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:06:20 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-20T16:06:20.503 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:06:20 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:06:20.503 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:06:20 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:06:20.503 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:06:20 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T16:06:20.503 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:06:20 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:06:20.504 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:06:20 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T16:06:20.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:06:20 smithi082 ceph-mon[32238]: purged_snaps scrub starts 2024-05-20T16:06:20.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:06:20 smithi082 ceph-mon[32238]: purged_snaps scrub ok 2024-05-20T16:06:20.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:06:20 smithi082 ceph-mon[32238]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-20T16:06:20.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:06:20 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-20T16:06:20.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:06:20 smithi082 ceph-mon[32238]: osd.0 [v2:172.21.15.82:6802/1484330167,v1:172.21.15.82:6803/1484330167] boot 2024-05-20T16:06:20.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:06:20 smithi082 ceph-mon[32238]: osdmap e8: 1 total, 1 up, 1 in 2024-05-20T16:06:20.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:06:20 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-20T16:06:20.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:06:20 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:06:20.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:06:20 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:06:20.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:06:20 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-20T16:06:20.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:06:20 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:06:20.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:06:20 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:06:20.611 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:06:20 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T16:06:20.611 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:06:20 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:06:20.611 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:06:20 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T16:06:20.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:06:20 smithi184 ceph-mon[36209]: purged_snaps scrub starts 2024-05-20T16:06:20.637 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:06:20 smithi184 ceph-mon[36209]: purged_snaps scrub ok 2024-05-20T16:06:20.637 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:06:20 smithi184 ceph-mon[36209]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-20T16:06:20.637 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:06:20 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-20T16:06:20.637 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:06:20 smithi184 ceph-mon[36209]: osd.0 [v2:172.21.15.82:6802/1484330167,v1:172.21.15.82:6803/1484330167] boot 2024-05-20T16:06:20.637 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:06:20 smithi184 ceph-mon[36209]: osdmap e8: 1 total, 1 up, 1 in 2024-05-20T16:06:20.637 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:06:20 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-20T16:06:20.637 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:06:20 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:06:20.638 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:06:20 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:06:20.638 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:06:20 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-20T16:06:20.638 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:06:20 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:06:20.638 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:06:20 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:06:20.638 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:06:20 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T16:06:20.638 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:06:20 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:06:20.638 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:06:20 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T16:06:21.126 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-20T16:06:21.124+0000 7fa16094c700 1 -- 172.21.15.160:0/874075746 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa15c103200 msgr2=0x7fa15c103610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:06:21.129 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-20T16:06:21.124+0000 7fa16094c700 1 --2- 172.21.15.160:0/874075746 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa15c103200 0x7fa15c103610 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fa14c009990 tx=0x7fa14c009ca0 comp rx=0 tx=0).stop 2024-05-20T16:06:21.129 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-20T16:06:21.125+0000 7fa16094c700 1 -- 172.21.15.160:0/874075746 shutdown_connections 2024-05-20T16:06:21.129 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-20T16:06:21.125+0000 7fa16094c700 1 --2- 172.21.15.160:0/874075746 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa15c103ba0 0x7fa15c10a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:06:21.129 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-20T16:06:21.125+0000 7fa16094c700 1 --2- 172.21.15.160:0/874075746 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa15c1020d0 0x7fa15c1024e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:06:21.130 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-20T16:06:21.125+0000 7fa16094c700 1 --2- 172.21.15.160:0/874075746 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa15c103200 0x7fa15c103610 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:06:21.130 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-20T16:06:21.125+0000 7fa16094c700 1 -- 172.21.15.160:0/874075746 >> 172.21.15.160:0/874075746 conn(0x7fa15c0fd6f0 msgr2=0x7fa15c0ffbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:06:21.130 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-20T16:06:21.125+0000 7fa16094c700 1 -- 172.21.15.160:0/874075746 shutdown_connections 2024-05-20T16:06:21.130 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-20T16:06:21.125+0000 7fa16094c700 1 -- 172.21.15.160:0/874075746 wait complete. 2024-05-20T16:06:21.132 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-20T16:06:21.125+0000 7fa16094c700 1 Processor -- start 2024-05-20T16:06:21.132 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-20T16:06:21.125+0000 7fa16094c700 1 -- start start 2024-05-20T16:06:21.132 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-20T16:06:21.126+0000 7fa16094c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa15c1020d0 0x7fa15c199880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:06:21.133 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-20T16:06:21.126+0000 7fa16094c700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa15c103200 0x7fa15c199dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:06:21.133 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-20T16:06:21.126+0000 7fa16094c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa15c103ba0 0x7fa15c1a0d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:06:21.133 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-20T16:06:21.126+0000 7fa16094c700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fa15c10dcb0 con 0x7fa15c1020d0 2024-05-20T16:06:21.133 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-20T16:06:21.126+0000 7fa16094c700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fa15c10db30 con 0x7fa15c103ba0 2024-05-20T16:06:21.133 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-20T16:06:21.126+0000 7fa16094c700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa15c10de30 con 0x7fa15c103200 2024-05-20T16:06:21.133 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-20T16:06:21.126+0000 7fa15ad9d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa15c103ba0 0x7fa15c1a0d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:06:21.133 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-20T16:06:21.126+0000 7fa15ad9d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa15c103ba0 0x7fa15c1a0d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:44390/0 (socket says 172.21.15.160:44390) 2024-05-20T16:06:21.133 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-20T16:06:21.126+0000 7fa15ad9d700 1 -- 172.21.15.160:0/1983391644 learned_addr learned my addr 172.21.15.160:0/1983391644 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-20T16:06:21.133 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-20T16:06:21.126+0000 7fa15a59c700 1 --2- 172.21.15.160:0/1983391644 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa15c1020d0 0x7fa15c199880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:06:21.133 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-20T16:06:21.126+0000 7fa159d9b700 1 --2- 172.21.15.160:0/1983391644 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa15c103200 0x7fa15c199dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:06:21.133 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-20T16:06:21.126+0000 7fa15ad9d700 1 -- 172.21.15.160:0/1983391644 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa15c103200 msgr2=0x7fa15c199dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:06:21.133 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-20T16:06:21.126+0000 7fa15ad9d700 1 --2- 172.21.15.160:0/1983391644 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa15c103200 0x7fa15c199dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:06:21.134 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-20T16:06:21.126+0000 7fa15ad9d700 1 -- 172.21.15.160:0/1983391644 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa15c1020d0 msgr2=0x7fa15c199880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:06:21.134 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-20T16:06:21.126+0000 7fa15ad9d700 1 --2- 172.21.15.160:0/1983391644 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa15c1020d0 0x7fa15c199880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:06:21.134 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-20T16:06:21.126+0000 7fa15ad9d700 1 -- 172.21.15.160:0/1983391644 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa15c1a13e0 con 0x7fa15c103ba0 2024-05-20T16:06:21.134 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-20T16:06:21.126+0000 7fa15a59c700 1 --2- 172.21.15.160:0/1983391644 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa15c1020d0 0x7fa15c199880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T16:06:21.134 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-20T16:06:21.126+0000 7fa15ad9d700 1 --2- 172.21.15.160:0/1983391644 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa15c103ba0 0x7fa15c1a0d60 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa15400ef30 tx=0x7fa15400c450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:06:21.134 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-20T16:06:21.126+0000 7fa14b7fe700 1 -- 172.21.15.160:0/1983391644 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1540092e0 con 0x7fa15c103ba0 2024-05-20T16:06:21.134 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-20T16:06:21.127+0000 7fa14b7fe700 1 -- 172.21.15.160:0/1983391644 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa154004bb0 con 0x7fa15c103ba0 2024-05-20T16:06:21.134 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-20T16:06:21.127+0000 7fa16094c700 1 -- 172.21.15.160:0/1983391644 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa15c1a1670 con 0x7fa15c103ba0 2024-05-20T16:06:21.134 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-20T16:06:21.127+0000 7fa14b7fe700 1 -- 172.21.15.160:0/1983391644 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa154004280 con 0x7fa15c103ba0 2024-05-20T16:06:21.134 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-20T16:06:21.127+0000 7fa16094c700 1 -- 172.21.15.160:0/1983391644 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa15c1a1b60 con 0x7fa15c103ba0 2024-05-20T16:06:21.134 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-20T16:06:21.127+0000 7fa16094c700 1 -- 172.21.15.160:0/1983391644 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa15c050170 con 0x7fa15c103ba0 2024-05-20T16:06:21.134 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-20T16:06:21.129+0000 7fa14b7fe700 1 -- 172.21.15.160:0/1983391644 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 15) ==== 137777+0+0 (secure 0 0 0) 0x7fa15401b030 con 0x7fa15c103ba0 2024-05-20T16:06:21.134 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-20T16:06:21.130+0000 7fa14b7fe700 1 --2- 172.21.15.160:0/1983391644 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fa1440a2b90 0x7fa1440a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:06:21.135 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-20T16:06:21.130+0000 7fa14b7fe700 1 -- 172.21.15.160:0/1983391644 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1767+0+0 (secure 0 0 0) 0x7fa1540cdb60 con 0x7fa15c103ba0 2024-05-20T16:06:21.135 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-20T16:06:21.130+0000 7fa15a59c700 1 --2- 172.21.15.160:0/1983391644 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fa1440a2b90 0x7fa1440a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:06:21.135 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-20T16:06:21.130+0000 7fa15a59c700 1 --2- 172.21.15.160:0/1983391644 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fa1440a2b90 0x7fa1440a50a0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fa150005fd0 tx=0x7fa150005f00 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:06:21.135 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-20T16:06:21.134+0000 7fa14b7fe700 1 -- 172.21.15.160:0/1983391644 <== mon.2 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fa1540cde20 con 0x7fa15c103ba0 2024-05-20T16:06:21.340 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-20T16:06:21.339+0000 7fa16094c700 1 -- 172.21.15.160:0/1983391644 --> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi160:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fa15c0ff6b0 con 0x7fa1440a2b90 2024-05-20T16:06:21.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:06:21 smithi160 ceph-mon[36347]: Detected new or changed devices on smithi082 2024-05-20T16:06:21.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:06:21 smithi160 ceph-mon[36347]: Adjusting osd_memory_target on smithi082 to 17088M 2024-05-20T16:06:21.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:06:21 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:06:21.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:06:21 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T16:06:21.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:06:21 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:06:21.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:06:21 smithi160 ceph-mon[36347]: osdmap e9: 1 total, 1 up, 1 in 2024-05-20T16:06:21.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:06:21 smithi082 ceph-mon[32238]: Detected new or changed devices on smithi082 2024-05-20T16:06:21.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:06:21 smithi082 ceph-mon[32238]: Adjusting osd_memory_target on smithi082 to 17088M 2024-05-20T16:06:21.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:06:21 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:06:21.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:06:21 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T16:06:21.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:06:21 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:06:21.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:06:21 smithi082 ceph-mon[32238]: osdmap e9: 1 total, 1 up, 1 in 2024-05-20T16:06:21.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:06:21 smithi184 ceph-mon[36209]: Detected new or changed devices on smithi082 2024-05-20T16:06:21.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:06:21 smithi184 ceph-mon[36209]: Adjusting osd_memory_target on smithi082 to 17088M 2024-05-20T16:06:21.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:06:21 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:06:21.637 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:06:21 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T16:06:21.637 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:06:21 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:06:21.637 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:06:21 smithi184 ceph-mon[36209]: osdmap e9: 1 total, 1 up, 1 in 2024-05-20T16:06:22.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:06:22 smithi160 ceph-mon[36347]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-20T16:06:22.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:06:22 smithi160 ceph-mon[36347]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi160:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-20T16:06:22.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:06:22 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-20T16:06:22.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:06:22 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-20T16:06:22.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:06:22 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:06:22.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:06:22 smithi082 ceph-mon[32238]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-20T16:06:22.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:06:22 smithi082 ceph-mon[32238]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi160:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-20T16:06:22.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:06:22 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-20T16:06:22.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:06:22 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-20T16:06:22.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:06:22 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:06:22.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:06:22 smithi184 ceph-mon[36209]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-20T16:06:22.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:06:22 smithi184 ceph-mon[36209]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi160:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-20T16:06:22.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:06:22 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-20T16:06:22.637 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:06:22 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-20T16:06:22.637 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:06:22 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:06:23.562 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:06:23 smithi160 ceph-mon[36347]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-20T16:06:23.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:06:23 smithi082 ceph-mon[32238]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-20T16:06:23.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:06:23 smithi184 ceph-mon[36209]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-20T16:06:24.537 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:06:24 smithi184 ceph-mon[36209]: from='client.? 172.21.15.160:0/3261044986' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "59f6a2ba-b5de-4161-81c5-92a417abacdf"}]: dispatch 2024-05-20T16:06:24.538 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:06:24 smithi184 ceph-mon[36209]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "59f6a2ba-b5de-4161-81c5-92a417abacdf"}]: dispatch 2024-05-20T16:06:24.538 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:06:24 smithi184 ceph-mon[36209]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "59f6a2ba-b5de-4161-81c5-92a417abacdf"}]': finished 2024-05-20T16:06:24.538 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:06:24 smithi184 ceph-mon[36209]: osdmap e10: 2 total, 1 up, 2 in 2024-05-20T16:06:24.538 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:06:24 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-20T16:06:24.562 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:06:24 smithi160 ceph-mon[36347]: from='client.? 172.21.15.160:0/3261044986' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "59f6a2ba-b5de-4161-81c5-92a417abacdf"}]: dispatch 2024-05-20T16:06:24.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:06:24 smithi160 ceph-mon[36347]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "59f6a2ba-b5de-4161-81c5-92a417abacdf"}]: dispatch 2024-05-20T16:06:24.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:06:24 smithi160 ceph-mon[36347]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "59f6a2ba-b5de-4161-81c5-92a417abacdf"}]': finished 2024-05-20T16:06:24.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:06:24 smithi160 ceph-mon[36347]: osdmap e10: 2 total, 1 up, 2 in 2024-05-20T16:06:24.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:06:24 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-20T16:06:24.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:06:24 smithi082 ceph-mon[32238]: from='client.? 172.21.15.160:0/3261044986' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "59f6a2ba-b5de-4161-81c5-92a417abacdf"}]: dispatch 2024-05-20T16:06:24.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:06:24 smithi082 ceph-mon[32238]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "59f6a2ba-b5de-4161-81c5-92a417abacdf"}]: dispatch 2024-05-20T16:06:24.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:06:24 smithi082 ceph-mon[32238]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "59f6a2ba-b5de-4161-81c5-92a417abacdf"}]': finished 2024-05-20T16:06:24.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:06:24 smithi082 ceph-mon[32238]: osdmap e10: 2 total, 1 up, 2 in 2024-05-20T16:06:24.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:06:24 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-20T16:06:25.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:06:25 smithi160 ceph-mon[36347]: from='client.? 172.21.15.160:0/701077382' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-20T16:06:25.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:06:25 smithi160 ceph-mon[36347]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-20T16:06:25.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:06:25 smithi082 ceph-mon[32238]: from='client.? 172.21.15.160:0/701077382' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-20T16:06:25.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:06:25 smithi082 ceph-mon[32238]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-20T16:06:25.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:06:25 smithi184 ceph-mon[36209]: from='client.? 172.21.15.160:0/701077382' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-20T16:06:25.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:06:25 smithi184 ceph-mon[36209]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-20T16:06:28.562 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:06:28 smithi160 ceph-mon[36347]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-20T16:06:28.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:06:28 smithi082 ceph-mon[32238]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-20T16:06:28.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:06:28 smithi184 ceph-mon[36209]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-20T16:06:30.313 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:06:30 smithi160 ceph-mon[36347]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-20T16:06:30.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:06:30 smithi082 ceph-mon[32238]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-20T16:06:30.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:06:30 smithi184 ceph-mon[36209]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-20T16:06:31.562 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:06:31 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-05-20T16:06:31.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:06:31 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:06:31.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:06:31 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-05-20T16:06:31.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:06:31 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:06:31.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:06:31 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-05-20T16:06:31.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:06:31 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:06:32.562 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:06:32 smithi160 ceph-mon[36347]: Deploying daemon osd.1 on smithi160 2024-05-20T16:06:32.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:06:32 smithi160 ceph-mon[36347]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-20T16:06:32.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:06:32 smithi082 ceph-mon[32238]: Deploying daemon osd.1 on smithi160 2024-05-20T16:06:32.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:06:32 smithi082 ceph-mon[32238]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-20T16:06:32.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:06:32 smithi184 ceph-mon[36209]: Deploying daemon osd.1 on smithi160 2024-05-20T16:06:32.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:06:32 smithi184 ceph-mon[36209]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-20T16:06:34.562 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:06:34 smithi160 ceph-mon[36347]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-20T16:06:34.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:06:34 smithi082 ceph-mon[32238]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-20T16:06:34.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:06:34 smithi184 ceph-mon[36209]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-20T16:06:36.543 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:06:36 smithi160 ceph-mon[36347]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-20T16:06:36.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:06:36 smithi082 ceph-mon[32238]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-20T16:06:36.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:06:36 smithi184 ceph-mon[36209]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-20T16:06:37.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:06:37 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:06:37.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:06:37 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:06:37.860 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:06:37 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T16:06:37.860 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:06:37 smithi082 ceph-mon[32238]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-20T16:06:37.886 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:06:37 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:06:37.886 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:06:37 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:06:37.886 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:06:37 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T16:06:37.886 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:06:37 smithi184 ceph-mon[36209]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-20T16:06:38.063 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:06:37 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:06:38.063 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:06:37 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:06:38.063 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:06:37 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T16:06:38.063 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:06:37 smithi160 ceph-mon[36347]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-20T16:06:40.005 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-20T16:06:40.003+0000 7fa14b7fe700 1 -- 172.21.15.160:0/1983391644 <== mgr.14150 v2:172.21.15.82:6800/379530823 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fa15c0ff6b0 con 0x7fa1440a2b90 2024-05-20T16:06:40.005 INFO:teuthology.orchestra.run.smithi160.stdout:Created osd(s) 1 on host 'smithi160' 2024-05-20T16:06:40.007 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-20T16:06:40.006+0000 7fa16094c700 1 -- 172.21.15.160:0/1983391644 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fa1440a2b90 msgr2=0x7fa1440a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:06:40.007 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-20T16:06:40.006+0000 7fa16094c700 1 --2- 172.21.15.160:0/1983391644 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fa1440a2b90 0x7fa1440a50a0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fa150005fd0 tx=0x7fa150005f00 comp rx=0 tx=0).stop 2024-05-20T16:06:40.008 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-20T16:06:40.006+0000 7fa16094c700 1 -- 172.21.15.160:0/1983391644 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa15c103ba0 msgr2=0x7fa15c1a0d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:06:40.008 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-20T16:06:40.006+0000 7fa16094c700 1 --2- 172.21.15.160:0/1983391644 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa15c103ba0 0x7fa15c1a0d60 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa15400ef30 tx=0x7fa15400c450 comp rx=0 tx=0).stop 2024-05-20T16:06:40.008 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-20T16:06:40.006+0000 7fa16094c700 1 -- 172.21.15.160:0/1983391644 shutdown_connections 2024-05-20T16:06:40.008 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-20T16:06:40.006+0000 7fa16094c700 1 --2- 172.21.15.160:0/1983391644 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fa1440a2b90 0x7fa1440a50a0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:06:40.008 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-20T16:06:40.006+0000 7fa16094c700 1 --2- 172.21.15.160:0/1983391644 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa15c103ba0 0x7fa15c1a0d60 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:06:40.009 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-20T16:06:40.006+0000 7fa16094c700 1 --2- 172.21.15.160:0/1983391644 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa15c1020d0 0x7fa15c199880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:06:40.009 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-20T16:06:40.006+0000 7fa16094c700 1 --2- 172.21.15.160:0/1983391644 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa15c103200 0x7fa15c199dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:06:40.009 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-20T16:06:40.006+0000 7fa16094c700 1 -- 172.21.15.160:0/1983391644 >> 172.21.15.160:0/1983391644 conn(0x7fa15c0fd6f0 msgr2=0x7fa15c0fefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:06:40.009 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-20T16:06:40.006+0000 7fa16094c700 1 -- 172.21.15.160:0/1983391644 shutdown_connections 2024-05-20T16:06:40.009 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-20T16:06:40.006+0000 7fa16094c700 1 -- 172.21.15.160:0/1983391644 wait complete. 2024-05-20T16:06:40.519 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:06:40 smithi160 ceph-mon[36347]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-20T16:06:40.519 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:06:40 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:06:40.519 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:06:40 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:06:40.519 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:06:40 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:06:40.519 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:06:40 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T16:06:40.519 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:06:40 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:06:40.520 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:06:40 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:06:40.520 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:06:40 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:06:40.520 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:06:40 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T16:06:40.563 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:06:40 smithi082 ceph-mon[32238]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-20T16:06:40.563 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:06:40 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:06:40.564 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:06:40 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:06:40.564 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:06:40 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:06:40.564 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:06:40 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T16:06:40.564 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:06:40 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:06:40.564 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:06:40 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:06:40.564 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:06:40 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:06:40.564 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:06:40 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T16:06:40.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:06:40 smithi184 ceph-mon[36209]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-20T16:06:40.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:06:40 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:06:40.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:06:40 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:06:40.637 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:06:40 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:06:40.637 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:06:40 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T16:06:40.637 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:06:40 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:06:40.637 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:06:40 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:06:40.637 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:06:40 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:06:40.637 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:06:40 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T16:06:40.645 DEBUG:teuthology.orchestra.run.smithi160:osd.1> sudo journalctl -f -n 0 -u ceph-59d1600a-16c2-11ef-bc9a-c7b262605968@osd.1.service 2024-05-20T16:06:40.689 INFO:tasks.cephadm:Deploying osd.2 on smithi184 with /dev/vg_nvme/lv_4... 2024-05-20T16:06:40.690 DEBUG:teuthology.orchestra.run.smithi184:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 59d1600a-16c2-11ef-bc9a-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-05-20T16:06:40.933 INFO:teuthology.orchestra.run.smithi184.stderr:Inferring config /var/lib/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/mon.c/config 2024-05-20T16:06:41.324 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:06:41 smithi184 ceph-mon[36209]: from='osd.1 [v2:172.21.15.160:6800/1457003568,v1:172.21.15.160:6801/1457003568]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-20T16:06:41.324 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:06:41 smithi184 ceph-mon[36209]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-20T16:06:41.501 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:06:41 smithi160 ceph-mon[36347]: from='osd.1 [v2:172.21.15.160:6800/1457003568,v1:172.21.15.160:6801/1457003568]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-20T16:06:41.501 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:06:41 smithi160 ceph-mon[36347]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-20T16:06:41.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:06:41 smithi082 ceph-mon[32238]: from='osd.1 [v2:172.21.15.160:6800/1457003568,v1:172.21.15.160:6801/1457003568]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-20T16:06:41.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:06:41 smithi082 ceph-mon[32238]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-20T16:06:42.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:06:42 smithi160 ceph-mon[36347]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-20T16:06:42.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:06:42 smithi160 ceph-mon[36347]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-05-20T16:06:42.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:06:42 smithi160 ceph-mon[36347]: from='osd.1 [v2:172.21.15.160:6800/1457003568,v1:172.21.15.160:6801/1457003568]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]: dispatch 2024-05-20T16:06:42.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:06:42 smithi160 ceph-mon[36347]: osdmap e11: 2 total, 1 up, 2 in 2024-05-20T16:06:42.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:06:42 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-20T16:06:42.564 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:06:42 smithi160 ceph-mon[36347]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]: dispatch 2024-05-20T16:06:42.564 INFO:journalctl@ceph.osd.1.smithi160.stdout:May 20 16:06:42 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-1[40898]: 2024-05-20T16:06:42.257+0000 7f3552e57700 -1 osd.1 0 waiting for initial osdmap 2024-05-20T16:06:42.564 INFO:journalctl@ceph.osd.1.smithi160.stdout:May 20 16:06:42 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-1[40898]: 2024-05-20T16:06:42.260+0000 7f354ec0c700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-20T16:06:42.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:06:42 smithi082 ceph-mon[32238]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-20T16:06:42.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:06:42 smithi082 ceph-mon[32238]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-05-20T16:06:42.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:06:42 smithi082 ceph-mon[32238]: from='osd.1 [v2:172.21.15.160:6800/1457003568,v1:172.21.15.160:6801/1457003568]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]: dispatch 2024-05-20T16:06:42.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:06:42 smithi082 ceph-mon[32238]: osdmap e11: 2 total, 1 up, 2 in 2024-05-20T16:06:42.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:06:42 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-20T16:06:42.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:06:42 smithi082 ceph-mon[32238]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]: dispatch 2024-05-20T16:06:42.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:06:42 smithi184 ceph-mon[36209]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-20T16:06:42.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:06:42 smithi184 ceph-mon[36209]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-05-20T16:06:42.637 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:06:42 smithi184 ceph-mon[36209]: from='osd.1 [v2:172.21.15.160:6800/1457003568,v1:172.21.15.160:6801/1457003568]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]: dispatch 2024-05-20T16:06:42.637 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:06:42 smithi184 ceph-mon[36209]: osdmap e11: 2 total, 1 up, 2 in 2024-05-20T16:06:42.637 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:06:42 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-20T16:06:42.637 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:06:42 smithi184 ceph-mon[36209]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]: dispatch 2024-05-20T16:06:42.721 INFO:teuthology.orchestra.run.smithi184.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-05-20T16:06:42.722 INFO:teuthology.orchestra.run.smithi184.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-05-20T16:06:42.722 INFO:teuthology.orchestra.run.smithi184.stderr: stderr: 10+0 records in 2024-05-20T16:06:42.722 INFO:teuthology.orchestra.run.smithi184.stderr:10+0 records out 2024-05-20T16:06:42.722 INFO:teuthology.orchestra.run.smithi184.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0114952 s, 912 MB/s 2024-05-20T16:06:42.722 INFO:teuthology.orchestra.run.smithi184.stderr:--> Zapping successful for: 2024-05-20T16:06:43.319 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:06:43 smithi184 ceph-mon[36209]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]': finished 2024-05-20T16:06:43.319 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:06:43 smithi184 ceph-mon[36209]: osdmap e12: 2 total, 1 up, 2 in 2024-05-20T16:06:43.319 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:06:43 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-20T16:06:43.319 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:06:43 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-20T16:06:43.319 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:06:43 smithi184 ceph-mon[36209]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-20T16:06:43.319 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:06:43 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-20T16:06:43.320 DEBUG:teuthology.orchestra.run.smithi184:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 59d1600a-16c2-11ef-bc9a-c7b262605968 -- ceph orch daemon add osd smithi184:vg_nvme/lv_4 2024-05-20T16:06:43.561 INFO:teuthology.orchestra.run.smithi184.stderr:Inferring config /var/lib/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/mon.c/config 2024-05-20T16:06:43.562 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:06:43 smithi160 ceph-mon[36347]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]': finished 2024-05-20T16:06:43.562 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:06:43 smithi160 ceph-mon[36347]: osdmap e12: 2 total, 1 up, 2 in 2024-05-20T16:06:43.562 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:06:43 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-20T16:06:43.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:06:43 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-20T16:06:43.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:06:43 smithi160 ceph-mon[36347]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-20T16:06:43.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:06:43 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-20T16:06:43.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:06:43 smithi082 ceph-mon[32238]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]': finished 2024-05-20T16:06:43.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:06:43 smithi082 ceph-mon[32238]: osdmap e12: 2 total, 1 up, 2 in 2024-05-20T16:06:43.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:06:43 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-20T16:06:43.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:06:43 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-20T16:06:43.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:06:43 smithi082 ceph-mon[32238]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-20T16:06:43.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:06:43 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-20T16:06:44.549 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:06:44 smithi184 ceph-mon[36209]: purged_snaps scrub starts 2024-05-20T16:06:44.549 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:06:44 smithi184 ceph-mon[36209]: purged_snaps scrub ok 2024-05-20T16:06:44.549 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:06:44 smithi184 ceph-mon[36209]: osd.1 [v2:172.21.15.160:6800/1457003568,v1:172.21.15.160:6801/1457003568] boot 2024-05-20T16:06:44.549 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:06:44 smithi184 ceph-mon[36209]: osdmap e13: 2 total, 2 up, 2 in 2024-05-20T16:06:44.549 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:06:44 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-20T16:06:44.549 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:06:44 smithi184 ceph-mon[36209]: Detected new or changed devices on smithi160 2024-05-20T16:06:44.549 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:06:44 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:06:44.549 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:06:44 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:06:44.549 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:06:44 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-20T16:06:44.549 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:06:44 smithi184 ceph-mon[36209]: Adjusting osd_memory_target on smithi160 to 17088M 2024-05-20T16:06:44.550 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:06:44 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:06:44.550 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:06:44 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:06:44.550 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:06:44 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T16:06:44.550 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:06:44 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:06:44.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:06:44 smithi160 ceph-mon[36347]: purged_snaps scrub starts 2024-05-20T16:06:44.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:06:44 smithi160 ceph-mon[36347]: purged_snaps scrub ok 2024-05-20T16:06:44.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:06:44 smithi160 ceph-mon[36347]: osd.1 [v2:172.21.15.160:6800/1457003568,v1:172.21.15.160:6801/1457003568] boot 2024-05-20T16:06:44.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:06:44 smithi160 ceph-mon[36347]: osdmap e13: 2 total, 2 up, 2 in 2024-05-20T16:06:44.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:06:44 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-20T16:06:44.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:06:44 smithi160 ceph-mon[36347]: Detected new or changed devices on smithi160 2024-05-20T16:06:44.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:06:44 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:06:44.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:06:44 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:06:44.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:06:44 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-20T16:06:44.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:06:44 smithi160 ceph-mon[36347]: Adjusting osd_memory_target on smithi160 to 17088M 2024-05-20T16:06:44.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:06:44 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:06:44.564 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:06:44 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:06:44.564 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:06:44 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T16:06:44.564 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:06:44 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:06:44.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:06:44 smithi082 ceph-mon[32238]: purged_snaps scrub starts 2024-05-20T16:06:44.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:06:44 smithi082 ceph-mon[32238]: purged_snaps scrub ok 2024-05-20T16:06:44.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:06:44 smithi082 ceph-mon[32238]: osd.1 [v2:172.21.15.160:6800/1457003568,v1:172.21.15.160:6801/1457003568] boot 2024-05-20T16:06:44.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:06:44 smithi082 ceph-mon[32238]: osdmap e13: 2 total, 2 up, 2 in 2024-05-20T16:06:44.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:06:44 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-20T16:06:44.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:06:44 smithi082 ceph-mon[32238]: Detected new or changed devices on smithi160 2024-05-20T16:06:44.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:06:44 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:06:44.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:06:44 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:06:44.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:06:44 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-20T16:06:44.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:06:44 smithi082 ceph-mon[32238]: Adjusting osd_memory_target on smithi160 to 17088M 2024-05-20T16:06:44.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:06:44 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:06:44.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:06:44 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:06:44.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:06:44 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T16:06:44.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:06:44 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:06:45.082 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:06:45.081+0000 7f9200533700 1 -- 172.21.15.184:0/1465023798 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f91f8103010 msgr2=0x7f91f81034e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:06:45.082 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:06:45.081+0000 7f9200533700 1 --2- 172.21.15.184:0/1465023798 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f91f8103010 0x7f91f81034e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f91e8009990 tx=0x7f91e8009ca0 comp rx=0 tx=0).stop 2024-05-20T16:06:45.082 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:06:45.081+0000 7f9200533700 1 -- 172.21.15.184:0/1465023798 shutdown_connections 2024-05-20T16:06:45.082 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:06:45.081+0000 7f9200533700 1 --2- 172.21.15.184:0/1465023798 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f91f8102610 0x7f91f8102a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:06:45.082 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:06:45.081+0000 7f9200533700 1 --2- 172.21.15.184:0/1465023798 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f91f8103b70 0x7f91f810a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:06:45.082 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:06:45.081+0000 7f9200533700 1 --2- 172.21.15.184:0/1465023798 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f91f8103010 0x7f91f81034e0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:06:45.083 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:06:45.081+0000 7f9200533700 1 -- 172.21.15.184:0/1465023798 >> 172.21.15.184:0/1465023798 conn(0x7f91f80fcea0 msgr2=0x7f91f80ff350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:06:45.083 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:06:45.081+0000 7f9200533700 1 -- 172.21.15.184:0/1465023798 shutdown_connections 2024-05-20T16:06:45.083 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:06:45.081+0000 7f9200533700 1 -- 172.21.15.184:0/1465023798 wait complete. 2024-05-20T16:06:45.084 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:06:45.082+0000 7f9200533700 1 Processor -- start 2024-05-20T16:06:45.085 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:06:45.082+0000 7f9200533700 1 -- start start 2024-05-20T16:06:45.085 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:06:45.082+0000 7f9200533700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f91f8102610 0x7f91f8199800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:06:45.085 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:06:45.082+0000 7f9200533700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f91f8103010 0x7f91f8199d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:06:45.085 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:06:45.082+0000 7f9200533700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f91f8103b70 0x7f91f81a0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:06:45.085 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:06:45.082+0000 7f9200533700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f91f810ee00 con 0x7f91f8103010 2024-05-20T16:06:45.085 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:06:45.082+0000 7f9200533700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f91f810ec80 con 0x7f91f8103b70 2024-05-20T16:06:45.085 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:06:45.082+0000 7f9200533700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f91f810ef80 con 0x7f91f8102610 2024-05-20T16:06:45.085 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:06:45.082+0000 7f91fe2cf700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f91f8102610 0x7f91f8199800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:06:45.085 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:06:45.082+0000 7f91fe2cf700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f91f8102610 0x7f91f8199800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.184:49714/0 (socket says 172.21.15.184:49714) 2024-05-20T16:06:45.085 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:06:45.082+0000 7f91fe2cf700 1 -- 172.21.15.184:0/2699509028 learned_addr learned my addr 172.21.15.184:0/2699509028 (peer_addr_for_me v2:172.21.15.184:0/0) 2024-05-20T16:06:45.085 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:06:45.083+0000 7f91fead0700 1 --2- 172.21.15.184:0/2699509028 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f91f8103b70 0x7f91f81a0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:06:45.085 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:06:45.083+0000 7f91fdace700 1 --2- 172.21.15.184:0/2699509028 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f91f8103010 0x7f91f8199d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:06:45.086 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:06:45.083+0000 7f91fe2cf700 1 -- 172.21.15.184:0/2699509028 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f91f8103b70 msgr2=0x7f91f81a0ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:06:45.086 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:06:45.083+0000 7f91fe2cf700 1 --2- 172.21.15.184:0/2699509028 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f91f8103b70 0x7f91f81a0ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:06:45.086 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:06:45.083+0000 7f91fe2cf700 1 -- 172.21.15.184:0/2699509028 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f91f8103010 msgr2=0x7f91f8199d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:06:45.086 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:06:45.083+0000 7f91fe2cf700 1 --2- 172.21.15.184:0/2699509028 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f91f8103010 0x7f91f8199d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:06:45.086 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:06:45.083+0000 7f91fe2cf700 1 -- 172.21.15.184:0/2699509028 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91f81a1360 con 0x7f91f8102610 2024-05-20T16:06:45.086 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:06:45.083+0000 7f91fdace700 1 --2- 172.21.15.184:0/2699509028 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f91f8103010 0x7f91f8199d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T16:06:45.086 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:06:45.083+0000 7f91fe2cf700 1 --2- 172.21.15.184:0/2699509028 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f91f8102610 0x7f91f8199800 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f91f400d950 tx=0x7f91f400dd10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:06:45.086 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:06:45.083+0000 7f91ef7fe700 1 -- 172.21.15.184:0/2699509028 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91f4017070 con 0x7f91f8102610 2024-05-20T16:06:45.086 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:06:45.083+0000 7f91ef7fe700 1 -- 172.21.15.184:0/2699509028 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f91f400de30 con 0x7f91f8102610 2024-05-20T16:06:45.086 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:06:45.083+0000 7f9200533700 1 -- 172.21.15.184:0/2699509028 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91f81a1650 con 0x7f91f8102610 2024-05-20T16:06:45.086 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:06:45.083+0000 7f91ef7fe700 1 -- 172.21.15.184:0/2699509028 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91f4003c70 con 0x7f91f8102610 2024-05-20T16:06:45.086 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:06:45.083+0000 7f9200533700 1 -- 172.21.15.184:0/2699509028 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91f81a1b90 con 0x7f91f8102610 2024-05-20T16:06:45.086 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:06:45.084+0000 7f9200533700 1 -- 172.21.15.184:0/2699509028 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91f8050170 con 0x7f91f8102610 2024-05-20T16:06:45.087 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:06:45.085+0000 7f91ef7fe700 1 -- 172.21.15.184:0/2699509028 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 15) ==== 137777+0+0 (secure 0 0 0) 0x7f91f4009940 con 0x7f91f8102610 2024-05-20T16:06:45.087 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:06:45.086+0000 7f91ef7fe700 1 --2- 172.21.15.184:0/2699509028 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f91e40a2b90 0x7f91e40a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:06:45.087 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:06:45.086+0000 7f91ef7fe700 1 -- 172.21.15.184:0/2699509028 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 2309+0+0 (secure 0 0 0) 0x7f91f40cdb30 con 0x7f91f8102610 2024-05-20T16:06:45.087 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:06:45.086+0000 7f91fdace700 1 --2- 172.21.15.184:0/2699509028 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f91e40a2b90 0x7f91e40a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:06:45.088 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:06:45.087+0000 7f91fdace700 1 --2- 172.21.15.184:0/2699509028 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f91e40a2b90 0x7f91e40a50a0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f91e8000c00 tx=0x7f91e8005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:06:45.090 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:06:45.090+0000 7f91ef7fe700 1 -- 172.21.15.184:0/2699509028 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f91f4020030 con 0x7f91f8102610 2024-05-20T16:06:45.297 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:06:45.297+0000 7f9200533700 1 -- 172.21.15.184:0/2699509028 --> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi184:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f91f8062a80 con 0x7f91e40a2b90 2024-05-20T16:06:45.386 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:06:45 smithi184 ceph-mon[36209]: osdmap e14: 2 total, 2 up, 2 in 2024-05-20T16:06:45.386 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:06:45 smithi184 ceph-mon[36209]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-20T16:06:45.562 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:06:45 smithi160 ceph-mon[36347]: osdmap e14: 2 total, 2 up, 2 in 2024-05-20T16:06:45.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:06:45 smithi160 ceph-mon[36347]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-20T16:06:45.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:06:45 smithi082 ceph-mon[32238]: osdmap e14: 2 total, 2 up, 2 in 2024-05-20T16:06:45.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:06:45 smithi082 ceph-mon[32238]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-20T16:06:46.550 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:06:46 smithi184 ceph-mon[36209]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi184:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-20T16:06:46.550 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:06:46 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-20T16:06:46.550 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:06:46 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-20T16:06:46.550 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:06:46 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:06:46.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:06:46 smithi160 ceph-mon[36347]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi184:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-20T16:06:46.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:06:46 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-20T16:06:46.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:06:46 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-20T16:06:46.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:06:46 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:06:46.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:06:46 smithi082 ceph-mon[32238]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi184:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-20T16:06:46.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:06:46 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-20T16:06:46.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:06:46 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-20T16:06:46.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:06:46 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:06:47.386 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:06:47 smithi184 ceph-mon[36209]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-20T16:06:47.562 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:06:47 smithi160 ceph-mon[36347]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-20T16:06:47.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:06:47 smithi082 ceph-mon[32238]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-20T16:06:48.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:06:48 smithi160 ceph-mon[36347]: from='client.? 172.21.15.184:0/4019346293' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3bbb5587-b449-41f3-bd26-67ec3d6afdfc"}]: dispatch 2024-05-20T16:06:48.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:06:48 smithi160 ceph-mon[36347]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3bbb5587-b449-41f3-bd26-67ec3d6afdfc"}]: dispatch 2024-05-20T16:06:48.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:06:48 smithi160 ceph-mon[36347]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3bbb5587-b449-41f3-bd26-67ec3d6afdfc"}]': finished 2024-05-20T16:06:48.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:06:48 smithi160 ceph-mon[36347]: osdmap e15: 3 total, 2 up, 3 in 2024-05-20T16:06:48.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:06:48 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-20T16:06:48.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:06:48 smithi082 ceph-mon[32238]: from='client.? 172.21.15.184:0/4019346293' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3bbb5587-b449-41f3-bd26-67ec3d6afdfc"}]: dispatch 2024-05-20T16:06:48.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:06:48 smithi082 ceph-mon[32238]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3bbb5587-b449-41f3-bd26-67ec3d6afdfc"}]: dispatch 2024-05-20T16:06:48.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:06:48 smithi082 ceph-mon[32238]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3bbb5587-b449-41f3-bd26-67ec3d6afdfc"}]': finished 2024-05-20T16:06:48.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:06:48 smithi082 ceph-mon[32238]: osdmap e15: 3 total, 2 up, 3 in 2024-05-20T16:06:48.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:06:48 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-20T16:06:48.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:06:48 smithi184 ceph-mon[36209]: from='client.? 172.21.15.184:0/4019346293' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3bbb5587-b449-41f3-bd26-67ec3d6afdfc"}]: dispatch 2024-05-20T16:06:48.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:06:48 smithi184 ceph-mon[36209]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3bbb5587-b449-41f3-bd26-67ec3d6afdfc"}]: dispatch 2024-05-20T16:06:48.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:06:48 smithi184 ceph-mon[36209]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3bbb5587-b449-41f3-bd26-67ec3d6afdfc"}]': finished 2024-05-20T16:06:48.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:06:48 smithi184 ceph-mon[36209]: osdmap e15: 3 total, 2 up, 3 in 2024-05-20T16:06:48.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:06:48 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-20T16:06:49.562 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:06:49 smithi160 ceph-mon[36347]: from='client.? 172.21.15.184:0/3485505255' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-20T16:06:49.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:06:49 smithi160 ceph-mon[36347]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-20T16:06:49.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:06:49 smithi082 ceph-mon[32238]: from='client.? 172.21.15.184:0/3485505255' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-20T16:06:49.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:06:49 smithi082 ceph-mon[32238]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-20T16:06:49.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:06:49 smithi184 ceph-mon[36209]: from='client.? 172.21.15.184:0/3485505255' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-20T16:06:49.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:06:49 smithi184 ceph-mon[36209]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-20T16:06:52.266 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:06:52 smithi184 ceph-mon[36209]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-20T16:06:52.562 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:06:52 smithi160 ceph-mon[36347]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-20T16:06:52.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:06:52 smithi082 ceph-mon[32238]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-20T16:06:54.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:06:54 smithi160 ceph-mon[36347]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-20T16:06:54.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:06:54 smithi082 ceph-mon[32238]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-20T16:06:54.610 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:06:54 smithi184 ceph-mon[36209]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-20T16:06:55.531 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:06:55 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-05-20T16:06:55.531 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:06:55 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:06:55.562 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:06:55 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-05-20T16:06:55.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:06:55 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:06:55.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:06:55 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-05-20T16:06:55.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:06:55 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:06:56.486 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:06:56 smithi184 ceph-mon[36209]: Deploying daemon osd.2 on smithi184 2024-05-20T16:06:56.487 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:06:56 smithi184 ceph-mon[36209]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-20T16:06:56.562 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:06:56 smithi160 ceph-mon[36347]: Deploying daemon osd.2 on smithi184 2024-05-20T16:06:56.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:06:56 smithi160 ceph-mon[36347]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-20T16:06:56.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:06:56 smithi082 ceph-mon[32238]: Deploying daemon osd.2 on smithi184 2024-05-20T16:06:56.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:06:56 smithi082 ceph-mon[32238]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-20T16:06:58.386 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:06:58 smithi184 ceph-mon[36209]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-20T16:06:58.562 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:06:58 smithi160 ceph-mon[36347]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-20T16:06:58.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:06:58 smithi082 ceph-mon[32238]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-20T16:07:00.562 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:07:00 smithi160 ceph-mon[36347]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-20T16:07:00.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:07:00 smithi082 ceph-mon[32238]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-20T16:07:00.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:07:00 smithi184 ceph-mon[36209]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-20T16:07:01.562 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:07:01 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:07:01.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:07:01 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:07:01.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:07:01 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T16:07:01.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:07:01 smithi160 ceph-mon[36347]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-20T16:07:01.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:07:01 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:07:01.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:07:01 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:07:01.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:07:01 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T16:07:01.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:07:01 smithi082 ceph-mon[32238]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-20T16:07:01.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:07:01 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:07:01.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:07:01 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:07:01.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:07:01 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T16:07:01.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:07:01 smithi184 ceph-mon[36209]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-20T16:07:03.714 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:07:03.713+0000 7f91ef7fe700 1 -- 172.21.15.184:0/2699509028 <== mgr.14150 v2:172.21.15.82:6800/379530823 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f91f8062a80 con 0x7f91e40a2b90 2024-05-20T16:07:03.714 INFO:teuthology.orchestra.run.smithi184.stdout:Created osd(s) 2 on host 'smithi184' 2024-05-20T16:07:03.716 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:07:03.715+0000 7f9200533700 1 -- 172.21.15.184:0/2699509028 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f91e40a2b90 msgr2=0x7f91e40a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:07:03.717 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:07:03.715+0000 7f9200533700 1 --2- 172.21.15.184:0/2699509028 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f91e40a2b90 0x7f91e40a50a0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f91e8000c00 tx=0x7f91e8005fb0 comp rx=0 tx=0).stop 2024-05-20T16:07:03.717 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:07:03.716+0000 7f9200533700 1 -- 172.21.15.184:0/2699509028 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f91f8102610 msgr2=0x7f91f8199800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:07:03.717 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:07:03.716+0000 7f9200533700 1 --2- 172.21.15.184:0/2699509028 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f91f8102610 0x7f91f8199800 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f91f400d950 tx=0x7f91f400dd10 comp rx=0 tx=0).stop 2024-05-20T16:07:03.717 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:07:03.716+0000 7f9200533700 1 -- 172.21.15.184:0/2699509028 shutdown_connections 2024-05-20T16:07:03.717 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:07:03.716+0000 7f9200533700 1 --2- 172.21.15.184:0/2699509028 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f91e40a2b90 0x7f91e40a50a0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:07:03.717 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:07:03.716+0000 7f9200533700 1 --2- 172.21.15.184:0/2699509028 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f91f8103b70 0x7f91f81a0ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:07:03.717 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:07:03.716+0000 7f9200533700 1 --2- 172.21.15.184:0/2699509028 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f91f8102610 0x7f91f8199800 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:07:03.717 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:07:03.716+0000 7f9200533700 1 --2- 172.21.15.184:0/2699509028 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f91f8103010 0x7f91f8199d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:07:03.717 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:07:03.716+0000 7f9200533700 1 -- 172.21.15.184:0/2699509028 >> 172.21.15.184:0/2699509028 conn(0x7f91f80fcea0 msgr2=0x7f91f81065c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:07:03.717 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:07:03.716+0000 7f9200533700 1 -- 172.21.15.184:0/2699509028 shutdown_connections 2024-05-20T16:07:03.717 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:07:03.716+0000 7f9200533700 1 -- 172.21.15.184:0/2699509028 wait complete. 2024-05-20T16:07:04.281 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:07:04 smithi184 ceph-mon[36209]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-20T16:07:04.281 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:07:04 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:07:04.281 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:07:04 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:07:04.281 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:07:04 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:07:04.281 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:07:04 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:07:04.378 DEBUG:teuthology.orchestra.run.smithi184:osd.2> sudo journalctl -f -n 0 -u ceph-59d1600a-16c2-11ef-bc9a-c7b262605968@osd.2.service 2024-05-20T16:07:04.380 INFO:tasks.cephadm:Deploying osd.3 on smithi082 with /dev/vg_nvme/lv_3... 2024-05-20T16:07:04.380 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 59d1600a-16c2-11ef-bc9a-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-05-20T16:07:04.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:07:04 smithi160 ceph-mon[36347]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-20T16:07:04.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:07:04 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:07:04.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:07:04 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:07:04.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:07:04 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:07:04.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:07:04 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:07:04.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:07:04 smithi082 ceph-mon[32238]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-20T16:07:04.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:07:04 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:07:04.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:07:04 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:07:04.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:07:04 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:07:04.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:07:04 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:07:04.626 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T16:07:05.361 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:07:05 smithi082 ceph-mon[32238]: from='osd.2 [v2:172.21.15.184:6800/4051293934,v1:172.21.15.184:6801/4051293934]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-20T16:07:05.362 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:07:05 smithi082 ceph-mon[32238]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-20T16:07:05.562 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:07:05 smithi160 ceph-mon[36347]: from='osd.2 [v2:172.21.15.184:6800/4051293934,v1:172.21.15.184:6801/4051293934]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-20T16:07:05.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:07:05 smithi160 ceph-mon[36347]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-20T16:07:05.588 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:07:05 smithi184 ceph-mon[36209]: from='osd.2 [v2:172.21.15.184:6800/4051293934,v1:172.21.15.184:6801/4051293934]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-20T16:07:05.588 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:07:05 smithi184 ceph-mon[36209]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-20T16:07:06.530 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:07:06 smithi184 ceph-mon[36209]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-20T16:07:06.530 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:07:06 smithi184 ceph-mon[36209]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-05-20T16:07:06.530 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:07:06 smithi184 ceph-mon[36209]: osdmap e16: 3 total, 2 up, 3 in 2024-05-20T16:07:06.530 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:07:06 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-20T16:07:06.530 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:07:06 smithi184 ceph-mon[36209]: from='osd.2 [v2:172.21.15.184:6800/4051293934,v1:172.21.15.184:6801/4051293934]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]: dispatch 2024-05-20T16:07:06.530 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:07:06 smithi184 ceph-mon[36209]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]: dispatch 2024-05-20T16:07:06.531 INFO:journalctl@ceph.osd.2.smithi184.stdout:May 20 16:07:06 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-2[41326]: 2024-05-20T16:07:06.264+0000 7f52a0ac5700 -1 osd.2 0 waiting for initial osdmap 2024-05-20T16:07:06.531 INFO:journalctl@ceph.osd.2.smithi184.stdout:May 20 16:07:06 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-2[41326]: 2024-05-20T16:07:06.269+0000 7f529c87a700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-20T16:07:06.554 INFO:teuthology.orchestra.run.smithi082.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-05-20T16:07:06.554 INFO:teuthology.orchestra.run.smithi082.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-05-20T16:07:06.554 INFO:teuthology.orchestra.run.smithi082.stderr: stderr: 10+0 records in 2024-05-20T16:07:06.554 INFO:teuthology.orchestra.run.smithi082.stderr:10+0 records out 2024-05-20T16:07:06.554 INFO:teuthology.orchestra.run.smithi082.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0126817 s, 827 MB/s 2024-05-20T16:07:06.554 INFO:teuthology.orchestra.run.smithi082.stderr: stderr: 2024-05-20T16:07:06.554 INFO:teuthology.orchestra.run.smithi082.stderr:--> Zapping successful for: 2024-05-20T16:07:06.555 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:07:06 smithi082 ceph-mon[32238]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-20T16:07:06.555 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:07:06 smithi082 ceph-mon[32238]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-05-20T16:07:06.555 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:07:06 smithi082 ceph-mon[32238]: osdmap e16: 3 total, 2 up, 3 in 2024-05-20T16:07:06.555 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:07:06 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-20T16:07:06.555 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:07:06 smithi082 ceph-mon[32238]: from='osd.2 [v2:172.21.15.184:6800/4051293934,v1:172.21.15.184:6801/4051293934]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]: dispatch 2024-05-20T16:07:06.555 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:07:06 smithi082 ceph-mon[32238]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]: dispatch 2024-05-20T16:07:06.562 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:07:06 smithi160 ceph-mon[36347]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-20T16:07:06.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:07:06 smithi160 ceph-mon[36347]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-05-20T16:07:06.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:07:06 smithi160 ceph-mon[36347]: osdmap e16: 3 total, 2 up, 3 in 2024-05-20T16:07:06.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:07:06 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-20T16:07:06.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:07:06 smithi160 ceph-mon[36347]: from='osd.2 [v2:172.21.15.184:6800/4051293934,v1:172.21.15.184:6801/4051293934]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]: dispatch 2024-05-20T16:07:06.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:07:06 smithi160 ceph-mon[36347]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]: dispatch 2024-05-20T16:07:07.183 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 59d1600a-16c2-11ef-bc9a-c7b262605968 -- ceph orch daemon add osd smithi082:vg_nvme/lv_3 2024-05-20T16:07:07.386 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:07:07 smithi184 ceph-mon[36209]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]': finished 2024-05-20T16:07:07.386 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:07:07 smithi184 ceph-mon[36209]: osdmap e17: 3 total, 2 up, 3 in 2024-05-20T16:07:07.386 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:07:07 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-20T16:07:07.386 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:07:07 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-20T16:07:07.386 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:07:07 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:07:07.387 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:07:07 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:07:07.387 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:07:07 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-20T16:07:07.387 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:07:07 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:07:07.387 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:07:07 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:07:07.387 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:07:07 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T16:07:07.387 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:07:07 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:07:07.440 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T16:07:07.562 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:07:07 smithi160 ceph-mon[36347]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]': finished 2024-05-20T16:07:07.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:07:07 smithi160 ceph-mon[36347]: osdmap e17: 3 total, 2 up, 3 in 2024-05-20T16:07:07.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:07:07 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-20T16:07:07.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:07:07 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-20T16:07:07.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:07:07 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:07:07.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:07:07 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:07:07.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:07:07 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-20T16:07:07.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:07:07 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:07:07.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:07:07 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:07:07.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:07:07 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T16:07:07.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:07:07 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:07:07.564 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:07:07 smithi082 ceph-mon[32238]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]': finished 2024-05-20T16:07:07.564 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:07:07 smithi082 ceph-mon[32238]: osdmap e17: 3 total, 2 up, 3 in 2024-05-20T16:07:07.564 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:07:07 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-20T16:07:07.564 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:07:07 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-20T16:07:07.564 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:07:07 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:07:07.564 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:07:07 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:07:07.564 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:07:07 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-20T16:07:07.564 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:07:07 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:07:07.564 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:07:07 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:07:07.564 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:07:07 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T16:07:07.564 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:07:07 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:07:08.536 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:07:08 smithi082 ceph-mon[32238]: purged_snaps scrub starts 2024-05-20T16:07:08.537 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:07:08 smithi082 ceph-mon[32238]: purged_snaps scrub ok 2024-05-20T16:07:08.537 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:07:08 smithi082 ceph-mon[32238]: pgmap v78: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-20T16:07:08.537 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:07:08 smithi082 ceph-mon[32238]: Detected new or changed devices on smithi184 2024-05-20T16:07:08.537 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:07:08 smithi082 ceph-mon[32238]: Adjusting osd_memory_target on smithi184 to 17088M 2024-05-20T16:07:08.537 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:07:08 smithi082 ceph-mon[32238]: osd.2 [v2:172.21.15.184:6800/4051293934,v1:172.21.15.184:6801/4051293934] boot 2024-05-20T16:07:08.537 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:07:08 smithi082 ceph-mon[32238]: osdmap e18: 3 total, 3 up, 3 in 2024-05-20T16:07:08.537 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:07:08 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-20T16:07:08.537 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:07:08 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T16:07:08.537 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:07:08 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:07:08.537 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:07:08 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T16:07:08.537 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:07:08 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:07:08.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:07:08 smithi160 ceph-mon[36347]: purged_snaps scrub starts 2024-05-20T16:07:08.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:07:08 smithi160 ceph-mon[36347]: purged_snaps scrub ok 2024-05-20T16:07:08.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:07:08 smithi160 ceph-mon[36347]: pgmap v78: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-20T16:07:08.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:07:08 smithi160 ceph-mon[36347]: Detected new or changed devices on smithi184 2024-05-20T16:07:08.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:07:08 smithi160 ceph-mon[36347]: Adjusting osd_memory_target on smithi184 to 17088M 2024-05-20T16:07:08.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:07:08 smithi160 ceph-mon[36347]: osd.2 [v2:172.21.15.184:6800/4051293934,v1:172.21.15.184:6801/4051293934] boot 2024-05-20T16:07:08.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:07:08 smithi160 ceph-mon[36347]: osdmap e18: 3 total, 3 up, 3 in 2024-05-20T16:07:08.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:07:08 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-20T16:07:08.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:07:08 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T16:07:08.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:07:08 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:07:08.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:07:08 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T16:07:08.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:07:08 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:07:08.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:07:08 smithi184 ceph-mon[36209]: purged_snaps scrub starts 2024-05-20T16:07:08.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:07:08 smithi184 ceph-mon[36209]: purged_snaps scrub ok 2024-05-20T16:07:08.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:07:08 smithi184 ceph-mon[36209]: pgmap v78: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-20T16:07:08.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:07:08 smithi184 ceph-mon[36209]: Detected new or changed devices on smithi184 2024-05-20T16:07:08.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:07:08 smithi184 ceph-mon[36209]: Adjusting osd_memory_target on smithi184 to 17088M 2024-05-20T16:07:08.637 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:07:08 smithi184 ceph-mon[36209]: osd.2 [v2:172.21.15.184:6800/4051293934,v1:172.21.15.184:6801/4051293934] boot 2024-05-20T16:07:08.637 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:07:08 smithi184 ceph-mon[36209]: osdmap e18: 3 total, 3 up, 3 in 2024-05-20T16:07:08.637 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:07:08 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-20T16:07:08.637 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:07:08 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T16:07:08.637 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:07:08 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:07:08.637 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:07:08 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T16:07:08.637 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:07:08 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:07:09.021 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:07:09.019+0000 7f38f80e2700 1 -- 172.21.15.82:0/66045323 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f38f0102de0 msgr2=0x7f38f01031f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:07:09.021 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:07:09.019+0000 7f38f80e2700 1 --2- 172.21.15.82:0/66045323 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f38f0102de0 0x7f38f01031f0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f38e0009990 tx=0x7f38e0009ca0 comp rx=0 tx=0).stop 2024-05-20T16:07:09.022 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:07:09.019+0000 7f38f80e2700 1 -- 172.21.15.82:0/66045323 shutdown_connections 2024-05-20T16:07:09.022 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:07:09.019+0000 7f38f80e2700 1 --2- 172.21.15.82:0/66045323 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f38f0103860 0x7f38f010b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:07:09.022 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:07:09.019+0000 7f38f80e2700 1 --2- 172.21.15.82:0/66045323 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f38f0104830 0x7f38f0104c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:07:09.022 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:07:09.019+0000 7f38f80e2700 1 --2- 172.21.15.82:0/66045323 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f38f0102de0 0x7f38f01031f0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:07:09.022 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:07:09.019+0000 7f38f80e2700 1 -- 172.21.15.82:0/66045323 >> 172.21.15.82:0/66045323 conn(0x7f38f00fe850 msgr2=0x7f38f0100ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:07:09.022 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:07:09.019+0000 7f38f80e2700 1 -- 172.21.15.82:0/66045323 shutdown_connections 2024-05-20T16:07:09.022 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:07:09.019+0000 7f38f80e2700 1 -- 172.21.15.82:0/66045323 wait complete. 2024-05-20T16:07:09.022 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:07:09.020+0000 7f38f80e2700 1 Processor -- start 2024-05-20T16:07:09.022 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:07:09.020+0000 7f38f80e2700 1 -- start start 2024-05-20T16:07:09.022 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:07:09.020+0000 7f38f80e2700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f38f0102de0 0x7f38f00773f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:07:09.022 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:07:09.020+0000 7f38f80e2700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f38f0103860 0x7f38f0075aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:07:09.022 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:07:09.020+0000 7f38f80e2700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f38f0104830 0x7f38f0075fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:07:09.022 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:07:09.020+0000 7f38f80e2700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f38f010efb0 con 0x7f38f0104830 2024-05-20T16:07:09.022 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:07:09.020+0000 7f38f80e2700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f38f010ee30 con 0x7f38f0102de0 2024-05-20T16:07:09.023 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:07:09.020+0000 7f38f80e2700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f38f010f130 con 0x7f38f0103860 2024-05-20T16:07:09.023 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:07:09.020+0000 7f38f667f700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f38f0104830 0x7f38f0075fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:07:09.023 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:07:09.020+0000 7f38f667f700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f38f0104830 0x7f38f0075fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:55728/0 (socket says 172.21.15.82:55728) 2024-05-20T16:07:09.023 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:07:09.020+0000 7f38f667f700 1 -- 172.21.15.82:0/1641627007 learned_addr learned my addr 172.21.15.82:0/1641627007 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:07:09.023 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:07:09.020+0000 7f38f5e7e700 1 --2- 172.21.15.82:0/1641627007 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f38f0102de0 0x7f38f00773f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:07:09.023 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:07:09.021+0000 7f38f567d700 1 --2- 172.21.15.82:0/1641627007 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f38f0103860 0x7f38f0075aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:07:09.023 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:07:09.021+0000 7f38f667f700 1 -- 172.21.15.82:0/1641627007 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f38f0103860 msgr2=0x7f38f0075aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:07:09.023 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:07:09.021+0000 7f38f667f700 1 --2- 172.21.15.82:0/1641627007 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f38f0103860 0x7f38f0075aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:07:09.023 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:07:09.021+0000 7f38f667f700 1 -- 172.21.15.82:0/1641627007 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f38f0102de0 msgr2=0x7f38f00773f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:07:09.023 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:07:09.021+0000 7f38f667f700 1 --2- 172.21.15.82:0/1641627007 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f38f0102de0 0x7f38f00773f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:07:09.023 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:07:09.021+0000 7f38f667f700 1 -- 172.21.15.82:0/1641627007 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38f0076740 con 0x7f38f0104830 2024-05-20T16:07:09.023 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:07:09.021+0000 7f38f667f700 1 --2- 172.21.15.82:0/1641627007 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f38f0104830 0x7f38f0075fe0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f38e800b550 tx=0x7f38e800b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:07:09.023 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:07:09.021+0000 7f38e6ffd700 1 -- 172.21.15.82:0/1641627007 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38e8015020 con 0x7f38f0104830 2024-05-20T16:07:09.023 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:07:09.021+0000 7f38f80e2700 1 -- 172.21.15.82:0/1641627007 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38f01a3ab0 con 0x7f38f0104830 2024-05-20T16:07:09.024 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:07:09.021+0000 7f38e6ffd700 1 -- 172.21.15.82:0/1641627007 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f38e8011d10 con 0x7f38f0104830 2024-05-20T16:07:09.024 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:07:09.021+0000 7f38e6ffd700 1 -- 172.21.15.82:0/1641627007 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38e800f2e0 con 0x7f38f0104830 2024-05-20T16:07:09.024 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:07:09.021+0000 7f38f80e2700 1 -- 172.21.15.82:0/1641627007 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38f01a3fc0 con 0x7f38f0104830 2024-05-20T16:07:09.025 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:07:09.022+0000 7f38f80e2700 1 -- 172.21.15.82:0/1641627007 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38f0050170 con 0x7f38f0104830 2024-05-20T16:07:09.025 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:07:09.023+0000 7f38e6ffd700 1 -- 172.21.15.82:0/1641627007 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 15) ==== 137777+0+0 (secure 0 0 0) 0x7f38e8018020 con 0x7f38f0104830 2024-05-20T16:07:09.025 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:07:09.024+0000 7f38e6ffd700 1 --2- 172.21.15.82:0/1641627007 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f38dc0a2b90 0x7f38dc0a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:07:09.025 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:07:09.024+0000 7f38f5e7e700 1 --2- 172.21.15.82:0/1641627007 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f38dc0a2b90 0x7f38dc0a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:07:09.025 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:07:09.024+0000 7f38e6ffd700 1 -- 172.21.15.82:0/1641627007 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 2851+0+0 (secure 0 0 0) 0x7f38e80cd600 con 0x7f38f0104830 2024-05-20T16:07:09.025 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:07:09.024+0000 7f38f5e7e700 1 --2- 172.21.15.82:0/1641627007 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f38dc0a2b90 0x7f38dc0a50a0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f38f0104220 tx=0x7f38ec0094f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:07:09.027 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:07:09.027+0000 7f38e6ffd700 1 -- 172.21.15.82:0/1641627007 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f38e801f020 con 0x7f38f0104830 2024-05-20T16:07:09.234 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:07:09.234+0000 7f38f80e2700 1 -- 172.21.15.82:0/1641627007 --> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi082:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f38f002e000 con 0x7f38dc0a2b90 2024-05-20T16:07:09.813 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:07:09 smithi160 ceph-mon[36347]: osdmap e19: 3 total, 3 up, 3 in 2024-05-20T16:07:09.813 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:07:09 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-20T16:07:09.813 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:07:09 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-20T16:07:09.813 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:07:09 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:07:09.813 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:07:09 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-05-20T16:07:09.856 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:07:09 smithi082 ceph-mon[32238]: osdmap e19: 3 total, 3 up, 3 in 2024-05-20T16:07:09.856 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:07:09 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-20T16:07:09.857 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:07:09 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-20T16:07:09.857 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:07:09 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:07:09.857 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:07:09 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-05-20T16:07:09.886 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:07:09 smithi184 ceph-mon[36209]: osdmap e19: 3 total, 3 up, 3 in 2024-05-20T16:07:09.886 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:07:09 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-20T16:07:09.886 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:07:09 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-20T16:07:09.886 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:07:09 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:07:09.886 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:07:09 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-05-20T16:07:10.813 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:07:10 smithi160 ceph-mon[36347]: pgmap v81: 0 pgs: ; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-05-20T16:07:10.813 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:07:10 smithi160 ceph-mon[36347]: from='client.14277 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi082:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-20T16:07:10.813 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:07:10 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-05-20T16:07:10.813 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:07:10 smithi160 ceph-mon[36347]: osdmap e20: 3 total, 3 up, 3 in 2024-05-20T16:07:10.813 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:07:10 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-05-20T16:07:10.813 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:07:10 smithi082 ceph-mon[32238]: pgmap v81: 0 pgs: ; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-05-20T16:07:10.813 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:07:10 smithi082 ceph-mon[32238]: from='client.14277 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi082:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-20T16:07:10.814 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:07:10 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-05-20T16:07:10.814 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:07:10 smithi082 ceph-mon[32238]: osdmap e20: 3 total, 3 up, 3 in 2024-05-20T16:07:10.814 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:07:10 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-05-20T16:07:10.886 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:07:10 smithi184 ceph-mon[36209]: pgmap v81: 0 pgs: ; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-05-20T16:07:10.886 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:07:10 smithi184 ceph-mon[36209]: from='client.14277 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi082:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-20T16:07:10.886 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:07:10 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-05-20T16:07:10.887 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:07:10 smithi184 ceph-mon[36209]: osdmap e20: 3 total, 3 up, 3 in 2024-05-20T16:07:10.887 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:07:10 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-05-20T16:07:11.102 INFO:journalctl@ceph.osd.0.smithi082.stdout:May 20 16:07:10 smithi082 sudo[51353]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-20T16:07:11.102 INFO:journalctl@ceph.osd.0.smithi082.stdout:May 20 16:07:10 smithi082 sudo[51353]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-20T16:07:11.102 INFO:journalctl@ceph.osd.0.smithi082.stdout:May 20 16:07:10 smithi082 sudo[51353]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-20T16:07:11.103 INFO:journalctl@ceph.osd.0.smithi082.stdout:May 20 16:07:10 smithi082 sudo[51353]: pam_unix(sudo:session): session closed for user root 2024-05-20T16:07:11.359 INFO:journalctl@ceph.osd.0.smithi082.stdout:May 20 16:07:11 smithi082 sudo[51360]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-20T16:07:11.359 INFO:journalctl@ceph.osd.0.smithi082.stdout:May 20 16:07:11 smithi082 sudo[51360]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-20T16:07:11.359 INFO:journalctl@ceph.osd.0.smithi082.stdout:May 20 16:07:11 smithi082 sudo[51360]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-20T16:07:11.359 INFO:journalctl@ceph.osd.0.smithi082.stdout:May 20 16:07:11 smithi082 sudo[51360]: pam_unix(sudo:session): session closed for user root 2024-05-20T16:07:11.687 INFO:journalctl@ceph.osd.1.smithi160.stdout:May 20 16:07:11 smithi160 sudo[44577]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-20T16:07:11.687 INFO:journalctl@ceph.osd.1.smithi160.stdout:May 20 16:07:11 smithi160 sudo[44577]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-20T16:07:11.687 INFO:journalctl@ceph.osd.1.smithi160.stdout:May 20 16:07:11 smithi160 sudo[44577]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-20T16:07:11.687 INFO:journalctl@ceph.osd.1.smithi160.stdout:May 20 16:07:11 smithi160 sudo[44577]: pam_unix(sudo:session): session closed for user root 2024-05-20T16:07:11.688 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:07:11 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-05-20T16:07:11.688 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:07:11 smithi160 ceph-mon[36347]: osdmap e21: 3 total, 3 up, 3 in 2024-05-20T16:07:11.688 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:07:11 smithi160 ceph-mon[36347]: pgmap v84: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-05-20T16:07:11.688 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:07:11 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:07:11.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:07:11 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-05-20T16:07:11.860 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:07:11 smithi082 ceph-mon[32238]: osdmap e21: 3 total, 3 up, 3 in 2024-05-20T16:07:11.860 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:07:11 smithi082 ceph-mon[32238]: pgmap v84: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-05-20T16:07:11.860 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:07:11 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:07:11.886 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:07:11 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-05-20T16:07:11.886 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:07:11 smithi184 ceph-mon[36209]: osdmap e21: 3 total, 3 up, 3 in 2024-05-20T16:07:11.886 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:07:11 smithi184 ceph-mon[36209]: pgmap v84: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-05-20T16:07:11.886 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:07:11 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:07:12.062 INFO:journalctl@ceph.osd.1.smithi160.stdout:May 20 16:07:11 smithi160 sudo[44580]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-20T16:07:12.063 INFO:journalctl@ceph.osd.1.smithi160.stdout:May 20 16:07:11 smithi160 sudo[44580]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-20T16:07:12.063 INFO:journalctl@ceph.osd.1.smithi160.stdout:May 20 16:07:11 smithi160 sudo[44580]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-20T16:07:12.063 INFO:journalctl@ceph.osd.1.smithi160.stdout:May 20 16:07:11 smithi160 sudo[44580]: pam_unix(sudo:session): session closed for user root 2024-05-20T16:07:12.205 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:07:12.204+0000 7f38e6ffd700 1 -- 172.21.15.82:0/1641627007 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f38e8097b00 con 0x7f38f0104830 2024-05-20T16:07:12.216 INFO:journalctl@ceph.osd.2.smithi184.stdout:May 20 16:07:11 smithi184 sudo[45377]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-20T16:07:12.216 INFO:journalctl@ceph.osd.2.smithi184.stdout:May 20 16:07:11 smithi184 sudo[45377]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-20T16:07:12.216 INFO:journalctl@ceph.osd.2.smithi184.stdout:May 20 16:07:11 smithi184 sudo[45377]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-20T16:07:12.216 INFO:journalctl@ceph.osd.2.smithi184.stdout:May 20 16:07:11 smithi184 sudo[45377]: pam_unix(sudo:session): session closed for user root 2024-05-20T16:07:12.636 INFO:journalctl@ceph.osd.2.smithi184.stdout:May 20 16:07:12 smithi184 sudo[45380]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-20T16:07:12.636 INFO:journalctl@ceph.osd.2.smithi184.stdout:May 20 16:07:12 smithi184 sudo[45380]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-20T16:07:12.636 INFO:journalctl@ceph.osd.2.smithi184.stdout:May 20 16:07:12 smithi184 sudo[45380]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-20T16:07:12.636 INFO:journalctl@ceph.osd.2.smithi184.stdout:May 20 16:07:12 smithi184 sudo[45380]: pam_unix(sudo:session): session closed for user root 2024-05-20T16:07:12.770 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:07:12 smithi082 sudo[51444]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-20T16:07:12.770 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:07:12 smithi082 sudo[51444]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-20T16:07:12.770 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:07:12 smithi082 sudo[51444]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-20T16:07:12.770 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:07:12 smithi082 sudo[51444]: pam_unix(sudo:session): session closed for user root 2024-05-20T16:07:13.031 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:07:12 smithi184 ceph-mon[36209]: osdmap e22: 3 total, 3 up, 3 in 2024-05-20T16:07:13.032 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:07:12 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:07:13.032 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:07:12 smithi184 ceph-mon[36209]: from='client.? 172.21.15.82:0/3634220819' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ca651f90-09b8-4081-a6f9-40b31268a837"}]: dispatch 2024-05-20T16:07:13.032 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:07:12 smithi184 ceph-mon[36209]: from='client.? 172.21.15.82:0/3634220819' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ca651f90-09b8-4081-a6f9-40b31268a837"}]': finished 2024-05-20T16:07:13.032 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:07:12 smithi184 ceph-mon[36209]: osdmap e23: 4 total, 3 up, 4 in 2024-05-20T16:07:13.032 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:07:12 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-20T16:07:13.032 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:07:12 smithi184 ceph-mon[36209]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-05-20T16:07:13.032 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:07:12 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:07:13.032 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:07:12 smithi184 ceph-mon[36209]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-20T16:07:13.032 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:07:13 smithi184 sudo[45383]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-20T16:07:13.032 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:07:13 smithi184 sudo[45383]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-20T16:07:13.032 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:07:13 smithi184 sudo[45383]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-20T16:07:13.032 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:07:13 smithi184 sudo[45383]: pam_unix(sudo:session): session closed for user root 2024-05-20T16:07:13.062 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:07:12 smithi160 ceph-mon[36347]: osdmap e22: 3 total, 3 up, 3 in 2024-05-20T16:07:13.062 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:07:12 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:07:13.063 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:07:12 smithi160 ceph-mon[36347]: from='client.? 172.21.15.82:0/3634220819' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ca651f90-09b8-4081-a6f9-40b31268a837"}]: dispatch 2024-05-20T16:07:13.063 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:07:12 smithi160 ceph-mon[36347]: from='client.? 172.21.15.82:0/3634220819' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ca651f90-09b8-4081-a6f9-40b31268a837"}]': finished 2024-05-20T16:07:13.063 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:07:12 smithi160 ceph-mon[36347]: osdmap e23: 4 total, 3 up, 4 in 2024-05-20T16:07:13.063 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:07:12 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-20T16:07:13.063 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:07:12 smithi160 ceph-mon[36347]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-05-20T16:07:13.063 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:07:12 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:07:13.063 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:07:12 smithi160 ceph-mon[36347]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-20T16:07:13.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:07:12 smithi082 sudo[51491]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-20T16:07:13.110 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:07:12 smithi082 sudo[51491]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-20T16:07:13.110 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:07:12 smithi082 sudo[51491]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-20T16:07:13.110 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:07:12 smithi082 sudo[51491]: pam_unix(sudo:session): session closed for user root 2024-05-20T16:07:13.110 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:07:12 smithi082 ceph-mon[32238]: osdmap e22: 3 total, 3 up, 3 in 2024-05-20T16:07:13.110 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:07:12 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:07:13.110 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:07:12 smithi082 ceph-mon[32238]: from='client.? 172.21.15.82:0/3634220819' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ca651f90-09b8-4081-a6f9-40b31268a837"}]: dispatch 2024-05-20T16:07:13.110 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:07:12 smithi082 ceph-mon[32238]: from='client.? 172.21.15.82:0/3634220819' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ca651f90-09b8-4081-a6f9-40b31268a837"}]': finished 2024-05-20T16:07:13.110 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:07:12 smithi082 ceph-mon[32238]: osdmap e23: 4 total, 3 up, 4 in 2024-05-20T16:07:13.110 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:07:12 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-20T16:07:13.110 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:07:12 smithi082 ceph-mon[32238]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-05-20T16:07:13.110 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:07:12 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:07:13.110 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:07:12 smithi082 ceph-mon[32238]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-20T16:07:13.386 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:07:13 smithi184 sudo[45386]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-20T16:07:13.386 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:07:13 smithi184 sudo[45386]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-20T16:07:13.386 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:07:13 smithi184 sudo[45386]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-20T16:07:13.386 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:07:13 smithi184 sudo[45386]: pam_unix(sudo:session): session closed for user root 2024-05-20T16:07:13.759 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:07:13 smithi160 sudo[44583]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-20T16:07:13.760 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:07:13 smithi160 sudo[44583]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-20T16:07:13.760 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:07:13 smithi160 sudo[44583]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-20T16:07:13.760 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:07:13 smithi160 sudo[44583]: pam_unix(sudo:session): session closed for user root 2024-05-20T16:07:14.062 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:07:13 smithi160 sudo[44586]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-20T16:07:14.063 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:07:13 smithi160 sudo[44586]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-20T16:07:14.063 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:07:13 smithi160 sudo[44586]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-20T16:07:14.063 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:07:13 smithi160 sudo[44586]: pam_unix(sudo:session): session closed for user root 2024-05-20T16:07:14.063 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:07:13 smithi160 ceph-mon[36347]: from='client.? 172.21.15.82:0/175991486' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-20T16:07:14.063 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:07:13 smithi160 ceph-mon[36347]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-20T16:07:14.063 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:07:13 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-20T16:07:14.063 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:07:13 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-20T16:07:14.063 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:07:13 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-20T16:07:14.063 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:07:13 smithi160 ceph-mon[36347]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-20T16:07:14.064 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:07:13 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-20T16:07:14.064 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:07:13 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-20T16:07:14.064 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:07:13 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-20T16:07:14.064 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:07:13 smithi160 ceph-mon[36347]: pgmap v87: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-05-20T16:07:14.064 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:07:13 smithi160 ceph-mon[36347]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-20T16:07:14.064 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:07:13 smithi160 ceph-mon[36347]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-20T16:07:14.064 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:07:13 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-20T16:07:14.064 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:07:13 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-20T16:07:14.064 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:07:13 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-20T16:07:14.064 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:07:13 smithi160 ceph-mon[36347]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-20T16:07:14.110 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:07:13 smithi082 ceph-mon[32238]: from='client.? 172.21.15.82:0/175991486' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-20T16:07:14.110 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:07:13 smithi082 ceph-mon[32238]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-20T16:07:14.110 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:07:13 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-20T16:07:14.110 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:07:13 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-20T16:07:14.110 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:07:13 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-20T16:07:14.110 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:07:13 smithi082 ceph-mon[32238]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-20T16:07:14.110 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:07:13 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-20T16:07:14.110 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:07:13 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-20T16:07:14.110 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:07:13 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-20T16:07:14.110 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:07:13 smithi082 ceph-mon[32238]: pgmap v87: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-05-20T16:07:14.110 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:07:13 smithi082 ceph-mon[32238]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-20T16:07:14.110 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:07:13 smithi082 ceph-mon[32238]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-20T16:07:14.111 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:07:13 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-20T16:07:14.111 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:07:13 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-20T16:07:14.111 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:07:13 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-20T16:07:14.111 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:07:13 smithi082 ceph-mon[32238]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-20T16:07:14.136 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:07:13 smithi184 ceph-mon[36209]: from='client.? 172.21.15.82:0/175991486' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-20T16:07:14.136 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:07:13 smithi184 ceph-mon[36209]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-20T16:07:14.136 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:07:13 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-20T16:07:14.136 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:07:13 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-20T16:07:14.136 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:07:13 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-20T16:07:14.136 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:07:13 smithi184 ceph-mon[36209]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-20T16:07:14.136 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:07:13 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-20T16:07:14.137 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:07:13 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-20T16:07:14.137 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:07:13 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-20T16:07:14.137 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:07:13 smithi184 ceph-mon[36209]: pgmap v87: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-05-20T16:07:14.137 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:07:13 smithi184 ceph-mon[36209]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-20T16:07:14.137 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:07:13 smithi184 ceph-mon[36209]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-20T16:07:14.137 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:07:13 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-20T16:07:14.137 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:07:13 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-20T16:07:14.137 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:07:13 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-20T16:07:14.137 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:07:13 smithi184 ceph-mon[36209]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-20T16:07:16.331 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:07:16 smithi082 ceph-mon[32238]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-05-20T16:07:16.562 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:07:16 smithi160 ceph-mon[36347]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-05-20T16:07:16.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:07:16 smithi184 ceph-mon[36209]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-05-20T16:07:18.562 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:07:18 smithi160 ceph-mon[36347]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-05-20T16:07:18.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:07:18 smithi082 ceph-mon[32238]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-05-20T16:07:18.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:07:18 smithi184 ceph-mon[36209]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-05-20T16:07:20.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:07:20 smithi160 ceph-mon[36347]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-05-20T16:07:20.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:07:20 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-05-20T16:07:20.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:07:20 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:07:20.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:07:20 smithi082 ceph-mon[32238]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-05-20T16:07:20.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:07:20 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-05-20T16:07:20.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:07:20 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:07:20.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:07:20 smithi184 ceph-mon[36209]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-05-20T16:07:20.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:07:20 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-05-20T16:07:20.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:07:20 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:07:21.562 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:07:21 smithi160 ceph-mon[36347]: Deploying daemon osd.3 on smithi082 2024-05-20T16:07:21.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:07:21 smithi082 ceph-mon[32238]: Deploying daemon osd.3 on smithi082 2024-05-20T16:07:21.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:07:21 smithi184 ceph-mon[36209]: Deploying daemon osd.3 on smithi082 2024-05-20T16:07:22.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:07:22 smithi082 ceph-mon[32238]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-05-20T16:07:22.562 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:07:22 smithi160 ceph-mon[36347]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-05-20T16:07:22.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:07:22 smithi184 ceph-mon[36209]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-05-20T16:07:24.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:07:24 smithi082 ceph-mon[32238]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-05-20T16:07:24.562 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:07:24 smithi160 ceph-mon[36347]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-05-20T16:07:24.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:07:24 smithi184 ceph-mon[36209]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-05-20T16:07:26.312 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:07:25 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:07:26.313 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:07:25 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:07:26.313 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:07:25 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T16:07:26.313 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:07:25 smithi160 ceph-mon[36347]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-05-20T16:07:26.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:07:25 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:07:26.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:07:25 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:07:26.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:07:25 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T16:07:26.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:07:25 smithi082 ceph-mon[32238]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-05-20T16:07:26.386 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:07:25 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:07:26.386 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:07:25 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:07:26.386 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:07:25 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T16:07:26.386 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:07:25 smithi184 ceph-mon[36209]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-05-20T16:07:28.360 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:07:28 smithi082 ceph-mon[32238]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-05-20T16:07:28.453 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:07:28.453+0000 7f38e6ffd700 1 -- 172.21.15.82:0/1641627007 <== mgr.14150 v2:172.21.15.82:6800/379530823 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f38f002e000 con 0x7f38dc0a2b90 2024-05-20T16:07:28.454 INFO:teuthology.orchestra.run.smithi082.stdout:Created osd(s) 3 on host 'smithi082' 2024-05-20T16:07:28.456 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:07:28.455+0000 7f38f80e2700 1 -- 172.21.15.82:0/1641627007 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f38dc0a2b90 msgr2=0x7f38dc0a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:07:28.456 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:07:28.455+0000 7f38f80e2700 1 --2- 172.21.15.82:0/1641627007 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f38dc0a2b90 0x7f38dc0a50a0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f38f0104220 tx=0x7f38ec0094f0 comp rx=0 tx=0).stop 2024-05-20T16:07:28.456 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:07:28.455+0000 7f38f80e2700 1 -- 172.21.15.82:0/1641627007 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f38f0104830 msgr2=0x7f38f0075fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:07:28.456 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:07:28.455+0000 7f38f80e2700 1 --2- 172.21.15.82:0/1641627007 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f38f0104830 0x7f38f0075fe0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f38e800b550 tx=0x7f38e800b910 comp rx=0 tx=0).stop 2024-05-20T16:07:28.456 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:07:28.456+0000 7f38f80e2700 1 -- 172.21.15.82:0/1641627007 shutdown_connections 2024-05-20T16:07:28.456 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:07:28.456+0000 7f38f80e2700 1 --2- 172.21.15.82:0/1641627007 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f38dc0a2b90 0x7f38dc0a50a0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:07:28.456 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:07:28.456+0000 7f38f80e2700 1 --2- 172.21.15.82:0/1641627007 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f38f0102de0 0x7f38f00773f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:07:28.456 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:07:28.456+0000 7f38f80e2700 1 --2- 172.21.15.82:0/1641627007 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f38f0104830 0x7f38f0075fe0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:07:28.457 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:07:28.456+0000 7f38f80e2700 1 --2- 172.21.15.82:0/1641627007 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f38f0103860 0x7f38f0075aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:07:28.457 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:07:28.456+0000 7f38f80e2700 1 -- 172.21.15.82:0/1641627007 >> 172.21.15.82:0/1641627007 conn(0x7f38f00fe850 msgr2=0x7f38f0107550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:07:28.457 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:07:28.456+0000 7f38f80e2700 1 -- 172.21.15.82:0/1641627007 shutdown_connections 2024-05-20T16:07:28.457 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:07:28.456+0000 7f38f80e2700 1 -- 172.21.15.82:0/1641627007 wait complete. 2024-05-20T16:07:28.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:07:28 smithi160 ceph-mon[36347]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-05-20T16:07:28.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:07:28 smithi184 ceph-mon[36209]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-05-20T16:07:29.199 DEBUG:teuthology.orchestra.run.smithi082:osd.3> sudo journalctl -f -n 0 -u ceph-59d1600a-16c2-11ef-bc9a-c7b262605968@osd.3.service 2024-05-20T16:07:29.201 INFO:tasks.cephadm:Deploying osd.4 on smithi160 with /dev/vg_nvme/lv_3... 2024-05-20T16:07:29.201 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 59d1600a-16c2-11ef-bc9a-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-05-20T16:07:29.443 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/mon.b/config 2024-05-20T16:07:29.505 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:07:29 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:07:29.505 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:07:29 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:07:29.505 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:07:29 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:07:29.505 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:07:29 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:07:29.505 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:07:29 smithi082 ceph-mon[32238]: from='osd.3 [v2:172.21.15.82:6810/3947485599,v1:172.21.15.82:6811/3947485599]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-20T16:07:29.505 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:07:29 smithi082 ceph-mon[32238]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-05-20T16:07:29.562 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:07:29 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:07:29.562 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:07:29 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:07:29.562 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:07:29 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:07:29.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:07:29 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:07:29.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:07:29 smithi160 ceph-mon[36347]: from='osd.3 [v2:172.21.15.82:6810/3947485599,v1:172.21.15.82:6811/3947485599]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-20T16:07:29.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:07:29 smithi160 ceph-mon[36347]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-05-20T16:07:29.886 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:07:29 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:07:29.886 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:07:29 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:07:29.886 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:07:29 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:07:29.886 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:07:29 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:07:29.886 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:07:29 smithi184 ceph-mon[36209]: from='osd.3 [v2:172.21.15.82:6810/3947485599,v1:172.21.15.82:6811/3947485599]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-20T16:07:29.886 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:07:29 smithi184 ceph-mon[36209]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-05-20T16:07:31.312 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:07:30 smithi160 ceph-mon[36347]: from='osd.3 [v2:172.21.15.82:6810/3947485599,v1:172.21.15.82:6811/3947485599]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-05-20T16:07:31.313 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:07:30 smithi160 ceph-mon[36347]: osdmap e24: 4 total, 3 up, 4 in 2024-05-20T16:07:31.313 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:07:30 smithi160 ceph-mon[36347]: from='osd.3 [v2:172.21.15.82:6810/3947485599,v1:172.21.15.82:6811/3947485599]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]: dispatch 2024-05-20T16:07:31.313 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:07:30 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-20T16:07:31.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:07:30 smithi082 ceph-mon[32238]: from='osd.3 [v2:172.21.15.82:6810/3947485599,v1:172.21.15.82:6811/3947485599]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-05-20T16:07:31.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:07:30 smithi082 ceph-mon[32238]: osdmap e24: 4 total, 3 up, 4 in 2024-05-20T16:07:31.360 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:07:30 smithi082 ceph-mon[32238]: from='osd.3 [v2:172.21.15.82:6810/3947485599,v1:172.21.15.82:6811/3947485599]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]: dispatch 2024-05-20T16:07:31.360 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:07:30 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-20T16:07:31.360 INFO:journalctl@ceph.osd.3.smithi082.stdout:May 20 16:07:30 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-3[53439]: 2024-05-20T16:07:30.913+0000 7ff25c838700 -1 osd.3 0 waiting for initial osdmap 2024-05-20T16:07:31.360 INFO:journalctl@ceph.osd.3.smithi082.stdout:May 20 16:07:30 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-3[53439]: 2024-05-20T16:07:30.919+0000 7ff2585ed700 -1 osd.3 25 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-20T16:07:31.386 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:07:30 smithi184 ceph-mon[36209]: from='osd.3 [v2:172.21.15.82:6810/3947485599,v1:172.21.15.82:6811/3947485599]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-05-20T16:07:31.386 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:07:30 smithi184 ceph-mon[36209]: osdmap e24: 4 total, 3 up, 4 in 2024-05-20T16:07:31.386 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:07:30 smithi184 ceph-mon[36209]: from='osd.3 [v2:172.21.15.82:6810/3947485599,v1:172.21.15.82:6811/3947485599]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]: dispatch 2024-05-20T16:07:31.386 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:07:30 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-20T16:07:31.433 INFO:teuthology.orchestra.run.smithi160.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-05-20T16:07:31.434 INFO:teuthology.orchestra.run.smithi160.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-05-20T16:07:31.434 INFO:teuthology.orchestra.run.smithi160.stderr: stderr: 10+0 records in 2024-05-20T16:07:31.434 INFO:teuthology.orchestra.run.smithi160.stderr:10+0 records out 2024-05-20T16:07:31.434 INFO:teuthology.orchestra.run.smithi160.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0114963 s, 912 MB/s 2024-05-20T16:07:31.434 INFO:teuthology.orchestra.run.smithi160.stderr:--> Zapping successful for: 2024-05-20T16:07:32.001 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:07:31 smithi082 ceph-mon[32238]: from='osd.3 [v2:172.21.15.82:6810/3947485599,v1:172.21.15.82:6811/3947485599]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]': finished 2024-05-20T16:07:32.001 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:07:31 smithi082 ceph-mon[32238]: osdmap e25: 4 total, 3 up, 4 in 2024-05-20T16:07:32.002 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:07:31 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-20T16:07:32.002 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:07:31 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-20T16:07:32.002 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:07:31 smithi082 ceph-mon[32238]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-05-20T16:07:32.002 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:07:31 smithi082 ceph-mon[32238]: osd.3 [v2:172.21.15.82:6810/3947485599,v1:172.21.15.82:6811/3947485599] boot 2024-05-20T16:07:32.002 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:07:31 smithi082 ceph-mon[32238]: osdmap e26: 4 total, 4 up, 4 in 2024-05-20T16:07:32.002 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:07:31 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-20T16:07:32.156 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 59d1600a-16c2-11ef-bc9a-c7b262605968 -- ceph orch daemon add osd smithi160:vg_nvme/lv_3 2024-05-20T16:07:32.179 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:07:31 smithi160 ceph-mon[36347]: from='osd.3 [v2:172.21.15.82:6810/3947485599,v1:172.21.15.82:6811/3947485599]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]': finished 2024-05-20T16:07:32.179 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:07:31 smithi160 ceph-mon[36347]: osdmap e25: 4 total, 3 up, 4 in 2024-05-20T16:07:32.179 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:07:31 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-20T16:07:32.179 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:07:31 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-20T16:07:32.179 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:07:31 smithi160 ceph-mon[36347]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-05-20T16:07:32.179 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:07:31 smithi160 ceph-mon[36347]: osd.3 [v2:172.21.15.82:6810/3947485599,v1:172.21.15.82:6811/3947485599] boot 2024-05-20T16:07:32.180 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:07:31 smithi160 ceph-mon[36347]: osdmap e26: 4 total, 4 up, 4 in 2024-05-20T16:07:32.180 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:07:31 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-20T16:07:32.386 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:07:31 smithi184 ceph-mon[36209]: from='osd.3 [v2:172.21.15.82:6810/3947485599,v1:172.21.15.82:6811/3947485599]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]': finished 2024-05-20T16:07:32.386 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:07:31 smithi184 ceph-mon[36209]: osdmap e25: 4 total, 3 up, 4 in 2024-05-20T16:07:32.386 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:07:31 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-20T16:07:32.386 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:07:31 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-20T16:07:32.386 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:07:31 smithi184 ceph-mon[36209]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-05-20T16:07:32.387 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:07:31 smithi184 ceph-mon[36209]: osd.3 [v2:172.21.15.82:6810/3947485599,v1:172.21.15.82:6811/3947485599] boot 2024-05-20T16:07:32.387 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:07:31 smithi184 ceph-mon[36209]: osdmap e26: 4 total, 4 up, 4 in 2024-05-20T16:07:32.387 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:07:31 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-20T16:07:32.406 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/mon.b/config 2024-05-20T16:07:33.291 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:07:32 smithi160 ceph-mon[36347]: purged_snaps scrub starts 2024-05-20T16:07:33.291 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:07:32 smithi160 ceph-mon[36347]: purged_snaps scrub ok 2024-05-20T16:07:33.291 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:07:32 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:07:33.291 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:07:32 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:07:33.291 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:07:32 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-20T16:07:33.291 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:07:32 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-20T16:07:33.291 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:07:32 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:07:33.291 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:07:32 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:07:33.291 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:07:32 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T16:07:33.292 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:07:32 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:07:33.292 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:07:32 smithi160 ceph-mon[36347]: osdmap e27: 4 total, 4 up, 4 in 2024-05-20T16:07:33.292 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:07:32 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T16:07:33.292 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:07:32 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:07:33.292 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:07:32 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T16:07:33.292 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:07:32 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:07:33.360 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:07:32 smithi082 ceph-mon[32238]: purged_snaps scrub starts 2024-05-20T16:07:33.360 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:07:32 smithi082 ceph-mon[32238]: purged_snaps scrub ok 2024-05-20T16:07:33.360 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:07:32 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:07:33.360 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:07:32 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:07:33.360 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:07:32 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-20T16:07:33.360 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:07:32 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-20T16:07:33.360 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:07:32 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:07:33.360 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:07:32 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:07:33.360 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:07:32 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T16:07:33.360 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:07:32 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:07:33.361 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:07:32 smithi082 ceph-mon[32238]: osdmap e27: 4 total, 4 up, 4 in 2024-05-20T16:07:33.361 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:07:32 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T16:07:33.361 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:07:32 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:07:33.361 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:07:32 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T16:07:33.361 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:07:32 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:07:33.386 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:07:32 smithi184 ceph-mon[36209]: purged_snaps scrub starts 2024-05-20T16:07:33.386 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:07:32 smithi184 ceph-mon[36209]: purged_snaps scrub ok 2024-05-20T16:07:33.386 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:07:32 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:07:33.386 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:07:32 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:07:33.386 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:07:32 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-20T16:07:33.386 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:07:32 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-20T16:07:33.387 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:07:32 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:07:33.387 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:07:32 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:07:33.387 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:07:32 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T16:07:33.387 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:07:32 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:07:33.387 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:07:32 smithi184 ceph-mon[36209]: osdmap e27: 4 total, 4 up, 4 in 2024-05-20T16:07:33.387 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:07:32 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T16:07:33.387 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:07:32 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:07:33.387 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:07:32 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T16:07:33.387 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:07:32 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:07:34.169 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-20T16:07:34.168+0000 7f7961f34700 1 -- 172.21.15.160:0/3747299035 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f795c102de0 msgr2=0x7f795c1031f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:07:34.170 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-20T16:07:34.168+0000 7f7961f34700 1 --2- 172.21.15.160:0/3747299035 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f795c102de0 0x7f795c1031f0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f7944009990 tx=0x7f7944009ca0 comp rx=0 tx=0).stop 2024-05-20T16:07:34.170 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-20T16:07:34.168+0000 7f7961f34700 1 -- 172.21.15.160:0/3747299035 shutdown_connections 2024-05-20T16:07:34.170 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-20T16:07:34.168+0000 7f7961f34700 1 --2- 172.21.15.160:0/3747299035 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f795c103860 0x7f795c10b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:07:34.170 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-20T16:07:34.168+0000 7f7961f34700 1 --2- 172.21.15.160:0/3747299035 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f795c104830 0x7f795c104c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:07:34.170 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-20T16:07:34.168+0000 7f7961f34700 1 --2- 172.21.15.160:0/3747299035 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f795c102de0 0x7f795c1031f0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:07:34.170 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-20T16:07:34.168+0000 7f7961f34700 1 -- 172.21.15.160:0/3747299035 >> 172.21.15.160:0/3747299035 conn(0x7f795c0fe850 msgr2=0x7f795c100ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:07:34.170 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-20T16:07:34.168+0000 7f7961f34700 1 -- 172.21.15.160:0/3747299035 shutdown_connections 2024-05-20T16:07:34.170 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-20T16:07:34.168+0000 7f7961f34700 1 -- 172.21.15.160:0/3747299035 wait complete. 2024-05-20T16:07:34.170 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-20T16:07:34.169+0000 7f7961f34700 1 Processor -- start 2024-05-20T16:07:34.170 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-20T16:07:34.169+0000 7f7961f34700 1 -- start start 2024-05-20T16:07:34.172 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-20T16:07:34.169+0000 7f7961f34700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f795c102de0 0x7f795c0773f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:07:34.172 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-20T16:07:34.169+0000 7f7961f34700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f795c103860 0x7f795c075aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:07:34.173 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-20T16:07:34.169+0000 7f7961f34700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f795c104830 0x7f795c075fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:07:34.173 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-20T16:07:34.169+0000 7f7961f34700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f795c10eec0 con 0x7f795c103860 2024-05-20T16:07:34.173 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-20T16:07:34.169+0000 7f7961f34700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f795c10ed40 con 0x7f795c104830 2024-05-20T16:07:34.173 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-20T16:07:34.169+0000 7f7961f34700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f795c10f040 con 0x7f795c102de0 2024-05-20T16:07:34.173 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-20T16:07:34.169+0000 7f795bfff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f795c104830 0x7f795c075fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:07:34.173 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-20T16:07:34.169+0000 7f795bfff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f795c104830 0x7f795c075fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:41822/0 (socket says 172.21.15.160:41822) 2024-05-20T16:07:34.173 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-20T16:07:34.169+0000 7f795bfff700 1 -- 172.21.15.160:0/2509721545 learned_addr learned my addr 172.21.15.160:0/2509721545 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-20T16:07:34.173 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-20T16:07:34.170+0000 7f795b7fe700 1 --2- 172.21.15.160:0/2509721545 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f795c102de0 0x7f795c0773f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:07:34.173 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-20T16:07:34.170+0000 7f7952ffd700 1 --2- 172.21.15.160:0/2509721545 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f795c103860 0x7f795c075aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:07:34.173 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-20T16:07:34.170+0000 7f795bfff700 1 -- 172.21.15.160:0/2509721545 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f795c102de0 msgr2=0x7f795c0773f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:07:34.173 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-20T16:07:34.170+0000 7f795bfff700 1 --2- 172.21.15.160:0/2509721545 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f795c102de0 0x7f795c0773f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:07:34.173 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-20T16:07:34.170+0000 7f795bfff700 1 -- 172.21.15.160:0/2509721545 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f795c103860 msgr2=0x7f795c075aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:07:34.174 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-20T16:07:34.170+0000 7f795bfff700 1 --2- 172.21.15.160:0/2509721545 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f795c103860 0x7f795c075aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:07:34.174 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-20T16:07:34.170+0000 7f795bfff700 1 -- 172.21.15.160:0/2509721545 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f795c076740 con 0x7f795c104830 2024-05-20T16:07:34.174 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-20T16:07:34.170+0000 7f795b7fe700 1 --2- 172.21.15.160:0/2509721545 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f795c102de0 0x7f795c0773f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T16:07:34.174 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-20T16:07:34.170+0000 7f795bfff700 1 --2- 172.21.15.160:0/2509721545 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f795c104830 0x7f795c075fe0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f7954007ee0 tx=0x7f795400d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:07:34.174 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-20T16:07:34.170+0000 7f79597fa700 1 -- 172.21.15.160:0/2509721545 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f795400ddd0 con 0x7f795c104830 2024-05-20T16:07:34.174 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-20T16:07:34.170+0000 7f7961f34700 1 -- 172.21.15.160:0/2509721545 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f795c1a3a40 con 0x7f795c104830 2024-05-20T16:07:34.174 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-20T16:07:34.170+0000 7f79597fa700 1 -- 172.21.15.160:0/2509721545 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7954004020 con 0x7f795c104830 2024-05-20T16:07:34.174 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-20T16:07:34.170+0000 7f79597fa700 1 -- 172.21.15.160:0/2509721545 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7954015660 con 0x7f795c104830 2024-05-20T16:07:34.174 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-20T16:07:34.171+0000 7f7961f34700 1 -- 172.21.15.160:0/2509721545 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f795c1a3ed0 con 0x7f795c104830 2024-05-20T16:07:34.175 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-20T16:07:34.172+0000 7f7961f34700 1 -- 172.21.15.160:0/2509721545 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f795c050170 con 0x7f795c104830 2024-05-20T16:07:34.175 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-20T16:07:34.172+0000 7f79597fa700 1 -- 172.21.15.160:0/2509721545 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f79540041d0 con 0x7f795c104830 2024-05-20T16:07:34.175 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-20T16:07:34.173+0000 7f79597fa700 1 --2- 172.21.15.160:0/2509721545 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f79480a2c20 0x7f79480a5130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:07:34.175 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-20T16:07:34.173+0000 7f79597fa700 1 -- 172.21.15.160:0/2509721545 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 3698+0+0 (secure 0 0 0) 0x7f79540cd9d0 con 0x7f795c104830 2024-05-20T16:07:34.176 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-20T16:07:34.173+0000 7f795b7fe700 1 --2- 172.21.15.160:0/2509721545 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f79480a2c20 0x7f79480a5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:07:34.176 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-20T16:07:34.174+0000 7f795b7fe700 1 --2- 172.21.15.160:0/2509721545 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f79480a2c20 0x7f79480a5130 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f795c104220 tx=0x7f794c00b5f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:07:34.176 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:07:33 smithi160 ceph-mon[36347]: Detected new or changed devices on smithi082 2024-05-20T16:07:34.176 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:07:33 smithi160 ceph-mon[36347]: Adjusting osd_memory_target on smithi082 to 8544M 2024-05-20T16:07:34.176 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:07:33 smithi160 ceph-mon[36347]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-05-20T16:07:34.176 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:07:33 smithi160 ceph-mon[36347]: osdmap e28: 4 total, 4 up, 4 in 2024-05-20T16:07:34.179 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-20T16:07:34.178+0000 7f79597fa700 1 -- 172.21.15.160:0/2509721545 <== mon.2 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f7954010070 con 0x7f795c104830 2024-05-20T16:07:34.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:07:33 smithi082 ceph-mon[32238]: Detected new or changed devices on smithi082 2024-05-20T16:07:34.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:07:33 smithi082 ceph-mon[32238]: Adjusting osd_memory_target on smithi082 to 8544M 2024-05-20T16:07:34.360 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:07:33 smithi082 ceph-mon[32238]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-05-20T16:07:34.360 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:07:33 smithi082 ceph-mon[32238]: osdmap e28: 4 total, 4 up, 4 in 2024-05-20T16:07:34.382 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-20T16:07:34.381+0000 7f7961f34700 1 -- 172.21.15.160:0/2509721545 --> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi160:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f795c02e040 con 0x7f79480a2c20 2024-05-20T16:07:34.386 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:07:33 smithi184 ceph-mon[36209]: Detected new or changed devices on smithi082 2024-05-20T16:07:34.386 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:07:33 smithi184 ceph-mon[36209]: Adjusting osd_memory_target on smithi082 to 8544M 2024-05-20T16:07:34.386 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:07:33 smithi184 ceph-mon[36209]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-05-20T16:07:34.386 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:07:33 smithi184 ceph-mon[36209]: osdmap e28: 4 total, 4 up, 4 in 2024-05-20T16:07:35.247 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:07:34 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-20T16:07:35.247 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:07:34 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-20T16:07:35.247 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:07:34 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:07:35.247 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:07:34 smithi160 ceph-mon[36347]: osdmap e29: 4 total, 4 up, 4 in 2024-05-20T16:07:35.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:07:34 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-20T16:07:35.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:07:34 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-20T16:07:35.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:07:34 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:07:35.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:07:34 smithi082 ceph-mon[32238]: osdmap e29: 4 total, 4 up, 4 in 2024-05-20T16:07:35.386 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:07:34 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-20T16:07:35.386 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:07:34 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-20T16:07:35.386 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:07:34 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:07:35.386 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:07:34 smithi184 ceph-mon[36209]: osdmap e29: 4 total, 4 up, 4 in 2024-05-20T16:07:36.136 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:07:35 smithi184 ceph-mon[36209]: from='client.24212 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi160:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-20T16:07:36.136 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:07:35 smithi184 ceph-mon[36209]: pgmap v104: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-05-20T16:07:36.136 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:07:35 smithi184 ceph-mon[36209]: osdmap e30: 4 total, 4 up, 4 in 2024-05-20T16:07:36.293 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:07:35 smithi160 ceph-mon[36347]: from='client.24212 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi160:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-20T16:07:36.293 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:07:35 smithi160 ceph-mon[36347]: pgmap v104: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-05-20T16:07:36.293 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:07:35 smithi160 ceph-mon[36347]: osdmap e30: 4 total, 4 up, 4 in 2024-05-20T16:07:36.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:07:35 smithi082 ceph-mon[32238]: from='client.24212 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi160:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-20T16:07:36.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:07:35 smithi082 ceph-mon[32238]: pgmap v104: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-05-20T16:07:36.360 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:07:35 smithi082 ceph-mon[32238]: osdmap e30: 4 total, 4 up, 4 in 2024-05-20T16:07:37.554 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:07:37 smithi160 ceph-mon[36347]: from='client.? 172.21.15.160:0/2818215795' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cb3dfd56-f677-40cb-9c96-8319ea5ac788"}]: dispatch 2024-05-20T16:07:37.554 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:07:37 smithi160 ceph-mon[36347]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cb3dfd56-f677-40cb-9c96-8319ea5ac788"}]: dispatch 2024-05-20T16:07:37.554 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:07:37 smithi160 ceph-mon[36347]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cb3dfd56-f677-40cb-9c96-8319ea5ac788"}]': finished 2024-05-20T16:07:37.554 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:07:37 smithi160 ceph-mon[36347]: osdmap e31: 5 total, 4 up, 5 in 2024-05-20T16:07:37.554 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:07:37 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-20T16:07:37.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:07:37 smithi082 ceph-mon[32238]: from='client.? 172.21.15.160:0/2818215795' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cb3dfd56-f677-40cb-9c96-8319ea5ac788"}]: dispatch 2024-05-20T16:07:37.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:07:37 smithi082 ceph-mon[32238]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cb3dfd56-f677-40cb-9c96-8319ea5ac788"}]: dispatch 2024-05-20T16:07:37.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:07:37 smithi082 ceph-mon[32238]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cb3dfd56-f677-40cb-9c96-8319ea5ac788"}]': finished 2024-05-20T16:07:37.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:07:37 smithi082 ceph-mon[32238]: osdmap e31: 5 total, 4 up, 5 in 2024-05-20T16:07:37.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:07:37 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-20T16:07:37.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:07:37 smithi184 ceph-mon[36209]: from='client.? 172.21.15.160:0/2818215795' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cb3dfd56-f677-40cb-9c96-8319ea5ac788"}]: dispatch 2024-05-20T16:07:37.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:07:37 smithi184 ceph-mon[36209]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cb3dfd56-f677-40cb-9c96-8319ea5ac788"}]: dispatch 2024-05-20T16:07:37.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:07:37 smithi184 ceph-mon[36209]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cb3dfd56-f677-40cb-9c96-8319ea5ac788"}]': finished 2024-05-20T16:07:37.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:07:37 smithi184 ceph-mon[36209]: osdmap e31: 5 total, 4 up, 5 in 2024-05-20T16:07:37.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:07:37 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-20T16:07:38.562 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:07:38 smithi160 ceph-mon[36347]: pgmap v107: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-05-20T16:07:38.562 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:07:38 smithi160 ceph-mon[36347]: from='client.? 172.21.15.160:0/222395990' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-20T16:07:38.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:07:38 smithi082 ceph-mon[32238]: pgmap v107: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-05-20T16:07:38.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:07:38 smithi082 ceph-mon[32238]: from='client.? 172.21.15.160:0/222395990' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-20T16:07:38.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:07:38 smithi184 ceph-mon[36209]: pgmap v107: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-05-20T16:07:38.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:07:38 smithi184 ceph-mon[36209]: from='client.? 172.21.15.160:0/222395990' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-20T16:07:40.562 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:07:40 smithi160 ceph-mon[36347]: pgmap v108: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 101 KiB/s, 0 objects/s recovering 2024-05-20T16:07:40.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:07:40 smithi082 ceph-mon[32238]: pgmap v108: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 101 KiB/s, 0 objects/s recovering 2024-05-20T16:07:40.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:07:40 smithi184 ceph-mon[36209]: pgmap v108: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 101 KiB/s, 0 objects/s recovering 2024-05-20T16:07:42.276 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:07:42 smithi160 ceph-mon[36347]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-05-20T16:07:42.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:07:42 smithi082 ceph-mon[32238]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-05-20T16:07:42.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:07:42 smithi184 ceph-mon[36209]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-05-20T16:07:44.313 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:07:44 smithi160 ceph-mon[36347]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-05-20T16:07:44.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:07:44 smithi082 ceph-mon[32238]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-05-20T16:07:44.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:07:44 smithi184 ceph-mon[36209]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-05-20T16:07:45.491 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:07:45 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-05-20T16:07:45.491 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:07:45 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:07:45.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:07:45 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-05-20T16:07:45.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:07:45 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:07:45.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:07:45 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-05-20T16:07:45.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:07:45 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:07:46.544 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:07:46 smithi160 ceph-mon[36347]: Deploying daemon osd.4 on smithi160 2024-05-20T16:07:46.544 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:07:46 smithi160 ceph-mon[36347]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-05-20T16:07:46.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:07:46 smithi082 ceph-mon[32238]: Deploying daemon osd.4 on smithi160 2024-05-20T16:07:46.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:07:46 smithi082 ceph-mon[32238]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-05-20T16:07:46.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:07:46 smithi184 ceph-mon[36209]: Deploying daemon osd.4 on smithi160 2024-05-20T16:07:46.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:07:46 smithi184 ceph-mon[36209]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-05-20T16:07:48.562 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:07:48 smithi160 ceph-mon[36347]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-05-20T16:07:48.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:07:48 smithi082 ceph-mon[32238]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-05-20T16:07:48.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:07:48 smithi184 ceph-mon[36209]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-05-20T16:07:50.478 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:07:50 smithi160 ceph-mon[36347]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-05-20T16:07:50.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:07:50 smithi082 ceph-mon[32238]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-05-20T16:07:50.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:07:50 smithi184 ceph-mon[36209]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-05-20T16:07:51.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:07:51 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:07:51.813 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:07:51 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T16:07:51.813 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:07:51 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:07:51.813 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:07:51 smithi160 ceph-mon[36347]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-05-20T16:07:51.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:07:51 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:07:51.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:07:51 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T16:07:51.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:07:51 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:07:51.860 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:07:51 smithi082 ceph-mon[32238]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-05-20T16:07:51.886 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:07:51 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:07:51.886 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:07:51 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T16:07:51.886 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:07:51 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:07:51.886 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:07:51 smithi184 ceph-mon[36209]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-05-20T16:07:54.403 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:07:54 smithi160 ceph-mon[36347]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-05-20T16:07:54.403 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:07:54 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:07:54.404 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:07:54 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:07:54.404 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:07:54 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:07:54.404 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:07:54 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T16:07:54.404 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:07:54 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:07:54.451 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-20T16:07:54.449+0000 7f79597fa700 1 -- 172.21.15.160:0/2509721545 <== mgr.14150 v2:172.21.15.82:6800/379530823 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f795c02e040 con 0x7f79480a2c20 2024-05-20T16:07:54.451 INFO:teuthology.orchestra.run.smithi160.stdout:Created osd(s) 4 on host 'smithi160' 2024-05-20T16:07:54.453 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-20T16:07:54.452+0000 7f7961f34700 1 -- 172.21.15.160:0/2509721545 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f79480a2c20 msgr2=0x7f79480a5130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:07:54.453 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-20T16:07:54.452+0000 7f7961f34700 1 --2- 172.21.15.160:0/2509721545 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f79480a2c20 0x7f79480a5130 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f795c104220 tx=0x7f794c00b5f0 comp rx=0 tx=0).stop 2024-05-20T16:07:54.454 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-20T16:07:54.452+0000 7f7961f34700 1 -- 172.21.15.160:0/2509721545 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f795c104830 msgr2=0x7f795c075fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:07:54.454 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-20T16:07:54.452+0000 7f7961f34700 1 --2- 172.21.15.160:0/2509721545 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f795c104830 0x7f795c075fe0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f7954007ee0 tx=0x7f795400d3b0 comp rx=0 tx=0).stop 2024-05-20T16:07:54.454 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-20T16:07:54.452+0000 7f7961f34700 1 -- 172.21.15.160:0/2509721545 shutdown_connections 2024-05-20T16:07:54.454 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-20T16:07:54.452+0000 7f7961f34700 1 --2- 172.21.15.160:0/2509721545 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f79480a2c20 0x7f79480a5130 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:07:54.454 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-20T16:07:54.452+0000 7f7961f34700 1 --2- 172.21.15.160:0/2509721545 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f795c104830 0x7f795c075fe0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:07:54.454 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-20T16:07:54.452+0000 7f7961f34700 1 --2- 172.21.15.160:0/2509721545 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f795c102de0 0x7f795c0773f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:07:54.454 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-20T16:07:54.452+0000 7f7961f34700 1 --2- 172.21.15.160:0/2509721545 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f795c103860 0x7f795c075aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:07:54.454 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-20T16:07:54.452+0000 7f7961f34700 1 -- 172.21.15.160:0/2509721545 >> 172.21.15.160:0/2509721545 conn(0x7f795c0fe850 msgr2=0x7f795c107550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:07:54.454 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-20T16:07:54.452+0000 7f7961f34700 1 -- 172.21.15.160:0/2509721545 shutdown_connections 2024-05-20T16:07:54.454 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-20T16:07:54.452+0000 7f7961f34700 1 -- 172.21.15.160:0/2509721545 wait complete. 2024-05-20T16:07:54.589 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:07:54 smithi184 ceph-mon[36209]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-05-20T16:07:54.589 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:07:54 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:07:54.590 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:07:54 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:07:54.590 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:07:54 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:07:54.590 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:07:54 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T16:07:54.590 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:07:54 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:07:54.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:07:54 smithi082 ceph-mon[32238]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-05-20T16:07:54.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:07:54 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:07:54.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:07:54 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:07:54.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:07:54 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:07:54.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:07:54 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T16:07:54.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:07:54 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:07:55.167 DEBUG:teuthology.orchestra.run.smithi160:osd.4> sudo journalctl -f -n 0 -u ceph-59d1600a-16c2-11ef-bc9a-c7b262605968@osd.4.service 2024-05-20T16:07:55.209 INFO:tasks.cephadm:Deploying osd.5 on smithi184 with /dev/vg_nvme/lv_3... 2024-05-20T16:07:55.209 DEBUG:teuthology.orchestra.run.smithi184:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 59d1600a-16c2-11ef-bc9a-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-05-20T16:07:55.416 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:07:55 smithi160 ceph-mon[36347]: from='osd.4 [v2:172.21.15.160:6808/4160830014,v1:172.21.15.160:6809/4160830014]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-20T16:07:55.417 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:07:55 smithi160 ceph-mon[36347]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-20T16:07:55.417 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:07:55 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T16:07:55.417 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:07:55 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:07:55.417 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:07:55 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:07:55.453 INFO:teuthology.orchestra.run.smithi184.stderr:Inferring config /var/lib/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/mon.c/config 2024-05-20T16:07:55.569 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:07:55 smithi184 ceph-mon[36209]: from='osd.4 [v2:172.21.15.160:6808/4160830014,v1:172.21.15.160:6809/4160830014]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-20T16:07:55.570 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:07:55 smithi184 ceph-mon[36209]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-20T16:07:55.570 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:07:55 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T16:07:55.570 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:07:55 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:07:55.570 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:07:55 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:07:55.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:07:55 smithi082 ceph-mon[32238]: from='osd.4 [v2:172.21.15.160:6808/4160830014,v1:172.21.15.160:6809/4160830014]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-20T16:07:55.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:07:55 smithi082 ceph-mon[32238]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-20T16:07:55.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:07:55 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T16:07:55.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:07:55 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:07:55.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:07:55 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:07:56.423 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:07:56 smithi160 ceph-mon[36347]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-05-20T16:07:56.423 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:07:56 smithi160 ceph-mon[36347]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-05-20T16:07:56.423 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:07:56 smithi160 ceph-mon[36347]: from='osd.4 [v2:172.21.15.160:6808/4160830014,v1:172.21.15.160:6809/4160830014]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]: dispatch 2024-05-20T16:07:56.423 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:07:56 smithi160 ceph-mon[36347]: osdmap e32: 5 total, 4 up, 5 in 2024-05-20T16:07:56.423 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:07:56 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-20T16:07:56.423 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:07:56 smithi160 ceph-mon[36347]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]: dispatch 2024-05-20T16:07:56.424 INFO:journalctl@ceph.osd.4.smithi160.stdout:May 20 16:07:56 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-4[47371]: 2024-05-20T16:07:56.274+0000 7f60f3a5c700 -1 osd.4 0 waiting for initial osdmap 2024-05-20T16:07:56.424 INFO:journalctl@ceph.osd.4.smithi160.stdout:May 20 16:07:56 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-4[47371]: 2024-05-20T16:07:56.280+0000 7f60ef811700 -1 osd.4 33 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-20T16:07:56.550 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:07:56 smithi184 ceph-mon[36209]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-05-20T16:07:56.550 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:07:56 smithi184 ceph-mon[36209]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-05-20T16:07:56.550 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:07:56 smithi184 ceph-mon[36209]: from='osd.4 [v2:172.21.15.160:6808/4160830014,v1:172.21.15.160:6809/4160830014]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]: dispatch 2024-05-20T16:07:56.550 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:07:56 smithi184 ceph-mon[36209]: osdmap e32: 5 total, 4 up, 5 in 2024-05-20T16:07:56.550 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:07:56 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-20T16:07:56.551 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:07:56 smithi184 ceph-mon[36209]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]: dispatch 2024-05-20T16:07:56.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:07:56 smithi082 ceph-mon[32238]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-05-20T16:07:56.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:07:56 smithi082 ceph-mon[32238]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-05-20T16:07:56.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:07:56 smithi082 ceph-mon[32238]: from='osd.4 [v2:172.21.15.160:6808/4160830014,v1:172.21.15.160:6809/4160830014]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]: dispatch 2024-05-20T16:07:56.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:07:56 smithi082 ceph-mon[32238]: osdmap e32: 5 total, 4 up, 5 in 2024-05-20T16:07:56.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:07:56 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-20T16:07:56.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:07:56 smithi082 ceph-mon[32238]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]: dispatch 2024-05-20T16:07:57.394 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:07:57 smithi184 ceph-mon[36209]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]': finished 2024-05-20T16:07:57.394 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:07:57 smithi184 ceph-mon[36209]: osdmap e33: 5 total, 4 up, 5 in 2024-05-20T16:07:57.395 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:07:57 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-20T16:07:57.395 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:07:57 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-20T16:07:57.395 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:07:57 smithi184 ceph-mon[36209]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-05-20T16:07:57.395 INFO:teuthology.orchestra.run.smithi184.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-05-20T16:07:57.395 INFO:teuthology.orchestra.run.smithi184.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-05-20T16:07:57.395 INFO:teuthology.orchestra.run.smithi184.stderr: stderr: 10+0 records in 2024-05-20T16:07:57.395 INFO:teuthology.orchestra.run.smithi184.stderr:10+0 records out 2024-05-20T16:07:57.395 INFO:teuthology.orchestra.run.smithi184.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0116331 s, 901 MB/s 2024-05-20T16:07:57.395 INFO:teuthology.orchestra.run.smithi184.stderr: stderr: 2024-05-20T16:07:57.395 INFO:teuthology.orchestra.run.smithi184.stderr:--> Zapping successful for: 2024-05-20T16:07:57.562 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:07:57 smithi160 ceph-mon[36347]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]': finished 2024-05-20T16:07:57.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:07:57 smithi160 ceph-mon[36347]: osdmap e33: 5 total, 4 up, 5 in 2024-05-20T16:07:57.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:07:57 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-20T16:07:57.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:07:57 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-20T16:07:57.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:07:57 smithi160 ceph-mon[36347]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-05-20T16:07:57.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:07:57 smithi082 ceph-mon[32238]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]': finished 2024-05-20T16:07:57.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:07:57 smithi082 ceph-mon[32238]: osdmap e33: 5 total, 4 up, 5 in 2024-05-20T16:07:57.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:07:57 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-20T16:07:57.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:07:57 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-20T16:07:57.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:07:57 smithi082 ceph-mon[32238]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-05-20T16:07:58.053 DEBUG:teuthology.orchestra.run.smithi184:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 59d1600a-16c2-11ef-bc9a-c7b262605968 -- ceph orch daemon add osd smithi184:vg_nvme/lv_3 2024-05-20T16:07:58.294 INFO:teuthology.orchestra.run.smithi184.stderr:Inferring config /var/lib/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/mon.c/config 2024-05-20T16:07:58.562 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:07:58 smithi160 ceph-mon[36347]: purged_snaps scrub starts 2024-05-20T16:07:58.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:07:58 smithi160 ceph-mon[36347]: purged_snaps scrub ok 2024-05-20T16:07:58.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:07:58 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-20T16:07:58.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:07:58 smithi160 ceph-mon[36347]: osd.4 [v2:172.21.15.160:6808/4160830014,v1:172.21.15.160:6809/4160830014] boot 2024-05-20T16:07:58.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:07:58 smithi160 ceph-mon[36347]: osdmap e34: 5 total, 5 up, 5 in 2024-05-20T16:07:58.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:07:58 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-20T16:07:58.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:07:58 smithi160 ceph-mon[36347]: Detected new or changed devices on smithi160 2024-05-20T16:07:58.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:07:58 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:07:58.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:07:58 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:07:58.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:07:58 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-20T16:07:58.564 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:07:58 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-20T16:07:58.564 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:07:58 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:07:58.564 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:07:58 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:07:58.564 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:07:58 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T16:07:58.581 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:07:58 smithi184 ceph-mon[36209]: purged_snaps scrub starts 2024-05-20T16:07:58.581 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:07:58 smithi184 ceph-mon[36209]: purged_snaps scrub ok 2024-05-20T16:07:58.581 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:07:58 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-20T16:07:58.581 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:07:58 smithi184 ceph-mon[36209]: osd.4 [v2:172.21.15.160:6808/4160830014,v1:172.21.15.160:6809/4160830014] boot 2024-05-20T16:07:58.581 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:07:58 smithi184 ceph-mon[36209]: osdmap e34: 5 total, 5 up, 5 in 2024-05-20T16:07:58.581 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:07:58 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-20T16:07:58.581 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:07:58 smithi184 ceph-mon[36209]: Detected new or changed devices on smithi160 2024-05-20T16:07:58.581 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:07:58 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:07:58.581 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:07:58 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:07:58.581 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:07:58 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-20T16:07:58.581 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:07:58 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-20T16:07:58.582 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:07:58 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:07:58.582 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:07:58 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:07:58.582 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:07:58 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T16:07:58.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:07:58 smithi082 ceph-mon[32238]: purged_snaps scrub starts 2024-05-20T16:07:58.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:07:58 smithi082 ceph-mon[32238]: purged_snaps scrub ok 2024-05-20T16:07:58.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:07:58 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-20T16:07:58.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:07:58 smithi082 ceph-mon[32238]: osd.4 [v2:172.21.15.160:6808/4160830014,v1:172.21.15.160:6809/4160830014] boot 2024-05-20T16:07:58.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:07:58 smithi082 ceph-mon[32238]: osdmap e34: 5 total, 5 up, 5 in 2024-05-20T16:07:58.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:07:58 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-20T16:07:58.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:07:58 smithi082 ceph-mon[32238]: Detected new or changed devices on smithi160 2024-05-20T16:07:58.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:07:58 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:07:58.611 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:07:58 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:07:58.611 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:07:58 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-20T16:07:58.611 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:07:58 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-20T16:07:58.611 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:07:58 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:07:58.611 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:07:58 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:07:58.611 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:07:58 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T16:07:59.364 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:07:59 smithi184 ceph-mon[36209]: Adjusting osd_memory_target on smithi160 to 8544M 2024-05-20T16:07:59.364 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:07:59 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:07:59.364 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:07:59 smithi184 ceph-mon[36209]: osdmap e35: 5 total, 5 up, 5 in 2024-05-20T16:07:59.365 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:07:59 smithi184 ceph-mon[36209]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2024-05-20T16:07:59.562 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:07:59 smithi160 ceph-mon[36347]: Adjusting osd_memory_target on smithi160 to 8544M 2024-05-20T16:07:59.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:07:59 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:07:59.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:07:59 smithi160 ceph-mon[36347]: osdmap e35: 5 total, 5 up, 5 in 2024-05-20T16:07:59.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:07:59 smithi160 ceph-mon[36347]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2024-05-20T16:07:59.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:07:59 smithi082 ceph-mon[32238]: Adjusting osd_memory_target on smithi160 to 8544M 2024-05-20T16:07:59.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:07:59 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:07:59.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:07:59 smithi082 ceph-mon[32238]: osdmap e35: 5 total, 5 up, 5 in 2024-05-20T16:07:59.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:07:59 smithi082 ceph-mon[32238]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2024-05-20T16:07:59.875 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:07:59.872+0000 7f66fb1c3700 1 -- 172.21.15.184:0/1567977375 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f66f41020c0 msgr2=0x7f66f41024d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:07:59.875 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:07:59.872+0000 7f66fb1c3700 1 --2- 172.21.15.184:0/1567977375 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f66f41020c0 0x7f66f41024d0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f66e8009990 tx=0x7f66e8009ca0 comp rx=0 tx=0).stop 2024-05-20T16:07:59.875 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:07:59.872+0000 7f66fb1c3700 1 -- 172.21.15.184:0/1567977375 shutdown_connections 2024-05-20T16:07:59.875 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:07:59.872+0000 7f66fb1c3700 1 --2- 172.21.15.184:0/1567977375 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f66f4103b90 0x7f66f410a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:07:59.875 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:07:59.872+0000 7f66fb1c3700 1 --2- 172.21.15.184:0/1567977375 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f66f41020c0 0x7f66f41024d0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:07:59.876 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:07:59.872+0000 7f66fb1c3700 1 --2- 172.21.15.184:0/1567977375 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f66f41031f0 0x7f66f4103600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:07:59.876 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:07:59.872+0000 7f66fb1c3700 1 -- 172.21.15.184:0/1567977375 >> 172.21.15.184:0/1567977375 conn(0x7f66f40fd700 msgr2=0x7f66f40ffbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:07:59.876 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:07:59.872+0000 7f66fb1c3700 1 -- 172.21.15.184:0/1567977375 shutdown_connections 2024-05-20T16:07:59.876 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:07:59.872+0000 7f66fb1c3700 1 -- 172.21.15.184:0/1567977375 wait complete. 2024-05-20T16:07:59.876 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:07:59.873+0000 7f66fb1c3700 1 Processor -- start 2024-05-20T16:07:59.876 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:07:59.873+0000 7f66fb1c3700 1 -- start start 2024-05-20T16:07:59.876 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:07:59.873+0000 7f66fb1c3700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f66f41020c0 0x7f66f406af50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:07:59.876 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:07:59.873+0000 7f66fb1c3700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f66f41031f0 0x7f66f419e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:07:59.876 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:07:59.873+0000 7f66fb1c3700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f66f4103b90 0x7f66f41a0b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:07:59.876 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:07:59.873+0000 7f66fb1c3700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f66f410dc60 con 0x7f66f41031f0 2024-05-20T16:07:59.876 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:07:59.873+0000 7f66fb1c3700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f66f410dae0 con 0x7f66f41020c0 2024-05-20T16:07:59.876 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:07:59.873+0000 7f66fb1c3700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f66f410dde0 con 0x7f66f4103b90 2024-05-20T16:07:59.876 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:07:59.874+0000 7f66f9760700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f66f4103b90 0x7f66f41a0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:07:59.877 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:07:59.874+0000 7f66f9760700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f66f4103b90 0x7f66f41a0b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.184:39392/0 (socket says 172.21.15.184:39392) 2024-05-20T16:07:59.877 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:07:59.874+0000 7f66f9760700 1 -- 172.21.15.184:0/638484707 learned_addr learned my addr 172.21.15.184:0/638484707 (peer_addr_for_me v2:172.21.15.184:0/0) 2024-05-20T16:07:59.877 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:07:59.874+0000 7f66f8f5f700 1 --2- 172.21.15.184:0/638484707 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f66f41020c0 0x7f66f406af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:07:59.877 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:07:59.874+0000 7f66f3fff700 1 --2- 172.21.15.184:0/638484707 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f66f41031f0 0x7f66f419e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:07:59.877 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:07:59.874+0000 7f66f9760700 1 -- 172.21.15.184:0/638484707 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f66f41020c0 msgr2=0x7f66f406af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:07:59.877 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:07:59.874+0000 7f66f9760700 1 --2- 172.21.15.184:0/638484707 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f66f41020c0 0x7f66f406af50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:07:59.877 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:07:59.874+0000 7f66f9760700 1 -- 172.21.15.184:0/638484707 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f66f41031f0 msgr2=0x7f66f419e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:07:59.877 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:07:59.874+0000 7f66f9760700 1 --2- 172.21.15.184:0/638484707 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f66f41031f0 0x7f66f419e5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:07:59.877 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:07:59.874+0000 7f66f9760700 1 -- 172.21.15.184:0/638484707 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66f41a1320 con 0x7f66f4103b90 2024-05-20T16:07:59.877 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:07:59.874+0000 7f66f9760700 1 --2- 172.21.15.184:0/638484707 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f66f4103b90 0x7f66f41a0b80 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f66ec007ee0 tx=0x7f66ec00d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:07:59.877 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:07:59.874+0000 7f66f1ffb700 1 -- 172.21.15.184:0/638484707 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66ec00dbd0 con 0x7f66f4103b90 2024-05-20T16:07:59.877 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:07:59.874+0000 7f66fb1c3700 1 -- 172.21.15.184:0/638484707 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66f41a1610 con 0x7f66f4103b90 2024-05-20T16:07:59.877 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:07:59.874+0000 7f66f1ffb700 1 -- 172.21.15.184:0/638484707 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f66ec00dd70 con 0x7f66f4103b90 2024-05-20T16:07:59.878 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:07:59.874+0000 7f66f1ffb700 1 -- 172.21.15.184:0/638484707 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66ec005a60 con 0x7f66f4103b90 2024-05-20T16:07:59.878 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:07:59.874+0000 7f66fb1c3700 1 -- 172.21.15.184:0/638484707 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66f41a1b00 con 0x7f66f4103b90 2024-05-20T16:07:59.878 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:07:59.875+0000 7f66fb1c3700 1 -- 172.21.15.184:0/638484707 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66f4050170 con 0x7f66f4103b90 2024-05-20T16:07:59.879 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:07:59.876+0000 7f66f1ffb700 1 -- 172.21.15.184:0/638484707 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f66ec01a020 con 0x7f66f4103b90 2024-05-20T16:07:59.879 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:07:59.877+0000 7f66f1ffb700 1 --2- 172.21.15.184:0/638484707 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f66e40a2c20 0x7f66e40a5130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:07:59.879 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:07:59.877+0000 7f66f1ffb700 1 -- 172.21.15.184:0/638484707 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 4101+0+0 (secure 0 0 0) 0x7f66ec0cd740 con 0x7f66f4103b90 2024-05-20T16:07:59.879 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:07:59.877+0000 7f66f8f5f700 1 --2- 172.21.15.184:0/638484707 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f66e40a2c20 0x7f66e40a5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:07:59.879 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:07:59.878+0000 7f66f8f5f700 1 --2- 172.21.15.184:0/638484707 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f66e40a2c20 0x7f66e40a5130 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f66e80098e0 tx=0x7f66e801a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:07:59.882 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:07:59.882+0000 7f66f1ffb700 1 -- 172.21.15.184:0/638484707 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f66ec010070 con 0x7f66f4103b90 2024-05-20T16:08:00.088 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:08:00.087+0000 7f66fb1c3700 1 -- 172.21.15.184:0/638484707 --> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi184:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f66f4062b40 con 0x7f66e40a2c20 2024-05-20T16:08:00.386 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:08:00 smithi184 ceph-mon[36209]: from='client.24235 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi184:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-20T16:08:00.386 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:08:00 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-20T16:08:00.386 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:08:00 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-20T16:08:00.386 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:08:00 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:08:00.562 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:08:00 smithi160 ceph-mon[36347]: from='client.24235 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi184:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-20T16:08:00.562 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:08:00 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-20T16:08:00.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:08:00 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-20T16:08:00.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:08:00 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:08:00.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:08:00 smithi082 ceph-mon[32238]: from='client.24235 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi184:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-20T16:08:00.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:08:00 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-20T16:08:00.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:08:00 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-20T16:08:00.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:08:00 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:08:01.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:08:01 smithi160 ceph-mon[36347]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2024-05-20T16:08:01.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:08:01 smithi082 ceph-mon[32238]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2024-05-20T16:08:01.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:08:01 smithi184 ceph-mon[36209]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2024-05-20T16:08:03.219 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:08:02 smithi184 ceph-mon[36209]: from='client.? 172.21.15.184:0/3249583426' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c70cfcd1-a4ff-461f-8c73-479b061200fb"}]: dispatch 2024-05-20T16:08:03.220 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:08:02 smithi184 ceph-mon[36209]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c70cfcd1-a4ff-461f-8c73-479b061200fb"}]: dispatch 2024-05-20T16:08:03.220 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:08:02 smithi184 ceph-mon[36209]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c70cfcd1-a4ff-461f-8c73-479b061200fb"}]': finished 2024-05-20T16:08:03.220 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:08:02 smithi184 ceph-mon[36209]: osdmap e36: 6 total, 5 up, 6 in 2024-05-20T16:08:03.220 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:08:02 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-20T16:08:03.312 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:08:02 smithi160 ceph-mon[36347]: from='client.? 172.21.15.184:0/3249583426' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c70cfcd1-a4ff-461f-8c73-479b061200fb"}]: dispatch 2024-05-20T16:08:03.313 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:08:02 smithi160 ceph-mon[36347]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c70cfcd1-a4ff-461f-8c73-479b061200fb"}]: dispatch 2024-05-20T16:08:03.313 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:08:02 smithi160 ceph-mon[36347]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c70cfcd1-a4ff-461f-8c73-479b061200fb"}]': finished 2024-05-20T16:08:03.313 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:08:02 smithi160 ceph-mon[36347]: osdmap e36: 6 total, 5 up, 6 in 2024-05-20T16:08:03.313 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:08:02 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-20T16:08:03.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:08:02 smithi082 ceph-mon[32238]: from='client.? 172.21.15.184:0/3249583426' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c70cfcd1-a4ff-461f-8c73-479b061200fb"}]: dispatch 2024-05-20T16:08:03.360 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:08:02 smithi082 ceph-mon[32238]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c70cfcd1-a4ff-461f-8c73-479b061200fb"}]: dispatch 2024-05-20T16:08:03.360 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:08:02 smithi082 ceph-mon[32238]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c70cfcd1-a4ff-461f-8c73-479b061200fb"}]': finished 2024-05-20T16:08:03.360 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:08:02 smithi082 ceph-mon[32238]: osdmap e36: 6 total, 5 up, 6 in 2024-05-20T16:08:03.360 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:08:02 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-20T16:08:04.312 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:08:03 smithi160 ceph-mon[36347]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2024-05-20T16:08:04.313 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:08:03 smithi160 ceph-mon[36347]: from='client.? 172.21.15.184:0/2263371587' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-20T16:08:04.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:08:03 smithi082 ceph-mon[32238]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2024-05-20T16:08:04.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:08:03 smithi082 ceph-mon[32238]: from='client.? 172.21.15.184:0/2263371587' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-20T16:08:04.386 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:08:03 smithi184 ceph-mon[36209]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2024-05-20T16:08:04.386 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:08:03 smithi184 ceph-mon[36209]: from='client.? 172.21.15.184:0/2263371587' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-20T16:08:06.553 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:08:06 smithi184 ceph-mon[36209]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2024-05-20T16:08:06.562 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:08:06 smithi160 ceph-mon[36347]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2024-05-20T16:08:06.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:08:06 smithi082 ceph-mon[32238]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2024-05-20T16:08:08.386 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:08:08 smithi184 ceph-mon[36209]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2024-05-20T16:08:08.562 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:08:08 smithi160 ceph-mon[36347]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2024-05-20T16:08:08.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:08:08 smithi082 ceph-mon[32238]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2024-05-20T16:08:10.562 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:08:10 smithi160 ceph-mon[36347]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2024-05-20T16:08:10.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:08:10 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-05-20T16:08:10.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:08:10 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:08:10.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:08:10 smithi082 ceph-mon[32238]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2024-05-20T16:08:10.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:08:10 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-05-20T16:08:10.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:08:10 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:08:10.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:08:10 smithi184 ceph-mon[36209]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2024-05-20T16:08:10.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:08:10 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-05-20T16:08:10.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:08:10 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:08:11.386 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:08:11 smithi184 ceph-mon[36209]: Deploying daemon osd.5 on smithi184 2024-05-20T16:08:11.562 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:08:11 smithi160 ceph-mon[36347]: Deploying daemon osd.5 on smithi184 2024-05-20T16:08:11.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:08:11 smithi082 ceph-mon[32238]: Deploying daemon osd.5 on smithi184 2024-05-20T16:08:12.386 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:08:12 smithi184 ceph-mon[36209]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2024-05-20T16:08:12.562 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:08:12 smithi160 ceph-mon[36347]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2024-05-20T16:08:12.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:08:12 smithi082 ceph-mon[32238]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2024-05-20T16:08:13.562 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:08:13 smithi160 ceph-mon[36347]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2024-05-20T16:08:13.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:08:13 smithi082 ceph-mon[32238]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2024-05-20T16:08:13.638 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:08:13 smithi184 ceph-mon[36209]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2024-05-20T16:08:16.347 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:08:16 smithi184 ceph-mon[36209]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2024-05-20T16:08:16.347 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:08:16 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:08:16.347 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:08:16 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:08:16.347 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:08:16 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T16:08:16.562 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:08:16 smithi160 ceph-mon[36347]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2024-05-20T16:08:16.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:08:16 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:08:16.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:08:16 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:08:16.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:08:16 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T16:08:16.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:08:16 smithi082 ceph-mon[32238]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2024-05-20T16:08:16.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:08:16 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:08:16.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:08:16 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:08:16.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:08:16 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T16:08:18.562 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:08:18 smithi160 ceph-mon[36347]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2024-05-20T16:08:18.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:08:18 smithi082 ceph-mon[32238]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2024-05-20T16:08:18.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:08:18 smithi184 ceph-mon[36209]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2024-05-20T16:08:19.305 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:08:19.304+0000 7f66f1ffb700 1 -- 172.21.15.184:0/638484707 <== mgr.14150 v2:172.21.15.82:6800/379530823 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f66f4062b40 con 0x7f66e40a2c20 2024-05-20T16:08:19.305 INFO:teuthology.orchestra.run.smithi184.stdout:Created osd(s) 5 on host 'smithi184' 2024-05-20T16:08:19.307 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:08:19.307+0000 7f66fb1c3700 1 -- 172.21.15.184:0/638484707 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f66e40a2c20 msgr2=0x7f66e40a5130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:08:19.308 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:08:19.307+0000 7f66fb1c3700 1 --2- 172.21.15.184:0/638484707 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f66e40a2c20 0x7f66e40a5130 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f66e80098e0 tx=0x7f66e801a040 comp rx=0 tx=0).stop 2024-05-20T16:08:19.308 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:08:19.307+0000 7f66fb1c3700 1 -- 172.21.15.184:0/638484707 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f66f4103b90 msgr2=0x7f66f41a0b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:08:19.308 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:08:19.307+0000 7f66fb1c3700 1 --2- 172.21.15.184:0/638484707 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f66f4103b90 0x7f66f41a0b80 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f66ec007ee0 tx=0x7f66ec00d3b0 comp rx=0 tx=0).stop 2024-05-20T16:08:19.308 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:08:19.307+0000 7f66fb1c3700 1 -- 172.21.15.184:0/638484707 shutdown_connections 2024-05-20T16:08:19.308 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:08:19.307+0000 7f66fb1c3700 1 --2- 172.21.15.184:0/638484707 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f66e40a2c20 0x7f66e40a5130 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:08:19.308 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:08:19.307+0000 7f66fb1c3700 1 --2- 172.21.15.184:0/638484707 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f66f41020c0 0x7f66f406af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:08:19.308 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:08:19.307+0000 7f66fb1c3700 1 --2- 172.21.15.184:0/638484707 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f66f4103b90 0x7f66f41a0b80 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:08:19.308 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:08:19.307+0000 7f66fb1c3700 1 --2- 172.21.15.184:0/638484707 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f66f41031f0 0x7f66f419e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:08:19.308 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:08:19.307+0000 7f66fb1c3700 1 -- 172.21.15.184:0/638484707 >> 172.21.15.184:0/638484707 conn(0x7f66f40fd700 msgr2=0x7f66f4106420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:08:19.308 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:08:19.307+0000 7f66fb1c3700 1 -- 172.21.15.184:0/638484707 shutdown_connections 2024-05-20T16:08:19.308 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:08:19.307+0000 7f66fb1c3700 1 -- 172.21.15.184:0/638484707 wait complete. 2024-05-20T16:08:19.537 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:08:19 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:08:19.537 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:08:19 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:08:19.537 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:08:19 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:08:19.538 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:08:19 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T16:08:19.538 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:08:19 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:08:19.538 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:08:19 smithi184 ceph-mon[36209]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2024-05-20T16:08:19.538 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:08:19 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:08:19.538 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:08:19 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:08:19.538 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:08:19 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T16:08:19.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:08:19 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:08:19.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:08:19 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:08:19.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:08:19 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:08:19.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:08:19 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T16:08:19.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:08:19 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:08:19.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:08:19 smithi082 ceph-mon[32238]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2024-05-20T16:08:19.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:08:19 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:08:19.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:08:19 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:08:19.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:08:19 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T16:08:19.813 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:08:19 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:08:19.813 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:08:19 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:08:19.813 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:08:19 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:08:19.813 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:08:19 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T16:08:19.813 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:08:19 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:08:19.814 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:08:19 smithi160 ceph-mon[36347]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2024-05-20T16:08:19.814 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:08:19 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:08:19.814 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:08:19 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:08:19.814 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:08:19 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T16:08:20.005 DEBUG:teuthology.orchestra.run.smithi184:osd.5> sudo journalctl -f -n 0 -u ceph-59d1600a-16c2-11ef-bc9a-c7b262605968@osd.5.service 2024-05-20T16:08:20.007 INFO:tasks.cephadm:Deploying osd.6 on smithi082 with /dev/vg_nvme/lv_2... 2024-05-20T16:08:20.007 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 59d1600a-16c2-11ef-bc9a-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-05-20T16:08:20.256 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T16:08:20.566 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:08:20 smithi184 ceph-mon[36209]: from='osd.5 [v2:172.21.15.184:6808/1046282672,v1:172.21.15.184:6809/1046282672]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-20T16:08:20.567 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:08:20 smithi184 ceph-mon[36209]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-20T16:08:20.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:08:20 smithi082 ceph-mon[32238]: from='osd.5 [v2:172.21.15.184:6808/1046282672,v1:172.21.15.184:6809/1046282672]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-20T16:08:20.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:08:20 smithi082 ceph-mon[32238]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-20T16:08:20.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:08:20 smithi160 ceph-mon[36347]: from='osd.5 [v2:172.21.15.184:6808/1046282672,v1:172.21.15.184:6809/1046282672]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-20T16:08:20.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:08:20 smithi160 ceph-mon[36347]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-20T16:08:21.542 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:08:21 smithi184 ceph-mon[36209]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-05-20T16:08:21.542 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:08:21 smithi184 ceph-mon[36209]: osdmap e37: 6 total, 5 up, 6 in 2024-05-20T16:08:21.542 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:08:21 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-20T16:08:21.542 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:08:21 smithi184 ceph-mon[36209]: from='osd.5 [v2:172.21.15.184:6808/1046282672,v1:172.21.15.184:6809/1046282672]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]: dispatch 2024-05-20T16:08:21.542 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:08:21 smithi184 ceph-mon[36209]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]: dispatch 2024-05-20T16:08:21.542 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:08:21 smithi184 ceph-mon[36209]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2024-05-20T16:08:21.543 INFO:journalctl@ceph.osd.5.smithi184.stdout:May 20 16:08:21 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-5[48238]: 2024-05-20T16:08:21.369+0000 7fc3d2b7c700 -1 osd.5 0 waiting for initial osdmap 2024-05-20T16:08:21.543 INFO:journalctl@ceph.osd.5.smithi184.stdout:May 20 16:08:21 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-5[48238]: 2024-05-20T16:08:21.375+0000 7fc3ce931700 -1 osd.5 38 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-20T16:08:21.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:08:21 smithi082 ceph-mon[32238]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-05-20T16:08:21.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:08:21 smithi082 ceph-mon[32238]: osdmap e37: 6 total, 5 up, 6 in 2024-05-20T16:08:21.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:08:21 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-20T16:08:21.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:08:21 smithi082 ceph-mon[32238]: from='osd.5 [v2:172.21.15.184:6808/1046282672,v1:172.21.15.184:6809/1046282672]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]: dispatch 2024-05-20T16:08:21.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:08:21 smithi082 ceph-mon[32238]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]: dispatch 2024-05-20T16:08:21.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:08:21 smithi082 ceph-mon[32238]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2024-05-20T16:08:21.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:08:21 smithi160 ceph-mon[36347]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-05-20T16:08:21.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:08:21 smithi160 ceph-mon[36347]: osdmap e37: 6 total, 5 up, 6 in 2024-05-20T16:08:21.813 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:08:21 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-20T16:08:21.813 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:08:21 smithi160 ceph-mon[36347]: from='osd.5 [v2:172.21.15.184:6808/1046282672,v1:172.21.15.184:6809/1046282672]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]: dispatch 2024-05-20T16:08:21.813 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:08:21 smithi160 ceph-mon[36347]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]: dispatch 2024-05-20T16:08:21.813 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:08:21 smithi160 ceph-mon[36347]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2024-05-20T16:08:22.299 INFO:teuthology.orchestra.run.smithi082.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-05-20T16:08:22.299 INFO:teuthology.orchestra.run.smithi082.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-05-20T16:08:22.299 INFO:teuthology.orchestra.run.smithi082.stderr: stderr: 10+0 records in 2024-05-20T16:08:22.300 INFO:teuthology.orchestra.run.smithi082.stderr:10+0 records out 2024-05-20T16:08:22.300 INFO:teuthology.orchestra.run.smithi082.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0120625 s, 869 MB/s 2024-05-20T16:08:22.300 INFO:teuthology.orchestra.run.smithi082.stderr:--> Zapping successful for: 2024-05-20T16:08:22.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:08:22 smithi082 ceph-mon[32238]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]': finished 2024-05-20T16:08:22.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:08:22 smithi082 ceph-mon[32238]: osdmap e38: 6 total, 5 up, 6 in 2024-05-20T16:08:22.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:08:22 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-20T16:08:22.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:08:22 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-20T16:08:22.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:08:22 smithi082 ceph-mon[32238]: osd.5 [v2:172.21.15.184:6808/1046282672,v1:172.21.15.184:6809/1046282672] boot 2024-05-20T16:08:22.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:08:22 smithi082 ceph-mon[32238]: osdmap e39: 6 total, 6 up, 6 in 2024-05-20T16:08:22.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:08:22 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-20T16:08:22.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:08:22 smithi184 ceph-mon[36209]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]': finished 2024-05-20T16:08:22.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:08:22 smithi184 ceph-mon[36209]: osdmap e38: 6 total, 5 up, 6 in 2024-05-20T16:08:22.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:08:22 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-20T16:08:22.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:08:22 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-20T16:08:22.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:08:22 smithi184 ceph-mon[36209]: osd.5 [v2:172.21.15.184:6808/1046282672,v1:172.21.15.184:6809/1046282672] boot 2024-05-20T16:08:22.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:08:22 smithi184 ceph-mon[36209]: osdmap e39: 6 total, 6 up, 6 in 2024-05-20T16:08:22.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:08:22 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-20T16:08:22.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:08:22 smithi160 ceph-mon[36347]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]': finished 2024-05-20T16:08:22.813 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:08:22 smithi160 ceph-mon[36347]: osdmap e38: 6 total, 5 up, 6 in 2024-05-20T16:08:22.813 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:08:22 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-20T16:08:22.813 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:08:22 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-20T16:08:22.813 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:08:22 smithi160 ceph-mon[36347]: osd.5 [v2:172.21.15.184:6808/1046282672,v1:172.21.15.184:6809/1046282672] boot 2024-05-20T16:08:22.813 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:08:22 smithi160 ceph-mon[36347]: osdmap e39: 6 total, 6 up, 6 in 2024-05-20T16:08:22.813 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:08:22 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-20T16:08:22.928 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 59d1600a-16c2-11ef-bc9a-c7b262605968 -- ceph orch daemon add osd smithi082:vg_nvme/lv_2 2024-05-20T16:08:23.206 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T16:08:23.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:08:23 smithi082 ceph-mon[32238]: purged_snaps scrub starts 2024-05-20T16:08:23.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:08:23 smithi082 ceph-mon[32238]: purged_snaps scrub ok 2024-05-20T16:08:23.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:08:23 smithi082 ceph-mon[32238]: osdmap e40: 6 total, 6 up, 6 in 2024-05-20T16:08:23.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:08:23 smithi082 ceph-mon[32238]: Detected new or changed devices on smithi184 2024-05-20T16:08:23.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:08:23 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:08:23.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:08:23 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:08:23.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:08:23 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-20T16:08:23.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:08:23 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-20T16:08:23.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:08:23 smithi082 ceph-mon[32238]: Adjusting osd_memory_target on smithi184 to 8544M 2024-05-20T16:08:23.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:08:23 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:08:23.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:08:23 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:08:23.611 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:08:23 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T16:08:23.611 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:08:23 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:08:23.611 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:08:23 smithi082 ceph-mon[32238]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2024-05-20T16:08:23.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:08:23 smithi160 ceph-mon[36347]: purged_snaps scrub starts 2024-05-20T16:08:23.813 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:08:23 smithi160 ceph-mon[36347]: purged_snaps scrub ok 2024-05-20T16:08:23.813 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:08:23 smithi160 ceph-mon[36347]: osdmap e40: 6 total, 6 up, 6 in 2024-05-20T16:08:23.813 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:08:23 smithi160 ceph-mon[36347]: Detected new or changed devices on smithi184 2024-05-20T16:08:23.813 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:08:23 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:08:23.813 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:08:23 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:08:23.813 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:08:23 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-20T16:08:23.813 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:08:23 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-20T16:08:23.813 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:08:23 smithi160 ceph-mon[36347]: Adjusting osd_memory_target on smithi184 to 8544M 2024-05-20T16:08:23.813 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:08:23 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:08:23.813 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:08:23 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:08:23.813 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:08:23 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T16:08:23.813 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:08:23 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:08:23.813 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:08:23 smithi160 ceph-mon[36347]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2024-05-20T16:08:23.886 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:08:23 smithi184 ceph-mon[36209]: purged_snaps scrub starts 2024-05-20T16:08:23.886 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:08:23 smithi184 ceph-mon[36209]: purged_snaps scrub ok 2024-05-20T16:08:23.886 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:08:23 smithi184 ceph-mon[36209]: osdmap e40: 6 total, 6 up, 6 in 2024-05-20T16:08:23.886 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:08:23 smithi184 ceph-mon[36209]: Detected new or changed devices on smithi184 2024-05-20T16:08:23.887 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:08:23 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:08:23.887 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:08:23 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:08:23.887 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:08:23 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-20T16:08:23.887 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:08:23 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-20T16:08:23.887 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:08:23 smithi184 ceph-mon[36209]: Adjusting osd_memory_target on smithi184 to 8544M 2024-05-20T16:08:23.887 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:08:23 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:08:23.887 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:08:23 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:08:23.887 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:08:23 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T16:08:23.887 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:08:23 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:08:23.887 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:08:23 smithi184 ceph-mon[36209]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2024-05-20T16:08:24.702 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:08:24 smithi082 ceph-mon[32238]: osdmap e41: 6 total, 6 up, 6 in 2024-05-20T16:08:24.787 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:08:24 smithi184 ceph-mon[36209]: osdmap e41: 6 total, 6 up, 6 in 2024-05-20T16:08:24.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:08:24 smithi160 ceph-mon[36347]: osdmap e41: 6 total, 6 up, 6 in 2024-05-20T16:08:24.965 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:08:24.964+0000 7fec8f3c5700 1 -- 172.21.15.82:0/1301217906 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fec88103ba0 msgr2=0x7fec8810a450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:08:24.965 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:08:24.964+0000 7fec8f3c5700 1 --2- 172.21.15.82:0/1301217906 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fec88103ba0 0x7fec8810a450 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fec8000b0a0 tx=0x7fec8000b3b0 comp rx=0 tx=0).stop 2024-05-20T16:08:24.965 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:08:24.964+0000 7fec8f3c5700 1 -- 172.21.15.82:0/1301217906 shutdown_connections 2024-05-20T16:08:24.965 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:08:24.964+0000 7fec8f3c5700 1 --2- 172.21.15.82:0/1301217906 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fec881020d0 0x7fec881024e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:08:24.965 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:08:24.964+0000 7fec8f3c5700 1 --2- 172.21.15.82:0/1301217906 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fec88103ba0 0x7fec8810a450 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:08:24.965 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:08:24.964+0000 7fec8f3c5700 1 --2- 172.21.15.82:0/1301217906 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fec88103200 0x7fec88103610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:08:24.965 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:08:24.964+0000 7fec8f3c5700 1 -- 172.21.15.82:0/1301217906 >> 172.21.15.82:0/1301217906 conn(0x7fec880fd710 msgr2=0x7fec880ffbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:08:24.965 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:08:24.964+0000 7fec8f3c5700 1 -- 172.21.15.82:0/1301217906 shutdown_connections 2024-05-20T16:08:24.965 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:08:24.964+0000 7fec8f3c5700 1 -- 172.21.15.82:0/1301217906 wait complete. 2024-05-20T16:08:24.967 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:08:24.965+0000 7fec8f3c5700 1 Processor -- start 2024-05-20T16:08:24.967 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:08:24.965+0000 7fec8f3c5700 1 -- start start 2024-05-20T16:08:24.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:08:24.965+0000 7fec8f3c5700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fec881020d0 0x7fec88198830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:08:24.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:08:24.965+0000 7fec8f3c5700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fec88103200 0x7fec88198d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:08:24.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:08:24.965+0000 7fec8f3c5700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fec88103ba0 0x7fec8819fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:08:24.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:08:24.965+0000 7fec8f3c5700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fec8810dd80 con 0x7fec881020d0 2024-05-20T16:08:24.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:08:24.966+0000 7fec8f3c5700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fec8810dc00 con 0x7fec88103200 2024-05-20T16:08:24.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:08:24.966+0000 7fec8f3c5700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fec8810df00 con 0x7fec88103ba0 2024-05-20T16:08:24.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:08:24.966+0000 7fec8d161700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fec881020d0 0x7fec88198830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:08:24.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:08:24.966+0000 7fec8d161700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fec881020d0 0x7fec88198830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:50356/0 (socket says 172.21.15.82:50356) 2024-05-20T16:08:24.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:08:24.966+0000 7fec8d161700 1 -- 172.21.15.82:0/2858407789 learned_addr learned my addr 172.21.15.82:0/2858407789 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:08:24.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:08:24.966+0000 7fec8d962700 1 --2- 172.21.15.82:0/2858407789 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fec88103ba0 0x7fec8819fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:08:24.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:08:24.966+0000 7fec8c960700 1 --2- 172.21.15.82:0/2858407789 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fec88103200 0x7fec88198d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:08:24.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:08:24.966+0000 7fec8d161700 1 -- 172.21.15.82:0/2858407789 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fec88103ba0 msgr2=0x7fec8819fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:08:24.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:08:24.966+0000 7fec8d161700 1 --2- 172.21.15.82:0/2858407789 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fec88103ba0 0x7fec8819fd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:08:24.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:08:24.966+0000 7fec8d161700 1 -- 172.21.15.82:0/2858407789 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fec88103200 msgr2=0x7fec88198d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:08:24.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:08:24.966+0000 7fec8d161700 1 --2- 172.21.15.82:0/2858407789 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fec88103200 0x7fec88198d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:08:24.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:08:24.966+0000 7fec8d161700 1 -- 172.21.15.82:0/2858407789 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec881a0390 con 0x7fec881020d0 2024-05-20T16:08:24.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:08:24.966+0000 7fec8c960700 1 --2- 172.21.15.82:0/2858407789 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fec88103200 0x7fec88198d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T16:08:24.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:08:24.966+0000 7fec8d161700 1 --2- 172.21.15.82:0/2858407789 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fec881020d0 0x7fec88198830 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fec8400d990 tx=0x7fec8400dca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:08:24.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:08:24.966+0000 7fec7e7fc700 1 -- 172.21.15.82:0/2858407789 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec8401b070 con 0x7fec881020d0 2024-05-20T16:08:24.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:08:24.966+0000 7fec7e7fc700 1 -- 172.21.15.82:0/2858407789 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fec8400ebd0 con 0x7fec881020d0 2024-05-20T16:08:24.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:08:24.966+0000 7fec7e7fc700 1 -- 172.21.15.82:0/2858407789 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec840145e0 con 0x7fec881020d0 2024-05-20T16:08:24.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:08:24.966+0000 7fec8f3c5700 1 -- 172.21.15.82:0/2858407789 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec880748d0 con 0x7fec881020d0 2024-05-20T16:08:24.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:08:24.966+0000 7fec8f3c5700 1 -- 172.21.15.82:0/2858407789 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec88074ee0 con 0x7fec881020d0 2024-05-20T16:08:24.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:08:24.967+0000 7fec8f3c5700 1 -- 172.21.15.82:0/2858407789 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec88050170 con 0x7fec881020d0 2024-05-20T16:08:24.970 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:08:24.969+0000 7fec7e7fc700 1 -- 172.21.15.82:0/2858407789 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fec8401e070 con 0x7fec881020d0 2024-05-20T16:08:24.970 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:08:24.970+0000 7fec7e7fc700 1 --2- 172.21.15.82:0/2858407789 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fec740a2ca0 0x7fec740a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:08:24.971 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:08:24.970+0000 7fec8c960700 1 --2- 172.21.15.82:0/2858407789 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fec740a2ca0 0x7fec740a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:08:24.971 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:08:24.970+0000 7fec7e7fc700 1 -- 172.21.15.82:0/2858407789 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 4533+0+0 (secure 0 0 0) 0x7fec840cfab0 con 0x7fec881020d0 2024-05-20T16:08:24.971 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:08:24.970+0000 7fec8c960700 1 --2- 172.21.15.82:0/2858407789 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fec740a2ca0 0x7fec740a51b0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fec881998e0 tx=0x7fec78008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:08:24.974 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:08:24.974+0000 7fec7e7fc700 1 -- 172.21.15.82:0/2858407789 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fec8409d960 con 0x7fec881020d0 2024-05-20T16:08:25.179 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:08:25.179+0000 7fec8f3c5700 1 -- 172.21.15.82:0/2858407789 --> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi082:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fec880751a0 con 0x7fec740a2ca0 2024-05-20T16:08:25.794 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:08:25 smithi082 ceph-mon[32238]: osdmap e42: 6 total, 6 up, 6 in 2024-05-20T16:08:25.794 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:08:25 smithi082 ceph-mon[32238]: from='client.14355 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi082:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-20T16:08:25.794 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:08:25 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-20T16:08:25.794 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:08:25 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-20T16:08:25.794 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:08:25 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:08:25.794 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:08:25 smithi082 ceph-mon[32238]: pgmap v142: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail; 150 KiB/s, 0 objects/s recovering 2024-05-20T16:08:25.803 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:08:25 smithi184 ceph-mon[36209]: osdmap e42: 6 total, 6 up, 6 in 2024-05-20T16:08:25.803 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:08:25 smithi184 ceph-mon[36209]: from='client.14355 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi082:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-20T16:08:25.803 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:08:25 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-20T16:08:25.804 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:08:25 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-20T16:08:25.804 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:08:25 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:08:25.804 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:08:25 smithi184 ceph-mon[36209]: pgmap v142: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail; 150 KiB/s, 0 objects/s recovering 2024-05-20T16:08:25.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:08:25 smithi160 ceph-mon[36347]: osdmap e42: 6 total, 6 up, 6 in 2024-05-20T16:08:25.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:08:25 smithi160 ceph-mon[36347]: from='client.14355 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi082:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-20T16:08:25.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:08:25 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-20T16:08:25.813 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:08:25 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-20T16:08:25.813 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:08:25 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:08:25.813 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:08:25 smithi160 ceph-mon[36347]: pgmap v142: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail; 150 KiB/s, 0 objects/s recovering 2024-05-20T16:08:26.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:08:26 smithi160 ceph-mon[36347]: osdmap e43: 6 total, 6 up, 6 in 2024-05-20T16:08:26.821 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:08:26 smithi082 ceph-mon[32238]: osdmap e43: 6 total, 6 up, 6 in 2024-05-20T16:08:26.886 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:08:26 smithi184 ceph-mon[36209]: osdmap e43: 6 total, 6 up, 6 in 2024-05-20T16:08:27.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:08:27 smithi160 ceph-mon[36347]: pgmap v144: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 123 KiB/s, 0 objects/s recovering 2024-05-20T16:08:27.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:08:27 smithi082 ceph-mon[32238]: pgmap v144: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 123 KiB/s, 0 objects/s recovering 2024-05-20T16:08:27.885 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:08:27 smithi184 ceph-mon[36209]: pgmap v144: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 123 KiB/s, 0 objects/s recovering 2024-05-20T16:08:28.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:08:28 smithi160 ceph-mon[36347]: from='client.? 172.21.15.82:0/4079261532' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d47ce001-9dd1-4830-bcc9-888601187c6d"}]: dispatch 2024-05-20T16:08:28.813 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:08:28 smithi160 ceph-mon[36347]: from='client.? 172.21.15.82:0/4079261532' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d47ce001-9dd1-4830-bcc9-888601187c6d"}]': finished 2024-05-20T16:08:28.813 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:08:28 smithi160 ceph-mon[36347]: osdmap e44: 7 total, 6 up, 7 in 2024-05-20T16:08:28.813 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:08:28 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-20T16:08:28.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:08:28 smithi082 ceph-mon[32238]: from='client.? 172.21.15.82:0/4079261532' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d47ce001-9dd1-4830-bcc9-888601187c6d"}]: dispatch 2024-05-20T16:08:28.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:08:28 smithi082 ceph-mon[32238]: from='client.? 172.21.15.82:0/4079261532' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d47ce001-9dd1-4830-bcc9-888601187c6d"}]': finished 2024-05-20T16:08:28.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:08:28 smithi082 ceph-mon[32238]: osdmap e44: 7 total, 6 up, 7 in 2024-05-20T16:08:28.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:08:28 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-20T16:08:28.886 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:08:28 smithi184 ceph-mon[36209]: from='client.? 172.21.15.82:0/4079261532' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d47ce001-9dd1-4830-bcc9-888601187c6d"}]: dispatch 2024-05-20T16:08:28.886 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:08:28 smithi184 ceph-mon[36209]: from='client.? 172.21.15.82:0/4079261532' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d47ce001-9dd1-4830-bcc9-888601187c6d"}]': finished 2024-05-20T16:08:28.886 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:08:28 smithi184 ceph-mon[36209]: osdmap e44: 7 total, 6 up, 7 in 2024-05-20T16:08:28.886 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:08:28 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-20T16:08:29.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:08:29 smithi160 ceph-mon[36347]: from='client.? 172.21.15.82:0/3184765539' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-20T16:08:29.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:08:29 smithi160 ceph-mon[36347]: pgmap v146: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 101 KiB/s, 0 objects/s recovering 2024-05-20T16:08:29.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:08:29 smithi082 ceph-mon[32238]: from='client.? 172.21.15.82:0/3184765539' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-20T16:08:29.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:08:29 smithi082 ceph-mon[32238]: pgmap v146: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 101 KiB/s, 0 objects/s recovering 2024-05-20T16:08:29.886 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:08:29 smithi184 ceph-mon[36209]: from='client.? 172.21.15.82:0/3184765539' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-20T16:08:29.886 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:08:29 smithi184 ceph-mon[36209]: pgmap v146: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 101 KiB/s, 0 objects/s recovering 2024-05-20T16:08:32.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:08:32 smithi082 ceph-mon[32238]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2024-05-20T16:08:32.562 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:08:32 smithi160 ceph-mon[36347]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2024-05-20T16:08:32.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:08:32 smithi184 ceph-mon[36209]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2024-05-20T16:08:34.274 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:08:34 smithi082 ceph-mon[32238]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2024-05-20T16:08:34.562 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:08:34 smithi160 ceph-mon[36347]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2024-05-20T16:08:34.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:08:34 smithi184 ceph-mon[36209]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2024-05-20T16:08:36.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:08:36 smithi082 ceph-mon[32238]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2024-05-20T16:08:36.360 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:08:36 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-05-20T16:08:36.360 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:08:36 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:08:36.562 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:08:36 smithi160 ceph-mon[36347]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2024-05-20T16:08:36.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:08:36 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-05-20T16:08:36.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:08:36 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:08:36.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:08:36 smithi184 ceph-mon[36209]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2024-05-20T16:08:36.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:08:36 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-05-20T16:08:36.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:08:36 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:08:37.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:08:37 smithi082 ceph-mon[32238]: Deploying daemon osd.6 on smithi082 2024-05-20T16:08:37.386 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:08:37 smithi184 ceph-mon[36209]: Deploying daemon osd.6 on smithi082 2024-05-20T16:08:37.562 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:08:37 smithi160 ceph-mon[36347]: Deploying daemon osd.6 on smithi082 2024-05-20T16:08:38.272 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:08:38 smithi160 ceph-mon[36347]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2024-05-20T16:08:38.497 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:08:38 smithi082 ceph-mon[32238]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2024-05-20T16:08:38.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:08:38 smithi184 ceph-mon[36209]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2024-05-20T16:08:39.494 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:08:39 smithi082 ceph-mon[32238]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2024-05-20T16:08:39.562 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:08:39 smithi160 ceph-mon[36347]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2024-05-20T16:08:39.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:08:39 smithi184 ceph-mon[36209]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2024-05-20T16:08:42.562 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:08:42 smithi160 ceph-mon[36347]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2024-05-20T16:08:42.562 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:08:42 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:08:42.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:08:42 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:08:42.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:08:42 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T16:08:42.568 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:08:42 smithi082 ceph-mon[32238]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2024-05-20T16:08:42.568 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:08:42 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:08:42.568 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:08:42 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:08:42.568 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:08:42 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T16:08:42.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:08:42 smithi184 ceph-mon[36209]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2024-05-20T16:08:42.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:08:42 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:08:42.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:08:42 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:08:42.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:08:42 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T16:08:44.562 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:08:44 smithi160 ceph-mon[36347]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2024-05-20T16:08:44.583 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:08:44 smithi082 ceph-mon[32238]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2024-05-20T16:08:44.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:08:44 smithi184 ceph-mon[36209]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2024-05-20T16:08:45.040 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:08:45.040+0000 7fec7e7fc700 1 -- 172.21.15.82:0/2858407789 <== mgr.14150 v2:172.21.15.82:6800/379530823 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fec880751a0 con 0x7fec740a2ca0 2024-05-20T16:08:45.040 INFO:teuthology.orchestra.run.smithi082.stdout:Created osd(s) 6 on host 'smithi082' 2024-05-20T16:08:45.042 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:08:45.042+0000 7fec8f3c5700 1 -- 172.21.15.82:0/2858407789 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fec740a2ca0 msgr2=0x7fec740a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:08:45.043 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:08:45.042+0000 7fec8f3c5700 1 --2- 172.21.15.82:0/2858407789 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fec740a2ca0 0x7fec740a51b0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fec881998e0 tx=0x7fec78008040 comp rx=0 tx=0).stop 2024-05-20T16:08:45.043 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:08:45.042+0000 7fec8f3c5700 1 -- 172.21.15.82:0/2858407789 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fec881020d0 msgr2=0x7fec88198830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:08:45.043 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:08:45.042+0000 7fec8f3c5700 1 --2- 172.21.15.82:0/2858407789 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fec881020d0 0x7fec88198830 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fec8400d990 tx=0x7fec8400dca0 comp rx=0 tx=0).stop 2024-05-20T16:08:45.043 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:08:45.042+0000 7fec8f3c5700 1 -- 172.21.15.82:0/2858407789 shutdown_connections 2024-05-20T16:08:45.043 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:08:45.042+0000 7fec8f3c5700 1 --2- 172.21.15.82:0/2858407789 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fec740a2ca0 0x7fec740a51b0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:08:45.043 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:08:45.042+0000 7fec8f3c5700 1 --2- 172.21.15.82:0/2858407789 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fec88103ba0 0x7fec8819fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:08:45.043 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:08:45.042+0000 7fec8f3c5700 1 --2- 172.21.15.82:0/2858407789 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fec881020d0 0x7fec88198830 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:08:45.043 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:08:45.042+0000 7fec8f3c5700 1 --2- 172.21.15.82:0/2858407789 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fec88103200 0x7fec88198d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:08:45.043 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:08:45.042+0000 7fec8f3c5700 1 -- 172.21.15.82:0/2858407789 >> 172.21.15.82:0/2858407789 conn(0x7fec880fd710 msgr2=0x7fec88106430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:08:45.043 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:08:45.042+0000 7fec8f3c5700 1 -- 172.21.15.82:0/2858407789 shutdown_connections 2024-05-20T16:08:45.043 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:08:45.042+0000 7fec8f3c5700 1 -- 172.21.15.82:0/2858407789 wait complete. 2024-05-20T16:08:45.682 DEBUG:teuthology.orchestra.run.smithi082:osd.6> sudo journalctl -f -n 0 -u ceph-59d1600a-16c2-11ef-bc9a-c7b262605968@osd.6.service 2024-05-20T16:08:45.684 INFO:tasks.cephadm:Deploying osd.7 on smithi160 with /dev/vg_nvme/lv_2... 2024-05-20T16:08:45.685 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 59d1600a-16c2-11ef-bc9a-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-05-20T16:08:45.930 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/mon.b/config 2024-05-20T16:08:46.304 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:08:45 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:08:46.305 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:08:45 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:08:46.305 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:08:45 smithi160 ceph-mon[36347]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2024-05-20T16:08:46.305 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:08:45 smithi160 ceph-mon[36347]: from='osd.6 [v2:172.21.15.82:6818/2789736499,v1:172.21.15.82:6819/2789736499]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-05-20T16:08:46.305 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:08:45 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:08:46.305 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:08:45 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:08:46.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:08:45 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:08:46.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:08:45 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:08:46.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:08:45 smithi082 ceph-mon[32238]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2024-05-20T16:08:46.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:08:45 smithi082 ceph-mon[32238]: from='osd.6 [v2:172.21.15.82:6818/2789736499,v1:172.21.15.82:6819/2789736499]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-05-20T16:08:46.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:08:45 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:08:46.360 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:08:45 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:08:46.386 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:08:45 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:08:46.386 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:08:45 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:08:46.386 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:08:45 smithi184 ceph-mon[36209]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2024-05-20T16:08:46.386 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:08:45 smithi184 ceph-mon[36209]: from='osd.6 [v2:172.21.15.82:6818/2789736499,v1:172.21.15.82:6819/2789736499]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-05-20T16:08:46.386 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:08:45 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:08:46.386 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:08:45 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:08:47.312 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:08:47 smithi160 ceph-mon[36347]: from='osd.6 [v2:172.21.15.82:6818/2789736499,v1:172.21.15.82:6819/2789736499]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-05-20T16:08:47.313 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:08:47 smithi160 ceph-mon[36347]: osdmap e45: 7 total, 6 up, 7 in 2024-05-20T16:08:47.313 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:08:47 smithi160 ceph-mon[36347]: from='osd.6 [v2:172.21.15.82:6818/2789736499,v1:172.21.15.82:6819/2789736499]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]: dispatch 2024-05-20T16:08:47.313 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:08:47 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-20T16:08:47.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:08:47 smithi082 ceph-mon[32238]: from='osd.6 [v2:172.21.15.82:6818/2789736499,v1:172.21.15.82:6819/2789736499]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-05-20T16:08:47.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:08:47 smithi082 ceph-mon[32238]: osdmap e45: 7 total, 6 up, 7 in 2024-05-20T16:08:47.360 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:08:47 smithi082 ceph-mon[32238]: from='osd.6 [v2:172.21.15.82:6818/2789736499,v1:172.21.15.82:6819/2789736499]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]: dispatch 2024-05-20T16:08:47.360 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:08:47 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-20T16:08:47.386 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:08:47 smithi184 ceph-mon[36209]: from='osd.6 [v2:172.21.15.82:6818/2789736499,v1:172.21.15.82:6819/2789736499]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-05-20T16:08:47.386 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:08:47 smithi184 ceph-mon[36209]: osdmap e45: 7 total, 6 up, 7 in 2024-05-20T16:08:47.386 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:08:47 smithi184 ceph-mon[36209]: from='osd.6 [v2:172.21.15.82:6818/2789736499,v1:172.21.15.82:6819/2789736499]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]: dispatch 2024-05-20T16:08:47.386 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:08:47 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-20T16:08:47.713 INFO:journalctl@ceph.osd.6.smithi082.stdout:May 20 16:08:47 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-6[60057]: 2024-05-20T16:08:47.466+0000 7f3f1adf5700 -1 osd.6 0 waiting for initial osdmap 2024-05-20T16:08:47.713 INFO:journalctl@ceph.osd.6.smithi082.stdout:May 20 16:08:47 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-6[60057]: 2024-05-20T16:08:47.475+0000 7f3f12ba2700 -1 osd.6 46 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-20T16:08:48.188 INFO:teuthology.orchestra.run.smithi160.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-05-20T16:08:48.188 INFO:teuthology.orchestra.run.smithi160.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-05-20T16:08:48.188 INFO:teuthology.orchestra.run.smithi160.stderr: stderr: 10+0 records in 2024-05-20T16:08:48.188 INFO:teuthology.orchestra.run.smithi160.stderr:10+0 records out 2024-05-20T16:08:48.188 INFO:teuthology.orchestra.run.smithi160.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0113457 s, 924 MB/s 2024-05-20T16:08:48.188 INFO:teuthology.orchestra.run.smithi160.stderr:--> Zapping successful for: 2024-05-20T16:08:48.312 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:08:48 smithi160 ceph-mon[36347]: from='osd.6 [v2:172.21.15.82:6818/2789736499,v1:172.21.15.82:6819/2789736499]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]': finished 2024-05-20T16:08:48.313 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:08:48 smithi160 ceph-mon[36347]: osdmap e46: 7 total, 6 up, 7 in 2024-05-20T16:08:48.313 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:08:48 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-20T16:08:48.313 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:08:48 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-20T16:08:48.313 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:08:48 smithi160 ceph-mon[36347]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2024-05-20T16:08:48.313 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:08:48 smithi160 ceph-mon[36347]: from='osd.6 [v2:172.21.15.82:6818/2789736499,v1:172.21.15.82:6819/2789736499]' entity='osd.6' 2024-05-20T16:08:48.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:08:48 smithi082 ceph-mon[32238]: from='osd.6 [v2:172.21.15.82:6818/2789736499,v1:172.21.15.82:6819/2789736499]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]': finished 2024-05-20T16:08:48.360 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:08:48 smithi082 ceph-mon[32238]: osdmap e46: 7 total, 6 up, 7 in 2024-05-20T16:08:48.360 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:08:48 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-20T16:08:48.360 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:08:48 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-20T16:08:48.360 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:08:48 smithi082 ceph-mon[32238]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2024-05-20T16:08:48.360 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:08:48 smithi082 ceph-mon[32238]: from='osd.6 [v2:172.21.15.82:6818/2789736499,v1:172.21.15.82:6819/2789736499]' entity='osd.6' 2024-05-20T16:08:48.386 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:08:48 smithi184 ceph-mon[36209]: from='osd.6 [v2:172.21.15.82:6818/2789736499,v1:172.21.15.82:6819/2789736499]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]': finished 2024-05-20T16:08:48.386 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:08:48 smithi184 ceph-mon[36209]: osdmap e46: 7 total, 6 up, 7 in 2024-05-20T16:08:48.386 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:08:48 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-20T16:08:48.386 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:08:48 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-20T16:08:48.386 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:08:48 smithi184 ceph-mon[36209]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2024-05-20T16:08:48.386 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:08:48 smithi184 ceph-mon[36209]: from='osd.6 [v2:172.21.15.82:6818/2789736499,v1:172.21.15.82:6819/2789736499]' entity='osd.6' 2024-05-20T16:08:48.920 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 59d1600a-16c2-11ef-bc9a-c7b262605968 -- ceph orch daemon add osd smithi160:vg_nvme/lv_2 2024-05-20T16:08:49.161 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/mon.b/config 2024-05-20T16:08:49.282 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:08:49 smithi160 ceph-mon[36347]: purged_snaps scrub starts 2024-05-20T16:08:49.282 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:08:49 smithi160 ceph-mon[36347]: purged_snaps scrub ok 2024-05-20T16:08:49.283 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:08:49 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-20T16:08:49.283 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:08:49 smithi160 ceph-mon[36347]: osd.6 [v2:172.21.15.82:6818/2789736499,v1:172.21.15.82:6819/2789736499] boot 2024-05-20T16:08:49.283 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:08:49 smithi160 ceph-mon[36347]: osdmap e47: 7 total, 7 up, 7 in 2024-05-20T16:08:49.283 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:08:49 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-20T16:08:49.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:08:49 smithi082 ceph-mon[32238]: purged_snaps scrub starts 2024-05-20T16:08:49.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:08:49 smithi082 ceph-mon[32238]: purged_snaps scrub ok 2024-05-20T16:08:49.360 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:08:49 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-20T16:08:49.360 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:08:49 smithi082 ceph-mon[32238]: osd.6 [v2:172.21.15.82:6818/2789736499,v1:172.21.15.82:6819/2789736499] boot 2024-05-20T16:08:49.360 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:08:49 smithi082 ceph-mon[32238]: osdmap e47: 7 total, 7 up, 7 in 2024-05-20T16:08:49.360 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:08:49 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-20T16:08:49.386 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:08:49 smithi184 ceph-mon[36209]: purged_snaps scrub starts 2024-05-20T16:08:49.386 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:08:49 smithi184 ceph-mon[36209]: purged_snaps scrub ok 2024-05-20T16:08:49.386 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:08:49 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-20T16:08:49.386 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:08:49 smithi184 ceph-mon[36209]: osd.6 [v2:172.21.15.82:6818/2789736499,v1:172.21.15.82:6819/2789736499] boot 2024-05-20T16:08:49.386 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:08:49 smithi184 ceph-mon[36209]: osdmap e47: 7 total, 7 up, 7 in 2024-05-20T16:08:49.386 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:08:49 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-20T16:08:50.312 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:08:50 smithi160 ceph-mon[36347]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 192 MiB used, 626 GiB / 626 GiB avail 2024-05-20T16:08:50.313 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:08:50 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:08:50.313 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:08:50 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:08:50.313 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:08:50 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-20T16:08:50.313 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:08:50 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-20T16:08:50.313 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:08:50 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-20T16:08:50.313 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:08:50 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:08:50.313 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:08:50 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:08:50.313 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:08:50 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T16:08:50.313 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:08:50 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:08:50.313 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:08:50 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T16:08:50.313 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:08:50 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:08:50.314 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:08:50 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T16:08:50.314 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:08:50 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:08:50.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:08:50 smithi082 ceph-mon[32238]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 192 MiB used, 626 GiB / 626 GiB avail 2024-05-20T16:08:50.360 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:08:50 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:08:50.360 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:08:50 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:08:50.360 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:08:50 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-20T16:08:50.360 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:08:50 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-20T16:08:50.360 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:08:50 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-20T16:08:50.360 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:08:50 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:08:50.360 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:08:50 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:08:50.360 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:08:50 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T16:08:50.360 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:08:50 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:08:50.360 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:08:50 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T16:08:50.360 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:08:50 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:08:50.360 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:08:50 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T16:08:50.361 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:08:50 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:08:50.386 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:08:50 smithi184 ceph-mon[36209]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 192 MiB used, 626 GiB / 626 GiB avail 2024-05-20T16:08:50.386 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:08:50 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:08:50.386 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:08:50 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:08:50.386 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:08:50 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-20T16:08:50.386 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:08:50 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-20T16:08:50.387 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:08:50 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-20T16:08:50.387 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:08:50 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:08:50.387 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:08:50 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:08:50.387 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:08:50 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T16:08:50.387 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:08:50 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:08:50.387 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:08:50 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T16:08:50.387 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:08:50 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:08:50.387 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:08:50 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T16:08:50.387 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:08:50 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:08:50.904 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-20T16:08:50.901+0000 7fc24174c700 1 -- 172.21.15.160:0/3456112310 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc23c103b90 msgr2=0x7fc23c10a440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:08:50.904 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-20T16:08:50.901+0000 7fc24174c700 1 --2- 172.21.15.160:0/3456112310 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc23c103b90 0x7fc23c10a440 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fc23400b0a0 tx=0x7fc23400b3b0 comp rx=0 tx=0).stop 2024-05-20T16:08:50.904 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-20T16:08:50.901+0000 7fc24174c700 1 -- 172.21.15.160:0/3456112310 shutdown_connections 2024-05-20T16:08:50.904 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-20T16:08:50.901+0000 7fc24174c700 1 --2- 172.21.15.160:0/3456112310 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc23c103b90 0x7fc23c10a440 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:08:50.904 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-20T16:08:50.901+0000 7fc24174c700 1 --2- 172.21.15.160:0/3456112310 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc23c1020c0 0x7fc23c1024d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:08:50.904 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-20T16:08:50.901+0000 7fc24174c700 1 --2- 172.21.15.160:0/3456112310 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc23c1031f0 0x7fc23c103600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:08:50.904 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-20T16:08:50.901+0000 7fc24174c700 1 -- 172.21.15.160:0/3456112310 >> 172.21.15.160:0/3456112310 conn(0x7fc23c0fd700 msgr2=0x7fc23c0ffbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:08:50.904 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-20T16:08:50.901+0000 7fc24174c700 1 -- 172.21.15.160:0/3456112310 shutdown_connections 2024-05-20T16:08:50.905 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-20T16:08:50.901+0000 7fc24174c700 1 -- 172.21.15.160:0/3456112310 wait complete. 2024-05-20T16:08:50.905 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-20T16:08:50.901+0000 7fc24174c700 1 Processor -- start 2024-05-20T16:08:50.905 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-20T16:08:50.902+0000 7fc24174c700 1 -- start start 2024-05-20T16:08:50.905 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-20T16:08:50.902+0000 7fc24174c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc23c1020c0 0x7fc23c06af50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:08:50.905 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-20T16:08:50.902+0000 7fc24174c700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc23c1031f0 0x7fc23c19e550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:08:50.905 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-20T16:08:50.902+0000 7fc24174c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc23c103b90 0x7fc23c1a0b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:08:50.905 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-20T16:08:50.902+0000 7fc24174c700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fc23c10dcb0 con 0x7fc23c103b90 2024-05-20T16:08:50.905 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-20T16:08:50.902+0000 7fc24174c700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fc23c10db30 con 0x7fc23c1020c0 2024-05-20T16:08:50.905 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-20T16:08:50.902+0000 7fc24174c700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fc23c10de30 con 0x7fc23c1031f0 2024-05-20T16:08:50.905 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-20T16:08:50.902+0000 7fc23affd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc23c1020c0 0x7fc23c06af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:08:50.905 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-20T16:08:50.902+0000 7fc23affd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc23c1020c0 0x7fc23c06af50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:37526/0 (socket says 172.21.15.160:37526) 2024-05-20T16:08:50.905 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-20T16:08:50.902+0000 7fc23affd700 1 -- 172.21.15.160:0/966551042 learned_addr learned my addr 172.21.15.160:0/966551042 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-20T16:08:50.905 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-20T16:08:50.902+0000 7fc23b7fe700 1 --2- 172.21.15.160:0/966551042 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc23c103b90 0x7fc23c1a0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:08:50.905 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-20T16:08:50.902+0000 7fc23affd700 1 -- 172.21.15.160:0/966551042 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc23c1031f0 msgr2=0x7fc23c19e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:08:50.906 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-20T16:08:50.902+0000 7fc23affd700 1 --2- 172.21.15.160:0/966551042 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc23c1031f0 0x7fc23c19e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:08:50.906 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-20T16:08:50.902+0000 7fc23affd700 1 -- 172.21.15.160:0/966551042 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc23c103b90 msgr2=0x7fc23c1a0b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:08:50.906 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-20T16:08:50.902+0000 7fc23affd700 1 --2- 172.21.15.160:0/966551042 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc23c103b90 0x7fc23c1a0b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:08:50.906 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-20T16:08:50.902+0000 7fc23affd700 1 -- 172.21.15.160:0/966551042 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc23c1a1350 con 0x7fc23c1020c0 2024-05-20T16:08:50.906 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-20T16:08:50.902+0000 7fc23b7fe700 1 --2- 172.21.15.160:0/966551042 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc23c103b90 0x7fc23c1a0b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T16:08:50.906 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-20T16:08:50.902+0000 7fc23affd700 1 --2- 172.21.15.160:0/966551042 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc23c1020c0 0x7fc23c06af50 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fc22c00d950 tx=0x7fc22c00dd10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:08:50.906 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-20T16:08:50.902+0000 7fc223fff700 1 -- 172.21.15.160:0/966551042 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc22c017070 con 0x7fc23c1020c0 2024-05-20T16:08:50.906 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-20T16:08:50.903+0000 7fc223fff700 1 -- 172.21.15.160:0/966551042 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc22c00de30 con 0x7fc23c1020c0 2024-05-20T16:08:50.906 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-20T16:08:50.903+0000 7fc24174c700 1 -- 172.21.15.160:0/966551042 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc23c1a1640 con 0x7fc23c1020c0 2024-05-20T16:08:50.906 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-20T16:08:50.903+0000 7fc223fff700 1 -- 172.21.15.160:0/966551042 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc22c00f3c0 con 0x7fc23c1020c0 2024-05-20T16:08:50.906 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-20T16:08:50.903+0000 7fc24174c700 1 -- 172.21.15.160:0/966551042 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc23c1a1b30 con 0x7fc23c1020c0 2024-05-20T16:08:50.907 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-20T16:08:50.903+0000 7fc24174c700 1 -- 172.21.15.160:0/966551042 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc23c050170 con 0x7fc23c1020c0 2024-05-20T16:08:50.907 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-20T16:08:50.905+0000 7fc223fff700 1 -- 172.21.15.160:0/966551042 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fc22c01a070 con 0x7fc23c1020c0 2024-05-20T16:08:50.907 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-20T16:08:50.905+0000 7fc223fff700 1 --2- 172.21.15.160:0/966551042 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fc2240a2c20 0x7fc2240a5130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:08:50.907 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-20T16:08:50.906+0000 7fc223fff700 1 -- 172.21.15.160:0/966551042 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 4965+0+0 (secure 0 0 0) 0x7fc22c0cdc90 con 0x7fc23c1020c0 2024-05-20T16:08:50.908 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-20T16:08:50.906+0000 7fc23a7fc700 1 --2- 172.21.15.160:0/966551042 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fc2240a2c20 0x7fc2240a5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:08:50.908 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-20T16:08:50.906+0000 7fc23a7fc700 1 --2- 172.21.15.160:0/966551042 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fc2240a2c20 0x7fc2240a5130 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fc230005950 tx=0x7fc2300058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:08:50.910 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-20T16:08:50.909+0000 7fc223fff700 1 -- 172.21.15.160:0/966551042 <== mon.2 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fc22c09b900 con 0x7fc23c1020c0 2024-05-20T16:08:51.114 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-20T16:08:51.114+0000 7fc24174c700 1 -- 172.21.15.160:0/966551042 --> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi160:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fc23c062a80 con 0x7fc2240a2c20 2024-05-20T16:08:51.312 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:08:51 smithi160 ceph-mon[36347]: Detected new or changed devices on smithi082 2024-05-20T16:08:51.312 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:08:51 smithi160 ceph-mon[36347]: Adjusting osd_memory_target on smithi082 to 5696M 2024-05-20T16:08:51.312 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:08:51 smithi160 ceph-mon[36347]: osdmap e48: 7 total, 7 up, 7 in 2024-05-20T16:08:51.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:08:51 smithi082 ceph-mon[32238]: Detected new or changed devices on smithi082 2024-05-20T16:08:51.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:08:51 smithi082 ceph-mon[32238]: Adjusting osd_memory_target on smithi082 to 5696M 2024-05-20T16:08:51.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:08:51 smithi082 ceph-mon[32238]: osdmap e48: 7 total, 7 up, 7 in 2024-05-20T16:08:51.386 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:08:51 smithi184 ceph-mon[36209]: Detected new or changed devices on smithi082 2024-05-20T16:08:51.386 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:08:51 smithi184 ceph-mon[36209]: Adjusting osd_memory_target on smithi082 to 5696M 2024-05-20T16:08:51.386 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:08:51 smithi184 ceph-mon[36209]: osdmap e48: 7 total, 7 up, 7 in 2024-05-20T16:08:52.312 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:08:52 smithi160 ceph-mon[36347]: from='client.24275 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi160:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-20T16:08:52.312 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:08:52 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-20T16:08:52.313 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:08:52 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-20T16:08:52.313 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:08:52 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:08:52.313 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:08:52 smithi160 ceph-mon[36347]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 192 MiB used, 626 GiB / 626 GiB avail 2024-05-20T16:08:52.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:08:52 smithi082 ceph-mon[32238]: from='client.24275 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi160:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-20T16:08:52.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:08:52 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-20T16:08:52.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:08:52 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-20T16:08:52.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:08:52 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:08:52.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:08:52 smithi082 ceph-mon[32238]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 192 MiB used, 626 GiB / 626 GiB avail 2024-05-20T16:08:52.386 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:08:52 smithi184 ceph-mon[36209]: from='client.24275 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi160:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-20T16:08:52.386 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:08:52 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-20T16:08:52.386 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:08:52 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-20T16:08:52.386 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:08:52 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:08:52.386 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:08:52 smithi184 ceph-mon[36209]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 192 MiB used, 626 GiB / 626 GiB avail 2024-05-20T16:08:54.562 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:08:54 smithi160 ceph-mon[36347]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 192 MiB used, 626 GiB / 626 GiB avail 2024-05-20T16:08:54.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:08:54 smithi160 ceph-mon[36347]: from='client.? 172.21.15.160:0/2282700912' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "25c54f43-18cb-41e4-83b4-2732b5a93889"}]: dispatch 2024-05-20T16:08:54.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:08:54 smithi160 ceph-mon[36347]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "25c54f43-18cb-41e4-83b4-2732b5a93889"}]: dispatch 2024-05-20T16:08:54.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:08:54 smithi160 ceph-mon[36347]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "25c54f43-18cb-41e4-83b4-2732b5a93889"}]': finished 2024-05-20T16:08:54.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:08:54 smithi160 ceph-mon[36347]: osdmap e49: 8 total, 7 up, 8 in 2024-05-20T16:08:54.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:08:54 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-20T16:08:54.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:08:54 smithi082 ceph-mon[32238]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 192 MiB used, 626 GiB / 626 GiB avail 2024-05-20T16:08:54.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:08:54 smithi082 ceph-mon[32238]: from='client.? 172.21.15.160:0/2282700912' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "25c54f43-18cb-41e4-83b4-2732b5a93889"}]: dispatch 2024-05-20T16:08:54.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:08:54 smithi082 ceph-mon[32238]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "25c54f43-18cb-41e4-83b4-2732b5a93889"}]: dispatch 2024-05-20T16:08:54.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:08:54 smithi082 ceph-mon[32238]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "25c54f43-18cb-41e4-83b4-2732b5a93889"}]': finished 2024-05-20T16:08:54.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:08:54 smithi082 ceph-mon[32238]: osdmap e49: 8 total, 7 up, 8 in 2024-05-20T16:08:54.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:08:54 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-20T16:08:54.638 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:08:54 smithi184 ceph-mon[36209]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 192 MiB used, 626 GiB / 626 GiB avail 2024-05-20T16:08:54.638 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:08:54 smithi184 ceph-mon[36209]: from='client.? 172.21.15.160:0/2282700912' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "25c54f43-18cb-41e4-83b4-2732b5a93889"}]: dispatch 2024-05-20T16:08:54.638 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:08:54 smithi184 ceph-mon[36209]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "25c54f43-18cb-41e4-83b4-2732b5a93889"}]: dispatch 2024-05-20T16:08:54.638 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:08:54 smithi184 ceph-mon[36209]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "25c54f43-18cb-41e4-83b4-2732b5a93889"}]': finished 2024-05-20T16:08:54.638 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:08:54 smithi184 ceph-mon[36209]: osdmap e49: 8 total, 7 up, 8 in 2024-05-20T16:08:54.638 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:08:54 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-20T16:08:55.562 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:08:55 smithi160 ceph-mon[36347]: from='client.? 172.21.15.160:0/3151751944' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-20T16:08:55.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:08:55 smithi082 ceph-mon[32238]: from='client.? 172.21.15.160:0/3151751944' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-20T16:08:55.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:08:55 smithi184 ceph-mon[36209]: from='client.? 172.21.15.160:0/3151751944' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-20T16:08:56.562 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:08:56 smithi160 ceph-mon[36347]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 192 MiB used, 626 GiB / 626 GiB avail 2024-05-20T16:08:56.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:08:56 smithi082 ceph-mon[32238]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 192 MiB used, 626 GiB / 626 GiB avail 2024-05-20T16:08:56.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:08:56 smithi184 ceph-mon[36209]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 192 MiB used, 626 GiB / 626 GiB avail 2024-05-20T16:08:58.312 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:08:58 smithi160 ceph-mon[36347]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 192 MiB used, 626 GiB / 626 GiB avail 2024-05-20T16:08:58.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:08:58 smithi082 ceph-mon[32238]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 192 MiB used, 626 GiB / 626 GiB avail 2024-05-20T16:08:58.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:08:58 smithi184 ceph-mon[36209]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 192 MiB used, 626 GiB / 626 GiB avail 2024-05-20T16:08:59.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:08:59 smithi082 ceph-mon[32238]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 192 MiB used, 626 GiB / 626 GiB avail 2024-05-20T16:08:59.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:08:59 smithi184 ceph-mon[36209]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 192 MiB used, 626 GiB / 626 GiB avail 2024-05-20T16:08:59.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:08:59 smithi160 ceph-mon[36347]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 192 MiB used, 626 GiB / 626 GiB avail 2024-05-20T16:09:02.421 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:09:02 smithi160 ceph-mon[36347]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 192 MiB used, 626 GiB / 626 GiB avail 2024-05-20T16:09:02.422 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:09:02 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-05-20T16:09:02.422 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:09:02 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:09:02.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:09:02 smithi082 ceph-mon[32238]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 192 MiB used, 626 GiB / 626 GiB avail 2024-05-20T16:09:02.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:09:02 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-05-20T16:09:02.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:09:02 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:09:02.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:09:02 smithi184 ceph-mon[36209]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 192 MiB used, 626 GiB / 626 GiB avail 2024-05-20T16:09:02.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:09:02 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-05-20T16:09:02.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:09:02 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:09:03.479 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:09:03 smithi160 ceph-mon[36347]: Deploying daemon osd.7 on smithi160 2024-05-20T16:09:03.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:09:03 smithi082 ceph-mon[32238]: Deploying daemon osd.7 on smithi160 2024-05-20T16:09:03.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:09:03 smithi184 ceph-mon[36209]: Deploying daemon osd.7 on smithi160 2024-05-20T16:09:04.483 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:09:04 smithi160 ceph-mon[36347]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 192 MiB used, 626 GiB / 626 GiB avail 2024-05-20T16:09:04.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:09:04 smithi082 ceph-mon[32238]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 192 MiB used, 626 GiB / 626 GiB avail 2024-05-20T16:09:04.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:09:04 smithi184 ceph-mon[36209]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 192 MiB used, 626 GiB / 626 GiB avail 2024-05-20T16:09:06.314 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:09:06 smithi160 ceph-mon[36347]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 192 MiB used, 626 GiB / 626 GiB avail 2024-05-20T16:09:06.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:09:06 smithi082 ceph-mon[32238]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 192 MiB used, 626 GiB / 626 GiB avail 2024-05-20T16:09:06.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:09:06 smithi184 ceph-mon[36209]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 192 MiB used, 626 GiB / 626 GiB avail 2024-05-20T16:09:07.520 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:09:07 smithi160 ceph-mon[36347]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 192 MiB used, 626 GiB / 626 GiB avail 2024-05-20T16:09:07.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:09:07 smithi082 ceph-mon[32238]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 192 MiB used, 626 GiB / 626 GiB avail 2024-05-20T16:09:07.635 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:09:07 smithi184 ceph-mon[36209]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 192 MiB used, 626 GiB / 626 GiB avail 2024-05-20T16:09:08.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:09:08 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:09:08.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:09:08 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:09:08.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:09:08 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T16:09:08.807 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:09:08 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:09:08.807 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:09:08 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:09:08.808 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:09:08 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T16:09:08.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:09:08 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:09:08.860 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:09:08 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:09:08.860 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:09:08 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T16:09:09.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:09:09 smithi160 ceph-mon[36347]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 192 MiB used, 626 GiB / 626 GiB avail 2024-05-20T16:09:09.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:09:09 smithi082 ceph-mon[32238]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 192 MiB used, 626 GiB / 626 GiB avail 2024-05-20T16:09:09.886 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:09:09 smithi184 ceph-mon[36209]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 192 MiB used, 626 GiB / 626 GiB avail 2024-05-20T16:09:11.534 INFO:teuthology.orchestra.run.smithi160.stdout:Created osd(s) 7 on host 'smithi160' 2024-05-20T16:09:11.534 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-20T16:09:11.533+0000 7fc223fff700 1 -- 172.21.15.160:0/966551042 <== mgr.14150 v2:172.21.15.82:6800/379530823 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fc23c062a80 con 0x7fc2240a2c20 2024-05-20T16:09:11.536 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-20T16:09:11.535+0000 7fc24174c700 1 -- 172.21.15.160:0/966551042 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fc2240a2c20 msgr2=0x7fc2240a5130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:09:11.536 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-20T16:09:11.535+0000 7fc24174c700 1 --2- 172.21.15.160:0/966551042 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fc2240a2c20 0x7fc2240a5130 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fc230005950 tx=0x7fc2300058e0 comp rx=0 tx=0).stop 2024-05-20T16:09:11.536 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-20T16:09:11.535+0000 7fc24174c700 1 -- 172.21.15.160:0/966551042 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc23c1020c0 msgr2=0x7fc23c06af50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:09:11.536 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-20T16:09:11.535+0000 7fc24174c700 1 --2- 172.21.15.160:0/966551042 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc23c1020c0 0x7fc23c06af50 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fc22c00d950 tx=0x7fc22c00dd10 comp rx=0 tx=0).stop 2024-05-20T16:09:11.536 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-20T16:09:11.535+0000 7fc24174c700 1 -- 172.21.15.160:0/966551042 shutdown_connections 2024-05-20T16:09:11.536 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-20T16:09:11.535+0000 7fc24174c700 1 --2- 172.21.15.160:0/966551042 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fc2240a2c20 0x7fc2240a5130 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:09:11.536 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-20T16:09:11.535+0000 7fc24174c700 1 --2- 172.21.15.160:0/966551042 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc23c1020c0 0x7fc23c06af50 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:09:11.536 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-20T16:09:11.535+0000 7fc24174c700 1 --2- 172.21.15.160:0/966551042 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc23c103b90 0x7fc23c1a0b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:09:11.537 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-20T16:09:11.535+0000 7fc24174c700 1 --2- 172.21.15.160:0/966551042 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc23c1031f0 0x7fc23c19e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:09:11.537 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-20T16:09:11.535+0000 7fc24174c700 1 -- 172.21.15.160:0/966551042 >> 172.21.15.160:0/966551042 conn(0x7fc23c0fd700 msgr2=0x7fc23c106420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:09:11.537 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-20T16:09:11.535+0000 7fc24174c700 1 -- 172.21.15.160:0/966551042 shutdown_connections 2024-05-20T16:09:11.537 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-20T16:09:11.535+0000 7fc24174c700 1 -- 172.21.15.160:0/966551042 wait complete. 2024-05-20T16:09:11.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:09:11 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:09:11.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:09:11 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:09:11.813 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:09:11 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:09:11.813 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:09:11 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T16:09:11.813 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:09:11 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:09:11.813 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:09:11 smithi160 ceph-mon[36347]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 192 MiB used, 626 GiB / 626 GiB avail 2024-05-20T16:09:11.813 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:09:11 smithi160 ceph-mon[36347]: from='osd.7 [v2:172.21.15.160:6816/2340683378,v1:172.21.15.160:6817/2340683378]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-20T16:09:11.813 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:09:11 smithi160 ceph-mon[36347]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-20T16:09:11.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:09:11 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:09:11.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:09:11 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:09:11.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:09:11 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:09:11.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:09:11 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T16:09:11.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:09:11 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:09:11.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:09:11 smithi082 ceph-mon[32238]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 192 MiB used, 626 GiB / 626 GiB avail 2024-05-20T16:09:11.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:09:11 smithi082 ceph-mon[32238]: from='osd.7 [v2:172.21.15.160:6816/2340683378,v1:172.21.15.160:6817/2340683378]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-20T16:09:11.860 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:09:11 smithi082 ceph-mon[32238]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-20T16:09:11.886 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:09:11 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:09:11.886 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:09:11 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:09:11.886 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:09:11 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:09:11.886 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:09:11 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T16:09:11.886 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:09:11 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:09:11.886 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:09:11 smithi184 ceph-mon[36209]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 192 MiB used, 626 GiB / 626 GiB avail 2024-05-20T16:09:11.886 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:09:11 smithi184 ceph-mon[36209]: from='osd.7 [v2:172.21.15.160:6816/2340683378,v1:172.21.15.160:6817/2340683378]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-20T16:09:11.886 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:09:11 smithi184 ceph-mon[36209]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-20T16:09:12.167 DEBUG:teuthology.orchestra.run.smithi160:osd.7> sudo journalctl -f -n 0 -u ceph-59d1600a-16c2-11ef-bc9a-c7b262605968@osd.7.service 2024-05-20T16:09:12.170 INFO:tasks.cephadm:Deploying osd.8 on smithi184 with /dev/vg_nvme/lv_2... 2024-05-20T16:09:12.170 DEBUG:teuthology.orchestra.run.smithi184:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 59d1600a-16c2-11ef-bc9a-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-05-20T16:09:12.415 INFO:teuthology.orchestra.run.smithi184.stderr:Inferring config /var/lib/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/mon.c/config 2024-05-20T16:09:12.727 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:09:12 smithi160 ceph-mon[36347]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-05-20T16:09:12.727 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:09:12 smithi160 ceph-mon[36347]: osdmap e50: 8 total, 7 up, 8 in 2024-05-20T16:09:12.727 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:09:12 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-20T16:09:12.727 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:09:12 smithi160 ceph-mon[36347]: from='osd.7 [v2:172.21.15.160:6816/2340683378,v1:172.21.15.160:6817/2340683378]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]: dispatch 2024-05-20T16:09:12.727 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:09:12 smithi160 ceph-mon[36347]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]: dispatch 2024-05-20T16:09:12.727 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:09:12 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:09:12.727 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:09:12 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:09:12.727 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:09:12 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T16:09:12.752 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:09:12 smithi184 ceph-mon[36209]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-05-20T16:09:12.752 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:09:12 smithi184 ceph-mon[36209]: osdmap e50: 8 total, 7 up, 8 in 2024-05-20T16:09:12.752 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:09:12 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-20T16:09:12.752 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:09:12 smithi184 ceph-mon[36209]: from='osd.7 [v2:172.21.15.160:6816/2340683378,v1:172.21.15.160:6817/2340683378]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]: dispatch 2024-05-20T16:09:12.752 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:09:12 smithi184 ceph-mon[36209]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]: dispatch 2024-05-20T16:09:12.753 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:09:12 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:09:12.753 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:09:12 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:09:12.753 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:09:12 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T16:09:12.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:09:12 smithi082 ceph-mon[32238]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-05-20T16:09:12.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:09:12 smithi082 ceph-mon[32238]: osdmap e50: 8 total, 7 up, 8 in 2024-05-20T16:09:12.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:09:12 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-20T16:09:12.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:09:12 smithi082 ceph-mon[32238]: from='osd.7 [v2:172.21.15.160:6816/2340683378,v1:172.21.15.160:6817/2340683378]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]: dispatch 2024-05-20T16:09:12.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:09:12 smithi082 ceph-mon[32238]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]: dispatch 2024-05-20T16:09:12.860 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:09:12 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:09:12.860 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:09:12 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:09:12.860 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:09:12 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T16:09:13.063 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 20 16:09:12 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-7[54382]: 2024-05-20T16:09:12.920+0000 7f68a91a0700 -1 osd.7 0 waiting for initial osdmap 2024-05-20T16:09:13.063 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 20 16:09:12 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-7[54382]: 2024-05-20T16:09:12.929+0000 7f68a0f4d700 -1 osd.7 51 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-20T16:09:13.726 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:09:13 smithi160 ceph-mon[36347]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]': finished 2024-05-20T16:09:13.726 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:09:13 smithi160 ceph-mon[36347]: osdmap e51: 8 total, 7 up, 8 in 2024-05-20T16:09:13.726 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:09:13 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-20T16:09:13.726 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:09:13 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-20T16:09:13.726 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:09:13 smithi160 ceph-mon[36347]: from='osd.7 ' entity='osd.7' 2024-05-20T16:09:13.727 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:09:13 smithi160 ceph-mon[36347]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 192 MiB used, 626 GiB / 626 GiB avail 2024-05-20T16:09:13.816 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:09:13 smithi184 ceph-mon[36209]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]': finished 2024-05-20T16:09:13.817 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:09:13 smithi184 ceph-mon[36209]: osdmap e51: 8 total, 7 up, 8 in 2024-05-20T16:09:13.817 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:09:13 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-20T16:09:13.817 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:09:13 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-20T16:09:13.817 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:09:13 smithi184 ceph-mon[36209]: from='osd.7 ' entity='osd.7' 2024-05-20T16:09:13.817 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:09:13 smithi184 ceph-mon[36209]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 192 MiB used, 626 GiB / 626 GiB avail 2024-05-20T16:09:13.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:09:13 smithi082 ceph-mon[32238]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]': finished 2024-05-20T16:09:13.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:09:13 smithi082 ceph-mon[32238]: osdmap e51: 8 total, 7 up, 8 in 2024-05-20T16:09:13.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:09:13 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-20T16:09:13.860 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:09:13 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-20T16:09:13.860 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:09:13 smithi082 ceph-mon[32238]: from='osd.7 ' entity='osd.7' 2024-05-20T16:09:13.860 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:09:13 smithi082 ceph-mon[32238]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 192 MiB used, 626 GiB / 626 GiB avail 2024-05-20T16:09:14.388 INFO:teuthology.orchestra.run.smithi184.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-05-20T16:09:14.388 INFO:teuthology.orchestra.run.smithi184.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-05-20T16:09:14.388 INFO:teuthology.orchestra.run.smithi184.stderr: stderr: 10+0 records in 2024-05-20T16:09:14.388 INFO:teuthology.orchestra.run.smithi184.stderr:10+0 records out 2024-05-20T16:09:14.388 INFO:teuthology.orchestra.run.smithi184.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0116249 s, 902 MB/s 2024-05-20T16:09:14.388 INFO:teuthology.orchestra.run.smithi184.stderr:--> Zapping successful for: 2024-05-20T16:09:14.723 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:09:14 smithi184 ceph-mon[36209]: purged_snaps scrub starts 2024-05-20T16:09:14.723 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:09:14 smithi184 ceph-mon[36209]: purged_snaps scrub ok 2024-05-20T16:09:14.723 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:09:14 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-20T16:09:14.723 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:09:14 smithi184 ceph-mon[36209]: osd.7 [v2:172.21.15.160:6816/2340683378,v1:172.21.15.160:6817/2340683378] boot 2024-05-20T16:09:14.723 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:09:14 smithi184 ceph-mon[36209]: osdmap e52: 8 total, 8 up, 8 in 2024-05-20T16:09:14.723 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:09:14 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-20T16:09:14.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:09:14 smithi160 ceph-mon[36347]: purged_snaps scrub starts 2024-05-20T16:09:14.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:09:14 smithi160 ceph-mon[36347]: purged_snaps scrub ok 2024-05-20T16:09:14.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:09:14 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-20T16:09:14.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:09:14 smithi160 ceph-mon[36347]: osd.7 [v2:172.21.15.160:6816/2340683378,v1:172.21.15.160:6817/2340683378] boot 2024-05-20T16:09:14.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:09:14 smithi160 ceph-mon[36347]: osdmap e52: 8 total, 8 up, 8 in 2024-05-20T16:09:14.813 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:09:14 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-20T16:09:14.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:09:14 smithi082 ceph-mon[32238]: purged_snaps scrub starts 2024-05-20T16:09:14.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:09:14 smithi082 ceph-mon[32238]: purged_snaps scrub ok 2024-05-20T16:09:14.860 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:09:14 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-20T16:09:14.860 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:09:14 smithi082 ceph-mon[32238]: osd.7 [v2:172.21.15.160:6816/2340683378,v1:172.21.15.160:6817/2340683378] boot 2024-05-20T16:09:14.860 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:09:14 smithi082 ceph-mon[32238]: osdmap e52: 8 total, 8 up, 8 in 2024-05-20T16:09:14.860 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:09:14 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-20T16:09:15.067 DEBUG:teuthology.orchestra.run.smithi184:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 59d1600a-16c2-11ef-bc9a-c7b262605968 -- ceph orch daemon add osd smithi184:vg_nvme/lv_2 2024-05-20T16:09:15.314 INFO:teuthology.orchestra.run.smithi184.stderr:Inferring config /var/lib/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/mon.c/config 2024-05-20T16:09:15.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:09:15 smithi160 ceph-mon[36347]: osdmap e53: 8 total, 8 up, 8 in 2024-05-20T16:09:15.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:09:15 smithi160 ceph-mon[36347]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 223 MiB used, 715 GiB / 715 GiB avail 2024-05-20T16:09:15.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:09:15 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:09:15.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:09:15 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:09:15.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:09:15 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-20T16:09:15.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:09:15 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-20T16:09:15.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:09:15 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-20T16:09:15.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:09:15 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:09:15.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:09:15 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:09:15.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:09:15 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T16:09:15.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:09:15 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:09:15.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:09:15 smithi184 ceph-mon[36209]: osdmap e53: 8 total, 8 up, 8 in 2024-05-20T16:09:15.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:09:15 smithi184 ceph-mon[36209]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 223 MiB used, 715 GiB / 715 GiB avail 2024-05-20T16:09:15.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:09:15 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:09:15.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:09:15 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:09:15.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:09:15 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-20T16:09:15.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:09:15 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-20T16:09:15.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:09:15 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-20T16:09:15.637 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:09:15 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:09:15.637 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:09:15 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:09:15.637 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:09:15 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T16:09:15.637 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:09:15 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:09:15.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:09:15 smithi082 ceph-mon[32238]: osdmap e53: 8 total, 8 up, 8 in 2024-05-20T16:09:15.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:09:15 smithi082 ceph-mon[32238]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 223 MiB used, 715 GiB / 715 GiB avail 2024-05-20T16:09:15.860 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:09:15 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:09:15.860 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:09:15 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:09:15.860 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:09:15 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-20T16:09:15.860 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:09:15 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-20T16:09:15.860 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:09:15 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-20T16:09:15.860 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:09:15 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:09:15.860 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:09:15 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:09:15.860 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:09:15 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T16:09:15.860 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:09:15 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:09:16.690 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:09:16 smithi184 ceph-mon[36209]: Detected new or changed devices on smithi160 2024-05-20T16:09:16.691 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:09:16 smithi184 ceph-mon[36209]: Adjusting osd_memory_target on smithi160 to 5696M 2024-05-20T16:09:16.691 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:09:16 smithi184 ceph-mon[36209]: osdmap e54: 8 total, 8 up, 8 in 2024-05-20T16:09:16.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:09:16 smithi160 ceph-mon[36347]: Detected new or changed devices on smithi160 2024-05-20T16:09:16.813 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:09:16 smithi160 ceph-mon[36347]: Adjusting osd_memory_target on smithi160 to 5696M 2024-05-20T16:09:16.813 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:09:16 smithi160 ceph-mon[36347]: osdmap e54: 8 total, 8 up, 8 in 2024-05-20T16:09:16.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:09:16 smithi082 ceph-mon[32238]: Detected new or changed devices on smithi160 2024-05-20T16:09:16.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:09:16 smithi082 ceph-mon[32238]: Adjusting osd_memory_target on smithi160 to 5696M 2024-05-20T16:09:16.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:09:16 smithi082 ceph-mon[32238]: osdmap e54: 8 total, 8 up, 8 in 2024-05-20T16:09:16.944 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:09:16.943+0000 7fa7f79b9700 1 -- 172.21.15.184:0/1266134870 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa7f01020c0 msgr2=0x7fa7f01024d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:09:16.944 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:09:16.943+0000 7fa7f79b9700 1 --2- 172.21.15.184:0/1266134870 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa7f01020c0 0x7fa7f01024d0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fa7ec009990 tx=0x7fa7ec009ca0 comp rx=0 tx=0).stop 2024-05-20T16:09:16.944 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:09:16.943+0000 7fa7f79b9700 1 -- 172.21.15.184:0/1266134870 shutdown_connections 2024-05-20T16:09:16.944 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:09:16.943+0000 7fa7f79b9700 1 --2- 172.21.15.184:0/1266134870 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa7f0103b90 0x7fa7f010a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:09:16.944 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:09:16.943+0000 7fa7f79b9700 1 --2- 172.21.15.184:0/1266134870 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa7f01020c0 0x7fa7f01024d0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:09:16.944 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:09:16.943+0000 7fa7f79b9700 1 --2- 172.21.15.184:0/1266134870 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa7f01031f0 0x7fa7f0103600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:09:16.945 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:09:16.943+0000 7fa7f79b9700 1 -- 172.21.15.184:0/1266134870 >> 172.21.15.184:0/1266134870 conn(0x7fa7f00fd700 msgr2=0x7fa7f00ffbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:09:16.945 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:09:16.944+0000 7fa7f79b9700 1 -- 172.21.15.184:0/1266134870 shutdown_connections 2024-05-20T16:09:16.945 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:09:16.944+0000 7fa7f79b9700 1 -- 172.21.15.184:0/1266134870 wait complete. 2024-05-20T16:09:16.946 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:09:16.944+0000 7fa7f79b9700 1 Processor -- start 2024-05-20T16:09:16.946 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:09:16.944+0000 7fa7f79b9700 1 -- start start 2024-05-20T16:09:16.947 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:09:16.944+0000 7fa7f79b9700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa7f01020c0 0x7fa7f0105e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:09:16.947 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:09:16.944+0000 7fa7f79b9700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa7f01031f0 0x7fa7f0104530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:09:16.947 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:09:16.944+0000 7fa7f79b9700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa7f0103b90 0x7fa7f0104a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:09:16.947 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:09:16.944+0000 7fa7f79b9700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fa7f010dc60 con 0x7fa7f01031f0 2024-05-20T16:09:16.947 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:09:16.944+0000 7fa7f79b9700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fa7f010dae0 con 0x7fa7f01020c0 2024-05-20T16:09:16.947 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:09:16.944+0000 7fa7f79b9700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa7f010dde0 con 0x7fa7f0103b90 2024-05-20T16:09:16.947 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:09:16.945+0000 7fa7f5f56700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa7f0103b90 0x7fa7f0104a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:09:16.947 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:09:16.945+0000 7fa7f5f56700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa7f0103b90 0x7fa7f0104a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.184:55418/0 (socket says 172.21.15.184:55418) 2024-05-20T16:09:16.947 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:09:16.945+0000 7fa7f5f56700 1 -- 172.21.15.184:0/1359595923 learned_addr learned my addr 172.21.15.184:0/1359595923 (peer_addr_for_me v2:172.21.15.184:0/0) 2024-05-20T16:09:16.947 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:09:16.945+0000 7fa7f5755700 1 --2- 172.21.15.184:0/1359595923 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa7f01020c0 0x7fa7f0105e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:09:16.947 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:09:16.945+0000 7fa7f4f54700 1 --2- 172.21.15.184:0/1359595923 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa7f01031f0 0x7fa7f0104530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:09:16.947 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:09:16.945+0000 7fa7f5f56700 1 -- 172.21.15.184:0/1359595923 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa7f01020c0 msgr2=0x7fa7f0105e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:09:16.948 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:09:16.945+0000 7fa7f5f56700 1 --2- 172.21.15.184:0/1359595923 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa7f01020c0 0x7fa7f0105e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:09:16.948 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:09:16.945+0000 7fa7f5f56700 1 -- 172.21.15.184:0/1359595923 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa7f01031f0 msgr2=0x7fa7f0104530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:09:16.948 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:09:16.945+0000 7fa7f5f56700 1 --2- 172.21.15.184:0/1359595923 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa7f01031f0 0x7fa7f0104530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:09:16.948 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:09:16.945+0000 7fa7f5f56700 1 -- 172.21.15.184:0/1359595923 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7f0105180 con 0x7fa7f0103b90 2024-05-20T16:09:16.948 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:09:16.945+0000 7fa7f5f56700 1 --2- 172.21.15.184:0/1359595923 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa7f0103b90 0x7fa7f0104a70 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fa7e800ada0 tx=0x7fa7e800c330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:09:16.948 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:09:16.945+0000 7fa7e67fc700 1 -- 172.21.15.184:0/1359595923 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7e8017020 con 0x7fa7f0103b90 2024-05-20T16:09:16.948 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:09:16.945+0000 7fa7e67fc700 1 -- 172.21.15.184:0/1359595923 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa7e800cdb0 con 0x7fa7f0103b90 2024-05-20T16:09:16.948 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:09:16.945+0000 7fa7f79b9700 1 -- 172.21.15.184:0/1359595923 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7f01a39a0 con 0x7fa7f0103b90 2024-05-20T16:09:16.948 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:09:16.945+0000 7fa7e67fc700 1 -- 172.21.15.184:0/1359595923 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7e80053e0 con 0x7fa7f0103b90 2024-05-20T16:09:16.948 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:09:16.946+0000 7fa7f79b9700 1 -- 172.21.15.184:0/1359595923 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7f01a3e00 con 0x7fa7f0103b90 2024-05-20T16:09:16.949 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:09:16.947+0000 7fa7f79b9700 1 -- 172.21.15.184:0/1359595923 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7f0050170 con 0x7fa7f0103b90 2024-05-20T16:09:16.949 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:09:16.948+0000 7fa7e67fc700 1 -- 172.21.15.184:0/1359595923 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fa7e800c8d0 con 0x7fa7f0103b90 2024-05-20T16:09:16.949 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:09:16.948+0000 7fa7e67fc700 1 --2- 172.21.15.184:0/1359595923 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fa7dc0a2c20 0x7fa7dc0a5130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:09:16.949 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:09:16.949+0000 7fa7e67fc700 1 -- 172.21.15.184:0/1359595923 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 5397+0+0 (secure 0 0 0) 0x7fa7e80cdbe0 con 0x7fa7f0103b90 2024-05-20T16:09:16.949 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:09:16.949+0000 7fa7f5755700 1 --2- 172.21.15.184:0/1359595923 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fa7dc0a2c20 0x7fa7dc0a5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:09:16.950 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:09:16.949+0000 7fa7f5755700 1 --2- 172.21.15.184:0/1359595923 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fa7dc0a2c20 0x7fa7dc0a5130 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fa7ec006010 tx=0x7fa7ec01a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:09:16.953 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:09:16.953+0000 7fa7e67fc700 1 -- 172.21.15.184:0/1359595923 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fa7e8021020 con 0x7fa7f0103b90 2024-05-20T16:09:17.158 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:09:17.158+0000 7fa7f79b9700 1 -- 172.21.15.184:0/1359595923 --> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi184:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fa7f00ff730 con 0x7fa7dc0a2c20 2024-05-20T16:09:17.771 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:09:17 smithi184 ceph-mon[36209]: osdmap e55: 8 total, 8 up, 8 in 2024-05-20T16:09:17.771 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:09:17 smithi184 ceph-mon[36209]: from='client.24295 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi184:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-20T16:09:17.772 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:09:17 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-20T16:09:17.772 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:09:17 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-20T16:09:17.772 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:09:17 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:09:17.772 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:09:17 smithi184 ceph-mon[36209]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 223 MiB used, 715 GiB / 715 GiB avail 2024-05-20T16:09:17.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:09:17 smithi160 ceph-mon[36347]: osdmap e55: 8 total, 8 up, 8 in 2024-05-20T16:09:17.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:09:17 smithi160 ceph-mon[36347]: from='client.24295 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi184:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-20T16:09:17.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:09:17 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-20T16:09:17.813 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:09:17 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-20T16:09:17.813 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:09:17 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:09:17.813 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:09:17 smithi160 ceph-mon[36347]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 223 MiB used, 715 GiB / 715 GiB avail 2024-05-20T16:09:17.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:09:17 smithi082 ceph-mon[32238]: osdmap e55: 8 total, 8 up, 8 in 2024-05-20T16:09:17.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:09:17 smithi082 ceph-mon[32238]: from='client.24295 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi184:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-20T16:09:17.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:09:17 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-20T16:09:17.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:09:17 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-20T16:09:17.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:09:17 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:09:17.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:09:17 smithi082 ceph-mon[32238]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 223 MiB used, 715 GiB / 715 GiB avail 2024-05-20T16:09:18.793 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:09:18 smithi184 ceph-mon[36209]: osdmap e56: 8 total, 8 up, 8 in 2024-05-20T16:09:18.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:09:18 smithi160 ceph-mon[36347]: osdmap e56: 8 total, 8 up, 8 in 2024-05-20T16:09:18.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:09:18 smithi082 ceph-mon[32238]: osdmap e56: 8 total, 8 up, 8 in 2024-05-20T16:09:19.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:09:19 smithi184 ceph-mon[36209]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 223 MiB used, 715 GiB / 715 GiB avail 2024-05-20T16:09:19.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:09:19 smithi160 ceph-mon[36347]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 223 MiB used, 715 GiB / 715 GiB avail 2024-05-20T16:09:19.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:09:19 smithi082 ceph-mon[32238]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 223 MiB used, 715 GiB / 715 GiB avail 2024-05-20T16:09:20.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:09:20 smithi160 ceph-mon[36347]: from='client.? 172.21.15.184:0/831438080' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1f8b478d-287f-45f5-8526-d1a58c5bc07c"}]: dispatch 2024-05-20T16:09:20.813 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:09:20 smithi160 ceph-mon[36347]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1f8b478d-287f-45f5-8526-d1a58c5bc07c"}]: dispatch 2024-05-20T16:09:20.813 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:09:20 smithi160 ceph-mon[36347]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1f8b478d-287f-45f5-8526-d1a58c5bc07c"}]': finished 2024-05-20T16:09:20.813 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:09:20 smithi160 ceph-mon[36347]: osdmap e57: 9 total, 8 up, 9 in 2024-05-20T16:09:20.813 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:09:20 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-20T16:09:20.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:09:20 smithi082 ceph-mon[32238]: from='client.? 172.21.15.184:0/831438080' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1f8b478d-287f-45f5-8526-d1a58c5bc07c"}]: dispatch 2024-05-20T16:09:20.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:09:20 smithi082 ceph-mon[32238]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1f8b478d-287f-45f5-8526-d1a58c5bc07c"}]: dispatch 2024-05-20T16:09:20.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:09:20 smithi082 ceph-mon[32238]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1f8b478d-287f-45f5-8526-d1a58c5bc07c"}]': finished 2024-05-20T16:09:20.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:09:20 smithi082 ceph-mon[32238]: osdmap e57: 9 total, 8 up, 9 in 2024-05-20T16:09:20.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:09:20 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-20T16:09:20.886 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:09:20 smithi184 ceph-mon[36209]: from='client.? 172.21.15.184:0/831438080' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1f8b478d-287f-45f5-8526-d1a58c5bc07c"}]: dispatch 2024-05-20T16:09:20.886 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:09:20 smithi184 ceph-mon[36209]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1f8b478d-287f-45f5-8526-d1a58c5bc07c"}]: dispatch 2024-05-20T16:09:20.886 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:09:20 smithi184 ceph-mon[36209]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1f8b478d-287f-45f5-8526-d1a58c5bc07c"}]': finished 2024-05-20T16:09:20.886 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:09:20 smithi184 ceph-mon[36209]: osdmap e57: 9 total, 8 up, 9 in 2024-05-20T16:09:20.886 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:09:20 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-20T16:09:21.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:09:21 smithi160 ceph-mon[36347]: from='client.? 172.21.15.184:0/3370703349' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-20T16:09:21.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:09:21 smithi160 ceph-mon[36347]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-05-20T16:09:21.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:09:21 smithi082 ceph-mon[32238]: from='client.? 172.21.15.184:0/3370703349' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-20T16:09:21.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:09:21 smithi082 ceph-mon[32238]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-05-20T16:09:21.886 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:09:21 smithi184 ceph-mon[36209]: from='client.? 172.21.15.184:0/3370703349' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-20T16:09:21.886 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:09:21 smithi184 ceph-mon[36209]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-05-20T16:09:24.386 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:09:24 smithi184 ceph-mon[36209]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-05-20T16:09:24.562 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:09:24 smithi160 ceph-mon[36347]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-05-20T16:09:24.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:09:24 smithi082 ceph-mon[32238]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-05-20T16:09:26.562 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:09:26 smithi160 ceph-mon[36347]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-05-20T16:09:26.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:09:26 smithi082 ceph-mon[32238]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-05-20T16:09:26.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:09:26 smithi184 ceph-mon[36209]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-05-20T16:09:28.287 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:09:28 smithi184 ceph-mon[36209]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-05-20T16:09:28.287 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:09:28 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-05-20T16:09:28.288 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:09:28 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:09:28.562 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:09:28 smithi160 ceph-mon[36347]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-05-20T16:09:28.562 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:09:28 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-05-20T16:09:28.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:09:28 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:09:28.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:09:28 smithi082 ceph-mon[32238]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-05-20T16:09:28.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:09:28 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-05-20T16:09:28.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:09:28 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:09:29.383 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:09:29 smithi184 ceph-mon[36209]: Deploying daemon osd.8 on smithi184 2024-05-20T16:09:29.562 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:09:29 smithi160 ceph-mon[36347]: Deploying daemon osd.8 on smithi184 2024-05-20T16:09:29.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:09:29 smithi082 ceph-mon[32238]: Deploying daemon osd.8 on smithi184 2024-05-20T16:09:30.386 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:09:30 smithi184 ceph-mon[36209]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-05-20T16:09:30.562 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:09:30 smithi160 ceph-mon[36347]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-05-20T16:09:30.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:09:30 smithi082 ceph-mon[32238]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-05-20T16:09:31.396 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:09:31 smithi184 ceph-mon[36209]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-05-20T16:09:31.562 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:09:31 smithi160 ceph-mon[36347]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-05-20T16:09:31.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:09:31 smithi082 ceph-mon[32238]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-05-20T16:09:34.562 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:09:34 smithi160 ceph-mon[36347]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-05-20T16:09:34.562 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:09:34 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:09:34.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:09:34 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:09:34.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:09:34 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T16:09:34.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:09:34 smithi082 ceph-mon[32238]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-05-20T16:09:34.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:09:34 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:09:34.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:09:34 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:09:34.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:09:34 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T16:09:34.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:09:34 smithi184 ceph-mon[36209]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-05-20T16:09:34.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:09:34 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:09:34.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:09:34 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:09:34.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:09:34 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T16:09:36.562 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:09:36 smithi160 ceph-mon[36347]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-05-20T16:09:36.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:09:36 smithi082 ceph-mon[32238]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-05-20T16:09:36.615 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:09:36 smithi184 ceph-mon[36209]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-05-20T16:09:37.329 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:09:37.327+0000 7fa7e67fc700 1 -- 172.21.15.184:0/1359595923 <== mgr.14150 v2:172.21.15.82:6800/379530823 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fa7f00ff730 con 0x7fa7dc0a2c20 2024-05-20T16:09:37.329 INFO:teuthology.orchestra.run.smithi184.stdout:Created osd(s) 8 on host 'smithi184' 2024-05-20T16:09:37.330 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:09:37.330+0000 7fa7f79b9700 1 -- 172.21.15.184:0/1359595923 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fa7dc0a2c20 msgr2=0x7fa7dc0a5130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:09:37.330 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:09:37.330+0000 7fa7f79b9700 1 --2- 172.21.15.184:0/1359595923 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fa7dc0a2c20 0x7fa7dc0a5130 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fa7ec006010 tx=0x7fa7ec01a040 comp rx=0 tx=0).stop 2024-05-20T16:09:37.330 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:09:37.330+0000 7fa7f79b9700 1 -- 172.21.15.184:0/1359595923 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa7f0103b90 msgr2=0x7fa7f0104a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:09:37.330 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:09:37.330+0000 7fa7f79b9700 1 --2- 172.21.15.184:0/1359595923 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa7f0103b90 0x7fa7f0104a70 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fa7e800ada0 tx=0x7fa7e800c330 comp rx=0 tx=0).stop 2024-05-20T16:09:37.330 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:09:37.330+0000 7fa7f79b9700 1 -- 172.21.15.184:0/1359595923 shutdown_connections 2024-05-20T16:09:37.331 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:09:37.330+0000 7fa7f79b9700 1 --2- 172.21.15.184:0/1359595923 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fa7dc0a2c20 0x7fa7dc0a5130 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:09:37.331 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:09:37.330+0000 7fa7f79b9700 1 --2- 172.21.15.184:0/1359595923 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa7f01020c0 0x7fa7f0105e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:09:37.331 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:09:37.330+0000 7fa7f79b9700 1 --2- 172.21.15.184:0/1359595923 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa7f0103b90 0x7fa7f0104a70 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:09:37.331 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:09:37.330+0000 7fa7f79b9700 1 --2- 172.21.15.184:0/1359595923 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa7f01031f0 0x7fa7f0104530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:09:37.331 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:09:37.330+0000 7fa7f79b9700 1 -- 172.21.15.184:0/1359595923 >> 172.21.15.184:0/1359595923 conn(0x7fa7f00fd700 msgr2=0x7fa7f00ff020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:09:37.331 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:09:37.330+0000 7fa7f79b9700 1 -- 172.21.15.184:0/1359595923 shutdown_connections 2024-05-20T16:09:37.331 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:09:37.330+0000 7fa7f79b9700 1 -- 172.21.15.184:0/1359595923 wait complete. 2024-05-20T16:09:37.562 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:09:37 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:09:37.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:09:37 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:09:37.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:09:37 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:09:37.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:09:37 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T16:09:37.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:09:37 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:09:37.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:09:37 smithi160 ceph-mon[36347]: from='osd.8 [v2:172.21.15.184:6816/3831576372,v1:172.21.15.184:6817/3831576372]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-20T16:09:37.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:09:37 smithi160 ceph-mon[36347]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-20T16:09:37.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:09:37 smithi160 ceph-mon[36347]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-05-20T16:09:37.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:09:37 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:09:37.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:09:37 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:09:37.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:09:37 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T16:09:37.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:09:37 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:09:37.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:09:37 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:09:37.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:09:37 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:09:37.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:09:37 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T16:09:37.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:09:37 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:09:37.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:09:37 smithi082 ceph-mon[32238]: from='osd.8 [v2:172.21.15.184:6816/3831576372,v1:172.21.15.184:6817/3831576372]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-20T16:09:37.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:09:37 smithi082 ceph-mon[32238]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-20T16:09:37.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:09:37 smithi082 ceph-mon[32238]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-05-20T16:09:37.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:09:37 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:09:37.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:09:37 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:09:37.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:09:37 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T16:09:37.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:09:37 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:09:37.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:09:37 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:09:37.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:09:37 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:09:37.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:09:37 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T16:09:37.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:09:37 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:09:37.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:09:37 smithi184 ceph-mon[36209]: from='osd.8 [v2:172.21.15.184:6816/3831576372,v1:172.21.15.184:6817/3831576372]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-20T16:09:37.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:09:37 smithi184 ceph-mon[36209]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-20T16:09:37.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:09:37 smithi184 ceph-mon[36209]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-05-20T16:09:37.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:09:37 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:09:37.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:09:37 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:09:37.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:09:37 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T16:09:37.928 DEBUG:teuthology.orchestra.run.smithi184:osd.8> sudo journalctl -f -n 0 -u ceph-59d1600a-16c2-11ef-bc9a-c7b262605968@osd.8.service 2024-05-20T16:09:37.930 INFO:tasks.cephadm:Deploying osd.9 on smithi082 with /dev/vg_nvme/lv_1... 2024-05-20T16:09:37.930 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 59d1600a-16c2-11ef-bc9a-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-05-20T16:09:38.181 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T16:09:38.556 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:09:38 smithi082 ceph-mon[32238]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-05-20T16:09:38.556 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:09:38 smithi082 ceph-mon[32238]: from='osd.8 [v2:172.21.15.184:6816/3831576372,v1:172.21.15.184:6817/3831576372]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]: dispatch 2024-05-20T16:09:38.556 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:09:38 smithi082 ceph-mon[32238]: osdmap e58: 9 total, 8 up, 9 in 2024-05-20T16:09:38.556 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:09:38 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-20T16:09:38.556 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:09:38 smithi082 ceph-mon[32238]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]: dispatch 2024-05-20T16:09:38.562 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:09:38 smithi184 ceph-mon[36209]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-05-20T16:09:38.562 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:09:38 smithi184 ceph-mon[36209]: from='osd.8 [v2:172.21.15.184:6816/3831576372,v1:172.21.15.184:6817/3831576372]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]: dispatch 2024-05-20T16:09:38.562 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:09:38 smithi184 ceph-mon[36209]: osdmap e58: 9 total, 8 up, 9 in 2024-05-20T16:09:38.563 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:09:38 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-20T16:09:38.563 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:09:38 smithi184 ceph-mon[36209]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]: dispatch 2024-05-20T16:09:38.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:09:38 smithi160 ceph-mon[36347]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-05-20T16:09:38.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:09:38 smithi160 ceph-mon[36347]: from='osd.8 [v2:172.21.15.184:6816/3831576372,v1:172.21.15.184:6817/3831576372]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]: dispatch 2024-05-20T16:09:38.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:09:38 smithi160 ceph-mon[36347]: osdmap e58: 9 total, 8 up, 9 in 2024-05-20T16:09:38.813 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:09:38 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-20T16:09:38.813 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:09:38 smithi160 ceph-mon[36347]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]: dispatch 2024-05-20T16:09:38.886 INFO:journalctl@ceph.osd.8.smithi184.stdout:May 20 16:09:38 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-8[55074]: 2024-05-20T16:09:38.800+0000 7f4a3efbb700 -1 osd.8 0 waiting for initial osdmap 2024-05-20T16:09:38.886 INFO:journalctl@ceph.osd.8.smithi184.stdout:May 20 16:09:38 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-8[55074]: 2024-05-20T16:09:38.810+0000 7f4a36d68700 -1 osd.8 59 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-20T16:09:39.386 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:09:39 smithi184 ceph-mon[36209]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]': finished 2024-05-20T16:09:39.386 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:09:39 smithi184 ceph-mon[36209]: osdmap e59: 9 total, 8 up, 9 in 2024-05-20T16:09:39.386 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:09:39 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-20T16:09:39.386 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:09:39 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-20T16:09:39.386 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:09:39 smithi184 ceph-mon[36209]: from='osd.8 ' entity='osd.8' 2024-05-20T16:09:39.387 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:09:39 smithi184 ceph-mon[36209]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-05-20T16:09:39.595 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:09:39 smithi082 ceph-mon[32238]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]': finished 2024-05-20T16:09:39.595 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:09:39 smithi082 ceph-mon[32238]: osdmap e59: 9 total, 8 up, 9 in 2024-05-20T16:09:39.596 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:09:39 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-20T16:09:39.596 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:09:39 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-20T16:09:39.596 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:09:39 smithi082 ceph-mon[32238]: from='osd.8 ' entity='osd.8' 2024-05-20T16:09:39.596 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:09:39 smithi082 ceph-mon[32238]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-05-20T16:09:39.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:09:39 smithi160 ceph-mon[36347]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]': finished 2024-05-20T16:09:39.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:09:39 smithi160 ceph-mon[36347]: osdmap e59: 9 total, 8 up, 9 in 2024-05-20T16:09:39.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:09:39 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-20T16:09:39.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:09:39 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-20T16:09:39.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:09:39 smithi160 ceph-mon[36347]: from='osd.8 ' entity='osd.8' 2024-05-20T16:09:39.813 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:09:39 smithi160 ceph-mon[36347]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-05-20T16:09:40.294 INFO:teuthology.orchestra.run.smithi082.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-05-20T16:09:40.294 INFO:teuthology.orchestra.run.smithi082.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-05-20T16:09:40.294 INFO:teuthology.orchestra.run.smithi082.stderr: stderr: 10+0 records in 2024-05-20T16:09:40.294 INFO:teuthology.orchestra.run.smithi082.stderr:10+0 records out 2024-05-20T16:09:40.294 INFO:teuthology.orchestra.run.smithi082.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0114179 s, 918 MB/s 2024-05-20T16:09:40.294 INFO:teuthology.orchestra.run.smithi082.stderr: stderr: 2024-05-20T16:09:40.294 INFO:teuthology.orchestra.run.smithi082.stderr:--> Zapping successful for: 2024-05-20T16:09:40.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:09:40 smithi082 ceph-mon[32238]: purged_snaps scrub starts 2024-05-20T16:09:40.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:09:40 smithi082 ceph-mon[32238]: purged_snaps scrub ok 2024-05-20T16:09:40.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:09:40 smithi082 ceph-mon[32238]: osd.8 [v2:172.21.15.184:6816/3831576372,v1:172.21.15.184:6817/3831576372] boot 2024-05-20T16:09:40.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:09:40 smithi082 ceph-mon[32238]: osdmap e60: 9 total, 9 up, 9 in 2024-05-20T16:09:40.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:09:40 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-20T16:09:40.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:09:40 smithi082 ceph-mon[32238]: osdmap e61: 9 total, 9 up, 9 in 2024-05-20T16:09:40.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:09:40 smithi184 ceph-mon[36209]: purged_snaps scrub starts 2024-05-20T16:09:40.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:09:40 smithi184 ceph-mon[36209]: purged_snaps scrub ok 2024-05-20T16:09:40.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:09:40 smithi184 ceph-mon[36209]: osd.8 [v2:172.21.15.184:6816/3831576372,v1:172.21.15.184:6817/3831576372] boot 2024-05-20T16:09:40.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:09:40 smithi184 ceph-mon[36209]: osdmap e60: 9 total, 9 up, 9 in 2024-05-20T16:09:40.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:09:40 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-20T16:09:40.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:09:40 smithi184 ceph-mon[36209]: osdmap e61: 9 total, 9 up, 9 in 2024-05-20T16:09:40.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:09:40 smithi160 ceph-mon[36347]: purged_snaps scrub starts 2024-05-20T16:09:40.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:09:40 smithi160 ceph-mon[36347]: purged_snaps scrub ok 2024-05-20T16:09:40.813 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:09:40 smithi160 ceph-mon[36347]: osd.8 [v2:172.21.15.184:6816/3831576372,v1:172.21.15.184:6817/3831576372] boot 2024-05-20T16:09:40.813 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:09:40 smithi160 ceph-mon[36347]: osdmap e60: 9 total, 9 up, 9 in 2024-05-20T16:09:40.813 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:09:40 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-20T16:09:40.813 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:09:40 smithi160 ceph-mon[36347]: osdmap e61: 9 total, 9 up, 9 in 2024-05-20T16:09:41.036 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 59d1600a-16c2-11ef-bc9a-c7b262605968 -- ceph orch daemon add osd smithi082:vg_nvme/lv_1 2024-05-20T16:09:41.290 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T16:09:42.312 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:09:42 smithi160 ceph-mon[36347]: Detected new or changed devices on smithi184 2024-05-20T16:09:42.312 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:09:42 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:09:42.313 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:09:42 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:09:42.313 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:09:42 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-20T16:09:42.313 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:09:42 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-20T16:09:42.313 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:09:42 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-20T16:09:42.313 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:09:42 smithi160 ceph-mon[36347]: Adjusting osd_memory_target on smithi184 to 5696M 2024-05-20T16:09:42.313 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:09:42 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:09:42.313 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:09:42 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:09:42.313 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:09:42 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T16:09:42.313 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:09:42 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:09:42.313 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:09:42 smithi160 ceph-mon[36347]: pgmap v199: 1 pgs: 1 remapped+peering; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-05-20T16:09:42.313 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:09:42 smithi160 ceph-mon[36347]: osdmap e62: 9 total, 9 up, 9 in 2024-05-20T16:09:42.313 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:09:42 smithi160 ceph-mon[36347]: osdmap e63: 9 total, 9 up, 9 in 2024-05-20T16:09:42.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:09:42 smithi082 ceph-mon[32238]: Detected new or changed devices on smithi184 2024-05-20T16:09:42.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:09:42 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:09:42.360 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:09:42 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:09:42.360 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:09:42 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-20T16:09:42.360 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:09:42 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-20T16:09:42.360 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:09:42 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-20T16:09:42.360 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:09:42 smithi082 ceph-mon[32238]: Adjusting osd_memory_target on smithi184 to 5696M 2024-05-20T16:09:42.360 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:09:42 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:09:42.360 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:09:42 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:09:42.360 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:09:42 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T16:09:42.360 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:09:42 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:09:42.360 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:09:42 smithi082 ceph-mon[32238]: pgmap v199: 1 pgs: 1 remapped+peering; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-05-20T16:09:42.360 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:09:42 smithi082 ceph-mon[32238]: osdmap e62: 9 total, 9 up, 9 in 2024-05-20T16:09:42.360 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:09:42 smithi082 ceph-mon[32238]: osdmap e63: 9 total, 9 up, 9 in 2024-05-20T16:09:42.386 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:09:42 smithi184 ceph-mon[36209]: Detected new or changed devices on smithi184 2024-05-20T16:09:42.386 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:09:42 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:09:42.386 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:09:42 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:09:42.386 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:09:42 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-20T16:09:42.386 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:09:42 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-20T16:09:42.386 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:09:42 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-20T16:09:42.387 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:09:42 smithi184 ceph-mon[36209]: Adjusting osd_memory_target on smithi184 to 5696M 2024-05-20T16:09:42.387 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:09:42 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:09:42.387 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:09:42 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:09:42.387 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:09:42 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T16:09:42.387 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:09:42 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:09:42.387 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:09:42 smithi184 ceph-mon[36209]: pgmap v199: 1 pgs: 1 remapped+peering; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-05-20T16:09:42.387 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:09:42 smithi184 ceph-mon[36209]: osdmap e62: 9 total, 9 up, 9 in 2024-05-20T16:09:42.387 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:09:42 smithi184 ceph-mon[36209]: osdmap e63: 9 total, 9 up, 9 in 2024-05-20T16:09:42.869 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:09:42.868+0000 7fcc9334a700 1 -- 172.21.15.82:0/2110566189 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcc8c106d80 msgr2=0x7fcc8c107190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:09:42.869 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:09:42.868+0000 7fcc9334a700 1 --2- 172.21.15.82:0/2110566189 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcc8c106d80 0x7fcc8c107190 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fcc8400b0a0 tx=0x7fcc8400b3b0 comp rx=0 tx=0).stop 2024-05-20T16:09:42.869 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:09:42.869+0000 7fcc9334a700 1 -- 172.21.15.82:0/2110566189 shutdown_connections 2024-05-20T16:09:42.869 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:09:42.869+0000 7fcc9334a700 1 --2- 172.21.15.82:0/2110566189 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcc8c106d80 0x7fcc8c107190 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:09:42.869 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:09:42.869+0000 7fcc9334a700 1 --2- 172.21.15.82:0/2110566189 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcc8c101ac0 0x7fcc8c103ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:09:42.869 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:09:42.869+0000 7fcc9334a700 1 --2- 172.21.15.82:0/2110566189 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcc8c104420 0x7fcc8c106840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:09:42.869 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:09:42.869+0000 7fcc9334a700 1 -- 172.21.15.82:0/2110566189 >> 172.21.15.82:0/2110566189 conn(0x7fcc8c0fd710 msgr2=0x7fcc8c0ffbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:09:42.870 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:09:42.869+0000 7fcc9334a700 1 -- 172.21.15.82:0/2110566189 shutdown_connections 2024-05-20T16:09:42.870 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:09:42.869+0000 7fcc9334a700 1 -- 172.21.15.82:0/2110566189 wait complete. 2024-05-20T16:09:42.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:09:42.869+0000 7fcc9334a700 1 Processor -- start 2024-05-20T16:09:42.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:09:42.870+0000 7fcc9334a700 1 -- start start 2024-05-20T16:09:42.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:09:42.870+0000 7fcc9334a700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcc8c101ac0 0x7fcc8c19fde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:09:42.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:09:42.870+0000 7fcc9334a700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcc8c104420 0x7fcc8c1a0320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:09:42.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:09:42.870+0000 7fcc9334a700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcc8c106d80 0x7fcc8c1a7280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:09:42.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:09:42.870+0000 7fcc9334a700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fcc8c115380 con 0x7fcc8c106d80 2024-05-20T16:09:42.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:09:42.870+0000 7fcc9334a700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fcc8c115200 con 0x7fcc8c101ac0 2024-05-20T16:09:42.873 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:09:42.870+0000 7fcc9334a700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fcc8c115500 con 0x7fcc8c104420 2024-05-20T16:09:42.873 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:09:42.870+0000 7fcc918e7700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcc8c106d80 0x7fcc8c1a7280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:09:42.873 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:09:42.870+0000 7fcc918e7700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcc8c106d80 0x7fcc8c1a7280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41880/0 (socket says 172.21.15.82:41880) 2024-05-20T16:09:42.873 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:09:42.870+0000 7fcc918e7700 1 -- 172.21.15.82:0/3741083480 learned_addr learned my addr 172.21.15.82:0/3741083480 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:09:42.873 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:09:42.870+0000 7fcc908e5700 1 --2- 172.21.15.82:0/3741083480 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcc8c104420 0x7fcc8c1a0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:09:42.873 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:09:42.870+0000 7fcc910e6700 1 --2- 172.21.15.82:0/3741083480 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcc8c101ac0 0x7fcc8c19fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:09:42.873 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:09:42.870+0000 7fcc918e7700 1 -- 172.21.15.82:0/3741083480 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcc8c104420 msgr2=0x7fcc8c1a0320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:09:42.873 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:09:42.870+0000 7fcc918e7700 1 --2- 172.21.15.82:0/3741083480 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcc8c104420 0x7fcc8c1a0320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:09:42.873 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:09:42.870+0000 7fcc918e7700 1 -- 172.21.15.82:0/3741083480 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcc8c101ac0 msgr2=0x7fcc8c19fde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:09:42.873 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:09:42.870+0000 7fcc918e7700 1 --2- 172.21.15.82:0/3741083480 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcc8c101ac0 0x7fcc8c19fde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:09:42.873 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:09:42.870+0000 7fcc918e7700 1 -- 172.21.15.82:0/3741083480 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc8c1a7950 con 0x7fcc8c106d80 2024-05-20T16:09:42.874 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:09:42.870+0000 7fcc918e7700 1 --2- 172.21.15.82:0/3741083480 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcc8c106d80 0x7fcc8c1a7280 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fcc84006070 tx=0x7fcc84004db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:09:42.874 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:09:42.871+0000 7fcc827fc700 1 -- 172.21.15.82:0/3741083480 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc84021070 con 0x7fcc8c106d80 2024-05-20T16:09:42.874 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:09:42.871+0000 7fcc827fc700 1 -- 172.21.15.82:0/3741083480 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcc84007c00 con 0x7fcc8c106d80 2024-05-20T16:09:42.874 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:09:42.871+0000 7fcc9334a700 1 -- 172.21.15.82:0/3741083480 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc8c1a7be0 con 0x7fcc8c106d80 2024-05-20T16:09:42.874 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:09:42.871+0000 7fcc827fc700 1 -- 172.21.15.82:0/3741083480 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc8401d450 con 0x7fcc8c106d80 2024-05-20T16:09:42.874 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:09:42.871+0000 7fcc9334a700 1 -- 172.21.15.82:0/3741083480 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc8c1a80e0 con 0x7fcc8c106d80 2024-05-20T16:09:42.874 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:09:42.872+0000 7fcc9334a700 1 -- 172.21.15.82:0/3741083480 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc8c050170 con 0x7fcc8c106d80 2024-05-20T16:09:42.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:09:42.873+0000 7fcc827fc700 1 -- 172.21.15.82:0/3741083480 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fcc84003a00 con 0x7fcc8c106d80 2024-05-20T16:09:42.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:09:42.874+0000 7fcc827fc700 1 --2- 172.21.15.82:0/3741083480 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fcc780a2ca0 0x7fcc780a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:09:42.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:09:42.874+0000 7fcc910e6700 1 --2- 172.21.15.82:0/3741083480 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fcc780a2ca0 0x7fcc780a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:09:42.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:09:42.874+0000 7fcc827fc700 1 -- 172.21.15.82:0/3741083480 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 5829+0+0 (secure 0 0 0) 0x7fcc8400f020 con 0x7fcc8c106d80 2024-05-20T16:09:42.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:09:42.874+0000 7fcc910e6700 1 --2- 172.21.15.82:0/3741083480 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fcc780a2ca0 0x7fcc780a51b0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fcc8c1a11e0 tx=0x7fcc88009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:09:42.877 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:09:42.877+0000 7fcc827fc700 1 -- 172.21.15.82:0/3741083480 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fcc8402a020 con 0x7fcc8c106d80 2024-05-20T16:09:43.082 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:09:43.082+0000 7fcc9334a700 1 -- 172.21.15.82:0/3741083480 --> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi082:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fcc8c062a80 con 0x7fcc780a2ca0 2024-05-20T16:09:43.800 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:09:43 smithi082 ceph-mon[32238]: osdmap e64: 9 total, 9 up, 9 in 2024-05-20T16:09:43.800 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:09:43 smithi082 ceph-mon[32238]: from='client.14406 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi082:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-20T16:09:43.801 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:09:43 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-20T16:09:43.801 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:09:43 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-20T16:09:43.801 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:09:43 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:09:43.801 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:09:43 smithi082 ceph-mon[32238]: pgmap v203: 1 pgs: 1 remapped+peering; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-05-20T16:09:43.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:09:43 smithi160 ceph-mon[36347]: osdmap e64: 9 total, 9 up, 9 in 2024-05-20T16:09:43.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:09:43 smithi160 ceph-mon[36347]: from='client.14406 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi082:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-20T16:09:43.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:09:43 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-20T16:09:43.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:09:43 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-20T16:09:43.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:09:43 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:09:43.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:09:43 smithi160 ceph-mon[36347]: pgmap v203: 1 pgs: 1 remapped+peering; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-05-20T16:09:43.886 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:09:43 smithi184 ceph-mon[36209]: osdmap e64: 9 total, 9 up, 9 in 2024-05-20T16:09:43.886 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:09:43 smithi184 ceph-mon[36209]: from='client.14406 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi082:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-20T16:09:43.886 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:09:43 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-20T16:09:43.886 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:09:43 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-20T16:09:43.886 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:09:43 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:09:43.886 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:09:43 smithi184 ceph-mon[36209]: pgmap v203: 1 pgs: 1 remapped+peering; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-05-20T16:09:46.562 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:09:46 smithi160 ceph-mon[36347]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 119 KiB/s, 0 objects/s recovering 2024-05-20T16:09:46.562 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:09:46 smithi160 ceph-mon[36347]: from='client.? 172.21.15.82:0/2140184152' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0d69e319-954f-4b18-95ea-31794f18dc3a"}]: dispatch 2024-05-20T16:09:46.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:09:46 smithi160 ceph-mon[36347]: from='client.? 172.21.15.82:0/2140184152' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0d69e319-954f-4b18-95ea-31794f18dc3a"}]': finished 2024-05-20T16:09:46.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:09:46 smithi160 ceph-mon[36347]: osdmap e65: 10 total, 9 up, 10 in 2024-05-20T16:09:46.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:09:46 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-20T16:09:46.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:09:46 smithi082 ceph-mon[32238]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 119 KiB/s, 0 objects/s recovering 2024-05-20T16:09:46.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:09:46 smithi082 ceph-mon[32238]: from='client.? 172.21.15.82:0/2140184152' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0d69e319-954f-4b18-95ea-31794f18dc3a"}]: dispatch 2024-05-20T16:09:46.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:09:46 smithi082 ceph-mon[32238]: from='client.? 172.21.15.82:0/2140184152' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0d69e319-954f-4b18-95ea-31794f18dc3a"}]': finished 2024-05-20T16:09:46.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:09:46 smithi082 ceph-mon[32238]: osdmap e65: 10 total, 9 up, 10 in 2024-05-20T16:09:46.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:09:46 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-20T16:09:46.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:09:46 smithi184 ceph-mon[36209]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 119 KiB/s, 0 objects/s recovering 2024-05-20T16:09:46.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:09:46 smithi184 ceph-mon[36209]: from='client.? 172.21.15.82:0/2140184152' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0d69e319-954f-4b18-95ea-31794f18dc3a"}]: dispatch 2024-05-20T16:09:46.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:09:46 smithi184 ceph-mon[36209]: from='client.? 172.21.15.82:0/2140184152' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0d69e319-954f-4b18-95ea-31794f18dc3a"}]': finished 2024-05-20T16:09:46.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:09:46 smithi184 ceph-mon[36209]: osdmap e65: 10 total, 9 up, 10 in 2024-05-20T16:09:46.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:09:46 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-20T16:09:47.562 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:09:47 smithi160 ceph-mon[36347]: from='client.? 172.21.15.82:0/1000279560' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-20T16:09:47.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:09:47 smithi082 ceph-mon[32238]: from='client.? 172.21.15.82:0/1000279560' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-20T16:09:47.635 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:09:47 smithi184 ceph-mon[36209]: from='client.? 172.21.15.82:0/1000279560' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-20T16:09:48.562 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:09:48 smithi160 ceph-mon[36347]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 98 KiB/s, 0 objects/s recovering 2024-05-20T16:09:48.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:09:48 smithi082 ceph-mon[32238]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 98 KiB/s, 0 objects/s recovering 2024-05-20T16:09:48.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:09:48 smithi184 ceph-mon[36209]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 98 KiB/s, 0 objects/s recovering 2024-05-20T16:09:49.587 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:09:49 smithi082 ceph-mon[32238]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 75 KiB/s, 0 objects/s recovering 2024-05-20T16:09:49.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:09:49 smithi184 ceph-mon[36209]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 75 KiB/s, 0 objects/s recovering 2024-05-20T16:09:49.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:09:49 smithi160 ceph-mon[36347]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 75 KiB/s, 0 objects/s recovering 2024-05-20T16:09:52.349 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:09:52 smithi082 ceph-mon[32238]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail; 66 KiB/s, 0 objects/s recovering 2024-05-20T16:09:52.562 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:09:52 smithi160 ceph-mon[36347]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail; 66 KiB/s, 0 objects/s recovering 2024-05-20T16:09:52.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:09:52 smithi184 ceph-mon[36209]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail; 66 KiB/s, 0 objects/s recovering 2024-05-20T16:09:54.376 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:09:54 smithi082 ceph-mon[32238]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-20T16:09:54.376 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:09:54 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-05-20T16:09:54.376 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:09:54 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:09:54.562 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:09:54 smithi160 ceph-mon[36347]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-20T16:09:54.562 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:09:54 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-05-20T16:09:54.562 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:09:54 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:09:54.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:09:54 smithi184 ceph-mon[36209]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-20T16:09:54.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:09:54 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-05-20T16:09:54.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:09:54 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:09:55.491 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:09:55 smithi082 ceph-mon[32238]: Deploying daemon osd.9 on smithi082 2024-05-20T16:09:55.562 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:09:55 smithi160 ceph-mon[36347]: Deploying daemon osd.9 on smithi082 2024-05-20T16:09:55.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:09:55 smithi184 ceph-mon[36209]: Deploying daemon osd.9 on smithi082 2024-05-20T16:09:56.562 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:09:56 smithi160 ceph-mon[36347]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail 2024-05-20T16:09:56.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:09:56 smithi082 ceph-mon[32238]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail 2024-05-20T16:09:56.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:09:56 smithi184 ceph-mon[36209]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail 2024-05-20T16:09:57.562 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:09:57 smithi160 ceph-mon[36347]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail 2024-05-20T16:09:57.603 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:09:57 smithi082 ceph-mon[32238]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail 2024-05-20T16:09:57.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:09:57 smithi184 ceph-mon[36209]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail 2024-05-20T16:10:00.508 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:10:00 smithi082 ceph-mon[32238]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail 2024-05-20T16:10:00.508 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:10:00 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:10:00.508 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:10:00 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:10:00.508 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:10:00 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T16:10:00.508 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:10:00 smithi082 ceph-mon[32238]: overall HEALTH_OK 2024-05-20T16:10:00.562 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:10:00 smithi160 ceph-mon[36347]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail 2024-05-20T16:10:00.562 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:10:00 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:10:00.562 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:10:00 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:10:00.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:10:00 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T16:10:00.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:10:00 smithi160 ceph-mon[36347]: overall HEALTH_OK 2024-05-20T16:10:00.635 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:10:00 smithi184 ceph-mon[36209]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail 2024-05-20T16:10:00.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:10:00 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:10:00.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:10:00 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:10:00.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:10:00 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T16:10:00.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:10:00 smithi184 ceph-mon[36209]: overall HEALTH_OK 2024-05-20T16:10:02.562 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:10:02 smithi160 ceph-mon[36347]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail 2024-05-20T16:10:02.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:10:02 smithi082 ceph-mon[32238]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail 2024-05-20T16:10:02.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:10:02 smithi184 ceph-mon[36209]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail 2024-05-20T16:10:03.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:10:03.644+0000 7fcc827fc700 1 -- 172.21.15.82:0/3741083480 <== mgr.14150 v2:172.21.15.82:6800/379530823 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fcc8c062a80 con 0x7fcc780a2ca0 2024-05-20T16:10:03.645 INFO:teuthology.orchestra.run.smithi082.stdout:Created osd(s) 9 on host 'smithi082' 2024-05-20T16:10:03.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:10:03.647+0000 7fcc9334a700 1 -- 172.21.15.82:0/3741083480 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fcc780a2ca0 msgr2=0x7fcc780a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:10:03.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:10:03.647+0000 7fcc9334a700 1 --2- 172.21.15.82:0/3741083480 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fcc780a2ca0 0x7fcc780a51b0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fcc8c1a11e0 tx=0x7fcc88009260 comp rx=0 tx=0).stop 2024-05-20T16:10:03.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:10:03.647+0000 7fcc9334a700 1 -- 172.21.15.82:0/3741083480 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcc8c106d80 msgr2=0x7fcc8c1a7280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:10:03.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:10:03.647+0000 7fcc9334a700 1 --2- 172.21.15.82:0/3741083480 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcc8c106d80 0x7fcc8c1a7280 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fcc84006070 tx=0x7fcc84004db0 comp rx=0 tx=0).stop 2024-05-20T16:10:03.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:10:03.647+0000 7fcc9334a700 1 -- 172.21.15.82:0/3741083480 shutdown_connections 2024-05-20T16:10:03.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:10:03.647+0000 7fcc9334a700 1 --2- 172.21.15.82:0/3741083480 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fcc780a2ca0 0x7fcc780a51b0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:10:03.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:10:03.647+0000 7fcc9334a700 1 --2- 172.21.15.82:0/3741083480 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcc8c101ac0 0x7fcc8c19fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:10:03.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:10:03.647+0000 7fcc9334a700 1 --2- 172.21.15.82:0/3741083480 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcc8c106d80 0x7fcc8c1a7280 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:10:03.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:10:03.647+0000 7fcc9334a700 1 --2- 172.21.15.82:0/3741083480 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcc8c104420 0x7fcc8c1a0320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:10:03.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:10:03.647+0000 7fcc9334a700 1 -- 172.21.15.82:0/3741083480 >> 172.21.15.82:0/3741083480 conn(0x7fcc8c0fd710 msgr2=0x7fcc8c102900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:10:03.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:10:03.647+0000 7fcc9334a700 1 -- 172.21.15.82:0/3741083480 shutdown_connections 2024-05-20T16:10:03.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:10:03.647+0000 7fcc9334a700 1 -- 172.21.15.82:0/3741083480 wait complete. 2024-05-20T16:10:03.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:10:03 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:10:03.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:10:03 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:10:03.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:10:03 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:10:03.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:10:03 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T16:10:03.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:10:03 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:10:03.860 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:10:03 smithi082 ceph-mon[32238]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail 2024-05-20T16:10:03.860 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:10:03 smithi082 ceph-mon[32238]: from='osd.9 [v2:172.21.15.82:6826/716330782,v1:172.21.15.82:6827/716330782]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-05-20T16:10:03.860 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:10:03 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:10:03.860 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:10:03 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:10:03.860 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:10:03 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T16:10:03.886 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:10:03 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:10:03.886 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:10:03 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:10:03.886 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:10:03 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:10:03.886 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:10:03 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T16:10:03.886 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:10:03 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:10:03.886 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:10:03 smithi184 ceph-mon[36209]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail 2024-05-20T16:10:03.886 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:10:03 smithi184 ceph-mon[36209]: from='osd.9 [v2:172.21.15.82:6826/716330782,v1:172.21.15.82:6827/716330782]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-05-20T16:10:03.886 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:10:03 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:10:03.886 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:10:03 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:10:03.886 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:10:03 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T16:10:04.062 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:10:03 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:10:04.062 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:10:03 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:10:04.063 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:10:03 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:10:04.063 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:10:03 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T16:10:04.063 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:10:03 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:10:04.063 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:10:03 smithi160 ceph-mon[36347]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail 2024-05-20T16:10:04.063 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:10:03 smithi160 ceph-mon[36347]: from='osd.9 [v2:172.21.15.82:6826/716330782,v1:172.21.15.82:6827/716330782]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-05-20T16:10:04.063 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:10:03 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:10:04.063 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:10:03 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:10:04.063 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:10:03 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T16:10:04.299 DEBUG:teuthology.orchestra.run.smithi082:osd.9> sudo journalctl -f -n 0 -u ceph-59d1600a-16c2-11ef-bc9a-c7b262605968@osd.9.service 2024-05-20T16:10:04.301 INFO:tasks.cephadm:Deploying osd.10 on smithi160 with /dev/vg_nvme/lv_1... 2024-05-20T16:10:04.302 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 59d1600a-16c2-11ef-bc9a-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-05-20T16:10:04.548 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/mon.b/config 2024-05-20T16:10:04.810 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:10:04 smithi082 ceph-mon[32238]: from='osd.9 [v2:172.21.15.82:6826/716330782,v1:172.21.15.82:6827/716330782]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-05-20T16:10:04.810 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:10:04 smithi082 ceph-mon[32238]: osdmap e66: 10 total, 9 up, 10 in 2024-05-20T16:10:04.810 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:10:04 smithi082 ceph-mon[32238]: from='osd.9 [v2:172.21.15.82:6826/716330782,v1:172.21.15.82:6827/716330782]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]: dispatch 2024-05-20T16:10:04.810 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:10:04 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-20T16:10:04.811 INFO:journalctl@ceph.osd.9.smithi082.stdout:May 20 16:10:04 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-9[66777]: 2024-05-20T16:10:04.659+0000 7fa18aa09700 -1 osd.9 0 waiting for initial osdmap 2024-05-20T16:10:04.811 INFO:journalctl@ceph.osd.9.smithi082.stdout:May 20 16:10:04 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-9[66777]: 2024-05-20T16:10:04.673+0000 7fa1867be700 -1 osd.9 67 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-20T16:10:04.939 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:10:04 smithi160 ceph-mon[36347]: from='osd.9 [v2:172.21.15.82:6826/716330782,v1:172.21.15.82:6827/716330782]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-05-20T16:10:04.939 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:10:04 smithi160 ceph-mon[36347]: osdmap e66: 10 total, 9 up, 10 in 2024-05-20T16:10:04.939 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:10:04 smithi160 ceph-mon[36347]: from='osd.9 [v2:172.21.15.82:6826/716330782,v1:172.21.15.82:6827/716330782]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]: dispatch 2024-05-20T16:10:04.939 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:10:04 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-20T16:10:05.136 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:10:04 smithi184 ceph-mon[36209]: from='osd.9 [v2:172.21.15.82:6826/716330782,v1:172.21.15.82:6827/716330782]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-05-20T16:10:05.136 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:10:04 smithi184 ceph-mon[36209]: osdmap e66: 10 total, 9 up, 10 in 2024-05-20T16:10:05.136 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:10:04 smithi184 ceph-mon[36209]: from='osd.9 [v2:172.21.15.82:6826/716330782,v1:172.21.15.82:6827/716330782]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]: dispatch 2024-05-20T16:10:05.136 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:10:04 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-20T16:10:05.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:10:05 smithi082 ceph-mon[32238]: from='osd.9 [v2:172.21.15.82:6826/716330782,v1:172.21.15.82:6827/716330782]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]': finished 2024-05-20T16:10:05.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:10:05 smithi082 ceph-mon[32238]: osdmap e67: 10 total, 9 up, 10 in 2024-05-20T16:10:05.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:10:05 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-20T16:10:05.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:10:05 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-20T16:10:05.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:10:05 smithi082 ceph-mon[32238]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail 2024-05-20T16:10:05.915 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:10:05 smithi160 ceph-mon[36347]: from='osd.9 [v2:172.21.15.82:6826/716330782,v1:172.21.15.82:6827/716330782]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]': finished 2024-05-20T16:10:05.916 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:10:05 smithi160 ceph-mon[36347]: osdmap e67: 10 total, 9 up, 10 in 2024-05-20T16:10:05.916 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:10:05 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-20T16:10:05.916 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:10:05 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-20T16:10:05.916 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:10:05 smithi160 ceph-mon[36347]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail 2024-05-20T16:10:06.136 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:10:05 smithi184 ceph-mon[36209]: from='osd.9 [v2:172.21.15.82:6826/716330782,v1:172.21.15.82:6827/716330782]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]': finished 2024-05-20T16:10:06.136 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:10:05 smithi184 ceph-mon[36209]: osdmap e67: 10 total, 9 up, 10 in 2024-05-20T16:10:06.136 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:10:05 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-20T16:10:06.136 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:10:05 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-20T16:10:06.136 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:10:05 smithi184 ceph-mon[36209]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail 2024-05-20T16:10:06.649 INFO:teuthology.orchestra.run.smithi160.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-05-20T16:10:06.649 INFO:teuthology.orchestra.run.smithi160.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-05-20T16:10:06.649 INFO:teuthology.orchestra.run.smithi160.stderr: stderr: 10+0 records in 2024-05-20T16:10:06.649 INFO:teuthology.orchestra.run.smithi160.stderr:10+0 records out 2024-05-20T16:10:06.649 INFO:teuthology.orchestra.run.smithi160.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0115891 s, 905 MB/s 2024-05-20T16:10:06.649 INFO:teuthology.orchestra.run.smithi160.stderr:--> Zapping successful for: 2024-05-20T16:10:06.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:10:06 smithi160 ceph-mon[36347]: purged_snaps scrub starts 2024-05-20T16:10:06.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:10:06 smithi160 ceph-mon[36347]: purged_snaps scrub ok 2024-05-20T16:10:06.813 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:10:06 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-20T16:10:06.813 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:10:06 smithi160 ceph-mon[36347]: osd.9 [v2:172.21.15.82:6826/716330782,v1:172.21.15.82:6827/716330782] boot 2024-05-20T16:10:06.813 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:10:06 smithi160 ceph-mon[36347]: osdmap e68: 10 total, 10 up, 10 in 2024-05-20T16:10:06.813 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:10:06 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-20T16:10:06.976 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:10:06 smithi082 ceph-mon[32238]: purged_snaps scrub starts 2024-05-20T16:10:06.976 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:10:06 smithi082 ceph-mon[32238]: purged_snaps scrub ok 2024-05-20T16:10:06.976 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:10:06 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-20T16:10:06.977 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:10:06 smithi082 ceph-mon[32238]: osd.9 [v2:172.21.15.82:6826/716330782,v1:172.21.15.82:6827/716330782] boot 2024-05-20T16:10:06.977 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:10:06 smithi082 ceph-mon[32238]: osdmap e68: 10 total, 10 up, 10 in 2024-05-20T16:10:06.977 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:10:06 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-20T16:10:07.136 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:10:06 smithi184 ceph-mon[36209]: purged_snaps scrub starts 2024-05-20T16:10:07.136 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:10:06 smithi184 ceph-mon[36209]: purged_snaps scrub ok 2024-05-20T16:10:07.136 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:10:06 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-20T16:10:07.136 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:10:06 smithi184 ceph-mon[36209]: osd.9 [v2:172.21.15.82:6826/716330782,v1:172.21.15.82:6827/716330782] boot 2024-05-20T16:10:07.136 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:10:06 smithi184 ceph-mon[36209]: osdmap e68: 10 total, 10 up, 10 in 2024-05-20T16:10:07.136 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:10:06 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-20T16:10:07.388 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 59d1600a-16c2-11ef-bc9a-c7b262605968 -- ceph orch daemon add osd smithi160:vg_nvme/lv_1 2024-05-20T16:10:07.628 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/mon.b/config 2024-05-20T16:10:07.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:10:07 smithi082 ceph-mon[32238]: osdmap e69: 10 total, 10 up, 10 in 2024-05-20T16:10:07.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:10:07 smithi082 ceph-mon[32238]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-05-20T16:10:07.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:10:07 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:10:07.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:10:07 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:10:07.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:10:07 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-20T16:10:07.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:10:07 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-20T16:10:07.860 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:10:07 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-20T16:10:07.860 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:10:07 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-20T16:10:07.860 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:10:07 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:10:07.860 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:10:07 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:10:07.860 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:10:07 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T16:10:07.860 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:10:07 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:10:07.996 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:10:07 smithi160 ceph-mon[36347]: osdmap e69: 10 total, 10 up, 10 in 2024-05-20T16:10:07.997 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:10:07 smithi160 ceph-mon[36347]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-05-20T16:10:07.997 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:10:07 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:10:07.997 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:10:07 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:10:07.997 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:10:07 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-20T16:10:07.997 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:10:07 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-20T16:10:07.997 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:10:07 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-20T16:10:07.997 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:10:07 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-20T16:10:07.997 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:10:07 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:10:07.997 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:10:07 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:10:07.997 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:10:07 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T16:10:07.997 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:10:07 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:10:08.136 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:10:07 smithi184 ceph-mon[36209]: osdmap e69: 10 total, 10 up, 10 in 2024-05-20T16:10:08.136 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:10:07 smithi184 ceph-mon[36209]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-05-20T16:10:08.136 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:10:07 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:10:08.136 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:10:07 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:10:08.136 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:10:07 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-20T16:10:08.136 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:10:07 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-20T16:10:08.136 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:10:07 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-20T16:10:08.136 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:10:07 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-20T16:10:08.137 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:10:07 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:10:08.137 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:10:07 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:10:08.137 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:10:07 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T16:10:08.137 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:10:07 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:10:08.750 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:10:08 smithi160 ceph-mon[36347]: Detected new or changed devices on smithi082 2024-05-20T16:10:08.750 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:10:08 smithi160 ceph-mon[36347]: Adjusting osd_memory_target on smithi082 to 4272M 2024-05-20T16:10:09.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:10:08 smithi082 ceph-mon[32238]: Detected new or changed devices on smithi082 2024-05-20T16:10:09.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:10:08 smithi082 ceph-mon[32238]: Adjusting osd_memory_target on smithi082 to 4272M 2024-05-20T16:10:09.136 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:10:08 smithi184 ceph-mon[36209]: Detected new or changed devices on smithi082 2024-05-20T16:10:09.136 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:10:08 smithi184 ceph-mon[36209]: Adjusting osd_memory_target on smithi082 to 4272M 2024-05-20T16:10:09.309 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-20T16:10:09.308+0000 7f250ac16700 1 -- 172.21.15.160:0/3261161089 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f25041020d0 msgr2=0x7f25041024e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:10:09.309 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-20T16:10:09.308+0000 7f250ac16700 1 --2- 172.21.15.160:0/3261161089 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f25041020d0 0x7f25041024e0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f24f8009990 tx=0x7f24f8009ca0 comp rx=0 tx=0).stop 2024-05-20T16:10:09.310 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-20T16:10:09.308+0000 7f250ac16700 1 -- 172.21.15.160:0/3261161089 shutdown_connections 2024-05-20T16:10:09.310 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-20T16:10:09.308+0000 7f250ac16700 1 --2- 172.21.15.160:0/3261161089 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f25041020d0 0x7f25041024e0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:10:09.310 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-20T16:10:09.308+0000 7f250ac16700 1 --2- 172.21.15.160:0/3261161089 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2504103ba0 0x7f250410a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:10:09.310 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-20T16:10:09.308+0000 7f250ac16700 1 --2- 172.21.15.160:0/3261161089 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2504103200 0x7f2504103610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:10:09.310 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-20T16:10:09.308+0000 7f250ac16700 1 -- 172.21.15.160:0/3261161089 >> 172.21.15.160:0/3261161089 conn(0x7f25040fd710 msgr2=0x7f25040ffbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:10:09.310 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-20T16:10:09.308+0000 7f250ac16700 1 -- 172.21.15.160:0/3261161089 shutdown_connections 2024-05-20T16:10:09.310 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-20T16:10:09.308+0000 7f250ac16700 1 -- 172.21.15.160:0/3261161089 wait complete. 2024-05-20T16:10:09.312 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-20T16:10:09.309+0000 7f250ac16700 1 Processor -- start 2024-05-20T16:10:09.312 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-20T16:10:09.309+0000 7f250ac16700 1 -- start start 2024-05-20T16:10:09.312 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-20T16:10:09.310+0000 7f250ac16700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f25041020d0 0x7f25041987c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:10:09.312 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-20T16:10:09.310+0000 7f250ac16700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2504103200 0x7f2504198d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:10:09.312 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-20T16:10:09.310+0000 7f250ac16700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2504103ba0 0x7f250419fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:10:09.312 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-20T16:10:09.310+0000 7f250ac16700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f250410dce0 con 0x7f2504103ba0 2024-05-20T16:10:09.312 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-20T16:10:09.310+0000 7f250ac16700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f250410db60 con 0x7f25041020d0 2024-05-20T16:10:09.312 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-20T16:10:09.310+0000 7f250ac16700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f250410de60 con 0x7f2504103200 2024-05-20T16:10:09.312 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-20T16:10:09.310+0000 7f25089b2700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f25041020d0 0x7f25041987c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:10:09.313 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-20T16:10:09.310+0000 7f25089b2700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f25041020d0 0x7f25041987c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:41060/0 (socket says 172.21.15.160:41060) 2024-05-20T16:10:09.313 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-20T16:10:09.310+0000 7f25089b2700 1 -- 172.21.15.160:0/727799072 learned_addr learned my addr 172.21.15.160:0/727799072 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-20T16:10:09.313 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-20T16:10:09.310+0000 7f2503fff700 1 --2- 172.21.15.160:0/727799072 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2504103200 0x7f2504198d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:10:09.313 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-20T16:10:09.310+0000 7f25089b2700 1 -- 172.21.15.160:0/727799072 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2504103200 msgr2=0x7f2504198d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:10:09.313 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-20T16:10:09.310+0000 7f25089b2700 1 --2- 172.21.15.160:0/727799072 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2504103200 0x7f2504198d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:10:09.313 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-20T16:10:09.310+0000 7f25089b2700 1 -- 172.21.15.160:0/727799072 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2504103ba0 msgr2=0x7f250419fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:10:09.313 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-20T16:10:09.310+0000 7f25089b2700 1 --2- 172.21.15.160:0/727799072 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2504103ba0 0x7f250419fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:10:09.313 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-20T16:10:09.310+0000 7f25089b2700 1 -- 172.21.15.160:0/727799072 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25041a0320 con 0x7f25041020d0 2024-05-20T16:10:09.313 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-20T16:10:09.310+0000 7f25089b2700 1 --2- 172.21.15.160:0/727799072 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f25041020d0 0x7f25041987c0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f24f80098e0 tx=0x7f24f8009f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:10:09.313 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-20T16:10:09.310+0000 7f2501ffb700 1 -- 172.21.15.160:0/727799072 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24f8027070 con 0x7f25041020d0 2024-05-20T16:10:09.313 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-20T16:10:09.310+0000 7f250ac16700 1 -- 172.21.15.160:0/727799072 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25041a05b0 con 0x7f25041020d0 2024-05-20T16:10:09.313 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-20T16:10:09.310+0000 7f2501ffb700 1 -- 172.21.15.160:0/727799072 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f24f8022470 con 0x7f25041020d0 2024-05-20T16:10:09.313 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-20T16:10:09.310+0000 7f2501ffb700 1 -- 172.21.15.160:0/727799072 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24f800f6c0 con 0x7f25041020d0 2024-05-20T16:10:09.314 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-20T16:10:09.311+0000 7f250ac16700 1 -- 172.21.15.160:0/727799072 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25041a0ae0 con 0x7f25041020d0 2024-05-20T16:10:09.314 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-20T16:10:09.311+0000 7f250ac16700 1 -- 172.21.15.160:0/727799072 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2504050170 con 0x7f25041020d0 2024-05-20T16:10:09.314 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-20T16:10:09.313+0000 7f2501ffb700 1 -- 172.21.15.160:0/727799072 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f24f802a070 con 0x7f25041020d0 2024-05-20T16:10:09.314 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-20T16:10:09.313+0000 7f2501ffb700 1 --2- 172.21.15.160:0/727799072 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f24f40a2c20 0x7f24f40a5130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:10:09.315 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-20T16:10:09.313+0000 7f2501ffb700 1 -- 172.21.15.160:0/727799072 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 6261+0+0 (secure 0 0 0) 0x7f24f80cfbd0 con 0x7f25041020d0 2024-05-20T16:10:09.315 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-20T16:10:09.314+0000 7f2503fff700 1 --2- 172.21.15.160:0/727799072 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f24f40a2c20 0x7f24f40a5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:10:09.315 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-20T16:10:09.314+0000 7f2503fff700 1 --2- 172.21.15.160:0/727799072 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f24f40a2c20 0x7f24f40a5130 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f24f0005950 tx=0x7f24f0005ed0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:10:09.318 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-20T16:10:09.317+0000 7f2501ffb700 1 -- 172.21.15.160:0/727799072 <== mon.2 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f24f8013070 con 0x7f25041020d0 2024-05-20T16:10:09.520 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-20T16:10:09.520+0000 7f250ac16700 1 -- 172.21.15.160:0/727799072 --> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi160:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f2504062a80 con 0x7f24f40a2c20 2024-05-20T16:10:10.062 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:10:09 smithi160 ceph-mon[36347]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-05-20T16:10:10.062 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:10:09 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-20T16:10:10.062 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:10:09 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-20T16:10:10.062 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:10:09 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:10:10.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:10:09 smithi082 ceph-mon[32238]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-05-20T16:10:10.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:10:09 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-20T16:10:10.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:10:09 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-20T16:10:10.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:10:09 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:10:10.136 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:10:09 smithi184 ceph-mon[36209]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-05-20T16:10:10.136 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:10:09 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-20T16:10:10.136 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:10:09 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-20T16:10:10.136 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:10:09 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:10:11.010 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:10:10 smithi082 ceph-mon[32238]: from='client.24347 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi160:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-20T16:10:11.062 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:10:10 smithi160 ceph-mon[36347]: from='client.24347 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi160:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-20T16:10:11.136 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:10:10 smithi184 ceph-mon[36209]: from='client.24347 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi160:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-20T16:10:12.022 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:10:11 smithi082 ceph-mon[32238]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-05-20T16:10:12.062 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:10:11 smithi160 ceph-mon[36347]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-05-20T16:10:12.135 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:10:11 smithi184 ceph-mon[36209]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-05-20T16:10:13.062 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:10:12 smithi160 ceph-mon[36347]: from='client.? 172.21.15.160:0/4278218539' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "39dc1b78-61cb-42ca-b5b6-1e44ac2b6500"}]: dispatch 2024-05-20T16:10:13.062 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:10:12 smithi160 ceph-mon[36347]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "39dc1b78-61cb-42ca-b5b6-1e44ac2b6500"}]: dispatch 2024-05-20T16:10:13.062 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:10:12 smithi160 ceph-mon[36347]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "39dc1b78-61cb-42ca-b5b6-1e44ac2b6500"}]': finished 2024-05-20T16:10:13.062 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:10:12 smithi160 ceph-mon[36347]: osdmap e70: 11 total, 10 up, 11 in 2024-05-20T16:10:13.062 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:10:12 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-20T16:10:13.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:10:12 smithi082 ceph-mon[32238]: from='client.? 172.21.15.160:0/4278218539' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "39dc1b78-61cb-42ca-b5b6-1e44ac2b6500"}]: dispatch 2024-05-20T16:10:13.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:10:12 smithi082 ceph-mon[32238]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "39dc1b78-61cb-42ca-b5b6-1e44ac2b6500"}]: dispatch 2024-05-20T16:10:13.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:10:12 smithi082 ceph-mon[32238]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "39dc1b78-61cb-42ca-b5b6-1e44ac2b6500"}]': finished 2024-05-20T16:10:13.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:10:12 smithi082 ceph-mon[32238]: osdmap e70: 11 total, 10 up, 11 in 2024-05-20T16:10:13.110 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:10:12 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-20T16:10:13.135 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:10:12 smithi184 ceph-mon[36209]: from='client.? 172.21.15.160:0/4278218539' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "39dc1b78-61cb-42ca-b5b6-1e44ac2b6500"}]: dispatch 2024-05-20T16:10:13.136 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:10:12 smithi184 ceph-mon[36209]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "39dc1b78-61cb-42ca-b5b6-1e44ac2b6500"}]: dispatch 2024-05-20T16:10:13.136 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:10:12 smithi184 ceph-mon[36209]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "39dc1b78-61cb-42ca-b5b6-1e44ac2b6500"}]': finished 2024-05-20T16:10:13.136 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:10:12 smithi184 ceph-mon[36209]: osdmap e70: 11 total, 10 up, 11 in 2024-05-20T16:10:13.136 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:10:12 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-20T16:10:14.062 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:10:13 smithi160 ceph-mon[36347]: from='client.? 172.21.15.160:0/761634570' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-20T16:10:14.063 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:10:13 smithi160 ceph-mon[36347]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-05-20T16:10:14.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:10:13 smithi082 ceph-mon[32238]: from='client.? 172.21.15.160:0/761634570' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-20T16:10:14.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:10:13 smithi082 ceph-mon[32238]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-05-20T16:10:14.136 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:10:13 smithi184 ceph-mon[36209]: from='client.? 172.21.15.160:0/761634570' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-20T16:10:14.136 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:10:13 smithi184 ceph-mon[36209]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-05-20T16:10:16.501 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:10:16 smithi160 ceph-mon[36347]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-05-20T16:10:16.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:10:16 smithi082 ceph-mon[32238]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-05-20T16:10:16.635 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:10:16 smithi184 ceph-mon[36209]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-05-20T16:10:18.562 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:10:18 smithi160 ceph-mon[36347]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-05-20T16:10:18.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:10:18 smithi082 ceph-mon[32238]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-05-20T16:10:18.635 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:10:18 smithi184 ceph-mon[36209]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-05-20T16:10:20.562 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:10:20 smithi160 ceph-mon[36347]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-05-20T16:10:20.562 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:10:20 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-05-20T16:10:20.562 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:10:20 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:10:20.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:10:20 smithi082 ceph-mon[32238]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-05-20T16:10:20.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:10:20 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-05-20T16:10:20.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:10:20 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:10:20.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:10:20 smithi184 ceph-mon[36209]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-05-20T16:10:20.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:10:20 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-05-20T16:10:20.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:10:20 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:10:21.562 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:10:21 smithi160 ceph-mon[36347]: Deploying daemon osd.10 on smithi160 2024-05-20T16:10:21.562 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:10:21 smithi160 ceph-mon[36347]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-05-20T16:10:21.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:10:21 smithi082 ceph-mon[32238]: Deploying daemon osd.10 on smithi160 2024-05-20T16:10:21.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:10:21 smithi082 ceph-mon[32238]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-05-20T16:10:21.635 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:10:21 smithi184 ceph-mon[36209]: Deploying daemon osd.10 on smithi160 2024-05-20T16:10:21.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:10:21 smithi184 ceph-mon[36209]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-05-20T16:10:24.562 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:10:24 smithi160 ceph-mon[36347]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-05-20T16:10:24.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:10:24 smithi082 ceph-mon[32238]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-05-20T16:10:24.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:10:24 smithi184 ceph-mon[36209]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-05-20T16:10:26.403 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:10:26 smithi160 ceph-mon[36347]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-05-20T16:10:26.403 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:10:26 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:10:26.403 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:10:26 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:10:26.403 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:10:26 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T16:10:26.537 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:10:26 smithi184 ceph-mon[36209]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-05-20T16:10:26.537 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:10:26 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:10:26.538 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:10:26 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:10:26.538 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:10:26 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T16:10:26.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:10:26 smithi082 ceph-mon[32238]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-05-20T16:10:26.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:10:26 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:10:26.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:10:26 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:10:26.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:10:26 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T16:10:27.395 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:10:27 smithi160 ceph-mon[36347]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-05-20T16:10:27.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:10:27 smithi082 ceph-mon[32238]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-05-20T16:10:27.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:10:27 smithi184 ceph-mon[36209]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-05-20T16:10:30.339 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-20T16:10:30.340+0000 7f2501ffb700 1 -- 172.21.15.160:0/727799072 <== mgr.14150 v2:172.21.15.82:6800/379530823 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f2504062a80 con 0x7f24f40a2c20 2024-05-20T16:10:30.340 INFO:teuthology.orchestra.run.smithi160.stdout:Created osd(s) 10 on host 'smithi160' 2024-05-20T16:10:30.342 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-20T16:10:30.342+0000 7f250ac16700 1 -- 172.21.15.160:0/727799072 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f24f40a2c20 msgr2=0x7f24f40a5130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:10:30.342 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-20T16:10:30.342+0000 7f250ac16700 1 --2- 172.21.15.160:0/727799072 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f24f40a2c20 0x7f24f40a5130 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f24f0005950 tx=0x7f24f0005ed0 comp rx=0 tx=0).stop 2024-05-20T16:10:30.342 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-20T16:10:30.342+0000 7f250ac16700 1 -- 172.21.15.160:0/727799072 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f25041020d0 msgr2=0x7f25041987c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:10:30.343 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-20T16:10:30.342+0000 7f250ac16700 1 --2- 172.21.15.160:0/727799072 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f25041020d0 0x7f25041987c0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f24f80098e0 tx=0x7f24f8009f90 comp rx=0 tx=0).stop 2024-05-20T16:10:30.343 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-20T16:10:30.342+0000 7f250ac16700 1 -- 172.21.15.160:0/727799072 shutdown_connections 2024-05-20T16:10:30.343 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-20T16:10:30.342+0000 7f250ac16700 1 --2- 172.21.15.160:0/727799072 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f24f40a2c20 0x7f24f40a5130 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:10:30.343 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-20T16:10:30.342+0000 7f250ac16700 1 --2- 172.21.15.160:0/727799072 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f25041020d0 0x7f25041987c0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:10:30.343 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-20T16:10:30.342+0000 7f250ac16700 1 --2- 172.21.15.160:0/727799072 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2504103ba0 0x7f250419fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:10:30.343 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-20T16:10:30.342+0000 7f250ac16700 1 --2- 172.21.15.160:0/727799072 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2504103200 0x7f2504198d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:10:30.343 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-20T16:10:30.342+0000 7f250ac16700 1 -- 172.21.15.160:0/727799072 >> 172.21.15.160:0/727799072 conn(0x7f25040fd710 msgr2=0x7f2504106430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:10:30.343 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-20T16:10:30.343+0000 7f250ac16700 1 -- 172.21.15.160:0/727799072 shutdown_connections 2024-05-20T16:10:30.343 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-20T16:10:30.343+0000 7f250ac16700 1 -- 172.21.15.160:0/727799072 wait complete. 2024-05-20T16:10:30.562 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:10:30 smithi160 ceph-mon[36347]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-05-20T16:10:30.562 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:10:30 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:10:30.562 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:10:30 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:10:30.562 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:10:30 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:10:30.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:10:30 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T16:10:30.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:10:30 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:10:30.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:10:30 smithi160 ceph-mon[36347]: from='osd.10 [v2:172.21.15.160:6824/2327428014,v1:172.21.15.160:6825/2327428014]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-20T16:10:30.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:10:30 smithi160 ceph-mon[36347]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-20T16:10:30.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:10:30 smithi082 ceph-mon[32238]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-05-20T16:10:30.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:10:30 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:10:30.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:10:30 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:10:30.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:10:30 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:10:30.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:10:30 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T16:10:30.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:10:30 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:10:30.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:10:30 smithi082 ceph-mon[32238]: from='osd.10 [v2:172.21.15.160:6824/2327428014,v1:172.21.15.160:6825/2327428014]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-20T16:10:30.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:10:30 smithi082 ceph-mon[32238]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-20T16:10:30.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:10:30 smithi184 ceph-mon[36209]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-05-20T16:10:30.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:10:30 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:10:30.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:10:30 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:10:30.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:10:30 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:10:30.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:10:30 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T16:10:30.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:10:30 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:10:30.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:10:30 smithi184 ceph-mon[36209]: from='osd.10 [v2:172.21.15.160:6824/2327428014,v1:172.21.15.160:6825/2327428014]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-20T16:10:30.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:10:30 smithi184 ceph-mon[36209]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-20T16:10:31.030 DEBUG:teuthology.orchestra.run.smithi160:osd.10> sudo journalctl -f -n 0 -u ceph-59d1600a-16c2-11ef-bc9a-c7b262605968@osd.10.service 2024-05-20T16:10:31.033 INFO:tasks.cephadm:Deploying osd.11 on smithi184 with /dev/vg_nvme/lv_1... 2024-05-20T16:10:31.033 DEBUG:teuthology.orchestra.run.smithi184:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 59d1600a-16c2-11ef-bc9a-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-05-20T16:10:31.281 INFO:teuthology.orchestra.run.smithi184.stderr:Inferring config /var/lib/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/mon.c/config 2024-05-20T16:10:31.385 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:10:31 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:10:31.386 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:10:31 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T16:10:31.386 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:10:31 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:10:31.386 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:10:31 smithi184 ceph-mon[36209]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-05-20T16:10:31.386 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:10:31 smithi184 ceph-mon[36209]: osdmap e71: 11 total, 10 up, 11 in 2024-05-20T16:10:31.386 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:10:31 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-20T16:10:31.386 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:10:31 smithi184 ceph-mon[36209]: from='osd.10 [v2:172.21.15.160:6824/2327428014,v1:172.21.15.160:6825/2327428014]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]: dispatch 2024-05-20T16:10:31.386 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:10:31 smithi184 ceph-mon[36209]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]: dispatch 2024-05-20T16:10:31.514 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:10:31 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:10:31.515 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:10:31 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T16:10:31.515 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:10:31 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:10:31.515 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:10:31 smithi160 ceph-mon[36347]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-05-20T16:10:31.515 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:10:31 smithi160 ceph-mon[36347]: osdmap e71: 11 total, 10 up, 11 in 2024-05-20T16:10:31.515 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:10:31 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-20T16:10:31.515 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:10:31 smithi160 ceph-mon[36347]: from='osd.10 [v2:172.21.15.160:6824/2327428014,v1:172.21.15.160:6825/2327428014]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]: dispatch 2024-05-20T16:10:31.515 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:10:31 smithi160 ceph-mon[36347]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]: dispatch 2024-05-20T16:10:31.515 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:10:31 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:10:31.438+0000 7f332e908700 -1 osd.10 0 waiting for initial osdmap 2024-05-20T16:10:31.515 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:10:31 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:10:31.450+0000 7f332a6bd700 -1 osd.10 72 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-20T16:10:31.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:10:31 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:10:31.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:10:31 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T16:10:31.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:10:31 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:10:31.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:10:31 smithi082 ceph-mon[32238]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-05-20T16:10:31.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:10:31 smithi082 ceph-mon[32238]: osdmap e71: 11 total, 10 up, 11 in 2024-05-20T16:10:31.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:10:31 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-20T16:10:31.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:10:31 smithi082 ceph-mon[32238]: from='osd.10 [v2:172.21.15.160:6824/2327428014,v1:172.21.15.160:6825/2327428014]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]: dispatch 2024-05-20T16:10:31.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:10:31 smithi082 ceph-mon[32238]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]: dispatch 2024-05-20T16:10:32.562 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:10:32 smithi160 ceph-mon[36347]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-05-20T16:10:32.562 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:10:32 smithi160 ceph-mon[36347]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]': finished 2024-05-20T16:10:32.562 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:10:32 smithi160 ceph-mon[36347]: osdmap e72: 11 total, 10 up, 11 in 2024-05-20T16:10:32.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:10:32 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-20T16:10:32.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:10:32 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-20T16:10:32.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:10:32 smithi160 ceph-mon[36347]: osd.10 [v2:172.21.15.160:6824/2327428014,v1:172.21.15.160:6825/2327428014] boot 2024-05-20T16:10:32.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:10:32 smithi160 ceph-mon[36347]: osdmap e73: 11 total, 11 up, 11 in 2024-05-20T16:10:32.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:10:32 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-20T16:10:32.572 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:10:32 smithi184 ceph-mon[36209]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-05-20T16:10:32.573 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:10:32 smithi184 ceph-mon[36209]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]': finished 2024-05-20T16:10:32.573 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:10:32 smithi184 ceph-mon[36209]: osdmap e72: 11 total, 10 up, 11 in 2024-05-20T16:10:32.573 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:10:32 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-20T16:10:32.573 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:10:32 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-20T16:10:32.573 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:10:32 smithi184 ceph-mon[36209]: osd.10 [v2:172.21.15.160:6824/2327428014,v1:172.21.15.160:6825/2327428014] boot 2024-05-20T16:10:32.573 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:10:32 smithi184 ceph-mon[36209]: osdmap e73: 11 total, 11 up, 11 in 2024-05-20T16:10:32.573 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:10:32 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-20T16:10:32.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:10:32 smithi082 ceph-mon[32238]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-05-20T16:10:32.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:10:32 smithi082 ceph-mon[32238]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]': finished 2024-05-20T16:10:32.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:10:32 smithi082 ceph-mon[32238]: osdmap e72: 11 total, 10 up, 11 in 2024-05-20T16:10:32.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:10:32 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-20T16:10:32.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:10:32 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-20T16:10:32.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:10:32 smithi082 ceph-mon[32238]: osd.10 [v2:172.21.15.160:6824/2327428014,v1:172.21.15.160:6825/2327428014] boot 2024-05-20T16:10:32.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:10:32 smithi082 ceph-mon[32238]: osdmap e73: 11 total, 11 up, 11 in 2024-05-20T16:10:32.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:10:32 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-20T16:10:33.435 INFO:teuthology.orchestra.run.smithi184.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-05-20T16:10:33.435 INFO:teuthology.orchestra.run.smithi184.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-05-20T16:10:33.435 INFO:teuthology.orchestra.run.smithi184.stderr: stderr: 10+0 records in 2024-05-20T16:10:33.435 INFO:teuthology.orchestra.run.smithi184.stderr:10+0 records out 2024-05-20T16:10:33.435 INFO:teuthology.orchestra.run.smithi184.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0113874 s, 921 MB/s 2024-05-20T16:10:33.435 INFO:teuthology.orchestra.run.smithi184.stderr:--> Zapping successful for: 2024-05-20T16:10:33.771 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:10:33 smithi184 ceph-mon[36209]: purged_snaps scrub starts 2024-05-20T16:10:33.771 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:10:33 smithi184 ceph-mon[36209]: purged_snaps scrub ok 2024-05-20T16:10:33.771 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:10:33 smithi184 ceph-mon[36209]: osdmap e74: 11 total, 11 up, 11 in 2024-05-20T16:10:33.771 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:10:33 smithi184 ceph-mon[36209]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 310 MiB used, 983 GiB / 983 GiB avail 2024-05-20T16:10:33.774 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:10:33 smithi160 ceph-mon[36347]: purged_snaps scrub starts 2024-05-20T16:10:33.774 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:10:33 smithi160 ceph-mon[36347]: purged_snaps scrub ok 2024-05-20T16:10:33.774 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:10:33 smithi160 ceph-mon[36347]: osdmap e74: 11 total, 11 up, 11 in 2024-05-20T16:10:33.774 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:10:33 smithi160 ceph-mon[36347]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 310 MiB used, 983 GiB / 983 GiB avail 2024-05-20T16:10:33.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:10:33 smithi082 ceph-mon[32238]: purged_snaps scrub starts 2024-05-20T16:10:33.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:10:33 smithi082 ceph-mon[32238]: purged_snaps scrub ok 2024-05-20T16:10:33.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:10:33 smithi082 ceph-mon[32238]: osdmap e74: 11 total, 11 up, 11 in 2024-05-20T16:10:33.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:10:33 smithi082 ceph-mon[32238]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 310 MiB used, 983 GiB / 983 GiB avail 2024-05-20T16:10:34.084 DEBUG:teuthology.orchestra.run.smithi184:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 59d1600a-16c2-11ef-bc9a-c7b262605968 -- ceph orch daemon add osd smithi184:vg_nvme/lv_1 2024-05-20T16:10:34.323 INFO:teuthology.orchestra.run.smithi184.stderr:Inferring config /var/lib/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/mon.c/config 2024-05-20T16:10:34.743 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:10:34 smithi184 ceph-mon[36209]: osdmap e75: 11 total, 11 up, 11 in 2024-05-20T16:10:34.743 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:10:34 smithi184 ceph-mon[36209]: Detected new or changed devices on smithi160 2024-05-20T16:10:34.743 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:10:34 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:10:34.743 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:10:34 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:10:34.743 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:10:34 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-20T16:10:34.743 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:10:34 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-20T16:10:34.743 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:10:34 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-20T16:10:34.743 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:10:34 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-20T16:10:34.743 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:10:34 smithi184 ceph-mon[36209]: Adjusting osd_memory_target on smithi160 to 4272M 2024-05-20T16:10:34.743 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:10:34 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:10:34.743 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:10:34 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:10:34.743 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:10:34 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T16:10:34.743 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:10:34 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:10:34.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:10:34 smithi160 ceph-mon[36347]: osdmap e75: 11 total, 11 up, 11 in 2024-05-20T16:10:34.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:10:34 smithi160 ceph-mon[36347]: Detected new or changed devices on smithi160 2024-05-20T16:10:34.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:10:34 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:10:34.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:10:34 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:10:34.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:10:34 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-20T16:10:34.813 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:10:34 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-20T16:10:34.813 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:10:34 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-20T16:10:34.813 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:10:34 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-20T16:10:34.813 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:10:34 smithi160 ceph-mon[36347]: Adjusting osd_memory_target on smithi160 to 4272M 2024-05-20T16:10:34.813 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:10:34 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:10:34.813 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:10:34 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:10:34.813 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:10:34 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T16:10:34.813 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:10:34 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:10:34.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:10:34 smithi082 ceph-mon[32238]: osdmap e75: 11 total, 11 up, 11 in 2024-05-20T16:10:34.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:10:34 smithi082 ceph-mon[32238]: Detected new or changed devices on smithi160 2024-05-20T16:10:34.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:10:34 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:10:34.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:10:34 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:10:34.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:10:34 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-20T16:10:34.860 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:10:34 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-20T16:10:34.860 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:10:34 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-20T16:10:34.860 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:10:34 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-20T16:10:34.860 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:10:34 smithi082 ceph-mon[32238]: Adjusting osd_memory_target on smithi160 to 4272M 2024-05-20T16:10:34.860 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:10:34 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:10:34.860 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:10:34 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:10:34.860 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:10:34 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T16:10:34.860 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:10:34 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:10:35.733 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:10:35 smithi184 ceph-mon[36209]: osdmap e76: 11 total, 11 up, 11 in 2024-05-20T16:10:35.733 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:10:35 smithi184 ceph-mon[36209]: pgmap v241: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail; 152 KiB/s, 0 objects/s recovering 2024-05-20T16:10:35.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:10:35 smithi160 ceph-mon[36347]: osdmap e76: 11 total, 11 up, 11 in 2024-05-20T16:10:35.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:10:35 smithi160 ceph-mon[36347]: pgmap v241: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail; 152 KiB/s, 0 objects/s recovering 2024-05-20T16:10:35.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:10:35 smithi082 ceph-mon[32238]: osdmap e76: 11 total, 11 up, 11 in 2024-05-20T16:10:35.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:10:35 smithi082 ceph-mon[32238]: pgmap v241: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail; 152 KiB/s, 0 objects/s recovering 2024-05-20T16:10:36.014 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:10:36.014+0000 7f53814d1700 1 -- 172.21.15.184:0/1489498527 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f537c103cc0 msgr2=0x7f537c1060e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:10:36.014 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:10:36.014+0000 7f53814d1700 1 --2- 172.21.15.184:0/1489498527 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f537c103cc0 0x7f537c1060e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f536c00b0a0 tx=0x7f536c00b3b0 comp rx=0 tx=0).stop 2024-05-20T16:10:36.014 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:10:36.014+0000 7f53814d1700 1 -- 172.21.15.184:0/1489498527 shutdown_connections 2024-05-20T16:10:36.014 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:10:36.014+0000 7f53814d1700 1 --2- 172.21.15.184:0/1489498527 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f537c108f80 0x7f537c10b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:10:36.014 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:10:36.014+0000 7f53814d1700 1 --2- 172.21.15.184:0/1489498527 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f537c103cc0 0x7f537c1060e0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:10:36.014 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:10:36.014+0000 7f53814d1700 1 --2- 172.21.15.184:0/1489498527 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f537c106620 0x7f537c108a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:10:36.014 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:10:36.014+0000 7f53814d1700 1 -- 172.21.15.184:0/1489498527 >> 172.21.15.184:0/1489498527 conn(0x7f537c0fd700 msgr2=0x7f537c0ffbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:10:36.014 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:10:36.014+0000 7f53814d1700 1 -- 172.21.15.184:0/1489498527 shutdown_connections 2024-05-20T16:10:36.014 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:10:36.014+0000 7f53814d1700 1 -- 172.21.15.184:0/1489498527 wait complete. 2024-05-20T16:10:36.014 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:10:36.015+0000 7f53814d1700 1 Processor -- start 2024-05-20T16:10:36.016 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:10:36.015+0000 7f53814d1700 1 -- start start 2024-05-20T16:10:36.016 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:10:36.015+0000 7f53814d1700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f537c103cc0 0x7f537c103520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:10:36.016 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:10:36.015+0000 7f53814d1700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f537c106620 0x7f537c101bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:10:36.016 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:10:36.015+0000 7f53814d1700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f537c108f80 0x7f537c102110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:10:36.016 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:10:36.015+0000 7f53814d1700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f537c10edd0 con 0x7f537c108f80 2024-05-20T16:10:36.017 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:10:36.015+0000 7f53814d1700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f537c10ec50 con 0x7f537c106620 2024-05-20T16:10:36.017 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:10:36.015+0000 7f53814d1700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f537c10ef50 con 0x7f537c103cc0 2024-05-20T16:10:36.017 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:10:36.015+0000 7f537affd700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f537c103cc0 0x7f537c103520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:10:36.017 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:10:36.015+0000 7f537affd700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f537c103cc0 0x7f537c103520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.184:39954/0 (socket says 172.21.15.184:39954) 2024-05-20T16:10:36.017 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:10:36.015+0000 7f537affd700 1 -- 172.21.15.184:0/3489308290 learned_addr learned my addr 172.21.15.184:0/3489308290 (peer_addr_for_me v2:172.21.15.184:0/0) 2024-05-20T16:10:36.017 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:10:36.015+0000 7f537a7fc700 1 --2- 172.21.15.184:0/3489308290 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f537c106620 0x7f537c101bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:10:36.017 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:10:36.015+0000 7f537b7fe700 1 --2- 172.21.15.184:0/3489308290 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f537c108f80 0x7f537c102110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:10:36.017 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:10:36.016+0000 7f537affd700 1 -- 172.21.15.184:0/3489308290 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f537c106620 msgr2=0x7f537c101bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:10:36.017 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:10:36.016+0000 7f537affd700 1 --2- 172.21.15.184:0/3489308290 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f537c106620 0x7f537c101bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:10:36.017 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:10:36.016+0000 7f537affd700 1 -- 172.21.15.184:0/3489308290 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f537c108f80 msgr2=0x7f537c102110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:10:36.017 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:10:36.016+0000 7f537affd700 1 --2- 172.21.15.184:0/3489308290 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f537c108f80 0x7f537c102110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:10:36.017 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:10:36.016+0000 7f537affd700 1 -- 172.21.15.184:0/3489308290 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f537c1027e0 con 0x7f537c103cc0 2024-05-20T16:10:36.017 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:10:36.016+0000 7f537affd700 1 --2- 172.21.15.184:0/3489308290 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f537c103cc0 0x7f537c103520 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f536c00d010 tx=0x7f536c004dd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:10:36.018 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:10:36.016+0000 7f5373fff700 1 -- 172.21.15.184:0/3489308290 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f536c021070 con 0x7f537c103cc0 2024-05-20T16:10:36.018 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:10:36.016+0000 7f53814d1700 1 -- 172.21.15.184:0/3489308290 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f537c1a3950 con 0x7f537c103cc0 2024-05-20T16:10:36.018 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:10:36.016+0000 7f5373fff700 1 -- 172.21.15.184:0/3489308290 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f536c01d930 con 0x7f537c103cc0 2024-05-20T16:10:36.018 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:10:36.016+0000 7f5373fff700 1 -- 172.21.15.184:0/3489308290 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f536c005570 con 0x7f537c103cc0 2024-05-20T16:10:36.018 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:10:36.016+0000 7f53814d1700 1 -- 172.21.15.184:0/3489308290 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f537c1a3cf0 con 0x7f537c103cc0 2024-05-20T16:10:36.018 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:10:36.017+0000 7f53814d1700 1 -- 172.21.15.184:0/3489308290 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f537c050170 con 0x7f537c103cc0 2024-05-20T16:10:36.019 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:10:36.018+0000 7f5373fff700 1 -- 172.21.15.184:0/3489308290 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f536c024070 con 0x7f537c103cc0 2024-05-20T16:10:36.019 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:10:36.019+0000 7f5373fff700 1 --2- 172.21.15.184:0/3489308290 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f53680a2c20 0x7f53680a5130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:10:36.019 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:10:36.019+0000 7f5373fff700 1 -- 172.21.15.184:0/3489308290 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 6694+0+0 (secure 0 0 0) 0x7f536c00f020 con 0x7f537c103cc0 2024-05-20T16:10:36.019 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:10:36.019+0000 7f537a7fc700 1 --2- 172.21.15.184:0/3489308290 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f53680a2c20 0x7f53680a5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:10:36.019 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:10:36.020+0000 7f537a7fc700 1 --2- 172.21.15.184:0/3489308290 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f53680a2c20 0x7f53680a5130 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f537c102ed0 tx=0x7f536400a380 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:10:36.023 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:10:36.023+0000 7f5373fff700 1 -- 172.21.15.184:0/3489308290 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f536c02b020 con 0x7f537c103cc0 2024-05-20T16:10:36.227 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:10:36.228+0000 7f53814d1700 1 -- 172.21.15.184:0/3489308290 --> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi184:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f537c02e0b0 con 0x7f53680a2c20 2024-05-20T16:10:36.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:10:36 smithi160 ceph-mon[36347]: osdmap e77: 11 total, 11 up, 11 in 2024-05-20T16:10:36.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:10:36 smithi160 ceph-mon[36347]: from='client.24364 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi184:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-20T16:10:36.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:10:36 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-20T16:10:36.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:10:36 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-20T16:10:36.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:10:36 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:10:36.841 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:10:36 smithi184 ceph-mon[36209]: osdmap e77: 11 total, 11 up, 11 in 2024-05-20T16:10:36.841 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:10:36 smithi184 ceph-mon[36209]: from='client.24364 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi184:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-20T16:10:36.841 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:10:36 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-20T16:10:36.841 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:10:36 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-20T16:10:36.841 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:10:36 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:10:36.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:10:36 smithi082 ceph-mon[32238]: osdmap e77: 11 total, 11 up, 11 in 2024-05-20T16:10:36.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:10:36 smithi082 ceph-mon[32238]: from='client.24364 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi184:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-20T16:10:36.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:10:36 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-20T16:10:36.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:10:36 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-20T16:10:36.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:10:36 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:10:37.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:10:37 smithi160 ceph-mon[36347]: pgmap v243: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail; 123 KiB/s, 0 objects/s recovering 2024-05-20T16:10:37.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:10:37 smithi082 ceph-mon[32238]: pgmap v243: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail; 123 KiB/s, 0 objects/s recovering 2024-05-20T16:10:37.871 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:10:37 smithi184 ceph-mon[36209]: pgmap v243: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail; 123 KiB/s, 0 objects/s recovering 2024-05-20T16:10:39.386 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:10:39 smithi184 ceph-mon[36209]: from='client.? 172.21.15.184:0/1121742457' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4306b3c1-707a-4010-987c-209ea0e948e3"}]: dispatch 2024-05-20T16:10:39.386 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:10:39 smithi184 ceph-mon[36209]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4306b3c1-707a-4010-987c-209ea0e948e3"}]: dispatch 2024-05-20T16:10:39.386 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:10:39 smithi184 ceph-mon[36209]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4306b3c1-707a-4010-987c-209ea0e948e3"}]': finished 2024-05-20T16:10:39.386 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:10:39 smithi184 ceph-mon[36209]: osdmap e78: 12 total, 11 up, 12 in 2024-05-20T16:10:39.386 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:10:39 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-20T16:10:39.562 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:10:39 smithi160 ceph-mon[36347]: from='client.? 172.21.15.184:0/1121742457' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4306b3c1-707a-4010-987c-209ea0e948e3"}]: dispatch 2024-05-20T16:10:39.562 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:10:39 smithi160 ceph-mon[36347]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4306b3c1-707a-4010-987c-209ea0e948e3"}]: dispatch 2024-05-20T16:10:39.562 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:10:39 smithi160 ceph-mon[36347]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4306b3c1-707a-4010-987c-209ea0e948e3"}]': finished 2024-05-20T16:10:39.562 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:10:39 smithi160 ceph-mon[36347]: osdmap e78: 12 total, 11 up, 12 in 2024-05-20T16:10:39.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:10:39 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-20T16:10:39.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:10:39 smithi082 ceph-mon[32238]: from='client.? 172.21.15.184:0/1121742457' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4306b3c1-707a-4010-987c-209ea0e948e3"}]: dispatch 2024-05-20T16:10:39.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:10:39 smithi082 ceph-mon[32238]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4306b3c1-707a-4010-987c-209ea0e948e3"}]: dispatch 2024-05-20T16:10:39.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:10:39 smithi082 ceph-mon[32238]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4306b3c1-707a-4010-987c-209ea0e948e3"}]': finished 2024-05-20T16:10:39.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:10:39 smithi082 ceph-mon[32238]: osdmap e78: 12 total, 11 up, 12 in 2024-05-20T16:10:39.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:10:39 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-20T16:10:40.385 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:10:40 smithi184 ceph-mon[36209]: pgmap v245: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail; 101 KiB/s, 0 objects/s recovering 2024-05-20T16:10:40.386 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:10:40 smithi184 ceph-mon[36209]: from='client.? 172.21.15.184:0/2378478332' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-20T16:10:40.562 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:10:40 smithi160 ceph-mon[36347]: pgmap v245: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail; 101 KiB/s, 0 objects/s recovering 2024-05-20T16:10:40.562 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:10:40 smithi160 ceph-mon[36347]: from='client.? 172.21.15.184:0/2378478332' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-20T16:10:40.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:10:40 smithi082 ceph-mon[32238]: pgmap v245: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail; 101 KiB/s, 0 objects/s recovering 2024-05-20T16:10:40.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:10:40 smithi082 ceph-mon[32238]: from='client.? 172.21.15.184:0/2378478332' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-20T16:10:42.562 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:10:42 smithi160 ceph-mon[36347]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail 2024-05-20T16:10:42.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:10:42 smithi082 ceph-mon[32238]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail 2024-05-20T16:10:42.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:10:42 smithi184 ceph-mon[36209]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail 2024-05-20T16:10:44.562 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:10:44 smithi160 ceph-mon[36347]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail 2024-05-20T16:10:44.584 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:10:44 smithi184 ceph-mon[36209]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail 2024-05-20T16:10:44.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:10:44 smithi082 ceph-mon[32238]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail 2024-05-20T16:10:45.551 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:10:45 smithi184 ceph-mon[36209]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail 2024-05-20T16:10:45.562 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:10:45 smithi160 ceph-mon[36347]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail 2024-05-20T16:10:45.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:10:45 smithi082 ceph-mon[32238]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail 2024-05-20T16:10:46.885 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:10:46 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-05-20T16:10:46.886 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:10:46 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:10:47.062 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:10:46 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-05-20T16:10:47.063 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:10:46 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:10:47.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:10:46 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-05-20T16:10:47.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:10:46 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:10:47.886 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:10:47 smithi184 ceph-mon[36209]: Deploying daemon osd.11 on smithi184 2024-05-20T16:10:47.886 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:10:47 smithi184 ceph-mon[36209]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail 2024-05-20T16:10:48.062 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:10:47 smithi160 ceph-mon[36347]: Deploying daemon osd.11 on smithi184 2024-05-20T16:10:48.063 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:10:47 smithi160 ceph-mon[36347]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail 2024-05-20T16:10:48.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:10:47 smithi082 ceph-mon[32238]: Deploying daemon osd.11 on smithi184 2024-05-20T16:10:48.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:10:47 smithi082 ceph-mon[32238]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail 2024-05-20T16:10:50.562 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:10:50 smithi160 ceph-mon[36347]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail 2024-05-20T16:10:50.607 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:10:50 smithi184 ceph-mon[36209]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail 2024-05-20T16:10:50.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:10:50 smithi082 ceph-mon[32238]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail 2024-05-20T16:10:51.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:10:51 smithi082 ceph-mon[32238]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail 2024-05-20T16:10:51.613 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:10:51 smithi184 ceph-mon[36209]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail 2024-05-20T16:10:51.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:10:51 smithi160 ceph-mon[36347]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail 2024-05-20T16:10:53.886 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:10:53 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:10:53.886 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:10:53 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:10:53.886 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:10:53 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T16:10:53.886 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:10:53 smithi184 ceph-mon[36209]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail 2024-05-20T16:10:54.062 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:10:53 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:10:54.062 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:10:53 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:10:54.062 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:10:53 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T16:10:54.062 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:10:53 smithi160 ceph-mon[36347]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail 2024-05-20T16:10:54.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:10:53 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:10:54.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:10:53 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:10:54.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:10:53 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T16:10:54.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:10:53 smithi082 ceph-mon[32238]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail 2024-05-20T16:10:56.562 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:10:56 smithi160 ceph-mon[36347]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail 2024-05-20T16:10:56.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:10:56 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:10:56.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:10:56 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:10:56.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:10:56 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:10:56.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:10:56 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T16:10:56.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:10:56 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:10:56.576 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:10:56 smithi184 ceph-mon[36209]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail 2024-05-20T16:10:56.576 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:10:56 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:10:56.577 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:10:56 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:10:56.577 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:10:56 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:10:56.577 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:10:56 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T16:10:56.577 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:10:56 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:10:56.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:10:56 smithi082 ceph-mon[32238]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail 2024-05-20T16:10:56.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:10:56 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:10:56.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:10:56 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:10:56.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:10:56 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:10:56.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:10:56 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T16:10:56.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:10:56 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:10:56.695 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:10:56.695+0000 7f5373fff700 1 -- 172.21.15.184:0/3489308290 <== mgr.14150 v2:172.21.15.82:6800/379530823 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f537c02e0b0 con 0x7f53680a2c20 2024-05-20T16:10:56.696 INFO:teuthology.orchestra.run.smithi184.stdout:Created osd(s) 11 on host 'smithi184' 2024-05-20T16:10:56.698 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:10:56.697+0000 7f53814d1700 1 -- 172.21.15.184:0/3489308290 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f53680a2c20 msgr2=0x7f53680a5130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:10:56.698 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:10:56.697+0000 7f53814d1700 1 --2- 172.21.15.184:0/3489308290 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f53680a2c20 0x7f53680a5130 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f537c102ed0 tx=0x7f536400a380 comp rx=0 tx=0).stop 2024-05-20T16:10:56.698 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:10:56.697+0000 7f53814d1700 1 -- 172.21.15.184:0/3489308290 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f537c103cc0 msgr2=0x7f537c103520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:10:56.698 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:10:56.697+0000 7f53814d1700 1 --2- 172.21.15.184:0/3489308290 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f537c103cc0 0x7f537c103520 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f536c00d010 tx=0x7f536c004dd0 comp rx=0 tx=0).stop 2024-05-20T16:10:56.698 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:10:56.697+0000 7f53814d1700 1 -- 172.21.15.184:0/3489308290 shutdown_connections 2024-05-20T16:10:56.698 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:10:56.697+0000 7f53814d1700 1 --2- 172.21.15.184:0/3489308290 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f53680a2c20 0x7f53680a5130 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:10:56.698 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:10:56.697+0000 7f53814d1700 1 --2- 172.21.15.184:0/3489308290 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f537c108f80 0x7f537c102110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:10:56.698 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:10:56.697+0000 7f53814d1700 1 --2- 172.21.15.184:0/3489308290 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f537c103cc0 0x7f537c103520 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:10:56.698 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:10:56.697+0000 7f53814d1700 1 --2- 172.21.15.184:0/3489308290 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f537c106620 0x7f537c101bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:10:56.699 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:10:56.697+0000 7f53814d1700 1 -- 172.21.15.184:0/3489308290 >> 172.21.15.184:0/3489308290 conn(0x7f537c0fd700 msgr2=0x7f537c104750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:10:56.699 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:10:56.698+0000 7f53814d1700 1 -- 172.21.15.184:0/3489308290 shutdown_connections 2024-05-20T16:10:56.699 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:10:56.698+0000 7f53814d1700 1 -- 172.21.15.184:0/3489308290 wait complete. 2024-05-20T16:10:57.346 DEBUG:teuthology.orchestra.run.smithi184:osd.11> sudo journalctl -f -n 0 -u ceph-59d1600a-16c2-11ef-bc9a-c7b262605968@osd.11.service 2024-05-20T16:10:57.348 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-05-20T16:10:57.348 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 59d1600a-16c2-11ef-bc9a-c7b262605968 -- ceph osd stat -f json 2024-05-20T16:10:57.552 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:10:57 smithi184 ceph-mon[36209]: from='osd.11 [v2:172.21.15.184:6824/3594364650,v1:172.21.15.184:6825/3594364650]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-20T16:10:57.552 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:10:57 smithi184 ceph-mon[36209]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-20T16:10:57.552 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:10:57 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:10:57.552 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:10:57 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:10:57.553 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:10:57 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T16:10:57.562 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:10:57 smithi160 ceph-mon[36347]: from='osd.11 [v2:172.21.15.184:6824/3594364650,v1:172.21.15.184:6825/3594364650]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-20T16:10:57.562 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:10:57 smithi160 ceph-mon[36347]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-20T16:10:57.562 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:10:57 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:10:57.562 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:10:57 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:10:57.562 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:10:57 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T16:10:57.598 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T16:10:57.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:10:57 smithi082 ceph-mon[32238]: from='osd.11 [v2:172.21.15.184:6824/3594364650,v1:172.21.15.184:6825/3594364650]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-20T16:10:57.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:10:57 smithi082 ceph-mon[32238]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-20T16:10:57.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:10:57 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:10:57.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:10:57 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:10:57.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:10:57 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T16:10:58.562 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:10:58 smithi160 ceph-mon[36347]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail 2024-05-20T16:10:58.562 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:10:58 smithi160 ceph-mon[36347]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-05-20T16:10:58.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:10:58 smithi160 ceph-mon[36347]: osdmap e79: 12 total, 11 up, 12 in 2024-05-20T16:10:58.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:10:58 smithi160 ceph-mon[36347]: from='osd.11 [v2:172.21.15.184:6824/3594364650,v1:172.21.15.184:6825/3594364650]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]: dispatch 2024-05-20T16:10:58.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:10:58 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-20T16:10:58.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:10:58 smithi160 ceph-mon[36347]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]: dispatch 2024-05-20T16:10:58.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:10:58 smithi160 ceph-mon[36347]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]': finished 2024-05-20T16:10:58.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:10:58 smithi160 ceph-mon[36347]: osdmap e80: 12 total, 11 up, 12 in 2024-05-20T16:10:58.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:10:58 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-20T16:10:58.563 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:10:58 smithi184 ceph-mon[36209]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail 2024-05-20T16:10:58.564 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:10:58 smithi184 ceph-mon[36209]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-05-20T16:10:58.564 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:10:58 smithi184 ceph-mon[36209]: osdmap e79: 12 total, 11 up, 12 in 2024-05-20T16:10:58.564 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:10:58 smithi184 ceph-mon[36209]: from='osd.11 [v2:172.21.15.184:6824/3594364650,v1:172.21.15.184:6825/3594364650]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]: dispatch 2024-05-20T16:10:58.564 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:10:58 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-20T16:10:58.564 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:10:58 smithi184 ceph-mon[36209]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]: dispatch 2024-05-20T16:10:58.564 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:10:58 smithi184 ceph-mon[36209]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]': finished 2024-05-20T16:10:58.564 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:10:58 smithi184 ceph-mon[36209]: osdmap e80: 12 total, 11 up, 12 in 2024-05-20T16:10:58.564 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:10:58 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-20T16:10:58.564 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:10:58 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:10:58.303+0000 7f99d10ec700 -1 osd.11 0 waiting for initial osdmap 2024-05-20T16:10:58.564 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:10:58 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:10:58.320+0000 7f99ccea1700 -1 osd.11 80 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-20T16:10:58.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:10:58 smithi082 ceph-mon[32238]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail 2024-05-20T16:10:58.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:10:58 smithi082 ceph-mon[32238]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-05-20T16:10:58.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:10:58 smithi082 ceph-mon[32238]: osdmap e79: 12 total, 11 up, 12 in 2024-05-20T16:10:58.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:10:58 smithi082 ceph-mon[32238]: from='osd.11 [v2:172.21.15.184:6824/3594364650,v1:172.21.15.184:6825/3594364650]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]: dispatch 2024-05-20T16:10:58.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:10:58 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-20T16:10:58.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:10:58 smithi082 ceph-mon[32238]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]: dispatch 2024-05-20T16:10:58.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:10:58 smithi082 ceph-mon[32238]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]': finished 2024-05-20T16:10:58.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:10:58 smithi082 ceph-mon[32238]: osdmap e80: 12 total, 11 up, 12 in 2024-05-20T16:10:58.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:10:58 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-20T16:10:59.414 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:10:59.412+0000 7fc52b852700 1 -- 172.21.15.82:0/3730655212 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc524101860 msgr2=0x7fc524101c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:10:59.415 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:10:59.412+0000 7fc52b852700 1 --2- 172.21.15.82:0/3730655212 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc524101860 0x7fc524101c70 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fc52000b0a0 tx=0x7fc52000b3b0 comp rx=0 tx=0).stop 2024-05-20T16:10:59.415 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:10:59.412+0000 7fc52b852700 1 -- 172.21.15.82:0/3730655212 shutdown_connections 2024-05-20T16:10:59.415 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:10:59.412+0000 7fc52b852700 1 --2- 172.21.15.82:0/3730655212 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc524103330 0x7fc524109be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:10:59.415 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:10:59.412+0000 7fc52b852700 1 --2- 172.21.15.82:0/3730655212 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc524101860 0x7fc524101c70 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:10:59.415 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:10:59.412+0000 7fc52b852700 1 --2- 172.21.15.82:0/3730655212 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc524102990 0x7fc524102da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:10:59.415 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:10:59.412+0000 7fc52b852700 1 -- 172.21.15.82:0/3730655212 >> 172.21.15.82:0/3730655212 conn(0x7fc5240fcea0 msgr2=0x7fc5240ff350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:10:59.415 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:10:59.413+0000 7fc52b852700 1 -- 172.21.15.82:0/3730655212 shutdown_connections 2024-05-20T16:10:59.415 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:10:59.413+0000 7fc52b852700 1 -- 172.21.15.82:0/3730655212 wait complete. 2024-05-20T16:10:59.415 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:10:59.413+0000 7fc52b852700 1 Processor -- start 2024-05-20T16:10:59.415 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:10:59.413+0000 7fc52b852700 1 -- start start 2024-05-20T16:10:59.416 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:10:59.413+0000 7fc52b852700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc524101860 0x7fc524199860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:10:59.416 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:10:59.413+0000 7fc52b852700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc524102990 0x7fc524199da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:10:59.416 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:10:59.413+0000 7fc52b852700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc524103330 0x7fc5241a0d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:10:59.416 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:10:59.413+0000 7fc52b852700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fc52410cba0 con 0x7fc524103330 2024-05-20T16:10:59.416 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:10:59.414+0000 7fc52b852700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fc52410ca20 con 0x7fc524101860 2024-05-20T16:10:59.416 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:10:59.414+0000 7fc52b852700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fc52410cd20 con 0x7fc524102990 2024-05-20T16:10:59.416 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:10:59.414+0000 7fc529def700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc524103330 0x7fc5241a0d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:10:59.416 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:10:59.414+0000 7fc5295ee700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc524101860 0x7fc524199860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:10:59.416 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:10:59.414+0000 7fc529def700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc524103330 0x7fc5241a0d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41650/0 (socket says 172.21.15.82:41650) 2024-05-20T16:10:59.416 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:10:59.414+0000 7fc529def700 1 -- 172.21.15.82:0/2946680082 learned_addr learned my addr 172.21.15.82:0/2946680082 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:10:59.416 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:10:59.414+0000 7fc528ded700 1 --2- 172.21.15.82:0/2946680082 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc524102990 0x7fc524199da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:10:59.416 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:10:59.414+0000 7fc529def700 1 -- 172.21.15.82:0/2946680082 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc524102990 msgr2=0x7fc524199da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:10:59.416 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:10:59.414+0000 7fc529def700 1 --2- 172.21.15.82:0/2946680082 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc524102990 0x7fc524199da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:10:59.416 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:10:59.414+0000 7fc529def700 1 -- 172.21.15.82:0/2946680082 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc524101860 msgr2=0x7fc524199860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:10:59.417 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:10:59.414+0000 7fc529def700 1 --2- 172.21.15.82:0/2946680082 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc524101860 0x7fc524199860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:10:59.417 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:10:59.414+0000 7fc529def700 1 -- 172.21.15.82:0/2946680082 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5241a13c0 con 0x7fc524103330 2024-05-20T16:10:59.417 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:10:59.414+0000 7fc529def700 1 --2- 172.21.15.82:0/2946680082 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc524103330 0x7fc5241a0d40 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fc51c00ea50 tx=0x7fc51c00ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:10:59.417 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:10:59.414+0000 7fc51a7fc700 1 -- 172.21.15.82:0/2946680082 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc51c00ccd0 con 0x7fc524103330 2024-05-20T16:10:59.417 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:10:59.414+0000 7fc51a7fc700 1 -- 172.21.15.82:0/2946680082 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc51c004d10 con 0x7fc524103330 2024-05-20T16:10:59.417 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:10:59.414+0000 7fc51a7fc700 1 -- 172.21.15.82:0/2946680082 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc51c0155f0 con 0x7fc524103330 2024-05-20T16:10:59.417 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:10:59.414+0000 7fc52b852700 1 -- 172.21.15.82:0/2946680082 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5241a16b0 con 0x7fc524103330 2024-05-20T16:10:59.417 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:10:59.414+0000 7fc52b852700 1 -- 172.21.15.82:0/2946680082 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5241a1c40 con 0x7fc524103330 2024-05-20T16:10:59.418 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:10:59.415+0000 7fc52b852700 1 -- 172.21.15.82:0/2946680082 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc524050170 con 0x7fc524103330 2024-05-20T16:10:59.418 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:10:59.417+0000 7fc51a7fc700 1 -- 172.21.15.82:0/2946680082 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fc51c004ec0 con 0x7fc524103330 2024-05-20T16:10:59.418 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:10:59.418+0000 7fc51a7fc700 1 --2- 172.21.15.82:0/2946680082 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fc5100a2ca0 0x7fc5100a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:10:59.418 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:10:59.418+0000 7fc5295ee700 1 --2- 172.21.15.82:0/2946680082 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fc5100a2ca0 0x7fc5100a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:10:59.418 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:10:59.418+0000 7fc51a7fc700 1 -- 172.21.15.82:0/2946680082 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fc51c0ced80 con 0x7fc524103330 2024-05-20T16:10:59.418 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:10:59.418+0000 7fc5295ee700 1 --2- 172.21.15.82:0/2946680082 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fc5100a2ca0 0x7fc5100a51b0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fc520009bc0 tx=0x7fc52000b700 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:10:59.421 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:10:59.422+0000 7fc51a7fc700 1 -- 172.21.15.82:0/2946680082 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fc51c0142f0 con 0x7fc524103330 2024-05-20T16:10:59.562 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:10:59 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-20T16:10:59.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:10:59 smithi160 ceph-mon[36347]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail 2024-05-20T16:10:59.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:10:59 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-20T16:10:59.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:10:59 smithi082 ceph-mon[32238]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail 2024-05-20T16:10:59.617 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:10:59.617+0000 7fc52b852700 1 -- 172.21.15.82:0/2946680082 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7fc524063250 con 0x7fc524103330 2024-05-20T16:10:59.618 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:10:59.618+0000 7fc51a7fc700 1 -- 172.21.15.82:0/2946680082 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v81) ==== 74+0+133 (secure 0 0 0) 0x7fc51c09c180 con 0x7fc524103330 2024-05-20T16:10:59.618 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-05-20T16:10:59.620 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:10:59.620+0000 7fc52b852700 1 -- 172.21.15.82:0/2946680082 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fc5100a2ca0 msgr2=0x7fc5100a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:10:59.620 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:10:59.620+0000 7fc52b852700 1 --2- 172.21.15.82:0/2946680082 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fc5100a2ca0 0x7fc5100a51b0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fc520009bc0 tx=0x7fc52000b700 comp rx=0 tx=0).stop 2024-05-20T16:10:59.620 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:10:59.620+0000 7fc52b852700 1 -- 172.21.15.82:0/2946680082 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc524103330 msgr2=0x7fc5241a0d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:10:59.620 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:10:59.620+0000 7fc52b852700 1 --2- 172.21.15.82:0/2946680082 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc524103330 0x7fc5241a0d40 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fc51c00ea50 tx=0x7fc51c00ed60 comp rx=0 tx=0).stop 2024-05-20T16:10:59.620 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:10:59.620+0000 7fc52b852700 1 -- 172.21.15.82:0/2946680082 shutdown_connections 2024-05-20T16:10:59.620 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:10:59.620+0000 7fc52b852700 1 --2- 172.21.15.82:0/2946680082 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fc5100a2ca0 0x7fc5100a51b0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:10:59.620 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:10:59.620+0000 7fc52b852700 1 --2- 172.21.15.82:0/2946680082 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc524101860 0x7fc524199860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:10:59.621 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:10:59.620+0000 7fc52b852700 1 --2- 172.21.15.82:0/2946680082 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc524103330 0x7fc5241a0d40 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:10:59.621 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:10:59.620+0000 7fc52b852700 1 --2- 172.21.15.82:0/2946680082 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc524102990 0x7fc524199da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:10:59.621 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:10:59.620+0000 7fc52b852700 1 -- 172.21.15.82:0/2946680082 >> 172.21.15.82:0/2946680082 conn(0x7fc5240fcea0 msgr2=0x7fc524105bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:10:59.621 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:10:59.620+0000 7fc52b852700 1 -- 172.21.15.82:0/2946680082 shutdown_connections 2024-05-20T16:10:59.621 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:10:59.620+0000 7fc52b852700 1 -- 172.21.15.82:0/2946680082 wait complete. 2024-05-20T16:10:59.635 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:10:59 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-20T16:10:59.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:10:59 smithi184 ceph-mon[36209]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail 2024-05-20T16:11:00.327 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":81,"num_osds":12,"num_up_osds":12,"osd_up_since":1716221459,"num_in_osds":12,"osd_in_since":1716221439,"num_remapped_pgs":0} 2024-05-20T16:11:00.327 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell --fsid 59d1600a-16c2-11ef-bc9a-c7b262605968 -- ceph osd dump --format=json 2024-05-20T16:11:00.385 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:00 smithi184 ceph-mon[36209]: purged_snaps scrub starts 2024-05-20T16:11:00.386 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:00 smithi184 ceph-mon[36209]: purged_snaps scrub ok 2024-05-20T16:11:00.386 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:00 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-20T16:11:00.386 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:00 smithi184 ceph-mon[36209]: osd.11 [v2:172.21.15.184:6824/3594364650,v1:172.21.15.184:6825/3594364650] boot 2024-05-20T16:11:00.386 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:00 smithi184 ceph-mon[36209]: osdmap e81: 12 total, 12 up, 12 in 2024-05-20T16:11:00.386 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:00 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-20T16:11:00.386 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:00 smithi184 ceph-mon[36209]: from='client.? 172.21.15.82:0/2946680082' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-20T16:11:00.562 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:00 smithi160 ceph-mon[36347]: purged_snaps scrub starts 2024-05-20T16:11:00.562 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:00 smithi160 ceph-mon[36347]: purged_snaps scrub ok 2024-05-20T16:11:00.562 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:00 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-20T16:11:00.562 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:00 smithi160 ceph-mon[36347]: osd.11 [v2:172.21.15.184:6824/3594364650,v1:172.21.15.184:6825/3594364650] boot 2024-05-20T16:11:00.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:00 smithi160 ceph-mon[36347]: osdmap e81: 12 total, 12 up, 12 in 2024-05-20T16:11:00.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:00 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-20T16:11:00.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:00 smithi160 ceph-mon[36347]: from='client.? 172.21.15.82:0/2946680082' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-20T16:11:00.583 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T16:11:00.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:00 smithi082 ceph-mon[32238]: purged_snaps scrub starts 2024-05-20T16:11:00.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:00 smithi082 ceph-mon[32238]: purged_snaps scrub ok 2024-05-20T16:11:00.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:00 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-20T16:11:00.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:00 smithi082 ceph-mon[32238]: osd.11 [v2:172.21.15.184:6824/3594364650,v1:172.21.15.184:6825/3594364650] boot 2024-05-20T16:11:00.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:00 smithi082 ceph-mon[32238]: osdmap e81: 12 total, 12 up, 12 in 2024-05-20T16:11:00.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:00 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-20T16:11:00.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:00 smithi082 ceph-mon[32238]: from='client.? 172.21.15.82:0/2946680082' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-20T16:11:01.577 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:01 smithi082 ceph-mon[32238]: osdmap e82: 12 total, 12 up, 12 in 2024-05-20T16:11:01.577 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:01 smithi082 ceph-mon[32238]: Detected new or changed devices on smithi184 2024-05-20T16:11:01.577 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:01 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:11:01.577 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:01 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:11:01.578 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:01 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-20T16:11:01.578 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:01 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-20T16:11:01.578 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:01 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-20T16:11:01.578 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:01 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-20T16:11:01.578 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:01 smithi082 ceph-mon[32238]: Adjusting osd_memory_target on smithi184 to 4272M 2024-05-20T16:11:01.578 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:01 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:11:01.578 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:01 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:11:01.578 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:01 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T16:11:01.578 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:01 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:11:01.578 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:01 smithi082 ceph-mon[32238]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-20T16:11:01.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:01 smithi184 ceph-mon[36209]: osdmap e82: 12 total, 12 up, 12 in 2024-05-20T16:11:01.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:01 smithi184 ceph-mon[36209]: Detected new or changed devices on smithi184 2024-05-20T16:11:01.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:01 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:11:01.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:01 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:11:01.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:01 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-20T16:11:01.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:01 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-20T16:11:01.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:01 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-20T16:11:01.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:01 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-20T16:11:01.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:01 smithi184 ceph-mon[36209]: Adjusting osd_memory_target on smithi184 to 4272M 2024-05-20T16:11:01.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:01 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:11:01.637 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:01 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:11:01.637 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:01 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T16:11:01.637 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:01 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:11:01.637 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:01 smithi184 ceph-mon[36209]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-20T16:11:01.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:01 smithi160 ceph-mon[36347]: osdmap e82: 12 total, 12 up, 12 in 2024-05-20T16:11:01.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:01 smithi160 ceph-mon[36347]: Detected new or changed devices on smithi184 2024-05-20T16:11:01.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:01 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:11:01.813 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:01 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:11:01.813 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:01 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-20T16:11:01.813 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:01 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-20T16:11:01.813 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:01 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-20T16:11:01.813 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:01 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-20T16:11:01.813 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:01 smithi160 ceph-mon[36347]: Adjusting osd_memory_target on smithi184 to 4272M 2024-05-20T16:11:01.813 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:01 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:11:01.813 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:01 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:11:01.813 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:01 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T16:11:01.813 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:01 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:11:01.813 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:01 smithi160 ceph-mon[36347]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-20T16:11:02.227 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:02.221+0000 7f5b7d3a9700 1 -- 172.21.15.82:0/1516317813 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5b78107d80 msgr2=0x7f5b78108190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:11:02.228 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:02.221+0000 7f5b7d3a9700 1 --2- 172.21.15.82:0/1516317813 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5b78107d80 0x7f5b78108190 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f5b600099c0 tx=0x7f5b60009cd0 comp rx=0 tx=0).stop 2024-05-20T16:11:02.228 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:02.221+0000 7f5b7d3a9700 1 -- 172.21.15.82:0/1516317813 shutdown_connections 2024-05-20T16:11:02.228 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:02.221+0000 7f5b7d3a9700 1 --2- 172.21.15.82:0/1516317813 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5b781086d0 0x7f5b78110030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:02.228 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:02.221+0000 7f5b7d3a9700 1 --2- 172.21.15.82:0/1516317813 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5b78103450 0x7f5b78103860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:02.228 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:02.221+0000 7f5b7d3a9700 1 --2- 172.21.15.82:0/1516317813 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5b78107d80 0x7f5b78108190 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:02.228 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:02.221+0000 7f5b7d3a9700 1 -- 172.21.15.82:0/1516317813 >> 172.21.15.82:0/1516317813 conn(0x7f5b780fed20 msgr2=0x7f5b78101170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:11:02.228 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:02.221+0000 7f5b7d3a9700 1 -- 172.21.15.82:0/1516317813 shutdown_connections 2024-05-20T16:11:02.228 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:02.221+0000 7f5b7d3a9700 1 -- 172.21.15.82:0/1516317813 wait complete. 2024-05-20T16:11:02.228 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:02.222+0000 7f5b7d3a9700 1 Processor -- start 2024-05-20T16:11:02.228 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:02.222+0000 7f5b7d3a9700 1 -- start start 2024-05-20T16:11:02.228 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:02.222+0000 7f5b7d3a9700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5b78103450 0x7f5b78199d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:11:02.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:02.222+0000 7f5b7d3a9700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5b78107d80 0x7f5b7819a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:11:02.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:02.222+0000 7f5b7d3a9700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5b781086d0 0x7f5b7819e590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:11:02.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:02.222+0000 7f5b7d3a9700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f5b78113430 con 0x7f5b78103450 2024-05-20T16:11:02.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:02.222+0000 7f5b7d3a9700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f5b781132b0 con 0x7f5b78107d80 2024-05-20T16:11:02.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:02.223+0000 7f5b7d3a9700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f5b781135b0 con 0x7f5b781086d0 2024-05-20T16:11:02.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:02.223+0000 7f5b76ffd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5b78103450 0x7f5b78199d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:11:02.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:02.223+0000 7f5b76ffd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5b78103450 0x7f5b78199d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:45920/0 (socket says 172.21.15.82:45920) 2024-05-20T16:11:02.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:02.223+0000 7f5b76ffd700 1 -- 172.21.15.82:0/757408655 learned_addr learned my addr 172.21.15.82:0/757408655 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:11:02.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:02.223+0000 7f5b777fe700 1 --2- 172.21.15.82:0/757408655 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5b781086d0 0x7f5b7819e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:11:02.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:02.223+0000 7f5b6ffff700 1 --2- 172.21.15.82:0/757408655 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5b78107d80 0x7f5b7819a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:11:02.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:02.223+0000 7f5b76ffd700 1 -- 172.21.15.82:0/757408655 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5b781086d0 msgr2=0x7f5b7819e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:11:02.230 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:02.223+0000 7f5b76ffd700 1 --2- 172.21.15.82:0/757408655 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5b781086d0 0x7f5b7819e590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:02.230 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:02.223+0000 7f5b76ffd700 1 -- 172.21.15.82:0/757408655 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5b78107d80 msgr2=0x7f5b7819a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:11:02.230 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:02.223+0000 7f5b76ffd700 1 --2- 172.21.15.82:0/757408655 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5b78107d80 0x7f5b7819a300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:02.230 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:02.223+0000 7f5b76ffd700 1 -- 172.21.15.82:0/757408655 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b7819ec30 con 0x7f5b78103450 2024-05-20T16:11:02.230 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:02.223+0000 7f5b777fe700 1 --2- 172.21.15.82:0/757408655 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5b781086d0 0x7f5b7819e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T16:11:02.230 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:02.223+0000 7f5b6ffff700 1 --2- 172.21.15.82:0/757408655 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5b78107d80 0x7f5b7819a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T16:11:02.230 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:02.223+0000 7f5b76ffd700 1 --2- 172.21.15.82:0/757408655 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5b78103450 0x7f5b78199d60 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f5b6800d950 tx=0x7f5b6800dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:11:02.230 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:02.223+0000 7f5b74ff9700 1 -- 172.21.15.82:0/757408655 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b68016070 con 0x7f5b78103450 2024-05-20T16:11:02.230 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:02.223+0000 7f5b7d3a9700 1 -- 172.21.15.82:0/757408655 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b7819ef20 con 0x7f5b78103450 2024-05-20T16:11:02.230 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:02.223+0000 7f5b74ff9700 1 -- 172.21.15.82:0/757408655 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b68009b40 con 0x7f5b78103450 2024-05-20T16:11:02.230 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:02.223+0000 7f5b74ff9700 1 -- 172.21.15.82:0/757408655 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b6800f620 con 0x7f5b78103450 2024-05-20T16:11:02.230 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:02.223+0000 7f5b7d3a9700 1 -- 172.21.15.82:0/757408655 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b781a6840 con 0x7f5b78103450 2024-05-20T16:11:02.231 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:02.224+0000 7f5b7d3a9700 1 -- 172.21.15.82:0/757408655 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b78050170 con 0x7f5b78103450 2024-05-20T16:11:02.231 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:02.225+0000 7f5b74ff9700 1 -- 172.21.15.82:0/757408655 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f5b68019070 con 0x7f5b78103450 2024-05-20T16:11:02.231 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:02.226+0000 7f5b74ff9700 1 --2- 172.21.15.82:0/757408655 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f5b640a2c20 0x7f5b640a5130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:11:02.231 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:02.226+0000 7f5b6ffff700 1 --2- 172.21.15.82:0/757408655 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f5b640a2c20 0x7f5b640a5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:11:02.231 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:02.226+0000 7f5b74ff9700 1 -- 172.21.15.82:0/757408655 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f5b680cebc0 con 0x7f5b78103450 2024-05-20T16:11:02.231 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:02.227+0000 7f5b6ffff700 1 --2- 172.21.15.82:0/757408655 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f5b640a2c20 0x7f5b640a5130 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f5b60006010 tx=0x7f5b60005e80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:11:02.231 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:02.230+0000 7f5b74ff9700 1 -- 172.21.15.82:0/757408655 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f5b6809bfc0 con 0x7f5b78103450 2024-05-20T16:11:02.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:02.425+0000 7f5b7d3a9700 1 -- 172.21.15.82:0/757408655 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5b7802dc60 con 0x7f5b78103450 2024-05-20T16:11:02.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:02.426+0000 7f5b74ff9700 1 -- 172.21.15.82:0/757408655 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+18481 (secure 0 0 0) 0x7f5b6809b8c0 con 0x7f5b78103450 2024-05-20T16:11:02.426 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-05-20T16:11:02.426 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":82,"fsid":"59d1600a-16c2-11ef-bc9a-c7b262605968","created":"2024-05-20T16:03:46.611556+0000","modified":"2024-05-20T16:11:00.310642+0000","last_up_change":"2024-05-20T16:10:59.303479+0000","last_in_change":"2024-05-20T16:10:39.063561+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-20T16:07:09.493727+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bae842a9-03ec-490e-b56d-f7957235cdc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":1484330167},{"type":"v1","addr":"172.21.15.82:6803","nonce":1484330167}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":1484330167},{"type":"v1","addr":"172.21.15.82:6805","nonce":1484330167}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1484330167},{"type":"v1","addr":"172.21.15.82:6809","nonce":1484330167}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":1484330167},{"type":"v1","addr":"172.21.15.82:6807","nonce":1484330167}]},"public_addr":"172.21.15.82:6803/1484330167","cluster_addr":"172.21.15.82:6805/1484330167","heartbeat_back_addr":"172.21.15.82:6809/1484330167","heartbeat_front_addr":"172.21.15.82:6807/1484330167","state":["exists","up"]},{"osd":1,"uuid":"59f6a2ba-b5de-4161-81c5-92a417abacdf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":1457003568},{"type":"v1","addr":"172.21.15.160:6801","nonce":1457003568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":1457003568},{"type":"v1","addr":"172.21.15.160:6803","nonce":1457003568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6806","nonce":1457003568},{"type":"v1","addr":"172.21.15.160:6807","nonce":1457003568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":1457003568},{"type":"v1","addr":"172.21.15.160:6805","nonce":1457003568}]},"public_addr":"172.21.15.160:6801/1457003568","cluster_addr":"172.21.15.160:6803/1457003568","heartbeat_back_addr":"172.21.15.160:6807/1457003568","heartbeat_front_addr":"172.21.15.160:6805/1457003568","state":["exists","up"]},{"osd":2,"uuid":"3bbb5587-b449-41f3-bd26-67ec3d6afdfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":4051293934},{"type":"v1","addr":"172.21.15.184:6801","nonce":4051293934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":4051293934},{"type":"v1","addr":"172.21.15.184:6803","nonce":4051293934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":4051293934},{"type":"v1","addr":"172.21.15.184:6807","nonce":4051293934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":4051293934},{"type":"v1","addr":"172.21.15.184:6805","nonce":4051293934}]},"public_addr":"172.21.15.184:6801/4051293934","cluster_addr":"172.21.15.184:6803/4051293934","heartbeat_back_addr":"172.21.15.184:6807/4051293934","heartbeat_front_addr":"172.21.15.184:6805/4051293934","state":["exists","up"]},{"osd":3,"uuid":"ca651f90-09b8-4081-a6f9-40b31268a837","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":3947485599},{"type":"v1","addr":"172.21.15.82:6811","nonce":3947485599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":3947485599},{"type":"v1","addr":"172.21.15.82:6813","nonce":3947485599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":3947485599},{"type":"v1","addr":"172.21.15.82:6817","nonce":3947485599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":3947485599},{"type":"v1","addr":"172.21.15.82:6815","nonce":3947485599}]},"public_addr":"172.21.15.82:6811/3947485599","cluster_addr":"172.21.15.82:6813/3947485599","heartbeat_back_addr":"172.21.15.82:6817/3947485599","heartbeat_front_addr":"172.21.15.82:6815/3947485599","state":["exists","up"]},{"osd":4,"uuid":"cb3dfd56-f677-40cb-9c96-8319ea5ac788","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":4160830014},{"type":"v1","addr":"172.21.15.160:6809","nonce":4160830014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":4160830014},{"type":"v1","addr":"172.21.15.160:6811","nonce":4160830014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":4160830014},{"type":"v1","addr":"172.21.15.160:6815","nonce":4160830014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6812","nonce":4160830014},{"type":"v1","addr":"172.21.15.160:6813","nonce":4160830014}]},"public_addr":"172.21.15.160:6809/4160830014","cluster_addr":"172.21.15.160:6811/4160830014","heartbeat_back_addr":"172.21.15.160:6815/4160830014","heartbeat_front_addr":"172.21.15.160:6813/4160830014","state":["exists","up"]},{"osd":5,"uuid":"c70cfcd1-a4ff-461f-8c73-479b061200fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":1046282672},{"type":"v1","addr":"172.21.15.184:6809","nonce":1046282672}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":1046282672},{"type":"v1","addr":"172.21.15.184:6811","nonce":1046282672}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":1046282672},{"type":"v1","addr":"172.21.15.184:6815","nonce":1046282672}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":1046282672},{"type":"v1","addr":"172.21.15.184:6813","nonce":1046282672}]},"public_addr":"172.21.15.184:6809/1046282672","cluster_addr":"172.21.15.184:6811/1046282672","heartbeat_back_addr":"172.21.15.184:6815/1046282672","heartbeat_front_addr":"172.21.15.184:6813/1046282672","state":["exists","up"]},{"osd":6,"uuid":"d47ce001-9dd1-4830-bcc9-888601187c6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":2789736499},{"type":"v1","addr":"172.21.15.82:6819","nonce":2789736499}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":2789736499},{"type":"v1","addr":"172.21.15.82:6821","nonce":2789736499}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":2789736499},{"type":"v1","addr":"172.21.15.82:6825","nonce":2789736499}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":2789736499},{"type":"v1","addr":"172.21.15.82:6823","nonce":2789736499}]},"public_addr":"172.21.15.82:6819/2789736499","cluster_addr":"172.21.15.82:6821/2789736499","heartbeat_back_addr":"172.21.15.82:6825/2789736499","heartbeat_front_addr":"172.21.15.82:6823/2789736499","state":["exists","up"]},{"osd":7,"uuid":"25c54f43-18cb-41e4-83b4-2732b5a93889","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6816","nonce":2340683378},{"type":"v1","addr":"172.21.15.160:6817","nonce":2340683378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":2340683378},{"type":"v1","addr":"172.21.15.160:6819","nonce":2340683378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":2340683378},{"type":"v1","addr":"172.21.15.160:6823","nonce":2340683378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6820","nonce":2340683378},{"type":"v1","addr":"172.21.15.160:6821","nonce":2340683378}]},"public_addr":"172.21.15.160:6817/2340683378","cluster_addr":"172.21.15.160:6819/2340683378","heartbeat_back_addr":"172.21.15.160:6823/2340683378","heartbeat_front_addr":"172.21.15.160:6821/2340683378","state":["exists","up"]},{"osd":8,"uuid":"1f8b478d-287f-45f5-8526-d1a58c5bc07c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":3831576372},{"type":"v1","addr":"172.21.15.184:6817","nonce":3831576372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":3831576372},{"type":"v1","addr":"172.21.15.184:6819","nonce":3831576372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":3831576372},{"type":"v1","addr":"172.21.15.184:6823","nonce":3831576372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":3831576372},{"type":"v1","addr":"172.21.15.184:6821","nonce":3831576372}]},"public_addr":"172.21.15.184:6817/3831576372","cluster_addr":"172.21.15.184:6819/3831576372","heartbeat_back_addr":"172.21.15.184:6823/3831576372","heartbeat_front_addr":"172.21.15.184:6821/3831576372","state":["exists","up"]},{"osd":9,"uuid":"0d69e319-954f-4b18-95ea-31794f18dc3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":716330782},{"type":"v1","addr":"172.21.15.82:6827","nonce":716330782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":716330782},{"type":"v1","addr":"172.21.15.82:6829","nonce":716330782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":716330782},{"type":"v1","addr":"172.21.15.82:6833","nonce":716330782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":716330782},{"type":"v1","addr":"172.21.15.82:6831","nonce":716330782}]},"public_addr":"172.21.15.82:6827/716330782","cluster_addr":"172.21.15.82:6829/716330782","heartbeat_back_addr":"172.21.15.82:6833/716330782","heartbeat_front_addr":"172.21.15.82:6831/716330782","state":["exists","up"]},{"osd":10,"uuid":"39dc1b78-61cb-42ca-b5b6-1e44ac2b6500","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":2327428014},{"type":"v1","addr":"172.21.15.160:6825","nonce":2327428014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":2327428014},{"type":"v1","addr":"172.21.15.160:6827","nonce":2327428014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":2327428014},{"type":"v1","addr":"172.21.15.160:6831","nonce":2327428014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":2327428014},{"type":"v1","addr":"172.21.15.160:6829","nonce":2327428014}]},"public_addr":"172.21.15.160:6825/2327428014","cluster_addr":"172.21.15.160:6827/2327428014","heartbeat_back_addr":"172.21.15.160:6831/2327428014","heartbeat_front_addr":"172.21.15.160:6829/2327428014","state":["exists","up"]},{"osd":11,"uuid":"4306b3c1-707a-4010-987c-209ea0e948e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":3594364650},{"type":"v1","addr":"172.21.15.184:6825","nonce":3594364650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":3594364650},{"type":"v1","addr":"172.21.15.184:6827","nonce":3594364650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3594364650},{"type":"v1","addr":"172.21.15.184:6831","nonce":3594364650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3594364650},{"type":"v1","addr":"172.21.15.184:6829","nonce":3594364650}]},"public_addr":"172.21.15.184:6825/3594364650","cluster_addr":"172.21.15.184:6827/3594364650","heartbeat_back_addr":"172.21.15.184:6831/3594364650","heartbeat_front_addr":"172.21.15.184:6829/3594364650","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:06:17.855998+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:06:41.473270+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:07:05.304108+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:07:29.887472+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:07:55.387286+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:08:20.579669+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:08:46.333051+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:09:12.289668+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:09:38.180696+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:10:04.471459+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:10:31.041912+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:10:57.477266+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.82:0/2005424509":"2024-05-21T16:04:45.161949+0000","172.21.15.82:6801/1763049308":"2024-05-21T16:04:45.161949+0000","172.21.15.82:6801/1959459348":"2024-05-21T16:04:19.208291+0000","172.21.15.82:0/3131904556":"2024-05-21T16:04:19.208291+0000","172.21.15.82:6800/1959459348":"2024-05-21T16:04:19.208291+0000","172.21.15.82:0/2456966342":"2024-05-21T16:04:45.161949+0000","172.21.15.82:0/4179978833":"2024-05-21T16:04:19.208291+0000","172.21.15.82:0/1602862225":"2024-05-21T16:04:45.161949+0000","172.21.15.82:0/106101180":"2024-05-21T16:04:19.208291+0000","172.21.15.82:6800/1763049308":"2024-05-21T16:04:45.161949+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-20T16:11:02.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:02.428+0000 7f5b7d3a9700 1 -- 172.21.15.82:0/757408655 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f5b640a2c20 msgr2=0x7f5b640a5130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:11:02.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:02.428+0000 7f5b7d3a9700 1 --2- 172.21.15.82:0/757408655 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f5b640a2c20 0x7f5b640a5130 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f5b60006010 tx=0x7f5b60005e80 comp rx=0 tx=0).stop 2024-05-20T16:11:02.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:02.428+0000 7f5b7d3a9700 1 -- 172.21.15.82:0/757408655 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5b78103450 msgr2=0x7f5b78199d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:11:02.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:02.428+0000 7f5b7d3a9700 1 --2- 172.21.15.82:0/757408655 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5b78103450 0x7f5b78199d60 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f5b6800d950 tx=0x7f5b6800dd10 comp rx=0 tx=0).stop 2024-05-20T16:11:02.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:02.428+0000 7f5b7d3a9700 1 -- 172.21.15.82:0/757408655 shutdown_connections 2024-05-20T16:11:02.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:02.428+0000 7f5b7d3a9700 1 --2- 172.21.15.82:0/757408655 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f5b640a2c20 0x7f5b640a5130 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:02.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:02.428+0000 7f5b7d3a9700 1 --2- 172.21.15.82:0/757408655 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5b781086d0 0x7f5b7819e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:02.429 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:02.428+0000 7f5b7d3a9700 1 --2- 172.21.15.82:0/757408655 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5b78103450 0x7f5b78199d60 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:02.429 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:02.428+0000 7f5b7d3a9700 1 --2- 172.21.15.82:0/757408655 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5b78107d80 0x7f5b7819a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:02.429 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:02.428+0000 7f5b7d3a9700 1 -- 172.21.15.82:0/757408655 >> 172.21.15.82:0/757408655 conn(0x7f5b780fed20 msgr2=0x7f5b78107520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:11:02.429 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:02.428+0000 7f5b7d3a9700 1 -- 172.21.15.82:0/757408655 shutdown_connections 2024-05-20T16:11:02.429 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:02.428+0000 7f5b7d3a9700 1 -- 172.21.15.82:0/757408655 wait complete. 2024-05-20T16:11:02.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:02 smithi160 ceph-mon[36347]: from='client.? 172.21.15.82:0/757408655' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-20T16:11:02.838 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:02 smithi082 ceph-mon[32238]: from='client.? 172.21.15.82:0/757408655' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-20T16:11:02.886 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:02 smithi184 ceph-mon[36209]: from='client.? 172.21.15.82:0/757408655' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-20T16:11:03.068 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-20T16:07:09.493727+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-20T16:11:03.069 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell --fsid 59d1600a-16c2-11ef-bc9a-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-05-20T16:11:03.309 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T16:11:03.692 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:03 smithi082 ceph-mon[32238]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-20T16:11:03.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:03 smithi160 ceph-mon[36347]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-20T16:11:03.886 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:03 smithi184 ceph-mon[36209]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-20T16:11:05.138 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:05.138+0000 7f91c315a700 1 -- 172.21.15.82:0/4133227163 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f91bc106c70 msgr2=0x7f91bc107080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:11:05.139 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:05.138+0000 7f91c315a700 1 --2- 172.21.15.82:0/4133227163 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f91bc106c70 0x7f91bc107080 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f91a80099c0 tx=0x7f91a8009cd0 comp rx=0 tx=0).stop 2024-05-20T16:11:05.139 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:05.138+0000 7f91c315a700 1 -- 172.21.15.82:0/4133227163 shutdown_connections 2024-05-20T16:11:05.139 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:05.138+0000 7f91c315a700 1 --2- 172.21.15.82:0/4133227163 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f91bc1075c0 0x7f91bc10ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:05.139 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:05.138+0000 7f91c315a700 1 --2- 172.21.15.82:0/4133227163 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f91bc102340 0x7f91bc102750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:05.139 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:05.138+0000 7f91c315a700 1 --2- 172.21.15.82:0/4133227163 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f91bc106c70 0x7f91bc107080 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:05.139 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:05.138+0000 7f91c315a700 1 -- 172.21.15.82:0/4133227163 >> 172.21.15.82:0/4133227163 conn(0x7f91bc0fdc40 msgr2=0x7f91bc100060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:11:05.139 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:05.138+0000 7f91c315a700 1 -- 172.21.15.82:0/4133227163 shutdown_connections 2024-05-20T16:11:05.139 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:05.138+0000 7f91c315a700 1 -- 172.21.15.82:0/4133227163 wait complete. 2024-05-20T16:11:05.141 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:05.139+0000 7f91c315a700 1 Processor -- start 2024-05-20T16:11:05.141 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:05.139+0000 7f91c315a700 1 -- start start 2024-05-20T16:11:05.141 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:05.139+0000 7f91c315a700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f91bc102340 0x7f91bc19e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:11:05.141 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:05.139+0000 7f91c315a700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f91bc106c70 0x7f91bc19eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:11:05.141 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:05.139+0000 7f91c315a700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f91bc1075c0 0x7f91bc198a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:11:05.142 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:05.139+0000 7f91c315a700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f91bc112320 con 0x7f91bc1075c0 2024-05-20T16:11:05.142 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:05.139+0000 7f91c315a700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f91bc1121a0 con 0x7f91bc106c70 2024-05-20T16:11:05.142 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:05.139+0000 7f91c315a700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f91bc1124a0 con 0x7f91bc102340 2024-05-20T16:11:05.142 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:05.139+0000 7f91c16f7700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f91bc1075c0 0x7f91bc198a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:11:05.142 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:05.139+0000 7f91c16f7700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f91bc1075c0 0x7f91bc198a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:45950/0 (socket says 172.21.15.82:45950) 2024-05-20T16:11:05.142 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:05.139+0000 7f91c16f7700 1 -- 172.21.15.82:0/4111659958 learned_addr learned my addr 172.21.15.82:0/4111659958 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:11:05.142 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:05.140+0000 7f91bbfff700 1 --2- 172.21.15.82:0/4111659958 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f91bc106c70 0x7f91bc19eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:11:05.142 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:05.140+0000 7f91c0ef6700 1 --2- 172.21.15.82:0/4111659958 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f91bc102340 0x7f91bc19e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:11:05.142 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:05.140+0000 7f91c16f7700 1 -- 172.21.15.82:0/4111659958 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f91bc102340 msgr2=0x7f91bc19e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:11:05.142 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:05.140+0000 7f91c16f7700 1 --2- 172.21.15.82:0/4111659958 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f91bc102340 0x7f91bc19e860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:05.142 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:05.140+0000 7f91c16f7700 1 -- 172.21.15.82:0/4111659958 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f91bc106c70 msgr2=0x7f91bc19eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:11:05.142 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:05.140+0000 7f91c16f7700 1 --2- 172.21.15.82:0/4111659958 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f91bc106c70 0x7f91bc19eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:05.143 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:05.140+0000 7f91c16f7700 1 -- 172.21.15.82:0/4111659958 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91bc1990c0 con 0x7f91bc1075c0 2024-05-20T16:11:05.143 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:05.140+0000 7f91c16f7700 1 --2- 172.21.15.82:0/4111659958 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f91bc1075c0 0x7f91bc198a80 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f91b400ef30 tx=0x7f91b400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:11:05.143 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:05.140+0000 7f91b9ffb700 1 -- 172.21.15.82:0/4111659958 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91b40092e0 con 0x7f91bc1075c0 2024-05-20T16:11:05.143 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:05.140+0000 7f91b9ffb700 1 -- 172.21.15.82:0/4111659958 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f91b4004bb0 con 0x7f91bc1075c0 2024-05-20T16:11:05.143 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:05.140+0000 7f91c315a700 1 -- 172.21.15.82:0/4111659958 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91bc199350 con 0x7f91bc1075c0 2024-05-20T16:11:05.143 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:05.140+0000 7f91b9ffb700 1 -- 172.21.15.82:0/4111659958 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91b4004280 con 0x7f91bc1075c0 2024-05-20T16:11:05.143 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:05.140+0000 7f91c315a700 1 -- 172.21.15.82:0/4111659958 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91bc199860 con 0x7f91bc1075c0 2024-05-20T16:11:05.143 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:05.141+0000 7f91c315a700 1 -- 172.21.15.82:0/4111659958 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91bc050170 con 0x7f91bc1075c0 2024-05-20T16:11:05.144 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:05.142+0000 7f91b9ffb700 1 -- 172.21.15.82:0/4111659958 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f91b401b030 con 0x7f91bc1075c0 2024-05-20T16:11:05.144 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:05.143+0000 7f91b9ffb700 1 --2- 172.21.15.82:0/4111659958 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f91ac0a2ca0 0x7f91ac0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:11:05.144 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:05.143+0000 7f91c0ef6700 1 --2- 172.21.15.82:0/4111659958 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f91ac0a2ca0 0x7f91ac0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:11:05.144 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:05.143+0000 7f91b9ffb700 1 -- 172.21.15.82:0/4111659958 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f91b40ce5b0 con 0x7f91bc1075c0 2024-05-20T16:11:05.144 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:05.144+0000 7f91c0ef6700 1 --2- 172.21.15.82:0/4111659958 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f91ac0a2ca0 0x7f91ac0a51b0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f91bc199bc0 tx=0x7f91b000a380 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:11:05.147 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:05.148+0000 7f91b9ffb700 1 -- 172.21.15.82:0/4111659958 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f91b4022020 con 0x7f91bc1075c0 2024-05-20T16:11:05.344 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:05.344+0000 7f91c315a700 1 -- 172.21.15.82:0/4111659958 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f91bc02dc60 con 0x7f91bc1075c0 2024-05-20T16:11:05.344 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:05.344+0000 7f91b9ffb700 1 -- 172.21.15.82:0/4111659958 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) ==== 93+0+10 (secure 0 0 0) 0x7f91b4010040 con 0x7f91bc1075c0 2024-05-20T16:11:05.344 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 1 2024-05-20T16:11:05.347 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:05.347+0000 7f91c315a700 1 -- 172.21.15.82:0/4111659958 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f91ac0a2ca0 msgr2=0x7f91ac0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:11:05.347 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:05.347+0000 7f91c315a700 1 --2- 172.21.15.82:0/4111659958 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f91ac0a2ca0 0x7f91ac0a51b0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f91bc199bc0 tx=0x7f91b000a380 comp rx=0 tx=0).stop 2024-05-20T16:11:05.347 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:05.347+0000 7f91c315a700 1 -- 172.21.15.82:0/4111659958 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f91bc1075c0 msgr2=0x7f91bc198a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:11:05.347 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:05.347+0000 7f91c315a700 1 --2- 172.21.15.82:0/4111659958 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f91bc1075c0 0x7f91bc198a80 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f91b400ef30 tx=0x7f91b400c3a0 comp rx=0 tx=0).stop 2024-05-20T16:11:05.348 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:05.347+0000 7f91c315a700 1 -- 172.21.15.82:0/4111659958 shutdown_connections 2024-05-20T16:11:05.348 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:05.347+0000 7f91c315a700 1 --2- 172.21.15.82:0/4111659958 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f91ac0a2ca0 0x7f91ac0a51b0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:05.348 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:05.347+0000 7f91c315a700 1 --2- 172.21.15.82:0/4111659958 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f91bc1075c0 0x7f91bc198a80 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:05.348 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:05.347+0000 7f91c315a700 1 --2- 172.21.15.82:0/4111659958 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f91bc102340 0x7f91bc19e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:05.348 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:05.347+0000 7f91c315a700 1 --2- 172.21.15.82:0/4111659958 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f91bc106c70 0x7f91bc19eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:05.348 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:05.347+0000 7f91c315a700 1 -- 172.21.15.82:0/4111659958 >> 172.21.15.82:0/4111659958 conn(0x7f91bc0fdc40 msgr2=0x7f91bc108520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:11:05.348 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:05.347+0000 7f91c315a700 1 -- 172.21.15.82:0/4111659958 shutdown_connections 2024-05-20T16:11:05.348 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:05.347+0000 7f91c315a700 1 -- 172.21.15.82:0/4111659958 wait complete. 2024-05-20T16:11:06.101 INFO:tasks.cephadm:Adding mds.a on smithi082 2024-05-20T16:11:06.102 INFO:tasks.cephadm:Adding mds.d on smithi082 2024-05-20T16:11:06.102 INFO:tasks.cephadm:Adding mds.g on smithi082 2024-05-20T16:11:06.102 INFO:tasks.cephadm:Adding mds.j on smithi082 2024-05-20T16:11:06.102 INFO:tasks.cephadm:Adding mds.b on smithi160 2024-05-20T16:11:06.102 INFO:tasks.cephadm:Adding mds.e on smithi160 2024-05-20T16:11:06.102 INFO:tasks.cephadm:Adding mds.h on smithi160 2024-05-20T16:11:06.102 INFO:tasks.cephadm:Adding mds.k on smithi160 2024-05-20T16:11:06.102 INFO:tasks.cephadm:Adding mds.c on smithi184 2024-05-20T16:11:06.102 INFO:tasks.cephadm:Adding mds.f on smithi184 2024-05-20T16:11:06.102 INFO:tasks.cephadm:Adding mds.i on smithi184 2024-05-20T16:11:06.102 INFO:tasks.cephadm:Adding mds.l on smithi184 2024-05-20T16:11:06.103 DEBUG:teuthology.orchestra.run.smithi184:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 59d1600a-16c2-11ef-bc9a-c7b262605968 -- ceph orch apply mds all '12;smithi082=a;smithi082=d;smithi082=g;smithi082=j;smithi160=b;smithi160=e;smithi160=h;smithi160=k;smithi184=c;smithi184=f;smithi184=i;smithi184=l' 2024-05-20T16:11:06.350 INFO:teuthology.orchestra.run.smithi184.stderr:Inferring config /var/lib/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/mon.c/config 2024-05-20T16:11:06.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:06 smithi082 ceph-mon[32238]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-20T16:11:06.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:06 smithi082 ceph-mon[32238]: from='client.? 172.21.15.82:0/4111659958' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-20T16:11:06.386 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:06 smithi184 ceph-mon[36209]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-20T16:11:06.386 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:06 smithi184 ceph-mon[36209]: from='client.? 172.21.15.82:0/4111659958' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-20T16:11:06.562 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:06 smithi160 ceph-mon[36347]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-20T16:11:06.562 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:06 smithi160 ceph-mon[36347]: from='client.? 172.21.15.82:0/4111659958' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-20T16:11:07.562 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:07 smithi160 ceph-mon[36347]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-20T16:11:07.606 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:07 smithi184 ceph-mon[36209]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-20T16:11:07.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:07 smithi082 ceph-mon[32238]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-20T16:11:07.977 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:11:07.974+0000 7f14282b9700 1 -- 172.21.15.184:0/4215904104 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1420103bb0 msgr2=0x7f142010a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:11:07.977 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:11:07.974+0000 7f14282b9700 1 --2- 172.21.15.184:0/4215904104 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1420103bb0 0x7f142010a460 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f141800b0a0 tx=0x7f141800b3b0 comp rx=0 tx=0).stop 2024-05-20T16:11:07.977 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:11:07.974+0000 7f14282b9700 1 -- 172.21.15.184:0/4215904104 shutdown_connections 2024-05-20T16:11:07.977 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:11:07.974+0000 7f14282b9700 1 --2- 172.21.15.184:0/4215904104 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f14201020e0 0x7f14201024f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:07.977 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:11:07.974+0000 7f14282b9700 1 --2- 172.21.15.184:0/4215904104 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1420103bb0 0x7f142010a460 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:07.977 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:11:07.974+0000 7f14282b9700 1 --2- 172.21.15.184:0/4215904104 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1420103210 0x7f1420103620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:07.978 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:11:07.974+0000 7f14282b9700 1 -- 172.21.15.184:0/4215904104 >> 172.21.15.184:0/4215904104 conn(0x7f14200fd740 msgr2=0x7f14200ffbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:11:07.978 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:11:07.974+0000 7f14282b9700 1 -- 172.21.15.184:0/4215904104 shutdown_connections 2024-05-20T16:11:07.978 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:11:07.974+0000 7f14282b9700 1 -- 172.21.15.184:0/4215904104 wait complete. 2024-05-20T16:11:07.978 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:11:07.975+0000 7f14282b9700 1 Processor -- start 2024-05-20T16:11:07.978 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:11:07.975+0000 7f14282b9700 1 -- start start 2024-05-20T16:11:07.978 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:11:07.975+0000 7f14282b9700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f14201020e0 0x7f1420075330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:11:07.978 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:11:07.975+0000 7f14282b9700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1420103210 0x7f142019e570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:11:07.978 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:11:07.975+0000 7f14282b9700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1420103bb0 0x7f14201a0b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:11:07.978 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:11:07.975+0000 7f14282b9700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f142010dcb0 con 0x7f1420103210 2024-05-20T16:11:07.978 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:11:07.975+0000 7f14282b9700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f142010db30 con 0x7f1420103bb0 2024-05-20T16:11:07.978 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:11:07.975+0000 7f14282b9700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f142010de30 con 0x7f14201020e0 2024-05-20T16:11:07.978 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:11:07.975+0000 7f1426055700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f14201020e0 0x7f1420075330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:11:07.979 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:11:07.975+0000 7f1426055700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f14201020e0 0x7f1420075330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.184:53926/0 (socket says 172.21.15.184:53926) 2024-05-20T16:11:07.979 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:11:07.975+0000 7f1426055700 1 -- 172.21.15.184:0/3916712930 learned_addr learned my addr 172.21.15.184:0/3916712930 (peer_addr_for_me v2:172.21.15.184:0/0) 2024-05-20T16:11:07.979 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:11:07.975+0000 7f1426856700 1 --2- 172.21.15.184:0/3916712930 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1420103bb0 0x7f14201a0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:11:07.979 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:11:07.975+0000 7f1425854700 1 --2- 172.21.15.184:0/3916712930 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1420103210 0x7f142019e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:11:07.979 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:11:07.975+0000 7f1426055700 1 -- 172.21.15.184:0/3916712930 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1420103bb0 msgr2=0x7f14201a0b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:11:07.979 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:11:07.975+0000 7f1426055700 1 --2- 172.21.15.184:0/3916712930 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1420103bb0 0x7f14201a0b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:07.979 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:11:07.975+0000 7f1426055700 1 -- 172.21.15.184:0/3916712930 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1420103210 msgr2=0x7f142019e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:11:07.979 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:11:07.975+0000 7f1426055700 1 --2- 172.21.15.184:0/3916712930 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1420103210 0x7f142019e570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:07.979 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:11:07.975+0000 7f1426055700 1 -- 172.21.15.184:0/3916712930 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14201a1370 con 0x7f14201020e0 2024-05-20T16:11:07.979 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:11:07.975+0000 7f1425854700 1 --2- 172.21.15.184:0/3916712930 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1420103210 0x7f142019e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T16:11:07.979 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:11:07.976+0000 7f1426055700 1 --2- 172.21.15.184:0/3916712930 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f14201020e0 0x7f1420075330 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f141c00d950 tx=0x7f141c00dc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:11:07.979 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:11:07.976+0000 7f14177fe700 1 -- 172.21.15.184:0/3916712930 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f141c017070 con 0x7f14201020e0 2024-05-20T16:11:07.979 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:11:07.976+0000 7f14177fe700 1 -- 172.21.15.184:0/3916712930 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f141c009c10 con 0x7f14201020e0 2024-05-20T16:11:07.980 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:11:07.976+0000 7f14282b9700 1 -- 172.21.15.184:0/3916712930 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14201a1660 con 0x7f14201020e0 2024-05-20T16:11:07.980 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:11:07.976+0000 7f14177fe700 1 -- 172.21.15.184:0/3916712930 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f141c003be0 con 0x7f14201020e0 2024-05-20T16:11:07.980 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:11:07.976+0000 7f14282b9700 1 -- 172.21.15.184:0/3916712930 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14201a1ba0 con 0x7f14201020e0 2024-05-20T16:11:07.980 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:11:07.977+0000 7f14282b9700 1 -- 172.21.15.184:0/3916712930 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1420050170 con 0x7f14201020e0 2024-05-20T16:11:07.980 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:11:07.978+0000 7f14177fe700 1 -- 172.21.15.184:0/3916712930 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f141c004220 con 0x7f14201020e0 2024-05-20T16:11:07.980 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:11:07.979+0000 7f14177fe700 1 --2- 172.21.15.184:0/3916712930 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f140c0a2c20 0x7f140c0a5130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:11:07.981 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:11:07.979+0000 7f14177fe700 1 -- 172.21.15.184:0/3916712930 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f141c0cee40 con 0x7f14201020e0 2024-05-20T16:11:07.981 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:11:07.979+0000 7f1425854700 1 --2- 172.21.15.184:0/3916712930 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f140c0a2c20 0x7f140c0a5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:11:07.981 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:11:07.979+0000 7f1425854700 1 --2- 172.21.15.184:0/3916712930 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f140c0a2c20 0x7f140c0a5130 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f1410005950 tx=0x7f14100058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:11:07.983 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:11:07.983+0000 7f14177fe700 1 -- 172.21.15.184:0/3916712930 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f141c020030 con 0x7f14201020e0 2024-05-20T16:11:08.188 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:11:08.188+0000 7f14282b9700 1 -- 172.21.15.184:0/3916712930 --> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi082=a;smithi082=d;smithi082=g;smithi082=j;smithi160=b;smithi160=e;smithi160=h;smithi160=k;smithi184=c;smithi184=f;smithi184=i;smithi184=l", "target": ["mon-mgr", ""]}) -- 0x7f1420062d40 con 0x7f140c0a2c20 2024-05-20T16:11:08.197 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:11:08.197+0000 7f14177fe700 1 -- 172.21.15.184:0/3916712930 <== mgr.14150 v2:172.21.15.82:6800/379530823 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7f1420062d40 con 0x7f140c0a2c20 2024-05-20T16:11:08.197 INFO:teuthology.orchestra.run.smithi184.stdout:Scheduled mds.all update... 2024-05-20T16:11:08.200 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:11:08.199+0000 7f14282b9700 1 -- 172.21.15.184:0/3916712930 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f140c0a2c20 msgr2=0x7f140c0a5130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:11:08.200 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:11:08.199+0000 7f14282b9700 1 --2- 172.21.15.184:0/3916712930 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f140c0a2c20 0x7f140c0a5130 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f1410005950 tx=0x7f14100058e0 comp rx=0 tx=0).stop 2024-05-20T16:11:08.200 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:11:08.199+0000 7f14282b9700 1 -- 172.21.15.184:0/3916712930 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f14201020e0 msgr2=0x7f1420075330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:11:08.200 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:11:08.199+0000 7f14282b9700 1 --2- 172.21.15.184:0/3916712930 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f14201020e0 0x7f1420075330 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f141c00d950 tx=0x7f141c00dc60 comp rx=0 tx=0).stop 2024-05-20T16:11:08.200 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:11:08.199+0000 7f14282b9700 1 -- 172.21.15.184:0/3916712930 shutdown_connections 2024-05-20T16:11:08.200 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:11:08.199+0000 7f14282b9700 1 --2- 172.21.15.184:0/3916712930 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f140c0a2c20 0x7f140c0a5130 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:08.200 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:11:08.199+0000 7f14282b9700 1 --2- 172.21.15.184:0/3916712930 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1420103bb0 0x7f14201a0b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:08.200 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:11:08.199+0000 7f14282b9700 1 --2- 172.21.15.184:0/3916712930 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f14201020e0 0x7f1420075330 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:08.200 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:11:08.199+0000 7f14282b9700 1 --2- 172.21.15.184:0/3916712930 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1420103210 0x7f142019e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:08.201 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:11:08.199+0000 7f14282b9700 1 -- 172.21.15.184:0/3916712930 >> 172.21.15.184:0/3916712930 conn(0x7f14200fd740 msgr2=0x7f1420106440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:11:08.201 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:11:08.200+0000 7f14282b9700 1 -- 172.21.15.184:0/3916712930 shutdown_connections 2024-05-20T16:11:08.201 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-20T16:11:08.200+0000 7f14282b9700 1 -- 172.21.15.184:0/3916712930 wait complete. 2024-05-20T16:11:08.904 DEBUG:teuthology.orchestra.run.smithi082:mds.a> sudo journalctl -f -n 0 -u ceph-59d1600a-16c2-11ef-bc9a-c7b262605968@mds.a.service 2024-05-20T16:11:08.907 DEBUG:teuthology.orchestra.run.smithi082:mds.d> sudo journalctl -f -n 0 -u ceph-59d1600a-16c2-11ef-bc9a-c7b262605968@mds.d.service 2024-05-20T16:11:08.952 DEBUG:teuthology.orchestra.run.smithi082:mds.g> sudo journalctl -f -n 0 -u ceph-59d1600a-16c2-11ef-bc9a-c7b262605968@mds.g.service 2024-05-20T16:11:08.995 DEBUG:teuthology.orchestra.run.smithi082:mds.j> sudo journalctl -f -n 0 -u ceph-59d1600a-16c2-11ef-bc9a-c7b262605968@mds.j.service 2024-05-20T16:11:09.038 DEBUG:teuthology.orchestra.run.smithi160:mds.b> sudo journalctl -f -n 0 -u ceph-59d1600a-16c2-11ef-bc9a-c7b262605968@mds.b.service 2024-05-20T16:11:09.040 DEBUG:teuthology.orchestra.run.smithi160:mds.e> sudo journalctl -f -n 0 -u ceph-59d1600a-16c2-11ef-bc9a-c7b262605968@mds.e.service 2024-05-20T16:11:09.083 DEBUG:teuthology.orchestra.run.smithi160:mds.h> sudo journalctl -f -n 0 -u ceph-59d1600a-16c2-11ef-bc9a-c7b262605968@mds.h.service 2024-05-20T16:11:09.126 DEBUG:teuthology.orchestra.run.smithi160:mds.k> sudo journalctl -f -n 0 -u ceph-59d1600a-16c2-11ef-bc9a-c7b262605968@mds.k.service 2024-05-20T16:11:09.169 DEBUG:teuthology.orchestra.run.smithi184:mds.c> sudo journalctl -f -n 0 -u ceph-59d1600a-16c2-11ef-bc9a-c7b262605968@mds.c.service 2024-05-20T16:11:09.172 DEBUG:teuthology.orchestra.run.smithi184:mds.f> sudo journalctl -f -n 0 -u ceph-59d1600a-16c2-11ef-bc9a-c7b262605968@mds.f.service 2024-05-20T16:11:09.215 DEBUG:teuthology.orchestra.run.smithi184:mds.i> sudo journalctl -f -n 0 -u ceph-59d1600a-16c2-11ef-bc9a-c7b262605968@mds.i.service 2024-05-20T16:11:09.258 DEBUG:teuthology.orchestra.run.smithi184:mds.l> sudo journalctl -f -n 0 -u ceph-59d1600a-16c2-11ef-bc9a-c7b262605968@mds.l.service 2024-05-20T16:11:09.302 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-05-20T16:11:09.302 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-20T16:11:09.312 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:09 smithi160 ceph-mon[36347]: from='client.24406 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi082=a;smithi082=d;smithi082=g;smithi082=j;smithi160=b;smithi160=e;smithi160=h;smithi160=k;smithi184=c;smithi184=f;smithi184=i;smithi184=l", "target": ["mon-mgr", ""]}]: dispatch 2024-05-20T16:11:09.312 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:09 smithi160 ceph-mon[36347]: Saving service mds.all spec with placement smithi082=a;smithi082=d;smithi082=g;smithi082=j;smithi160=b;smithi160=e;smithi160=h;smithi160=k;smithi184=c;smithi184=f;smithi184=i;smithi184=l;count:12 2024-05-20T16:11:09.312 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:09 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:11:09.312 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:09 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T16:11:09.312 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:09 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:11:09.312 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:09 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T16:11:09.312 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:09 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:11:09.312 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:09 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-20T16:11:09.313 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:09 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-20T16:11:09.313 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:09 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:11:09.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:09.426+0000 7fccb70e2640 1 -- 172.21.15.82:0/3811585085 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fccb0101200 msgr2=0x7fccb010ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:11:09.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:09.426+0000 7fccb70e2640 1 --2- 172.21.15.82:0/3811585085 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fccb0101200 0x7fccb010ac60 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fcca8009ca0 tx=0x7fcca802a8c0 comp rx=0 tx=0).stop 2024-05-20T16:11:09.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:09.426+0000 7fccb70e2640 1 -- 172.21.15.82:0/3811585085 shutdown_connections 2024-05-20T16:11:09.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:09.426+0000 7fccb70e2640 1 --2- 172.21.15.82:0/3811585085 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fccb0101200 0x7fccb010ac60 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:09.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:09.426+0000 7fccb70e2640 1 --2- 172.21.15.82:0/3811585085 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fccb0100700 0x7fccb0100b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:09.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:09.426+0000 7fccb70e2640 1 --2- 172.21.15.82:0/3811585085 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fccb0104050 0x7fccb0104460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:09.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:09.426+0000 7fccb70e2640 1 -- 172.21.15.82:0/3811585085 >> 172.21.15.82:0/3811585085 conn(0x7fccb00fc360 msgr2=0x7fccb00fe7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:11:09.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:09.426+0000 7fccb70e2640 1 -- 172.21.15.82:0/3811585085 shutdown_connections 2024-05-20T16:11:09.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:09.426+0000 7fccb70e2640 1 -- 172.21.15.82:0/3811585085 wait complete. 2024-05-20T16:11:09.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:09.426+0000 7fccb70e2640 1 Processor -- start 2024-05-20T16:11:09.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:09.426+0000 7fccb70e2640 1 -- start start 2024-05-20T16:11:09.429 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:09.427+0000 7fccb70e2640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fccb0100700 0x7fccb01991f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:11:09.429 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:09.427+0000 7fccb70e2640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fccb0101200 0x7fccb0199730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:11:09.429 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:09.427+0000 7fccb70e2640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fccb0104050 0x7fccb01a06c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:11:09.429 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:09.427+0000 7fccb70e2640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fccb010e710 con 0x7fccb0101200 2024-05-20T16:11:09.429 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:09.427+0000 7fccb70e2640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fccb010e590 con 0x7fccb0100700 2024-05-20T16:11:09.429 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:09.427+0000 7fccb70e2640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fccb010e890 con 0x7fccb0104050 2024-05-20T16:11:09.429 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:09.427+0000 7fccaffff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fccb0101200 0x7fccb0199730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:11:09.429 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:09.427+0000 7fccaffff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fccb0101200 0x7fccb0199730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:45962/0 (socket says 172.21.15.82:45962) 2024-05-20T16:11:09.429 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:09.427+0000 7fccaffff640 1 -- 172.21.15.82:0/2582965044 learned_addr learned my addr 172.21.15.82:0/2582965044 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:11:09.429 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:09.427+0000 7fccb567f640 1 --2- 172.21.15.82:0/2582965044 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fccb0104050 0x7fccb01a06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:11:09.429 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:09.427+0000 7fccb4e7e640 1 --2- 172.21.15.82:0/2582965044 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fccb0100700 0x7fccb01991f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:11:09.429 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:09.427+0000 7fccaffff640 1 -- 172.21.15.82:0/2582965044 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fccb0104050 msgr2=0x7fccb01a06c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:11:09.430 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:09.427+0000 7fccaffff640 1 --2- 172.21.15.82:0/2582965044 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fccb0104050 0x7fccb01a06c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:09.430 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:09.427+0000 7fccaffff640 1 -- 172.21.15.82:0/2582965044 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fccb0100700 msgr2=0x7fccb01991f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:11:09.430 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:09.427+0000 7fccaffff640 1 --2- 172.21.15.82:0/2582965044 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fccb0100700 0x7fccb01991f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:09.430 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:09.427+0000 7fccaffff640 1 -- 172.21.15.82:0/2582965044 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccb01a0d30 con 0x7fccb0101200 2024-05-20T16:11:09.430 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:09.427+0000 7fccaffff640 1 --2- 172.21.15.82:0/2582965044 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fccb0101200 0x7fccb0199730 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fcc9c00e9f0 tx=0x7fcc9c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:11:09.430 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:09.427+0000 7fccadffb640 1 -- 172.21.15.82:0/2582965044 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc9c00cd80 con 0x7fccb0101200 2024-05-20T16:11:09.430 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:09.427+0000 7fccb70e2640 1 -- 172.21.15.82:0/2582965044 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fccb01a0fc0 con 0x7fccb0101200 2024-05-20T16:11:09.430 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:09.427+0000 7fccadffb640 1 -- 172.21.15.82:0/2582965044 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcc9c010910 con 0x7fccb0101200 2024-05-20T16:11:09.430 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:09.427+0000 7fccadffb640 1 -- 172.21.15.82:0/2582965044 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc9c0188f0 con 0x7fccb0101200 2024-05-20T16:11:09.430 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:09.428+0000 7fccb70e2640 1 -- 172.21.15.82:0/2582965044 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fccb01a1500 con 0x7fccb0101200 2024-05-20T16:11:09.431 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:09.429+0000 7fccb70e2640 1 -- 172.21.15.82:0/2582965044 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fccb01099f0 con 0x7fccb0101200 2024-05-20T16:11:09.431 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:09.430+0000 7fccadffb640 1 -- 172.21.15.82:0/2582965044 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fcc9c010ac0 con 0x7fccb0101200 2024-05-20T16:11:09.431 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:09.430+0000 7fccadffb640 1 --2- 172.21.15.82:0/2582965044 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fcca00a2b90 0x7fcca00a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:11:09.431 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:09.431+0000 7fccb4e7e640 1 --2- 172.21.15.82:0/2582965044 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fcca00a2b90 0x7fcca00a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:11:09.431 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:09.431+0000 7fccadffb640 1 -- 172.21.15.82:0/2582965044 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fcc9c0cefe0 con 0x7fccb0101200 2024-05-20T16:11:09.431 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:09.431+0000 7fccb4e7e640 1 --2- 172.21.15.82:0/2582965044 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fcca00a2b90 0x7fcca00a50a0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fcca4005fd0 tx=0x7fcca400b5b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:11:09.435 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:09.435+0000 7fccadffb640 1 -- 172.21.15.82:0/2582965044 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fcc9c09c3e0 con 0x7fccb0101200 2024-05-20T16:11:09.453 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:09 smithi184 ceph-mon[36209]: from='client.24406 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi082=a;smithi082=d;smithi082=g;smithi082=j;smithi160=b;smithi160=e;smithi160=h;smithi160=k;smithi184=c;smithi184=f;smithi184=i;smithi184=l", "target": ["mon-mgr", ""]}]: dispatch 2024-05-20T16:11:09.453 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:09 smithi184 ceph-mon[36209]: Saving service mds.all spec with placement smithi082=a;smithi082=d;smithi082=g;smithi082=j;smithi160=b;smithi160=e;smithi160=h;smithi160=k;smithi184=c;smithi184=f;smithi184=i;smithi184=l;count:12 2024-05-20T16:11:09.453 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:09 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:11:09.453 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:09 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T16:11:09.453 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:09 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:11:09.453 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:09 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T16:11:09.453 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:09 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:11:09.453 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:09 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-20T16:11:09.453 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:09 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-20T16:11:09.454 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:09 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:11:09.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:09 smithi082 ceph-mon[32238]: from='client.24406 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi082=a;smithi082=d;smithi082=g;smithi082=j;smithi160=b;smithi160=e;smithi160=h;smithi160=k;smithi184=c;smithi184=f;smithi184=i;smithi184=l", "target": ["mon-mgr", ""]}]: dispatch 2024-05-20T16:11:09.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:09 smithi082 ceph-mon[32238]: Saving service mds.all spec with placement smithi082=a;smithi082=d;smithi082=g;smithi082=j;smithi160=b;smithi160=e;smithi160=h;smithi160=k;smithi184=c;smithi184=f;smithi184=i;smithi184=l;count:12 2024-05-20T16:11:09.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:09 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:11:09.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:09 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T16:11:09.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:09 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:11:09.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:09 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T16:11:09.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:09 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:11:09.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:09 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-20T16:11:09.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:09 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-20T16:11:09.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:09 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:11:09.613 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:09.613+0000 7fccb70e2640 1 -- 172.21.15.82:0/2582965044 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fccb0062750 con 0x7fccb0101200 2024-05-20T16:11:09.614 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:09.614+0000 7fccadffb640 1 -- 172.21.15.82:0/2582965044 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+18481 (secure 0 0 0) 0x7fcc9c09bce0 con 0x7fccb0101200 2024-05-20T16:11:09.615 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-05-20T16:11:09.615 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":82,"fsid":"59d1600a-16c2-11ef-bc9a-c7b262605968","created":"2024-05-20T16:03:46.611556+0000","modified":"2024-05-20T16:11:00.310642+0000","last_up_change":"2024-05-20T16:10:59.303479+0000","last_in_change":"2024-05-20T16:10:39.063561+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-20T16:07:09.493727+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bae842a9-03ec-490e-b56d-f7957235cdc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":1484330167},{"type":"v1","addr":"172.21.15.82:6803","nonce":1484330167}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":1484330167},{"type":"v1","addr":"172.21.15.82:6805","nonce":1484330167}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1484330167},{"type":"v1","addr":"172.21.15.82:6809","nonce":1484330167}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":1484330167},{"type":"v1","addr":"172.21.15.82:6807","nonce":1484330167}]},"public_addr":"172.21.15.82:6803/1484330167","cluster_addr":"172.21.15.82:6805/1484330167","heartbeat_back_addr":"172.21.15.82:6809/1484330167","heartbeat_front_addr":"172.21.15.82:6807/1484330167","state":["exists","up"]},{"osd":1,"uuid":"59f6a2ba-b5de-4161-81c5-92a417abacdf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":1457003568},{"type":"v1","addr":"172.21.15.160:6801","nonce":1457003568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":1457003568},{"type":"v1","addr":"172.21.15.160:6803","nonce":1457003568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6806","nonce":1457003568},{"type":"v1","addr":"172.21.15.160:6807","nonce":1457003568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":1457003568},{"type":"v1","addr":"172.21.15.160:6805","nonce":1457003568}]},"public_addr":"172.21.15.160:6801/1457003568","cluster_addr":"172.21.15.160:6803/1457003568","heartbeat_back_addr":"172.21.15.160:6807/1457003568","heartbeat_front_addr":"172.21.15.160:6805/1457003568","state":["exists","up"]},{"osd":2,"uuid":"3bbb5587-b449-41f3-bd26-67ec3d6afdfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":4051293934},{"type":"v1","addr":"172.21.15.184:6801","nonce":4051293934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":4051293934},{"type":"v1","addr":"172.21.15.184:6803","nonce":4051293934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":4051293934},{"type":"v1","addr":"172.21.15.184:6807","nonce":4051293934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":4051293934},{"type":"v1","addr":"172.21.15.184:6805","nonce":4051293934}]},"public_addr":"172.21.15.184:6801/4051293934","cluster_addr":"172.21.15.184:6803/4051293934","heartbeat_back_addr":"172.21.15.184:6807/4051293934","heartbeat_front_addr":"172.21.15.184:6805/4051293934","state":["exists","up"]},{"osd":3,"uuid":"ca651f90-09b8-4081-a6f9-40b31268a837","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":3947485599},{"type":"v1","addr":"172.21.15.82:6811","nonce":3947485599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":3947485599},{"type":"v1","addr":"172.21.15.82:6813","nonce":3947485599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":3947485599},{"type":"v1","addr":"172.21.15.82:6817","nonce":3947485599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":3947485599},{"type":"v1","addr":"172.21.15.82:6815","nonce":3947485599}]},"public_addr":"172.21.15.82:6811/3947485599","cluster_addr":"172.21.15.82:6813/3947485599","heartbeat_back_addr":"172.21.15.82:6817/3947485599","heartbeat_front_addr":"172.21.15.82:6815/3947485599","state":["exists","up"]},{"osd":4,"uuid":"cb3dfd56-f677-40cb-9c96-8319ea5ac788","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":4160830014},{"type":"v1","addr":"172.21.15.160:6809","nonce":4160830014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":4160830014},{"type":"v1","addr":"172.21.15.160:6811","nonce":4160830014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":4160830014},{"type":"v1","addr":"172.21.15.160:6815","nonce":4160830014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6812","nonce":4160830014},{"type":"v1","addr":"172.21.15.160:6813","nonce":4160830014}]},"public_addr":"172.21.15.160:6809/4160830014","cluster_addr":"172.21.15.160:6811/4160830014","heartbeat_back_addr":"172.21.15.160:6815/4160830014","heartbeat_front_addr":"172.21.15.160:6813/4160830014","state":["exists","up"]},{"osd":5,"uuid":"c70cfcd1-a4ff-461f-8c73-479b061200fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":1046282672},{"type":"v1","addr":"172.21.15.184:6809","nonce":1046282672}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":1046282672},{"type":"v1","addr":"172.21.15.184:6811","nonce":1046282672}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":1046282672},{"type":"v1","addr":"172.21.15.184:6815","nonce":1046282672}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":1046282672},{"type":"v1","addr":"172.21.15.184:6813","nonce":1046282672}]},"public_addr":"172.21.15.184:6809/1046282672","cluster_addr":"172.21.15.184:6811/1046282672","heartbeat_back_addr":"172.21.15.184:6815/1046282672","heartbeat_front_addr":"172.21.15.184:6813/1046282672","state":["exists","up"]},{"osd":6,"uuid":"d47ce001-9dd1-4830-bcc9-888601187c6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":2789736499},{"type":"v1","addr":"172.21.15.82:6819","nonce":2789736499}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":2789736499},{"type":"v1","addr":"172.21.15.82:6821","nonce":2789736499}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":2789736499},{"type":"v1","addr":"172.21.15.82:6825","nonce":2789736499}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":2789736499},{"type":"v1","addr":"172.21.15.82:6823","nonce":2789736499}]},"public_addr":"172.21.15.82:6819/2789736499","cluster_addr":"172.21.15.82:6821/2789736499","heartbeat_back_addr":"172.21.15.82:6825/2789736499","heartbeat_front_addr":"172.21.15.82:6823/2789736499","state":["exists","up"]},{"osd":7,"uuid":"25c54f43-18cb-41e4-83b4-2732b5a93889","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6816","nonce":2340683378},{"type":"v1","addr":"172.21.15.160:6817","nonce":2340683378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":2340683378},{"type":"v1","addr":"172.21.15.160:6819","nonce":2340683378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":2340683378},{"type":"v1","addr":"172.21.15.160:6823","nonce":2340683378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6820","nonce":2340683378},{"type":"v1","addr":"172.21.15.160:6821","nonce":2340683378}]},"public_addr":"172.21.15.160:6817/2340683378","cluster_addr":"172.21.15.160:6819/2340683378","heartbeat_back_addr":"172.21.15.160:6823/2340683378","heartbeat_front_addr":"172.21.15.160:6821/2340683378","state":["exists","up"]},{"osd":8,"uuid":"1f8b478d-287f-45f5-8526-d1a58c5bc07c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":3831576372},{"type":"v1","addr":"172.21.15.184:6817","nonce":3831576372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":3831576372},{"type":"v1","addr":"172.21.15.184:6819","nonce":3831576372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":3831576372},{"type":"v1","addr":"172.21.15.184:6823","nonce":3831576372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":3831576372},{"type":"v1","addr":"172.21.15.184:6821","nonce":3831576372}]},"public_addr":"172.21.15.184:6817/3831576372","cluster_addr":"172.21.15.184:6819/3831576372","heartbeat_back_addr":"172.21.15.184:6823/3831576372","heartbeat_front_addr":"172.21.15.184:6821/3831576372","state":["exists","up"]},{"osd":9,"uuid":"0d69e319-954f-4b18-95ea-31794f18dc3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":716330782},{"type":"v1","addr":"172.21.15.82:6827","nonce":716330782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":716330782},{"type":"v1","addr":"172.21.15.82:6829","nonce":716330782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":716330782},{"type":"v1","addr":"172.21.15.82:6833","nonce":716330782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":716330782},{"type":"v1","addr":"172.21.15.82:6831","nonce":716330782}]},"public_addr":"172.21.15.82:6827/716330782","cluster_addr":"172.21.15.82:6829/716330782","heartbeat_back_addr":"172.21.15.82:6833/716330782","heartbeat_front_addr":"172.21.15.82:6831/716330782","state":["exists","up"]},{"osd":10,"uuid":"39dc1b78-61cb-42ca-b5b6-1e44ac2b6500","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":2327428014},{"type":"v1","addr":"172.21.15.160:6825","nonce":2327428014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":2327428014},{"type":"v1","addr":"172.21.15.160:6827","nonce":2327428014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":2327428014},{"type":"v1","addr":"172.21.15.160:6831","nonce":2327428014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":2327428014},{"type":"v1","addr":"172.21.15.160:6829","nonce":2327428014}]},"public_addr":"172.21.15.160:6825/2327428014","cluster_addr":"172.21.15.160:6827/2327428014","heartbeat_back_addr":"172.21.15.160:6831/2327428014","heartbeat_front_addr":"172.21.15.160:6829/2327428014","state":["exists","up"]},{"osd":11,"uuid":"4306b3c1-707a-4010-987c-209ea0e948e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":3594364650},{"type":"v1","addr":"172.21.15.184:6825","nonce":3594364650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":3594364650},{"type":"v1","addr":"172.21.15.184:6827","nonce":3594364650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3594364650},{"type":"v1","addr":"172.21.15.184:6831","nonce":3594364650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3594364650},{"type":"v1","addr":"172.21.15.184:6829","nonce":3594364650}]},"public_addr":"172.21.15.184:6825/3594364650","cluster_addr":"172.21.15.184:6827/3594364650","heartbeat_back_addr":"172.21.15.184:6831/3594364650","heartbeat_front_addr":"172.21.15.184:6829/3594364650","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:06:17.855998+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:06:41.473270+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:07:05.304108+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:07:29.887472+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:07:55.387286+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:08:20.579669+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:08:46.333051+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:09:12.289668+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:09:38.180696+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:10:04.471459+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:10:31.041912+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:10:57.477266+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.82:0/2005424509":"2024-05-21T16:04:45.161949+0000","172.21.15.82:6801/1763049308":"2024-05-21T16:04:45.161949+0000","172.21.15.82:6801/1959459348":"2024-05-21T16:04:19.208291+0000","172.21.15.82:0/3131904556":"2024-05-21T16:04:19.208291+0000","172.21.15.82:6800/1959459348":"2024-05-21T16:04:19.208291+0000","172.21.15.82:0/2456966342":"2024-05-21T16:04:45.161949+0000","172.21.15.82:0/4179978833":"2024-05-21T16:04:19.208291+0000","172.21.15.82:0/1602862225":"2024-05-21T16:04:45.161949+0000","172.21.15.82:0/106101180":"2024-05-21T16:04:19.208291+0000","172.21.15.82:6800/1763049308":"2024-05-21T16:04:45.161949+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-20T16:11:09.617 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:09.617+0000 7fccb70e2640 1 -- 172.21.15.82:0/2582965044 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fcca00a2b90 msgr2=0x7fcca00a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:11:09.617 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:09.617+0000 7fccb70e2640 1 --2- 172.21.15.82:0/2582965044 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fcca00a2b90 0x7fcca00a50a0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fcca4005fd0 tx=0x7fcca400b5b0 comp rx=0 tx=0).stop 2024-05-20T16:11:09.617 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:09.617+0000 7fccb70e2640 1 -- 172.21.15.82:0/2582965044 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fccb0101200 msgr2=0x7fccb0199730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:11:09.617 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:09.617+0000 7fccb70e2640 1 --2- 172.21.15.82:0/2582965044 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fccb0101200 0x7fccb0199730 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fcc9c00e9f0 tx=0x7fcc9c00eeb0 comp rx=0 tx=0).stop 2024-05-20T16:11:09.617 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:09.617+0000 7fccb70e2640 1 -- 172.21.15.82:0/2582965044 shutdown_connections 2024-05-20T16:11:09.617 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:09.617+0000 7fccb70e2640 1 --2- 172.21.15.82:0/2582965044 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fcca00a2b90 0x7fcca00a50a0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:09.617 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:09.617+0000 7fccb70e2640 1 --2- 172.21.15.82:0/2582965044 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fccb0101200 0x7fccb0199730 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:09.617 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:09.617+0000 7fccb70e2640 1 --2- 172.21.15.82:0/2582965044 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fccb0104050 0x7fccb01a06c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:09.617 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:09.617+0000 7fccb70e2640 1 --2- 172.21.15.82:0/2582965044 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fccb0100700 0x7fccb01991f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:09.617 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:09.617+0000 7fccb70e2640 1 -- 172.21.15.82:0/2582965044 >> 172.21.15.82:0/2582965044 conn(0x7fccb00fc360 msgr2=0x7fccb0106bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:11:09.617 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:09.617+0000 7fccb70e2640 1 -- 172.21.15.82:0/2582965044 shutdown_connections 2024-05-20T16:11:09.617 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:09.617+0000 7fccb70e2640 1 -- 172.21.15.82:0/2582965044 wait complete. 2024-05-20T16:11:09.628 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-20T16:07:09.493727+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-20T16:11:09.629 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-20T16:11:09.745 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:09.743+0000 7f45fa58f640 1 -- 172.21.15.82:0/3369560320 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f45f4102bd0 msgr2=0x7f45f4102fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:11:09.745 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:09.743+0000 7f45fa58f640 1 --2- 172.21.15.82:0/3369560320 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f45f4102bd0 0x7f45f4102fe0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f45dc009900 tx=0x7f45dc02a410 comp rx=0 tx=0).stop 2024-05-20T16:11:09.745 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:09.743+0000 7f45fa58f640 1 -- 172.21.15.82:0/3369560320 shutdown_connections 2024-05-20T16:11:09.745 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:09.743+0000 7f45fa58f640 1 --2- 172.21.15.82:0/3369560320 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f45f4102bd0 0x7f45f4102fe0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:09.745 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:09.743+0000 7f45fa58f640 1 --2- 172.21.15.82:0/3369560320 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f45f41078d0 0x7f45f4109cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:09.745 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:09.743+0000 7f45fa58f640 1 --2- 172.21.15.82:0/3369560320 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f45f4101990 0x7f45f4102690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:09.746 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:09.743+0000 7f45fa58f640 1 -- 172.21.15.82:0/3369560320 >> 172.21.15.82:0/3369560320 conn(0x7f45f40fcb80 msgr2=0x7f45f40ff030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:11:09.746 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:09.743+0000 7f45fa58f640 1 -- 172.21.15.82:0/3369560320 shutdown_connections 2024-05-20T16:11:09.746 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:09.743+0000 7f45fa58f640 1 -- 172.21.15.82:0/3369560320 wait complete. 2024-05-20T16:11:09.746 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:09.743+0000 7f45fa58f640 1 Processor -- start 2024-05-20T16:11:09.746 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:09.743+0000 7f45fa58f640 1 -- start start 2024-05-20T16:11:09.746 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:09.744+0000 7f45fa58f640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f45f4101990 0x7f45f41995b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:11:09.746 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:09.744+0000 7f45fa58f640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f45f4102bd0 0x7f45f4199af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:11:09.746 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:09.744+0000 7f45fa58f640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f45f41078d0 0x7f45f41a0a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:11:09.746 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:09.744+0000 7f45fa58f640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f45f410e770 con 0x7f45f4101990 2024-05-20T16:11:09.746 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:09.744+0000 7f45fa58f640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f45f410e5f0 con 0x7f45f41078d0 2024-05-20T16:11:09.746 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:09.744+0000 7f45fa58f640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f45f410e8f0 con 0x7f45f4102bd0 2024-05-20T16:11:09.746 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:09.744+0000 7f45f3fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f45f4101990 0x7f45f41995b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:11:09.746 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:09.744+0000 7f45f3fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f45f4101990 0x7f45f41995b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:54484/0 (socket says 172.21.15.82:54484) 2024-05-20T16:11:09.747 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:09.744+0000 7f45f3fff640 1 -- 172.21.15.82:0/2109652067 learned_addr learned my addr 172.21.15.82:0/2109652067 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:11:09.747 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:09.744+0000 7f45ebfff640 1 --2- 172.21.15.82:0/2109652067 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f45f4102bd0 0x7f45f4199af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:11:09.747 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:09.744+0000 7f45f8b2c640 1 --2- 172.21.15.82:0/2109652067 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f45f41078d0 0x7f45f41a0a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:11:09.747 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:09.744+0000 7f45f3fff640 1 -- 172.21.15.82:0/2109652067 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f45f4102bd0 msgr2=0x7f45f4199af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:11:09.747 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:09.744+0000 7f45f3fff640 1 --2- 172.21.15.82:0/2109652067 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f45f4102bd0 0x7f45f4199af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:09.747 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:09.744+0000 7f45f3fff640 1 -- 172.21.15.82:0/2109652067 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f45f41078d0 msgr2=0x7f45f41a0a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:11:09.747 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:09.744+0000 7f45f3fff640 1 --2- 172.21.15.82:0/2109652067 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f45f41078d0 0x7f45f41a0a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:09.747 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:09.744+0000 7f45f3fff640 1 -- 172.21.15.82:0/2109652067 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45f41a10f0 con 0x7f45f4101990 2024-05-20T16:11:09.747 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:09.744+0000 7f45f8b2c640 1 --2- 172.21.15.82:0/2109652067 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f45f41078d0 0x7f45f41a0a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T16:11:09.747 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:09.744+0000 7f45f3fff640 1 --2- 172.21.15.82:0/2109652067 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f45f4101990 0x7f45f41995b0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f45e400c970 tx=0x7f45e400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:11:09.747 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:09.744+0000 7f45f1ffb640 1 -- 172.21.15.82:0/2109652067 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45e4004220 con 0x7f45f4101990 2024-05-20T16:11:09.747 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:09.744+0000 7f45fa58f640 1 -- 172.21.15.82:0/2109652067 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45f41a13e0 con 0x7f45f4101990 2024-05-20T16:11:09.748 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:09.744+0000 7f45f1ffb640 1 -- 172.21.15.82:0/2109652067 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f45e40043c0 con 0x7f45f4101990 2024-05-20T16:11:09.748 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:09.744+0000 7f45f1ffb640 1 -- 172.21.15.82:0/2109652067 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45e400f450 con 0x7f45f4101990 2024-05-20T16:11:09.748 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:09.744+0000 7f45fa58f640 1 -- 172.21.15.82:0/2109652067 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45f41a1950 con 0x7f45f4101990 2024-05-20T16:11:09.748 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:09.746+0000 7f45fa58f640 1 -- 172.21.15.82:0/2109652067 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45f41066e0 con 0x7f45f4101990 2024-05-20T16:11:09.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:09.747+0000 7f45f1ffb640 1 -- 172.21.15.82:0/2109652067 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f45e40079b0 con 0x7f45f4101990 2024-05-20T16:11:09.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:09.747+0000 7f45f1ffb640 1 --2- 172.21.15.82:0/2109652067 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f45e00a2b90 0x7f45e00a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:11:09.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:09.748+0000 7f45ebfff640 1 --2- 172.21.15.82:0/2109652067 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f45e00a2b90 0x7f45e00a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:11:09.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:09.748+0000 7f45f1ffb640 1 -- 172.21.15.82:0/2109652067 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f45e40cec40 con 0x7f45f4101990 2024-05-20T16:11:09.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:09.748+0000 7f45ebfff640 1 --2- 172.21.15.82:0/2109652067 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f45e00a2b90 0x7f45e00a50a0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f45dc005ab0 tx=0x7f45dc0315b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:11:09.751 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:09.751+0000 7f45f1ffb640 1 -- 172.21.15.82:0/2109652067 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f45e4014030 con 0x7f45f4101990 2024-05-20T16:11:09.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:09.925+0000 7f45fa58f640 1 -- 172.21.15.82:0/2109652067 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f45f402d3c0 con 0x7f45f4101990 2024-05-20T16:11:09.926 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:09.926+0000 7f45f1ffb640 1 -- 172.21.15.82:0/2109652067 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) ==== 93+0+10 (secure 0 0 0) 0x7f45e409c040 con 0x7f45f4101990 2024-05-20T16:11:09.926 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 1 2024-05-20T16:11:09.928 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:09.928+0000 7f45fa58f640 1 -- 172.21.15.82:0/2109652067 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f45e00a2b90 msgr2=0x7f45e00a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:11:09.928 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:09.928+0000 7f45fa58f640 1 --2- 172.21.15.82:0/2109652067 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f45e00a2b90 0x7f45e00a50a0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f45dc005ab0 tx=0x7f45dc0315b0 comp rx=0 tx=0).stop 2024-05-20T16:11:09.928 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:09.928+0000 7f45fa58f640 1 -- 172.21.15.82:0/2109652067 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f45f4101990 msgr2=0x7f45f41995b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:11:09.928 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:09.928+0000 7f45fa58f640 1 --2- 172.21.15.82:0/2109652067 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f45f4101990 0x7f45f41995b0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f45e400c970 tx=0x7f45e400ce30 comp rx=0 tx=0).stop 2024-05-20T16:11:09.928 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:09.928+0000 7f45fa58f640 1 -- 172.21.15.82:0/2109652067 shutdown_connections 2024-05-20T16:11:09.929 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:09.928+0000 7f45fa58f640 1 --2- 172.21.15.82:0/2109652067 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f45e00a2b90 0x7f45e00a50a0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:09.929 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:09.928+0000 7f45fa58f640 1 --2- 172.21.15.82:0/2109652067 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f45f41078d0 0x7f45f41a0a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:09.929 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:09.928+0000 7f45fa58f640 1 --2- 172.21.15.82:0/2109652067 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f45f4102bd0 0x7f45f4199af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:09.929 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:09.928+0000 7f45fa58f640 1 --2- 172.21.15.82:0/2109652067 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f45f4101990 0x7f45f41995b0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:09.929 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:09.928+0000 7f45fa58f640 1 -- 172.21.15.82:0/2109652067 >> 172.21.15.82:0/2109652067 conn(0x7f45f40fcb80 msgr2=0x7f45f41038b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:11:09.929 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:09.928+0000 7f45fa58f640 1 -- 172.21.15.82:0/2109652067 shutdown_connections 2024-05-20T16:11:09.929 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:09.928+0000 7f45fa58f640 1 -- 172.21.15.82:0/2109652067 wait complete. 2024-05-20T16:11:09.940 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-20T16:11:10.053 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:10.051+0000 7efd27e5a640 1 -- 172.21.15.82:0/3260450487 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efd20102ab0 msgr2=0x7efd201037b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:11:10.053 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:10.051+0000 7efd27e5a640 1 --2- 172.21.15.82:0/3260450487 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efd20102ab0 0x7efd201037b0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7efd1c009ca0 tx=0x7efd1c02a8c0 comp rx=0 tx=0).stop 2024-05-20T16:11:10.053 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:10.051+0000 7efd27e5a640 1 -- 172.21.15.82:0/3260450487 shutdown_connections 2024-05-20T16:11:10.053 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:10.051+0000 7efd27e5a640 1 --2- 172.21.15.82:0/3260450487 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efd201089f0 0x7efd2010ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:10.053 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:10.051+0000 7efd27e5a640 1 --2- 172.21.15.82:0/3260450487 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efd20103cf0 0x7efd20104100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:10.053 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:10.051+0000 7efd27e5a640 1 --2- 172.21.15.82:0/3260450487 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efd20102ab0 0x7efd201037b0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:10.053 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:10.051+0000 7efd27e5a640 1 -- 172.21.15.82:0/3260450487 >> 172.21.15.82:0/3260450487 conn(0x7efd200fdcc0 msgr2=0x7efd20100150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:11:10.053 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:10.051+0000 7efd27e5a640 1 -- 172.21.15.82:0/3260450487 shutdown_connections 2024-05-20T16:11:10.054 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:10.051+0000 7efd27e5a640 1 -- 172.21.15.82:0/3260450487 wait complete. 2024-05-20T16:11:10.054 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:10.051+0000 7efd27e5a640 1 Processor -- start 2024-05-20T16:11:10.054 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:10.051+0000 7efd27e5a640 1 -- start start 2024-05-20T16:11:10.054 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:10.052+0000 7efd27e5a640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efd20102ab0 0x7efd20199270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:11:10.054 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:10.052+0000 7efd27e5a640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efd20103cf0 0x7efd201997b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:11:10.054 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:10.052+0000 7efd27e5a640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efd201089f0 0x7efd201a0740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:11:10.054 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:10.052+0000 7efd27e5a640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7efd2010e730 con 0x7efd20103cf0 2024-05-20T16:11:10.054 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:10.052+0000 7efd27e5a640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7efd2010e5b0 con 0x7efd201089f0 2024-05-20T16:11:10.054 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:10.052+0000 7efd27e5a640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7efd2010e8b0 con 0x7efd20102ab0 2024-05-20T16:11:10.054 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:10.052+0000 7efd253f5640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efd20103cf0 0x7efd201997b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:11:10.054 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:10.052+0000 7efd253f5640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efd20103cf0 0x7efd201997b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:54500/0 (socket says 172.21.15.82:54500) 2024-05-20T16:11:10.054 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:10.052+0000 7efd253f5640 1 -- 172.21.15.82:0/1174851387 learned_addr learned my addr 172.21.15.82:0/1174851387 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:11:10.054 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:10.052+0000 7efd25bf6640 1 --2- 172.21.15.82:0/1174851387 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efd20102ab0 0x7efd20199270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:11:10.055 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:10.052+0000 7efd263f7640 1 --2- 172.21.15.82:0/1174851387 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efd201089f0 0x7efd201a0740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:11:10.055 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:10.052+0000 7efd253f5640 1 -- 172.21.15.82:0/1174851387 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efd20102ab0 msgr2=0x7efd20199270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:11:10.055 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:10.052+0000 7efd253f5640 1 --2- 172.21.15.82:0/1174851387 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efd20102ab0 0x7efd20199270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:10.055 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:10.052+0000 7efd253f5640 1 -- 172.21.15.82:0/1174851387 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efd201089f0 msgr2=0x7efd201a0740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:11:10.055 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:10.052+0000 7efd253f5640 1 --2- 172.21.15.82:0/1174851387 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efd201089f0 0x7efd201a0740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:10.055 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:10.052+0000 7efd253f5640 1 -- 172.21.15.82:0/1174851387 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd201a0db0 con 0x7efd20103cf0 2024-05-20T16:11:10.055 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:10.052+0000 7efd253f5640 1 --2- 172.21.15.82:0/1174851387 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efd20103cf0 0x7efd201997b0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7efd1000e9f0 tx=0x7efd1000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:11:10.055 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:10.052+0000 7efd16ffd640 1 -- 172.21.15.82:0/1174851387 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd1000ccc0 con 0x7efd20103cf0 2024-05-20T16:11:10.055 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:10.052+0000 7efd16ffd640 1 -- 172.21.15.82:0/1174851387 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efd1000ce60 con 0x7efd20103cf0 2024-05-20T16:11:10.055 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:10.052+0000 7efd16ffd640 1 -- 172.21.15.82:0/1174851387 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd10010690 con 0x7efd20103cf0 2024-05-20T16:11:10.055 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:10.052+0000 7efd27e5a640 1 -- 172.21.15.82:0/1174851387 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd201a10a0 con 0x7efd20103cf0 2024-05-20T16:11:10.055 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:10.053+0000 7efd27e5a640 1 -- 172.21.15.82:0/1174851387 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd201a15a0 con 0x7efd20103cf0 2024-05-20T16:11:10.056 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:10.054+0000 7efd27e5a640 1 -- 172.21.15.82:0/1174851387 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd20107800 con 0x7efd20103cf0 2024-05-20T16:11:10.056 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:10.055+0000 7efd16ffd640 1 -- 172.21.15.82:0/1174851387 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7efd100046c0 con 0x7efd20103cf0 2024-05-20T16:11:10.056 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:10.055+0000 7efd16ffd640 1 --2- 172.21.15.82:0/1174851387 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7efd0c0a2b90 0x7efd0c0a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:11:10.056 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:10.056+0000 7efd25bf6640 1 --2- 172.21.15.82:0/1174851387 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7efd0c0a2b90 0x7efd0c0a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:11:10.056 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:10.056+0000 7efd16ffd640 1 -- 172.21.15.82:0/1174851387 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7efd100cf130 con 0x7efd20103cf0 2024-05-20T16:11:10.057 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:10.056+0000 7efd25bf6640 1 --2- 172.21.15.82:0/1174851387 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7efd0c0a2b90 0x7efd0c0a50a0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7efd1c006010 tx=0x7efd1c000f40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:11:10.060 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:10.060+0000 7efd16ffd640 1 -- 172.21.15.82:0/1174851387 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7efd1009c530 con 0x7efd20103cf0 2024-05-20T16:11:10.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:10.275+0000 7efd27e5a640 1 -- 172.21.15.82:0/1174851387 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efd2002d2a0 con 0x7efd20103cf0 2024-05-20T16:11:10.276 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:10.276+0000 7efd16ffd640 1 -- 172.21.15.82:0/1174851387 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+569 (secure 0 0 0) 0x7efd1009be30 con 0x7efd20103cf0 2024-05-20T16:11:10.276 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-05-20T16:11:10.276 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-05-20T16:11:10.276 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 1 2024-05-20T16:11:10.278 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:10.278+0000 7efd27e5a640 1 -- 172.21.15.82:0/1174851387 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7efd0c0a2b90 msgr2=0x7efd0c0a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:11:10.278 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:10.278+0000 7efd27e5a640 1 --2- 172.21.15.82:0/1174851387 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7efd0c0a2b90 0x7efd0c0a50a0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7efd1c006010 tx=0x7efd1c000f40 comp rx=0 tx=0).stop 2024-05-20T16:11:10.278 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:10.278+0000 7efd27e5a640 1 -- 172.21.15.82:0/1174851387 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efd20103cf0 msgr2=0x7efd201997b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:11:10.278 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:10.278+0000 7efd27e5a640 1 --2- 172.21.15.82:0/1174851387 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efd20103cf0 0x7efd201997b0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7efd1000e9f0 tx=0x7efd1000eeb0 comp rx=0 tx=0).stop 2024-05-20T16:11:10.278 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:10.278+0000 7efd27e5a640 1 -- 172.21.15.82:0/1174851387 shutdown_connections 2024-05-20T16:11:10.278 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:10.278+0000 7efd27e5a640 1 --2- 172.21.15.82:0/1174851387 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7efd0c0a2b90 0x7efd0c0a50a0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:10.278 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:10.278+0000 7efd27e5a640 1 --2- 172.21.15.82:0/1174851387 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efd20103cf0 0x7efd201997b0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:10.278 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:10.278+0000 7efd27e5a640 1 --2- 172.21.15.82:0/1174851387 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efd201089f0 0x7efd201a0740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:10.278 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:10.278+0000 7efd27e5a640 1 --2- 172.21.15.82:0/1174851387 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efd20102ab0 0x7efd20199270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:10.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:10.278+0000 7efd27e5a640 1 -- 172.21.15.82:0/1174851387 >> 172.21.15.82:0/1174851387 conn(0x7efd200fdcc0 msgr2=0x7efd201049d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:11:10.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:10.278+0000 7efd27e5a640 1 -- 172.21.15.82:0/1174851387 shutdown_connections 2024-05-20T16:11:10.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:10.278+0000 7efd27e5a640 1 -- 172.21.15.82:0/1174851387 wait complete. 2024-05-20T16:11:10.562 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:10 smithi160 ceph-mon[36347]: Deploying daemon mds.l on smithi184 2024-05-20T16:11:10.562 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:10 smithi160 ceph-mon[36347]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-20T16:11:10.562 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:10 smithi160 ceph-mon[36347]: from='client.? 172.21.15.82:0/2582965044' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-20T16:11:10.562 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:10 smithi160 ceph-mon[36347]: from='client.? 172.21.15.82:0/2109652067' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-20T16:11:10.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:10 smithi082 ceph-mon[32238]: Deploying daemon mds.l on smithi184 2024-05-20T16:11:10.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:10 smithi082 ceph-mon[32238]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-20T16:11:10.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:10 smithi082 ceph-mon[32238]: from='client.? 172.21.15.82:0/2582965044' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-20T16:11:10.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:10 smithi082 ceph-mon[32238]: from='client.? 172.21.15.82:0/2109652067' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-20T16:11:10.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:10 smithi184 ceph-mon[36209]: Deploying daemon mds.l on smithi184 2024-05-20T16:11:10.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:10 smithi184 ceph-mon[36209]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-20T16:11:10.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:10 smithi184 ceph-mon[36209]: from='client.? 172.21.15.82:0/2582965044' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-20T16:11:10.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:10 smithi184 ceph-mon[36209]: from='client.? 172.21.15.82:0/2109652067' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-20T16:11:11.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:11 smithi082 ceph-mon[32238]: from='client.? 172.21.15.82:0/1174851387' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T16:11:11.386 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:11 smithi184 ceph-mon[36209]: from='client.? 172.21.15.82:0/1174851387' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T16:11:11.562 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:11 smithi160 ceph-mon[36347]: from='client.? 172.21.15.82:0/1174851387' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T16:11:12.136 INFO:journalctl@ceph.mds.l.smithi184.stdout:May 20 16:11:11 smithi184 systemd[1]: Started Ceph mds.l for 59d1600a-16c2-11ef-bc9a-c7b262605968. 2024-05-20T16:11:12.290 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-20T16:11:12.394 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:12 smithi082 ceph-mon[32238]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-20T16:11:12.395 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:12 smithi082 ceph-mon[32238]: mds.? [v2:172.21.15.184:6832/4015144007,v1:172.21.15.184:6833/4015144007] up:boot 2024-05-20T16:11:12.395 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:12 smithi082 ceph-mon[32238]: fsmap 1 up:standby 2024-05-20T16:11:12.395 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:12 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-20T16:11:12.395 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:12 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:11:12.395 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:12 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:11:12.395 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:12 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:11:12.395 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:12 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-20T16:11:12.395 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:12 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-20T16:11:12.395 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:12 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:11:12.405 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:12.405+0000 7fe8b3a2b640 1 -- 172.21.15.82:0/460029943 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe8ac102900 msgr2=0x7fe8ac102d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:11:12.406 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:12.405+0000 7fe8b3a2b640 1 --2- 172.21.15.82:0/460029943 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe8ac102900 0x7fe8ac102d90 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fe8a8009900 tx=0x7fe8a802a410 comp rx=0 tx=0).stop 2024-05-20T16:11:12.406 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:12.405+0000 7fe8b3a2b640 1 -- 172.21.15.82:0/460029943 shutdown_connections 2024-05-20T16:11:12.406 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:12.405+0000 7fe8b3a2b640 1 --2- 172.21.15.82:0/460029943 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe8ac102900 0x7fe8ac102d90 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:12.406 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:12.405+0000 7fe8b3a2b640 1 --2- 172.21.15.82:0/460029943 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe8ac103420 0x7fe8ac109b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:12.406 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:12.405+0000 7fe8b3a2b640 1 --2- 172.21.15.82:0/460029943 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe8ac101f20 0x7fe8ac102330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:12.406 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:12.405+0000 7fe8b3a2b640 1 -- 172.21.15.82:0/460029943 >> 172.21.15.82:0/460029943 conn(0x7fe8ac0fc360 msgr2=0x7fe8ac0fe7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:11:12.406 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:12.405+0000 7fe8b3a2b640 1 -- 172.21.15.82:0/460029943 shutdown_connections 2024-05-20T16:11:12.406 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:12.405+0000 7fe8b3a2b640 1 -- 172.21.15.82:0/460029943 wait complete. 2024-05-20T16:11:12.408 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:12.406+0000 7fe8b3a2b640 1 Processor -- start 2024-05-20T16:11:12.409 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:12.406+0000 7fe8b3a2b640 1 -- start start 2024-05-20T16:11:12.409 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:12.406+0000 7fe8b3a2b640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe8ac101f20 0x7fe8ac1011e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:11:12.409 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:12.406+0000 7fe8b3a2b640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe8ac102900 0x7fe8ac19e040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:11:12.409 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:12.406+0000 7fe8b3a2b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe8ac103420 0x7fe8ac1a05b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:11:12.409 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:12.406+0000 7fe8b3a2b640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fe8ac10d640 con 0x7fe8ac103420 2024-05-20T16:11:12.409 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:12.406+0000 7fe8b3a2b640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fe8ac10d4c0 con 0x7fe8ac101f20 2024-05-20T16:11:12.409 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:12.406+0000 7fe8b3a2b640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fe8ac10d7c0 con 0x7fe8ac102900 2024-05-20T16:11:12.409 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:12.406+0000 7fe8b1fc8640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe8ac103420 0x7fe8ac1a05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:11:12.409 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:12.406+0000 7fe8b1fc8640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe8ac103420 0x7fe8ac1a05b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:54532/0 (socket says 172.21.15.82:54532) 2024-05-20T16:11:12.409 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:12.406+0000 7fe8b1fc8640 1 -- 172.21.15.82:0/1024368136 learned_addr learned my addr 172.21.15.82:0/1024368136 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:11:12.409 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:12.407+0000 7fe8b17c7640 1 --2- 172.21.15.82:0/1024368136 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe8ac101f20 0x7fe8ac1011e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:11:12.410 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:12.407+0000 7fe8b0fc6640 1 --2- 172.21.15.82:0/1024368136 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe8ac102900 0x7fe8ac19e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:11:12.410 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:12.407+0000 7fe8b1fc8640 1 -- 172.21.15.82:0/1024368136 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe8ac102900 msgr2=0x7fe8ac19e040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:11:12.410 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:12.407+0000 7fe8b1fc8640 1 --2- 172.21.15.82:0/1024368136 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe8ac102900 0x7fe8ac19e040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:12.410 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:12.407+0000 7fe8b1fc8640 1 -- 172.21.15.82:0/1024368136 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe8ac101f20 msgr2=0x7fe8ac1011e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:11:12.410 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:12.407+0000 7fe8b1fc8640 1 --2- 172.21.15.82:0/1024368136 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe8ac101f20 0x7fe8ac1011e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:12.410 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:12.407+0000 7fe8b1fc8640 1 -- 172.21.15.82:0/1024368136 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8ac1a0dd0 con 0x7fe8ac103420 2024-05-20T16:11:12.410 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:12.407+0000 7fe8b1fc8640 1 --2- 172.21.15.82:0/1024368136 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe8ac103420 0x7fe8ac1a05b0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fe8a4009fd0 tx=0x7fe8a400b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:11:12.410 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:12.407+0000 7fe8a27fc640 1 -- 172.21.15.82:0/1024368136 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8a4017070 con 0x7fe8ac103420 2024-05-20T16:11:12.410 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:12.407+0000 7fe8a27fc640 1 -- 172.21.15.82:0/1024368136 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe8a4003e50 con 0x7fe8ac103420 2024-05-20T16:11:12.410 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:12.407+0000 7fe8b3a2b640 1 -- 172.21.15.82:0/1024368136 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8ac1a10c0 con 0x7fe8ac103420 2024-05-20T16:11:12.410 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:12.407+0000 7fe8a27fc640 1 -- 172.21.15.82:0/1024368136 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8a40106c0 con 0x7fe8ac103420 2024-05-20T16:11:12.410 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:12.407+0000 7fe8b3a2b640 1 -- 172.21.15.82:0/1024368136 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8ac1a15c0 con 0x7fe8ac103420 2024-05-20T16:11:12.410 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:12.408+0000 7fe8b3a2b640 1 -- 172.21.15.82:0/1024368136 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8ac1068c0 con 0x7fe8ac103420 2024-05-20T16:11:12.411 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:12.409+0000 7fe8a27fc640 1 -- 172.21.15.82:0/1024368136 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fe8a4003970 con 0x7fe8ac103420 2024-05-20T16:11:12.411 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:12.410+0000 7fe8a27fc640 1 --2- 172.21.15.82:0/1024368136 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fe8980a2b90 0x7fe8980a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:11:12.411 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:12.410+0000 7fe8b17c7640 1 --2- 172.21.15.82:0/1024368136 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fe8980a2b90 0x7fe8980a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:11:12.411 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:12.410+0000 7fe8a27fc640 1 -- 172.21.15.82:0/1024368136 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fe8a40ce940 con 0x7fe8ac103420 2024-05-20T16:11:12.411 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:12.411+0000 7fe8b17c7640 1 --2- 172.21.15.82:0/1024368136 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fe8980a2b90 0x7fe8980a50a0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fe89c005fd0 tx=0x7fe89c00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:11:12.414 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:12.415+0000 7fe8a27fc640 1 -- 172.21.15.82:0/1024368136 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fe8a409bd40 con 0x7fe8ac103420 2024-05-20T16:11:12.490 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:12 smithi160 ceph-mon[36347]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-20T16:11:12.490 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:12 smithi160 ceph-mon[36347]: mds.? [v2:172.21.15.184:6832/4015144007,v1:172.21.15.184:6833/4015144007] up:boot 2024-05-20T16:11:12.490 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:12 smithi160 ceph-mon[36347]: fsmap 1 up:standby 2024-05-20T16:11:12.490 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:12 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-20T16:11:12.490 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:12 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:11:12.490 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:12 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:11:12.490 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:12 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:11:12.490 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:12 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-20T16:11:12.490 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:12 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-20T16:11:12.491 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:12 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:11:12.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:12 smithi184 ceph-mon[36209]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-20T16:11:12.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:12 smithi184 ceph-mon[36209]: mds.? [v2:172.21.15.184:6832/4015144007,v1:172.21.15.184:6833/4015144007] up:boot 2024-05-20T16:11:12.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:12 smithi184 ceph-mon[36209]: fsmap 1 up:standby 2024-05-20T16:11:12.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:12 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-20T16:11:12.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:12 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:11:12.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:12 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:11:12.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:12 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:11:12.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:12 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-20T16:11:12.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:12 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-20T16:11:12.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:12 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:11:12.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:12.637+0000 7fe8b3a2b640 1 -- 172.21.15.82:0/1024368136 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe8ac062750 con 0x7fe8ac103420 2024-05-20T16:11:12.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:12.638+0000 7fe8a27fc640 1 -- 172.21.15.82:0/1024368136 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1396 (secure 0 0 0) 0x7fe8a4097020 con 0x7fe8ac103420 2024-05-20T16:11:12.638 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-05-20T16:11:12.638 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6833/4015144007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":4015144007},{"type":"v1","addr":"172.21.15.184:6833","nonce":4015144007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-05-20T16:11:12.638 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 2 2024-05-20T16:11:12.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:12.640+0000 7fe8b3a2b640 1 -- 172.21.15.82:0/1024368136 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fe8980a2b90 msgr2=0x7fe8980a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:11:12.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:12.640+0000 7fe8b3a2b640 1 --2- 172.21.15.82:0/1024368136 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fe8980a2b90 0x7fe8980a50a0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fe89c005fd0 tx=0x7fe89c00b390 comp rx=0 tx=0).stop 2024-05-20T16:11:12.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:12.640+0000 7fe8b3a2b640 1 -- 172.21.15.82:0/1024368136 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe8ac103420 msgr2=0x7fe8ac1a05b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:11:12.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:12.640+0000 7fe8b3a2b640 1 --2- 172.21.15.82:0/1024368136 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe8ac103420 0x7fe8ac1a05b0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fe8a4009fd0 tx=0x7fe8a400b260 comp rx=0 tx=0).stop 2024-05-20T16:11:12.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:12.640+0000 7fe8b3a2b640 1 -- 172.21.15.82:0/1024368136 shutdown_connections 2024-05-20T16:11:12.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:12.640+0000 7fe8b3a2b640 1 --2- 172.21.15.82:0/1024368136 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fe8980a2b90 0x7fe8980a50a0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:12.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:12.640+0000 7fe8b3a2b640 1 --2- 172.21.15.82:0/1024368136 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe8ac103420 0x7fe8ac1a05b0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:12.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:12.640+0000 7fe8b3a2b640 1 --2- 172.21.15.82:0/1024368136 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe8ac102900 0x7fe8ac19e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:12.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:12.640+0000 7fe8b3a2b640 1 --2- 172.21.15.82:0/1024368136 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe8ac101f20 0x7fe8ac1011e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:12.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:12.640+0000 7fe8b3a2b640 1 -- 172.21.15.82:0/1024368136 >> 172.21.15.82:0/1024368136 conn(0x7fe8ac0fc360 msgr2=0x7fe8ac103a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:11:12.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:12.640+0000 7fe8b3a2b640 1 -- 172.21.15.82:0/1024368136 shutdown_connections 2024-05-20T16:11:12.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:12.640+0000 7fe8b3a2b640 1 -- 172.21.15.82:0/1024368136 wait complete. 2024-05-20T16:11:13.518 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:13 smithi160 ceph-mon[36347]: Deploying daemon mds.k on smithi160 2024-05-20T16:11:13.518 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:13 smithi160 ceph-mon[36347]: from='client.? 172.21.15.82:0/1024368136' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T16:11:13.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:13 smithi082 ceph-mon[32238]: Deploying daemon mds.k on smithi160 2024-05-20T16:11:13.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:13 smithi082 ceph-mon[32238]: from='client.? 172.21.15.82:0/1024368136' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T16:11:13.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:13 smithi184 ceph-mon[36209]: Deploying daemon mds.k on smithi160 2024-05-20T16:11:13.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:13 smithi184 ceph-mon[36209]: from='client.? 172.21.15.82:0/1024368136' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T16:11:14.217 INFO:journalctl@ceph.mds.k.smithi160.stdout:May 20 16:11:14 smithi160 systemd[1]: Starting Ceph mds.k for 59d1600a-16c2-11ef-bc9a-c7b262605968... 2024-05-20T16:11:14.562 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:14 smithi160 ceph-mon[36347]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-20T16:11:14.562 INFO:journalctl@ceph.mds.k.smithi160.stdout:May 20 16:11:14 smithi160 podman[65994]: 2024-05-20 16:11:14.189406253 +0000 UTC m=+0.017699829 image pull cfcc61b2d476f611479310eb042aa05f11dd5f60dbb65e3b0d7e0e11d024cdb3 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708 2024-05-20T16:11:14.563 INFO:journalctl@ceph.mds.k.smithi160.stdout:May 20 16:11:14 smithi160 podman[65994]: 2024-05-20 16:11:14.301601295 +0000 UTC m=+0.129894859 container create d601d02073592d37fbf7da1fb7daeb0799a3c21be670fc353d723ad8100a1a49 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-k, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240514, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-20T16:11:14.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:14 smithi082 ceph-mon[32238]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-20T16:11:14.635 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:14 smithi184 ceph-mon[36209]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-20T16:11:14.653 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-20T16:11:14.780 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:14.779+0000 7faf0c35e640 1 -- 172.21.15.82:0/2411209184 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faf04102ab0 msgr2=0x7faf041037b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:11:14.781 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:14.779+0000 7faf0c35e640 1 --2- 172.21.15.82:0/2411209184 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faf04102ab0 0x7faf041037b0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7faf00009900 tx=0x7faf0002a410 comp rx=0 tx=0).stop 2024-05-20T16:11:14.781 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:14.779+0000 7faf0c35e640 1 -- 172.21.15.82:0/2411209184 shutdown_connections 2024-05-20T16:11:14.781 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:14.779+0000 7faf0c35e640 1 --2- 172.21.15.82:0/2411209184 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faf041089f0 0x7faf0410ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:14.781 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:14.779+0000 7faf0c35e640 1 --2- 172.21.15.82:0/2411209184 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faf04103cf0 0x7faf04104100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:14.781 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:14.779+0000 7faf0c35e640 1 --2- 172.21.15.82:0/2411209184 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faf04102ab0 0x7faf041037b0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:14.781 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:14.779+0000 7faf0c35e640 1 -- 172.21.15.82:0/2411209184 >> 172.21.15.82:0/2411209184 conn(0x7faf040fdcc0 msgr2=0x7faf04100150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:11:14.781 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:14.779+0000 7faf0c35e640 1 -- 172.21.15.82:0/2411209184 shutdown_connections 2024-05-20T16:11:14.781 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:14.779+0000 7faf0c35e640 1 -- 172.21.15.82:0/2411209184 wait complete. 2024-05-20T16:11:14.783 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:14.780+0000 7faf0c35e640 1 Processor -- start 2024-05-20T16:11:14.783 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:14.780+0000 7faf0c35e640 1 -- start start 2024-05-20T16:11:14.783 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:14.780+0000 7faf0c35e640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faf04102ab0 0x7faf04199300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:11:14.783 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:14.780+0000 7faf0c35e640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faf04103cf0 0x7faf04199840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:11:14.783 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:14.780+0000 7faf0c35e640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faf041089f0 0x7faf041a07d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:11:14.783 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:14.780+0000 7faf0c35e640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7faf0410e700 con 0x7faf041089f0 2024-05-20T16:11:14.783 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:14.780+0000 7faf0c35e640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7faf0410e580 con 0x7faf04103cf0 2024-05-20T16:11:14.783 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:14.780+0000 7faf0c35e640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7faf0410e880 con 0x7faf04102ab0 2024-05-20T16:11:14.783 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:14.780+0000 7faf0a8fb640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faf041089f0 0x7faf041a07d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:11:14.783 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:14.780+0000 7faf0a8fb640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faf041089f0 0x7faf041a07d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:54564/0 (socket says 172.21.15.82:54564) 2024-05-20T16:11:14.784 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:14.781+0000 7faf0a8fb640 1 -- 172.21.15.82:0/1980175794 learned_addr learned my addr 172.21.15.82:0/1980175794 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:11:14.784 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:14.781+0000 7faf0a0fa640 1 --2- 172.21.15.82:0/1980175794 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faf04102ab0 0x7faf04199300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:11:14.784 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:14.781+0000 7faf098f9640 1 --2- 172.21.15.82:0/1980175794 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faf04103cf0 0x7faf04199840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:11:14.784 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:14.781+0000 7faf0a8fb640 1 -- 172.21.15.82:0/1980175794 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faf04102ab0 msgr2=0x7faf04199300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:11:14.784 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:14.781+0000 7faf0a8fb640 1 --2- 172.21.15.82:0/1980175794 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faf04102ab0 0x7faf04199300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:14.784 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:14.781+0000 7faf0a8fb640 1 -- 172.21.15.82:0/1980175794 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faf04103cf0 msgr2=0x7faf04199840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:11:14.784 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:14.781+0000 7faf0a8fb640 1 --2- 172.21.15.82:0/1980175794 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faf04103cf0 0x7faf04199840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:14.784 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:14.781+0000 7faf0a8fb640 1 -- 172.21.15.82:0/1980175794 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf041a0d90 con 0x7faf041089f0 2024-05-20T16:11:14.784 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:14.781+0000 7faf0a0fa640 1 --2- 172.21.15.82:0/1980175794 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faf04102ab0 0x7faf04199300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T16:11:14.784 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:14.781+0000 7faf0a8fb640 1 --2- 172.21.15.82:0/1980175794 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faf041089f0 0x7faf041a07d0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7faefc00cad0 tx=0x7faefc00ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:11:14.784 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:14.781+0000 7faefb7fe640 1 -- 172.21.15.82:0/1980175794 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faefc00ad60 con 0x7faf041089f0 2024-05-20T16:11:14.784 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:14.781+0000 7faefb7fe640 1 -- 172.21.15.82:0/1980175794 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faefc005020 con 0x7faf041089f0 2024-05-20T16:11:14.784 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:14.781+0000 7faf0c35e640 1 -- 172.21.15.82:0/1980175794 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf041a1080 con 0x7faf041089f0 2024-05-20T16:11:14.784 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:14.781+0000 7faefb7fe640 1 -- 172.21.15.82:0/1980175794 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faefc00f6b0 con 0x7faf041089f0 2024-05-20T16:11:14.785 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:14.781+0000 7faf0c35e640 1 -- 172.21.15.82:0/1980175794 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf041a15d0 con 0x7faf041089f0 2024-05-20T16:11:14.785 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:14.782+0000 7faf0c35e640 1 -- 172.21.15.82:0/1980175794 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf04107800 con 0x7faf041089f0 2024-05-20T16:11:14.785 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:14.784+0000 7faefb7fe640 1 -- 172.21.15.82:0/1980175794 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7faefc005560 con 0x7faf041089f0 2024-05-20T16:11:14.785 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:14.784+0000 7faefb7fe640 1 --2- 172.21.15.82:0/1980175794 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7faef00a2b90 0x7faef00a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:11:14.785 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:14.785+0000 7faf0a0fa640 1 --2- 172.21.15.82:0/1980175794 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7faef00a2b90 0x7faef00a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:11:14.786 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:14.785+0000 7faf0a0fa640 1 --2- 172.21.15.82:0/1980175794 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7faef00a2b90 0x7faef00a50a0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7faf00009230 tx=0x7faf00031510 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:11:14.786 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:14.785+0000 7faefb7fe640 1 -- 172.21.15.82:0/1980175794 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7faefc099d60 con 0x7faf041089f0 2024-05-20T16:11:14.791 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:14.790+0000 7faefb7fe640 1 -- 172.21.15.82:0/1980175794 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7faefc014030 con 0x7faf041089f0 2024-05-20T16:11:15.015 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:15.014+0000 7faf0c35e640 1 -- 172.21.15.82:0/1980175794 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7faf0402d2a0 con 0x7faf041089f0 2024-05-20T16:11:15.015 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:15.014+0000 7faefb7fe640 1 -- 172.21.15.82:0/1980175794 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1396 (secure 0 0 0) 0x7faefc09c3c0 con 0x7faf041089f0 2024-05-20T16:11:15.015 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-05-20T16:11:15.015 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6833/4015144007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":4015144007},{"type":"v1","addr":"172.21.15.184:6833","nonce":4015144007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-05-20T16:11:15.016 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 2 2024-05-20T16:11:15.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:15.016+0000 7faf0c35e640 1 -- 172.21.15.82:0/1980175794 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7faef00a2b90 msgr2=0x7faef00a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:11:15.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:15.016+0000 7faf0c35e640 1 --2- 172.21.15.82:0/1980175794 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7faef00a2b90 0x7faef00a50a0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7faf00009230 tx=0x7faf00031510 comp rx=0 tx=0).stop 2024-05-20T16:11:15.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:15.016+0000 7faf0c35e640 1 -- 172.21.15.82:0/1980175794 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faf041089f0 msgr2=0x7faf041a07d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:11:15.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:15.016+0000 7faf0c35e640 1 --2- 172.21.15.82:0/1980175794 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faf041089f0 0x7faf041a07d0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7faefc00cad0 tx=0x7faefc00ced0 comp rx=0 tx=0).stop 2024-05-20T16:11:15.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:15.017+0000 7faf0c35e640 1 -- 172.21.15.82:0/1980175794 shutdown_connections 2024-05-20T16:11:15.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:15.017+0000 7faf0c35e640 1 --2- 172.21.15.82:0/1980175794 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7faef00a2b90 0x7faef00a50a0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:15.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:15.017+0000 7faf0c35e640 1 --2- 172.21.15.82:0/1980175794 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faf041089f0 0x7faf041a07d0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:15.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:15.017+0000 7faf0c35e640 1 --2- 172.21.15.82:0/1980175794 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faf04103cf0 0x7faf04199840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:15.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:15.017+0000 7faf0c35e640 1 --2- 172.21.15.82:0/1980175794 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faf04102ab0 0x7faf04199300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:15.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:15.017+0000 7faf0c35e640 1 -- 172.21.15.82:0/1980175794 >> 172.21.15.82:0/1980175794 conn(0x7faf040fdcc0 msgr2=0x7faf041049d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:11:15.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:15.017+0000 7faf0c35e640 1 -- 172.21.15.82:0/1980175794 shutdown_connections 2024-05-20T16:11:15.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:15.017+0000 7faf0c35e640 1 -- 172.21.15.82:0/1980175794 wait complete. 2024-05-20T16:11:15.062 INFO:journalctl@ceph.mds.k.smithi160.stdout:May 20 16:11:14 smithi160 podman[65994]: 2024-05-20 16:11:14.570437234 +0000 UTC m=+0.398730798 container init d601d02073592d37fbf7da1fb7daeb0799a3c21be670fc353d723ad8100a1a49 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-k, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240514, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, org.label-schema.vendor=CentOS) 2024-05-20T16:11:15.062 INFO:journalctl@ceph.mds.k.smithi160.stdout:May 20 16:11:14 smithi160 podman[65994]: 2024-05-20 16:11:14.573192871 +0000 UTC m=+0.401486434 container start d601d02073592d37fbf7da1fb7daeb0799a3c21be670fc353d723ad8100a1a49 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-k, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240514, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-05-20T16:11:15.062 INFO:journalctl@ceph.mds.k.smithi160.stdout:May 20 16:11:14 smithi160 ceph-mds[66013]: -- 172.21.15.160:0/9104466 <== mon.2 v2:172.21.15.160:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5628d217ba40 con 0x5628d21f3600 2024-05-20T16:11:15.062 INFO:journalctl@ceph.mds.k.smithi160.stdout:May 20 16:11:14 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-k[66009]: starting mds.k at 2024-05-20T16:11:15.062 INFO:journalctl@ceph.mds.k.smithi160.stdout:May 20 16:11:14 smithi160 bash[65994]: d601d02073592d37fbf7da1fb7daeb0799a3c21be670fc353d723ad8100a1a49 2024-05-20T16:11:15.063 INFO:journalctl@ceph.mds.k.smithi160.stdout:May 20 16:11:14 smithi160 systemd[1]: Started Ceph mds.k for 59d1600a-16c2-11ef-bc9a-c7b262605968. 2024-05-20T16:11:16.062 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:15 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:11:16.062 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:15 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:11:16.062 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:15 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:11:16.063 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:15 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-20T16:11:16.063 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:15 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-20T16:11:16.063 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:15 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:11:16.063 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:15 smithi160 ceph-mon[36347]: Deploying daemon mds.j on smithi082 2024-05-20T16:11:16.063 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:15 smithi160 ceph-mon[36347]: from='client.? 172.21.15.82:0/1980175794' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T16:11:16.063 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:15 smithi160 ceph-mon[36347]: mds.? [v2:172.21.15.160:6832/975243372,v1:172.21.15.160:6833/975243372] up:boot 2024-05-20T16:11:16.063 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:15 smithi160 ceph-mon[36347]: fsmap 2 up:standby 2024-05-20T16:11:16.063 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:15 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-20T16:11:16.063 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:15 smithi160 ceph-mon[36347]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-20T16:11:16.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:15 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:11:16.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:15 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:11:16.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:15 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:11:16.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:15 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-20T16:11:16.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:15 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-20T16:11:16.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:15 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:11:16.110 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:15 smithi082 ceph-mon[32238]: Deploying daemon mds.j on smithi082 2024-05-20T16:11:16.110 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:15 smithi082 ceph-mon[32238]: from='client.? 172.21.15.82:0/1980175794' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T16:11:16.110 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:15 smithi082 ceph-mon[32238]: mds.? [v2:172.21.15.160:6832/975243372,v1:172.21.15.160:6833/975243372] up:boot 2024-05-20T16:11:16.110 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:15 smithi082 ceph-mon[32238]: fsmap 2 up:standby 2024-05-20T16:11:16.110 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:15 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-20T16:11:16.110 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:15 smithi082 ceph-mon[32238]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-20T16:11:16.136 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:15 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:11:16.136 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:15 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:11:16.136 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:15 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:11:16.136 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:15 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-20T16:11:16.136 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:15 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-20T16:11:16.136 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:15 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:11:16.136 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:15 smithi184 ceph-mon[36209]: Deploying daemon mds.j on smithi082 2024-05-20T16:11:16.136 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:15 smithi184 ceph-mon[36209]: from='client.? 172.21.15.82:0/1980175794' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T16:11:16.136 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:15 smithi184 ceph-mon[36209]: mds.? [v2:172.21.15.160:6832/975243372,v1:172.21.15.160:6833/975243372] up:boot 2024-05-20T16:11:16.136 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:15 smithi184 ceph-mon[36209]: fsmap 2 up:standby 2024-05-20T16:11:16.136 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:15 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-20T16:11:16.136 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:15 smithi184 ceph-mon[36209]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-20T16:11:17.030 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-20T16:11:17.158 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:17.156+0000 7f99f3661640 1 -- 172.21.15.82:0/3544037393 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f99ec102e80 msgr2=0x7f99ec101760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:11:17.158 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:17.156+0000 7f99f3661640 1 --2- 172.21.15.82:0/3544037393 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f99ec102e80 0x7f99ec101760 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f99e8009900 tx=0x7f99e802a410 comp rx=0 tx=0).stop 2024-05-20T16:11:17.158 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:17.156+0000 7f99f3661640 1 -- 172.21.15.82:0/3544037393 shutdown_connections 2024-05-20T16:11:17.158 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:17.156+0000 7f99f3661640 1 --2- 172.21.15.82:0/3544037393 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f99ec1079b0 0x7f99ec109dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:17.158 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:17.156+0000 7f99f3661640 1 --2- 172.21.15.82:0/3544037393 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f99ec103360 0x7f99ec101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:17.158 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:17.156+0000 7f99f3661640 1 --2- 172.21.15.82:0/3544037393 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f99ec102e80 0x7f99ec101760 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:17.158 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:17.156+0000 7f99f3661640 1 -- 172.21.15.82:0/3544037393 >> 172.21.15.82:0/3544037393 conn(0x7f99ec0fc2e0 msgr2=0x7f99ec0fe7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:11:17.159 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:17.156+0000 7f99f3661640 1 -- 172.21.15.82:0/3544037393 shutdown_connections 2024-05-20T16:11:17.159 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:17.156+0000 7f99f3661640 1 -- 172.21.15.82:0/3544037393 wait complete. 2024-05-20T16:11:17.159 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:17.156+0000 7f99f3661640 1 Processor -- start 2024-05-20T16:11:17.159 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:17.156+0000 7f99f3661640 1 -- start start 2024-05-20T16:11:17.159 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:17.157+0000 7f99f3661640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f99ec102e80 0x7f99ec1010e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:11:17.159 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:17.157+0000 7f99f3661640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f99ec1079b0 0x7f99ec19dc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:11:17.159 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:17.157+0000 7f99f3661640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f99ec19e170 0x7f99ec1a05b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:11:17.159 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:17.157+0000 7f99f3661640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f99ec10d5e0 con 0x7f99ec1079b0 2024-05-20T16:11:17.159 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:17.157+0000 7f99f3661640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f99ec10d460 con 0x7f99ec19e170 2024-05-20T16:11:17.159 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:17.157+0000 7f99f3661640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f99ec10d760 con 0x7f99ec102e80 2024-05-20T16:11:17.159 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:17.157+0000 7f99f0bfc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f99ec1079b0 0x7f99ec19dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:11:17.159 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:17.157+0000 7f99f0bfc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f99ec1079b0 0x7f99ec19dc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:54594/0 (socket says 172.21.15.82:54594) 2024-05-20T16:11:17.159 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:17.157+0000 7f99f0bfc640 1 -- 172.21.15.82:0/3499883970 learned_addr learned my addr 172.21.15.82:0/3499883970 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:11:17.159 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:17.157+0000 7f99f13fd640 1 --2- 172.21.15.82:0/3499883970 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f99ec102e80 0x7f99ec1010e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:11:17.160 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:17.157+0000 7f99f1bfe640 1 --2- 172.21.15.82:0/3499883970 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f99ec19e170 0x7f99ec1a05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:11:17.160 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:17.157+0000 7f99f0bfc640 1 -- 172.21.15.82:0/3499883970 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f99ec102e80 msgr2=0x7f99ec1010e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:11:17.160 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:17.157+0000 7f99f0bfc640 1 --2- 172.21.15.82:0/3499883970 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f99ec102e80 0x7f99ec1010e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:17.160 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:17.157+0000 7f99f0bfc640 1 -- 172.21.15.82:0/3499883970 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f99ec19e170 msgr2=0x7f99ec1a05b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:11:17.160 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:17.157+0000 7f99f0bfc640 1 --2- 172.21.15.82:0/3499883970 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f99ec19e170 0x7f99ec1a05b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:17.160 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:17.157+0000 7f99f0bfc640 1 -- 172.21.15.82:0/3499883970 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99ec1a0dd0 con 0x7f99ec1079b0 2024-05-20T16:11:17.160 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:17.157+0000 7f99f1bfe640 1 --2- 172.21.15.82:0/3499883970 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f99ec19e170 0x7f99ec1a05b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T16:11:17.160 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:17.157+0000 7f99f0bfc640 1 --2- 172.21.15.82:0/3499883970 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f99ec1079b0 0x7f99ec19dc30 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f99dc00e9f0 tx=0x7f99dc00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:11:17.160 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:17.157+0000 7f99e27fc640 1 -- 172.21.15.82:0/3499883970 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99dc00cd80 con 0x7f99ec1079b0 2024-05-20T16:11:17.160 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:17.157+0000 7f99e27fc640 1 -- 172.21.15.82:0/3499883970 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f99dc005500 con 0x7f99ec1079b0 2024-05-20T16:11:17.160 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:17.157+0000 7f99f3661640 1 -- 172.21.15.82:0/3499883970 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99ec1a10c0 con 0x7f99ec1079b0 2024-05-20T16:11:17.160 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:17.157+0000 7f99e27fc640 1 -- 172.21.15.82:0/3499883970 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99dc010690 con 0x7f99ec1079b0 2024-05-20T16:11:17.160 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:17.157+0000 7f99f3661640 1 -- 172.21.15.82:0/3499883970 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99ec1a15c0 con 0x7f99ec1079b0 2024-05-20T16:11:17.161 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:17.158+0000 7f99f3661640 1 -- 172.21.15.82:0/3499883970 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99ec1060b0 con 0x7f99ec1079b0 2024-05-20T16:11:17.161 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:17.161+0000 7f99e27fc640 1 -- 172.21.15.82:0/3499883970 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f99dc0046c0 con 0x7f99ec1079b0 2024-05-20T16:11:17.161 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:17.161+0000 7f99e27fc640 1 --2- 172.21.15.82:0/3499883970 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f99d80a2b90 0x7f99d80a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:11:17.161 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:17.161+0000 7f99f13fd640 1 --2- 172.21.15.82:0/3499883970 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f99d80a2b90 0x7f99d80a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:11:17.161 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:17.161+0000 7f99e27fc640 1 -- 172.21.15.82:0/3499883970 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f99dc0cf260 con 0x7f99ec1079b0 2024-05-20T16:11:17.162 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:17.162+0000 7f99f13fd640 1 --2- 172.21.15.82:0/3499883970 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f99d80a2b90 0x7f99d80a50a0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f99e8009230 tx=0x7f99e8031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:11:17.164 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:17.165+0000 7f99e27fc640 1 -- 172.21.15.82:0/3499883970 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f99dc053850 con 0x7f99ec1079b0 2024-05-20T16:11:17.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:17.390+0000 7f99f3661640 1 -- 172.21.15.82:0/3499883970 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f99ec02d2a0 con 0x7f99ec1079b0 2024-05-20T16:11:17.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:17.391+0000 7f99e27fc640 1 -- 172.21.15.82:0/3499883970 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2221 (secure 0 0 0) 0x7f99dc09c660 con 0x7f99ec1079b0 2024-05-20T16:11:17.391 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-05-20T16:11:17.391 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6833/4015144007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":4015144007},{"type":"v1","addr":"172.21.15.184:6833","nonce":4015144007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6833/975243372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6832","nonce":975243372},{"type":"v1","addr":"172.21.15.160:6833","nonce":975243372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-05-20T16:11:17.391 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 3 2024-05-20T16:11:17.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:17.393+0000 7f99f3661640 1 -- 172.21.15.82:0/3499883970 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f99d80a2b90 msgr2=0x7f99d80a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:11:17.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:17.393+0000 7f99f3661640 1 --2- 172.21.15.82:0/3499883970 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f99d80a2b90 0x7f99d80a50a0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f99e8009230 tx=0x7f99e8031400 comp rx=0 tx=0).stop 2024-05-20T16:11:17.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:17.393+0000 7f99f3661640 1 -- 172.21.15.82:0/3499883970 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f99ec1079b0 msgr2=0x7f99ec19dc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:11:17.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:17.393+0000 7f99f3661640 1 --2- 172.21.15.82:0/3499883970 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f99ec1079b0 0x7f99ec19dc30 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f99dc00e9f0 tx=0x7f99dc00edf0 comp rx=0 tx=0).stop 2024-05-20T16:11:17.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:17.393+0000 7f99f3661640 1 -- 172.21.15.82:0/3499883970 shutdown_connections 2024-05-20T16:11:17.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:17.393+0000 7f99f3661640 1 --2- 172.21.15.82:0/3499883970 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f99d80a2b90 0x7f99d80a50a0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:17.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:17.393+0000 7f99f3661640 1 --2- 172.21.15.82:0/3499883970 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f99ec1079b0 0x7f99ec19dc30 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:17.394 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:17.393+0000 7f99f3661640 1 --2- 172.21.15.82:0/3499883970 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f99ec19e170 0x7f99ec1a05b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:17.394 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:17.393+0000 7f99f3661640 1 --2- 172.21.15.82:0/3499883970 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f99ec102e80 0x7f99ec1010e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:17.394 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:17.393+0000 7f99f3661640 1 -- 172.21.15.82:0/3499883970 >> 172.21.15.82:0/3499883970 conn(0x7f99ec0fc2e0 msgr2=0x7f99ec0fe700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:11:17.394 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:17.393+0000 7f99f3661640 1 -- 172.21.15.82:0/3499883970 shutdown_connections 2024-05-20T16:11:17.396 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:17.393+0000 7f99f3661640 1 -- 172.21.15.82:0/3499883970 wait complete. 2024-05-20T16:11:17.859 INFO:journalctl@ceph.mds.j.smithi082.stdout:May 20 16:11:17 smithi082 systemd[1]: Started Ceph mds.j for 59d1600a-16c2-11ef-bc9a-c7b262605968. 2024-05-20T16:11:18.548 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:18 smithi082 ceph-mon[32238]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-20T16:11:18.548 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:18 smithi082 ceph-mon[32238]: from='client.? 172.21.15.82:0/3499883970' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T16:11:18.548 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:18 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:11:18.548 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:18 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:11:18.548 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:18 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:11:18.548 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:18 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-20T16:11:18.548 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:18 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-20T16:11:18.548 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:18 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:11:18.562 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:18 smithi160 ceph-mon[36347]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-20T16:11:18.562 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:18 smithi160 ceph-mon[36347]: from='client.? 172.21.15.82:0/3499883970' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T16:11:18.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:18 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:11:18.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:18 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:11:18.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:18 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:11:18.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:18 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-20T16:11:18.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:18 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-20T16:11:18.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:18 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:11:18.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:18 smithi184 ceph-mon[36209]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-20T16:11:18.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:18 smithi184 ceph-mon[36209]: from='client.? 172.21.15.82:0/3499883970' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T16:11:18.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:18 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:11:18.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:18 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:11:18.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:18 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:11:18.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:18 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-20T16:11:18.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:18 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-20T16:11:18.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:18 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:11:19.405 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-20T16:11:19.537 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:19.534+0000 7fc17581b640 1 -- 172.21.15.82:0/3349348292 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc170105bf0 msgr2=0x7fc170108090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:11:19.537 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:19.534+0000 7fc17581b640 1 --2- 172.21.15.82:0/3349348292 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc170105bf0 0x7fc170108090 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fc158009900 tx=0x7fc15802a410 comp rx=0 tx=0).stop 2024-05-20T16:11:19.537 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:19.534+0000 7fc17581b640 1 -- 172.21.15.82:0/3349348292 shutdown_connections 2024-05-20T16:11:19.537 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:19.534+0000 7fc17581b640 1 --2- 172.21.15.82:0/3349348292 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc170105bf0 0x7fc170108090 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:19.537 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:19.534+0000 7fc17581b640 1 --2- 172.21.15.82:0/3349348292 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc170108720 0x7fc17010ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:19.537 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:19.534+0000 7fc17581b640 1 --2- 172.21.15.82:0/3349348292 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc170103200 0x7fc170105620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:19.537 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:19.534+0000 7fc17581b640 1 -- 172.21.15.82:0/3349348292 >> 172.21.15.82:0/3349348292 conn(0x7fc1700fcba0 msgr2=0x7fc1700ff030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:11:19.537 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:19.534+0000 7fc17581b640 1 -- 172.21.15.82:0/3349348292 shutdown_connections 2024-05-20T16:11:19.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:19.534+0000 7fc17581b640 1 -- 172.21.15.82:0/3349348292 wait complete. 2024-05-20T16:11:19.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:19.535+0000 7fc17581b640 1 Processor -- start 2024-05-20T16:11:19.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:19.535+0000 7fc17581b640 1 -- start start 2024-05-20T16:11:19.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:19.535+0000 7fc17581b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc170103200 0x7fc170199310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:11:19.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:19.535+0000 7fc17581b640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc170105bf0 0x7fc170199850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:11:19.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:19.535+0000 7fc17581b640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc170108720 0x7fc1701a07e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:11:19.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:19.535+0000 7fc17581b640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fc17010e700 con 0x7fc170103200 2024-05-20T16:11:19.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:19.535+0000 7fc17581b640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fc17010e580 con 0x7fc170105bf0 2024-05-20T16:11:19.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:19.535+0000 7fc17581b640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fc17010e880 con 0x7fc170108720 2024-05-20T16:11:19.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:19.535+0000 7fc16effd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc170103200 0x7fc170199310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:11:19.539 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:19.535+0000 7fc16effd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc170103200 0x7fc170199310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43242/0 (socket says 172.21.15.82:43242) 2024-05-20T16:11:19.539 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:19.535+0000 7fc16effd640 1 -- 172.21.15.82:0/2213641137 learned_addr learned my addr 172.21.15.82:0/2213641137 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:11:19.539 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:19.536+0000 7fc16f7fe640 1 --2- 172.21.15.82:0/2213641137 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc170108720 0x7fc1701a07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:11:19.539 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:19.536+0000 7fc167fff640 1 --2- 172.21.15.82:0/2213641137 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc170105bf0 0x7fc170199850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:11:19.539 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:19.536+0000 7fc16effd640 1 -- 172.21.15.82:0/2213641137 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc170108720 msgr2=0x7fc1701a07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:11:19.539 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:19.536+0000 7fc16effd640 1 --2- 172.21.15.82:0/2213641137 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc170108720 0x7fc1701a07e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:19.539 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:19.536+0000 7fc16effd640 1 -- 172.21.15.82:0/2213641137 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc170105bf0 msgr2=0x7fc170199850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:11:19.539 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:19.536+0000 7fc16effd640 1 --2- 172.21.15.82:0/2213641137 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc170105bf0 0x7fc170199850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:19.539 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:19.536+0000 7fc16effd640 1 -- 172.21.15.82:0/2213641137 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1701a0e50 con 0x7fc170103200 2024-05-20T16:11:19.539 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:19.536+0000 7fc16f7fe640 1 --2- 172.21.15.82:0/2213641137 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc170108720 0x7fc1701a07e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T16:11:19.539 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:19.536+0000 7fc16effd640 1 --2- 172.21.15.82:0/2213641137 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc170103200 0x7fc170199310 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fc16000d740 tx=0x7fc16000dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:11:19.539 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:19.536+0000 7fc174819640 1 -- 172.21.15.82:0/2213641137 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc160017070 con 0x7fc170103200 2024-05-20T16:11:19.539 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:19.536+0000 7fc17581b640 1 -- 172.21.15.82:0/2213641137 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1701a1140 con 0x7fc170103200 2024-05-20T16:11:19.539 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:19.536+0000 7fc174819640 1 -- 172.21.15.82:0/2213641137 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc16000de50 con 0x7fc170103200 2024-05-20T16:11:19.540 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:19.536+0000 7fc174819640 1 -- 172.21.15.82:0/2213641137 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc160020060 con 0x7fc170103200 2024-05-20T16:11:19.540 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:19.536+0000 7fc17581b640 1 -- 172.21.15.82:0/2213641137 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1701a1690 con 0x7fc170103200 2024-05-20T16:11:19.540 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:19.537+0000 7fc17581b640 1 -- 172.21.15.82:0/2213641137 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1700fe6c0 con 0x7fc170103200 2024-05-20T16:11:19.540 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:19.539+0000 7fc174819640 1 -- 172.21.15.82:0/2213641137 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fc160004da0 con 0x7fc170103200 2024-05-20T16:11:19.540 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:19.539+0000 7fc174819640 1 --2- 172.21.15.82:0/2213641137 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fc15c0a2b90 0x7fc15c0a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:11:19.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:19.539+0000 7fc167fff640 1 --2- 172.21.15.82:0/2213641137 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fc15c0a2b90 0x7fc15c0a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:11:19.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:19.539+0000 7fc174819640 1 -- 172.21.15.82:0/2213641137 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fc1600ceac0 con 0x7fc170103200 2024-05-20T16:11:19.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:19.540+0000 7fc167fff640 1 --2- 172.21.15.82:0/2213641137 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fc15c0a2b90 0x7fc15c0a50a0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fc158031610 tx=0x7fc158000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:11:19.547 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:19.547+0000 7fc174819640 1 -- 172.21.15.82:0/2213641137 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fc16009bec0 con 0x7fc170103200 2024-05-20T16:11:19.562 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:19 smithi160 ceph-mon[36347]: Deploying daemon mds.d on smithi082 2024-05-20T16:11:19.562 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:19 smithi160 ceph-mon[36347]: mds.? [v2:172.21.15.82:6834/2044534217,v1:172.21.15.82:6835/2044534217] up:boot 2024-05-20T16:11:19.562 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:19 smithi160 ceph-mon[36347]: fsmap 3 up:standby 2024-05-20T16:11:19.562 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:19 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-20T16:11:19.562 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:19 smithi160 ceph-mon[36347]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-20T16:11:19.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:19 smithi082 ceph-mon[32238]: Deploying daemon mds.d on smithi082 2024-05-20T16:11:19.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:19 smithi082 ceph-mon[32238]: mds.? [v2:172.21.15.82:6834/2044534217,v1:172.21.15.82:6835/2044534217] up:boot 2024-05-20T16:11:19.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:19 smithi082 ceph-mon[32238]: fsmap 3 up:standby 2024-05-20T16:11:19.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:19 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-20T16:11:19.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:19 smithi082 ceph-mon[32238]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-20T16:11:19.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:19 smithi184 ceph-mon[36209]: Deploying daemon mds.d on smithi082 2024-05-20T16:11:19.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:19 smithi184 ceph-mon[36209]: mds.? [v2:172.21.15.82:6834/2044534217,v1:172.21.15.82:6835/2044534217] up:boot 2024-05-20T16:11:19.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:19 smithi184 ceph-mon[36209]: fsmap 3 up:standby 2024-05-20T16:11:19.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:19 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-20T16:11:19.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:19 smithi184 ceph-mon[36209]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-20T16:11:19.779 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:19.778+0000 7fc17581b640 1 -- 172.21.15.82:0/2213641137 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc17002d2a0 con 0x7fc170103200 2024-05-20T16:11:19.779 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:19.779+0000 7fc174819640 1 -- 172.21.15.82:0/2213641137 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3046 (secure 0 0 0) 0x7fc16009b7c0 con 0x7fc170103200 2024-05-20T16:11:19.779 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-05-20T16:11:19.780 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/2044534217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":2044534217},{"type":"v1","addr":"172.21.15.82:6835","nonce":2044534217}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6833/4015144007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":4015144007},{"type":"v1","addr":"172.21.15.184:6833","nonce":4015144007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6833/975243372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6832","nonce":975243372},{"type":"v1","addr":"172.21.15.160:6833","nonce":975243372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-05-20T16:11:19.780 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 4 2024-05-20T16:11:19.781 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:19.781+0000 7fc17581b640 1 -- 172.21.15.82:0/2213641137 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fc15c0a2b90 msgr2=0x7fc15c0a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:11:19.781 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:19.781+0000 7fc17581b640 1 --2- 172.21.15.82:0/2213641137 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fc15c0a2b90 0x7fc15c0a50a0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fc158031610 tx=0x7fc158000bc0 comp rx=0 tx=0).stop 2024-05-20T16:11:19.781 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:19.781+0000 7fc17581b640 1 -- 172.21.15.82:0/2213641137 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc170103200 msgr2=0x7fc170199310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:11:19.782 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:19.781+0000 7fc17581b640 1 --2- 172.21.15.82:0/2213641137 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc170103200 0x7fc170199310 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fc16000d740 tx=0x7fc16000dc00 comp rx=0 tx=0).stop 2024-05-20T16:11:19.782 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:19.781+0000 7fc17581b640 1 -- 172.21.15.82:0/2213641137 shutdown_connections 2024-05-20T16:11:19.782 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:19.781+0000 7fc17581b640 1 --2- 172.21.15.82:0/2213641137 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fc15c0a2b90 0x7fc15c0a50a0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:19.782 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:19.781+0000 7fc17581b640 1 --2- 172.21.15.82:0/2213641137 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc170108720 0x7fc1701a07e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:19.782 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:19.781+0000 7fc17581b640 1 --2- 172.21.15.82:0/2213641137 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc170105bf0 0x7fc170199850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:19.782 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:19.781+0000 7fc17581b640 1 --2- 172.21.15.82:0/2213641137 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc170103200 0x7fc170199310 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:19.782 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:19.781+0000 7fc17581b640 1 -- 172.21.15.82:0/2213641137 >> 172.21.15.82:0/2213641137 conn(0x7fc1700fcba0 msgr2=0x7fc170103e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:11:19.782 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:19.781+0000 7fc17581b640 1 -- 172.21.15.82:0/2213641137 shutdown_connections 2024-05-20T16:11:19.782 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:19.781+0000 7fc17581b640 1 -- 172.21.15.82:0/2213641137 wait complete. 2024-05-20T16:11:20.059 INFO:journalctl@ceph.mds.d.smithi082.stdout:May 20 16:11:19 smithi082 systemd[1]: Starting Ceph mds.d for 59d1600a-16c2-11ef-bc9a-c7b262605968... 2024-05-20T16:11:20.351 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:20 smithi082 ceph-mon[32238]: from='client.? 172.21.15.82:0/2213641137' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T16:11:20.351 INFO:journalctl@ceph.mds.d.smithi082.stdout:May 20 16:11:20 smithi082 podman[72448]: 2024-05-20 16:11:19.961820539 +0000 UTC m=+0.016885503 image pull cfcc61b2d476f611479310eb042aa05f11dd5f60dbb65e3b0d7e0e11d024cdb3 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708 2024-05-20T16:11:20.351 INFO:journalctl@ceph.mds.d.smithi082.stdout:May 20 16:11:20 smithi082 podman[72448]: 2024-05-20 16:11:20.073331945 +0000 UTC m=+0.128396894 container create 98a34b38a26205de4d53a8c54e1f2151826bffe593bbe7dac9d2edb99fb48b9f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-d, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240514, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c) 2024-05-20T16:11:20.351 INFO:journalctl@ceph.mds.d.smithi082.stdout:May 20 16:11:20 smithi082 podman[72448]: 2024-05-20 16:11:20.351621893 +0000 UTC m=+0.406686843 container init 98a34b38a26205de4d53a8c54e1f2151826bffe593bbe7dac9d2edb99fb48b9f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-d, ceph=True, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240514, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-05-20T16:11:20.562 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:20 smithi160 ceph-mon[36347]: from='client.? 172.21.15.82:0/2213641137' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T16:11:20.609 INFO:journalctl@ceph.mds.d.smithi082.stdout:May 20 16:11:20 smithi082 podman[72448]: 2024-05-20 16:11:20.354608982 +0000 UTC m=+0.409673938 container start 98a34b38a26205de4d53a8c54e1f2151826bffe593bbe7dac9d2edb99fb48b9f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-d, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240514, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-05-20T16:11:20.609 INFO:journalctl@ceph.mds.d.smithi082.stdout:May 20 16:11:20 smithi082 ceph-mds[72467]: -- 172.21.15.82:0/3035837896 <== mon.0 v2:172.21.15.82:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x562d351b3a40 con 0x562d3522b600 2024-05-20T16:11:20.609 INFO:journalctl@ceph.mds.d.smithi082.stdout:May 20 16:11:20 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-d[72463]: starting mds.d at 2024-05-20T16:11:20.609 INFO:journalctl@ceph.mds.d.smithi082.stdout:May 20 16:11:20 smithi082 bash[72448]: 98a34b38a26205de4d53a8c54e1f2151826bffe593bbe7dac9d2edb99fb48b9f 2024-05-20T16:11:20.609 INFO:journalctl@ceph.mds.d.smithi082.stdout:May 20 16:11:20 smithi082 systemd[1]: Started Ceph mds.d for 59d1600a-16c2-11ef-bc9a-c7b262605968. 2024-05-20T16:11:20.635 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:20 smithi184 ceph-mon[36209]: from='client.? 172.21.15.82:0/2213641137' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T16:11:21.794 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-20T16:11:21.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:21 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:11:21.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:21 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:11:21.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:21 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:11:21.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:21 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-20T16:11:21.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:21 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-20T16:11:21.813 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:21 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:11:21.813 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:21 smithi160 ceph-mon[36347]: Deploying daemon mds.e on smithi160 2024-05-20T16:11:21.813 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:21 smithi160 ceph-mon[36347]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-20T16:11:21.813 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:21 smithi160 ceph-mon[36347]: mds.? [v2:172.21.15.82:6836/2635635097,v1:172.21.15.82:6837/2635635097] up:boot 2024-05-20T16:11:21.813 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:21 smithi160 ceph-mon[36347]: fsmap 4 up:standby 2024-05-20T16:11:21.813 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:21 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-20T16:11:21.823 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:21 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:11:21.823 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:21 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:11:21.823 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:21 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:11:21.823 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:21 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-20T16:11:21.823 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:21 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-20T16:11:21.823 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:21 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:11:21.823 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:21 smithi082 ceph-mon[32238]: Deploying daemon mds.e on smithi160 2024-05-20T16:11:21.823 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:21 smithi082 ceph-mon[32238]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-20T16:11:21.823 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:21 smithi082 ceph-mon[32238]: mds.? [v2:172.21.15.82:6836/2635635097,v1:172.21.15.82:6837/2635635097] up:boot 2024-05-20T16:11:21.823 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:21 smithi082 ceph-mon[32238]: fsmap 4 up:standby 2024-05-20T16:11:21.824 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:21 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-20T16:11:21.886 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:21 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:11:21.886 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:21 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:11:21.886 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:21 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:11:21.886 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:21 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-20T16:11:21.886 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:21 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-20T16:11:21.886 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:21 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:11:21.886 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:21 smithi184 ceph-mon[36209]: Deploying daemon mds.e on smithi160 2024-05-20T16:11:21.886 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:21 smithi184 ceph-mon[36209]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-20T16:11:21.887 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:21 smithi184 ceph-mon[36209]: mds.? [v2:172.21.15.82:6836/2635635097,v1:172.21.15.82:6837/2635635097] up:boot 2024-05-20T16:11:21.887 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:21 smithi184 ceph-mon[36209]: fsmap 4 up:standby 2024-05-20T16:11:21.887 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:21 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-20T16:11:21.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:21.922+0000 7f11223bc640 1 -- 172.21.15.82:0/4201731595 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f111c1019b0 msgr2=0x7f111c1026b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:11:21.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:21.922+0000 7f11223bc640 1 --2- 172.21.15.82:0/4201731595 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f111c1019b0 0x7f111c1026b0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f110c009900 tx=0x7f110c02a410 comp rx=0 tx=0).stop 2024-05-20T16:11:21.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:21.922+0000 7f11223bc640 1 -- 172.21.15.82:0/4201731595 shutdown_connections 2024-05-20T16:11:21.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:21.922+0000 7f11223bc640 1 --2- 172.21.15.82:0/4201731595 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f111c1078f0 0x7f111c109d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:21.923 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:21.922+0000 7f11223bc640 1 --2- 172.21.15.82:0/4201731595 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f111c102bf0 0x7f111c103000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:21.923 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:21.922+0000 7f11223bc640 1 --2- 172.21.15.82:0/4201731595 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f111c1019b0 0x7f111c1026b0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:21.923 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:21.922+0000 7f11223bc640 1 -- 172.21.15.82:0/4201731595 >> 172.21.15.82:0/4201731595 conn(0x7f111c0fcba0 msgr2=0x7f111c0ff050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:11:21.923 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:21.922+0000 7f11223bc640 1 -- 172.21.15.82:0/4201731595 shutdown_connections 2024-05-20T16:11:21.923 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:21.922+0000 7f11223bc640 1 -- 172.21.15.82:0/4201731595 wait complete. 2024-05-20T16:11:21.923 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:21.922+0000 7f11223bc640 1 Processor -- start 2024-05-20T16:11:21.923 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:21.923+0000 7f11223bc640 1 -- start start 2024-05-20T16:11:21.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:21.923+0000 7f11223bc640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f111c1019b0 0x7f111c1981b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:11:21.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:21.923+0000 7f11223bc640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f111c102bf0 0x7f111c1986f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:11:21.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:21.923+0000 7f11223bc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f111c1078f0 0x7f111c19f680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:11:21.926 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:21.923+0000 7f11223bc640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f111c10d600 con 0x7f111c1078f0 2024-05-20T16:11:21.926 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:21.923+0000 7f11223bc640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f111c10d480 con 0x7f111c102bf0 2024-05-20T16:11:21.926 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:21.923+0000 7f11223bc640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f111c10d780 con 0x7f111c1019b0 2024-05-20T16:11:21.926 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:21.923+0000 7f1120959640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f111c1078f0 0x7f111c19f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:11:21.926 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:21.923+0000 7f1120959640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f111c1078f0 0x7f111c19f680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43276/0 (socket says 172.21.15.82:43276) 2024-05-20T16:11:21.926 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:21.923+0000 7f1120959640 1 -- 172.21.15.82:0/2803254977 learned_addr learned my addr 172.21.15.82:0/2803254977 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:11:21.926 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:21.923+0000 7f111b7fe640 1 --2- 172.21.15.82:0/2803254977 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f111c102bf0 0x7f111c1986f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:11:21.926 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:21.923+0000 7f111bfff640 1 --2- 172.21.15.82:0/2803254977 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f111c1019b0 0x7f111c1981b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:11:21.926 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:21.923+0000 7f1120959640 1 -- 172.21.15.82:0/2803254977 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f111c1019b0 msgr2=0x7f111c1981b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:11:21.926 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:21.923+0000 7f1120959640 1 --2- 172.21.15.82:0/2803254977 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f111c1019b0 0x7f111c1981b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:21.927 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:21.923+0000 7f1120959640 1 -- 172.21.15.82:0/2803254977 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f111c102bf0 msgr2=0x7f111c1986f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:11:21.927 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:21.923+0000 7f1120959640 1 --2- 172.21.15.82:0/2803254977 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f111c102bf0 0x7f111c1986f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:21.927 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:21.923+0000 7f1120959640 1 -- 172.21.15.82:0/2803254977 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f111c19fcf0 con 0x7f111c1078f0 2024-05-20T16:11:21.927 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:21.923+0000 7f111bfff640 1 --2- 172.21.15.82:0/2803254977 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f111c1019b0 0x7f111c1981b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T16:11:21.927 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:21.924+0000 7f1120959640 1 --2- 172.21.15.82:0/2803254977 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f111c1078f0 0x7f111c19f680 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f111400eb10 tx=0x7f111400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:11:21.927 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:21.924+0000 7f11197fa640 1 -- 172.21.15.82:0/2803254977 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1114004b20 con 0x7f111c1078f0 2024-05-20T16:11:21.927 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:21.924+0000 7f11197fa640 1 -- 172.21.15.82:0/2803254977 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1114004cc0 con 0x7f111c1078f0 2024-05-20T16:11:21.927 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:21.924+0000 7f11223bc640 1 -- 172.21.15.82:0/2803254977 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f111c19ffe0 con 0x7f111c1078f0 2024-05-20T16:11:21.927 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:21.924+0000 7f11197fa640 1 -- 172.21.15.82:0/2803254977 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1114010760 con 0x7f111c1078f0 2024-05-20T16:11:21.927 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:21.924+0000 7f11223bc640 1 -- 172.21.15.82:0/2803254977 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f111c1a0440 con 0x7f111c1078f0 2024-05-20T16:11:21.927 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:21.925+0000 7f11223bc640 1 -- 172.21.15.82:0/2803254977 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f111c106700 con 0x7f111c1078f0 2024-05-20T16:11:21.928 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:21.927+0000 7f11197fa640 1 -- 172.21.15.82:0/2803254977 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f1114010900 con 0x7f111c1078f0 2024-05-20T16:11:21.928 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:21.928+0000 7f11197fa640 1 --2- 172.21.15.82:0/2803254977 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f11040a2b90 0x7f11040a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:11:21.928 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:21.928+0000 7f11197fa640 1 -- 172.21.15.82:0/2803254977 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f11140cf640 con 0x7f111c1078f0 2024-05-20T16:11:21.928 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:21.928+0000 7f111bfff640 1 --2- 172.21.15.82:0/2803254977 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f11040a2b90 0x7f11040a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:11:21.929 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:21.928+0000 7f111bfff640 1 --2- 172.21.15.82:0/2803254977 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f11040a2b90 0x7f11040a50a0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f110c009230 tx=0x7f110c000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:11:21.931 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:21.931+0000 7f11197fa640 1 -- 172.21.15.82:0/2803254977 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f111409ca40 con 0x7f111c1078f0 2024-05-20T16:11:22.144 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:22.144+0000 7f11223bc640 1 -- 172.21.15.82:0/2803254977 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f111c02d4f0 con 0x7f111c1078f0 2024-05-20T16:11:22.144 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:22.144+0000 7f11197fa640 1 -- 172.21.15.82:0/2803254977 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3871 (secure 0 0 0) 0x7f111409c340 con 0x7f111c1078f0 2024-05-20T16:11:22.145 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-05-20T16:11:22.145 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/2044534217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":2044534217},{"type":"v1","addr":"172.21.15.82:6835","nonce":2044534217}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/2635635097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":2635635097},{"type":"v1","addr":"172.21.15.82:6837","nonce":2635635097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6833/4015144007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":4015144007},{"type":"v1","addr":"172.21.15.184:6833","nonce":4015144007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6833/975243372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6832","nonce":975243372},{"type":"v1","addr":"172.21.15.160:6833","nonce":975243372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-05-20T16:11:22.145 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 5 2024-05-20T16:11:22.147 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:22.146+0000 7f11223bc640 1 -- 172.21.15.82:0/2803254977 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f11040a2b90 msgr2=0x7f11040a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:11:22.147 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:22.146+0000 7f11223bc640 1 --2- 172.21.15.82:0/2803254977 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f11040a2b90 0x7f11040a50a0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f110c009230 tx=0x7f110c000bc0 comp rx=0 tx=0).stop 2024-05-20T16:11:22.147 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:22.146+0000 7f11223bc640 1 -- 172.21.15.82:0/2803254977 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f111c1078f0 msgr2=0x7f111c19f680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:11:22.147 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:22.146+0000 7f11223bc640 1 --2- 172.21.15.82:0/2803254977 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f111c1078f0 0x7f111c19f680 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f111400eb10 tx=0x7f111400c2d0 comp rx=0 tx=0).stop 2024-05-20T16:11:22.147 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:22.147+0000 7f11223bc640 1 -- 172.21.15.82:0/2803254977 shutdown_connections 2024-05-20T16:11:22.147 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:22.147+0000 7f11223bc640 1 --2- 172.21.15.82:0/2803254977 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f11040a2b90 0x7f11040a50a0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:22.147 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:22.147+0000 7f11223bc640 1 --2- 172.21.15.82:0/2803254977 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f111c1078f0 0x7f111c19f680 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:22.147 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:22.147+0000 7f11223bc640 1 --2- 172.21.15.82:0/2803254977 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f111c102bf0 0x7f111c1986f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:22.147 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:22.147+0000 7f11223bc640 1 --2- 172.21.15.82:0/2803254977 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f111c1019b0 0x7f111c1981b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:22.147 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:22.147+0000 7f11223bc640 1 -- 172.21.15.82:0/2803254977 >> 172.21.15.82:0/2803254977 conn(0x7f111c0fcba0 msgr2=0x7f111c1038d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:11:22.147 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:22.147+0000 7f11223bc640 1 -- 172.21.15.82:0/2803254977 shutdown_connections 2024-05-20T16:11:22.148 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:22.147+0000 7f11223bc640 1 -- 172.21.15.82:0/2803254977 wait complete. 2024-05-20T16:11:22.638 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:22 smithi160 ceph-mon[36347]: from='client.? 172.21.15.82:0/2803254977' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T16:11:22.844 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:22 smithi082 ceph-mon[32238]: from='client.? 172.21.15.82:0/2803254977' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T16:11:22.885 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:22 smithi184 ceph-mon[36209]: from='client.? 172.21.15.82:0/2803254977' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T16:11:23.033 INFO:journalctl@ceph.mds.e.smithi160.stdout:May 20 16:11:22 smithi160 systemd[1]: Starting Ceph mds.e for 59d1600a-16c2-11ef-bc9a-c7b262605968... 2024-05-20T16:11:23.285 INFO:journalctl@ceph.mds.e.smithi160.stdout:May 20 16:11:23 smithi160 podman[66308]: 2024-05-20 16:11:22.936205425 +0000 UTC m=+0.016597840 image pull cfcc61b2d476f611479310eb042aa05f11dd5f60dbb65e3b0d7e0e11d024cdb3 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708 2024-05-20T16:11:23.285 INFO:journalctl@ceph.mds.e.smithi160.stdout:May 20 16:11:23 smithi160 podman[66308]: 2024-05-20 16:11:23.156799447 +0000 UTC m=+0.237191846 container create cc8764d81953378b45630be4d1aef6d729048f6ad7342e09a40857ddbeeeac0a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-e, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240514, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-20T16:11:23.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:23 smithi160 ceph-mon[36347]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-20T16:11:23.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:23 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:11:23.812 INFO:journalctl@ceph.mds.e.smithi160.stdout:May 20 16:11:23 smithi160 podman[66308]: 2024-05-20 16:11:23.409299585 +0000 UTC m=+0.489691989 container init cc8764d81953378b45630be4d1aef6d729048f6ad7342e09a40857ddbeeeac0a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-e, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, org.label-schema.build-date=20240514, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-20T16:11:23.813 INFO:journalctl@ceph.mds.e.smithi160.stdout:May 20 16:11:23 smithi160 podman[66308]: 2024-05-20 16:11:23.411847542 +0000 UTC m=+0.492239948 container start cc8764d81953378b45630be4d1aef6d729048f6ad7342e09a40857ddbeeeac0a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-e, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240514, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-20T16:11:23.813 INFO:journalctl@ceph.mds.e.smithi160.stdout:May 20 16:11:23 smithi160 ceph-mds[66327]: -- 172.21.15.160:0/332233692 <== mon.2 v2:172.21.15.160:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x555c6f759a40 con 0x555c6f7d1180 2024-05-20T16:11:23.813 INFO:journalctl@ceph.mds.e.smithi160.stdout:May 20 16:11:23 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-e[66323]: starting mds.e at 2024-05-20T16:11:23.813 INFO:journalctl@ceph.mds.e.smithi160.stdout:May 20 16:11:23 smithi160 bash[66308]: cc8764d81953378b45630be4d1aef6d729048f6ad7342e09a40857ddbeeeac0a 2024-05-20T16:11:23.813 INFO:journalctl@ceph.mds.e.smithi160.stdout:May 20 16:11:23 smithi160 systemd[1]: Started Ceph mds.e for 59d1600a-16c2-11ef-bc9a-c7b262605968. 2024-05-20T16:11:23.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:23 smithi082 ceph-mon[32238]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-20T16:11:23.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:23 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:11:23.885 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:23 smithi184 ceph-mon[36209]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-20T16:11:23.885 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:23 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:11:24.159 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-20T16:11:24.286 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:24.283+0000 7f204f834640 1 -- 172.21.15.82:0/3322910960 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2048102ac0 msgr2=0x7f20481037c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:11:24.286 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:24.283+0000 7f204f834640 1 --2- 172.21.15.82:0/3322910960 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2048102ac0 0x7f20481037c0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f2044009900 tx=0x7f204402a410 comp rx=0 tx=0).stop 2024-05-20T16:11:24.286 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:24.283+0000 7f204f834640 1 -- 172.21.15.82:0/3322910960 shutdown_connections 2024-05-20T16:11:24.286 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:24.283+0000 7f204f834640 1 --2- 172.21.15.82:0/3322910960 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2048108a00 0x7f204810ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:24.286 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:24.283+0000 7f204f834640 1 --2- 172.21.15.82:0/3322910960 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2048103d00 0x7f2048104110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:24.286 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:24.283+0000 7f204f834640 1 --2- 172.21.15.82:0/3322910960 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2048102ac0 0x7f20481037c0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:24.286 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:24.284+0000 7f204f834640 1 -- 172.21.15.82:0/3322910960 >> 172.21.15.82:0/3322910960 conn(0x7f20480fdcf0 msgr2=0x7f2048100160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:11:24.286 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:24.284+0000 7f204f834640 1 -- 172.21.15.82:0/3322910960 shutdown_connections 2024-05-20T16:11:24.287 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:24.284+0000 7f204f834640 1 -- 172.21.15.82:0/3322910960 wait complete. 2024-05-20T16:11:24.287 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:24.284+0000 7f204f834640 1 Processor -- start 2024-05-20T16:11:24.287 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:24.284+0000 7f204f834640 1 -- start start 2024-05-20T16:11:24.287 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:24.284+0000 7f204f834640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2048102ac0 0x7f2048199310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:11:24.287 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:24.284+0000 7f204f834640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2048103d00 0x7f2048199850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:11:24.287 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:24.284+0000 7f204f834640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2048108a00 0x7f20481a07e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:11:24.287 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:24.284+0000 7f204f834640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f204810e710 con 0x7f2048102ac0 2024-05-20T16:11:24.287 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:24.284+0000 7f204f834640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f204810e590 con 0x7f2048108a00 2024-05-20T16:11:24.287 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:24.284+0000 7f204f834640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f204810e890 con 0x7f2048103d00 2024-05-20T16:11:24.287 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:24.285+0000 7f204d5d0640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2048102ac0 0x7f2048199310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:11:24.288 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:24.285+0000 7f204d5d0640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2048102ac0 0x7f2048199310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43286/0 (socket says 172.21.15.82:43286) 2024-05-20T16:11:24.288 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:24.285+0000 7f204d5d0640 1 -- 172.21.15.82:0/3149332056 learned_addr learned my addr 172.21.15.82:0/3149332056 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:11:24.288 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:24.285+0000 7f204ddd1640 1 --2- 172.21.15.82:0/3149332056 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2048108a00 0x7f20481a07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:11:24.288 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:24.285+0000 7f204cdcf640 1 --2- 172.21.15.82:0/3149332056 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2048103d00 0x7f2048199850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:11:24.288 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:24.285+0000 7f204d5d0640 1 -- 172.21.15.82:0/3149332056 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2048103d00 msgr2=0x7f2048199850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:11:24.288 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:24.285+0000 7f204d5d0640 1 --2- 172.21.15.82:0/3149332056 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2048103d00 0x7f2048199850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:24.288 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:24.285+0000 7f204d5d0640 1 -- 172.21.15.82:0/3149332056 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2048108a00 msgr2=0x7f20481a07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:11:24.288 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:24.285+0000 7f204d5d0640 1 --2- 172.21.15.82:0/3149332056 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2048108a00 0x7f20481a07e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:24.289 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:24.285+0000 7f204d5d0640 1 -- 172.21.15.82:0/3149332056 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20481a0e50 con 0x7f2048102ac0 2024-05-20T16:11:24.289 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:24.285+0000 7f204cdcf640 1 --2- 172.21.15.82:0/3149332056 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2048103d00 0x7f2048199850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T16:11:24.289 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:24.285+0000 7f204d5d0640 1 --2- 172.21.15.82:0/3149332056 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2048102ac0 0x7f2048199310 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f2044006010 tx=0x7f2044031b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:11:24.289 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:24.285+0000 7f203e7fc640 1 -- 172.21.15.82:0/3149332056 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f204403f030 con 0x7f2048102ac0 2024-05-20T16:11:24.289 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:24.285+0000 7f204f834640 1 -- 172.21.15.82:0/3149332056 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20481a10e0 con 0x7f2048102ac0 2024-05-20T16:11:24.290 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:24.285+0000 7f203e7fc640 1 -- 172.21.15.82:0/3149332056 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2044003ad0 con 0x7f2048102ac0 2024-05-20T16:11:24.290 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:24.285+0000 7f203e7fc640 1 -- 172.21.15.82:0/3149332056 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2044035070 con 0x7f2048102ac0 2024-05-20T16:11:24.290 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:24.285+0000 7f204f834640 1 -- 172.21.15.82:0/3149332056 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20481a15d0 con 0x7f2048102ac0 2024-05-20T16:11:24.292 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:24.286+0000 7f204f834640 1 -- 172.21.15.82:0/3149332056 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2048107810 con 0x7f2048102ac0 2024-05-20T16:11:24.292 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:24.288+0000 7f203e7fc640 1 -- 172.21.15.82:0/3149332056 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f2044042030 con 0x7f2048102ac0 2024-05-20T16:11:24.292 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:24.288+0000 7f203e7fc640 1 --2- 172.21.15.82:0/3149332056 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f20340a2b90 0x7f20340a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:11:24.293 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:24.288+0000 7f204cdcf640 1 --2- 172.21.15.82:0/3149332056 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f20340a2b90 0x7f20340a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:11:24.293 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:24.288+0000 7f203e7fc640 1 -- 172.21.15.82:0/3149332056 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f20440ef190 con 0x7f2048102ac0 2024-05-20T16:11:24.293 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:24.289+0000 7f204cdcf640 1 --2- 172.21.15.82:0/3149332056 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f20340a2b90 0x7f20340a50a0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f2038005fd0 tx=0x7f2038005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:11:24.294 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:24.293+0000 7f203e7fc640 1 -- 172.21.15.82:0/3149332056 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f20440bc590 con 0x7f2048102ac0 2024-05-20T16:11:24.506 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:24.506+0000 7f204f834640 1 -- 172.21.15.82:0/3149332056 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f204802d2a0 con 0x7f2048102ac0 2024-05-20T16:11:24.506 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:24.507+0000 7f203e7fc640 1 -- 172.21.15.82:0/3149332056 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4696 (secure 0 0 0) 0x7f20440bbe90 con 0x7f2048102ac0 2024-05-20T16:11:24.507 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-05-20T16:11:24.507 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/2044534217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":2044534217},{"type":"v1","addr":"172.21.15.82:6835","nonce":2044534217}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/2635635097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":2635635097},{"type":"v1","addr":"172.21.15.82:6837","nonce":2635635097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6833/4015144007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":4015144007},{"type":"v1","addr":"172.21.15.184:6833","nonce":4015144007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6833/975243372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6832","nonce":975243372},{"type":"v1","addr":"172.21.15.160:6833","nonce":975243372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24470,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6835/664068363","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":664068363},{"type":"v1","addr":"172.21.15.160:6835","nonce":664068363}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-05-20T16:11:24.507 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 6 2024-05-20T16:11:24.509 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:24.509+0000 7f204f834640 1 -- 172.21.15.82:0/3149332056 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f20340a2b90 msgr2=0x7f20340a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:11:24.509 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:24.509+0000 7f204f834640 1 --2- 172.21.15.82:0/3149332056 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f20340a2b90 0x7f20340a50a0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f2038005fd0 tx=0x7f2038005850 comp rx=0 tx=0).stop 2024-05-20T16:11:24.509 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:24.509+0000 7f204f834640 1 -- 172.21.15.82:0/3149332056 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2048102ac0 msgr2=0x7f2048199310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:11:24.509 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:24.509+0000 7f204f834640 1 --2- 172.21.15.82:0/3149332056 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2048102ac0 0x7f2048199310 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f2044006010 tx=0x7f2044031b60 comp rx=0 tx=0).stop 2024-05-20T16:11:24.509 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:24.509+0000 7f204f834640 1 -- 172.21.15.82:0/3149332056 shutdown_connections 2024-05-20T16:11:24.509 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:24.509+0000 7f204f834640 1 --2- 172.21.15.82:0/3149332056 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f20340a2b90 0x7f20340a50a0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:24.509 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:24.509+0000 7f204f834640 1 --2- 172.21.15.82:0/3149332056 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2048108a00 0x7f20481a07e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:24.509 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:24.509+0000 7f204f834640 1 --2- 172.21.15.82:0/3149332056 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2048103d00 0x7f2048199850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:24.509 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:24.509+0000 7f204f834640 1 --2- 172.21.15.82:0/3149332056 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2048102ac0 0x7f2048199310 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:24.509 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:24.509+0000 7f204f834640 1 -- 172.21.15.82:0/3149332056 >> 172.21.15.82:0/3149332056 conn(0x7f20480fdcf0 msgr2=0x7f20481049e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:11:24.510 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:24.509+0000 7f204f834640 1 -- 172.21.15.82:0/3149332056 shutdown_connections 2024-05-20T16:11:24.510 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:24.509+0000 7f204f834640 1 -- 172.21.15.82:0/3149332056 wait complete. 2024-05-20T16:11:24.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:24 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:11:24.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:24 smithi082 ceph-mon[32238]: mds.? [v2:172.21.15.160:6834/664068363,v1:172.21.15.160:6835/664068363] up:boot 2024-05-20T16:11:24.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:24 smithi082 ceph-mon[32238]: fsmap 5 up:standby 2024-05-20T16:11:24.860 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:24 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-20T16:11:24.860 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:24 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:11:24.860 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:24 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-20T16:11:24.860 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:24 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-20T16:11:24.860 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:24 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:11:24.860 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:24 smithi082 ceph-mon[32238]: Deploying daemon mds.f on smithi184 2024-05-20T16:11:24.860 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:24 smithi082 ceph-mon[32238]: from='client.? 172.21.15.82:0/3149332056' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T16:11:24.886 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:24 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:11:24.886 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:24 smithi184 ceph-mon[36209]: mds.? [v2:172.21.15.160:6834/664068363,v1:172.21.15.160:6835/664068363] up:boot 2024-05-20T16:11:24.886 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:24 smithi184 ceph-mon[36209]: fsmap 5 up:standby 2024-05-20T16:11:24.886 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:24 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-20T16:11:24.886 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:24 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:11:24.886 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:24 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-20T16:11:24.886 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:24 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-20T16:11:24.886 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:24 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:11:24.886 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:24 smithi184 ceph-mon[36209]: Deploying daemon mds.f on smithi184 2024-05-20T16:11:24.886 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:24 smithi184 ceph-mon[36209]: from='client.? 172.21.15.82:0/3149332056' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T16:11:25.062 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:24 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:11:25.062 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:24 smithi160 ceph-mon[36347]: mds.? [v2:172.21.15.160:6834/664068363,v1:172.21.15.160:6835/664068363] up:boot 2024-05-20T16:11:25.062 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:24 smithi160 ceph-mon[36347]: fsmap 5 up:standby 2024-05-20T16:11:25.062 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:24 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-20T16:11:25.062 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:24 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:11:25.063 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:24 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-20T16:11:25.063 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:24 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-20T16:11:25.063 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:24 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:11:25.063 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:24 smithi160 ceph-mon[36347]: Deploying daemon mds.f on smithi184 2024-05-20T16:11:25.063 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:24 smithi160 ceph-mon[36347]: from='client.? 172.21.15.82:0/3149332056' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T16:11:25.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:25 smithi082 ceph-mon[32238]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-20T16:11:25.868 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:25 smithi184 ceph-mon[36209]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-20T16:11:25.868 INFO:journalctl@ceph.mds.f.smithi184.stdout:May 20 16:11:25 smithi184 systemd[1]: Starting Ceph mds.f for 59d1600a-16c2-11ef-bc9a-c7b262605968... 2024-05-20T16:11:26.062 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:25 smithi160 ceph-mon[36347]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-20T16:11:26.137 INFO:journalctl@ceph.mds.f.smithi184.stdout:May 20 16:11:25 smithi184 podman[67075]: 2024-05-20 16:11:25.868855952 +0000 UTC m=+0.102558437 container create 70d70110f88e5e8cf7ccfbbff95d95264f2c1dac9ad285f40b3cdb7dbce6e49c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-f, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240514, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-05-20T16:11:26.137 INFO:journalctl@ceph.mds.f.smithi184.stdout:May 20 16:11:25 smithi184 podman[67075]: 2024-05-20 16:11:25.783464647 +0000 UTC m=+0.017167104 image pull cfcc61b2d476f611479310eb042aa05f11dd5f60dbb65e3b0d7e0e11d024cdb3 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708 2024-05-20T16:11:26.137 INFO:journalctl@ceph.mds.f.smithi184.stdout:May 20 16:11:26 smithi184 podman[67075]: 2024-05-20 16:11:26.113625706 +0000 UTC m=+0.347328143 container init 70d70110f88e5e8cf7ccfbbff95d95264f2c1dac9ad285f40b3cdb7dbce6e49c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-f, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240514, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-05-20T16:11:26.137 INFO:journalctl@ceph.mds.f.smithi184.stdout:May 20 16:11:26 smithi184 podman[67075]: 2024-05-20 16:11:26.116299934 +0000 UTC m=+0.350002386 container start 70d70110f88e5e8cf7ccfbbff95d95264f2c1dac9ad285f40b3cdb7dbce6e49c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-f, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240514, io.buildah.version=1.33.5, ceph=True) 2024-05-20T16:11:26.506 INFO:journalctl@ceph.mds.f.smithi184.stdout:May 20 16:11:26 smithi184 ceph-mds[67094]: -- 172.21.15.184:0/963991997 <== mon.1 v2:172.21.15.184:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55cecf085a40 con 0x55cecf0fd600 2024-05-20T16:11:26.506 INFO:journalctl@ceph.mds.f.smithi184.stdout:May 20 16:11:26 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-f[67090]: starting mds.f at 2024-05-20T16:11:26.506 INFO:journalctl@ceph.mds.f.smithi184.stdout:May 20 16:11:26 smithi184 bash[67075]: 70d70110f88e5e8cf7ccfbbff95d95264f2c1dac9ad285f40b3cdb7dbce6e49c 2024-05-20T16:11:26.506 INFO:journalctl@ceph.mds.f.smithi184.stdout:May 20 16:11:26 smithi184 systemd[1]: Started Ceph mds.f for 59d1600a-16c2-11ef-bc9a-c7b262605968. 2024-05-20T16:11:26.521 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-20T16:11:26.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:26.646+0000 7fafd1596640 1 -- 172.21.15.82:0/848641387 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fafcc1089c0 msgr2=0x7fafcc10ade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:11:26.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:26.646+0000 7fafd1596640 1 --2- 172.21.15.82:0/848641387 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fafcc1089c0 0x7fafcc10ade0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fafc4009ca0 tx=0x7fafc402a8c0 comp rx=0 tx=0).stop 2024-05-20T16:11:26.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:26.646+0000 7fafd1596640 1 -- 172.21.15.82:0/848641387 shutdown_connections 2024-05-20T16:11:26.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:26.646+0000 7fafd1596640 1 --2- 172.21.15.82:0/848641387 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fafcc1089c0 0x7fafcc10ade0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:26.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:26.646+0000 7fafd1596640 1 --2- 172.21.15.82:0/848641387 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fafcc103cc0 0x7fafcc1040d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:26.650 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:26.646+0000 7fafd1596640 1 --2- 172.21.15.82:0/848641387 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fafcc102a80 0x7fafcc103780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:26.650 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:26.646+0000 7fafd1596640 1 -- 172.21.15.82:0/848641387 >> 172.21.15.82:0/848641387 conn(0x7fafcc0fdc70 msgr2=0x7fafcc100120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:11:26.650 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:26.646+0000 7fafd1596640 1 -- 172.21.15.82:0/848641387 shutdown_connections 2024-05-20T16:11:26.650 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:26.646+0000 7fafd1596640 1 -- 172.21.15.82:0/848641387 wait complete. 2024-05-20T16:11:26.650 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:26.647+0000 7fafd1596640 1 Processor -- start 2024-05-20T16:11:26.650 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:26.647+0000 7fafd1596640 1 -- start start 2024-05-20T16:11:26.650 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:26.647+0000 7fafd1596640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fafcc102a80 0x7fafcc199220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:11:26.650 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:26.647+0000 7fafd1596640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fafcc103cc0 0x7fafcc199760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:11:26.650 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:26.647+0000 7fafd1596640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fafcc1089c0 0x7fafcc1a06f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:11:26.650 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:26.647+0000 7fafd1596640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fafcc10e700 con 0x7fafcc1089c0 2024-05-20T16:11:26.650 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:26.647+0000 7fafd1596640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fafcc10e580 con 0x7fafcc103cc0 2024-05-20T16:11:26.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:26.647+0000 7fafd1596640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fafcc10e880 con 0x7fafcc102a80 2024-05-20T16:11:26.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:26.647+0000 7fafcb7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fafcc1089c0 0x7fafcc1a06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:11:26.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:26.647+0000 7fafcb7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fafcc1089c0 0x7fafcc1a06f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43306/0 (socket says 172.21.15.82:43306) 2024-05-20T16:11:26.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:26.647+0000 7fafcb7fe640 1 -- 172.21.15.82:0/956020994 learned_addr learned my addr 172.21.15.82:0/956020994 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:11:26.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:26.647+0000 7fafcaffd640 1 --2- 172.21.15.82:0/956020994 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fafcc102a80 0x7fafcc199220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:11:26.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:26.647+0000 7fafc3fff640 1 --2- 172.21.15.82:0/956020994 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fafcc103cc0 0x7fafcc199760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:11:26.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:26.648+0000 7fafcb7fe640 1 -- 172.21.15.82:0/956020994 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fafcc102a80 msgr2=0x7fafcc199220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:11:26.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:26.648+0000 7fafcb7fe640 1 --2- 172.21.15.82:0/956020994 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fafcc102a80 0x7fafcc199220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:26.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:26.648+0000 7fafcb7fe640 1 -- 172.21.15.82:0/956020994 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fafcc103cc0 msgr2=0x7fafcc199760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:11:26.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:26.648+0000 7fafcb7fe640 1 --2- 172.21.15.82:0/956020994 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fafcc103cc0 0x7fafcc199760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:26.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:26.648+0000 7fafcb7fe640 1 -- 172.21.15.82:0/956020994 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafcc1a0cb0 con 0x7fafcc1089c0 2024-05-20T16:11:26.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:26.648+0000 7fafc3fff640 1 --2- 172.21.15.82:0/956020994 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fafcc103cc0 0x7fafcc199760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T16:11:26.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:26.648+0000 7fafcb7fe640 1 --2- 172.21.15.82:0/956020994 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fafcc1089c0 0x7fafcc1a06f0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fafc4005cc0 tx=0x7fafc4004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:11:26.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:26.648+0000 7fafc8ff9640 1 -- 172.21.15.82:0/956020994 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafc4003b70 con 0x7fafcc1089c0 2024-05-20T16:11:26.652 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:26.648+0000 7fafc8ff9640 1 -- 172.21.15.82:0/956020994 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fafc4003d10 con 0x7fafcc1089c0 2024-05-20T16:11:26.652 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:26.648+0000 7fafc8ff9640 1 -- 172.21.15.82:0/956020994 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafc402fbd0 con 0x7fafcc1089c0 2024-05-20T16:11:26.652 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:26.648+0000 7fafd1596640 1 -- 172.21.15.82:0/956020994 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fafcc1a0f40 con 0x7fafcc1089c0 2024-05-20T16:11:26.652 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:26.648+0000 7fafd1596640 1 -- 172.21.15.82:0/956020994 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fafcc1a1350 con 0x7fafcc1089c0 2024-05-20T16:11:26.652 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:26.649+0000 7fafd1596640 1 -- 172.21.15.82:0/956020994 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fafcc069f50 con 0x7fafcc1089c0 2024-05-20T16:11:26.653 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:26.651+0000 7fafc8ff9640 1 -- 172.21.15.82:0/956020994 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fafc402fd70 con 0x7fafcc1089c0 2024-05-20T16:11:26.653 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:26.652+0000 7fafc8ff9640 1 --2- 172.21.15.82:0/956020994 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fafb80a2b90 0x7fafb80a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:11:26.653 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:26.652+0000 7fafcaffd640 1 --2- 172.21.15.82:0/956020994 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fafb80a2b90 0x7fafb80a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:11:26.653 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:26.652+0000 7fafc8ff9640 1 -- 172.21.15.82:0/956020994 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fafc40f0070 con 0x7fafcc1089c0 2024-05-20T16:11:26.653 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:26.652+0000 7fafcaffd640 1 --2- 172.21.15.82:0/956020994 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fafb80a2b90 0x7fafb80a50a0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fafcc103680 tx=0x7fafbc005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:11:26.655 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:26.655+0000 7fafc8ff9640 1 -- 172.21.15.82:0/956020994 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fafc40bd410 con 0x7fafcc1089c0 2024-05-20T16:11:26.874 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:26.875+0000 7fafd1596640 1 -- 172.21.15.82:0/956020994 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fafcc1077d0 con 0x7fafcc1089c0 2024-05-20T16:11:26.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:26.875+0000 7fafc8ff9640 1 -- 172.21.15.82:0/956020994 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5524 (secure 0 0 0) 0x7fafc40bcd10 con 0x7fafcc1089c0 2024-05-20T16:11:26.875 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-05-20T16:11:26.875 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/2044534217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":2044534217},{"type":"v1","addr":"172.21.15.82:6835","nonce":2044534217}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/2635635097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":2635635097},{"type":"v1","addr":"172.21.15.82:6837","nonce":2635635097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6833/4015144007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":4015144007},{"type":"v1","addr":"172.21.15.184:6833","nonce":4015144007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6833/975243372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6832","nonce":975243372},{"type":"v1","addr":"172.21.15.160:6833","nonce":975243372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24470,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6835/664068363","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":664068363},{"type":"v1","addr":"172.21.15.160:6835","nonce":664068363}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24478,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/4242156090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":4242156090},{"type":"v1","addr":"172.21.15.184:6835","nonce":4242156090}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[]} 2024-05-20T16:11:26.876 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 7 2024-05-20T16:11:26.877 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:26.877+0000 7fafd1596640 1 -- 172.21.15.82:0/956020994 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fafb80a2b90 msgr2=0x7fafb80a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:11:26.877 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:26.877+0000 7fafd1596640 1 --2- 172.21.15.82:0/956020994 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fafb80a2b90 0x7fafb80a50a0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fafcc103680 tx=0x7fafbc005f90 comp rx=0 tx=0).stop 2024-05-20T16:11:26.878 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:26.877+0000 7fafd1596640 1 -- 172.21.15.82:0/956020994 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fafcc1089c0 msgr2=0x7fafcc1a06f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:11:26.878 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:26.877+0000 7fafd1596640 1 --2- 172.21.15.82:0/956020994 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fafcc1089c0 0x7fafcc1a06f0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fafc4005cc0 tx=0x7fafc4004ab0 comp rx=0 tx=0).stop 2024-05-20T16:11:26.878 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:26.877+0000 7fafd1596640 1 -- 172.21.15.82:0/956020994 shutdown_connections 2024-05-20T16:11:26.878 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:26.877+0000 7fafd1596640 1 --2- 172.21.15.82:0/956020994 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fafb80a2b90 0x7fafb80a50a0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:26.878 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:26.877+0000 7fafd1596640 1 --2- 172.21.15.82:0/956020994 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fafcc1089c0 0x7fafcc1a06f0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:26.878 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:26.877+0000 7fafd1596640 1 --2- 172.21.15.82:0/956020994 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fafcc103cc0 0x7fafcc199760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:26.878 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:26.877+0000 7fafd1596640 1 --2- 172.21.15.82:0/956020994 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fafcc102a80 0x7fafcc199220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:26.878 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:26.877+0000 7fafd1596640 1 -- 172.21.15.82:0/956020994 >> 172.21.15.82:0/956020994 conn(0x7fafcc0fdc70 msgr2=0x7fafcc1049a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:11:26.878 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:26.877+0000 7fafd1596640 1 -- 172.21.15.82:0/956020994 shutdown_connections 2024-05-20T16:11:26.878 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:26.877+0000 7fafd1596640 1 -- 172.21.15.82:0/956020994 wait complete. 2024-05-20T16:11:27.488 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:27 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:11:27.489 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:27 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:11:27.489 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:27 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:11:27.489 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:27 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-20T16:11:27.489 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:27 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-20T16:11:27.489 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:27 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:11:27.489 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:27 smithi184 ceph-mon[36209]: mds.? [v2:172.21.15.184:6834/4242156090,v1:172.21.15.184:6835/4242156090] up:boot 2024-05-20T16:11:27.489 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:27 smithi184 ceph-mon[36209]: fsmap 6 up:standby 2024-05-20T16:11:27.489 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:27 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-20T16:11:27.489 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:27 smithi184 ceph-mon[36209]: from='client.? 172.21.15.82:0/956020994' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T16:11:27.562 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:27 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:11:27.562 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:27 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:11:27.562 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:27 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:11:27.562 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:27 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-20T16:11:27.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:27 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-20T16:11:27.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:27 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:11:27.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:27 smithi160 ceph-mon[36347]: mds.? [v2:172.21.15.184:6834/4242156090,v1:172.21.15.184:6835/4242156090] up:boot 2024-05-20T16:11:27.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:27 smithi160 ceph-mon[36347]: fsmap 6 up:standby 2024-05-20T16:11:27.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:27 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-20T16:11:27.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:27 smithi160 ceph-mon[36347]: from='client.? 172.21.15.82:0/956020994' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T16:11:27.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:27 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:11:27.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:27 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:11:27.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:27 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:11:27.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:27 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-20T16:11:27.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:27 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-20T16:11:27.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:27 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:11:27.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:27 smithi082 ceph-mon[32238]: mds.? [v2:172.21.15.184:6834/4242156090,v1:172.21.15.184:6835/4242156090] up:boot 2024-05-20T16:11:27.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:27 smithi082 ceph-mon[32238]: fsmap 6 up:standby 2024-05-20T16:11:27.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:27 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-20T16:11:27.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:27 smithi082 ceph-mon[32238]: from='client.? 172.21.15.82:0/956020994' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T16:11:28.562 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:28 smithi160 ceph-mon[36347]: Deploying daemon mds.c on smithi184 2024-05-20T16:11:28.562 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:28 smithi160 ceph-mon[36347]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-20T16:11:28.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:28 smithi082 ceph-mon[32238]: Deploying daemon mds.c on smithi184 2024-05-20T16:11:28.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:28 smithi082 ceph-mon[32238]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-20T16:11:28.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:28 smithi184 ceph-mon[36209]: Deploying daemon mds.c on smithi184 2024-05-20T16:11:28.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:28 smithi184 ceph-mon[36209]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-20T16:11:28.636 INFO:journalctl@ceph.mds.c.smithi184.stdout:May 20 16:11:28 smithi184 systemd[1]: Starting Ceph mds.c for 59d1600a-16c2-11ef-bc9a-c7b262605968... 2024-05-20T16:11:28.889 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-20T16:11:29.017 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:29.014+0000 7f947dc06640 1 -- 172.21.15.82:0/2830874394 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9478103cd0 msgr2=0x7f94781040e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:11:29.017 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:29.014+0000 7f947dc06640 1 --2- 172.21.15.82:0/2830874394 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9478103cd0 0x7f94781040e0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f9460009900 tx=0x7f946002a410 comp rx=0 tx=0).stop 2024-05-20T16:11:29.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:29.014+0000 7f947dc06640 1 -- 172.21.15.82:0/2830874394 shutdown_connections 2024-05-20T16:11:29.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:29.014+0000 7f947dc06640 1 --2- 172.21.15.82:0/2830874394 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9478103cd0 0x7f94781040e0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:29.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:29.014+0000 7f947dc06640 1 --2- 172.21.15.82:0/2830874394 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f94781089d0 0x7f947810adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:29.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:29.014+0000 7f947dc06640 1 --2- 172.21.15.82:0/2830874394 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9478102a90 0x7f9478103790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:29.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:29.014+0000 7f947dc06640 1 -- 172.21.15.82:0/2830874394 >> 172.21.15.82:0/2830874394 conn(0x7f94780fdc80 msgr2=0x7f9478100130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:11:29.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:29.014+0000 7f947dc06640 1 -- 172.21.15.82:0/2830874394 shutdown_connections 2024-05-20T16:11:29.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:29.014+0000 7f947dc06640 1 -- 172.21.15.82:0/2830874394 wait complete. 2024-05-20T16:11:29.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:29.015+0000 7f947dc06640 1 Processor -- start 2024-05-20T16:11:29.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:29.015+0000 7f947dc06640 1 -- start start 2024-05-20T16:11:29.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:29.015+0000 7f947dc06640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9478102a90 0x7f9478199280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:11:29.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:29.015+0000 7f947dc06640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9478103cd0 0x7f94781997c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:11:29.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:29.015+0000 7f947dc06640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f94781089d0 0x7f94781a0750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:11:29.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:29.015+0000 7f947dc06640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f947810e710 con 0x7f9478102a90 2024-05-20T16:11:29.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:29.015+0000 7f947dc06640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f947810e590 con 0x7f94781089d0 2024-05-20T16:11:29.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:29.015+0000 7f947dc06640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f947810e890 con 0x7f9478103cd0 2024-05-20T16:11:29.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:29.015+0000 7f94777fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9478102a90 0x7f9478199280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:11:29.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:29.015+0000 7f94777fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9478102a90 0x7f9478199280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43322/0 (socket says 172.21.15.82:43322) 2024-05-20T16:11:29.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:29.015+0000 7f94777fe640 1 -- 172.21.15.82:0/4281148089 learned_addr learned my addr 172.21.15.82:0/4281148089 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:11:29.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:29.015+0000 7f9477fff640 1 --2- 172.21.15.82:0/4281148089 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f94781089d0 0x7f94781a0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:11:29.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:29.015+0000 7f9476ffd640 1 --2- 172.21.15.82:0/4281148089 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9478103cd0 0x7f94781997c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:11:29.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:29.016+0000 7f94777fe640 1 -- 172.21.15.82:0/4281148089 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9478103cd0 msgr2=0x7f94781997c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:11:29.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:29.016+0000 7f94777fe640 1 --2- 172.21.15.82:0/4281148089 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9478103cd0 0x7f94781997c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:29.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:29.016+0000 7f94777fe640 1 -- 172.21.15.82:0/4281148089 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f94781089d0 msgr2=0x7f94781a0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:11:29.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:29.016+0000 7f94777fe640 1 --2- 172.21.15.82:0/4281148089 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f94781089d0 0x7f94781a0750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:29.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:29.016+0000 7f94777fe640 1 -- 172.21.15.82:0/4281148089 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94781a0dc0 con 0x7f9478102a90 2024-05-20T16:11:29.020 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:29.016+0000 7f94777fe640 1 --2- 172.21.15.82:0/4281148089 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9478102a90 0x7f9478199280 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f946800e9f0 tx=0x7f946800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:11:29.020 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:29.016+0000 7f9474ff9640 1 -- 172.21.15.82:0/4281148089 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f946800ccc0 con 0x7f9478102a90 2024-05-20T16:11:29.020 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:29.016+0000 7f9474ff9640 1 -- 172.21.15.82:0/4281148089 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f94680035f0 con 0x7f9478102a90 2024-05-20T16:11:29.020 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:29.016+0000 7f947dc06640 1 -- 172.21.15.82:0/4281148089 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94781a10b0 con 0x7f9478102a90 2024-05-20T16:11:29.020 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:29.016+0000 7f9474ff9640 1 -- 172.21.15.82:0/4281148089 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f946800ce60 con 0x7f9478102a90 2024-05-20T16:11:29.020 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:29.016+0000 7f947dc06640 1 -- 172.21.15.82:0/4281148089 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94781a1600 con 0x7f9478102a90 2024-05-20T16:11:29.020 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:29.017+0000 7f947dc06640 1 -- 172.21.15.82:0/4281148089 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94781077e0 con 0x7f9478102a90 2024-05-20T16:11:29.021 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:29.018+0000 7f9474ff9640 1 -- 172.21.15.82:0/4281148089 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f94680037a0 con 0x7f9478102a90 2024-05-20T16:11:29.021 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:29.019+0000 7f9474ff9640 1 --2- 172.21.15.82:0/4281148089 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f94640a2b90 0x7f94640a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:11:29.021 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:29.019+0000 7f9476ffd640 1 --2- 172.21.15.82:0/4281148089 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f94640a2b90 0x7f94640a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:11:29.021 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:29.019+0000 7f9474ff9640 1 -- 172.21.15.82:0/4281148089 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f94680cf020 con 0x7f9478102a90 2024-05-20T16:11:29.021 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:29.019+0000 7f9476ffd640 1 --2- 172.21.15.82:0/4281148089 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f94640a2b90 0x7f94640a50a0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f9460000c00 tx=0x7f9460031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:11:29.022 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:29.022+0000 7f9474ff9640 1 -- 172.21.15.82:0/4281148089 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f946809c420 con 0x7f9478102a90 2024-05-20T16:11:29.071 INFO:journalctl@ceph.mds.c.smithi184.stdout:May 20 16:11:28 smithi184 podman[67387]: 2024-05-20 16:11:28.610613555 +0000 UTC m=+0.017533758 image pull cfcc61b2d476f611479310eb042aa05f11dd5f60dbb65e3b0d7e0e11d024cdb3 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708 2024-05-20T16:11:29.071 INFO:journalctl@ceph.mds.c.smithi184.stdout:May 20 16:11:28 smithi184 podman[67387]: 2024-05-20 16:11:28.727277423 +0000 UTC m=+0.134197636 container create aa07be6b721540c695bab239d736767cd47bec50b4734f652b22c78a9c9c9d29 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-c, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240514, maintainer=Guillaume Abrioux ) 2024-05-20T16:11:29.239 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:29.239+0000 7f947dc06640 1 -- 172.21.15.82:0/4281148089 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f947802d2a0 con 0x7f9478102a90 2024-05-20T16:11:29.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:29.240+0000 7f9474ff9640 1 -- 172.21.15.82:0/4281148089 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5524 (secure 0 0 0) 0x7f946809bd20 con 0x7f9478102a90 2024-05-20T16:11:29.240 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-05-20T16:11:29.240 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/2044534217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":2044534217},{"type":"v1","addr":"172.21.15.82:6835","nonce":2044534217}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/2635635097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":2635635097},{"type":"v1","addr":"172.21.15.82:6837","nonce":2635635097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6833/4015144007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":4015144007},{"type":"v1","addr":"172.21.15.184:6833","nonce":4015144007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6833/975243372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6832","nonce":975243372},{"type":"v1","addr":"172.21.15.160:6833","nonce":975243372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24470,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6835/664068363","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":664068363},{"type":"v1","addr":"172.21.15.160:6835","nonce":664068363}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24478,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/4242156090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":4242156090},{"type":"v1","addr":"172.21.15.184:6835","nonce":4242156090}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[]} 2024-05-20T16:11:29.240 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 7 2024-05-20T16:11:29.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:29.242+0000 7f947dc06640 1 -- 172.21.15.82:0/4281148089 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f94640a2b90 msgr2=0x7f94640a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:11:29.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:29.242+0000 7f947dc06640 1 --2- 172.21.15.82:0/4281148089 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f94640a2b90 0x7f94640a50a0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f9460000c00 tx=0x7f9460031400 comp rx=0 tx=0).stop 2024-05-20T16:11:29.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:29.242+0000 7f947dc06640 1 -- 172.21.15.82:0/4281148089 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9478102a90 msgr2=0x7f9478199280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:11:29.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:29.242+0000 7f947dc06640 1 --2- 172.21.15.82:0/4281148089 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9478102a90 0x7f9478199280 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f946800e9f0 tx=0x7f946800eeb0 comp rx=0 tx=0).stop 2024-05-20T16:11:29.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:29.242+0000 7f947dc06640 1 -- 172.21.15.82:0/4281148089 shutdown_connections 2024-05-20T16:11:29.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:29.242+0000 7f947dc06640 1 --2- 172.21.15.82:0/4281148089 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f94640a2b90 0x7f94640a50a0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:29.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:29.242+0000 7f947dc06640 1 --2- 172.21.15.82:0/4281148089 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f94781089d0 0x7f94781a0750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:29.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:29.242+0000 7f947dc06640 1 --2- 172.21.15.82:0/4281148089 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9478103cd0 0x7f94781997c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:29.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:29.242+0000 7f947dc06640 1 --2- 172.21.15.82:0/4281148089 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9478102a90 0x7f9478199280 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:29.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:29.242+0000 7f947dc06640 1 -- 172.21.15.82:0/4281148089 >> 172.21.15.82:0/4281148089 conn(0x7f94780fdc80 msgr2=0x7f94781049b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:11:29.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:29.242+0000 7f947dc06640 1 -- 172.21.15.82:0/4281148089 shutdown_connections 2024-05-20T16:11:29.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:29.242+0000 7f947dc06640 1 -- 172.21.15.82:0/4281148089 wait complete. 2024-05-20T16:11:29.386 INFO:journalctl@ceph.mds.c.smithi184.stdout:May 20 16:11:29 smithi184 podman[67387]: 2024-05-20 16:11:29.072491896 +0000 UTC m=+0.479412099 container init aa07be6b721540c695bab239d736767cd47bec50b4734f652b22c78a9c9c9d29 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-c, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240514, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-05-20T16:11:29.386 INFO:journalctl@ceph.mds.c.smithi184.stdout:May 20 16:11:29 smithi184 podman[67387]: 2024-05-20 16:11:29.075841988 +0000 UTC m=+0.482762192 container start aa07be6b721540c695bab239d736767cd47bec50b4734f652b22c78a9c9c9d29 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-c, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240514, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-05-20T16:11:29.386 INFO:journalctl@ceph.mds.c.smithi184.stdout:May 20 16:11:29 smithi184 ceph-mds[67406]: -- 172.21.15.184:0/4146807564 <== mon.1 v2:172.21.15.184:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56061d0fba40 con 0x56061d173a80 2024-05-20T16:11:29.386 INFO:journalctl@ceph.mds.c.smithi184.stdout:May 20 16:11:29 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-c[67402]: starting mds.c at 2024-05-20T16:11:29.386 INFO:journalctl@ceph.mds.c.smithi184.stdout:May 20 16:11:29 smithi184 bash[67387]: aa07be6b721540c695bab239d736767cd47bec50b4734f652b22c78a9c9c9d29 2024-05-20T16:11:29.386 INFO:journalctl@ceph.mds.c.smithi184.stdout:May 20 16:11:29 smithi184 systemd[1]: Started Ceph mds.c for 59d1600a-16c2-11ef-bc9a-c7b262605968. 2024-05-20T16:11:30.562 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:30 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:11:30.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:30 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:11:30.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:30 smithi160 ceph-mon[36347]: from='client.? 172.21.15.82:0/4281148089' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T16:11:30.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:30 smithi160 ceph-mon[36347]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-20T16:11:30.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:30 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:11:30.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:30 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-20T16:11:30.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:30 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-20T16:11:30.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:30 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:11:30.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:30 smithi160 ceph-mon[36347]: Deploying daemon mds.h on smithi160 2024-05-20T16:11:30.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:30 smithi160 ceph-mon[36347]: mds.? [v2:172.21.15.184:6836/403262242,v1:172.21.15.184:6837/403262242] up:boot 2024-05-20T16:11:30.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:30 smithi160 ceph-mon[36347]: fsmap 7 up:standby 2024-05-20T16:11:30.564 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:30 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-20T16:11:30.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:30 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:11:30.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:30 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:11:30.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:30 smithi082 ceph-mon[32238]: from='client.? 172.21.15.82:0/4281148089' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T16:11:30.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:30 smithi082 ceph-mon[32238]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-20T16:11:30.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:30 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:11:30.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:30 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-20T16:11:30.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:30 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-20T16:11:30.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:30 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:11:30.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:30 smithi082 ceph-mon[32238]: Deploying daemon mds.h on smithi160 2024-05-20T16:11:30.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:30 smithi082 ceph-mon[32238]: mds.? [v2:172.21.15.184:6836/403262242,v1:172.21.15.184:6837/403262242] up:boot 2024-05-20T16:11:30.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:30 smithi082 ceph-mon[32238]: fsmap 7 up:standby 2024-05-20T16:11:30.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:30 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-20T16:11:30.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:30 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:11:30.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:30 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:11:30.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:30 smithi184 ceph-mon[36209]: from='client.? 172.21.15.82:0/4281148089' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T16:11:30.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:30 smithi184 ceph-mon[36209]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-20T16:11:30.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:30 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:11:30.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:30 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-20T16:11:30.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:30 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-20T16:11:30.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:30 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:11:30.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:30 smithi184 ceph-mon[36209]: Deploying daemon mds.h on smithi160 2024-05-20T16:11:30.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:30 smithi184 ceph-mon[36209]: mds.? [v2:172.21.15.184:6836/403262242,v1:172.21.15.184:6837/403262242] up:boot 2024-05-20T16:11:30.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:30 smithi184 ceph-mon[36209]: fsmap 7 up:standby 2024-05-20T16:11:30.637 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:30 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-20T16:11:31.254 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-20T16:11:31.384 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:31.384+0000 7fa4614a8640 1 -- 172.21.15.82:0/768142456 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa45c1078b0 msgr2=0x7fa45c109cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:11:31.384 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:31.384+0000 7fa4614a8640 1 --2- 172.21.15.82:0/768142456 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa45c1078b0 0x7fa45c109cd0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fa454009ca0 tx=0x7fa45402a8c0 comp rx=0 tx=0).stop 2024-05-20T16:11:31.384 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:31.384+0000 7fa4614a8640 1 -- 172.21.15.82:0/768142456 shutdown_connections 2024-05-20T16:11:31.384 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:31.384+0000 7fa4614a8640 1 --2- 172.21.15.82:0/768142456 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa45c1078b0 0x7fa45c109cd0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:31.384 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:31.384+0000 7fa4614a8640 1 --2- 172.21.15.82:0/768142456 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa45c102bb0 0x7fa45c102fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:31.384 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:31.384+0000 7fa4614a8640 1 --2- 172.21.15.82:0/768142456 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa45c101970 0x7fa45c102670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:31.384 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:31.384+0000 7fa4614a8640 1 -- 172.21.15.82:0/768142456 >> 172.21.15.82:0/768142456 conn(0x7fa45c0fcb80 msgr2=0x7fa45c0ff010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:11:31.384 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:31.384+0000 7fa4614a8640 1 -- 172.21.15.82:0/768142456 shutdown_connections 2024-05-20T16:11:31.385 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:31.384+0000 7fa4614a8640 1 -- 172.21.15.82:0/768142456 wait complete. 2024-05-20T16:11:31.385 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:31.384+0000 7fa4614a8640 1 Processor -- start 2024-05-20T16:11:31.387 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:31.385+0000 7fa4614a8640 1 -- start start 2024-05-20T16:11:31.387 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:31.385+0000 7fa4614a8640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa45c101970 0x7fa45c198110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:11:31.387 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:31.385+0000 7fa4614a8640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa45c102bb0 0x7fa45c198650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:11:31.387 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:31.385+0000 7fa4614a8640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa45c1078b0 0x7fa45c19f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:11:31.387 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:31.385+0000 7fa4614a8640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fa45c10d5d0 con 0x7fa45c101970 2024-05-20T16:11:31.387 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:31.385+0000 7fa4614a8640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fa45c10d450 con 0x7fa45c102bb0 2024-05-20T16:11:31.387 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:31.385+0000 7fa4614a8640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa45c10d750 con 0x7fa45c1078b0 2024-05-20T16:11:31.387 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:31.385+0000 7fa45affd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa45c101970 0x7fa45c198110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:11:31.387 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:31.385+0000 7fa45affd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa45c101970 0x7fa45c198110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:49356/0 (socket says 172.21.15.82:49356) 2024-05-20T16:11:31.387 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:31.385+0000 7fa45affd640 1 -- 172.21.15.82:0/433703251 learned_addr learned my addr 172.21.15.82:0/433703251 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:11:31.387 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:31.385+0000 7fa45a7fc640 1 --2- 172.21.15.82:0/433703251 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa45c102bb0 0x7fa45c198650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:11:31.387 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:31.385+0000 7fa45b7fe640 1 --2- 172.21.15.82:0/433703251 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa45c1078b0 0x7fa45c19f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:11:31.388 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:31.385+0000 7fa45affd640 1 -- 172.21.15.82:0/433703251 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa45c1078b0 msgr2=0x7fa45c19f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:11:31.388 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:31.385+0000 7fa45affd640 1 --2- 172.21.15.82:0/433703251 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa45c1078b0 0x7fa45c19f5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:31.388 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:31.385+0000 7fa45affd640 1 -- 172.21.15.82:0/433703251 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa45c102bb0 msgr2=0x7fa45c198650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:11:31.388 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:31.385+0000 7fa45affd640 1 --2- 172.21.15.82:0/433703251 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa45c102bb0 0x7fa45c198650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:31.388 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:31.385+0000 7fa45affd640 1 -- 172.21.15.82:0/433703251 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa45c19fba0 con 0x7fa45c101970 2024-05-20T16:11:31.388 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:31.385+0000 7fa45affd640 1 --2- 172.21.15.82:0/433703251 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa45c101970 0x7fa45c198110 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fa44c00e9f0 tx=0x7fa44c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:11:31.388 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:31.385+0000 7fa443fff640 1 -- 172.21.15.82:0/433703251 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa44c00ccc0 con 0x7fa45c101970 2024-05-20T16:11:31.388 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:31.386+0000 7fa4614a8640 1 -- 172.21.15.82:0/433703251 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa45c19fe90 con 0x7fa45c101970 2024-05-20T16:11:31.388 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:31.386+0000 7fa443fff640 1 -- 172.21.15.82:0/433703251 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa44c0035f0 con 0x7fa45c101970 2024-05-20T16:11:31.388 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:31.386+0000 7fa443fff640 1 -- 172.21.15.82:0/433703251 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa44c00ce60 con 0x7fa45c101970 2024-05-20T16:11:31.388 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:31.386+0000 7fa4614a8640 1 -- 172.21.15.82:0/433703251 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa45c1a0350 con 0x7fa45c101970 2024-05-20T16:11:31.388 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:31.386+0000 7fa4614a8640 1 -- 172.21.15.82:0/433703251 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa45c1066c0 con 0x7fa45c101970 2024-05-20T16:11:31.389 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:31.388+0000 7fa443fff640 1 -- 172.21.15.82:0/433703251 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fa44c0037a0 con 0x7fa45c101970 2024-05-20T16:11:31.389 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:31.388+0000 7fa443fff640 1 --2- 172.21.15.82:0/433703251 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fa4440a2b90 0x7fa4440a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:11:31.389 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:31.389+0000 7fa45a7fc640 1 --2- 172.21.15.82:0/433703251 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fa4440a2b90 0x7fa4440a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:11:31.389 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:31.389+0000 7fa443fff640 1 -- 172.21.15.82:0/433703251 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fa44c0cf060 con 0x7fa45c101970 2024-05-20T16:11:31.389 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:31.389+0000 7fa45a7fc640 1 --2- 172.21.15.82:0/433703251 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fa4440a2b90 0x7fa4440a50a0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fa45c1991b0 tx=0x7fa45000a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:11:31.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:31.392+0000 7fa443fff640 1 -- 172.21.15.82:0/433703251 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fa44c09c460 con 0x7fa45c101970 2024-05-20T16:11:31.562 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:31 smithi160 ceph-mon[36347]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-20T16:11:31.563 INFO:journalctl@ceph.mds.h.smithi160.stdout:May 20 16:11:31 smithi160 systemd[1]: Starting Ceph mds.h for 59d1600a-16c2-11ef-bc9a-c7b262605968... 2024-05-20T16:11:31.608 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:31.608+0000 7fa4614a8640 1 -- 172.21.15.82:0/433703251 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa45c02d2a0 con 0x7fa45c101970 2024-05-20T16:11:31.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:31 smithi082 ceph-mon[32238]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-20T16:11:31.610 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:31.609+0000 7fa443fff640 1 -- 172.21.15.82:0/433703251 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6349 (secure 0 0 0) 0x7fa44c09bd60 con 0x7fa45c101970 2024-05-20T16:11:31.610 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-05-20T16:11:31.610 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/2044534217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":2044534217},{"type":"v1","addr":"172.21.15.82:6835","nonce":2044534217}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/2635635097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":2635635097},{"type":"v1","addr":"172.21.15.82:6837","nonce":2635635097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6833/4015144007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":4015144007},{"type":"v1","addr":"172.21.15.184:6833","nonce":4015144007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6833/975243372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6832","nonce":975243372},{"type":"v1","addr":"172.21.15.160:6833","nonce":975243372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24470,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6835/664068363","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":664068363},{"type":"v1","addr":"172.21.15.160:6835","nonce":664068363}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24478,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/4242156090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":4242156090},{"type":"v1","addr":"172.21.15.184:6835","nonce":4242156090}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24496,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/403262242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":403262242},{"type":"v1","addr":"172.21.15.184:6837","nonce":403262242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-05-20T16:11:31.612 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 8 2024-05-20T16:11:31.612 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:31.611+0000 7fa4614a8640 1 -- 172.21.15.82:0/433703251 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fa4440a2b90 msgr2=0x7fa4440a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:11:31.612 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:31.611+0000 7fa4614a8640 1 --2- 172.21.15.82:0/433703251 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fa4440a2b90 0x7fa4440a50a0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fa45c1991b0 tx=0x7fa45000a2d0 comp rx=0 tx=0).stop 2024-05-20T16:11:31.612 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:31.611+0000 7fa4614a8640 1 -- 172.21.15.82:0/433703251 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa45c101970 msgr2=0x7fa45c198110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:11:31.613 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:31.611+0000 7fa4614a8640 1 --2- 172.21.15.82:0/433703251 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa45c101970 0x7fa45c198110 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fa44c00e9f0 tx=0x7fa44c00edf0 comp rx=0 tx=0).stop 2024-05-20T16:11:31.613 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:31.611+0000 7fa4614a8640 1 -- 172.21.15.82:0/433703251 shutdown_connections 2024-05-20T16:11:31.613 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:31.611+0000 7fa4614a8640 1 --2- 172.21.15.82:0/433703251 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fa4440a2b90 0x7fa4440a50a0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:31.613 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:31.611+0000 7fa4614a8640 1 --2- 172.21.15.82:0/433703251 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa45c1078b0 0x7fa45c19f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:31.613 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:31.611+0000 7fa4614a8640 1 --2- 172.21.15.82:0/433703251 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa45c102bb0 0x7fa45c198650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:31.613 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:31.611+0000 7fa4614a8640 1 --2- 172.21.15.82:0/433703251 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa45c101970 0x7fa45c198110 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:31.614 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:31.611+0000 7fa4614a8640 1 -- 172.21.15.82:0/433703251 >> 172.21.15.82:0/433703251 conn(0x7fa45c0fcb80 msgr2=0x7fa45c103890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:11:31.614 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:31.611+0000 7fa4614a8640 1 -- 172.21.15.82:0/433703251 shutdown_connections 2024-05-20T16:11:31.614 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:31.611+0000 7fa4614a8640 1 -- 172.21.15.82:0/433703251 wait complete. 2024-05-20T16:11:31.635 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:31 smithi184 ceph-mon[36209]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-20T16:11:31.988 INFO:journalctl@ceph.mds.h.smithi160.stdout:May 20 16:11:31 smithi160 podman[66625]: 2024-05-20 16:11:31.572068123 +0000 UTC m=+0.018094644 image pull cfcc61b2d476f611479310eb042aa05f11dd5f60dbb65e3b0d7e0e11d024cdb3 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708 2024-05-20T16:11:31.988 INFO:journalctl@ceph.mds.h.smithi160.stdout:May 20 16:11:31 smithi160 podman[66625]: 2024-05-20 16:11:31.678252206 +0000 UTC m=+0.124278708 container create 0e2acd3f66e0f0b71d50460f761fbd57505a06c0c7e29aefe2b3431a89436d99 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-h, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240514, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, ceph=True) 2024-05-20T16:11:31.988 INFO:journalctl@ceph.mds.h.smithi160.stdout:May 20 16:11:31 smithi160 podman[66625]: 2024-05-20 16:11:31.989152901 +0000 UTC m=+0.435179409 container init 0e2acd3f66e0f0b71d50460f761fbd57505a06c0c7e29aefe2b3431a89436d99 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-h, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240514, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-05-20T16:11:32.312 INFO:journalctl@ceph.mds.h.smithi160.stdout:May 20 16:11:31 smithi160 podman[66625]: 2024-05-20 16:11:31.992449608 +0000 UTC m=+0.438476118 container start 0e2acd3f66e0f0b71d50460f761fbd57505a06c0c7e29aefe2b3431a89436d99 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-h, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240514, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-20T16:11:32.312 INFO:journalctl@ceph.mds.h.smithi160.stdout:May 20 16:11:32 smithi160 ceph-mds[66644]: -- 172.21.15.160:0/247932998 <== mon.2 v2:172.21.15.160:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55db3df75a40 con 0x55db3dfed600 2024-05-20T16:11:32.312 INFO:journalctl@ceph.mds.h.smithi160.stdout:May 20 16:11:32 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-h[66640]: starting mds.h at 2024-05-20T16:11:32.312 INFO:journalctl@ceph.mds.h.smithi160.stdout:May 20 16:11:32 smithi160 bash[66625]: 0e2acd3f66e0f0b71d50460f761fbd57505a06c0c7e29aefe2b3431a89436d99 2024-05-20T16:11:32.312 INFO:journalctl@ceph.mds.h.smithi160.stdout:May 20 16:11:32 smithi160 systemd[1]: Started Ceph mds.h for 59d1600a-16c2-11ef-bc9a-c7b262605968. 2024-05-20T16:11:32.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:32 smithi082 ceph-mon[32238]: from='client.? 172.21.15.82:0/433703251' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T16:11:32.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:32 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:11:32.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:32 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:11:32.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:32 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:11:32.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:32 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-20T16:11:32.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:32 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-20T16:11:32.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:32 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:11:32.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:32 smithi082 ceph-mon[32238]: Deploying daemon mds.i on smithi184 2024-05-20T16:11:32.635 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:32 smithi184 ceph-mon[36209]: from='client.? 172.21.15.82:0/433703251' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T16:11:32.635 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:32 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:11:32.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:32 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:11:32.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:32 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:11:32.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:32 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-20T16:11:32.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:32 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-20T16:11:32.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:32 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:11:32.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:32 smithi184 ceph-mon[36209]: Deploying daemon mds.i on smithi184 2024-05-20T16:11:32.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:32 smithi160 ceph-mon[36347]: from='client.? 172.21.15.82:0/433703251' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T16:11:32.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:32 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:11:32.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:32 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:11:32.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:32 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:11:32.813 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:32 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-20T16:11:32.813 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:32 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-20T16:11:32.813 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:32 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:11:32.813 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:32 smithi160 ceph-mon[36347]: Deploying daemon mds.i on smithi184 2024-05-20T16:11:33.624 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-20T16:11:33.635 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:33 smithi184 ceph-mon[36209]: mds.? [v2:172.21.15.160:6836/2846188680,v1:172.21.15.160:6837/2846188680] up:boot 2024-05-20T16:11:33.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:33 smithi184 ceph-mon[36209]: fsmap 8 up:standby 2024-05-20T16:11:33.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:33 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-20T16:11:33.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:33 smithi184 ceph-mon[36209]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-20T16:11:33.650 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:33 smithi082 ceph-mon[32238]: mds.? [v2:172.21.15.160:6836/2846188680,v1:172.21.15.160:6837/2846188680] up:boot 2024-05-20T16:11:33.650 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:33 smithi082 ceph-mon[32238]: fsmap 8 up:standby 2024-05-20T16:11:33.651 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:33 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-20T16:11:33.651 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:33 smithi082 ceph-mon[32238]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-20T16:11:33.748 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:33.748+0000 7f8791be3640 1 -- 172.21.15.82:0/1991650788 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f878c101970 msgr2=0x7f878c102670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:11:33.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:33.748+0000 7f8791be3640 1 --2- 172.21.15.82:0/1991650788 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f878c101970 0x7f878c102670 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f877c009ca0 tx=0x7f877c02a8c0 comp rx=0 tx=0).stop 2024-05-20T16:11:33.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:33.748+0000 7f8791be3640 1 -- 172.21.15.82:0/1991650788 shutdown_connections 2024-05-20T16:11:33.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:33.748+0000 7f8791be3640 1 --2- 172.21.15.82:0/1991650788 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f878c1078b0 0x7f878c109cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:33.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:33.748+0000 7f8791be3640 1 --2- 172.21.15.82:0/1991650788 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f878c102bb0 0x7f878c102fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:33.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:33.748+0000 7f8791be3640 1 --2- 172.21.15.82:0/1991650788 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f878c101970 0x7f878c102670 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:33.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:33.748+0000 7f8791be3640 1 -- 172.21.15.82:0/1991650788 >> 172.21.15.82:0/1991650788 conn(0x7f878c0fcb80 msgr2=0x7f878c0ff010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:11:33.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:33.748+0000 7f8791be3640 1 -- 172.21.15.82:0/1991650788 shutdown_connections 2024-05-20T16:11:33.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:33.748+0000 7f8791be3640 1 -- 172.21.15.82:0/1991650788 wait complete. 2024-05-20T16:11:33.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:33.749+0000 7f8791be3640 1 Processor -- start 2024-05-20T16:11:33.751 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:33.749+0000 7f8791be3640 1 -- start start 2024-05-20T16:11:33.751 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:33.749+0000 7f8791be3640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f878c101970 0x7f878c198180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:11:33.751 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:33.749+0000 7f8791be3640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f878c102bb0 0x7f878c1986c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:11:33.751 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:33.749+0000 7f8791be3640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f878c1078b0 0x7f878c19f650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:11:33.751 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:33.749+0000 7f8791be3640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f878c10d5a0 con 0x7f878c101970 2024-05-20T16:11:33.752 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:33.749+0000 7f8791be3640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f878c10d420 con 0x7f878c102bb0 2024-05-20T16:11:33.752 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:33.749+0000 7f8791be3640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f878c10d720 con 0x7f878c1078b0 2024-05-20T16:11:33.752 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:33.749+0000 7f878b7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f878c101970 0x7f878c198180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:11:33.752 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:33.749+0000 7f878b7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f878c101970 0x7f878c198180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:49366/0 (socket says 172.21.15.82:49366) 2024-05-20T16:11:33.752 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:33.749+0000 7f878b7fe640 1 -- 172.21.15.82:0/3766232318 learned_addr learned my addr 172.21.15.82:0/3766232318 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:11:33.752 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:33.750+0000 7f878affd640 1 --2- 172.21.15.82:0/3766232318 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f878c102bb0 0x7f878c1986c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:11:33.752 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:33.750+0000 7f878bfff640 1 --2- 172.21.15.82:0/3766232318 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f878c1078b0 0x7f878c19f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:11:33.752 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:33.750+0000 7f878b7fe640 1 -- 172.21.15.82:0/3766232318 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f878c1078b0 msgr2=0x7f878c19f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:11:33.752 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:33.750+0000 7f878b7fe640 1 --2- 172.21.15.82:0/3766232318 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f878c1078b0 0x7f878c19f650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:33.752 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:33.750+0000 7f878b7fe640 1 -- 172.21.15.82:0/3766232318 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f878c102bb0 msgr2=0x7f878c1986c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:11:33.752 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:33.750+0000 7f878b7fe640 1 --2- 172.21.15.82:0/3766232318 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f878c102bb0 0x7f878c1986c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:33.752 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:33.750+0000 7f878b7fe640 1 -- 172.21.15.82:0/3766232318 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f878c19fc10 con 0x7f878c101970 2024-05-20T16:11:33.752 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:33.750+0000 7f878bfff640 1 --2- 172.21.15.82:0/3766232318 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f878c1078b0 0x7f878c19f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T16:11:33.752 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:33.750+0000 7f878b7fe640 1 --2- 172.21.15.82:0/3766232318 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f878c101970 0x7f878c198180 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f877c000f80 tx=0x7f877c02f840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:11:33.752 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:33.750+0000 7f8788ff9640 1 -- 172.21.15.82:0/3766232318 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f877c02fdc0 con 0x7f878c101970 2024-05-20T16:11:33.753 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:33.750+0000 7f8791be3640 1 -- 172.21.15.82:0/3766232318 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f878c19fea0 con 0x7f878c101970 2024-05-20T16:11:33.753 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:33.750+0000 7f8788ff9640 1 -- 172.21.15.82:0/3766232318 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f877c030aa0 con 0x7f878c101970 2024-05-20T16:11:33.753 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:33.750+0000 7f8788ff9640 1 -- 172.21.15.82:0/3766232318 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f877c046950 con 0x7f878c101970 2024-05-20T16:11:33.753 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:33.750+0000 7f8791be3640 1 -- 172.21.15.82:0/3766232318 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f878c1a0300 con 0x7f878c101970 2024-05-20T16:11:33.753 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:33.751+0000 7f8791be3640 1 -- 172.21.15.82:0/3766232318 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f878c1066c0 con 0x7f878c101970 2024-05-20T16:11:33.754 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:33.752+0000 7f8788ff9640 1 -- 172.21.15.82:0/3766232318 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f877c0305c0 con 0x7f878c101970 2024-05-20T16:11:33.754 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:33.753+0000 7f8788ff9640 1 --2- 172.21.15.82:0/3766232318 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f87740a2b90 0x7f87740a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:11:33.754 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:33.753+0000 7f878affd640 1 --2- 172.21.15.82:0/3766232318 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f87740a2b90 0x7f87740a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:11:33.754 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:33.753+0000 7f8788ff9640 1 -- 172.21.15.82:0/3766232318 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f877c0f0320 con 0x7f878c101970 2024-05-20T16:11:33.754 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:33.754+0000 7f878affd640 1 --2- 172.21.15.82:0/3766232318 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f87740a2b90 0x7f87740a50a0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f878c199220 tx=0x7f878000a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:11:33.757 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:33.758+0000 7f8788ff9640 1 -- 172.21.15.82:0/3766232318 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f877c0bd720 con 0x7f878c101970 2024-05-20T16:11:33.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:33 smithi160 ceph-mon[36347]: mds.? [v2:172.21.15.160:6836/2846188680,v1:172.21.15.160:6837/2846188680] up:boot 2024-05-20T16:11:33.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:33 smithi160 ceph-mon[36347]: fsmap 8 up:standby 2024-05-20T16:11:33.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:33 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-20T16:11:33.813 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:33 smithi160 ceph-mon[36347]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-20T16:11:33.970 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:33.970+0000 7f8791be3640 1 -- 172.21.15.82:0/3766232318 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f878c02d2a0 con 0x7f878c101970 2024-05-20T16:11:33.970 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:33.970+0000 7f8788ff9640 1 -- 172.21.15.82:0/3766232318 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7177 (secure 0 0 0) 0x7f877c0dd0e0 con 0x7f878c101970 2024-05-20T16:11:33.971 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-05-20T16:11:33.971 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/2044534217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":2044534217},{"type":"v1","addr":"172.21.15.82:6835","nonce":2044534217}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/2635635097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":2635635097},{"type":"v1","addr":"172.21.15.82:6837","nonce":2635635097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6833/4015144007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":4015144007},{"type":"v1","addr":"172.21.15.184:6833","nonce":4015144007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6833/975243372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6832","nonce":975243372},{"type":"v1","addr":"172.21.15.160:6833","nonce":975243372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24470,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6835/664068363","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":664068363},{"type":"v1","addr":"172.21.15.160:6835","nonce":664068363}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24478,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/4242156090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":4242156090},{"type":"v1","addr":"172.21.15.184:6835","nonce":4242156090}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24496,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/403262242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":403262242},{"type":"v1","addr":"172.21.15.184:6837","nonce":403262242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24506,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6837/2846188680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6836","nonce":2846188680},{"type":"v1","addr":"172.21.15.160:6837","nonce":2846188680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-05-20T16:11:33.971 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 9 2024-05-20T16:11:33.973 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:33.972+0000 7f8791be3640 1 -- 172.21.15.82:0/3766232318 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f87740a2b90 msgr2=0x7f87740a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:11:33.973 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:33.972+0000 7f8791be3640 1 --2- 172.21.15.82:0/3766232318 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f87740a2b90 0x7f87740a50a0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f878c199220 tx=0x7f878000a2d0 comp rx=0 tx=0).stop 2024-05-20T16:11:33.973 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:33.972+0000 7f8791be3640 1 -- 172.21.15.82:0/3766232318 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f878c101970 msgr2=0x7f878c198180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:11:33.973 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:33.972+0000 7f8791be3640 1 --2- 172.21.15.82:0/3766232318 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f878c101970 0x7f878c198180 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f877c000f80 tx=0x7f877c02f840 comp rx=0 tx=0).stop 2024-05-20T16:11:33.973 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:33.972+0000 7f8791be3640 1 -- 172.21.15.82:0/3766232318 shutdown_connections 2024-05-20T16:11:33.973 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:33.972+0000 7f8791be3640 1 --2- 172.21.15.82:0/3766232318 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f87740a2b90 0x7f87740a50a0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:33.973 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:33.972+0000 7f8791be3640 1 --2- 172.21.15.82:0/3766232318 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f878c1078b0 0x7f878c19f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:33.973 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:33.972+0000 7f8791be3640 1 --2- 172.21.15.82:0/3766232318 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f878c102bb0 0x7f878c1986c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:33.973 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:33.972+0000 7f8791be3640 1 --2- 172.21.15.82:0/3766232318 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f878c101970 0x7f878c198180 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:33.973 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:33.972+0000 7f8791be3640 1 -- 172.21.15.82:0/3766232318 >> 172.21.15.82:0/3766232318 conn(0x7f878c0fcb80 msgr2=0x7f878c103890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:11:33.974 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:33.973+0000 7f8791be3640 1 -- 172.21.15.82:0/3766232318 shutdown_connections 2024-05-20T16:11:33.974 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:33.973+0000 7f8791be3640 1 -- 172.21.15.82:0/3766232318 wait complete. 2024-05-20T16:11:34.362 INFO:journalctl@ceph.mds.i.smithi184.stdout:May 20 16:11:34 smithi184 systemd[1]: Starting Ceph mds.i for 59d1600a-16c2-11ef-bc9a-c7b262605968... 2024-05-20T16:11:34.632 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:34 smithi184 ceph-mon[36209]: from='client.? 172.21.15.82:0/3766232318' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T16:11:34.632 INFO:journalctl@ceph.mds.i.smithi184.stdout:May 20 16:11:34 smithi184 podman[67690]: 2024-05-20 16:11:34.401475004 +0000 UTC m=+0.017451341 image pull cfcc61b2d476f611479310eb042aa05f11dd5f60dbb65e3b0d7e0e11d024cdb3 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708 2024-05-20T16:11:34.632 INFO:journalctl@ceph.mds.i.smithi184.stdout:May 20 16:11:34 smithi184 podman[67690]: 2024-05-20 16:11:34.518464431 +0000 UTC m=+0.134440785 container create a27a46359d78869b0f838831347de84fe8bf0ccc74e75cee82130b40a2ad089c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-i, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240514, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-05-20T16:11:34.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:34 smithi160 ceph-mon[36347]: from='client.? 172.21.15.82:0/3766232318' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T16:11:34.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:34 smithi082 ceph-mon[32238]: from='client.? 172.21.15.82:0/3766232318' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T16:11:35.136 INFO:journalctl@ceph.mds.i.smithi184.stdout:May 20 16:11:34 smithi184 podman[67690]: 2024-05-20 16:11:34.738423938 +0000 UTC m=+0.354400261 container init a27a46359d78869b0f838831347de84fe8bf0ccc74e75cee82130b40a2ad089c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-i, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240514, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c) 2024-05-20T16:11:35.136 INFO:journalctl@ceph.mds.i.smithi184.stdout:May 20 16:11:34 smithi184 podman[67690]: 2024-05-20 16:11:34.741010315 +0000 UTC m=+0.356986650 container start a27a46359d78869b0f838831347de84fe8bf0ccc74e75cee82130b40a2ad089c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-i, org.label-schema.license=GPLv2, org.label-schema.build-date=20240514, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-05-20T16:11:35.136 INFO:journalctl@ceph.mds.i.smithi184.stdout:May 20 16:11:34 smithi184 ceph-mds[67709]: -- 172.21.15.184:0/3729441692 <== mon.1 v2:172.21.15.184:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e05cc67a40 con 0x55e05ccdfa80 2024-05-20T16:11:35.136 INFO:journalctl@ceph.mds.i.smithi184.stdout:May 20 16:11:34 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-i[67705]: starting mds.i at 2024-05-20T16:11:35.136 INFO:journalctl@ceph.mds.i.smithi184.stdout:May 20 16:11:34 smithi184 bash[67690]: a27a46359d78869b0f838831347de84fe8bf0ccc74e75cee82130b40a2ad089c 2024-05-20T16:11:35.136 INFO:journalctl@ceph.mds.i.smithi184.stdout:May 20 16:11:34 smithi184 systemd[1]: Started Ceph mds.i for 59d1600a-16c2-11ef-bc9a-c7b262605968. 2024-05-20T16:11:35.985 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-20T16:11:36.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:36.100+0000 7f4fb8777640 1 -- 172.21.15.82:0/4075689349 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4fb0102ab0 msgr2=0x7f4fb01037b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:11:36.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:36.100+0000 7f4fb8777640 1 --2- 172.21.15.82:0/4075689349 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4fb0102ab0 0x7f4fb01037b0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f4fa0009900 tx=0x7f4fa002a410 comp rx=0 tx=0).stop 2024-05-20T16:11:36.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:36.100+0000 7f4fb8777640 1 -- 172.21.15.82:0/4075689349 shutdown_connections 2024-05-20T16:11:36.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:36.100+0000 7f4fb8777640 1 --2- 172.21.15.82:0/4075689349 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4fb01089f0 0x7f4fb010ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:36.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:36.100+0000 7f4fb8777640 1 --2- 172.21.15.82:0/4075689349 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4fb0103cf0 0x7f4fb0104100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:36.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:36.100+0000 7f4fb8777640 1 --2- 172.21.15.82:0/4075689349 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4fb0102ab0 0x7f4fb01037b0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:36.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:36.100+0000 7f4fb8777640 1 -- 172.21.15.82:0/4075689349 >> 172.21.15.82:0/4075689349 conn(0x7f4fb00fdcc0 msgr2=0x7f4fb0100150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:11:36.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:36.100+0000 7f4fb8777640 1 -- 172.21.15.82:0/4075689349 shutdown_connections 2024-05-20T16:11:36.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:36.100+0000 7f4fb8777640 1 -- 172.21.15.82:0/4075689349 wait complete. 2024-05-20T16:11:36.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:36.101+0000 7f4fb8777640 1 Processor -- start 2024-05-20T16:11:36.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:36.101+0000 7f4fb8777640 1 -- start start 2024-05-20T16:11:36.103 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:36.101+0000 7f4fb8777640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4fb0102ab0 0x7f4fb01992e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:11:36.103 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:36.101+0000 7f4fb8777640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4fb0103cf0 0x7f4fb0199820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:11:36.103 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:36.101+0000 7f4fb8777640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4fb01089f0 0x7f4fb01a07b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:11:36.103 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:36.101+0000 7f4fb8777640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f4fb010e700 con 0x7f4fb0102ab0 2024-05-20T16:11:36.103 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:36.101+0000 7f4fb8777640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f4fb010e580 con 0x7f4fb01089f0 2024-05-20T16:11:36.103 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:36.101+0000 7f4fb8777640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f4fb010e880 con 0x7f4fb0103cf0 2024-05-20T16:11:36.103 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:36.101+0000 7f4fb6513640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4fb0102ab0 0x7f4fb01992e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:11:36.103 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:36.101+0000 7f4fb6513640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4fb0102ab0 0x7f4fb01992e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:49388/0 (socket says 172.21.15.82:49388) 2024-05-20T16:11:36.103 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:36.101+0000 7f4fb6513640 1 -- 172.21.15.82:0/3068266597 learned_addr learned my addr 172.21.15.82:0/3068266597 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:11:36.103 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:36.101+0000 7f4fb6d14640 1 --2- 172.21.15.82:0/3068266597 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4fb01089f0 0x7f4fb01a07b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:11:36.104 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:36.101+0000 7f4fb5d12640 1 --2- 172.21.15.82:0/3068266597 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4fb0103cf0 0x7f4fb0199820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:11:36.104 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:36.101+0000 7f4fb6513640 1 -- 172.21.15.82:0/3068266597 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4fb0103cf0 msgr2=0x7f4fb0199820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:11:36.104 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:36.101+0000 7f4fb6513640 1 --2- 172.21.15.82:0/3068266597 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4fb0103cf0 0x7f4fb0199820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:36.104 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:36.101+0000 7f4fb6513640 1 -- 172.21.15.82:0/3068266597 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4fb01089f0 msgr2=0x7f4fb01a07b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:11:36.104 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:36.101+0000 7f4fb6513640 1 --2- 172.21.15.82:0/3068266597 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4fb01089f0 0x7f4fb01a07b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:36.104 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:36.101+0000 7f4fb6513640 1 -- 172.21.15.82:0/3068266597 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fb01a0e20 con 0x7f4fb0102ab0 2024-05-20T16:11:36.104 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:36.101+0000 7f4fb5d12640 1 --2- 172.21.15.82:0/3068266597 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4fb0103cf0 0x7f4fb0199820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T16:11:36.104 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:36.101+0000 7f4fb6d14640 1 --2- 172.21.15.82:0/3068266597 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4fb01089f0 0x7f4fb01a07b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T16:11:36.104 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:36.102+0000 7f4fb6513640 1 --2- 172.21.15.82:0/3068266597 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4fb0102ab0 0x7f4fb01992e0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f4fa0006010 tx=0x7f4fa00043b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:11:36.104 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:36.102+0000 7f4fa77fe640 1 -- 172.21.15.82:0/3068266597 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fa003f030 con 0x7f4fb0102ab0 2024-05-20T16:11:36.104 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:36.102+0000 7f4fa77fe640 1 -- 172.21.15.82:0/3068266597 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4fa0003ce0 con 0x7f4fb0102ab0 2024-05-20T16:11:36.104 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:36.102+0000 7f4fb8777640 1 -- 172.21.15.82:0/3068266597 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4fb01a10b0 con 0x7f4fb0102ab0 2024-05-20T16:11:36.104 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:36.102+0000 7f4fa77fe640 1 -- 172.21.15.82:0/3068266597 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fa0035070 con 0x7f4fb0102ab0 2024-05-20T16:11:36.104 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:36.102+0000 7f4fb8777640 1 -- 172.21.15.82:0/3068266597 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4fb01a15a0 con 0x7f4fb0102ab0 2024-05-20T16:11:36.105 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:36.103+0000 7f4fb8777640 1 -- 172.21.15.82:0/3068266597 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4fb0107800 con 0x7f4fb0102ab0 2024-05-20T16:11:36.105 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:36.104+0000 7f4fa77fe640 1 -- 172.21.15.82:0/3068266597 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f4fa0042030 con 0x7f4fb0102ab0 2024-05-20T16:11:36.105 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:36.105+0000 7f4fa77fe640 1 --2- 172.21.15.82:0/3068266597 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f4f9c0a2b90 0x7f4f9c0a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:11:36.105 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:36.105+0000 7f4fb5d12640 1 --2- 172.21.15.82:0/3068266597 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f4f9c0a2b90 0x7f4f9c0a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:11:36.106 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:36.105+0000 7f4fa77fe640 1 -- 172.21.15.82:0/3068266597 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f4fa00ef490 con 0x7f4fb0102ab0 2024-05-20T16:11:36.106 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:36.105+0000 7f4fb5d12640 1 --2- 172.21.15.82:0/3068266597 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f4f9c0a2b90 0x7f4f9c0a50a0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f4fac0058c0 tx=0x7f4fac005d50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:11:36.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:36.108+0000 7f4fa77fe640 1 -- 172.21.15.82:0/3068266597 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f4fa00bc890 con 0x7f4fb0102ab0 2024-05-20T16:11:36.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:35 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:11:36.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:35 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:11:36.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:35 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:11:36.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:35 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-20T16:11:36.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:35 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-20T16:11:36.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:35 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:11:36.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:35 smithi082 ceph-mon[32238]: Deploying daemon mds.g on smithi082 2024-05-20T16:11:36.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:35 smithi082 ceph-mon[32238]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-20T16:11:36.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:35 smithi082 ceph-mon[32238]: mds.? [v2:172.21.15.184:6838/2199955067,v1:172.21.15.184:6839/2199955067] up:boot 2024-05-20T16:11:36.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:35 smithi082 ceph-mon[32238]: fsmap 9 up:standby 2024-05-20T16:11:36.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:35 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-20T16:11:36.136 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:35 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:11:36.136 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:35 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:11:36.136 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:35 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:11:36.136 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:35 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-20T16:11:36.136 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:35 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-20T16:11:36.136 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:35 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:11:36.136 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:35 smithi184 ceph-mon[36209]: Deploying daemon mds.g on smithi082 2024-05-20T16:11:36.136 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:35 smithi184 ceph-mon[36209]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-20T16:11:36.136 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:35 smithi184 ceph-mon[36209]: mds.? [v2:172.21.15.184:6838/2199955067,v1:172.21.15.184:6839/2199955067] up:boot 2024-05-20T16:11:36.136 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:35 smithi184 ceph-mon[36209]: fsmap 9 up:standby 2024-05-20T16:11:36.136 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:35 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-20T16:11:36.312 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:35 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:11:36.313 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:35 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:11:36.313 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:35 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:11:36.313 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:35 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-20T16:11:36.313 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:35 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-20T16:11:36.313 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:35 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:11:36.313 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:35 smithi160 ceph-mon[36347]: Deploying daemon mds.g on smithi082 2024-05-20T16:11:36.313 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:35 smithi160 ceph-mon[36347]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-20T16:11:36.313 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:35 smithi160 ceph-mon[36347]: mds.? [v2:172.21.15.184:6838/2199955067,v1:172.21.15.184:6839/2199955067] up:boot 2024-05-20T16:11:36.313 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:35 smithi160 ceph-mon[36347]: fsmap 9 up:standby 2024-05-20T16:11:36.313 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:35 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-20T16:11:36.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:36.325+0000 7f4fb8777640 1 -- 172.21.15.82:0/3068266597 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4fb002d2a0 con 0x7f4fb0102ab0 2024-05-20T16:11:36.326 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:36.326+0000 7f4fa77fe640 1 -- 172.21.15.82:0/3068266597 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+8007 (secure 0 0 0) 0x7f4fa00bc190 con 0x7f4fb0102ab0 2024-05-20T16:11:36.326 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-05-20T16:11:36.326 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/2044534217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":2044534217},{"type":"v1","addr":"172.21.15.82:6835","nonce":2044534217}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/2635635097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":2635635097},{"type":"v1","addr":"172.21.15.82:6837","nonce":2635635097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6833/4015144007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":4015144007},{"type":"v1","addr":"172.21.15.184:6833","nonce":4015144007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6833/975243372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6832","nonce":975243372},{"type":"v1","addr":"172.21.15.160:6833","nonce":975243372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24470,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6835/664068363","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":664068363},{"type":"v1","addr":"172.21.15.160:6835","nonce":664068363}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24478,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/4242156090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":4242156090},{"type":"v1","addr":"172.21.15.184:6835","nonce":4242156090}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24496,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/403262242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":403262242},{"type":"v1","addr":"172.21.15.184:6837","nonce":403262242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24506,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6837/2846188680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6836","nonce":2846188680},{"type":"v1","addr":"172.21.15.160:6837","nonce":2846188680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24511,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6839/2199955067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":2199955067},{"type":"v1","addr":"172.21.15.184:6839","nonce":2199955067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-05-20T16:11:36.326 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 10 2024-05-20T16:11:36.328 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:36.328+0000 7f4fb8777640 1 -- 172.21.15.82:0/3068266597 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f4f9c0a2b90 msgr2=0x7f4f9c0a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:11:36.328 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:36.328+0000 7f4fb8777640 1 --2- 172.21.15.82:0/3068266597 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f4f9c0a2b90 0x7f4f9c0a50a0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f4fac0058c0 tx=0x7f4fac005d50 comp rx=0 tx=0).stop 2024-05-20T16:11:36.328 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:36.328+0000 7f4fb8777640 1 -- 172.21.15.82:0/3068266597 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4fb0102ab0 msgr2=0x7f4fb01992e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:11:36.328 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:36.328+0000 7f4fb8777640 1 --2- 172.21.15.82:0/3068266597 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4fb0102ab0 0x7f4fb01992e0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f4fa0006010 tx=0x7f4fa00043b0 comp rx=0 tx=0).stop 2024-05-20T16:11:36.328 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:36.328+0000 7f4fb8777640 1 -- 172.21.15.82:0/3068266597 shutdown_connections 2024-05-20T16:11:36.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:36.328+0000 7f4fb8777640 1 --2- 172.21.15.82:0/3068266597 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f4f9c0a2b90 0x7f4f9c0a50a0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:36.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:36.328+0000 7f4fb8777640 1 --2- 172.21.15.82:0/3068266597 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4fb01089f0 0x7f4fb01a07b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:36.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:36.328+0000 7f4fb8777640 1 --2- 172.21.15.82:0/3068266597 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4fb0103cf0 0x7f4fb0199820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:36.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:36.328+0000 7f4fb8777640 1 --2- 172.21.15.82:0/3068266597 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4fb0102ab0 0x7f4fb01992e0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:36.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:36.328+0000 7f4fb8777640 1 -- 172.21.15.82:0/3068266597 >> 172.21.15.82:0/3068266597 conn(0x7f4fb00fdcc0 msgr2=0x7f4fb01049d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:11:36.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:36.328+0000 7f4fb8777640 1 -- 172.21.15.82:0/3068266597 shutdown_connections 2024-05-20T16:11:36.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:36.328+0000 7f4fb8777640 1 -- 172.21.15.82:0/3068266597 wait complete. 2024-05-20T16:11:37.193 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:36 smithi082 ceph-mon[32238]: from='client.? 172.21.15.82:0/3068266597' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T16:11:37.193 INFO:journalctl@ceph.mds.g.smithi082.stdout:May 20 16:11:36 smithi082 systemd[1]: Starting Ceph mds.g for 59d1600a-16c2-11ef-bc9a-c7b262605968... 2024-05-20T16:11:37.312 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:36 smithi160 ceph-mon[36347]: from='client.? 172.21.15.82:0/3068266597' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T16:11:37.386 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:36 smithi184 ceph-mon[36209]: from='client.? 172.21.15.82:0/3068266597' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T16:11:37.481 INFO:journalctl@ceph.mds.g.smithi082.stdout:May 20 16:11:37 smithi082 podman[73039]: 2024-05-20 16:11:37.096722812 +0000 UTC m=+0.018027447 image pull cfcc61b2d476f611479310eb042aa05f11dd5f60dbb65e3b0d7e0e11d024cdb3 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708 2024-05-20T16:11:37.481 INFO:journalctl@ceph.mds.g.smithi082.stdout:May 20 16:11:37 smithi082 podman[73039]: 2024-05-20 16:11:37.2293571 +0000 UTC m=+0.150661709 container create e65fc67c67e41761d3346ac331f0d1cc1292972c6f7458ce3838f10a24efb837 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-g, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240514, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5) 2024-05-20T16:11:37.481 INFO:journalctl@ceph.mds.g.smithi082.stdout:May 20 16:11:37 smithi082 podman[73039]: 2024-05-20 16:11:37.481887891 +0000 UTC m=+0.403192507 container init e65fc67c67e41761d3346ac331f0d1cc1292972c6f7458ce3838f10a24efb837 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-g, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, org.label-schema.build-date=20240514) 2024-05-20T16:11:37.859 INFO:journalctl@ceph.mds.g.smithi082.stdout:May 20 16:11:37 smithi082 podman[73039]: 2024-05-20 16:11:37.485035831 +0000 UTC m=+0.406340465 container start e65fc67c67e41761d3346ac331f0d1cc1292972c6f7458ce3838f10a24efb837 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-g, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240514, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-05-20T16:11:37.859 INFO:journalctl@ceph.mds.g.smithi082.stdout:May 20 16:11:37 smithi082 ceph-mds[73058]: -- 172.21.15.82:0/255088060 <== mon.0 v2:172.21.15.82:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e2f01cba40 con 0x55e2f0243a80 2024-05-20T16:11:37.859 INFO:journalctl@ceph.mds.g.smithi082.stdout:May 20 16:11:37 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-g[73054]: starting mds.g at 2024-05-20T16:11:37.859 INFO:journalctl@ceph.mds.g.smithi082.stdout:May 20 16:11:37 smithi082 bash[73039]: e65fc67c67e41761d3346ac331f0d1cc1292972c6f7458ce3838f10a24efb837 2024-05-20T16:11:37.859 INFO:journalctl@ceph.mds.g.smithi082.stdout:May 20 16:11:37 smithi082 systemd[1]: Started Ceph mds.g for 59d1600a-16c2-11ef-bc9a-c7b262605968. 2024-05-20T16:11:38.312 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:37 smithi160 ceph-mon[36347]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-20T16:11:38.312 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:37 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:11:38.312 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:37 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:11:38.312 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:37 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:11:38.312 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:37 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-20T16:11:38.312 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:37 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-20T16:11:38.313 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:37 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:11:38.321 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:37 smithi082 ceph-mon[32238]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-20T16:11:38.321 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:37 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:11:38.321 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:37 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:11:38.321 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:37 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:11:38.321 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:37 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-20T16:11:38.321 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:37 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-20T16:11:38.321 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:37 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:11:38.340 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-20T16:11:38.386 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:37 smithi184 ceph-mon[36209]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-20T16:11:38.386 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:37 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:11:38.386 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:37 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:11:38.386 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:37 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:11:38.386 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:37 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-20T16:11:38.386 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:37 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-20T16:11:38.386 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:37 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:11:38.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:38.459+0000 7f278c977640 1 -- 172.21.15.82:0/1469549175 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2788101f40 msgr2=0x7f2788102350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:11:38.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:38.459+0000 7f278c977640 1 --2- 172.21.15.82:0/1469549175 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2788101f40 0x7f2788102350 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f2778009900 tx=0x7f277802a410 comp rx=0 tx=0).stop 2024-05-20T16:11:38.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:38.459+0000 7f278c977640 1 -- 172.21.15.82:0/1469549175 shutdown_connections 2024-05-20T16:11:38.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:38.459+0000 7f278c977640 1 --2- 172.21.15.82:0/1469549175 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2788103480 0x7f2788109bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:38.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:38.459+0000 7f278c977640 1 --2- 172.21.15.82:0/1469549175 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2788102920 0x7f2788102df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:38.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:38.459+0000 7f278c977640 1 --2- 172.21.15.82:0/1469549175 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2788101f40 0x7f2788102350 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:38.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:38.459+0000 7f278c977640 1 -- 172.21.15.82:0/1469549175 >> 172.21.15.82:0/1469549175 conn(0x7f27880fc360 msgr2=0x7f27880fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:11:38.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:38.459+0000 7f278c977640 1 -- 172.21.15.82:0/1469549175 shutdown_connections 2024-05-20T16:11:38.460 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:38.459+0000 7f278c977640 1 -- 172.21.15.82:0/1469549175 wait complete. 2024-05-20T16:11:38.462 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:38.460+0000 7f278c977640 1 Processor -- start 2024-05-20T16:11:38.462 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:38.460+0000 7f278c977640 1 -- start start 2024-05-20T16:11:38.462 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:38.460+0000 7f278c977640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2788101f40 0x7f278819b080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:11:38.462 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:38.460+0000 7f278c977640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2788102920 0x7f278819b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:11:38.462 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:38.460+0000 7f278c977640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f27881a0540 0x7f27881a2960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:11:38.462 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:38.460+0000 7f278c977640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f278810c530 con 0x7f27881a0540 2024-05-20T16:11:38.462 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:38.460+0000 7f278c977640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f278810c3b0 con 0x7f2788101f40 2024-05-20T16:11:38.462 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:38.460+0000 7f278c977640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f278810c6b0 con 0x7f2788102920 2024-05-20T16:11:38.462 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:38.460+0000 7f2786d9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f27881a0540 0x7f27881a2960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:11:38.462 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:38.460+0000 7f2786d9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f27881a0540 0x7f27881a2960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:49422/0 (socket says 172.21.15.82:49422) 2024-05-20T16:11:38.462 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:38.460+0000 7f2786d9d640 1 -- 172.21.15.82:0/2842599068 learned_addr learned my addr 172.21.15.82:0/2842599068 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:11:38.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:38.460+0000 7f277ffff640 1 --2- 172.21.15.82:0/2842599068 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2788102920 0x7f278819b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:11:38.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:38.460+0000 7f278659c640 1 --2- 172.21.15.82:0/2842599068 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2788101f40 0x7f278819b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:11:38.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:38.461+0000 7f2786d9d640 1 -- 172.21.15.82:0/2842599068 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2788102920 msgr2=0x7f278819b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:11:38.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:38.461+0000 7f2786d9d640 1 --2- 172.21.15.82:0/2842599068 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2788102920 0x7f278819b5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:38.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:38.461+0000 7f2786d9d640 1 -- 172.21.15.82:0/2842599068 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2788101f40 msgr2=0x7f278819b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:11:38.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:38.461+0000 7f2786d9d640 1 --2- 172.21.15.82:0/2842599068 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2788101f40 0x7f278819b080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:38.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:38.461+0000 7f2786d9d640 1 -- 172.21.15.82:0/2842599068 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27881a2fd0 con 0x7f27881a0540 2024-05-20T16:11:38.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:38.461+0000 7f278659c640 1 --2- 172.21.15.82:0/2842599068 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2788101f40 0x7f278819b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T16:11:38.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:38.461+0000 7f2786d9d640 1 --2- 172.21.15.82:0/2842599068 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f27881a0540 0x7f27881a2960 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f278000da90 tx=0x7f278000df50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:11:38.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:38.461+0000 7f277f7fe640 1 -- 172.21.15.82:0/2842599068 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f278001b020 con 0x7f27881a0540 2024-05-20T16:11:38.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:38.461+0000 7f278c977640 1 -- 172.21.15.82:0/2842599068 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27881a32c0 con 0x7f27881a0540 2024-05-20T16:11:38.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:38.461+0000 7f277f7fe640 1 -- 172.21.15.82:0/2842599068 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f278000bc30 con 0x7f27881a0540 2024-05-20T16:11:38.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:38.461+0000 7f277f7fe640 1 -- 172.21.15.82:0/2842599068 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27800173f0 con 0x7f27881a0540 2024-05-20T16:11:38.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:38.461+0000 7f278c977640 1 -- 172.21.15.82:0/2842599068 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27881a3810 con 0x7f27881a0540 2024-05-20T16:11:38.464 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:38.462+0000 7f278c977640 1 -- 172.21.15.82:0/2842599068 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27881061d0 con 0x7f27881a0540 2024-05-20T16:11:38.464 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:38.463+0000 7f277f7fe640 1 -- 172.21.15.82:0/2842599068 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f2780004a40 con 0x7f27881a0540 2024-05-20T16:11:38.464 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:38.464+0000 7f277f7fe640 1 --2- 172.21.15.82:0/2842599068 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f27740a2b90 0x7f27740a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:11:38.464 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:38.464+0000 7f278659c640 1 --2- 172.21.15.82:0/2842599068 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f27740a2b90 0x7f27740a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:11:38.464 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:38.464+0000 7f277f7fe640 1 -- 172.21.15.82:0/2842599068 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f27800cf460 con 0x7f27881a0540 2024-05-20T16:11:38.465 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:38.465+0000 7f278659c640 1 --2- 172.21.15.82:0/2842599068 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f27740a2b90 0x7f27740a50a0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f2778009230 tx=0x7f27780314f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:11:38.468 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:38.468+0000 7f277f7fe640 1 -- 172.21.15.82:0/2842599068 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f2780025020 con 0x7f27881a0540 2024-05-20T16:11:38.693 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:38.693+0000 7f278c977640 1 -- 172.21.15.82:0/2842599068 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f278802d2a0 con 0x7f27881a0540 2024-05-20T16:11:38.693 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:38.693+0000 7f277f7fe640 1 -- 172.21.15.82:0/2842599068 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8833 (secure 0 0 0) 0x7f278009c8d0 con 0x7f27881a0540 2024-05-20T16:11:38.693 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-05-20T16:11:38.694 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/2044534217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":2044534217},{"type":"v1","addr":"172.21.15.82:6835","nonce":2044534217}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/2635635097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":2635635097},{"type":"v1","addr":"172.21.15.82:6837","nonce":2635635097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14625,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6839/2118233541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6838","nonce":2118233541},{"type":"v1","addr":"172.21.15.82:6839","nonce":2118233541}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6833/4015144007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":4015144007},{"type":"v1","addr":"172.21.15.184:6833","nonce":4015144007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6833/975243372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6832","nonce":975243372},{"type":"v1","addr":"172.21.15.160:6833","nonce":975243372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24470,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6835/664068363","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":664068363},{"type":"v1","addr":"172.21.15.160:6835","nonce":664068363}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24478,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/4242156090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":4242156090},{"type":"v1","addr":"172.21.15.184:6835","nonce":4242156090}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24496,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/403262242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":403262242},{"type":"v1","addr":"172.21.15.184:6837","nonce":403262242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24506,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6837/2846188680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6836","nonce":2846188680},{"type":"v1","addr":"172.21.15.160:6837","nonce":2846188680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24511,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6839/2199955067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":2199955067},{"type":"v1","addr":"172.21.15.184:6839","nonce":2199955067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-05-20T16:11:38.694 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 11 2024-05-20T16:11:38.697 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:38.697+0000 7f278c977640 1 -- 172.21.15.82:0/2842599068 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f27740a2b90 msgr2=0x7f27740a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:11:38.697 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:38.697+0000 7f278c977640 1 --2- 172.21.15.82:0/2842599068 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f27740a2b90 0x7f27740a50a0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f2778009230 tx=0x7f27780314f0 comp rx=0 tx=0).stop 2024-05-20T16:11:38.697 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:38.697+0000 7f278c977640 1 -- 172.21.15.82:0/2842599068 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f27881a0540 msgr2=0x7f27881a2960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:11:38.697 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:38.697+0000 7f278c977640 1 --2- 172.21.15.82:0/2842599068 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f27881a0540 0x7f27881a2960 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f278000da90 tx=0x7f278000df50 comp rx=0 tx=0).stop 2024-05-20T16:11:38.697 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:38.697+0000 7f278c977640 1 -- 172.21.15.82:0/2842599068 shutdown_connections 2024-05-20T16:11:38.697 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:38.697+0000 7f278c977640 1 --2- 172.21.15.82:0/2842599068 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f27740a2b90 0x7f27740a50a0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:38.697 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:38.697+0000 7f278c977640 1 --2- 172.21.15.82:0/2842599068 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f27881a0540 0x7f27881a2960 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:38.697 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:38.697+0000 7f278c977640 1 --2- 172.21.15.82:0/2842599068 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2788102920 0x7f278819b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:38.697 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:38.697+0000 7f278c977640 1 --2- 172.21.15.82:0/2842599068 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2788101f40 0x7f278819b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:38.697 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:38.697+0000 7f278c977640 1 -- 172.21.15.82:0/2842599068 >> 172.21.15.82:0/2842599068 conn(0x7f27880fc360 msgr2=0x7f27880fe780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:11:38.697 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:38.697+0000 7f278c977640 1 -- 172.21.15.82:0/2842599068 shutdown_connections 2024-05-20T16:11:38.697 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:38.697+0000 7f278c977640 1 -- 172.21.15.82:0/2842599068 wait complete. 2024-05-20T16:11:39.062 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:38 smithi160 ceph-mon[36347]: Deploying daemon mds.a on smithi082 2024-05-20T16:11:39.062 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:38 smithi160 ceph-mon[36347]: mds.? [v2:172.21.15.82:6838/2118233541,v1:172.21.15.82:6839/2118233541] up:boot 2024-05-20T16:11:39.062 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:38 smithi160 ceph-mon[36347]: fsmap 10 up:standby 2024-05-20T16:11:39.062 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:38 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-20T16:11:39.063 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:38 smithi160 ceph-mon[36347]: from='client.? 172.21.15.82:0/2842599068' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T16:11:39.306 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:38 smithi082 ceph-mon[32238]: Deploying daemon mds.a on smithi082 2024-05-20T16:11:39.307 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:38 smithi082 ceph-mon[32238]: mds.? [v2:172.21.15.82:6838/2118233541,v1:172.21.15.82:6839/2118233541] up:boot 2024-05-20T16:11:39.307 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:38 smithi082 ceph-mon[32238]: fsmap 10 up:standby 2024-05-20T16:11:39.307 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:38 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-20T16:11:39.307 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:38 smithi082 ceph-mon[32238]: from='client.? 172.21.15.82:0/2842599068' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T16:11:39.385 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:38 smithi184 ceph-mon[36209]: Deploying daemon mds.a on smithi082 2024-05-20T16:11:39.386 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:38 smithi184 ceph-mon[36209]: mds.? [v2:172.21.15.82:6838/2118233541,v1:172.21.15.82:6839/2118233541] up:boot 2024-05-20T16:11:39.386 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:38 smithi184 ceph-mon[36209]: fsmap 10 up:standby 2024-05-20T16:11:39.386 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:38 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-20T16:11:39.386 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:38 smithi184 ceph-mon[36209]: from='client.? 172.21.15.82:0/2842599068' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T16:11:39.859 INFO:journalctl@ceph.mds.a.smithi082.stdout:May 20 16:11:39 smithi082 systemd[1]: Starting Ceph mds.a for 59d1600a-16c2-11ef-bc9a-c7b262605968... 2024-05-20T16:11:40.254 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:39 smithi082 ceph-mon[32238]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-20T16:11:40.254 INFO:journalctl@ceph.mds.a.smithi082.stdout:May 20 16:11:39 smithi082 podman[73397]: 2024-05-20 16:11:39.890388618 +0000 UTC m=+0.017361234 image pull cfcc61b2d476f611479310eb042aa05f11dd5f60dbb65e3b0d7e0e11d024cdb3 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708 2024-05-20T16:11:40.254 INFO:journalctl@ceph.mds.a.smithi082.stdout:May 20 16:11:40 smithi082 podman[73397]: 2024-05-20 16:11:40.001816883 +0000 UTC m=+0.128789488 container create d27456f6239e54ef86124f6ffd278195f9b281d45c270422b0a0a61d4bf716ae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-a, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240514, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-05-20T16:11:40.254 INFO:journalctl@ceph.mds.a.smithi082.stdout:May 20 16:11:40 smithi082 podman[73397]: 2024-05-20 16:11:40.255159907 +0000 UTC m=+0.382132511 container init d27456f6239e54ef86124f6ffd278195f9b281d45c270422b0a0a61d4bf716ae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-a, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240514, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-05-20T16:11:40.312 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:39 smithi160 ceph-mon[36347]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-20T16:11:40.385 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:39 smithi184 ceph-mon[36209]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-20T16:11:40.609 INFO:journalctl@ceph.mds.a.smithi082.stdout:May 20 16:11:40 smithi082 podman[73397]: 2024-05-20 16:11:40.257984158 +0000 UTC m=+0.384956777 container start d27456f6239e54ef86124f6ffd278195f9b281d45c270422b0a0a61d4bf716ae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-a, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240514, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-20T16:11:40.609 INFO:journalctl@ceph.mds.a.smithi082.stdout:May 20 16:11:40 smithi082 ceph-mds[73416]: -- 172.21.15.82:0/1377315965 <== mon.0 v2:172.21.15.82:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56475f64fa40 con 0x56475f6c7180 2024-05-20T16:11:40.609 INFO:journalctl@ceph.mds.a.smithi082.stdout:May 20 16:11:40 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-a[73412]: starting mds.a at 2024-05-20T16:11:40.609 INFO:journalctl@ceph.mds.a.smithi082.stdout:May 20 16:11:40 smithi082 bash[73397]: d27456f6239e54ef86124f6ffd278195f9b281d45c270422b0a0a61d4bf716ae 2024-05-20T16:11:40.609 INFO:journalctl@ceph.mds.a.smithi082.stdout:May 20 16:11:40 smithi082 systemd[1]: Started Ceph mds.a for 59d1600a-16c2-11ef-bc9a-c7b262605968. 2024-05-20T16:11:40.713 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-20T16:11:40.828 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:40.828+0000 7febcd6e2640 1 -- 172.21.15.82:0/3943874152 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7febc8101990 msgr2=0x7febc8102690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:11:40.828 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:40.828+0000 7febcd6e2640 1 --2- 172.21.15.82:0/3943874152 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7febc8101990 0x7febc8102690 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7febb8009900 tx=0x7febb802a410 comp rx=0 tx=0).stop 2024-05-20T16:11:40.828 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:40.828+0000 7febcd6e2640 1 -- 172.21.15.82:0/3943874152 shutdown_connections 2024-05-20T16:11:40.828 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:40.828+0000 7febcd6e2640 1 --2- 172.21.15.82:0/3943874152 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7febc81078d0 0x7febc8109cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:40.828 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:40.828+0000 7febcd6e2640 1 --2- 172.21.15.82:0/3943874152 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7febc8102bd0 0x7febc8102fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:40.828 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:40.828+0000 7febcd6e2640 1 --2- 172.21.15.82:0/3943874152 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7febc8101990 0x7febc8102690 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:40.828 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:40.828+0000 7febcd6e2640 1 -- 172.21.15.82:0/3943874152 >> 172.21.15.82:0/3943874152 conn(0x7febc80fcb80 msgr2=0x7febc80ff030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:11:40.828 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:40.828+0000 7febcd6e2640 1 -- 172.21.15.82:0/3943874152 shutdown_connections 2024-05-20T16:11:40.828 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:40.828+0000 7febcd6e2640 1 -- 172.21.15.82:0/3943874152 wait complete. 2024-05-20T16:11:40.831 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:40.829+0000 7febcd6e2640 1 Processor -- start 2024-05-20T16:11:40.831 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:40.829+0000 7febcd6e2640 1 -- start start 2024-05-20T16:11:40.831 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:40.829+0000 7febcd6e2640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7febc8101990 0x7febc8198540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:11:40.831 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:40.829+0000 7febcd6e2640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7febc8102bd0 0x7febc8198a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:11:40.831 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:40.829+0000 7febcd6e2640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7febc81078d0 0x7febc819f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:11:40.831 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:40.829+0000 7febcd6e2640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7febc810d5e0 con 0x7febc8101990 2024-05-20T16:11:40.832 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:40.829+0000 7febcd6e2640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7febc810d460 con 0x7febc8102bd0 2024-05-20T16:11:40.832 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:40.829+0000 7febcd6e2640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7febc810d760 con 0x7febc81078d0 2024-05-20T16:11:40.832 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:40.829+0000 7febc6ffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7febc8101990 0x7febc8198540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:11:40.832 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:40.829+0000 7febc6ffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7febc8101990 0x7febc8198540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:52348/0 (socket says 172.21.15.82:52348) 2024-05-20T16:11:40.832 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:40.829+0000 7febc6ffd640 1 -- 172.21.15.82:0/636089298 learned_addr learned my addr 172.21.15.82:0/636089298 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:11:40.832 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:40.829+0000 7febbffff640 1 --2- 172.21.15.82:0/636089298 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7febc8102bd0 0x7febc8198a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:11:40.832 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:40.829+0000 7febc6ffd640 1 -- 172.21.15.82:0/636089298 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7febc81078d0 msgr2=0x7febc819f9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:11:40.832 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:40.829+0000 7febc77fe640 1 --2- 172.21.15.82:0/636089298 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7febc81078d0 0x7febc819f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:11:40.832 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:40.829+0000 7febc6ffd640 1 --2- 172.21.15.82:0/636089298 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7febc81078d0 0x7febc819f9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:40.832 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:40.829+0000 7febc6ffd640 1 -- 172.21.15.82:0/636089298 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7febc8102bd0 msgr2=0x7febc8198a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:11:40.832 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:40.829+0000 7febc6ffd640 1 --2- 172.21.15.82:0/636089298 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7febc8102bd0 0x7febc8198a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:40.832 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:40.829+0000 7febc6ffd640 1 -- 172.21.15.82:0/636089298 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febc81a0010 con 0x7febc8101990 2024-05-20T16:11:40.832 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:40.829+0000 7febbffff640 1 --2- 172.21.15.82:0/636089298 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7febc8102bd0 0x7febc8198a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T16:11:40.833 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:40.829+0000 7febc6ffd640 1 --2- 172.21.15.82:0/636089298 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7febc8101990 0x7febc8198540 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7febb8006010 tx=0x7febb80043b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:11:40.833 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:40.830+0000 7febc4ff9640 1 -- 172.21.15.82:0/636089298 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febb8046070 con 0x7febc8101990 2024-05-20T16:11:40.833 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:40.830+0000 7febc4ff9640 1 -- 172.21.15.82:0/636089298 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7febb8003ce0 con 0x7febc8101990 2024-05-20T16:11:40.833 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:40.830+0000 7febc4ff9640 1 -- 172.21.15.82:0/636089298 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febb8035050 con 0x7febc8101990 2024-05-20T16:11:40.833 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:40.830+0000 7febcd6e2640 1 -- 172.21.15.82:0/636089298 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7febc806a1a0 con 0x7febc8101990 2024-05-20T16:11:40.833 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:40.830+0000 7febcd6e2640 1 -- 172.21.15.82:0/636089298 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7febc806a6e0 con 0x7febc8101990 2024-05-20T16:11:40.833 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:40.831+0000 7febcd6e2640 1 -- 172.21.15.82:0/636089298 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7febc80682c0 con 0x7febc8101990 2024-05-20T16:11:40.834 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:40.832+0000 7febc4ff9640 1 -- 172.21.15.82:0/636089298 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7febb8049070 con 0x7febc8101990 2024-05-20T16:11:40.834 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:40.833+0000 7febc4ff9640 1 --2- 172.21.15.82:0/636089298 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7febb40a2c10 0x7febb40a5120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:11:40.834 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:40.833+0000 7febbffff640 1 --2- 172.21.15.82:0/636089298 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7febb40a2c10 0x7febb40a5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:11:40.834 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:40.833+0000 7febc4ff9640 1 -- 172.21.15.82:0/636089298 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7febb80f0980 con 0x7febc8101990 2024-05-20T16:11:40.834 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:40.833+0000 7febbffff640 1 --2- 172.21.15.82:0/636089298 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7febb40a2c10 0x7febb40a5120 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7febb0005fd0 tx=0x7febb0005ea0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:11:40.837 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:40.837+0000 7febc4ff9640 1 -- 172.21.15.82:0/636089298 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7febb807d850 con 0x7febc8101990 2024-05-20T16:11:41.055 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:41.055+0000 7febcd6e2640 1 -- 172.21.15.82:0/636089298 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7febc8106290 con 0x7febc8101990 2024-05-20T16:11:41.056 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:41.056+0000 7febc4ff9640 1 -- 172.21.15.82:0/636089298 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9659 (secure 0 0 0) 0x7febb80bdd80 con 0x7febc8101990 2024-05-20T16:11:41.056 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-05-20T16:11:41.056 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/2044534217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":2044534217},{"type":"v1","addr":"172.21.15.82:6835","nonce":2044534217}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/2635635097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":2635635097},{"type":"v1","addr":"172.21.15.82:6837","nonce":2635635097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14625,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6839/2118233541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6838","nonce":2118233541},{"type":"v1","addr":"172.21.15.82:6839","nonce":2118233541}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14637,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6841/2119517973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6840","nonce":2119517973},{"type":"v1","addr":"172.21.15.82:6841","nonce":2119517973}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6833/4015144007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":4015144007},{"type":"v1","addr":"172.21.15.184:6833","nonce":4015144007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6833/975243372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6832","nonce":975243372},{"type":"v1","addr":"172.21.15.160:6833","nonce":975243372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24470,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6835/664068363","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":664068363},{"type":"v1","addr":"172.21.15.160:6835","nonce":664068363}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24478,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/4242156090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":4242156090},{"type":"v1","addr":"172.21.15.184:6835","nonce":4242156090}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24496,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/403262242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":403262242},{"type":"v1","addr":"172.21.15.184:6837","nonce":403262242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24506,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6837/2846188680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6836","nonce":2846188680},{"type":"v1","addr":"172.21.15.160:6837","nonce":2846188680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24511,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6839/2199955067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":2199955067},{"type":"v1","addr":"172.21.15.184:6839","nonce":2199955067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-05-20T16:11:41.056 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 12 2024-05-20T16:11:41.058 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:41.058+0000 7febcd6e2640 1 -- 172.21.15.82:0/636089298 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7febb40a2c10 msgr2=0x7febb40a5120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:11:41.058 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:41.058+0000 7febcd6e2640 1 --2- 172.21.15.82:0/636089298 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7febb40a2c10 0x7febb40a5120 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7febb0005fd0 tx=0x7febb0005ea0 comp rx=0 tx=0).stop 2024-05-20T16:11:41.058 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:41.058+0000 7febcd6e2640 1 -- 172.21.15.82:0/636089298 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7febc8101990 msgr2=0x7febc8198540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:11:41.058 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:41.058+0000 7febcd6e2640 1 --2- 172.21.15.82:0/636089298 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7febc8101990 0x7febc8198540 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7febb8006010 tx=0x7febb80043b0 comp rx=0 tx=0).stop 2024-05-20T16:11:41.058 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:41.058+0000 7febcd6e2640 1 -- 172.21.15.82:0/636089298 shutdown_connections 2024-05-20T16:11:41.058 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:41.058+0000 7febcd6e2640 1 --2- 172.21.15.82:0/636089298 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7febb40a2c10 0x7febb40a5120 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:41.059 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:41.058+0000 7febcd6e2640 1 --2- 172.21.15.82:0/636089298 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7febc81078d0 0x7febc819f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:41.059 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:41.058+0000 7febcd6e2640 1 --2- 172.21.15.82:0/636089298 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7febc8102bd0 0x7febc8198a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:41.059 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:41.058+0000 7febcd6e2640 1 --2- 172.21.15.82:0/636089298 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7febc8101990 0x7febc8198540 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:41.059 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:41.058+0000 7febcd6e2640 1 -- 172.21.15.82:0/636089298 >> 172.21.15.82:0/636089298 conn(0x7febc80fcb80 msgr2=0x7febc81038b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:11:41.059 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:41.058+0000 7febcd6e2640 1 -- 172.21.15.82:0/636089298 shutdown_connections 2024-05-20T16:11:41.059 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:41.058+0000 7febcd6e2640 1 -- 172.21.15.82:0/636089298 wait complete. 2024-05-20T16:11:41.804 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:41 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:11:41.804 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:41 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:11:41.804 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:41 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:11:41.804 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:41 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-20T16:11:41.804 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:41 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-20T16:11:41.804 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:41 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:11:41.804 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:41 smithi160 ceph-mon[36347]: Deploying daemon mds.b on smithi160 2024-05-20T16:11:41.804 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:41 smithi160 ceph-mon[36347]: mds.? [v2:172.21.15.82:6840/2119517973,v1:172.21.15.82:6841/2119517973] up:boot 2024-05-20T16:11:41.804 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:41 smithi160 ceph-mon[36347]: fsmap 11 up:standby 2024-05-20T16:11:41.804 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:41 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-20T16:11:41.804 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:41 smithi160 ceph-mon[36347]: from='client.? 172.21.15.82:0/636089298' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T16:11:41.805 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:41 smithi160 ceph-mon[36347]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-20T16:11:41.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:41 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:11:41.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:41 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:11:41.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:41 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:11:41.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:41 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-20T16:11:41.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:41 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-20T16:11:41.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:41 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:11:41.860 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:41 smithi082 ceph-mon[32238]: Deploying daemon mds.b on smithi160 2024-05-20T16:11:41.860 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:41 smithi082 ceph-mon[32238]: mds.? [v2:172.21.15.82:6840/2119517973,v1:172.21.15.82:6841/2119517973] up:boot 2024-05-20T16:11:41.860 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:41 smithi082 ceph-mon[32238]: fsmap 11 up:standby 2024-05-20T16:11:41.860 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:41 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-20T16:11:41.860 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:41 smithi082 ceph-mon[32238]: from='client.? 172.21.15.82:0/636089298' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T16:11:41.860 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:41 smithi082 ceph-mon[32238]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-20T16:11:41.886 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:41 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:11:41.886 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:41 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:11:41.886 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:41 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:11:41.886 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:41 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-20T16:11:41.886 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:41 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-20T16:11:41.886 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:41 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:11:41.886 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:41 smithi184 ceph-mon[36209]: Deploying daemon mds.b on smithi160 2024-05-20T16:11:41.886 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:41 smithi184 ceph-mon[36209]: mds.? [v2:172.21.15.82:6840/2119517973,v1:172.21.15.82:6841/2119517973] up:boot 2024-05-20T16:11:41.886 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:41 smithi184 ceph-mon[36209]: fsmap 11 up:standby 2024-05-20T16:11:41.886 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:41 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-20T16:11:41.886 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:41 smithi184 ceph-mon[36209]: from='client.? 172.21.15.82:0/636089298' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T16:11:41.886 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:41 smithi184 ceph-mon[36209]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-20T16:11:42.812 INFO:journalctl@ceph.mds.b.smithi160.stdout:May 20 16:11:42 smithi160 systemd[1]: Starting Ceph mds.b for 59d1600a-16c2-11ef-bc9a-c7b262605968... 2024-05-20T16:11:43.071 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-20T16:11:43.197 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:43.197+0000 7f7e1e0a2640 1 -- 172.21.15.82:0/3533795737 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7e18103cf0 msgr2=0x7f7e18104100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:11:43.197 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:43.197+0000 7f7e1e0a2640 1 --2- 172.21.15.82:0/3533795737 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7e18103cf0 0x7f7e18104100 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f7e00009900 tx=0x7f7e0002a410 comp rx=0 tx=0).stop 2024-05-20T16:11:43.197 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:43.197+0000 7f7e1e0a2640 1 -- 172.21.15.82:0/3533795737 shutdown_connections 2024-05-20T16:11:43.197 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:43.197+0000 7f7e1e0a2640 1 --2- 172.21.15.82:0/3533795737 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7e18103cf0 0x7f7e18104100 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:43.197 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:43.197+0000 7f7e1e0a2640 1 --2- 172.21.15.82:0/3533795737 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7e181089f0 0x7f7e1810ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:43.197 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:43.197+0000 7f7e1e0a2640 1 --2- 172.21.15.82:0/3533795737 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7e18102ab0 0x7f7e181037b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:43.197 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:43.197+0000 7f7e1e0a2640 1 -- 172.21.15.82:0/3533795737 >> 172.21.15.82:0/3533795737 conn(0x7f7e180fdcc0 msgr2=0x7f7e18100150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:11:43.197 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:43.197+0000 7f7e1e0a2640 1 -- 172.21.15.82:0/3533795737 shutdown_connections 2024-05-20T16:11:43.198 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:43.197+0000 7f7e1e0a2640 1 -- 172.21.15.82:0/3533795737 wait complete. 2024-05-20T16:11:43.198 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:43.197+0000 7f7e1e0a2640 1 Processor -- start 2024-05-20T16:11:43.200 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:43.198+0000 7f7e1e0a2640 1 -- start start 2024-05-20T16:11:43.200 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:43.198+0000 7f7e1e0a2640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7e18102ab0 0x7f7e18199380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:11:43.200 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:43.198+0000 7f7e1e0a2640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7e18103cf0 0x7f7e181998c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:11:43.200 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:43.198+0000 7f7e1e0a2640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7e181089f0 0x7f7e181a07e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:11:43.200 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:43.198+0000 7f7e1e0a2640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f7e1810e770 con 0x7f7e181089f0 2024-05-20T16:11:43.200 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:43.198+0000 7f7e1e0a2640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f7e1810e5f0 con 0x7f7e18103cf0 2024-05-20T16:11:43.200 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:43.198+0000 7f7e1e0a2640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f7e1810e8f0 con 0x7f7e18102ab0 2024-05-20T16:11:43.200 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:43.198+0000 7f7e17fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7e181089f0 0x7f7e181a07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:11:43.201 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:43.198+0000 7f7e17fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7e181089f0 0x7f7e181a07e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:52360/0 (socket says 172.21.15.82:52360) 2024-05-20T16:11:43.201 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:43.198+0000 7f7e17fff640 1 -- 172.21.15.82:0/2920698104 learned_addr learned my addr 172.21.15.82:0/2920698104 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:11:43.201 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:43.198+0000 7f7e177fe640 1 --2- 172.21.15.82:0/2920698104 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7e18102ab0 0x7f7e18199380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:11:43.201 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:43.198+0000 7f7e16ffd640 1 --2- 172.21.15.82:0/2920698104 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7e18103cf0 0x7f7e181998c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:11:43.201 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:43.198+0000 7f7e17fff640 1 -- 172.21.15.82:0/2920698104 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7e18102ab0 msgr2=0x7f7e18199380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:11:43.201 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:43.198+0000 7f7e17fff640 1 --2- 172.21.15.82:0/2920698104 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7e18102ab0 0x7f7e18199380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:43.201 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:43.198+0000 7f7e17fff640 1 -- 172.21.15.82:0/2920698104 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7e18103cf0 msgr2=0x7f7e181998c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:11:43.201 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:43.198+0000 7f7e17fff640 1 --2- 172.21.15.82:0/2920698104 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7e18103cf0 0x7f7e181998c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:43.201 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:43.198+0000 7f7e17fff640 1 -- 172.21.15.82:0/2920698104 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e181a0e50 con 0x7f7e181089f0 2024-05-20T16:11:43.201 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:43.198+0000 7f7e16ffd640 1 --2- 172.21.15.82:0/2920698104 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7e18103cf0 0x7f7e181998c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T16:11:43.201 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:43.198+0000 7f7e17fff640 1 --2- 172.21.15.82:0/2920698104 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7e181089f0 0x7f7e181a07e0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f7e10009fd0 tx=0x7f7e1000b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:11:43.201 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:43.199+0000 7f7e14ff9640 1 -- 172.21.15.82:0/2920698104 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e10017070 con 0x7f7e181089f0 2024-05-20T16:11:43.201 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:43.199+0000 7f7e14ff9640 1 -- 172.21.15.82:0/2920698104 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7e10003e50 con 0x7f7e181089f0 2024-05-20T16:11:43.202 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:43.199+0000 7f7e1e0a2640 1 -- 172.21.15.82:0/2920698104 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e181a1140 con 0x7f7e181089f0 2024-05-20T16:11:43.202 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:43.199+0000 7f7e14ff9640 1 -- 172.21.15.82:0/2920698104 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e100106c0 con 0x7f7e181089f0 2024-05-20T16:11:43.202 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:43.199+0000 7f7e1e0a2640 1 -- 172.21.15.82:0/2920698104 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e181a1640 con 0x7f7e181089f0 2024-05-20T16:11:43.202 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:43.200+0000 7f7e1e0a2640 1 -- 172.21.15.82:0/2920698104 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e18107800 con 0x7f7e181089f0 2024-05-20T16:11:43.202 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:43.201+0000 7f7e14ff9640 1 -- 172.21.15.82:0/2920698104 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f7e10003970 con 0x7f7e181089f0 2024-05-20T16:11:43.203 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:43.202+0000 7f7e14ff9640 1 --2- 172.21.15.82:0/2920698104 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f7e040a2b90 0x7f7e040a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:11:43.203 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:43.202+0000 7f7e177fe640 1 --2- 172.21.15.82:0/2920698104 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f7e040a2b90 0x7f7e040a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:11:43.203 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:43.202+0000 7f7e14ff9640 1 -- 172.21.15.82:0/2920698104 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f7e100ce930 con 0x7f7e181089f0 2024-05-20T16:11:43.203 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:43.202+0000 7f7e177fe640 1 --2- 172.21.15.82:0/2920698104 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f7e040a2b90 0x7f7e040a50a0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f7e181036b0 tx=0x7f7e08005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:11:43.205 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:43.205+0000 7f7e14ff9640 1 -- 172.21.15.82:0/2920698104 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f7e1009bd30 con 0x7f7e181089f0 2024-05-20T16:11:43.210 INFO:journalctl@ceph.mds.b.smithi160.stdout:May 20 16:11:42 smithi160 podman[66929]: 2024-05-20 16:11:42.812340472 +0000 UTC m=+0.017157714 image pull cfcc61b2d476f611479310eb042aa05f11dd5f60dbb65e3b0d7e0e11d024cdb3 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708 2024-05-20T16:11:43.210 INFO:journalctl@ceph.mds.b.smithi160.stdout:May 20 16:11:42 smithi160 podman[66929]: 2024-05-20 16:11:42.941110338 +0000 UTC m=+0.145927589 container create adc717c8bfc17087d9177d0c6e21018a2d2162505b682db9843921ede41dd960 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240514, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-20T16:11:43.210 INFO:journalctl@ceph.mds.b.smithi160.stdout:May 20 16:11:43 smithi160 podman[66929]: 2024-05-20 16:11:43.210905288 +0000 UTC m=+0.415722523 container init adc717c8bfc17087d9177d0c6e21018a2d2162505b682db9843921ede41dd960 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-b, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240514, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-20T16:11:43.438 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:43.438+0000 7f7e1e0a2640 1 -- 172.21.15.82:0/2920698104 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7e1802d2a0 con 0x7f7e181089f0 2024-05-20T16:11:43.438 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:43.438+0000 7f7e14ff9640 1 -- 172.21.15.82:0/2920698104 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10488 (secure 0 0 0) 0x7f7e1009b630 con 0x7f7e181089f0 2024-05-20T16:11:43.439 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-05-20T16:11:43.439 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/2044534217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":2044534217},{"type":"v1","addr":"172.21.15.82:6835","nonce":2044534217}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/2635635097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":2635635097},{"type":"v1","addr":"172.21.15.82:6837","nonce":2635635097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14625,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6839/2118233541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6838","nonce":2118233541},{"type":"v1","addr":"172.21.15.82:6839","nonce":2118233541}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14637,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6841/2119517973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6840","nonce":2119517973},{"type":"v1","addr":"172.21.15.82:6841","nonce":2119517973}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6833/4015144007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":4015144007},{"type":"v1","addr":"172.21.15.184:6833","nonce":4015144007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6833/975243372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6832","nonce":975243372},{"type":"v1","addr":"172.21.15.160:6833","nonce":975243372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24470,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6835/664068363","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":664068363},{"type":"v1","addr":"172.21.15.160:6835","nonce":664068363}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24478,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/4242156090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":4242156090},{"type":"v1","addr":"172.21.15.184:6835","nonce":4242156090}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24496,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/403262242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":403262242},{"type":"v1","addr":"172.21.15.184:6837","nonce":403262242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24506,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6837/2846188680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6836","nonce":2846188680},{"type":"v1","addr":"172.21.15.160:6837","nonce":2846188680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24511,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6839/2199955067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":2199955067},{"type":"v1","addr":"172.21.15.184:6839","nonce":2199955067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24548,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6839/2765554482","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6838","nonce":2765554482},{"type":"v1","addr":"172.21.15.160:6839","nonce":2765554482}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-05-20T16:11:43.439 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 13 2024-05-20T16:11:43.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:43.441+0000 7f7e1e0a2640 1 -- 172.21.15.82:0/2920698104 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f7e040a2b90 msgr2=0x7f7e040a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:11:43.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:43.441+0000 7f7e1e0a2640 1 --2- 172.21.15.82:0/2920698104 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f7e040a2b90 0x7f7e040a50a0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f7e181036b0 tx=0x7f7e08005f90 comp rx=0 tx=0).stop 2024-05-20T16:11:43.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:43.441+0000 7f7e1e0a2640 1 -- 172.21.15.82:0/2920698104 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7e181089f0 msgr2=0x7f7e181a07e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:11:43.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:43.441+0000 7f7e1e0a2640 1 --2- 172.21.15.82:0/2920698104 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7e181089f0 0x7f7e181a07e0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f7e10009fd0 tx=0x7f7e1000b260 comp rx=0 tx=0).stop 2024-05-20T16:11:43.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:43.441+0000 7f7e1e0a2640 1 -- 172.21.15.82:0/2920698104 shutdown_connections 2024-05-20T16:11:43.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:43.441+0000 7f7e1e0a2640 1 --2- 172.21.15.82:0/2920698104 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f7e040a2b90 0x7f7e040a50a0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:43.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:43.441+0000 7f7e1e0a2640 1 --2- 172.21.15.82:0/2920698104 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7e181089f0 0x7f7e181a07e0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:43.442 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:43.441+0000 7f7e1e0a2640 1 --2- 172.21.15.82:0/2920698104 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7e18103cf0 0x7f7e181998c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:43.442 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:43.441+0000 7f7e1e0a2640 1 --2- 172.21.15.82:0/2920698104 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7e18102ab0 0x7f7e18199380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:43.442 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:43.441+0000 7f7e1e0a2640 1 -- 172.21.15.82:0/2920698104 >> 172.21.15.82:0/2920698104 conn(0x7f7e180fdcc0 msgr2=0x7f7e181049d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:11:43.442 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:43.441+0000 7f7e1e0a2640 1 -- 172.21.15.82:0/2920698104 shutdown_connections 2024-05-20T16:11:43.442 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:43.441+0000 7f7e1e0a2640 1 -- 172.21.15.82:0/2920698104 wait complete. 2024-05-20T16:11:43.453 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-20T16:11:43.562 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:43 smithi160 ceph-mon[36347]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-20T16:11:43.562 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:43 smithi160 ceph-mon[36347]: mds.? [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] up:boot 2024-05-20T16:11:43.562 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:43 smithi160 ceph-mon[36347]: fsmap 12 up:standby 2024-05-20T16:11:43.562 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:43 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-20T16:11:43.563 INFO:journalctl@ceph.mds.b.smithi160.stdout:May 20 16:11:43 smithi160 podman[66929]: 2024-05-20 16:11:43.213480386 +0000 UTC m=+0.418297633 container start adc717c8bfc17087d9177d0c6e21018a2d2162505b682db9843921ede41dd960 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-b, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240514, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-05-20T16:11:43.563 INFO:journalctl@ceph.mds.b.smithi160.stdout:May 20 16:11:43 smithi160 ceph-mds[66948]: -- 172.21.15.160:0/2677070440 <== mon.2 v2:172.21.15.160:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x563b40cffa40 con 0x563b40d77600 2024-05-20T16:11:43.563 INFO:journalctl@ceph.mds.b.smithi160.stdout:May 20 16:11:43 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-b[66944]: starting mds.b at 2024-05-20T16:11:43.563 INFO:journalctl@ceph.mds.b.smithi160.stdout:May 20 16:11:43 smithi160 bash[66929]: adc717c8bfc17087d9177d0c6e21018a2d2162505b682db9843921ede41dd960 2024-05-20T16:11:43.563 INFO:journalctl@ceph.mds.b.smithi160.stdout:May 20 16:11:43 smithi160 systemd[1]: Started Ceph mds.b for 59d1600a-16c2-11ef-bc9a-c7b262605968. 2024-05-20T16:11:43.572 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:43.572+0000 7fa829bc2640 1 -- 172.21.15.82:0/3661473919 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa824101970 msgr2=0x7fa824102670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:11:43.572 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:43.572+0000 7fa829bc2640 1 --2- 172.21.15.82:0/3661473919 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa824101970 0x7fa824102670 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fa814009900 tx=0x7fa81402a410 comp rx=0 tx=0).stop 2024-05-20T16:11:43.572 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:43.572+0000 7fa829bc2640 1 -- 172.21.15.82:0/3661473919 shutdown_connections 2024-05-20T16:11:43.572 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:43.572+0000 7fa829bc2640 1 --2- 172.21.15.82:0/3661473919 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa8241078b0 0x7fa824109cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:43.573 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:43.572+0000 7fa829bc2640 1 --2- 172.21.15.82:0/3661473919 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa824102bb0 0x7fa824102fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:43.573 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:43.572+0000 7fa829bc2640 1 --2- 172.21.15.82:0/3661473919 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa824101970 0x7fa824102670 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:43.573 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:43.572+0000 7fa829bc2640 1 -- 172.21.15.82:0/3661473919 >> 172.21.15.82:0/3661473919 conn(0x7fa8240fcb80 msgr2=0x7fa8240ff010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:11:43.573 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:43.572+0000 7fa829bc2640 1 -- 172.21.15.82:0/3661473919 shutdown_connections 2024-05-20T16:11:43.573 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:43.572+0000 7fa829bc2640 1 -- 172.21.15.82:0/3661473919 wait complete. 2024-05-20T16:11:43.575 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:43.573+0000 7fa829bc2640 1 Processor -- start 2024-05-20T16:11:43.575 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:43.573+0000 7fa829bc2640 1 -- start start 2024-05-20T16:11:43.575 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:43.573+0000 7fa829bc2640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa824101970 0x7fa8241981c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:11:43.575 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:43.573+0000 7fa829bc2640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa824102bb0 0x7fa824198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:11:43.575 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:43.573+0000 7fa829bc2640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa8241078b0 0x7fa82419f690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:11:43.576 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:43.573+0000 7fa829bc2640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fa82410d5c0 con 0x7fa8241078b0 2024-05-20T16:11:43.576 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:43.573+0000 7fa829bc2640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fa82410d440 con 0x7fa824101970 2024-05-20T16:11:43.576 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:43.573+0000 7fa829bc2640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa82410d740 con 0x7fa824102bb0 2024-05-20T16:11:43.576 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:43.573+0000 7fa823fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa8241078b0 0x7fa82419f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:11:43.576 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:43.573+0000 7fa823fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa8241078b0 0x7fa82419f690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:52382/0 (socket says 172.21.15.82:52382) 2024-05-20T16:11:43.576 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:43.573+0000 7fa823fff640 1 -- 172.21.15.82:0/2116278004 learned_addr learned my addr 172.21.15.82:0/2116278004 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:11:43.576 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:43.573+0000 7fa8237fe640 1 --2- 172.21.15.82:0/2116278004 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa824101970 0x7fa8241981c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:11:43.576 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:43.573+0000 7fa81bfff640 1 --2- 172.21.15.82:0/2116278004 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa824102bb0 0x7fa824198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:11:43.576 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:43.574+0000 7fa823fff640 1 -- 172.21.15.82:0/2116278004 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa824102bb0 msgr2=0x7fa824198700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:11:43.576 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:43.574+0000 7fa823fff640 1 --2- 172.21.15.82:0/2116278004 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa824102bb0 0x7fa824198700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:43.576 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:43.574+0000 7fa823fff640 1 -- 172.21.15.82:0/2116278004 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa824101970 msgr2=0x7fa8241981c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:11:43.576 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:43.574+0000 7fa823fff640 1 --2- 172.21.15.82:0/2116278004 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa824101970 0x7fa8241981c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:43.576 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:43.574+0000 7fa823fff640 1 -- 172.21.15.82:0/2116278004 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa82419fc50 con 0x7fa8241078b0 2024-05-20T16:11:43.576 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:43.574+0000 7fa823fff640 1 --2- 172.21.15.82:0/2116278004 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa8241078b0 0x7fa82419f690 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fa81c00cce0 tx=0x7fa81c00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:11:43.577 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:43.574+0000 7fa8217fa640 1 -- 172.21.15.82:0/2116278004 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa81c00f450 con 0x7fa8241078b0 2024-05-20T16:11:43.577 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:43.574+0000 7fa8217fa640 1 -- 172.21.15.82:0/2116278004 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa81c00fad0 con 0x7fa8241078b0 2024-05-20T16:11:43.577 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:43.574+0000 7fa8217fa640 1 -- 172.21.15.82:0/2116278004 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa81c018c10 con 0x7fa8241078b0 2024-05-20T16:11:43.577 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:43.574+0000 7fa829bc2640 1 -- 172.21.15.82:0/2116278004 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa82419ff40 con 0x7fa8241078b0 2024-05-20T16:11:43.577 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:43.574+0000 7fa829bc2640 1 -- 172.21.15.82:0/2116278004 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8241a03a0 con 0x7fa8241078b0 2024-05-20T16:11:43.577 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:43.575+0000 7fa829bc2640 1 -- 172.21.15.82:0/2116278004 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8241066c0 con 0x7fa8241078b0 2024-05-20T16:11:43.578 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:43.576+0000 7fa8217fa640 1 -- 172.21.15.82:0/2116278004 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fa81c00f5f0 con 0x7fa8241078b0 2024-05-20T16:11:43.578 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:43.577+0000 7fa8217fa640 1 --2- 172.21.15.82:0/2116278004 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fa8100a2c10 0x7fa8100a5120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:11:43.578 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:43.577+0000 7fa8237fe640 1 --2- 172.21.15.82:0/2116278004 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fa8100a2c10 0x7fa8100a5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:11:43.578 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:43.577+0000 7fa8217fa640 1 -- 172.21.15.82:0/2116278004 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fa81c0cf160 con 0x7fa8241078b0 2024-05-20T16:11:43.578 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:43.577+0000 7fa8237fe640 1 --2- 172.21.15.82:0/2116278004 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fa8100a2c10 0x7fa8100a5120 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fa814005ab0 tx=0x7fa814031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:11:43.581 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:43.581+0000 7fa8217fa640 1 -- 172.21.15.82:0/2116278004 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fa81c014030 con 0x7fa8241078b0 2024-05-20T16:11:43.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:43 smithi082 ceph-mon[32238]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-20T16:11:43.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:43 smithi082 ceph-mon[32238]: mds.? [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] up:boot 2024-05-20T16:11:43.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:43 smithi082 ceph-mon[32238]: fsmap 12 up:standby 2024-05-20T16:11:43.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:43 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-20T16:11:43.635 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:43 smithi184 ceph-mon[36209]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-20T16:11:43.635 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:43 smithi184 ceph-mon[36209]: mds.? [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] up:boot 2024-05-20T16:11:43.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:43 smithi184 ceph-mon[36209]: fsmap 12 up:standby 2024-05-20T16:11:43.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:43 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-20T16:11:43.799 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:43.799+0000 7fa829bc2640 1 -- 172.21.15.82:0/2116278004 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa82402d4c0 con 0x7fa8241078b0 2024-05-20T16:11:43.800 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:43.800+0000 7fa8217fa640 1 -- 172.21.15.82:0/2116278004 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+18481 (secure 0 0 0) 0x7fa81c09c560 con 0x7fa8241078b0 2024-05-20T16:11:43.801 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-05-20T16:11:43.801 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":82,"fsid":"59d1600a-16c2-11ef-bc9a-c7b262605968","created":"2024-05-20T16:03:46.611556+0000","modified":"2024-05-20T16:11:00.310642+0000","last_up_change":"2024-05-20T16:10:59.303479+0000","last_in_change":"2024-05-20T16:10:39.063561+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-20T16:07:09.493727+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bae842a9-03ec-490e-b56d-f7957235cdc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":1484330167},{"type":"v1","addr":"172.21.15.82:6803","nonce":1484330167}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":1484330167},{"type":"v1","addr":"172.21.15.82:6805","nonce":1484330167}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1484330167},{"type":"v1","addr":"172.21.15.82:6809","nonce":1484330167}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":1484330167},{"type":"v1","addr":"172.21.15.82:6807","nonce":1484330167}]},"public_addr":"172.21.15.82:6803/1484330167","cluster_addr":"172.21.15.82:6805/1484330167","heartbeat_back_addr":"172.21.15.82:6809/1484330167","heartbeat_front_addr":"172.21.15.82:6807/1484330167","state":["exists","up"]},{"osd":1,"uuid":"59f6a2ba-b5de-4161-81c5-92a417abacdf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":1457003568},{"type":"v1","addr":"172.21.15.160:6801","nonce":1457003568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":1457003568},{"type":"v1","addr":"172.21.15.160:6803","nonce":1457003568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6806","nonce":1457003568},{"type":"v1","addr":"172.21.15.160:6807","nonce":1457003568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":1457003568},{"type":"v1","addr":"172.21.15.160:6805","nonce":1457003568}]},"public_addr":"172.21.15.160:6801/1457003568","cluster_addr":"172.21.15.160:6803/1457003568","heartbeat_back_addr":"172.21.15.160:6807/1457003568","heartbeat_front_addr":"172.21.15.160:6805/1457003568","state":["exists","up"]},{"osd":2,"uuid":"3bbb5587-b449-41f3-bd26-67ec3d6afdfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":4051293934},{"type":"v1","addr":"172.21.15.184:6801","nonce":4051293934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":4051293934},{"type":"v1","addr":"172.21.15.184:6803","nonce":4051293934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":4051293934},{"type":"v1","addr":"172.21.15.184:6807","nonce":4051293934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":4051293934},{"type":"v1","addr":"172.21.15.184:6805","nonce":4051293934}]},"public_addr":"172.21.15.184:6801/4051293934","cluster_addr":"172.21.15.184:6803/4051293934","heartbeat_back_addr":"172.21.15.184:6807/4051293934","heartbeat_front_addr":"172.21.15.184:6805/4051293934","state":["exists","up"]},{"osd":3,"uuid":"ca651f90-09b8-4081-a6f9-40b31268a837","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":3947485599},{"type":"v1","addr":"172.21.15.82:6811","nonce":3947485599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":3947485599},{"type":"v1","addr":"172.21.15.82:6813","nonce":3947485599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":3947485599},{"type":"v1","addr":"172.21.15.82:6817","nonce":3947485599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":3947485599},{"type":"v1","addr":"172.21.15.82:6815","nonce":3947485599}]},"public_addr":"172.21.15.82:6811/3947485599","cluster_addr":"172.21.15.82:6813/3947485599","heartbeat_back_addr":"172.21.15.82:6817/3947485599","heartbeat_front_addr":"172.21.15.82:6815/3947485599","state":["exists","up"]},{"osd":4,"uuid":"cb3dfd56-f677-40cb-9c96-8319ea5ac788","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":4160830014},{"type":"v1","addr":"172.21.15.160:6809","nonce":4160830014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":4160830014},{"type":"v1","addr":"172.21.15.160:6811","nonce":4160830014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":4160830014},{"type":"v1","addr":"172.21.15.160:6815","nonce":4160830014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6812","nonce":4160830014},{"type":"v1","addr":"172.21.15.160:6813","nonce":4160830014}]},"public_addr":"172.21.15.160:6809/4160830014","cluster_addr":"172.21.15.160:6811/4160830014","heartbeat_back_addr":"172.21.15.160:6815/4160830014","heartbeat_front_addr":"172.21.15.160:6813/4160830014","state":["exists","up"]},{"osd":5,"uuid":"c70cfcd1-a4ff-461f-8c73-479b061200fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":1046282672},{"type":"v1","addr":"172.21.15.184:6809","nonce":1046282672}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":1046282672},{"type":"v1","addr":"172.21.15.184:6811","nonce":1046282672}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":1046282672},{"type":"v1","addr":"172.21.15.184:6815","nonce":1046282672}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":1046282672},{"type":"v1","addr":"172.21.15.184:6813","nonce":1046282672}]},"public_addr":"172.21.15.184:6809/1046282672","cluster_addr":"172.21.15.184:6811/1046282672","heartbeat_back_addr":"172.21.15.184:6815/1046282672","heartbeat_front_addr":"172.21.15.184:6813/1046282672","state":["exists","up"]},{"osd":6,"uuid":"d47ce001-9dd1-4830-bcc9-888601187c6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":2789736499},{"type":"v1","addr":"172.21.15.82:6819","nonce":2789736499}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":2789736499},{"type":"v1","addr":"172.21.15.82:6821","nonce":2789736499}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":2789736499},{"type":"v1","addr":"172.21.15.82:6825","nonce":2789736499}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":2789736499},{"type":"v1","addr":"172.21.15.82:6823","nonce":2789736499}]},"public_addr":"172.21.15.82:6819/2789736499","cluster_addr":"172.21.15.82:6821/2789736499","heartbeat_back_addr":"172.21.15.82:6825/2789736499","heartbeat_front_addr":"172.21.15.82:6823/2789736499","state":["exists","up"]},{"osd":7,"uuid":"25c54f43-18cb-41e4-83b4-2732b5a93889","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6816","nonce":2340683378},{"type":"v1","addr":"172.21.15.160:6817","nonce":2340683378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":2340683378},{"type":"v1","addr":"172.21.15.160:6819","nonce":2340683378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":2340683378},{"type":"v1","addr":"172.21.15.160:6823","nonce":2340683378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6820","nonce":2340683378},{"type":"v1","addr":"172.21.15.160:6821","nonce":2340683378}]},"public_addr":"172.21.15.160:6817/2340683378","cluster_addr":"172.21.15.160:6819/2340683378","heartbeat_back_addr":"172.21.15.160:6823/2340683378","heartbeat_front_addr":"172.21.15.160:6821/2340683378","state":["exists","up"]},{"osd":8,"uuid":"1f8b478d-287f-45f5-8526-d1a58c5bc07c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":3831576372},{"type":"v1","addr":"172.21.15.184:6817","nonce":3831576372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":3831576372},{"type":"v1","addr":"172.21.15.184:6819","nonce":3831576372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":3831576372},{"type":"v1","addr":"172.21.15.184:6823","nonce":3831576372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":3831576372},{"type":"v1","addr":"172.21.15.184:6821","nonce":3831576372}]},"public_addr":"172.21.15.184:6817/3831576372","cluster_addr":"172.21.15.184:6819/3831576372","heartbeat_back_addr":"172.21.15.184:6823/3831576372","heartbeat_front_addr":"172.21.15.184:6821/3831576372","state":["exists","up"]},{"osd":9,"uuid":"0d69e319-954f-4b18-95ea-31794f18dc3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":716330782},{"type":"v1","addr":"172.21.15.82:6827","nonce":716330782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":716330782},{"type":"v1","addr":"172.21.15.82:6829","nonce":716330782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":716330782},{"type":"v1","addr":"172.21.15.82:6833","nonce":716330782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":716330782},{"type":"v1","addr":"172.21.15.82:6831","nonce":716330782}]},"public_addr":"172.21.15.82:6827/716330782","cluster_addr":"172.21.15.82:6829/716330782","heartbeat_back_addr":"172.21.15.82:6833/716330782","heartbeat_front_addr":"172.21.15.82:6831/716330782","state":["exists","up"]},{"osd":10,"uuid":"39dc1b78-61cb-42ca-b5b6-1e44ac2b6500","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":2327428014},{"type":"v1","addr":"172.21.15.160:6825","nonce":2327428014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":2327428014},{"type":"v1","addr":"172.21.15.160:6827","nonce":2327428014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":2327428014},{"type":"v1","addr":"172.21.15.160:6831","nonce":2327428014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":2327428014},{"type":"v1","addr":"172.21.15.160:6829","nonce":2327428014}]},"public_addr":"172.21.15.160:6825/2327428014","cluster_addr":"172.21.15.160:6827/2327428014","heartbeat_back_addr":"172.21.15.160:6831/2327428014","heartbeat_front_addr":"172.21.15.160:6829/2327428014","state":["exists","up"]},{"osd":11,"uuid":"4306b3c1-707a-4010-987c-209ea0e948e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":3594364650},{"type":"v1","addr":"172.21.15.184:6825","nonce":3594364650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":3594364650},{"type":"v1","addr":"172.21.15.184:6827","nonce":3594364650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3594364650},{"type":"v1","addr":"172.21.15.184:6831","nonce":3594364650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3594364650},{"type":"v1","addr":"172.21.15.184:6829","nonce":3594364650}]},"public_addr":"172.21.15.184:6825/3594364650","cluster_addr":"172.21.15.184:6827/3594364650","heartbeat_back_addr":"172.21.15.184:6831/3594364650","heartbeat_front_addr":"172.21.15.184:6829/3594364650","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:06:17.855998+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:06:41.473270+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:07:05.304108+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:07:29.887472+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:07:55.387286+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:08:20.579669+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:08:46.333051+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:09:12.289668+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:09:38.180696+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:10:04.471459+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:10:31.041912+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:10:57.477266+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.82:0/2005424509":"2024-05-21T16:04:45.161949+0000","172.21.15.82:6801/1763049308":"2024-05-21T16:04:45.161949+0000","172.21.15.82:6801/1959459348":"2024-05-21T16:04:19.208291+0000","172.21.15.82:0/3131904556":"2024-05-21T16:04:19.208291+0000","172.21.15.82:6800/1959459348":"2024-05-21T16:04:19.208291+0000","172.21.15.82:0/2456966342":"2024-05-21T16:04:45.161949+0000","172.21.15.82:0/4179978833":"2024-05-21T16:04:19.208291+0000","172.21.15.82:0/1602862225":"2024-05-21T16:04:45.161949+0000","172.21.15.82:0/106101180":"2024-05-21T16:04:19.208291+0000","172.21.15.82:6800/1763049308":"2024-05-21T16:04:45.161949+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-20T16:11:43.804 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:43.804+0000 7fa829bc2640 1 -- 172.21.15.82:0/2116278004 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fa8100a2c10 msgr2=0x7fa8100a5120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:11:43.804 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:43.804+0000 7fa829bc2640 1 --2- 172.21.15.82:0/2116278004 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fa8100a2c10 0x7fa8100a5120 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fa814005ab0 tx=0x7fa814031400 comp rx=0 tx=0).stop 2024-05-20T16:11:43.804 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:43.804+0000 7fa829bc2640 1 -- 172.21.15.82:0/2116278004 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa8241078b0 msgr2=0x7fa82419f690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:11:43.804 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:43.804+0000 7fa829bc2640 1 --2- 172.21.15.82:0/2116278004 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa8241078b0 0x7fa82419f690 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fa81c00cce0 tx=0x7fa81c00a620 comp rx=0 tx=0).stop 2024-05-20T16:11:43.804 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:43.804+0000 7fa829bc2640 1 -- 172.21.15.82:0/2116278004 shutdown_connections 2024-05-20T16:11:43.804 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:43.804+0000 7fa829bc2640 1 --2- 172.21.15.82:0/2116278004 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fa8100a2c10 0x7fa8100a5120 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:43.804 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:43.804+0000 7fa829bc2640 1 --2- 172.21.15.82:0/2116278004 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa8241078b0 0x7fa82419f690 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:43.804 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:43.804+0000 7fa829bc2640 1 --2- 172.21.15.82:0/2116278004 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa824102bb0 0x7fa824198700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:43.804 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:43.804+0000 7fa829bc2640 1 --2- 172.21.15.82:0/2116278004 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa824101970 0x7fa8241981c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:43.804 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:43.804+0000 7fa829bc2640 1 -- 172.21.15.82:0/2116278004 >> 172.21.15.82:0/2116278004 conn(0x7fa8240fcb80 msgr2=0x7fa824103890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:11:43.805 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:43.804+0000 7fa829bc2640 1 -- 172.21.15.82:0/2116278004 shutdown_connections 2024-05-20T16:11:43.805 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:43.804+0000 7fa829bc2640 1 -- 172.21.15.82:0/2116278004 wait complete. 2024-05-20T16:11:43.820 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-20T16:07:09.493727+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-20T16:11:43.821 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-20T16:11:43.940 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:43.938+0000 7fa6064a2640 1 -- 172.21.15.82:0/926228837 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa600101990 msgr2=0x7fa600102690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:11:43.941 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:43.938+0000 7fa6064a2640 1 --2- 172.21.15.82:0/926228837 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa600101990 0x7fa600102690 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fa5f0009900 tx=0x7fa5f002a410 comp rx=0 tx=0).stop 2024-05-20T16:11:43.941 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:43.938+0000 7fa6064a2640 1 -- 172.21.15.82:0/926228837 shutdown_connections 2024-05-20T16:11:43.941 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:43.938+0000 7fa6064a2640 1 --2- 172.21.15.82:0/926228837 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa6001078d0 0x7fa600109cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:43.941 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:43.938+0000 7fa6064a2640 1 --2- 172.21.15.82:0/926228837 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa600102bd0 0x7fa600102fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:43.941 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:43.938+0000 7fa6064a2640 1 --2- 172.21.15.82:0/926228837 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa600101990 0x7fa600102690 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:43.941 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:43.938+0000 7fa6064a2640 1 -- 172.21.15.82:0/926228837 >> 172.21.15.82:0/926228837 conn(0x7fa6000fcb80 msgr2=0x7fa6000ff030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:11:43.941 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:43.938+0000 7fa6064a2640 1 -- 172.21.15.82:0/926228837 shutdown_connections 2024-05-20T16:11:43.941 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:43.938+0000 7fa6064a2640 1 -- 172.21.15.82:0/926228837 wait complete. 2024-05-20T16:11:43.941 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:43.939+0000 7fa6064a2640 1 Processor -- start 2024-05-20T16:11:43.941 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:43.939+0000 7fa6064a2640 1 -- start start 2024-05-20T16:11:43.942 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:43.939+0000 7fa6064a2640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa600101990 0x7fa6001995b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:11:43.942 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:43.939+0000 7fa6064a2640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa600102bd0 0x7fa600199af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:11:43.942 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:43.939+0000 7fa6064a2640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa6001078d0 0x7fa6001a0a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:11:43.942 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:43.939+0000 7fa6064a2640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fa60010d5c0 con 0x7fa6001078d0 2024-05-20T16:11:43.942 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:43.939+0000 7fa6064a2640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fa60010d440 con 0x7fa600101990 2024-05-20T16:11:43.942 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:43.939+0000 7fa6064a2640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa60010d740 con 0x7fa600102bd0 2024-05-20T16:11:43.942 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:43.939+0000 7fa604a3f640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa6001078d0 0x7fa6001a0a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:11:43.942 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:43.939+0000 7fa604a3f640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa6001078d0 0x7fa6001a0a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:52388/0 (socket says 172.21.15.82:52388) 2024-05-20T16:11:43.942 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:43.939+0000 7fa604a3f640 1 -- 172.21.15.82:0/1017125739 learned_addr learned my addr 172.21.15.82:0/1017125739 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:11:43.942 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:43.939+0000 7fa5fffff640 1 --2- 172.21.15.82:0/1017125739 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa600101990 0x7fa6001995b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:11:43.942 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:43.939+0000 7fa5f7fff640 1 --2- 172.21.15.82:0/1017125739 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa600102bd0 0x7fa600199af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:11:43.942 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:43.939+0000 7fa604a3f640 1 -- 172.21.15.82:0/1017125739 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa600102bd0 msgr2=0x7fa600199af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:11:43.943 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:43.939+0000 7fa604a3f640 1 --2- 172.21.15.82:0/1017125739 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa600102bd0 0x7fa600199af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:43.943 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:43.939+0000 7fa604a3f640 1 -- 172.21.15.82:0/1017125739 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa600101990 msgr2=0x7fa6001995b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:11:43.943 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:43.939+0000 7fa604a3f640 1 --2- 172.21.15.82:0/1017125739 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa600101990 0x7fa6001995b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:43.943 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:43.939+0000 7fa604a3f640 1 -- 172.21.15.82:0/1017125739 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6001a10f0 con 0x7fa6001078d0 2024-05-20T16:11:43.943 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:43.939+0000 7fa604a3f640 1 --2- 172.21.15.82:0/1017125739 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa6001078d0 0x7fa6001a0a80 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fa5f800eb10 tx=0x7fa5f800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:11:43.943 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:43.940+0000 7fa5fdffb640 1 -- 172.21.15.82:0/1017125739 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5f8004b20 con 0x7fa6001078d0 2024-05-20T16:11:43.943 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:43.940+0000 7fa5fdffb640 1 -- 172.21.15.82:0/1017125739 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa5f8004cc0 con 0x7fa6001078d0 2024-05-20T16:11:43.943 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:43.940+0000 7fa6064a2640 1 -- 172.21.15.82:0/1017125739 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6001a13e0 con 0x7fa6001078d0 2024-05-20T16:11:43.943 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:43.940+0000 7fa5fdffb640 1 -- 172.21.15.82:0/1017125739 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5f8010760 con 0x7fa6001078d0 2024-05-20T16:11:43.943 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:43.940+0000 7fa6064a2640 1 -- 172.21.15.82:0/1017125739 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6001a1930 con 0x7fa6001078d0 2024-05-20T16:11:43.943 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:43.941+0000 7fa6064a2640 1 -- 172.21.15.82:0/1017125739 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa60006b250 con 0x7fa6001078d0 2024-05-20T16:11:43.944 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:43.942+0000 7fa5fdffb640 1 -- 172.21.15.82:0/1017125739 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fa5f8005020 con 0x7fa6001078d0 2024-05-20T16:11:43.944 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:43.943+0000 7fa5fdffb640 1 --2- 172.21.15.82:0/1017125739 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fa5ec0a2c10 0x7fa5ec0a5120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:11:43.944 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:43.943+0000 7fa5fffff640 1 --2- 172.21.15.82:0/1017125739 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fa5ec0a2c10 0x7fa5ec0a5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:11:43.944 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:43.943+0000 7fa5fdffb640 1 -- 172.21.15.82:0/1017125739 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fa5f80cf0d0 con 0x7fa6001078d0 2024-05-20T16:11:43.944 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:43.943+0000 7fa5fffff640 1 --2- 172.21.15.82:0/1017125739 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fa5ec0a2c10 0x7fa5ec0a5120 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fa5f0005ab0 tx=0x7fa5f0031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:11:43.947 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:43.947+0000 7fa5fdffb640 1 -- 172.21.15.82:0/1017125739 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fa5f809c4d0 con 0x7fa6001078d0 2024-05-20T16:11:44.133 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:44.133+0000 7fa6064a2640 1 -- 172.21.15.82:0/1017125739 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fa6001a1bf0 con 0x7fa6001078d0 2024-05-20T16:11:44.133 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:44.133+0000 7fa5fdffb640 1 -- 172.21.15.82:0/1017125739 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) ==== 93+0+10 (secure 0 0 0) 0x7fa5f809bdd0 con 0x7fa6001078d0 2024-05-20T16:11:44.134 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 1 2024-05-20T16:11:44.136 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:44.136+0000 7fa6064a2640 1 -- 172.21.15.82:0/1017125739 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fa5ec0a2c10 msgr2=0x7fa5ec0a5120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:11:44.136 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:44.136+0000 7fa6064a2640 1 --2- 172.21.15.82:0/1017125739 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fa5ec0a2c10 0x7fa5ec0a5120 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fa5f0005ab0 tx=0x7fa5f0031400 comp rx=0 tx=0).stop 2024-05-20T16:11:44.136 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:44.136+0000 7fa6064a2640 1 -- 172.21.15.82:0/1017125739 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa6001078d0 msgr2=0x7fa6001a0a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:11:44.136 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:44.136+0000 7fa6064a2640 1 --2- 172.21.15.82:0/1017125739 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa6001078d0 0x7fa6001a0a80 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fa5f800eb10 tx=0x7fa5f800c2d0 comp rx=0 tx=0).stop 2024-05-20T16:11:44.136 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:44.136+0000 7fa6064a2640 1 -- 172.21.15.82:0/1017125739 shutdown_connections 2024-05-20T16:11:44.136 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:44.136+0000 7fa6064a2640 1 --2- 172.21.15.82:0/1017125739 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fa5ec0a2c10 0x7fa5ec0a5120 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:44.136 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:44.136+0000 7fa6064a2640 1 --2- 172.21.15.82:0/1017125739 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa6001078d0 0x7fa6001a0a80 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:44.136 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:44.136+0000 7fa6064a2640 1 --2- 172.21.15.82:0/1017125739 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa600102bd0 0x7fa600199af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:44.136 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:44.136+0000 7fa6064a2640 1 --2- 172.21.15.82:0/1017125739 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa600101990 0x7fa6001995b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:44.136 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:44.136+0000 7fa6064a2640 1 -- 172.21.15.82:0/1017125739 >> 172.21.15.82:0/1017125739 conn(0x7fa6000fcb80 msgr2=0x7fa6000fe2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:11:44.136 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:44.136+0000 7fa6064a2640 1 -- 172.21.15.82:0/1017125739 shutdown_connections 2024-05-20T16:11:44.136 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:44.136+0000 7fa6064a2640 1 -- 172.21.15.82:0/1017125739 wait complete. 2024-05-20T16:11:44.148 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-05-20T16:11:44.277 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:44.275+0000 7f7a22b16640 1 -- 172.21.15.82:0/2309563755 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7a1c105bb0 msgr2=0x7f7a1c108090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:11:44.277 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:44.275+0000 7f7a22b16640 1 --2- 172.21.15.82:0/2309563755 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7a1c105bb0 0x7f7a1c108090 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f7a08009900 tx=0x7f7a0802a410 comp rx=0 tx=0).stop 2024-05-20T16:11:44.277 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:44.275+0000 7f7a22b16640 1 -- 172.21.15.82:0/2309563755 shutdown_connections 2024-05-20T16:11:44.277 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:44.275+0000 7f7a22b16640 1 --2- 172.21.15.82:0/2309563755 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7a1c105bb0 0x7f7a1c108090 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:44.278 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:44.275+0000 7f7a22b16640 1 --2- 172.21.15.82:0/2309563755 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7a1c108720 0x7f7a1c10ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:44.278 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:44.275+0000 7f7a22b16640 1 --2- 172.21.15.82:0/2309563755 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7a1c1031c0 0x7f7a1c1055e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:44.278 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:44.275+0000 7f7a22b16640 1 -- 172.21.15.82:0/2309563755 >> 172.21.15.82:0/2309563755 conn(0x7f7a1c0fcb80 msgr2=0x7f7a1c0ff010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:11:44.278 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:44.275+0000 7f7a22b16640 1 -- 172.21.15.82:0/2309563755 shutdown_connections 2024-05-20T16:11:44.278 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:44.275+0000 7f7a22b16640 1 -- 172.21.15.82:0/2309563755 wait complete. 2024-05-20T16:11:44.278 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:44.276+0000 7f7a22b16640 1 Processor -- start 2024-05-20T16:11:44.278 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:44.276+0000 7f7a22b16640 1 -- start start 2024-05-20T16:11:44.278 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:44.276+0000 7f7a22b16640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7a1c1031c0 0x7f7a1c1993c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:11:44.278 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:44.276+0000 7f7a22b16640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7a1c105bb0 0x7f7a1c199900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:11:44.278 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:44.276+0000 7f7a22b16640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7a1c108720 0x7f7a1c1a0890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:11:44.278 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:44.276+0000 7f7a22b16640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f7a1c10e780 con 0x7f7a1c105bb0 2024-05-20T16:11:44.278 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:44.276+0000 7f7a22b16640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f7a1c10e600 con 0x7f7a1c108720 2024-05-20T16:11:44.278 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:44.276+0000 7f7a22b16640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f7a1c10e900 con 0x7f7a1c1031c0 2024-05-20T16:11:44.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:44.276+0000 7f7a1bfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7a1c105bb0 0x7f7a1c199900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:11:44.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:44.276+0000 7f7a1bfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7a1c105bb0 0x7f7a1c199900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:52410/0 (socket says 172.21.15.82:52410) 2024-05-20T16:11:44.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:44.276+0000 7f7a1bfff640 1 -- 172.21.15.82:0/720971714 learned_addr learned my addr 172.21.15.82:0/720971714 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:11:44.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:44.276+0000 7f7a208b2640 1 --2- 172.21.15.82:0/720971714 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7a1c1031c0 0x7f7a1c1993c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:11:44.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:44.276+0000 7f7a1bfff640 1 -- 172.21.15.82:0/720971714 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7a1c1031c0 msgr2=0x7f7a1c1993c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:11:44.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:44.276+0000 7f7a1bfff640 1 --2- 172.21.15.82:0/720971714 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7a1c1031c0 0x7f7a1c1993c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:44.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:44.276+0000 7f7a1bfff640 1 -- 172.21.15.82:0/720971714 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7a1c108720 msgr2=0x7f7a1c1a0890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:11:44.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:44.276+0000 7f7a210b3640 1 --2- 172.21.15.82:0/720971714 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7a1c108720 0x7f7a1c1a0890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:11:44.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:44.276+0000 7f7a1bfff640 1 --2- 172.21.15.82:0/720971714 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7a1c108720 0x7f7a1c1a0890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:44.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:44.276+0000 7f7a1bfff640 1 -- 172.21.15.82:0/720971714 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a1c1a0f00 con 0x7f7a1c105bb0 2024-05-20T16:11:44.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:44.276+0000 7f7a1bfff640 1 --2- 172.21.15.82:0/720971714 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7a1c105bb0 0x7f7a1c199900 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f7a08006010 tx=0x7f7a080043c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:11:44.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:44.277+0000 7f7a19ffb640 1 -- 172.21.15.82:0/720971714 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a0803f030 con 0x7f7a1c105bb0 2024-05-20T16:11:44.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:44.277+0000 7f7a19ffb640 1 -- 172.21.15.82:0/720971714 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7a08003ce0 con 0x7f7a1c105bb0 2024-05-20T16:11:44.280 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:44.277+0000 7f7a22b16640 1 -- 172.21.15.82:0/720971714 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a1c1a1190 con 0x7f7a1c105bb0 2024-05-20T16:11:44.280 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:44.277+0000 7f7a19ffb640 1 -- 172.21.15.82:0/720971714 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a08035070 con 0x7f7a1c105bb0 2024-05-20T16:11:44.280 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:44.277+0000 7f7a22b16640 1 -- 172.21.15.82:0/720971714 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a1c1a1700 con 0x7f7a1c105bb0 2024-05-20T16:11:44.280 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:44.278+0000 7f7a22b16640 1 -- 172.21.15.82:0/720971714 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a1c0fe6a0 con 0x7f7a1c105bb0 2024-05-20T16:11:44.281 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:44.279+0000 7f7a19ffb640 1 -- 172.21.15.82:0/720971714 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f7a08042030 con 0x7f7a1c105bb0 2024-05-20T16:11:44.281 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:44.280+0000 7f7a19ffb640 1 --2- 172.21.15.82:0/720971714 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f7a0c0a2b90 0x7f7a0c0a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:11:44.281 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:44.280+0000 7f7a208b2640 1 --2- 172.21.15.82:0/720971714 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f7a0c0a2b90 0x7f7a0c0a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:11:44.281 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:44.280+0000 7f7a19ffb640 1 -- 172.21.15.82:0/720971714 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f7a080ef570 con 0x7f7a1c105bb0 2024-05-20T16:11:44.281 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:44.280+0000 7f7a208b2640 1 --2- 172.21.15.82:0/720971714 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f7a0c0a2b90 0x7f7a0c0a50a0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f7a10009990 tx=0x7f7a10009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:11:44.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:44.283+0000 7f7a19ffb640 1 -- 172.21.15.82:0/720971714 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f7a080bc970 con 0x7f7a1c105bb0 2024-05-20T16:11:44.435 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:44 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:11:44.435 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:44 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:11:44.435 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:44 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:11:44.436 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:44 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:11:44.436 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:44 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:11:44.436 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:44 smithi082 ceph-mon[32238]: from='client.? 172.21.15.82:0/2920698104' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T16:11:44.436 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:44 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T16:11:44.436 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:44 smithi082 ceph-mon[32238]: from='client.? 172.21.15.82:0/2116278004' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-20T16:11:44.436 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:44 smithi082 ceph-mon[32238]: from='client.? 172.21.15.82:0/1017125739' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-20T16:11:44.469 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:44.469+0000 7f7a22b16640 1 -- 172.21.15.82:0/720971714 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f7a1c062750 con 0x7f7a1c105bb0 2024-05-20T16:11:44.470 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:44.470+0000 7f7a19ffb640 1 -- 172.21.15.82:0/720971714 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v82) ==== 84+0+65 (secure 0 0 0) 0x7f7a080bc270 con 0x7f7a1c105bb0 2024-05-20T16:11:44.470 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-05-20T16:11:44.470 INFO:teuthology.orchestra.run.smithi082.stdout:[ 2024-05-20T16:11:44.470 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-05-20T16:11:44.470 INFO:teuthology.orchestra.run.smithi082.stdout: "poolnum": 1, 2024-05-20T16:11:44.470 INFO:teuthology.orchestra.run.smithi082.stdout: "poolname": ".mgr" 2024-05-20T16:11:44.470 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-20T16:11:44.470 INFO:teuthology.orchestra.run.smithi082.stdout:] 2024-05-20T16:11:44.472 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:44.472+0000 7f7a22b16640 1 -- 172.21.15.82:0/720971714 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f7a0c0a2b90 msgr2=0x7f7a0c0a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:11:44.472 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:44.472+0000 7f7a22b16640 1 --2- 172.21.15.82:0/720971714 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f7a0c0a2b90 0x7f7a0c0a50a0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f7a10009990 tx=0x7f7a10009110 comp rx=0 tx=0).stop 2024-05-20T16:11:44.472 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:44.472+0000 7f7a22b16640 1 -- 172.21.15.82:0/720971714 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7a1c105bb0 msgr2=0x7f7a1c199900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:11:44.472 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:44.472+0000 7f7a22b16640 1 --2- 172.21.15.82:0/720971714 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7a1c105bb0 0x7f7a1c199900 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f7a08006010 tx=0x7f7a080043c0 comp rx=0 tx=0).stop 2024-05-20T16:11:44.473 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:44.472+0000 7f7a22b16640 1 -- 172.21.15.82:0/720971714 shutdown_connections 2024-05-20T16:11:44.473 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:44.472+0000 7f7a22b16640 1 --2- 172.21.15.82:0/720971714 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f7a0c0a2b90 0x7f7a0c0a50a0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:44.473 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:44.472+0000 7f7a22b16640 1 --2- 172.21.15.82:0/720971714 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7a1c105bb0 0x7f7a1c199900 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:44.473 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:44.472+0000 7f7a22b16640 1 --2- 172.21.15.82:0/720971714 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7a1c108720 0x7f7a1c1a0890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:44.473 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:44.472+0000 7f7a22b16640 1 --2- 172.21.15.82:0/720971714 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7a1c1031c0 0x7f7a1c1993c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:44.473 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:44.472+0000 7f7a22b16640 1 -- 172.21.15.82:0/720971714 >> 172.21.15.82:0/720971714 conn(0x7f7a1c0fcb80 msgr2=0x7f7a1c104120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:11:44.473 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:44.472+0000 7f7a22b16640 1 -- 172.21.15.82:0/720971714 shutdown_connections 2024-05-20T16:11:44.473 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:44.472+0000 7f7a22b16640 1 -- 172.21.15.82:0/720971714 wait complete. 2024-05-20T16:11:44.484 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-05-20T16:11:44.484 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-05-20T16:11:44.606 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:44.604+0000 7fc33efaa640 1 -- 172.21.15.82:0/4060430430 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc338105bf0 msgr2=0x7fc338108090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:11:44.606 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:44.604+0000 7fc33efaa640 1 --2- 172.21.15.82:0/4060430430 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc338105bf0 0x7fc338108090 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fc32c009900 tx=0x7fc32c02a410 comp rx=0 tx=0).stop 2024-05-20T16:11:44.607 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:44.604+0000 7fc33efaa640 1 -- 172.21.15.82:0/4060430430 shutdown_connections 2024-05-20T16:11:44.607 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:44.604+0000 7fc33efaa640 1 --2- 172.21.15.82:0/4060430430 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc338105bf0 0x7fc338108090 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:44.607 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:44.604+0000 7fc33efaa640 1 --2- 172.21.15.82:0/4060430430 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc338108720 0x7fc33810ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:44.607 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:44.604+0000 7fc33efaa640 1 --2- 172.21.15.82:0/4060430430 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc338103200 0x7fc338105620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:44.607 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:44.604+0000 7fc33efaa640 1 -- 172.21.15.82:0/4060430430 >> 172.21.15.82:0/4060430430 conn(0x7fc3380fcb80 msgr2=0x7fc3380ff030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:11:44.607 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:44.604+0000 7fc33efaa640 1 -- 172.21.15.82:0/4060430430 shutdown_connections 2024-05-20T16:11:44.607 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:44.604+0000 7fc33efaa640 1 -- 172.21.15.82:0/4060430430 wait complete. 2024-05-20T16:11:44.607 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:44.605+0000 7fc33efaa640 1 Processor -- start 2024-05-20T16:11:44.607 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:44.605+0000 7fc33efaa640 1 -- start start 2024-05-20T16:11:44.607 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:44.605+0000 7fc33efaa640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc338103200 0x7fc3381995c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:11:44.607 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:44.605+0000 7fc33efaa640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc338105bf0 0x7fc338199b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:11:44.608 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:44.605+0000 7fc33efaa640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc338108720 0x7fc3381a0a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:11:44.608 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:44.605+0000 7fc33efaa640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fc33810e700 con 0x7fc338108720 2024-05-20T16:11:44.608 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:44.605+0000 7fc33efaa640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fc33810e580 con 0x7fc338103200 2024-05-20T16:11:44.608 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:44.605+0000 7fc33efaa640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fc33810e880 con 0x7fc338105bf0 2024-05-20T16:11:44.608 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:44.605+0000 7fc33d547640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc338108720 0x7fc3381a0a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:11:44.608 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:44.605+0000 7fc33d547640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc338108720 0x7fc3381a0a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:52428/0 (socket says 172.21.15.82:52428) 2024-05-20T16:11:44.608 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:44.605+0000 7fc33d547640 1 -- 172.21.15.82:0/2241653693 learned_addr learned my addr 172.21.15.82:0/2241653693 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:11:44.608 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:44.605+0000 7fc33cd46640 1 --2- 172.21.15.82:0/2241653693 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc338103200 0x7fc3381995c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:11:44.608 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:44.605+0000 7fc337fff640 1 --2- 172.21.15.82:0/2241653693 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc338105bf0 0x7fc338199b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:11:44.608 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:44.605+0000 7fc33d547640 1 -- 172.21.15.82:0/2241653693 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc338105bf0 msgr2=0x7fc338199b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:11:44.608 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:44.605+0000 7fc33d547640 1 --2- 172.21.15.82:0/2241653693 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc338105bf0 0x7fc338199b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:44.608 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:44.605+0000 7fc33d547640 1 -- 172.21.15.82:0/2241653693 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc338103200 msgr2=0x7fc3381995c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:11:44.608 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:44.605+0000 7fc33d547640 1 --2- 172.21.15.82:0/2241653693 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc338103200 0x7fc3381995c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:44.609 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:44.605+0000 7fc33d547640 1 -- 172.21.15.82:0/2241653693 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3381a1100 con 0x7fc338108720 2024-05-20T16:11:44.609 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:44.605+0000 7fc33d547640 1 --2- 172.21.15.82:0/2241653693 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc338108720 0x7fc3381a0a90 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fc32800d750 tx=0x7fc32800dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:11:44.609 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:44.606+0000 7fc335ffb640 1 -- 172.21.15.82:0/2241653693 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc328017070 con 0x7fc338108720 2024-05-20T16:11:44.609 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:44.606+0000 7fc335ffb640 1 -- 172.21.15.82:0/2241653693 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc328005500 con 0x7fc338108720 2024-05-20T16:11:44.609 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:44.606+0000 7fc335ffb640 1 -- 172.21.15.82:0/2241653693 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc328004d90 con 0x7fc338108720 2024-05-20T16:11:44.609 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:44.606+0000 7fc33efaa640 1 -- 172.21.15.82:0/2241653693 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3381a13f0 con 0x7fc338108720 2024-05-20T16:11:44.609 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:44.606+0000 7fc33efaa640 1 -- 172.21.15.82:0/2241653693 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3381a1940 con 0x7fc338108720 2024-05-20T16:11:44.610 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:44.607+0000 7fc33efaa640 1 -- 172.21.15.82:0/2241653693 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc33806b250 con 0x7fc338108720 2024-05-20T16:11:44.610 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:44.608+0000 7fc335ffb640 1 -- 172.21.15.82:0/2241653693 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fc328004f30 con 0x7fc338108720 2024-05-20T16:11:44.610 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:44.609+0000 7fc335ffb640 1 --2- 172.21.15.82:0/2241653693 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fc3200a2c10 0x7fc3200a5120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:11:44.610 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:44.609+0000 7fc33cd46640 1 --2- 172.21.15.82:0/2241653693 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fc3200a2c10 0x7fc3200a5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:11:44.610 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:44.609+0000 7fc335ffb640 1 -- 172.21.15.82:0/2241653693 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fc3280ce410 con 0x7fc338108720 2024-05-20T16:11:44.610 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:44.609+0000 7fc33cd46640 1 --2- 172.21.15.82:0/2241653693 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fc3200a2c10 0x7fc3200a5120 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fc3380697c0 tx=0x7fc330009680 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:11:44.613 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:44.614+0000 7fc335ffb640 1 -- 172.21.15.82:0/2241653693 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fc32809b1c0 con 0x7fc338108720 2024-05-20T16:11:44.625 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:44 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:11:44.625 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:44 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:11:44.625 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:44 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:11:44.625 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:44 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:11:44.625 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:44 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:11:44.625 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:44 smithi160 ceph-mon[36347]: from='client.? 172.21.15.82:0/2920698104' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T16:11:44.625 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:44 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T16:11:44.625 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:44 smithi160 ceph-mon[36347]: from='client.? 172.21.15.82:0/2116278004' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-20T16:11:44.625 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:44 smithi160 ceph-mon[36347]: from='client.? 172.21.15.82:0/1017125739' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-20T16:11:44.635 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:44 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:11:44.635 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:44 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:11:44.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:44 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:11:44.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:44 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:11:44.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:44 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:11:44.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:44 smithi184 ceph-mon[36209]: from='client.? 172.21.15.82:0/2920698104' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T16:11:44.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:44 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T16:11:44.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:44 smithi184 ceph-mon[36209]: from='client.? 172.21.15.82:0/2116278004' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-20T16:11:44.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:44 smithi184 ceph-mon[36209]: from='client.? 172.21.15.82:0/1017125739' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-20T16:11:44.795 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:44.795+0000 7fc33efaa640 1 -- 172.21.15.82:0/2241653693 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fc3381a1ba0 con 0x7fc338108720 2024-05-20T16:11:45.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:45.367+0000 7fc335ffb640 1 -- 172.21.15.82:0/2241653693 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v83) ==== 138+0+0 (secure 0 0 0) 0x7fc32809f4c0 con 0x7fc338108720 2024-05-20T16:11:45.367 INFO:teuthology.orchestra.run.smithi082.stderr:pool 'cephfs_metadata' created 2024-05-20T16:11:45.370 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:45.370+0000 7fc33efaa640 1 -- 172.21.15.82:0/2241653693 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fc3200a2c10 msgr2=0x7fc3200a5120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:11:45.370 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:45.370+0000 7fc33efaa640 1 --2- 172.21.15.82:0/2241653693 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fc3200a2c10 0x7fc3200a5120 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fc3380697c0 tx=0x7fc330009680 comp rx=0 tx=0).stop 2024-05-20T16:11:45.370 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:45.370+0000 7fc33efaa640 1 -- 172.21.15.82:0/2241653693 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc338108720 msgr2=0x7fc3381a0a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:11:45.370 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:45.370+0000 7fc33efaa640 1 --2- 172.21.15.82:0/2241653693 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc338108720 0x7fc3381a0a90 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fc32800d750 tx=0x7fc32800dc10 comp rx=0 tx=0).stop 2024-05-20T16:11:45.370 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:45.370+0000 7fc33efaa640 1 -- 172.21.15.82:0/2241653693 shutdown_connections 2024-05-20T16:11:45.370 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:45.370+0000 7fc33efaa640 1 --2- 172.21.15.82:0/2241653693 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fc3200a2c10 0x7fc3200a5120 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:45.371 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:45.370+0000 7fc33efaa640 1 --2- 172.21.15.82:0/2241653693 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc338108720 0x7fc3381a0a90 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:45.371 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:45.370+0000 7fc33efaa640 1 --2- 172.21.15.82:0/2241653693 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc338105bf0 0x7fc338199b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:45.371 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:45.370+0000 7fc33efaa640 1 --2- 172.21.15.82:0/2241653693 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc338103200 0x7fc3381995c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:45.371 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:45.370+0000 7fc33efaa640 1 -- 172.21.15.82:0/2241653693 >> 172.21.15.82:0/2241653693 conn(0x7fc3380fcb80 msgr2=0x7fc338104ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:11:45.371 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:45.370+0000 7fc33efaa640 1 -- 172.21.15.82:0/2241653693 shutdown_connections 2024-05-20T16:11:45.371 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:45.370+0000 7fc33efaa640 1 -- 172.21.15.82:0/2241653693 wait complete. 2024-05-20T16:11:45.384 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-05-20T16:11:45.508 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:45.506+0000 7f4d12f3e640 1 -- 172.21.15.82:0/3815558949 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4d0c102bd0 msgr2=0x7f4d0c102fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:11:45.508 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:45.506+0000 7f4d12f3e640 1 --2- 172.21.15.82:0/3815558949 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4d0c102bd0 0x7f4d0c102fe0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f4cf8009900 tx=0x7f4cf802a410 comp rx=0 tx=0).stop 2024-05-20T16:11:45.508 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:45.506+0000 7f4d12f3e640 1 -- 172.21.15.82:0/3815558949 shutdown_connections 2024-05-20T16:11:45.509 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:45.506+0000 7f4d12f3e640 1 --2- 172.21.15.82:0/3815558949 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4d0c102bd0 0x7f4d0c102fe0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:45.509 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:45.506+0000 7f4d12f3e640 1 --2- 172.21.15.82:0/3815558949 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4d0c1078d0 0x7f4d0c109cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:45.509 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:45.506+0000 7f4d12f3e640 1 --2- 172.21.15.82:0/3815558949 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4d0c101990 0x7f4d0c102690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:45.509 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:45.506+0000 7f4d12f3e640 1 -- 172.21.15.82:0/3815558949 >> 172.21.15.82:0/3815558949 conn(0x7f4d0c0fcba0 msgr2=0x7f4d0c0ff030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:11:45.509 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:45.506+0000 7f4d12f3e640 1 -- 172.21.15.82:0/3815558949 shutdown_connections 2024-05-20T16:11:45.509 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:45.506+0000 7f4d12f3e640 1 -- 172.21.15.82:0/3815558949 wait complete. 2024-05-20T16:11:45.509 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:45.507+0000 7f4d12f3e640 1 Processor -- start 2024-05-20T16:11:45.509 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:45.507+0000 7f4d12f3e640 1 -- start start 2024-05-20T16:11:45.509 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:45.507+0000 7f4d12f3e640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4d0c101990 0x7f4d0c199350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:11:45.509 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:45.507+0000 7f4d12f3e640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4d0c102bd0 0x7f4d0c199890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:11:45.509 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:45.507+0000 7f4d12f3e640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4d0c1078d0 0x7f4d0c1a0820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:11:45.509 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:45.507+0000 7f4d12f3e640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f4d0c10e710 con 0x7f4d0c1078d0 2024-05-20T16:11:45.509 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:45.507+0000 7f4d12f3e640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f4d0c10e590 con 0x7f4d0c102bd0 2024-05-20T16:11:45.510 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:45.507+0000 7f4d12f3e640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f4d0c10e890 con 0x7f4d0c101990 2024-05-20T16:11:45.510 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:45.507+0000 7f4d114db640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4d0c1078d0 0x7f4d0c1a0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:11:45.510 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:45.507+0000 7f4d114db640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4d0c1078d0 0x7f4d0c1a0820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:52446/0 (socket says 172.21.15.82:52446) 2024-05-20T16:11:45.510 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:45.507+0000 7f4d114db640 1 -- 172.21.15.82:0/4179053626 learned_addr learned my addr 172.21.15.82:0/4179053626 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:11:45.510 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:45.507+0000 7f4d10cda640 1 --2- 172.21.15.82:0/4179053626 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4d0c101990 0x7f4d0c199350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:11:45.510 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:45.507+0000 7f4d0bfff640 1 --2- 172.21.15.82:0/4179053626 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4d0c102bd0 0x7f4d0c199890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:11:45.510 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:45.507+0000 7f4d114db640 1 -- 172.21.15.82:0/4179053626 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4d0c101990 msgr2=0x7f4d0c199350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:11:45.510 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:45.507+0000 7f4d114db640 1 --2- 172.21.15.82:0/4179053626 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4d0c101990 0x7f4d0c199350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:45.510 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:45.507+0000 7f4d114db640 1 -- 172.21.15.82:0/4179053626 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4d0c102bd0 msgr2=0x7f4d0c199890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:11:45.510 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:45.507+0000 7f4d114db640 1 --2- 172.21.15.82:0/4179053626 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4d0c102bd0 0x7f4d0c199890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:45.510 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:45.507+0000 7f4d114db640 1 -- 172.21.15.82:0/4179053626 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d0c1a0e90 con 0x7f4d0c1078d0 2024-05-20T16:11:45.510 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:45.508+0000 7f4d114db640 1 --2- 172.21.15.82:0/4179053626 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4d0c1078d0 0x7f4d0c1a0820 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f4d04009fd0 tx=0x7f4d0400b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:11:45.510 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:45.508+0000 7f4d09ffb640 1 -- 172.21.15.82:0/4179053626 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d04017070 con 0x7f4d0c1078d0 2024-05-20T16:11:45.511 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:45.508+0000 7f4d12f3e640 1 -- 172.21.15.82:0/4179053626 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d0c1a1180 con 0x7f4d0c1078d0 2024-05-20T16:11:45.511 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:45.508+0000 7f4d09ffb640 1 -- 172.21.15.82:0/4179053626 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d04003e50 con 0x7f4d0c1078d0 2024-05-20T16:11:45.511 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:45.508+0000 7f4d09ffb640 1 -- 172.21.15.82:0/4179053626 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d040106c0 con 0x7f4d0c1078d0 2024-05-20T16:11:45.511 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:45.508+0000 7f4d12f3e640 1 -- 172.21.15.82:0/4179053626 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d0c1a1720 con 0x7f4d0c1078d0 2024-05-20T16:11:45.512 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:45.509+0000 7f4d12f3e640 1 -- 172.21.15.82:0/4179053626 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d0c1066e0 con 0x7f4d0c1078d0 2024-05-20T16:11:45.512 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:45.510+0000 7f4d09ffb640 1 -- 172.21.15.82:0/4179053626 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f4d04003970 con 0x7f4d0c1078d0 2024-05-20T16:11:45.512 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:45.511+0000 7f4d09ffb640 1 --2- 172.21.15.82:0/4179053626 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f4cfc0a2b90 0x7f4cfc0a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:11:45.512 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:45.511+0000 7f4d10cda640 1 --2- 172.21.15.82:0/4179053626 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f4cfc0a2b90 0x7f4cfc0a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:11:45.512 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:45.511+0000 7f4d09ffb640 1 -- 172.21.15.82:0/4179053626 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7497+0+0 (secure 0 0 0) 0x7f4d040ced20 con 0x7f4d0c1078d0 2024-05-20T16:11:45.512 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:45.511+0000 7f4d10cda640 1 --2- 172.21.15.82:0/4179053626 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f4cfc0a2b90 0x7f4cfc0a50a0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f4d0c102590 tx=0x7f4d00005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:11:45.515 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:45.515+0000 7f4d09ffb640 1 -- 172.21.15.82:0/4179053626 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f4d0409b960 con 0x7f4d0c1078d0 2024-05-20T16:11:45.635 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:45 smithi184 ceph-mon[36209]: from='client.? 172.21.15.82:0/720971714' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-05-20T16:11:45.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:45 smithi184 ceph-mon[36209]: from='client.? 172.21.15.82:0/2241653693' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-05-20T16:11:45.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:45 smithi184 ceph-mon[36209]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-20T16:11:45.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:45 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:11:45.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:45 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:11:45.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:45 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:11:45.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:45 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:11:45.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:45 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:11:45.702 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:45.702+0000 7f4d12f3e640 1 -- 172.21.15.82:0/4179053626 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) -- 0x7f4d0c02d2a0 con 0x7f4d0c1078d0 2024-05-20T16:11:45.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:45 smithi160 ceph-mon[36347]: from='client.? 172.21.15.82:0/720971714' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-05-20T16:11:45.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:45 smithi160 ceph-mon[36347]: from='client.? 172.21.15.82:0/2241653693' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-05-20T16:11:45.813 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:45 smithi160 ceph-mon[36347]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-20T16:11:45.813 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:45 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:11:45.813 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:45 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:11:45.813 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:45 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:11:45.813 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:45 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:11:45.813 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:45 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:11:45.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:45 smithi082 ceph-mon[32238]: from='client.? 172.21.15.82:0/720971714' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-05-20T16:11:45.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:45 smithi082 ceph-mon[32238]: from='client.? 172.21.15.82:0/2241653693' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-05-20T16:11:45.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:45 smithi082 ceph-mon[32238]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-20T16:11:45.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:45 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:11:45.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:45 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:11:45.860 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:45 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:11:45.860 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:45 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:11:45.860 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:45 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:11:46.377 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:46.377+0000 7f4d09ffb640 1 -- 172.21.15.82:0/4179053626 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v84) ==== 171+0+0 (secure 0 0 0) 0x7f4d0409fc60 con 0x7f4d0c1078d0 2024-05-20T16:11:46.378 INFO:teuthology.orchestra.run.smithi082.stderr:pool 'cephfs_data' created 2024-05-20T16:11:46.380 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:46.380+0000 7f4d12f3e640 1 -- 172.21.15.82:0/4179053626 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f4cfc0a2b90 msgr2=0x7f4cfc0a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:11:46.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:46.380+0000 7f4d12f3e640 1 --2- 172.21.15.82:0/4179053626 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f4cfc0a2b90 0x7f4cfc0a50a0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f4d0c102590 tx=0x7f4d00005f90 comp rx=0 tx=0).stop 2024-05-20T16:11:46.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:46.380+0000 7f4d12f3e640 1 -- 172.21.15.82:0/4179053626 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4d0c1078d0 msgr2=0x7f4d0c1a0820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:11:46.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:46.380+0000 7f4d12f3e640 1 --2- 172.21.15.82:0/4179053626 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4d0c1078d0 0x7f4d0c1a0820 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f4d04009fd0 tx=0x7f4d0400b260 comp rx=0 tx=0).stop 2024-05-20T16:11:46.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:46.380+0000 7f4d12f3e640 1 -- 172.21.15.82:0/4179053626 shutdown_connections 2024-05-20T16:11:46.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:46.380+0000 7f4d12f3e640 1 --2- 172.21.15.82:0/4179053626 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f4cfc0a2b90 0x7f4cfc0a50a0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:46.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:46.380+0000 7f4d12f3e640 1 --2- 172.21.15.82:0/4179053626 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4d0c1078d0 0x7f4d0c1a0820 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:46.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:46.380+0000 7f4d12f3e640 1 --2- 172.21.15.82:0/4179053626 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4d0c102bd0 0x7f4d0c199890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:46.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:46.380+0000 7f4d12f3e640 1 --2- 172.21.15.82:0/4179053626 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4d0c101990 0x7f4d0c199350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:46.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:46.380+0000 7f4d12f3e640 1 -- 172.21.15.82:0/4179053626 >> 172.21.15.82:0/4179053626 conn(0x7f4d0c0fcba0 msgr2=0x7f4d0c1038b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:11:46.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:46.381+0000 7f4d12f3e640 1 -- 172.21.15.82:0/4179053626 shutdown_connections 2024-05-20T16:11:46.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:46.381+0000 7f4d12f3e640 1 -- 172.21.15.82:0/4179053626 wait complete. 2024-05-20T16:11:46.400 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-05-20T16:11:46.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:46.522+0000 7fc1b1b57640 1 -- 172.21.15.82:0/702878311 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc1ac1078e0 msgr2=0x7fc1ac109d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:11:46.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:46.522+0000 7fc1b1b57640 1 --2- 172.21.15.82:0/702878311 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc1ac1078e0 0x7fc1ac109d00 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fc1a4009ca0 tx=0x7fc1a402a8c0 comp rx=0 tx=0).stop 2024-05-20T16:11:46.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:46.522+0000 7fc1b1b57640 1 -- 172.21.15.82:0/702878311 shutdown_connections 2024-05-20T16:11:46.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:46.522+0000 7fc1b1b57640 1 --2- 172.21.15.82:0/702878311 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc1ac1078e0 0x7fc1ac109d00 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:46.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:46.522+0000 7fc1b1b57640 1 --2- 172.21.15.82:0/702878311 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc1ac102be0 0x7fc1ac102ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:46.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:46.522+0000 7fc1b1b57640 1 --2- 172.21.15.82:0/702878311 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc1ac1019a0 0x7fc1ac1026a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:46.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:46.522+0000 7fc1b1b57640 1 -- 172.21.15.82:0/702878311 >> 172.21.15.82:0/702878311 conn(0x7fc1ac0fcb70 msgr2=0x7fc1ac0ff040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:11:46.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:46.522+0000 7fc1b1b57640 1 -- 172.21.15.82:0/702878311 shutdown_connections 2024-05-20T16:11:46.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:46.522+0000 7fc1b1b57640 1 -- 172.21.15.82:0/702878311 wait complete. 2024-05-20T16:11:46.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:46.522+0000 7fc1b1b57640 1 Processor -- start 2024-05-20T16:11:46.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:46.523+0000 7fc1b1b57640 1 -- start start 2024-05-20T16:11:46.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:46.523+0000 7fc1b1b57640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc1ac1019a0 0x7fc1ac074810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:11:46.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:46.523+0000 7fc1b1b57640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc1ac102be0 0x7fc1ac19dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:11:46.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:46.523+0000 7fc1b1b57640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc1ac1078e0 0x7fc1ac1a0530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:11:46.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:46.523+0000 7fc1b1b57640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fc1ac10d620 con 0x7fc1ac102be0 2024-05-20T16:11:46.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:46.523+0000 7fc1b1b57640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fc1ac10d4a0 con 0x7fc1ac1078e0 2024-05-20T16:11:46.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:46.523+0000 7fc1b1b57640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fc1ac10d7a0 con 0x7fc1ac1019a0 2024-05-20T16:11:46.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:46.523+0000 7fc1a3fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc1ac102be0 0x7fc1ac19dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:11:46.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:46.523+0000 7fc1a3fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc1ac102be0 0x7fc1ac19dfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:52452/0 (socket says 172.21.15.82:52452) 2024-05-20T16:11:46.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:46.523+0000 7fc1a3fff640 1 -- 172.21.15.82:0/195426484 learned_addr learned my addr 172.21.15.82:0/195426484 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:11:46.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:46.523+0000 7fc1ab7fe640 1 --2- 172.21.15.82:0/195426484 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc1ac1019a0 0x7fc1ac074810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:11:46.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:46.523+0000 7fc1abfff640 1 --2- 172.21.15.82:0/195426484 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc1ac1078e0 0x7fc1ac1a0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:11:46.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:46.523+0000 7fc1a3fff640 1 -- 172.21.15.82:0/195426484 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc1ac1019a0 msgr2=0x7fc1ac074810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:11:46.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:46.523+0000 7fc1a3fff640 1 --2- 172.21.15.82:0/195426484 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc1ac1019a0 0x7fc1ac074810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:46.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:46.523+0000 7fc1a3fff640 1 -- 172.21.15.82:0/195426484 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc1ac1078e0 msgr2=0x7fc1ac1a0530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:11:46.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:46.523+0000 7fc1a3fff640 1 --2- 172.21.15.82:0/195426484 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc1ac1078e0 0x7fc1ac1a0530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:46.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:46.523+0000 7fc1a3fff640 1 -- 172.21.15.82:0/195426484 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1ac1a0d50 con 0x7fc1ac102be0 2024-05-20T16:11:46.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:46.523+0000 7fc1a3fff640 1 --2- 172.21.15.82:0/195426484 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc1ac102be0 0x7fc1ac19dfc0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fc19400e9f0 tx=0x7fc19400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:11:46.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:46.523+0000 7fc1a97fa640 1 -- 172.21.15.82:0/195426484 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc19400cd80 con 0x7fc1ac102be0 2024-05-20T16:11:46.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:46.523+0000 7fc1a97fa640 1 -- 172.21.15.82:0/195426484 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc194005500 con 0x7fc1ac102be0 2024-05-20T16:11:46.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:46.524+0000 7fc1b1b57640 1 -- 172.21.15.82:0/195426484 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1ac1a0fe0 con 0x7fc1ac102be0 2024-05-20T16:11:46.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:46.524+0000 7fc1a97fa640 1 -- 172.21.15.82:0/195426484 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc194010690 con 0x7fc1ac102be0 2024-05-20T16:11:46.527 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:46.524+0000 7fc1b1b57640 1 -- 172.21.15.82:0/195426484 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1ac1a1480 con 0x7fc1ac102be0 2024-05-20T16:11:46.527 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:46.524+0000 7fc1b1b57640 1 -- 172.21.15.82:0/195426484 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1ac1066f0 con 0x7fc1ac102be0 2024-05-20T16:11:46.528 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:46.526+0000 7fc1a97fa640 1 -- 172.21.15.82:0/195426484 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fc1940046c0 con 0x7fc1ac102be0 2024-05-20T16:11:46.528 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:46.527+0000 7fc1a97fa640 1 --2- 172.21.15.82:0/195426484 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fc1980a2b90 0x7fc1980a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:11:46.528 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:46.527+0000 7fc1ab7fe640 1 --2- 172.21.15.82:0/195426484 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fc1980a2b90 0x7fc1980a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:11:46.528 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:46.527+0000 7fc1a97fa640 1 -- 172.21.15.82:0/195426484 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7879+0+0 (secure 0 0 0) 0x7fc1940cf640 con 0x7fc1ac102be0 2024-05-20T16:11:46.528 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:46.527+0000 7fc1ab7fe640 1 --2- 172.21.15.82:0/195426484 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fc1980a2b90 0x7fc1980a50a0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fc1ac1025a0 tx=0x7fc19c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:11:46.530 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:46.530+0000 7fc1a97fa640 1 -- 172.21.15.82:0/195426484 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fc194053850 con 0x7fc1ac102be0 2024-05-20T16:11:46.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:46 smithi082 ceph-mon[32238]: from='client.? 172.21.15.82:0/2241653693' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-05-20T16:11:46.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:46 smithi082 ceph-mon[32238]: osdmap e83: 12 total, 12 up, 12 in 2024-05-20T16:11:46.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:46 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:11:46.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:46 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:11:46.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:46 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:11:46.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:46 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T16:11:46.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:46 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:11:46.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:46 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T16:11:46.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:46 smithi082 ceph-mon[32238]: from='client.? 172.21.15.82:0/4179053626' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-05-20T16:11:46.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:46 smithi184 ceph-mon[36209]: from='client.? 172.21.15.82:0/2241653693' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-05-20T16:11:46.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:46 smithi184 ceph-mon[36209]: osdmap e83: 12 total, 12 up, 12 in 2024-05-20T16:11:46.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:46 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:11:46.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:46 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:11:46.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:46 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:11:46.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:46 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T16:11:46.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:46 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:11:46.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:46 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T16:11:46.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:46 smithi184 ceph-mon[36209]: from='client.? 172.21.15.82:0/4179053626' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-05-20T16:11:46.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:46.749+0000 7fc1b1b57640 1 -- 172.21.15.82:0/195426484 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fc1ac02d2a0 con 0x7fc1ac102be0 2024-05-20T16:11:46.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:46 smithi160 ceph-mon[36347]: from='client.? 172.21.15.82:0/2241653693' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-05-20T16:11:46.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:46 smithi160 ceph-mon[36347]: osdmap e83: 12 total, 12 up, 12 in 2024-05-20T16:11:46.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:46 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:11:46.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:46 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:11:46.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:46 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:11:46.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:46 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T16:11:46.813 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:46 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:11:46.813 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:46 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T16:11:46.813 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:46 smithi160 ceph-mon[36347]: from='client.? 172.21.15.82:0/4179053626' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-05-20T16:11:47.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:47.379+0000 7fc1a97fa640 1 -- 172.21.15.82:0/195426484 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7fc19401f340 con 0x7fc1ac102be0 2024-05-20T16:11:47.379 INFO:teuthology.orchestra.run.smithi082.stderr:new fs with metadata pool 2 and data pool 3 2024-05-20T16:11:47.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:47.381+0000 7fc1b1b57640 1 -- 172.21.15.82:0/195426484 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fc1980a2b90 msgr2=0x7fc1980a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:11:47.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:47.381+0000 7fc1b1b57640 1 --2- 172.21.15.82:0/195426484 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fc1980a2b90 0x7fc1980a50a0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fc1ac1025a0 tx=0x7fc19c005f90 comp rx=0 tx=0).stop 2024-05-20T16:11:47.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:47.381+0000 7fc1b1b57640 1 -- 172.21.15.82:0/195426484 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc1ac102be0 msgr2=0x7fc1ac19dfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:11:47.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:47.381+0000 7fc1b1b57640 1 --2- 172.21.15.82:0/195426484 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc1ac102be0 0x7fc1ac19dfc0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fc19400e9f0 tx=0x7fc19400edf0 comp rx=0 tx=0).stop 2024-05-20T16:11:47.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:47.381+0000 7fc1b1b57640 1 -- 172.21.15.82:0/195426484 shutdown_connections 2024-05-20T16:11:47.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:47.381+0000 7fc1b1b57640 1 --2- 172.21.15.82:0/195426484 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fc1980a2b90 0x7fc1980a50a0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:47.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:47.381+0000 7fc1b1b57640 1 --2- 172.21.15.82:0/195426484 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc1ac102be0 0x7fc1ac19dfc0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:47.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:47.381+0000 7fc1b1b57640 1 --2- 172.21.15.82:0/195426484 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc1ac1078e0 0x7fc1ac1a0530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:47.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:47.381+0000 7fc1b1b57640 1 --2- 172.21.15.82:0/195426484 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc1ac1019a0 0x7fc1ac074810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:47.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:47.381+0000 7fc1b1b57640 1 -- 172.21.15.82:0/195426484 >> 172.21.15.82:0/195426484 conn(0x7fc1ac0fcb70 msgr2=0x7fc1ac1038c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:11:47.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:47.381+0000 7fc1b1b57640 1 -- 172.21.15.82:0/195426484 shutdown_connections 2024-05-20T16:11:47.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:47.381+0000 7fc1b1b57640 1 -- 172.21.15.82:0/195426484 wait complete. 2024-05-20T16:11:47.396 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-05-20T16:11:47.396 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-05-20T16:11:47.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:47.512+0000 7ff2921d5640 1 -- 172.21.15.82:0/15053751 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff28c1089e0 msgr2=0x7ff28c10ae00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:11:47.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:47.512+0000 7ff2921d5640 1 --2- 172.21.15.82:0/15053751 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff28c1089e0 0x7ff28c10ae00 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7ff284009ca0 tx=0x7ff28402a8c0 comp rx=0 tx=0).stop 2024-05-20T16:11:47.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:47.512+0000 7ff2921d5640 1 -- 172.21.15.82:0/15053751 shutdown_connections 2024-05-20T16:11:47.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:47.512+0000 7ff2921d5640 1 --2- 172.21.15.82:0/15053751 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff28c1089e0 0x7ff28c10ae00 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:47.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:47.512+0000 7ff2921d5640 1 --2- 172.21.15.82:0/15053751 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff28c103ce0 0x7ff28c1040f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:47.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:47.512+0000 7ff2921d5640 1 --2- 172.21.15.82:0/15053751 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff28c102aa0 0x7ff28c1037a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:47.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:47.512+0000 7ff2921d5640 1 -- 172.21.15.82:0/15053751 >> 172.21.15.82:0/15053751 conn(0x7ff28c0fdc90 msgr2=0x7ff28c100140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:11:47.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:47.513+0000 7ff2921d5640 1 -- 172.21.15.82:0/15053751 shutdown_connections 2024-05-20T16:11:47.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:47.513+0000 7ff2921d5640 1 -- 172.21.15.82:0/15053751 wait complete. 2024-05-20T16:11:47.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:47.513+0000 7ff2921d5640 1 Processor -- start 2024-05-20T16:11:47.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:47.513+0000 7ff2921d5640 1 -- start start 2024-05-20T16:11:47.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:47.513+0000 7ff2921d5640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff28c102aa0 0x7ff28c199260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:11:47.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:47.513+0000 7ff2921d5640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff28c103ce0 0x7ff28c1997a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:11:47.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:47.514+0000 7ff2921d5640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff28c1089e0 0x7ff28c1a0730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:11:47.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:47.514+0000 7ff2921d5640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7ff28c10e720 con 0x7ff28c102aa0 2024-05-20T16:11:47.521 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:47.514+0000 7ff2921d5640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7ff28c10e5a0 con 0x7ff28c103ce0 2024-05-20T16:11:47.521 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:47.514+0000 7ff2921d5640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ff28c10e8a0 con 0x7ff28c1089e0 2024-05-20T16:11:47.521 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:47.514+0000 7ff28b7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff28c102aa0 0x7ff28c199260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:11:47.521 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:47.514+0000 7ff28b7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff28c102aa0 0x7ff28c199260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:52470/0 (socket says 172.21.15.82:52470) 2024-05-20T16:11:47.521 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:47.514+0000 7ff28b7fe640 1 -- 172.21.15.82:0/3380463326 learned_addr learned my addr 172.21.15.82:0/3380463326 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:11:47.522 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:47.514+0000 7ff28bfff640 1 --2- 172.21.15.82:0/3380463326 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff28c1089e0 0x7ff28c1a0730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:11:47.522 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:47.514+0000 7ff282ffd640 1 --2- 172.21.15.82:0/3380463326 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff28c103ce0 0x7ff28c1997a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:11:47.522 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:47.514+0000 7ff28b7fe640 1 -- 172.21.15.82:0/3380463326 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff28c1089e0 msgr2=0x7ff28c1a0730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:11:47.522 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:47.514+0000 7ff28b7fe640 1 --2- 172.21.15.82:0/3380463326 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff28c1089e0 0x7ff28c1a0730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:47.522 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:47.514+0000 7ff28b7fe640 1 -- 172.21.15.82:0/3380463326 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff28c103ce0 msgr2=0x7ff28c1997a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:11:47.522 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:47.514+0000 7ff28b7fe640 1 --2- 172.21.15.82:0/3380463326 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff28c103ce0 0x7ff28c1997a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:47.523 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:47.514+0000 7ff28b7fe640 1 -- 172.21.15.82:0/3380463326 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff28c1a0da0 con 0x7ff28c102aa0 2024-05-20T16:11:47.523 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:47.514+0000 7ff282ffd640 1 --2- 172.21.15.82:0/3380463326 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff28c103ce0 0x7ff28c1997a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T16:11:47.523 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:47.514+0000 7ff28b7fe640 1 --2- 172.21.15.82:0/3380463326 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff28c102aa0 0x7ff28c199260 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7ff27c00e9f0 tx=0x7ff27c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:11:47.523 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:47.514+0000 7ff2897fa640 1 -- 172.21.15.82:0/3380463326 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff27c00ccc0 con 0x7ff28c102aa0 2024-05-20T16:11:47.523 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:47.514+0000 7ff2897fa640 1 -- 172.21.15.82:0/3380463326 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff27c0035f0 con 0x7ff28c102aa0 2024-05-20T16:11:47.523 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:47.514+0000 7ff2921d5640 1 -- 172.21.15.82:0/3380463326 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff28c1a1090 con 0x7ff28c102aa0 2024-05-20T16:11:47.523 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:47.514+0000 7ff2897fa640 1 -- 172.21.15.82:0/3380463326 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff27c00ce60 con 0x7ff28c102aa0 2024-05-20T16:11:47.523 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:47.514+0000 7ff2921d5640 1 -- 172.21.15.82:0/3380463326 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff28c1a1590 con 0x7ff28c102aa0 2024-05-20T16:11:47.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:47.515+0000 7ff2921d5640 1 -- 172.21.15.82:0/3380463326 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff28c069110 con 0x7ff28c102aa0 2024-05-20T16:11:47.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:47.516+0000 7ff2897fa640 1 -- 172.21.15.82:0/3380463326 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7ff27c0037a0 con 0x7ff28c102aa0 2024-05-20T16:11:47.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:47.517+0000 7ff2897fa640 1 --2- 172.21.15.82:0/3380463326 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7ff2780a2b90 0x7ff2780a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:11:47.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:47.517+0000 7ff282ffd640 1 --2- 172.21.15.82:0/3380463326 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7ff2780a2b90 0x7ff2780a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:11:47.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:47.517+0000 7ff2897fa640 1 -- 172.21.15.82:0/3380463326 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7ff27c0cf4a0 con 0x7ff28c102aa0 2024-05-20T16:11:47.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:47.518+0000 7ff282ffd640 1 --2- 172.21.15.82:0/3380463326 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7ff2780a2b90 0x7ff2780a50a0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7ff28c19a300 tx=0x7ff274005e70 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:11:47.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:47.522+0000 7ff2897fa640 1 -- 172.21.15.82:0/3380463326 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7ff27c09c550 con 0x7ff28c102aa0 2024-05-20T16:11:47.708 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:47.708+0000 7ff2921d5640 1 -- 172.21.15.82:0/3380463326 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) -- 0x7ff28c1077f0 con 0x7ff28c102aa0 2024-05-20T16:11:47.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:47 smithi160 ceph-mon[36347]: from='client.? 172.21.15.82:0/4179053626' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-05-20T16:11:47.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:47 smithi160 ceph-mon[36347]: osdmap e84: 12 total, 12 up, 12 in 2024-05-20T16:11:47.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:47 smithi160 ceph-mon[36347]: from='client.? 172.21.15.82:0/195426484' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-05-20T16:11:47.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:47 smithi160 ceph-mon[36347]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-05-20T16:11:47.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:47 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:11:47.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:47 smithi160 ceph-mon[36347]: osdmap e85: 12 total, 12 up, 12 in 2024-05-20T16:11:47.813 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:47 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:11:47.813 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:47 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:11:47.813 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:47 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:11:47.813 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:47 smithi160 ceph-mon[36347]: pgmap v286: 97 pgs: 10 creating+peering, 6 active+clean, 81 unknown; 577 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-20T16:11:47.813 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:47 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:11:47.813 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:47 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:11:47.814 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:47 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:11:47.814 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:47 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T16:11:47.814 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:47 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:11:47.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:47 smithi082 ceph-mon[32238]: from='client.? 172.21.15.82:0/4179053626' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-05-20T16:11:47.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:47 smithi082 ceph-mon[32238]: osdmap e84: 12 total, 12 up, 12 in 2024-05-20T16:11:47.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:47 smithi082 ceph-mon[32238]: from='client.? 172.21.15.82:0/195426484' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-05-20T16:11:47.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:47 smithi082 ceph-mon[32238]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-05-20T16:11:47.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:47 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:11:47.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:47 smithi082 ceph-mon[32238]: osdmap e85: 12 total, 12 up, 12 in 2024-05-20T16:11:47.860 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:47 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:11:47.860 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:47 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:11:47.860 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:47 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:11:47.860 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:47 smithi082 ceph-mon[32238]: pgmap v286: 97 pgs: 10 creating+peering, 6 active+clean, 81 unknown; 577 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-20T16:11:47.860 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:47 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:11:47.860 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:47 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:11:47.860 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:47 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:11:47.860 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:47 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T16:11:47.860 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:47 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:11:47.885 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:47 smithi184 ceph-mon[36209]: from='client.? 172.21.15.82:0/4179053626' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-05-20T16:11:47.886 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:47 smithi184 ceph-mon[36209]: osdmap e84: 12 total, 12 up, 12 in 2024-05-20T16:11:47.886 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:47 smithi184 ceph-mon[36209]: from='client.? 172.21.15.82:0/195426484' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-05-20T16:11:47.886 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:47 smithi184 ceph-mon[36209]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-05-20T16:11:47.886 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:47 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:11:47.886 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:47 smithi184 ceph-mon[36209]: osdmap e85: 12 total, 12 up, 12 in 2024-05-20T16:11:47.886 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:47 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:11:47.886 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:47 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:11:47.886 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:47 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:11:47.886 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:47 smithi184 ceph-mon[36209]: pgmap v286: 97 pgs: 10 creating+peering, 6 active+clean, 81 unknown; 577 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-20T16:11:47.886 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:47 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:11:47.886 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:47 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:11:47.886 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:47 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:11:47.886 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:47 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T16:11:47.887 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:47 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:11:48.167 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:48.167+0000 7ff2897fa640 1 -- 172.21.15.82:0/3380463326 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v86) ==== 157+0+0 (secure 0 0 0) 0x7ff27c09be50 con 0x7ff28c102aa0 2024-05-20T16:11:48.170 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:48.170+0000 7ff2921d5640 1 -- 172.21.15.82:0/3380463326 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7ff2780a2b90 msgr2=0x7ff2780a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:11:48.171 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:48.170+0000 7ff2921d5640 1 --2- 172.21.15.82:0/3380463326 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7ff2780a2b90 0x7ff2780a50a0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7ff28c19a300 tx=0x7ff274005e70 comp rx=0 tx=0).stop 2024-05-20T16:11:48.171 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:48.170+0000 7ff2921d5640 1 -- 172.21.15.82:0/3380463326 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff28c102aa0 msgr2=0x7ff28c199260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:11:48.171 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:48.170+0000 7ff2921d5640 1 --2- 172.21.15.82:0/3380463326 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff28c102aa0 0x7ff28c199260 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7ff27c00e9f0 tx=0x7ff27c00eeb0 comp rx=0 tx=0).stop 2024-05-20T16:11:48.171 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:48.170+0000 7ff2921d5640 1 -- 172.21.15.82:0/3380463326 shutdown_connections 2024-05-20T16:11:48.171 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:48.170+0000 7ff2921d5640 1 --2- 172.21.15.82:0/3380463326 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7ff2780a2b90 0x7ff2780a50a0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:48.171 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:48.170+0000 7ff2921d5640 1 --2- 172.21.15.82:0/3380463326 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff28c1089e0 0x7ff28c1a0730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:48.171 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:48.170+0000 7ff2921d5640 1 --2- 172.21.15.82:0/3380463326 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff28c103ce0 0x7ff28c1997a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:48.172 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:48.170+0000 7ff2921d5640 1 --2- 172.21.15.82:0/3380463326 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff28c102aa0 0x7ff28c199260 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:48.172 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:48.170+0000 7ff2921d5640 1 -- 172.21.15.82:0/3380463326 >> 172.21.15.82:0/3380463326 conn(0x7ff28c0fdc90 msgr2=0x7ff28c1049c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:11:48.172 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:48.170+0000 7ff2921d5640 1 -- 172.21.15.82:0/3380463326 shutdown_connections 2024-05-20T16:11:48.172 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:48.170+0000 7ff2921d5640 1 -- 172.21.15.82:0/3380463326 wait complete. 2024-05-20T16:11:48.183 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-05-20T16:11:48.316 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:48.313+0000 7f805c661640 1 -- 172.21.15.82:0/3765020305 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8054102380 msgr2=0x7f8054102790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:11:48.316 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:48.313+0000 7f805c661640 1 --2- 172.21.15.82:0/3765020305 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8054102380 0x7f8054102790 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f8044009900 tx=0x7f804402a410 comp rx=0 tx=0).stop 2024-05-20T16:11:48.316 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:48.313+0000 7f805c661640 1 -- 172.21.15.82:0/3765020305 shutdown_connections 2024-05-20T16:11:48.316 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:48.313+0000 7f805c661640 1 --2- 172.21.15.82:0/3765020305 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8054102380 0x7f8054102790 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:48.316 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:48.313+0000 7f805c661640 1 --2- 172.21.15.82:0/3765020305 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8054107080 0x7f80541094a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:48.316 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:48.313+0000 7f805c661640 1 --2- 172.21.15.82:0/3765020305 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8054101140 0x7f8054101e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:48.316 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:48.313+0000 7f805c661640 1 -- 172.21.15.82:0/3765020305 >> 172.21.15.82:0/3765020305 conn(0x7f80540fc330 msgr2=0x7f80540fe7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:11:48.317 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:48.313+0000 7f805c661640 1 -- 172.21.15.82:0/3765020305 shutdown_connections 2024-05-20T16:11:48.317 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:48.313+0000 7f805c661640 1 -- 172.21.15.82:0/3765020305 wait complete. 2024-05-20T16:11:48.317 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:48.314+0000 7f805c661640 1 Processor -- start 2024-05-20T16:11:48.317 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:48.314+0000 7f805c661640 1 -- start start 2024-05-20T16:11:48.317 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:48.314+0000 7f805c661640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8054101140 0x7f8054108ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:11:48.317 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:48.314+0000 7f805c661640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8054102380 0x7f805419e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:11:48.317 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:48.314+0000 7f805c661640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8054107080 0x7f80541a06f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:11:48.317 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:48.314+0000 7f805c661640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f805410cee0 con 0x7f8054107080 2024-05-20T16:11:48.317 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:48.314+0000 7f805c661640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f805410cd60 con 0x7f8054102380 2024-05-20T16:11:48.317 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:48.314+0000 7f805c661640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f805410d060 con 0x7f8054101140 2024-05-20T16:11:48.317 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:48.314+0000 7f805abfe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8054107080 0x7f80541a06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:11:48.317 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:48.314+0000 7f805abfe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8054107080 0x7f80541a06f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:52494/0 (socket says 172.21.15.82:52494) 2024-05-20T16:11:48.317 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:48.314+0000 7f805abfe640 1 -- 172.21.15.82:0/3659450028 learned_addr learned my addr 172.21.15.82:0/3659450028 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:11:48.318 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:48.314+0000 7f8059bfc640 1 --2- 172.21.15.82:0/3659450028 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8054102380 0x7f805419e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:11:48.318 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:48.315+0000 7f805a3fd640 1 --2- 172.21.15.82:0/3659450028 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8054101140 0x7f8054108ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:11:48.318 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:48.315+0000 7f805abfe640 1 -- 172.21.15.82:0/3659450028 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8054101140 msgr2=0x7f8054108ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:11:48.318 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:48.315+0000 7f805abfe640 1 --2- 172.21.15.82:0/3659450028 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8054101140 0x7f8054108ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:48.318 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:48.315+0000 7f805abfe640 1 -- 172.21.15.82:0/3659450028 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8054102380 msgr2=0x7f805419e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:11:48.318 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:48.315+0000 7f805abfe640 1 --2- 172.21.15.82:0/3659450028 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8054102380 0x7f805419e1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:48.318 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:48.315+0000 7f805abfe640 1 -- 172.21.15.82:0/3659450028 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80541a0f10 con 0x7f8054107080 2024-05-20T16:11:48.318 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:48.315+0000 7f805abfe640 1 --2- 172.21.15.82:0/3659450028 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8054107080 0x7f80541a06f0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f804c00b970 tx=0x7f804c00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:11:48.318 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:48.315+0000 7f804b7fe640 1 -- 172.21.15.82:0/3659450028 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f804c019070 con 0x7f8054107080 2024-05-20T16:11:48.318 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:48.315+0000 7f804b7fe640 1 -- 172.21.15.82:0/3659450028 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f804c00cb40 con 0x7f8054107080 2024-05-20T16:11:48.318 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:48.315+0000 7f805c661640 1 -- 172.21.15.82:0/3659450028 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80541a1200 con 0x7f8054107080 2024-05-20T16:11:48.318 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:48.315+0000 7f804b7fe640 1 -- 172.21.15.82:0/3659450028 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f804c007650 con 0x7f8054107080 2024-05-20T16:11:48.318 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:48.315+0000 7f805c661640 1 -- 172.21.15.82:0/3659450028 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80541a1750 con 0x7f8054107080 2024-05-20T16:11:48.319 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:48.316+0000 7f805c661640 1 -- 172.21.15.82:0/3659450028 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8054106780 con 0x7f8054107080 2024-05-20T16:11:48.319 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:48.317+0000 7f804b7fe640 1 -- 172.21.15.82:0/3659450028 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f804c01c070 con 0x7f8054107080 2024-05-20T16:11:48.319 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:48.318+0000 7f804b7fe640 1 --2- 172.21.15.82:0/3659450028 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f80400a2b90 0x7f80400a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:11:48.320 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:48.318+0000 7f805a3fd640 1 --2- 172.21.15.82:0/3659450028 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f80400a2b90 0x7f80400a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:11:48.320 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:48.318+0000 7f804b7fe640 1 -- 172.21.15.82:0/3659450028 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8277+0+0 (secure 0 0 0) 0x7f804c0ce660 con 0x7f8054107080 2024-05-20T16:11:48.320 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:48.318+0000 7f805a3fd640 1 --2- 172.21.15.82:0/3659450028 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f80400a2b90 0x7f80400a50a0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f8054101d40 tx=0x7f8050008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:11:48.322 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:48.322+0000 7f804b7fe640 1 -- 172.21.15.82:0/3659450028 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f804c09af40 con 0x7f8054107080 2024-05-20T16:11:48.504 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:48.504+0000 7f805c661640 1 -- 172.21.15.82:0/3659450028 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f805402d660 con 0x7f8054107080 2024-05-20T16:11:48.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:48 smithi082 ceph-mon[32238]: from='client.? 172.21.15.82:0/195426484' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-05-20T16:11:48.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:48 smithi082 ceph-mon[32238]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-05-20T16:11:48.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:48 smithi082 ceph-mon[32238]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-05-20T16:11:48.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:48 smithi082 ceph-mon[32238]: from='client.? 172.21.15.82:0/3380463326' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-05-20T16:11:48.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:48 smithi082 ceph-mon[32238]: from='client.? 172.21.15.82:0/3380463326' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-05-20T16:11:48.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:48 smithi082 ceph-mon[32238]: osdmap e86: 12 total, 12 up, 12 in 2024-05-20T16:11:48.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:48 smithi160 ceph-mon[36347]: from='client.? 172.21.15.82:0/195426484' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-05-20T16:11:48.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:48 smithi160 ceph-mon[36347]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-05-20T16:11:48.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:48 smithi160 ceph-mon[36347]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-05-20T16:11:48.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:48 smithi160 ceph-mon[36347]: from='client.? 172.21.15.82:0/3380463326' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-05-20T16:11:48.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:48 smithi160 ceph-mon[36347]: from='client.? 172.21.15.82:0/3380463326' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-05-20T16:11:48.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:48 smithi160 ceph-mon[36347]: osdmap e86: 12 total, 12 up, 12 in 2024-05-20T16:11:48.886 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:48 smithi184 ceph-mon[36209]: from='client.? 172.21.15.82:0/195426484' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-05-20T16:11:48.886 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:48 smithi184 ceph-mon[36209]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-05-20T16:11:48.886 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:48 smithi184 ceph-mon[36209]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-05-20T16:11:48.886 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:48 smithi184 ceph-mon[36209]: from='client.? 172.21.15.82:0/3380463326' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-05-20T16:11:48.887 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:48 smithi184 ceph-mon[36209]: from='client.? 172.21.15.82:0/3380463326' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-05-20T16:11:48.887 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:48 smithi184 ceph-mon[36209]: osdmap e86: 12 total, 12 up, 12 in 2024-05-20T16:11:49.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:49 smithi160 ceph-mon[36347]: mds.? [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] up:active 2024-05-20T16:11:49.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:49 smithi160 ceph-mon[36347]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-20T16:11:49.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:49 smithi160 ceph-mon[36347]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-20T16:11:49.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:49 smithi160 ceph-mon[36347]: from='client.? 172.21.15.82:0/3659450028' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-05-20T16:11:49.813 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:49 smithi160 ceph-mon[36347]: pgmap v288: 97 pgs: 10 creating+peering, 44 active+clean, 43 unknown; 578 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s wr, 4 op/s 2024-05-20T16:11:49.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:49 smithi082 ceph-mon[32238]: mds.? [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] up:active 2024-05-20T16:11:49.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:49 smithi082 ceph-mon[32238]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-20T16:11:49.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:49 smithi082 ceph-mon[32238]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-20T16:11:49.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:49 smithi082 ceph-mon[32238]: from='client.? 172.21.15.82:0/3659450028' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-05-20T16:11:49.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:49 smithi082 ceph-mon[32238]: pgmap v288: 97 pgs: 10 creating+peering, 44 active+clean, 43 unknown; 578 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s wr, 4 op/s 2024-05-20T16:11:49.886 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:49 smithi184 ceph-mon[36209]: mds.? [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] up:active 2024-05-20T16:11:49.886 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:49 smithi184 ceph-mon[36209]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-20T16:11:49.886 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:49 smithi184 ceph-mon[36209]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-20T16:11:49.886 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:49 smithi184 ceph-mon[36209]: from='client.? 172.21.15.82:0/3659450028' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-05-20T16:11:49.886 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:49 smithi184 ceph-mon[36209]: pgmap v288: 97 pgs: 10 creating+peering, 44 active+clean, 43 unknown; 578 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s wr, 4 op/s 2024-05-20T16:11:50.404 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:50.404+0000 7f804b7fe640 1 -- 172.21.15.82:0/3659450028 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v88) ==== 228+0+0 (secure 0 0 0) 0x7f804c09f240 con 0x7f8054107080 2024-05-20T16:11:50.404 INFO:teuthology.orchestra.run.smithi082.stderr:pool 'cephfs_data_ec' created 2024-05-20T16:11:50.407 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:50.407+0000 7f805c661640 1 -- 172.21.15.82:0/3659450028 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f80400a2b90 msgr2=0x7f80400a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:11:50.407 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:50.407+0000 7f805c661640 1 --2- 172.21.15.82:0/3659450028 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f80400a2b90 0x7f80400a50a0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f8054101d40 tx=0x7f8050008040 comp rx=0 tx=0).stop 2024-05-20T16:11:50.407 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:50.407+0000 7f805c661640 1 -- 172.21.15.82:0/3659450028 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8054107080 msgr2=0x7f80541a06f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:11:50.408 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:50.407+0000 7f805c661640 1 --2- 172.21.15.82:0/3659450028 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8054107080 0x7f80541a06f0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f804c00b970 tx=0x7f804c00be30 comp rx=0 tx=0).stop 2024-05-20T16:11:50.408 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:50.408+0000 7f805c661640 1 -- 172.21.15.82:0/3659450028 shutdown_connections 2024-05-20T16:11:50.408 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:50.408+0000 7f805c661640 1 --2- 172.21.15.82:0/3659450028 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f80400a2b90 0x7f80400a50a0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:50.409 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:50.408+0000 7f805c661640 1 --2- 172.21.15.82:0/3659450028 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8054107080 0x7f80541a06f0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:50.409 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:50.408+0000 7f805c661640 1 --2- 172.21.15.82:0/3659450028 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8054102380 0x7f805419e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:50.409 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:50.408+0000 7f805c661640 1 --2- 172.21.15.82:0/3659450028 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8054101140 0x7f8054108ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:50.409 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:50.408+0000 7f805c661640 1 -- 172.21.15.82:0/3659450028 >> 172.21.15.82:0/3659450028 conn(0x7f80540fc330 msgr2=0x7f80540fd170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:11:50.409 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:50.408+0000 7f805c661640 1 -- 172.21.15.82:0/3659450028 shutdown_connections 2024-05-20T16:11:50.409 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:50.408+0000 7f805c661640 1 -- 172.21.15.82:0/3659450028 wait complete. 2024-05-20T16:11:50.425 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-05-20T16:11:50.543 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:50.541+0000 7f307b0a0640 1 -- 172.21.15.82:0/176606657 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3074101990 msgr2=0x7f3074102690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:11:50.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:50.541+0000 7f307b0a0640 1 --2- 172.21.15.82:0/176606657 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3074101990 0x7f3074102690 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f3064009ca0 tx=0x7f306402a8c0 comp rx=0 tx=0).stop 2024-05-20T16:11:50.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:50.542+0000 7f307b0a0640 1 -- 172.21.15.82:0/176606657 shutdown_connections 2024-05-20T16:11:50.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:50.542+0000 7f307b0a0640 1 --2- 172.21.15.82:0/176606657 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f30741078d0 0x7f3074109cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:50.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:50.542+0000 7f307b0a0640 1 --2- 172.21.15.82:0/176606657 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3074102bd0 0x7f3074102fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:50.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:50.542+0000 7f307b0a0640 1 --2- 172.21.15.82:0/176606657 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3074101990 0x7f3074102690 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:50.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:50.542+0000 7f307b0a0640 1 -- 172.21.15.82:0/176606657 >> 172.21.15.82:0/176606657 conn(0x7f30740fcb80 msgr2=0x7f30740ff030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:11:50.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:50.542+0000 7f307b0a0640 1 -- 172.21.15.82:0/176606657 shutdown_connections 2024-05-20T16:11:50.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:50.542+0000 7f307b0a0640 1 -- 172.21.15.82:0/176606657 wait complete. 2024-05-20T16:11:50.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:50.542+0000 7f307b0a0640 1 Processor -- start 2024-05-20T16:11:50.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:50.542+0000 7f307b0a0640 1 -- start start 2024-05-20T16:11:50.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:50.542+0000 7f307b0a0640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3074101990 0x7f307410c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:11:50.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:50.542+0000 7f307b0a0640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3074102bd0 0x7f307410a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:11:50.545 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:50.542+0000 7f307b0a0640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f30741078d0 0x7f307410ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:11:50.545 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:50.543+0000 7f307b0a0640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f307410e750 con 0x7f3074102bd0 2024-05-20T16:11:50.545 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:50.543+0000 7f307b0a0640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f307410e5d0 con 0x7f3074101990 2024-05-20T16:11:50.545 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:50.543+0000 7f307b0a0640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f307410e8d0 con 0x7f30741078d0 2024-05-20T16:11:50.545 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:50.543+0000 7f3073fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3074102bd0 0x7f307410a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:11:50.545 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:50.543+0000 7f3073fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3074102bd0 0x7f307410a900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:56230/0 (socket says 172.21.15.82:56230) 2024-05-20T16:11:50.545 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:50.543+0000 7f3073fff640 1 -- 172.21.15.82:0/2451145426 learned_addr learned my addr 172.21.15.82:0/2451145426 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:11:50.545 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:50.543+0000 7f307963d640 1 --2- 172.21.15.82:0/2451145426 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f30741078d0 0x7f307410ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:11:50.545 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:50.543+0000 7f3078e3c640 1 --2- 172.21.15.82:0/2451145426 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3074101990 0x7f307410c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:11:50.545 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:50.543+0000 7f3073fff640 1 -- 172.21.15.82:0/2451145426 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f30741078d0 msgr2=0x7f307410ae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:11:50.545 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:50.543+0000 7f3073fff640 1 --2- 172.21.15.82:0/2451145426 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f30741078d0 0x7f307410ae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:50.545 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:50.543+0000 7f3073fff640 1 -- 172.21.15.82:0/2451145426 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3074101990 msgr2=0x7f307410c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:11:50.545 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:50.543+0000 7f3073fff640 1 --2- 172.21.15.82:0/2451145426 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3074101990 0x7f307410c1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:50.545 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:50.543+0000 7f3073fff640 1 -- 172.21.15.82:0/2451145426 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f307410b570 con 0x7f3074102bd0 2024-05-20T16:11:50.546 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:50.543+0000 7f3073fff640 1 --2- 172.21.15.82:0/2451145426 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3074102bd0 0x7f307410a900 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f305c00b4a0 tx=0x7f305c00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:11:50.546 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:50.543+0000 7f3071ffb640 1 -- 172.21.15.82:0/2451145426 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f305c014070 con 0x7f3074102bd0 2024-05-20T16:11:50.546 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:50.543+0000 7f3071ffb640 1 -- 172.21.15.82:0/2451145426 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f305c010af0 con 0x7f3074102bd0 2024-05-20T16:11:50.546 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:50.543+0000 7f307b0a0640 1 -- 172.21.15.82:0/2451145426 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30741a3780 con 0x7f3074102bd0 2024-05-20T16:11:50.546 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:50.543+0000 7f3071ffb640 1 -- 172.21.15.82:0/2451145426 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f305c017d10 con 0x7f3074102bd0 2024-05-20T16:11:50.546 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:50.543+0000 7f307b0a0640 1 -- 172.21.15.82:0/2451145426 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30741a3c60 con 0x7f3074102bd0 2024-05-20T16:11:50.547 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:50.544+0000 7f307b0a0640 1 -- 172.21.15.82:0/2451145426 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3074069f50 con 0x7f3074102bd0 2024-05-20T16:11:50.547 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:50.546+0000 7f3071ffb640 1 -- 172.21.15.82:0/2451145426 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f305c017070 con 0x7f3074102bd0 2024-05-20T16:11:50.547 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:50.546+0000 7f3071ffb640 1 --2- 172.21.15.82:0/2451145426 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f30600a2b90 0x7f30600a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:11:50.547 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:50.546+0000 7f3078e3c640 1 --2- 172.21.15.82:0/2451145426 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f30600a2b90 0x7f30600a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:11:50.547 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:50.546+0000 7f3071ffb640 1 -- 172.21.15.82:0/2451145426 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8770+0+0 (secure 0 0 0) 0x7f305c0ce630 con 0x7f3074102bd0 2024-05-20T16:11:50.547 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:50.547+0000 7f3078e3c640 1 --2- 172.21.15.82:0/2451145426 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f30600a2b90 0x7f30600a50a0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f3064006010 tx=0x7f3064000f40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:11:50.551 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:50.551+0000 7f3071ffb640 1 -- 172.21.15.82:0/2451145426 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f305c09b370 con 0x7f3074102bd0 2024-05-20T16:11:50.726 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:50.726+0000 7f307b0a0640 1 -- 172.21.15.82:0/2451145426 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) -- 0x7f307402d410 con 0x7f3074102bd0 2024-05-20T16:11:50.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:50 smithi160 ceph-mon[36347]: osdmap e87: 12 total, 12 up, 12 in 2024-05-20T16:11:50.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:50 smithi082 ceph-mon[32238]: osdmap e87: 12 total, 12 up, 12 in 2024-05-20T16:11:50.885 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:50 smithi184 ceph-mon[36209]: osdmap e87: 12 total, 12 up, 12 in 2024-05-20T16:11:51.413 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:51.413+0000 7f3071ffb640 1 -- 172.21.15.82:0/2451145426 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v89) ==== 169+0+0 (secure 0 0 0) 0x7f305c09ac70 con 0x7f3074102bd0 2024-05-20T16:11:51.413 INFO:teuthology.orchestra.run.smithi082.stderr:set pool 4 allow_ec_overwrites to true 2024-05-20T16:11:51.416 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:51.416+0000 7f307b0a0640 1 -- 172.21.15.82:0/2451145426 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f30600a2b90 msgr2=0x7f30600a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:11:51.416 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:51.416+0000 7f307b0a0640 1 --2- 172.21.15.82:0/2451145426 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f30600a2b90 0x7f30600a50a0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f3064006010 tx=0x7f3064000f40 comp rx=0 tx=0).stop 2024-05-20T16:11:51.416 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:51.416+0000 7f307b0a0640 1 -- 172.21.15.82:0/2451145426 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3074102bd0 msgr2=0x7f307410a900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:11:51.416 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:51.416+0000 7f307b0a0640 1 --2- 172.21.15.82:0/2451145426 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3074102bd0 0x7f307410a900 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f305c00b4a0 tx=0x7f305c00b960 comp rx=0 tx=0).stop 2024-05-20T16:11:51.416 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:51.416+0000 7f307b0a0640 1 -- 172.21.15.82:0/2451145426 shutdown_connections 2024-05-20T16:11:51.416 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:51.416+0000 7f307b0a0640 1 --2- 172.21.15.82:0/2451145426 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f30600a2b90 0x7f30600a50a0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:51.416 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:51.416+0000 7f307b0a0640 1 --2- 172.21.15.82:0/2451145426 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3074102bd0 0x7f307410a900 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:51.416 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:51.416+0000 7f307b0a0640 1 --2- 172.21.15.82:0/2451145426 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f30741078d0 0x7f307410ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:51.416 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:51.416+0000 7f307b0a0640 1 --2- 172.21.15.82:0/2451145426 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3074101990 0x7f307410c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:51.416 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:51.416+0000 7f307b0a0640 1 -- 172.21.15.82:0/2451145426 >> 172.21.15.82:0/2451145426 conn(0x7f30740fcb80 msgr2=0x7f30741038b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:11:51.417 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:51.416+0000 7f307b0a0640 1 -- 172.21.15.82:0/2451145426 shutdown_connections 2024-05-20T16:11:51.417 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:51.416+0000 7f307b0a0640 1 -- 172.21.15.82:0/2451145426 wait complete. 2024-05-20T16:11:51.436 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_ec 2024-05-20T16:11:51.559 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:51.557+0000 7fd08d21b640 1 -- 172.21.15.82:0/3066891022 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd088103ce0 msgr2=0x7fd0881040f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:11:51.560 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:51.557+0000 7fd08d21b640 1 --2- 172.21.15.82:0/3066891022 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd088103ce0 0x7fd0881040f0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fd070009900 tx=0x7fd07002a410 comp rx=0 tx=0).stop 2024-05-20T16:11:51.560 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:51.557+0000 7fd08d21b640 1 -- 172.21.15.82:0/3066891022 shutdown_connections 2024-05-20T16:11:51.560 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:51.557+0000 7fd08d21b640 1 --2- 172.21.15.82:0/3066891022 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd088103ce0 0x7fd0881040f0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:51.560 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:51.557+0000 7fd08d21b640 1 --2- 172.21.15.82:0/3066891022 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd0881089e0 0x7fd08810ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:51.560 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:51.557+0000 7fd08d21b640 1 --2- 172.21.15.82:0/3066891022 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd088102aa0 0x7fd0881037a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:51.560 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:51.557+0000 7fd08d21b640 1 -- 172.21.15.82:0/3066891022 >> 172.21.15.82:0/3066891022 conn(0x7fd0880fdc90 msgr2=0x7fd088100140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:11:51.560 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:51.557+0000 7fd08d21b640 1 -- 172.21.15.82:0/3066891022 shutdown_connections 2024-05-20T16:11:51.560 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:51.557+0000 7fd08d21b640 1 -- 172.21.15.82:0/3066891022 wait complete. 2024-05-20T16:11:51.560 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:51.558+0000 7fd08d21b640 1 Processor -- start 2024-05-20T16:11:51.561 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:51.558+0000 7fd08d21b640 1 -- start start 2024-05-20T16:11:51.561 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:51.558+0000 7fd08d21b640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd088102aa0 0x7fd088199330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:11:51.561 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:51.558+0000 7fd08d21b640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd088103ce0 0x7fd088199870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:11:51.561 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:51.558+0000 7fd08d21b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd0881089e0 0x7fd0881a0800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:11:51.561 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:51.558+0000 7fd08d21b640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fd08810e790 con 0x7fd0881089e0 2024-05-20T16:11:51.561 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:51.558+0000 7fd08d21b640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fd08810e610 con 0x7fd088102aa0 2024-05-20T16:11:51.561 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:51.558+0000 7fd08d21b640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd08810e910 con 0x7fd088103ce0 2024-05-20T16:11:51.561 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:51.558+0000 7fd08759e640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd0881089e0 0x7fd0881a0800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:11:51.561 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:51.558+0000 7fd08759e640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd0881089e0 0x7fd0881a0800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:56238/0 (socket says 172.21.15.82:56238) 2024-05-20T16:11:51.561 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:51.558+0000 7fd08759e640 1 -- 172.21.15.82:0/2752500248 learned_addr learned my addr 172.21.15.82:0/2752500248 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:11:51.562 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:51.558+0000 7fd086d9d640 1 --2- 172.21.15.82:0/2752500248 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd088102aa0 0x7fd088199330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:11:51.562 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:51.558+0000 7fd08659c640 1 --2- 172.21.15.82:0/2752500248 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd088103ce0 0x7fd088199870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:11:51.562 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:51.559+0000 7fd08759e640 1 -- 172.21.15.82:0/2752500248 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd088103ce0 msgr2=0x7fd088199870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:11:51.562 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:51.559+0000 7fd08759e640 1 --2- 172.21.15.82:0/2752500248 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd088103ce0 0x7fd088199870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:51.562 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:51.559+0000 7fd08759e640 1 -- 172.21.15.82:0/2752500248 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd088102aa0 msgr2=0x7fd088199330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:11:51.562 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:51.559+0000 7fd08759e640 1 --2- 172.21.15.82:0/2752500248 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd088102aa0 0x7fd088199330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:51.562 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:51.559+0000 7fd08759e640 1 -- 172.21.15.82:0/2752500248 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0881a0e70 con 0x7fd0881089e0 2024-05-20T16:11:51.562 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:51.559+0000 7fd08759e640 1 --2- 172.21.15.82:0/2752500248 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd0881089e0 0x7fd0881a0800 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fd078009fd0 tx=0x7fd07800b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:11:51.562 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:51.559+0000 7fd07ffff640 1 -- 172.21.15.82:0/2752500248 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd078011830 con 0x7fd0881089e0 2024-05-20T16:11:51.563 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:51.560+0000 7fd07ffff640 1 -- 172.21.15.82:0/2752500248 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd078016070 con 0x7fd0881089e0 2024-05-20T16:11:51.563 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:51.560+0000 7fd08d21b640 1 -- 172.21.15.82:0/2752500248 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0881a1160 con 0x7fd0881089e0 2024-05-20T16:11:51.563 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:51.560+0000 7fd07ffff640 1 -- 172.21.15.82:0/2752500248 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd07800f2e0 con 0x7fd0881089e0 2024-05-20T16:11:51.563 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:51.560+0000 7fd08d21b640 1 -- 172.21.15.82:0/2752500248 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0881a1730 con 0x7fd0881089e0 2024-05-20T16:11:51.564 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:51.561+0000 7fd08d21b640 1 -- 172.21.15.82:0/2752500248 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0881077f0 con 0x7fd0881089e0 2024-05-20T16:11:51.564 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:51.562+0000 7fd07ffff640 1 -- 172.21.15.82:0/2752500248 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fd0780119d0 con 0x7fd0881089e0 2024-05-20T16:11:51.564 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:51.563+0000 7fd07ffff640 1 --2- 172.21.15.82:0/2752500248 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fd0740a2c10 0x7fd0740a5120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:11:51.564 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:51.563+0000 7fd086d9d640 1 --2- 172.21.15.82:0/2752500248 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fd0740a2c10 0x7fd0740a5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:11:51.564 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:51.563+0000 7fd07ffff640 1 -- 172.21.15.82:0/2752500248 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8770+0+0 (secure 0 0 0) 0x7fd0780cee90 con 0x7fd0881089e0 2024-05-20T16:11:51.564 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:51.563+0000 7fd086d9d640 1 --2- 172.21.15.82:0/2752500248 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fd0740a2c10 0x7fd0740a5120 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fd080005c40 tx=0x7fd08000a590 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:11:51.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:51.569+0000 7fd07ffff640 1 -- 172.21.15.82:0/2752500248 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fd07809bbd0 con 0x7fd0881089e0 2024-05-20T16:11:51.786 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:51.786+0000 7fd08d21b640 1 -- 172.21.15.82:0/2752500248 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) -- 0x7fd08802d2a0 con 0x7fd0881089e0 2024-05-20T16:11:51.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:51 smithi160 ceph-mon[36347]: from='client.? 172.21.15.82:0/3659450028' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-05-20T16:11:51.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:51 smithi160 ceph-mon[36347]: osdmap e88: 12 total, 12 up, 12 in 2024-05-20T16:11:51.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:51 smithi160 ceph-mon[36347]: from='client.? 172.21.15.82:0/2451145426' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-05-20T16:11:51.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:51 smithi160 ceph-mon[36347]: pgmap v291: 129 pgs: 32 unknown, 10 creating+peering, 87 active+clean; 578 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s wr, 9 op/s 2024-05-20T16:11:51.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:51 smithi082 ceph-mon[32238]: from='client.? 172.21.15.82:0/3659450028' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-05-20T16:11:51.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:51 smithi082 ceph-mon[32238]: osdmap e88: 12 total, 12 up, 12 in 2024-05-20T16:11:51.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:51 smithi082 ceph-mon[32238]: from='client.? 172.21.15.82:0/2451145426' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-05-20T16:11:51.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:51 smithi082 ceph-mon[32238]: pgmap v291: 129 pgs: 32 unknown, 10 creating+peering, 87 active+clean; 578 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s wr, 9 op/s 2024-05-20T16:11:51.886 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:51 smithi184 ceph-mon[36209]: from='client.? 172.21.15.82:0/3659450028' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-05-20T16:11:51.886 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:51 smithi184 ceph-mon[36209]: osdmap e88: 12 total, 12 up, 12 in 2024-05-20T16:11:51.886 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:51 smithi184 ceph-mon[36209]: from='client.? 172.21.15.82:0/2451145426' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-05-20T16:11:51.886 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:51 smithi184 ceph-mon[36209]: pgmap v291: 129 pgs: 32 unknown, 10 creating+peering, 87 active+clean; 578 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s wr, 9 op/s 2024-05-20T16:11:52.563 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:52.563+0000 7fd07ffff640 1 -- 172.21.15.82:0/2752500248 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v17) ==== 137+0+0 (secure 0 0 0) 0x7fd07809b4d0 con 0x7fd0881089e0 2024-05-20T16:11:52.563 INFO:teuthology.orchestra.run.smithi082.stderr:added data pool 4 to fsmap 2024-05-20T16:11:52.565 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:52.565+0000 7fd08d21b640 1 -- 172.21.15.82:0/2752500248 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fd0740a2c10 msgr2=0x7fd0740a5120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:11:52.566 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:52.565+0000 7fd08d21b640 1 --2- 172.21.15.82:0/2752500248 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fd0740a2c10 0x7fd0740a5120 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fd080005c40 tx=0x7fd08000a590 comp rx=0 tx=0).stop 2024-05-20T16:11:52.566 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:52.565+0000 7fd08d21b640 1 -- 172.21.15.82:0/2752500248 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd0881089e0 msgr2=0x7fd0881a0800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:11:52.566 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:52.565+0000 7fd08d21b640 1 --2- 172.21.15.82:0/2752500248 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd0881089e0 0x7fd0881a0800 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fd078009fd0 tx=0x7fd07800b8e0 comp rx=0 tx=0).stop 2024-05-20T16:11:52.566 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:52.565+0000 7fd08d21b640 1 -- 172.21.15.82:0/2752500248 shutdown_connections 2024-05-20T16:11:52.566 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:52.565+0000 7fd08d21b640 1 --2- 172.21.15.82:0/2752500248 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fd0740a2c10 0x7fd0740a5120 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:52.566 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:52.565+0000 7fd08d21b640 1 --2- 172.21.15.82:0/2752500248 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd0881089e0 0x7fd0881a0800 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:52.566 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:52.565+0000 7fd08d21b640 1 --2- 172.21.15.82:0/2752500248 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd088103ce0 0x7fd088199870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:52.566 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:52.565+0000 7fd08d21b640 1 --2- 172.21.15.82:0/2752500248 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd088102aa0 0x7fd088199330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:52.566 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:52.565+0000 7fd08d21b640 1 -- 172.21.15.82:0/2752500248 >> 172.21.15.82:0/2752500248 conn(0x7fd0880fdc90 msgr2=0x7fd0881049c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:11:52.566 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:52.566+0000 7fd08d21b640 1 -- 172.21.15.82:0/2752500248 shutdown_connections 2024-05-20T16:11:52.566 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:52.566+0000 7fd08d21b640 1 -- 172.21.15.82:0/2752500248 wait complete. 2024-05-20T16:11:52.579 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-20T16:11:52.696 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:52.695+0000 7f3980efe640 1 -- 172.21.15.82:0/2372008123 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f397c102c00 msgr2=0x7f397c103010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:11:52.696 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:52.695+0000 7f3980efe640 1 --2- 172.21.15.82:0/2372008123 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f397c102c00 0x7f397c103010 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f3964009900 tx=0x7f396402a3f0 comp rx=0 tx=0).stop 2024-05-20T16:11:52.696 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:52.696+0000 7f3980efe640 1 -- 172.21.15.82:0/2372008123 shutdown_connections 2024-05-20T16:11:52.696 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:52.696+0000 7f3980efe640 1 --2- 172.21.15.82:0/2372008123 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f397c102c00 0x7f397c103010 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:52.696 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:52.696+0000 7f3980efe640 1 --2- 172.21.15.82:0/2372008123 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f397c107900 0x7f397c109d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:52.696 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:52.696+0000 7f3980efe640 1 --2- 172.21.15.82:0/2372008123 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f397c1019c0 0x7f397c1026c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:52.696 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:52.696+0000 7f3980efe640 1 -- 172.21.15.82:0/2372008123 >> 172.21.15.82:0/2372008123 conn(0x7f397c0fcbb0 msgr2=0x7f397c0ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:11:52.697 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:52.696+0000 7f3980efe640 1 -- 172.21.15.82:0/2372008123 shutdown_connections 2024-05-20T16:11:52.697 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:52.696+0000 7f3980efe640 1 -- 172.21.15.82:0/2372008123 wait complete. 2024-05-20T16:11:52.697 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:52.696+0000 7f3980efe640 1 Processor -- start 2024-05-20T16:11:52.699 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:52.697+0000 7f3980efe640 1 -- start start 2024-05-20T16:11:52.700 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:52.697+0000 7f3980efe640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f397c1019c0 0x7f397c198230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:11:52.700 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:52.697+0000 7f3980efe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f397c102c00 0x7f397c198770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:11:52.700 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:52.697+0000 7f3980efe640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f397c107900 0x7f397c19f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:11:52.700 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:52.697+0000 7f3980efe640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f397c10d690 con 0x7f397c102c00 2024-05-20T16:11:52.700 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:52.697+0000 7f3980efe640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f397c10d510 con 0x7f397c1019c0 2024-05-20T16:11:52.700 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:52.697+0000 7f3980efe640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f397c10d810 con 0x7f397c107900 2024-05-20T16:11:52.700 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:52.697+0000 7f3971d9b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f397c102c00 0x7f397c198770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:11:52.700 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:52.697+0000 7f3971d9b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f397c102c00 0x7f397c198770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:56260/0 (socket says 172.21.15.82:56260) 2024-05-20T16:11:52.700 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:52.697+0000 7f3971d9b640 1 -- 172.21.15.82:0/3114230317 learned_addr learned my addr 172.21.15.82:0/3114230317 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:11:52.700 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:52.697+0000 7f397ad9d640 1 --2- 172.21.15.82:0/3114230317 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f397c107900 0x7f397c19f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:11:52.700 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:52.697+0000 7f397a59c640 1 --2- 172.21.15.82:0/3114230317 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f397c1019c0 0x7f397c198230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:11:52.701 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:52.697+0000 7f3971d9b640 1 -- 172.21.15.82:0/3114230317 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f397c107900 msgr2=0x7f397c19f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:11:52.701 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:52.697+0000 7f3971d9b640 1 --2- 172.21.15.82:0/3114230317 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f397c107900 0x7f397c19f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:52.701 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:52.697+0000 7f3971d9b640 1 -- 172.21.15.82:0/3114230317 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f397c1019c0 msgr2=0x7f397c198230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:11:52.701 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:52.697+0000 7f3971d9b640 1 --2- 172.21.15.82:0/3114230317 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f397c1019c0 0x7f397c198230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:52.701 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:52.697+0000 7f3971d9b640 1 -- 172.21.15.82:0/3114230317 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f397c19fcc0 con 0x7f397c102c00 2024-05-20T16:11:52.701 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:52.697+0000 7f3971d9b640 1 --2- 172.21.15.82:0/3114230317 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f397c102c00 0x7f397c198770 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f3964031610 tx=0x7f3964032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:11:52.701 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:52.698+0000 7f3973fff640 1 -- 172.21.15.82:0/3114230317 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f396403f030 con 0x7f397c102c00 2024-05-20T16:11:52.701 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:52.698+0000 7f3973fff640 1 -- 172.21.15.82:0/3114230317 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3964003ce0 con 0x7f397c102c00 2024-05-20T16:11:52.701 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:52.698+0000 7f3980efe640 1 -- 172.21.15.82:0/3114230317 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f397c19ff50 con 0x7f397c102c00 2024-05-20T16:11:52.701 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:52.698+0000 7f3973fff640 1 -- 172.21.15.82:0/3114230317 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3964035070 con 0x7f397c102c00 2024-05-20T16:11:52.701 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:52.698+0000 7f3980efe640 1 -- 172.21.15.82:0/3114230317 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f397c1a0360 con 0x7f397c102c00 2024-05-20T16:11:52.701 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:52.699+0000 7f3980efe640 1 -- 172.21.15.82:0/3114230317 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f397c0682c0 con 0x7f397c102c00 2024-05-20T16:11:52.701 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:52.700+0000 7f3973fff640 1 -- 172.21.15.82:0/3114230317 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f3964042030 con 0x7f397c102c00 2024-05-20T16:11:52.702 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:52.701+0000 7f3973fff640 1 --2- 172.21.15.82:0/3114230317 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f39680a2b90 0x7f39680a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:11:52.702 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:52.701+0000 7f397a59c640 1 --2- 172.21.15.82:0/3114230317 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f39680a2b90 0x7f39680a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:11:52.702 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:52.701+0000 7f3973fff640 1 -- 172.21.15.82:0/3114230317 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f39640ef9d0 con 0x7f397c102c00 2024-05-20T16:11:52.702 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:52.701+0000 7f397a59c640 1 --2- 172.21.15.82:0/3114230317 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f39680a2b90 0x7f39680a50a0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f396c005fd0 tx=0x7f396c005ce0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:11:52.705 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:52.706+0000 7f3973fff640 1 -- 172.21.15.82:0/3114230317 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f39640bc6f0 con 0x7f397c102c00 2024-05-20T16:11:52.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:52 smithi160 ceph-mon[36347]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-20T16:11:52.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:52 smithi160 ceph-mon[36347]: from='client.? 172.21.15.82:0/2451145426' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-05-20T16:11:52.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:52 smithi160 ceph-mon[36347]: osdmap e89: 12 total, 12 up, 12 in 2024-05-20T16:11:52.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:52 smithi160 ceph-mon[36347]: from='client.? 172.21.15.82:0/2752500248' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-05-20T16:11:52.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:52 smithi082 ceph-mon[32238]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-20T16:11:52.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:52 smithi082 ceph-mon[32238]: from='client.? 172.21.15.82:0/2451145426' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-05-20T16:11:52.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:52 smithi082 ceph-mon[32238]: osdmap e89: 12 total, 12 up, 12 in 2024-05-20T16:11:52.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:52 smithi082 ceph-mon[32238]: from='client.? 172.21.15.82:0/2752500248' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-05-20T16:11:52.886 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:52 smithi184 ceph-mon[36209]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-20T16:11:52.886 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:52 smithi184 ceph-mon[36209]: from='client.? 172.21.15.82:0/2451145426' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-05-20T16:11:52.886 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:52 smithi184 ceph-mon[36209]: osdmap e89: 12 total, 12 up, 12 in 2024-05-20T16:11:52.886 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:52 smithi184 ceph-mon[36209]: from='client.? 172.21.15.82:0/2752500248' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-05-20T16:11:52.917 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:52.917+0000 7f3980efe640 1 -- 172.21.15.82:0/3114230317 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f397c106710 con 0x7f397c102c00 2024-05-20T16:11:52.918 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:52.918+0000 7f3973fff640 1 -- 172.21.15.82:0/3114230317 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+11806 (secure 0 0 0) 0x7f39640bbff0 con 0x7f397c102c00 2024-05-20T16:11:52.918 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-05-20T16:11:52.918 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/2044534217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":2044534217},{"type":"v1","addr":"172.21.15.82:6835","nonce":2044534217}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/2635635097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":2635635097},{"type":"v1","addr":"172.21.15.82:6837","nonce":2635635097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14625,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6839/2118233541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6838","nonce":2118233541},{"type":"v1","addr":"172.21.15.82:6839","nonce":2118233541}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14637,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6841/2119517973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6840","nonce":2119517973},{"type":"v1","addr":"172.21.15.82:6841","nonce":2119517973}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6833/4015144007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":4015144007},{"type":"v1","addr":"172.21.15.184:6833","nonce":4015144007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6833/975243372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6832","nonce":975243372},{"type":"v1","addr":"172.21.15.160:6833","nonce":975243372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24470,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6835/664068363","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":664068363},{"type":"v1","addr":"172.21.15.160:6835","nonce":664068363}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24478,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/4242156090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":4242156090},{"type":"v1","addr":"172.21.15.184:6835","nonce":4242156090}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24496,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/403262242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":403262242},{"type":"v1","addr":"172.21.15.184:6837","nonce":403262242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24506,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6837/2846188680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6836","nonce":2846188680},{"type":"v1","addr":"172.21.15.160:6837","nonce":2846188680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24511,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6839/2199955067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":2199955067},{"type":"v1","addr":"172.21.15.184:6839","nonce":2199955067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-20T16:11:46.751632+0000","modified":"2024-05-20T16:11:51.788350+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24548},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24548":{"gid":24548,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.160:6839/2765554482","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6838","nonce":2765554482},{"type":"v1","addr":"172.21.15.160:6839","nonce":2765554482}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24548,"qdb_cluster":[24548]},"id":1}]} 2024-05-20T16:11:52.919 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 17 2024-05-20T16:11:52.920 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:52.920+0000 7f3980efe640 1 -- 172.21.15.82:0/3114230317 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f39680a2b90 msgr2=0x7f39680a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:11:52.920 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:52.920+0000 7f3980efe640 1 --2- 172.21.15.82:0/3114230317 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f39680a2b90 0x7f39680a50a0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f396c005fd0 tx=0x7f396c005ce0 comp rx=0 tx=0).stop 2024-05-20T16:11:52.920 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:52.920+0000 7f3980efe640 1 -- 172.21.15.82:0/3114230317 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f397c102c00 msgr2=0x7f397c198770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:11:52.920 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:52.920+0000 7f3980efe640 1 --2- 172.21.15.82:0/3114230317 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f397c102c00 0x7f397c198770 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f3964031610 tx=0x7f3964032ce0 comp rx=0 tx=0).stop 2024-05-20T16:11:52.920 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:52.920+0000 7f3980efe640 1 -- 172.21.15.82:0/3114230317 shutdown_connections 2024-05-20T16:11:52.920 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:52.920+0000 7f3980efe640 1 --2- 172.21.15.82:0/3114230317 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f39680a2b90 0x7f39680a50a0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:52.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:52.920+0000 7f3980efe640 1 --2- 172.21.15.82:0/3114230317 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f397c102c00 0x7f397c198770 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:52.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:52.920+0000 7f3980efe640 1 --2- 172.21.15.82:0/3114230317 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f397c107900 0x7f397c19f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:52.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:52.920+0000 7f3980efe640 1 --2- 172.21.15.82:0/3114230317 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f397c1019c0 0x7f397c198230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:52.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:52.920+0000 7f3980efe640 1 -- 172.21.15.82:0/3114230317 >> 172.21.15.82:0/3114230317 conn(0x7f397c0fcbb0 msgr2=0x7f397c1038e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:11:52.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:52.920+0000 7f3980efe640 1 -- 172.21.15.82:0/3114230317 shutdown_connections 2024-05-20T16:11:52.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:52.920+0000 7f3980efe640 1 -- 172.21.15.82:0/3114230317 wait complete. 2024-05-20T16:11:52.932 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-20T16:11:53.046 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:53.044+0000 7f9193502640 1 -- 172.21.15.82:0/4074630544 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f918c101140 msgr2=0x7f918c101e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:11:53.046 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:53.044+0000 7f9193502640 1 --2- 172.21.15.82:0/4074630544 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f918c101140 0x7f918c101e40 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f9188009900 tx=0x7f918802a410 comp rx=0 tx=0).stop 2024-05-20T16:11:53.046 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:53.044+0000 7f9193502640 1 -- 172.21.15.82:0/4074630544 shutdown_connections 2024-05-20T16:11:53.046 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:53.044+0000 7f9193502640 1 --2- 172.21.15.82:0/4074630544 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f918c107080 0x7f918c1094a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:53.046 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:53.044+0000 7f9193502640 1 --2- 172.21.15.82:0/4074630544 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f918c102380 0x7f918c102790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:53.046 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:53.044+0000 7f9193502640 1 --2- 172.21.15.82:0/4074630544 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f918c101140 0x7f918c101e40 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:53.046 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:53.044+0000 7f9193502640 1 -- 172.21.15.82:0/4074630544 >> 172.21.15.82:0/4074630544 conn(0x7f918c0fc310 msgr2=0x7f918c0fe7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:11:53.046 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:53.044+0000 7f9193502640 1 -- 172.21.15.82:0/4074630544 shutdown_connections 2024-05-20T16:11:53.047 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:53.044+0000 7f9193502640 1 -- 172.21.15.82:0/4074630544 wait complete. 2024-05-20T16:11:53.047 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:53.044+0000 7f9193502640 1 Processor -- start 2024-05-20T16:11:53.047 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:53.044+0000 7f9193502640 1 -- start start 2024-05-20T16:11:53.047 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:53.045+0000 7f9193502640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f918c101140 0x7f918c199270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:11:53.047 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:53.045+0000 7f9193502640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f918c102380 0x7f918c1997b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:11:53.047 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:53.045+0000 7f9193502640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f918c107080 0x7f918c1a0740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:11:53.047 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:53.045+0000 7f9193502640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f918c10cdb0 con 0x7f918c107080 2024-05-20T16:11:53.047 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:53.045+0000 7f9193502640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f918c10cc30 con 0x7f918c102380 2024-05-20T16:11:53.047 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:53.045+0000 7f9193502640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f918c10cf30 con 0x7f918c101140 2024-05-20T16:11:53.047 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:53.045+0000 7f9191a9f640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f918c107080 0x7f918c1a0740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:11:53.047 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:53.045+0000 7f9191a9f640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f918c107080 0x7f918c1a0740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:56276/0 (socket says 172.21.15.82:56276) 2024-05-20T16:11:53.047 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:53.045+0000 7f9191a9f640 1 -- 172.21.15.82:0/19149921 learned_addr learned my addr 172.21.15.82:0/19149921 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:11:53.047 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:53.045+0000 7f9190a9d640 1 --2- 172.21.15.82:0/19149921 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f918c102380 0x7f918c1997b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:11:53.047 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:53.045+0000 7f919129e640 1 --2- 172.21.15.82:0/19149921 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f918c101140 0x7f918c199270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:11:53.048 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:53.045+0000 7f9191a9f640 1 -- 172.21.15.82:0/19149921 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f918c101140 msgr2=0x7f918c199270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:11:53.048 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:53.045+0000 7f9191a9f640 1 --2- 172.21.15.82:0/19149921 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f918c101140 0x7f918c199270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:53.048 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:53.045+0000 7f9191a9f640 1 -- 172.21.15.82:0/19149921 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f918c102380 msgr2=0x7f918c1997b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:11:53.048 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:53.045+0000 7f9191a9f640 1 --2- 172.21.15.82:0/19149921 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f918c102380 0x7f918c1997b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:53.048 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:53.045+0000 7f9191a9f640 1 -- 172.21.15.82:0/19149921 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f918c1a0db0 con 0x7f918c107080 2024-05-20T16:11:53.048 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:53.045+0000 7f919129e640 1 --2- 172.21.15.82:0/19149921 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f918c101140 0x7f918c199270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T16:11:53.048 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:53.045+0000 7f9191a9f640 1 --2- 172.21.15.82:0/19149921 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f918c107080 0x7f918c1a0740 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f918400cad0 tx=0x7f918400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:11:53.048 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:53.045+0000 7f91827fc640 1 -- 172.21.15.82:0/19149921 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f918400ae20 con 0x7f918c107080 2024-05-20T16:11:53.048 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:53.045+0000 7f91827fc640 1 -- 172.21.15.82:0/19149921 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9184005020 con 0x7f918c107080 2024-05-20T16:11:53.048 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:53.045+0000 7f9193502640 1 -- 172.21.15.82:0/19149921 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f918c1a10a0 con 0x7f918c107080 2024-05-20T16:11:53.048 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:53.045+0000 7f91827fc640 1 -- 172.21.15.82:0/19149921 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f918400f6f0 con 0x7f918c107080 2024-05-20T16:11:53.048 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:53.045+0000 7f9193502640 1 -- 172.21.15.82:0/19149921 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f918c1a15a0 con 0x7f918c107080 2024-05-20T16:11:53.049 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:53.047+0000 7f9193502640 1 -- 172.21.15.82:0/19149921 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f918c106780 con 0x7f918c107080 2024-05-20T16:11:53.049 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:53.048+0000 7f91827fc640 1 -- 172.21.15.82:0/19149921 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f91840051d0 con 0x7f918c107080 2024-05-20T16:11:53.049 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:53.049+0000 7f91827fc640 1 --2- 172.21.15.82:0/19149921 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f91780a2b90 0x7f91780a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:11:53.049 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:53.049+0000 7f919129e640 1 --2- 172.21.15.82:0/19149921 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f91780a2b90 0x7f91780a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:11:53.050 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:53.049+0000 7f91827fc640 1 -- 172.21.15.82:0/19149921 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f91840cf740 con 0x7f918c107080 2024-05-20T16:11:53.050 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:53.049+0000 7f919129e640 1 --2- 172.21.15.82:0/19149921 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f91780a2b90 0x7f91780a50a0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f9188000c00 tx=0x7f9188031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:11:53.052 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:53.052+0000 7f91827fc640 1 -- 172.21.15.82:0/19149921 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f9184014030 con 0x7f918c107080 2024-05-20T16:11:53.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:53.229+0000 7f9193502640 1 -- 172.21.15.82:0/19149921 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f918c062750 con 0x7f918c107080 2024-05-20T16:11:53.231 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:53.231+0000 7f91827fc640 1 -- 172.21.15.82:0/19149921 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24166 (secure 0 0 0) 0x7f918409c460 con 0x7f918c107080 2024-05-20T16:11:53.231 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-05-20T16:11:53.231 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":90,"fsid":"59d1600a-16c2-11ef-bc9a-c7b262605968","created":"2024-05-20T16:03:46.611556+0000","modified":"2024-05-20T16:11:51.788081+0000","last_up_change":"2024-05-20T16:10:59.303479+0000","last_in_change":"2024-05-20T16:10:39.063561+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-20T16:07:09.493727+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-20T16:11:45.058035+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-20T16:11:45.943862+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-20T16:11:49.637579+0000","flags":32773,"flags_names":"hashpspool,ec_overwrites,creating","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"bae842a9-03ec-490e-b56d-f7957235cdc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":1484330167},{"type":"v1","addr":"172.21.15.82:6803","nonce":1484330167}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":1484330167},{"type":"v1","addr":"172.21.15.82:6805","nonce":1484330167}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1484330167},{"type":"v1","addr":"172.21.15.82:6809","nonce":1484330167}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":1484330167},{"type":"v1","addr":"172.21.15.82:6807","nonce":1484330167}]},"public_addr":"172.21.15.82:6803/1484330167","cluster_addr":"172.21.15.82:6805/1484330167","heartbeat_back_addr":"172.21.15.82:6809/1484330167","heartbeat_front_addr":"172.21.15.82:6807/1484330167","state":["exists","up"]},{"osd":1,"uuid":"59f6a2ba-b5de-4161-81c5-92a417abacdf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":1457003568},{"type":"v1","addr":"172.21.15.160:6801","nonce":1457003568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":1457003568},{"type":"v1","addr":"172.21.15.160:6803","nonce":1457003568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6806","nonce":1457003568},{"type":"v1","addr":"172.21.15.160:6807","nonce":1457003568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":1457003568},{"type":"v1","addr":"172.21.15.160:6805","nonce":1457003568}]},"public_addr":"172.21.15.160:6801/1457003568","cluster_addr":"172.21.15.160:6803/1457003568","heartbeat_back_addr":"172.21.15.160:6807/1457003568","heartbeat_front_addr":"172.21.15.160:6805/1457003568","state":["exists","up"]},{"osd":2,"uuid":"3bbb5587-b449-41f3-bd26-67ec3d6afdfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":4051293934},{"type":"v1","addr":"172.21.15.184:6801","nonce":4051293934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":4051293934},{"type":"v1","addr":"172.21.15.184:6803","nonce":4051293934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":4051293934},{"type":"v1","addr":"172.21.15.184:6807","nonce":4051293934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":4051293934},{"type":"v1","addr":"172.21.15.184:6805","nonce":4051293934}]},"public_addr":"172.21.15.184:6801/4051293934","cluster_addr":"172.21.15.184:6803/4051293934","heartbeat_back_addr":"172.21.15.184:6807/4051293934","heartbeat_front_addr":"172.21.15.184:6805/4051293934","state":["exists","up"]},{"osd":3,"uuid":"ca651f90-09b8-4081-a6f9-40b31268a837","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":3947485599},{"type":"v1","addr":"172.21.15.82:6811","nonce":3947485599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":3947485599},{"type":"v1","addr":"172.21.15.82:6813","nonce":3947485599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":3947485599},{"type":"v1","addr":"172.21.15.82:6817","nonce":3947485599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":3947485599},{"type":"v1","addr":"172.21.15.82:6815","nonce":3947485599}]},"public_addr":"172.21.15.82:6811/3947485599","cluster_addr":"172.21.15.82:6813/3947485599","heartbeat_back_addr":"172.21.15.82:6817/3947485599","heartbeat_front_addr":"172.21.15.82:6815/3947485599","state":["exists","up"]},{"osd":4,"uuid":"cb3dfd56-f677-40cb-9c96-8319ea5ac788","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":4160830014},{"type":"v1","addr":"172.21.15.160:6809","nonce":4160830014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":4160830014},{"type":"v1","addr":"172.21.15.160:6811","nonce":4160830014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":4160830014},{"type":"v1","addr":"172.21.15.160:6815","nonce":4160830014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6812","nonce":4160830014},{"type":"v1","addr":"172.21.15.160:6813","nonce":4160830014}]},"public_addr":"172.21.15.160:6809/4160830014","cluster_addr":"172.21.15.160:6811/4160830014","heartbeat_back_addr":"172.21.15.160:6815/4160830014","heartbeat_front_addr":"172.21.15.160:6813/4160830014","state":["exists","up"]},{"osd":5,"uuid":"c70cfcd1-a4ff-461f-8c73-479b061200fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":1046282672},{"type":"v1","addr":"172.21.15.184:6809","nonce":1046282672}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":1046282672},{"type":"v1","addr":"172.21.15.184:6811","nonce":1046282672}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":1046282672},{"type":"v1","addr":"172.21.15.184:6815","nonce":1046282672}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":1046282672},{"type":"v1","addr":"172.21.15.184:6813","nonce":1046282672}]},"public_addr":"172.21.15.184:6809/1046282672","cluster_addr":"172.21.15.184:6811/1046282672","heartbeat_back_addr":"172.21.15.184:6815/1046282672","heartbeat_front_addr":"172.21.15.184:6813/1046282672","state":["exists","up"]},{"osd":6,"uuid":"d47ce001-9dd1-4830-bcc9-888601187c6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":2789736499},{"type":"v1","addr":"172.21.15.82:6819","nonce":2789736499}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":2789736499},{"type":"v1","addr":"172.21.15.82:6821","nonce":2789736499}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":2789736499},{"type":"v1","addr":"172.21.15.82:6825","nonce":2789736499}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":2789736499},{"type":"v1","addr":"172.21.15.82:6823","nonce":2789736499}]},"public_addr":"172.21.15.82:6819/2789736499","cluster_addr":"172.21.15.82:6821/2789736499","heartbeat_back_addr":"172.21.15.82:6825/2789736499","heartbeat_front_addr":"172.21.15.82:6823/2789736499","state":["exists","up"]},{"osd":7,"uuid":"25c54f43-18cb-41e4-83b4-2732b5a93889","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6816","nonce":2340683378},{"type":"v1","addr":"172.21.15.160:6817","nonce":2340683378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":2340683378},{"type":"v1","addr":"172.21.15.160:6819","nonce":2340683378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":2340683378},{"type":"v1","addr":"172.21.15.160:6823","nonce":2340683378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6820","nonce":2340683378},{"type":"v1","addr":"172.21.15.160:6821","nonce":2340683378}]},"public_addr":"172.21.15.160:6817/2340683378","cluster_addr":"172.21.15.160:6819/2340683378","heartbeat_back_addr":"172.21.15.160:6823/2340683378","heartbeat_front_addr":"172.21.15.160:6821/2340683378","state":["exists","up"]},{"osd":8,"uuid":"1f8b478d-287f-45f5-8526-d1a58c5bc07c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":3831576372},{"type":"v1","addr":"172.21.15.184:6817","nonce":3831576372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":3831576372},{"type":"v1","addr":"172.21.15.184:6819","nonce":3831576372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":3831576372},{"type":"v1","addr":"172.21.15.184:6823","nonce":3831576372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":3831576372},{"type":"v1","addr":"172.21.15.184:6821","nonce":3831576372}]},"public_addr":"172.21.15.184:6817/3831576372","cluster_addr":"172.21.15.184:6819/3831576372","heartbeat_back_addr":"172.21.15.184:6823/3831576372","heartbeat_front_addr":"172.21.15.184:6821/3831576372","state":["exists","up"]},{"osd":9,"uuid":"0d69e319-954f-4b18-95ea-31794f18dc3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":716330782},{"type":"v1","addr":"172.21.15.82:6827","nonce":716330782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":716330782},{"type":"v1","addr":"172.21.15.82:6829","nonce":716330782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":716330782},{"type":"v1","addr":"172.21.15.82:6833","nonce":716330782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":716330782},{"type":"v1","addr":"172.21.15.82:6831","nonce":716330782}]},"public_addr":"172.21.15.82:6827/716330782","cluster_addr":"172.21.15.82:6829/716330782","heartbeat_back_addr":"172.21.15.82:6833/716330782","heartbeat_front_addr":"172.21.15.82:6831/716330782","state":["exists","up"]},{"osd":10,"uuid":"39dc1b78-61cb-42ca-b5b6-1e44ac2b6500","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":2327428014},{"type":"v1","addr":"172.21.15.160:6825","nonce":2327428014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":2327428014},{"type":"v1","addr":"172.21.15.160:6827","nonce":2327428014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":2327428014},{"type":"v1","addr":"172.21.15.160:6831","nonce":2327428014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":2327428014},{"type":"v1","addr":"172.21.15.160:6829","nonce":2327428014}]},"public_addr":"172.21.15.160:6825/2327428014","cluster_addr":"172.21.15.160:6827/2327428014","heartbeat_back_addr":"172.21.15.160:6831/2327428014","heartbeat_front_addr":"172.21.15.160:6829/2327428014","state":["exists","up"]},{"osd":11,"uuid":"4306b3c1-707a-4010-987c-209ea0e948e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":3594364650},{"type":"v1","addr":"172.21.15.184:6825","nonce":3594364650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":3594364650},{"type":"v1","addr":"172.21.15.184:6827","nonce":3594364650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3594364650},{"type":"v1","addr":"172.21.15.184:6831","nonce":3594364650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3594364650},{"type":"v1","addr":"172.21.15.184:6829","nonce":3594364650}]},"public_addr":"172.21.15.184:6825/3594364650","cluster_addr":"172.21.15.184:6827/3594364650","heartbeat_back_addr":"172.21.15.184:6831/3594364650","heartbeat_front_addr":"172.21.15.184:6829/3594364650","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:06:17.855998+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:06:41.473270+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:07:05.304108+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:07:29.887472+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:07:55.387286+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:08:20.579669+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:08:46.333051+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:09:12.289668+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:09:38.180696+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:10:04.471459+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:10:31.041912+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:10:57.477266+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.82:0/2005424509":"2024-05-21T16:04:45.161949+0000","172.21.15.82:6801/1763049308":"2024-05-21T16:04:45.161949+0000","172.21.15.82:6801/1959459348":"2024-05-21T16:04:19.208291+0000","172.21.15.82:0/3131904556":"2024-05-21T16:04:19.208291+0000","172.21.15.82:6800/1959459348":"2024-05-21T16:04:19.208291+0000","172.21.15.82:0/2456966342":"2024-05-21T16:04:45.161949+0000","172.21.15.82:0/4179978833":"2024-05-21T16:04:19.208291+0000","172.21.15.82:0/1602862225":"2024-05-21T16:04:45.161949+0000","172.21.15.82:0/106101180":"2024-05-21T16:04:19.208291+0000","172.21.15.82:6800/1763049308":"2024-05-21T16:04:45.161949+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-20T16:11:53.233 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:53.233+0000 7f9193502640 1 -- 172.21.15.82:0/19149921 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f91780a2b90 msgr2=0x7f91780a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:11:53.233 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:53.233+0000 7f9193502640 1 --2- 172.21.15.82:0/19149921 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f91780a2b90 0x7f91780a50a0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f9188000c00 tx=0x7f9188031400 comp rx=0 tx=0).stop 2024-05-20T16:11:53.233 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:53.233+0000 7f9193502640 1 -- 172.21.15.82:0/19149921 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f918c107080 msgr2=0x7f918c1a0740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:11:53.233 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:53.233+0000 7f9193502640 1 --2- 172.21.15.82:0/19149921 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f918c107080 0x7f918c1a0740 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f918400cad0 tx=0x7f918400cf90 comp rx=0 tx=0).stop 2024-05-20T16:11:53.234 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:53.233+0000 7f9193502640 1 -- 172.21.15.82:0/19149921 shutdown_connections 2024-05-20T16:11:53.234 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:53.233+0000 7f9193502640 1 --2- 172.21.15.82:0/19149921 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f91780a2b90 0x7f91780a50a0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:53.234 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:53.233+0000 7f9193502640 1 --2- 172.21.15.82:0/19149921 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f918c107080 0x7f918c1a0740 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:53.234 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:53.233+0000 7f9193502640 1 --2- 172.21.15.82:0/19149921 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f918c102380 0x7f918c1997b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:53.234 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:53.233+0000 7f9193502640 1 --2- 172.21.15.82:0/19149921 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f918c101140 0x7f918c199270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:53.234 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:53.233+0000 7f9193502640 1 -- 172.21.15.82:0/19149921 >> 172.21.15.82:0/19149921 conn(0x7f918c0fc310 msgr2=0x7f918c107c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:11:53.234 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:53.233+0000 7f9193502640 1 -- 172.21.15.82:0/19149921 shutdown_connections 2024-05-20T16:11:53.234 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:53.233+0000 7f9193502640 1 -- 172.21.15.82:0/19149921 wait complete. 2024-05-20T16:11:53.245 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-20T16:11:53.359 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:53.357+0000 7fa38f00d640 1 -- 172.21.15.82:0/2004656446 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa3881078b0 msgr2=0x7fa388109cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:11:53.359 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:53.357+0000 7fa38f00d640 1 --2- 172.21.15.82:0/2004656446 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa3881078b0 0x7fa388109cd0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fa380009ca0 tx=0x7fa38002a8c0 comp rx=0 tx=0).stop 2024-05-20T16:11:53.359 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:53.357+0000 7fa38f00d640 1 -- 172.21.15.82:0/2004656446 shutdown_connections 2024-05-20T16:11:53.359 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:53.357+0000 7fa38f00d640 1 --2- 172.21.15.82:0/2004656446 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa3881078b0 0x7fa388109cd0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:53.359 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:53.357+0000 7fa38f00d640 1 --2- 172.21.15.82:0/2004656446 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa388102bb0 0x7fa388102fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:53.359 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:53.357+0000 7fa38f00d640 1 --2- 172.21.15.82:0/2004656446 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa388101970 0x7fa388102670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:53.359 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:53.357+0000 7fa38f00d640 1 -- 172.21.15.82:0/2004656446 >> 172.21.15.82:0/2004656446 conn(0x7fa3880fcb80 msgr2=0x7fa3880ff010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:11:53.359 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:53.357+0000 7fa38f00d640 1 -- 172.21.15.82:0/2004656446 shutdown_connections 2024-05-20T16:11:53.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:53.357+0000 7fa38f00d640 1 -- 172.21.15.82:0/2004656446 wait complete. 2024-05-20T16:11:53.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:53.357+0000 7fa38f00d640 1 Processor -- start 2024-05-20T16:11:53.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:53.357+0000 7fa38f00d640 1 -- start start 2024-05-20T16:11:53.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:53.358+0000 7fa38f00d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa388101970 0x7fa3880698d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:11:53.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:53.358+0000 7fa38f00d640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa388102bb0 0x7fa38819dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:11:53.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:53.358+0000 7fa38f00d640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa3881078b0 0x7fa3881a0550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:11:53.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:53.358+0000 7fa38f00d640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fa38810d640 con 0x7fa388101970 2024-05-20T16:11:53.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:53.358+0000 7fa38f00d640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fa38810d4c0 con 0x7fa388102bb0 2024-05-20T16:11:53.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:53.358+0000 7fa38f00d640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa38810d7c0 con 0x7fa3881078b0 2024-05-20T16:11:53.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:53.358+0000 7fa38cda9640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa388101970 0x7fa3880698d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:11:53.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:53.358+0000 7fa38cda9640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa388101970 0x7fa3880698d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:56294/0 (socket says 172.21.15.82:56294) 2024-05-20T16:11:53.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:53.358+0000 7fa38cda9640 1 -- 172.21.15.82:0/3084002116 learned_addr learned my addr 172.21.15.82:0/3084002116 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:11:53.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:53.358+0000 7fa387fff640 1 --2- 172.21.15.82:0/3084002116 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa388102bb0 0x7fa38819dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:11:53.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:53.358+0000 7fa38d5aa640 1 --2- 172.21.15.82:0/3084002116 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa3881078b0 0x7fa3881a0550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:11:53.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:53.358+0000 7fa38cda9640 1 -- 172.21.15.82:0/3084002116 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa3881078b0 msgr2=0x7fa3881a0550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:11:53.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:53.358+0000 7fa38cda9640 1 --2- 172.21.15.82:0/3084002116 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa3881078b0 0x7fa3881a0550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:53.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:53.358+0000 7fa38cda9640 1 -- 172.21.15.82:0/3084002116 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa388102bb0 msgr2=0x7fa38819dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:11:53.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:53.358+0000 7fa38cda9640 1 --2- 172.21.15.82:0/3084002116 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa388102bb0 0x7fa38819dfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:53.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:53.358+0000 7fa38cda9640 1 -- 172.21.15.82:0/3084002116 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3881a0d70 con 0x7fa388101970 2024-05-20T16:11:53.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:53.358+0000 7fa38cda9640 1 --2- 172.21.15.82:0/3084002116 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa388101970 0x7fa3880698d0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fa37c005fd0 tx=0x7fa37c00dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:11:53.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:53.358+0000 7fa385ffb640 1 -- 172.21.15.82:0/3084002116 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa37c017070 con 0x7fa388101970 2024-05-20T16:11:53.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:53.358+0000 7fa38f00d640 1 -- 172.21.15.82:0/3084002116 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3881a1060 con 0x7fa388101970 2024-05-20T16:11:53.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:53.358+0000 7fa385ffb640 1 -- 172.21.15.82:0/3084002116 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa37c009c30 con 0x7fa388101970 2024-05-20T16:11:53.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:53.358+0000 7fa385ffb640 1 -- 172.21.15.82:0/3084002116 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa37c003e40 con 0x7fa388101970 2024-05-20T16:11:53.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:53.359+0000 7fa38f00d640 1 -- 172.21.15.82:0/3084002116 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3881a15b0 con 0x7fa388101970 2024-05-20T16:11:53.362 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:53.360+0000 7fa38f00d640 1 -- 172.21.15.82:0/3084002116 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa388105fb0 con 0x7fa388101970 2024-05-20T16:11:53.362 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:53.361+0000 7fa385ffb640 1 -- 172.21.15.82:0/3084002116 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fa37c004000 con 0x7fa388101970 2024-05-20T16:11:53.362 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:53.362+0000 7fa385ffb640 1 --2- 172.21.15.82:0/3084002116 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fa3780a2b90 0x7fa3780a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:11:53.362 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:53.362+0000 7fa387fff640 1 --2- 172.21.15.82:0/3084002116 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fa3780a2b90 0x7fa3780a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:11:53.362 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:53.362+0000 7fa385ffb640 1 -- 172.21.15.82:0/3084002116 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7fa37c0cf810 con 0x7fa388101970 2024-05-20T16:11:53.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:53.363+0000 7fa387fff640 1 --2- 172.21.15.82:0/3084002116 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fa3780a2b90 0x7fa3780a50a0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fa374005fd0 tx=0x7fa374005cf0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:11:53.366 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:53.366+0000 7fa385ffb640 1 -- 172.21.15.82:0/3084002116 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fa37c020030 con 0x7fa388101970 2024-05-20T16:11:53.552 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:53.552+0000 7fa38f00d640 1 -- 172.21.15.82:0/3084002116 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa38802d2a0 con 0x7fa388101970 2024-05-20T16:11:53.554 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:53.554+0000 7fa385ffb640 1 -- 172.21.15.82:0/3084002116 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+24153 (secure 0 0 0) 0x7fa37c09c510 con 0x7fa388101970 2024-05-20T16:11:53.554 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-05-20T16:11:53.554 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":91,"fsid":"59d1600a-16c2-11ef-bc9a-c7b262605968","created":"2024-05-20T16:03:46.611556+0000","modified":"2024-05-20T16:11:53.416085+0000","last_up_change":"2024-05-20T16:10:59.303479+0000","last_in_change":"2024-05-20T16:10:39.063561+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-20T16:07:09.493727+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-20T16:11:45.058035+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-20T16:11:45.943862+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-20T16:11:49.637579+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"bae842a9-03ec-490e-b56d-f7957235cdc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":1484330167},{"type":"v1","addr":"172.21.15.82:6803","nonce":1484330167}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":1484330167},{"type":"v1","addr":"172.21.15.82:6805","nonce":1484330167}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1484330167},{"type":"v1","addr":"172.21.15.82:6809","nonce":1484330167}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":1484330167},{"type":"v1","addr":"172.21.15.82:6807","nonce":1484330167}]},"public_addr":"172.21.15.82:6803/1484330167","cluster_addr":"172.21.15.82:6805/1484330167","heartbeat_back_addr":"172.21.15.82:6809/1484330167","heartbeat_front_addr":"172.21.15.82:6807/1484330167","state":["exists","up"]},{"osd":1,"uuid":"59f6a2ba-b5de-4161-81c5-92a417abacdf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":1457003568},{"type":"v1","addr":"172.21.15.160:6801","nonce":1457003568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":1457003568},{"type":"v1","addr":"172.21.15.160:6803","nonce":1457003568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6806","nonce":1457003568},{"type":"v1","addr":"172.21.15.160:6807","nonce":1457003568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":1457003568},{"type":"v1","addr":"172.21.15.160:6805","nonce":1457003568}]},"public_addr":"172.21.15.160:6801/1457003568","cluster_addr":"172.21.15.160:6803/1457003568","heartbeat_back_addr":"172.21.15.160:6807/1457003568","heartbeat_front_addr":"172.21.15.160:6805/1457003568","state":["exists","up"]},{"osd":2,"uuid":"3bbb5587-b449-41f3-bd26-67ec3d6afdfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":4051293934},{"type":"v1","addr":"172.21.15.184:6801","nonce":4051293934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":4051293934},{"type":"v1","addr":"172.21.15.184:6803","nonce":4051293934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":4051293934},{"type":"v1","addr":"172.21.15.184:6807","nonce":4051293934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":4051293934},{"type":"v1","addr":"172.21.15.184:6805","nonce":4051293934}]},"public_addr":"172.21.15.184:6801/4051293934","cluster_addr":"172.21.15.184:6803/4051293934","heartbeat_back_addr":"172.21.15.184:6807/4051293934","heartbeat_front_addr":"172.21.15.184:6805/4051293934","state":["exists","up"]},{"osd":3,"uuid":"ca651f90-09b8-4081-a6f9-40b31268a837","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":3947485599},{"type":"v1","addr":"172.21.15.82:6811","nonce":3947485599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":3947485599},{"type":"v1","addr":"172.21.15.82:6813","nonce":3947485599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":3947485599},{"type":"v1","addr":"172.21.15.82:6817","nonce":3947485599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":3947485599},{"type":"v1","addr":"172.21.15.82:6815","nonce":3947485599}]},"public_addr":"172.21.15.82:6811/3947485599","cluster_addr":"172.21.15.82:6813/3947485599","heartbeat_back_addr":"172.21.15.82:6817/3947485599","heartbeat_front_addr":"172.21.15.82:6815/3947485599","state":["exists","up"]},{"osd":4,"uuid":"cb3dfd56-f677-40cb-9c96-8319ea5ac788","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":4160830014},{"type":"v1","addr":"172.21.15.160:6809","nonce":4160830014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":4160830014},{"type":"v1","addr":"172.21.15.160:6811","nonce":4160830014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":4160830014},{"type":"v1","addr":"172.21.15.160:6815","nonce":4160830014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6812","nonce":4160830014},{"type":"v1","addr":"172.21.15.160:6813","nonce":4160830014}]},"public_addr":"172.21.15.160:6809/4160830014","cluster_addr":"172.21.15.160:6811/4160830014","heartbeat_back_addr":"172.21.15.160:6815/4160830014","heartbeat_front_addr":"172.21.15.160:6813/4160830014","state":["exists","up"]},{"osd":5,"uuid":"c70cfcd1-a4ff-461f-8c73-479b061200fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":1046282672},{"type":"v1","addr":"172.21.15.184:6809","nonce":1046282672}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":1046282672},{"type":"v1","addr":"172.21.15.184:6811","nonce":1046282672}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":1046282672},{"type":"v1","addr":"172.21.15.184:6815","nonce":1046282672}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":1046282672},{"type":"v1","addr":"172.21.15.184:6813","nonce":1046282672}]},"public_addr":"172.21.15.184:6809/1046282672","cluster_addr":"172.21.15.184:6811/1046282672","heartbeat_back_addr":"172.21.15.184:6815/1046282672","heartbeat_front_addr":"172.21.15.184:6813/1046282672","state":["exists","up"]},{"osd":6,"uuid":"d47ce001-9dd1-4830-bcc9-888601187c6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":2789736499},{"type":"v1","addr":"172.21.15.82:6819","nonce":2789736499}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":2789736499},{"type":"v1","addr":"172.21.15.82:6821","nonce":2789736499}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":2789736499},{"type":"v1","addr":"172.21.15.82:6825","nonce":2789736499}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":2789736499},{"type":"v1","addr":"172.21.15.82:6823","nonce":2789736499}]},"public_addr":"172.21.15.82:6819/2789736499","cluster_addr":"172.21.15.82:6821/2789736499","heartbeat_back_addr":"172.21.15.82:6825/2789736499","heartbeat_front_addr":"172.21.15.82:6823/2789736499","state":["exists","up"]},{"osd":7,"uuid":"25c54f43-18cb-41e4-83b4-2732b5a93889","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6816","nonce":2340683378},{"type":"v1","addr":"172.21.15.160:6817","nonce":2340683378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":2340683378},{"type":"v1","addr":"172.21.15.160:6819","nonce":2340683378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":2340683378},{"type":"v1","addr":"172.21.15.160:6823","nonce":2340683378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6820","nonce":2340683378},{"type":"v1","addr":"172.21.15.160:6821","nonce":2340683378}]},"public_addr":"172.21.15.160:6817/2340683378","cluster_addr":"172.21.15.160:6819/2340683378","heartbeat_back_addr":"172.21.15.160:6823/2340683378","heartbeat_front_addr":"172.21.15.160:6821/2340683378","state":["exists","up"]},{"osd":8,"uuid":"1f8b478d-287f-45f5-8526-d1a58c5bc07c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":3831576372},{"type":"v1","addr":"172.21.15.184:6817","nonce":3831576372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":3831576372},{"type":"v1","addr":"172.21.15.184:6819","nonce":3831576372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":3831576372},{"type":"v1","addr":"172.21.15.184:6823","nonce":3831576372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":3831576372},{"type":"v1","addr":"172.21.15.184:6821","nonce":3831576372}]},"public_addr":"172.21.15.184:6817/3831576372","cluster_addr":"172.21.15.184:6819/3831576372","heartbeat_back_addr":"172.21.15.184:6823/3831576372","heartbeat_front_addr":"172.21.15.184:6821/3831576372","state":["exists","up"]},{"osd":9,"uuid":"0d69e319-954f-4b18-95ea-31794f18dc3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":716330782},{"type":"v1","addr":"172.21.15.82:6827","nonce":716330782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":716330782},{"type":"v1","addr":"172.21.15.82:6829","nonce":716330782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":716330782},{"type":"v1","addr":"172.21.15.82:6833","nonce":716330782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":716330782},{"type":"v1","addr":"172.21.15.82:6831","nonce":716330782}]},"public_addr":"172.21.15.82:6827/716330782","cluster_addr":"172.21.15.82:6829/716330782","heartbeat_back_addr":"172.21.15.82:6833/716330782","heartbeat_front_addr":"172.21.15.82:6831/716330782","state":["exists","up"]},{"osd":10,"uuid":"39dc1b78-61cb-42ca-b5b6-1e44ac2b6500","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":2327428014},{"type":"v1","addr":"172.21.15.160:6825","nonce":2327428014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":2327428014},{"type":"v1","addr":"172.21.15.160:6827","nonce":2327428014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":2327428014},{"type":"v1","addr":"172.21.15.160:6831","nonce":2327428014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":2327428014},{"type":"v1","addr":"172.21.15.160:6829","nonce":2327428014}]},"public_addr":"172.21.15.160:6825/2327428014","cluster_addr":"172.21.15.160:6827/2327428014","heartbeat_back_addr":"172.21.15.160:6831/2327428014","heartbeat_front_addr":"172.21.15.160:6829/2327428014","state":["exists","up"]},{"osd":11,"uuid":"4306b3c1-707a-4010-987c-209ea0e948e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":3594364650},{"type":"v1","addr":"172.21.15.184:6825","nonce":3594364650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":3594364650},{"type":"v1","addr":"172.21.15.184:6827","nonce":3594364650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3594364650},{"type":"v1","addr":"172.21.15.184:6831","nonce":3594364650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3594364650},{"type":"v1","addr":"172.21.15.184:6829","nonce":3594364650}]},"public_addr":"172.21.15.184:6825/3594364650","cluster_addr":"172.21.15.184:6827/3594364650","heartbeat_back_addr":"172.21.15.184:6831/3594364650","heartbeat_front_addr":"172.21.15.184:6829/3594364650","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:06:17.855998+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:06:41.473270+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:07:05.304108+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:07:29.887472+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:07:55.387286+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:08:20.579669+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:08:46.333051+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:09:12.289668+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:09:38.180696+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:10:04.471459+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:10:31.041912+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:10:57.477266+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.82:0/2005424509":"2024-05-21T16:04:45.161949+0000","172.21.15.82:6801/1763049308":"2024-05-21T16:04:45.161949+0000","172.21.15.82:6801/1959459348":"2024-05-21T16:04:19.208291+0000","172.21.15.82:0/3131904556":"2024-05-21T16:04:19.208291+0000","172.21.15.82:6800/1959459348":"2024-05-21T16:04:19.208291+0000","172.21.15.82:0/2456966342":"2024-05-21T16:04:45.161949+0000","172.21.15.82:0/4179978833":"2024-05-21T16:04:19.208291+0000","172.21.15.82:0/1602862225":"2024-05-21T16:04:45.161949+0000","172.21.15.82:0/106101180":"2024-05-21T16:04:19.208291+0000","172.21.15.82:6800/1763049308":"2024-05-21T16:04:45.161949+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-20T16:11:53.556 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:53.556+0000 7fa38f00d640 1 -- 172.21.15.82:0/3084002116 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fa3780a2b90 msgr2=0x7fa3780a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:11:53.556 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:53.556+0000 7fa38f00d640 1 --2- 172.21.15.82:0/3084002116 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fa3780a2b90 0x7fa3780a50a0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fa374005fd0 tx=0x7fa374005cf0 comp rx=0 tx=0).stop 2024-05-20T16:11:53.556 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:53.556+0000 7fa38f00d640 1 -- 172.21.15.82:0/3084002116 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa388101970 msgr2=0x7fa3880698d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:11:53.556 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:53.556+0000 7fa38f00d640 1 --2- 172.21.15.82:0/3084002116 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa388101970 0x7fa3880698d0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fa37c005fd0 tx=0x7fa37c00dde0 comp rx=0 tx=0).stop 2024-05-20T16:11:53.556 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:53.556+0000 7fa38f00d640 1 -- 172.21.15.82:0/3084002116 shutdown_connections 2024-05-20T16:11:53.556 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:53.556+0000 7fa38f00d640 1 --2- 172.21.15.82:0/3084002116 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fa3780a2b90 0x7fa3780a50a0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:53.556 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:53.556+0000 7fa38f00d640 1 --2- 172.21.15.82:0/3084002116 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa3881078b0 0x7fa3881a0550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:53.556 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:53.556+0000 7fa38f00d640 1 --2- 172.21.15.82:0/3084002116 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa388102bb0 0x7fa38819dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:53.557 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:53.556+0000 7fa38f00d640 1 --2- 172.21.15.82:0/3084002116 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa388101970 0x7fa3880698d0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:53.557 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:53.556+0000 7fa38f00d640 1 -- 172.21.15.82:0/3084002116 >> 172.21.15.82:0/3084002116 conn(0x7fa3880fcb80 msgr2=0x7fa3880ff010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:11:53.557 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:53.556+0000 7fa38f00d640 1 -- 172.21.15.82:0/3084002116 shutdown_connections 2024-05-20T16:11:53.557 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:53.556+0000 7fa38f00d640 1 -- 172.21.15.82:0/3084002116 wait complete. 2024-05-20T16:11:53.562 INFO:journalctl@ceph.mds.b.smithi160.stdout:May 20 16:11:53 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-b[66944]: 2024-05-20T16:11:53.310+0000 7ff20f5dd700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-20T16:11:53.562 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:53 smithi160 ceph-mon[36347]: osdmap e90: 12 total, 12 up, 12 in 2024-05-20T16:11:53.562 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:53 smithi160 ceph-mon[36347]: from='client.? 172.21.15.82:0/2752500248' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-05-20T16:11:53.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:53 smithi160 ceph-mon[36347]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-20T16:11:53.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:53 smithi160 ceph-mon[36347]: from='client.? 172.21.15.82:0/3114230317' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T16:11:53.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:53 smithi160 ceph-mon[36347]: from='client.? 172.21.15.82:0/19149921' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-20T16:11:53.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:53 smithi160 ceph-mon[36347]: pgmap v294: 129 pgs: 22 unknown, 107 active+clean; 579 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 6 op/s 2024-05-20T16:11:53.567 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:53 smithi082 ceph-mon[32238]: osdmap e90: 12 total, 12 up, 12 in 2024-05-20T16:11:53.567 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:53 smithi082 ceph-mon[32238]: from='client.? 172.21.15.82:0/2752500248' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-05-20T16:11:53.567 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:53 smithi082 ceph-mon[32238]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-20T16:11:53.567 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:53 smithi082 ceph-mon[32238]: from='client.? 172.21.15.82:0/3114230317' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T16:11:53.567 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:53 smithi082 ceph-mon[32238]: from='client.? 172.21.15.82:0/19149921' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-20T16:11:53.567 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:53 smithi082 ceph-mon[32238]: pgmap v294: 129 pgs: 22 unknown, 107 active+clean; 579 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 6 op/s 2024-05-20T16:11:53.585 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-20T16:11:53.698 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:53.696+0000 7f371f109640 1 -- 172.21.15.82:0/851703458 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3718102900 msgr2=0x7f3718102db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:11:53.699 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:53.696+0000 7f371f109640 1 --2- 172.21.15.82:0/851703458 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3718102900 0x7f3718102db0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f3704009900 tx=0x7f370402a410 comp rx=0 tx=0).stop 2024-05-20T16:11:53.699 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:53.696+0000 7f371f109640 1 -- 172.21.15.82:0/851703458 shutdown_connections 2024-05-20T16:11:53.699 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:53.696+0000 7f371f109640 1 --2- 172.21.15.82:0/851703458 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3718102900 0x7f3718102db0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:53.699 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:53.696+0000 7f371f109640 1 --2- 172.21.15.82:0/851703458 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3718103440 0x7f3718109b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:53.699 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:53.696+0000 7f371f109640 1 --2- 172.21.15.82:0/851703458 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3718101f20 0x7f3718102330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:53.699 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:53.696+0000 7f371f109640 1 -- 172.21.15.82:0/851703458 >> 172.21.15.82:0/851703458 conn(0x7f37180fc340 msgr2=0x7f37180fe7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:11:53.699 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:53.696+0000 7f371f109640 1 -- 172.21.15.82:0/851703458 shutdown_connections 2024-05-20T16:11:53.699 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:53.696+0000 7f371f109640 1 -- 172.21.15.82:0/851703458 wait complete. 2024-05-20T16:11:53.699 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:53.697+0000 7f371f109640 1 Processor -- start 2024-05-20T16:11:53.699 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:53.697+0000 7f371f109640 1 -- start start 2024-05-20T16:11:53.699 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:53.697+0000 7f371f109640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3718101f20 0x7f37181981b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:11:53.700 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:53.697+0000 7f371f109640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3718102900 0x7f37181986f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:11:53.700 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:53.697+0000 7f371f109640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3718103440 0x7f371819f680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:11:53.700 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:53.697+0000 7f371f109640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f371810d660 con 0x7f3718103440 2024-05-20T16:11:53.700 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:53.697+0000 7f371f109640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f371810d4e0 con 0x7f3718101f20 2024-05-20T16:11:53.700 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:53.697+0000 7f371f109640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f371810d7e0 con 0x7f3718102900 2024-05-20T16:11:53.700 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:53.697+0000 7f371d6a6640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3718103440 0x7f371819f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:11:53.700 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:53.697+0000 7f371d6a6640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3718103440 0x7f371819f680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:56302/0 (socket says 172.21.15.82:56302) 2024-05-20T16:11:53.700 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:53.697+0000 7f371d6a6640 1 -- 172.21.15.82:0/3285430907 learned_addr learned my addr 172.21.15.82:0/3285430907 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:11:53.700 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:53.698+0000 7f371cea5640 1 --2- 172.21.15.82:0/3285430907 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3718101f20 0x7f37181981b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:11:53.700 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:53.698+0000 7f371d6a6640 1 -- 172.21.15.82:0/3285430907 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3718102900 msgr2=0x7f37181986f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:11:53.700 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:53.698+0000 7f371d6a6640 1 --2- 172.21.15.82:0/3285430907 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3718102900 0x7f37181986f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:53.700 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:53.698+0000 7f371d6a6640 1 -- 172.21.15.82:0/3285430907 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3718101f20 msgr2=0x7f37181981b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:11:53.700 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:53.698+0000 7f371d6a6640 1 --2- 172.21.15.82:0/3285430907 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3718101f20 0x7f37181981b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:53.701 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:53.698+0000 7f371d6a6640 1 -- 172.21.15.82:0/3285430907 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f371819fcf0 con 0x7f3718103440 2024-05-20T16:11:53.701 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:53.698+0000 7f371d6a6640 1 --2- 172.21.15.82:0/3285430907 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3718103440 0x7f371819f680 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f371000eaa0 tx=0x7f371000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:11:53.701 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:53.698+0000 7f3715ffb640 1 -- 172.21.15.82:0/3285430907 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f371000cdb0 con 0x7f3718103440 2024-05-20T16:11:53.701 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:53.698+0000 7f3715ffb640 1 -- 172.21.15.82:0/3285430907 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3710005020 con 0x7f3718103440 2024-05-20T16:11:53.701 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:53.698+0000 7f371f109640 1 -- 172.21.15.82:0/3285430907 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f371819ffe0 con 0x7f3718103440 2024-05-20T16:11:53.701 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:53.698+0000 7f3715ffb640 1 -- 172.21.15.82:0/3285430907 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37100116b0 con 0x7f3718103440 2024-05-20T16:11:53.701 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:53.698+0000 7f371f109640 1 -- 172.21.15.82:0/3285430907 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37181a04a0 con 0x7f3718103440 2024-05-20T16:11:53.702 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:53.699+0000 7f371f109640 1 -- 172.21.15.82:0/3285430907 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37181068c0 con 0x7f3718103440 2024-05-20T16:11:53.702 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:53.700+0000 7f3715ffb640 1 -- 172.21.15.82:0/3285430907 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f3710005560 con 0x7f3718103440 2024-05-20T16:11:53.702 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:53.701+0000 7f3715ffb640 1 --2- 172.21.15.82:0/3285430907 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f37080a2b90 0x7f37080a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:11:53.702 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:53.701+0000 7f371cea5640 1 --2- 172.21.15.82:0/3285430907 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f37080a2b90 0x7f37080a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:11:53.702 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:53.701+0000 7f3715ffb640 1 -- 172.21.15.82:0/3285430907 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8802+0+0 (secure 0 0 0) 0x7f37100cf530 con 0x7f3718103440 2024-05-20T16:11:53.702 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:53.702+0000 7f371cea5640 1 --2- 172.21.15.82:0/3285430907 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f37080a2b90 0x7f37080a50a0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f370c005fd0 tx=0x7f370c005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:11:53.705 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:53.706+0000 7f3715ffb640 1 -- 172.21.15.82:0/3285430907 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f3710016030 con 0x7f3718103440 2024-05-20T16:11:53.886 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:53 smithi184 ceph-mon[36209]: osdmap e90: 12 total, 12 up, 12 in 2024-05-20T16:11:53.886 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:53 smithi184 ceph-mon[36209]: from='client.? 172.21.15.82:0/2752500248' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-05-20T16:11:53.886 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:53 smithi184 ceph-mon[36209]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-20T16:11:53.886 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:53 smithi184 ceph-mon[36209]: from='client.? 172.21.15.82:0/3114230317' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T16:11:53.886 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:53 smithi184 ceph-mon[36209]: from='client.? 172.21.15.82:0/19149921' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-20T16:11:53.886 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:53 smithi184 ceph-mon[36209]: pgmap v294: 129 pgs: 22 unknown, 107 active+clean; 579 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 6 op/s 2024-05-20T16:11:53.923 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:53.923+0000 7f371f109640 1 -- 172.21.15.82:0/3285430907 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f371802d2a0 con 0x7f3718103440 2024-05-20T16:11:53.924 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:53.924+0000 7f3715ffb640 1 -- 172.21.15.82:0/3285430907 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+11806 (secure 0 0 0) 0x7f371009c230 con 0x7f3718103440 2024-05-20T16:11:53.924 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-05-20T16:11:53.924 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/2044534217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":2044534217},{"type":"v1","addr":"172.21.15.82:6835","nonce":2044534217}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/2635635097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":2635635097},{"type":"v1","addr":"172.21.15.82:6837","nonce":2635635097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14625,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6839/2118233541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6838","nonce":2118233541},{"type":"v1","addr":"172.21.15.82:6839","nonce":2118233541}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14637,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6841/2119517973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6840","nonce":2119517973},{"type":"v1","addr":"172.21.15.82:6841","nonce":2119517973}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6833/4015144007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":4015144007},{"type":"v1","addr":"172.21.15.184:6833","nonce":4015144007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6833/975243372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6832","nonce":975243372},{"type":"v1","addr":"172.21.15.160:6833","nonce":975243372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24470,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6835/664068363","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":664068363},{"type":"v1","addr":"172.21.15.160:6835","nonce":664068363}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24478,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/4242156090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":4242156090},{"type":"v1","addr":"172.21.15.184:6835","nonce":4242156090}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24496,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/403262242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":403262242},{"type":"v1","addr":"172.21.15.184:6837","nonce":403262242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24506,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6837/2846188680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6836","nonce":2846188680},{"type":"v1","addr":"172.21.15.160:6837","nonce":2846188680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24511,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6839/2199955067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":2199955067},{"type":"v1","addr":"172.21.15.184:6839","nonce":2199955067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-20T16:11:46.751632+0000","modified":"2024-05-20T16:11:51.788350+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24548},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24548":{"gid":24548,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.160:6839/2765554482","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6838","nonce":2765554482},{"type":"v1","addr":"172.21.15.160:6839","nonce":2765554482}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24548,"qdb_cluster":[24548]},"id":1}]} 2024-05-20T16:11:53.925 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 17 2024-05-20T16:11:53.926 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:53.926+0000 7f371f109640 1 -- 172.21.15.82:0/3285430907 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f37080a2b90 msgr2=0x7f37080a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:11:53.927 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:53.926+0000 7f371f109640 1 --2- 172.21.15.82:0/3285430907 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f37080a2b90 0x7f37080a50a0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f370c005fd0 tx=0x7f370c005850 comp rx=0 tx=0).stop 2024-05-20T16:11:53.927 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:53.926+0000 7f371f109640 1 -- 172.21.15.82:0/3285430907 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3718103440 msgr2=0x7f371819f680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:11:53.927 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:53.926+0000 7f371f109640 1 --2- 172.21.15.82:0/3285430907 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3718103440 0x7f371819f680 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f371000eaa0 tx=0x7f371000ef60 comp rx=0 tx=0).stop 2024-05-20T16:11:53.927 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:53.926+0000 7f371f109640 1 -- 172.21.15.82:0/3285430907 shutdown_connections 2024-05-20T16:11:53.927 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:53.926+0000 7f371f109640 1 --2- 172.21.15.82:0/3285430907 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f37080a2b90 0x7f37080a50a0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:53.927 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:53.926+0000 7f371f109640 1 --2- 172.21.15.82:0/3285430907 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3718103440 0x7f371819f680 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:53.927 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:53.926+0000 7f371f109640 1 --2- 172.21.15.82:0/3285430907 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3718102900 0x7f37181986f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:53.927 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:53.926+0000 7f371f109640 1 --2- 172.21.15.82:0/3285430907 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3718101f20 0x7f37181981b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:53.927 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:53.926+0000 7f371f109640 1 -- 172.21.15.82:0/3285430907 >> 172.21.15.82:0/3285430907 conn(0x7f37180fc340 msgr2=0x7f3718103a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:11:53.927 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:53.926+0000 7f371f109640 1 -- 172.21.15.82:0/3285430907 shutdown_connections 2024-05-20T16:11:53.927 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:53.927+0000 7f371f109640 1 -- 172.21.15.82:0/3285430907 wait complete. 2024-05-20T16:11:53.938 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-20T16:11:54.051 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.049+0000 7fbd57b09640 1 -- 172.21.15.82:0/1290995019 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbd50101990 msgr2=0x7fbd50102690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:11:54.051 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.049+0000 7fbd57b09640 1 --2- 172.21.15.82:0/1290995019 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbd50101990 0x7fbd50102690 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fbd4c009900 tx=0x7fbd4c02a410 comp rx=0 tx=0).stop 2024-05-20T16:11:54.051 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.049+0000 7fbd57b09640 1 -- 172.21.15.82:0/1290995019 shutdown_connections 2024-05-20T16:11:54.051 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.049+0000 7fbd57b09640 1 --2- 172.21.15.82:0/1290995019 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbd501078d0 0x7fbd50109cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:54.051 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.049+0000 7fbd57b09640 1 --2- 172.21.15.82:0/1290995019 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbd50102bd0 0x7fbd50102fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:54.051 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.049+0000 7fbd57b09640 1 --2- 172.21.15.82:0/1290995019 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbd50101990 0x7fbd50102690 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:54.051 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.049+0000 7fbd57b09640 1 -- 172.21.15.82:0/1290995019 >> 172.21.15.82:0/1290995019 conn(0x7fbd500fcb80 msgr2=0x7fbd500ff030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:11:54.052 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.049+0000 7fbd57b09640 1 -- 172.21.15.82:0/1290995019 shutdown_connections 2024-05-20T16:11:54.052 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.049+0000 7fbd57b09640 1 -- 172.21.15.82:0/1290995019 wait complete. 2024-05-20T16:11:54.052 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.050+0000 7fbd57b09640 1 Processor -- start 2024-05-20T16:11:54.052 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.050+0000 7fbd57b09640 1 -- start start 2024-05-20T16:11:54.052 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.050+0000 7fbd57b09640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbd50101990 0x7fbd50198240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:11:54.052 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.050+0000 7fbd57b09640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbd50102bd0 0x7fbd50198780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:11:54.052 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.050+0000 7fbd57b09640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbd501078d0 0x7fbd5019f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:11:54.052 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.050+0000 7fbd57b09640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fbd5010d5e0 con 0x7fbd501078d0 2024-05-20T16:11:54.052 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.050+0000 7fbd57b09640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fbd5010d460 con 0x7fbd50101990 2024-05-20T16:11:54.052 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.050+0000 7fbd57b09640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fbd5010d760 con 0x7fbd50102bd0 2024-05-20T16:11:54.052 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.050+0000 7fbd560a6640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbd501078d0 0x7fbd5019f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:11:54.052 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.050+0000 7fbd560a6640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbd501078d0 0x7fbd5019f710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:56324/0 (socket says 172.21.15.82:56324) 2024-05-20T16:11:54.052 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.050+0000 7fbd560a6640 1 -- 172.21.15.82:0/3661896714 learned_addr learned my addr 172.21.15.82:0/3661896714 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:11:54.053 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.050+0000 7fbd550a4640 1 --2- 172.21.15.82:0/3661896714 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbd50102bd0 0x7fbd50198780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:11:54.053 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.050+0000 7fbd560a6640 1 -- 172.21.15.82:0/3661896714 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbd50102bd0 msgr2=0x7fbd50198780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:11:54.053 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.050+0000 7fbd560a6640 1 --2- 172.21.15.82:0/3661896714 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbd50102bd0 0x7fbd50198780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:54.053 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.050+0000 7fbd558a5640 1 --2- 172.21.15.82:0/3661896714 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbd50101990 0x7fbd50198240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:11:54.053 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.050+0000 7fbd560a6640 1 -- 172.21.15.82:0/3661896714 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbd50101990 msgr2=0x7fbd50198240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:11:54.053 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.050+0000 7fbd560a6640 1 --2- 172.21.15.82:0/3661896714 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbd50101990 0x7fbd50198240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:54.053 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.050+0000 7fbd560a6640 1 -- 172.21.15.82:0/3661896714 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd50073d80 con 0x7fbd501078d0 2024-05-20T16:11:54.053 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.050+0000 7fbd558a5640 1 --2- 172.21.15.82:0/3661896714 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbd50101990 0x7fbd50198240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T16:11:54.053 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.050+0000 7fbd560a6640 1 --2- 172.21.15.82:0/3661896714 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbd501078d0 0x7fbd5019f710 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fbd4800eaa0 tx=0x7fbd4800eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:11:54.053 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.051+0000 7fbd46ffd640 1 -- 172.21.15.82:0/3661896714 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd48004d20 con 0x7fbd501078d0 2024-05-20T16:11:54.053 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.051+0000 7fbd46ffd640 1 -- 172.21.15.82:0/3661896714 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbd48005020 con 0x7fbd501078d0 2024-05-20T16:11:54.053 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.051+0000 7fbd57b09640 1 -- 172.21.15.82:0/3661896714 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd50074070 con 0x7fbd501078d0 2024-05-20T16:11:54.053 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.051+0000 7fbd46ffd640 1 -- 172.21.15.82:0/3661896714 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd480116d0 con 0x7fbd501078d0 2024-05-20T16:11:54.053 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.051+0000 7fbd57b09640 1 -- 172.21.15.82:0/3661896714 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd500745c0 con 0x7fbd501078d0 2024-05-20T16:11:54.054 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.052+0000 7fbd57b09640 1 -- 172.21.15.82:0/3661896714 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd501066e0 con 0x7fbd501078d0 2024-05-20T16:11:54.054 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.053+0000 7fbd46ffd640 1 -- 172.21.15.82:0/3661896714 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fbd480051d0 con 0x7fbd501078d0 2024-05-20T16:11:54.055 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.053+0000 7fbd46ffd640 1 --2- 172.21.15.82:0/3661896714 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fbd3c0a2b90 0x7fbd3c0a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:11:54.055 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.054+0000 7fbd558a5640 1 --2- 172.21.15.82:0/3661896714 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fbd3c0a2b90 0x7fbd3c0a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:11:54.055 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.054+0000 7fbd46ffd640 1 -- 172.21.15.82:0/3661896714 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8802+0+0 (secure 0 0 0) 0x7fbd480cf880 con 0x7fbd501078d0 2024-05-20T16:11:54.055 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.054+0000 7fbd558a5640 1 --2- 172.21.15.82:0/3661896714 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fbd3c0a2b90 0x7fbd3c0a50a0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fbd4c000c00 tx=0x7fbd4c032c30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:11:54.057 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.057+0000 7fbd46ffd640 1 -- 172.21.15.82:0/3661896714 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fbd48016030 con 0x7fbd501078d0 2024-05-20T16:11:54.234 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.234+0000 7fbd57b09640 1 -- 172.21.15.82:0/3661896714 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbd5002d470 con 0x7fbd501078d0 2024-05-20T16:11:54.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.236+0000 7fbd46ffd640 1 -- 172.21.15.82:0/3661896714 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+24153 (secure 0 0 0) 0x7fbd4809c5a0 con 0x7fbd501078d0 2024-05-20T16:11:54.236 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-05-20T16:11:54.236 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":91,"fsid":"59d1600a-16c2-11ef-bc9a-c7b262605968","created":"2024-05-20T16:03:46.611556+0000","modified":"2024-05-20T16:11:53.416085+0000","last_up_change":"2024-05-20T16:10:59.303479+0000","last_in_change":"2024-05-20T16:10:39.063561+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-20T16:07:09.493727+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-20T16:11:45.058035+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-20T16:11:45.943862+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-20T16:11:49.637579+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"bae842a9-03ec-490e-b56d-f7957235cdc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":1484330167},{"type":"v1","addr":"172.21.15.82:6803","nonce":1484330167}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":1484330167},{"type":"v1","addr":"172.21.15.82:6805","nonce":1484330167}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1484330167},{"type":"v1","addr":"172.21.15.82:6809","nonce":1484330167}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":1484330167},{"type":"v1","addr":"172.21.15.82:6807","nonce":1484330167}]},"public_addr":"172.21.15.82:6803/1484330167","cluster_addr":"172.21.15.82:6805/1484330167","heartbeat_back_addr":"172.21.15.82:6809/1484330167","heartbeat_front_addr":"172.21.15.82:6807/1484330167","state":["exists","up"]},{"osd":1,"uuid":"59f6a2ba-b5de-4161-81c5-92a417abacdf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":1457003568},{"type":"v1","addr":"172.21.15.160:6801","nonce":1457003568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":1457003568},{"type":"v1","addr":"172.21.15.160:6803","nonce":1457003568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6806","nonce":1457003568},{"type":"v1","addr":"172.21.15.160:6807","nonce":1457003568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":1457003568},{"type":"v1","addr":"172.21.15.160:6805","nonce":1457003568}]},"public_addr":"172.21.15.160:6801/1457003568","cluster_addr":"172.21.15.160:6803/1457003568","heartbeat_back_addr":"172.21.15.160:6807/1457003568","heartbeat_front_addr":"172.21.15.160:6805/1457003568","state":["exists","up"]},{"osd":2,"uuid":"3bbb5587-b449-41f3-bd26-67ec3d6afdfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":4051293934},{"type":"v1","addr":"172.21.15.184:6801","nonce":4051293934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":4051293934},{"type":"v1","addr":"172.21.15.184:6803","nonce":4051293934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":4051293934},{"type":"v1","addr":"172.21.15.184:6807","nonce":4051293934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":4051293934},{"type":"v1","addr":"172.21.15.184:6805","nonce":4051293934}]},"public_addr":"172.21.15.184:6801/4051293934","cluster_addr":"172.21.15.184:6803/4051293934","heartbeat_back_addr":"172.21.15.184:6807/4051293934","heartbeat_front_addr":"172.21.15.184:6805/4051293934","state":["exists","up"]},{"osd":3,"uuid":"ca651f90-09b8-4081-a6f9-40b31268a837","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":3947485599},{"type":"v1","addr":"172.21.15.82:6811","nonce":3947485599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":3947485599},{"type":"v1","addr":"172.21.15.82:6813","nonce":3947485599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":3947485599},{"type":"v1","addr":"172.21.15.82:6817","nonce":3947485599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":3947485599},{"type":"v1","addr":"172.21.15.82:6815","nonce":3947485599}]},"public_addr":"172.21.15.82:6811/3947485599","cluster_addr":"172.21.15.82:6813/3947485599","heartbeat_back_addr":"172.21.15.82:6817/3947485599","heartbeat_front_addr":"172.21.15.82:6815/3947485599","state":["exists","up"]},{"osd":4,"uuid":"cb3dfd56-f677-40cb-9c96-8319ea5ac788","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":4160830014},{"type":"v1","addr":"172.21.15.160:6809","nonce":4160830014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":4160830014},{"type":"v1","addr":"172.21.15.160:6811","nonce":4160830014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":4160830014},{"type":"v1","addr":"172.21.15.160:6815","nonce":4160830014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6812","nonce":4160830014},{"type":"v1","addr":"172.21.15.160:6813","nonce":4160830014}]},"public_addr":"172.21.15.160:6809/4160830014","cluster_addr":"172.21.15.160:6811/4160830014","heartbeat_back_addr":"172.21.15.160:6815/4160830014","heartbeat_front_addr":"172.21.15.160:6813/4160830014","state":["exists","up"]},{"osd":5,"uuid":"c70cfcd1-a4ff-461f-8c73-479b061200fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":1046282672},{"type":"v1","addr":"172.21.15.184:6809","nonce":1046282672}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":1046282672},{"type":"v1","addr":"172.21.15.184:6811","nonce":1046282672}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":1046282672},{"type":"v1","addr":"172.21.15.184:6815","nonce":1046282672}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":1046282672},{"type":"v1","addr":"172.21.15.184:6813","nonce":1046282672}]},"public_addr":"172.21.15.184:6809/1046282672","cluster_addr":"172.21.15.184:6811/1046282672","heartbeat_back_addr":"172.21.15.184:6815/1046282672","heartbeat_front_addr":"172.21.15.184:6813/1046282672","state":["exists","up"]},{"osd":6,"uuid":"d47ce001-9dd1-4830-bcc9-888601187c6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":2789736499},{"type":"v1","addr":"172.21.15.82:6819","nonce":2789736499}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":2789736499},{"type":"v1","addr":"172.21.15.82:6821","nonce":2789736499}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":2789736499},{"type":"v1","addr":"172.21.15.82:6825","nonce":2789736499}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":2789736499},{"type":"v1","addr":"172.21.15.82:6823","nonce":2789736499}]},"public_addr":"172.21.15.82:6819/2789736499","cluster_addr":"172.21.15.82:6821/2789736499","heartbeat_back_addr":"172.21.15.82:6825/2789736499","heartbeat_front_addr":"172.21.15.82:6823/2789736499","state":["exists","up"]},{"osd":7,"uuid":"25c54f43-18cb-41e4-83b4-2732b5a93889","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6816","nonce":2340683378},{"type":"v1","addr":"172.21.15.160:6817","nonce":2340683378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":2340683378},{"type":"v1","addr":"172.21.15.160:6819","nonce":2340683378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":2340683378},{"type":"v1","addr":"172.21.15.160:6823","nonce":2340683378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6820","nonce":2340683378},{"type":"v1","addr":"172.21.15.160:6821","nonce":2340683378}]},"public_addr":"172.21.15.160:6817/2340683378","cluster_addr":"172.21.15.160:6819/2340683378","heartbeat_back_addr":"172.21.15.160:6823/2340683378","heartbeat_front_addr":"172.21.15.160:6821/2340683378","state":["exists","up"]},{"osd":8,"uuid":"1f8b478d-287f-45f5-8526-d1a58c5bc07c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":3831576372},{"type":"v1","addr":"172.21.15.184:6817","nonce":3831576372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":3831576372},{"type":"v1","addr":"172.21.15.184:6819","nonce":3831576372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":3831576372},{"type":"v1","addr":"172.21.15.184:6823","nonce":3831576372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":3831576372},{"type":"v1","addr":"172.21.15.184:6821","nonce":3831576372}]},"public_addr":"172.21.15.184:6817/3831576372","cluster_addr":"172.21.15.184:6819/3831576372","heartbeat_back_addr":"172.21.15.184:6823/3831576372","heartbeat_front_addr":"172.21.15.184:6821/3831576372","state":["exists","up"]},{"osd":9,"uuid":"0d69e319-954f-4b18-95ea-31794f18dc3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":716330782},{"type":"v1","addr":"172.21.15.82:6827","nonce":716330782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":716330782},{"type":"v1","addr":"172.21.15.82:6829","nonce":716330782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":716330782},{"type":"v1","addr":"172.21.15.82:6833","nonce":716330782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":716330782},{"type":"v1","addr":"172.21.15.82:6831","nonce":716330782}]},"public_addr":"172.21.15.82:6827/716330782","cluster_addr":"172.21.15.82:6829/716330782","heartbeat_back_addr":"172.21.15.82:6833/716330782","heartbeat_front_addr":"172.21.15.82:6831/716330782","state":["exists","up"]},{"osd":10,"uuid":"39dc1b78-61cb-42ca-b5b6-1e44ac2b6500","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":2327428014},{"type":"v1","addr":"172.21.15.160:6825","nonce":2327428014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":2327428014},{"type":"v1","addr":"172.21.15.160:6827","nonce":2327428014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":2327428014},{"type":"v1","addr":"172.21.15.160:6831","nonce":2327428014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":2327428014},{"type":"v1","addr":"172.21.15.160:6829","nonce":2327428014}]},"public_addr":"172.21.15.160:6825/2327428014","cluster_addr":"172.21.15.160:6827/2327428014","heartbeat_back_addr":"172.21.15.160:6831/2327428014","heartbeat_front_addr":"172.21.15.160:6829/2327428014","state":["exists","up"]},{"osd":11,"uuid":"4306b3c1-707a-4010-987c-209ea0e948e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":3594364650},{"type":"v1","addr":"172.21.15.184:6825","nonce":3594364650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":3594364650},{"type":"v1","addr":"172.21.15.184:6827","nonce":3594364650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3594364650},{"type":"v1","addr":"172.21.15.184:6831","nonce":3594364650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3594364650},{"type":"v1","addr":"172.21.15.184:6829","nonce":3594364650}]},"public_addr":"172.21.15.184:6825/3594364650","cluster_addr":"172.21.15.184:6827/3594364650","heartbeat_back_addr":"172.21.15.184:6831/3594364650","heartbeat_front_addr":"172.21.15.184:6829/3594364650","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:06:17.855998+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:06:41.473270+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:07:05.304108+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:07:29.887472+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:07:55.387286+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:08:20.579669+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:08:46.333051+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:09:12.289668+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:09:38.180696+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:10:04.471459+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:10:31.041912+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:10:57.477266+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.82:0/2005424509":"2024-05-21T16:04:45.161949+0000","172.21.15.82:6801/1763049308":"2024-05-21T16:04:45.161949+0000","172.21.15.82:6801/1959459348":"2024-05-21T16:04:19.208291+0000","172.21.15.82:0/3131904556":"2024-05-21T16:04:19.208291+0000","172.21.15.82:6800/1959459348":"2024-05-21T16:04:19.208291+0000","172.21.15.82:0/2456966342":"2024-05-21T16:04:45.161949+0000","172.21.15.82:0/4179978833":"2024-05-21T16:04:19.208291+0000","172.21.15.82:0/1602862225":"2024-05-21T16:04:45.161949+0000","172.21.15.82:0/106101180":"2024-05-21T16:04:19.208291+0000","172.21.15.82:6800/1763049308":"2024-05-21T16:04:45.161949+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-20T16:11:54.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.238+0000 7fbd57b09640 1 -- 172.21.15.82:0/3661896714 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fbd3c0a2b90 msgr2=0x7fbd3c0a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:11:54.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.238+0000 7fbd57b09640 1 --2- 172.21.15.82:0/3661896714 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fbd3c0a2b90 0x7fbd3c0a50a0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fbd4c000c00 tx=0x7fbd4c032c30 comp rx=0 tx=0).stop 2024-05-20T16:11:54.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.238+0000 7fbd57b09640 1 -- 172.21.15.82:0/3661896714 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbd501078d0 msgr2=0x7fbd5019f710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:11:54.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.238+0000 7fbd57b09640 1 --2- 172.21.15.82:0/3661896714 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbd501078d0 0x7fbd5019f710 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fbd4800eaa0 tx=0x7fbd4800eea0 comp rx=0 tx=0).stop 2024-05-20T16:11:54.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.238+0000 7fbd57b09640 1 -- 172.21.15.82:0/3661896714 shutdown_connections 2024-05-20T16:11:54.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.238+0000 7fbd57b09640 1 --2- 172.21.15.82:0/3661896714 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fbd3c0a2b90 0x7fbd3c0a50a0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:54.239 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.238+0000 7fbd57b09640 1 --2- 172.21.15.82:0/3661896714 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbd501078d0 0x7fbd5019f710 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:54.239 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.238+0000 7fbd57b09640 1 --2- 172.21.15.82:0/3661896714 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbd50102bd0 0x7fbd50198780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:54.239 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.238+0000 7fbd57b09640 1 --2- 172.21.15.82:0/3661896714 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbd50101990 0x7fbd50198240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:54.239 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.238+0000 7fbd57b09640 1 -- 172.21.15.82:0/3661896714 >> 172.21.15.82:0/3661896714 conn(0x7fbd500fcb80 msgr2=0x7fbd501038b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:11:54.239 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.238+0000 7fbd57b09640 1 -- 172.21.15.82:0/3661896714 shutdown_connections 2024-05-20T16:11:54.239 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.238+0000 7fbd57b09640 1 -- 172.21.15.82:0/3661896714 wait complete. 2024-05-20T16:11:54.250 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-20T16:11:46.751632+0000', 'modified': '2024-05-20T16:11:51.788350+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 24548}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24548': {'gid': 24548, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.160:6839/2765554482', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.160:6838', 'nonce': 2765554482}, {'type': 'v1', 'addr': '172.21.15.160:6839', 'nonce': 2765554482}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1, 'qdb_leader': 24548, 'qdb_cluster': [24548]} 2024-05-20T16:11:54.250 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-20T16:11:54.250 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-05-20T16:11:54.366 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.364+0000 7faf8400d640 1 -- 172.21.15.82:0/2774406687 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faf7c1019b0 msgr2=0x7faf7c1026b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:11:54.366 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.364+0000 7faf8400d640 1 --2- 172.21.15.82:0/2774406687 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faf7c1019b0 0x7faf7c1026b0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7faf78009900 tx=0x7faf7802a410 comp rx=0 tx=0).stop 2024-05-20T16:11:54.366 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.364+0000 7faf8400d640 1 -- 172.21.15.82:0/2774406687 shutdown_connections 2024-05-20T16:11:54.366 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.364+0000 7faf8400d640 1 --2- 172.21.15.82:0/2774406687 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faf7c1078f0 0x7faf7c109d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:54.366 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.364+0000 7faf8400d640 1 --2- 172.21.15.82:0/2774406687 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faf7c102bf0 0x7faf7c103000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:54.366 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.364+0000 7faf8400d640 1 --2- 172.21.15.82:0/2774406687 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faf7c1019b0 0x7faf7c1026b0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:54.366 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.364+0000 7faf8400d640 1 -- 172.21.15.82:0/2774406687 >> 172.21.15.82:0/2774406687 conn(0x7faf7c0fcba0 msgr2=0x7faf7c0ff050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:11:54.366 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.364+0000 7faf8400d640 1 -- 172.21.15.82:0/2774406687 shutdown_connections 2024-05-20T16:11:54.366 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.364+0000 7faf8400d640 1 -- 172.21.15.82:0/2774406687 wait complete. 2024-05-20T16:11:54.366 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.364+0000 7faf8400d640 1 Processor -- start 2024-05-20T16:11:54.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.364+0000 7faf8400d640 1 -- start start 2024-05-20T16:11:54.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.365+0000 7faf8400d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faf7c1019b0 0x7faf7c199420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:11:54.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.365+0000 7faf8400d640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faf7c102bf0 0x7faf7c199960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:11:54.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.365+0000 7faf8400d640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faf7c1078f0 0x7faf7c1a08f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:11:54.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.365+0000 7faf8400d640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7faf7c10e7f0 con 0x7faf7c1019b0 2024-05-20T16:11:54.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.365+0000 7faf8400d640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7faf7c10e670 con 0x7faf7c1078f0 2024-05-20T16:11:54.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.365+0000 7faf8400d640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7faf7c10e970 con 0x7faf7c102bf0 2024-05-20T16:11:54.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.365+0000 7faf81da9640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faf7c1019b0 0x7faf7c199420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:11:54.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.365+0000 7faf81da9640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faf7c1019b0 0x7faf7c199420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:56346/0 (socket says 172.21.15.82:56346) 2024-05-20T16:11:54.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.365+0000 7faf81da9640 1 -- 172.21.15.82:0/1258097124 learned_addr learned my addr 172.21.15.82:0/1258097124 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:11:54.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.365+0000 7faf815a8640 1 --2- 172.21.15.82:0/1258097124 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faf7c102bf0 0x7faf7c199960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:11:54.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.365+0000 7faf81da9640 1 -- 172.21.15.82:0/1258097124 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faf7c102bf0 msgr2=0x7faf7c199960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:11:54.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.365+0000 7faf81da9640 1 --2- 172.21.15.82:0/1258097124 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faf7c102bf0 0x7faf7c199960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:54.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.365+0000 7faf81da9640 1 -- 172.21.15.82:0/1258097124 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faf7c1078f0 msgr2=0x7faf7c1a08f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:11:54.368 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.365+0000 7faf81da9640 1 --2- 172.21.15.82:0/1258097124 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faf7c1078f0 0x7faf7c1a08f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:54.368 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.365+0000 7faf81da9640 1 -- 172.21.15.82:0/1258097124 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf7c1a0f60 con 0x7faf7c1019b0 2024-05-20T16:11:54.368 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.365+0000 7faf81da9640 1 --2- 172.21.15.82:0/1258097124 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faf7c1019b0 0x7faf7c199420 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7faf78006010 tx=0x7faf780043b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:11:54.368 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.365+0000 7faf72ffd640 1 -- 172.21.15.82:0/1258097124 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf7803f030 con 0x7faf7c1019b0 2024-05-20T16:11:54.368 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.365+0000 7faf72ffd640 1 -- 172.21.15.82:0/1258097124 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faf78003ce0 con 0x7faf7c1019b0 2024-05-20T16:11:54.368 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.365+0000 7faf8400d640 1 -- 172.21.15.82:0/1258097124 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf7c1a11f0 con 0x7faf7c1019b0 2024-05-20T16:11:54.368 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.365+0000 7faf72ffd640 1 -- 172.21.15.82:0/1258097124 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf78035070 con 0x7faf7c1019b0 2024-05-20T16:11:54.368 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.365+0000 7faf8400d640 1 -- 172.21.15.82:0/1258097124 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf7c1a1760 con 0x7faf7c1019b0 2024-05-20T16:11:54.375 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.368+0000 7faf72ffd640 1 -- 172.21.15.82:0/1258097124 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7faf78042030 con 0x7faf7c1019b0 2024-05-20T16:11:54.376 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.368+0000 7faf8af8e740 1 Processor -- start 2024-05-20T16:11:54.376 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.368+0000 7faf8af8e740 1 -- start start 2024-05-20T16:11:54.376 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.368+0000 7faf8af8e740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d7dc1dd270 0x55d7dc57d470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:11:54.376 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.368+0000 7faf8af8e740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d7dc1b43b0 0x55d7dc57d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:11:54.376 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.368+0000 7faf8af8e740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d7dc1bc590 0x55d7dc5848f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:11:54.376 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.368+0000 7faf8af8e740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55d7dc4e14d0 con 0x55d7dc1b43b0 2024-05-20T16:11:54.376 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.368+0000 7faf8af8e740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x55d7dc4893e0 con 0x55d7dc1dd270 2024-05-20T16:11:54.376 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.368+0000 7faf8af8e740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55d7dc4dd490 con 0x55d7dc1bc590 2024-05-20T16:11:54.376 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.368+0000 7faf81da9640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d7dc1b43b0 0x55d7dc57d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:11:54.376 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.369+0000 7faf81da9640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d7dc1b43b0 0x55d7dc57d9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:56352/0 (socket says 172.21.15.82:56352) 2024-05-20T16:11:54.376 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.369+0000 7faf81da9640 1 -- 172.21.15.82:0/1730247399 learned_addr learned my addr 172.21.15.82:0/1730247399 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:11:54.376 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.369+0000 7faf72ffd640 1 --2- 172.21.15.82:0/1258097124 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7faf680a2b90 0x7faf680a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:11:54.377 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.369+0000 7faf815a8640 1 --2- 172.21.15.82:0/1730247399 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d7dc1bc590 0x55d7dc5848f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:11:54.377 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.369+0000 7faf825aa640 1 --2- 172.21.15.82:0/1730247399 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d7dc1dd270 0x55d7dc57d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:11:54.377 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.369+0000 7faf825aa640 1 --2- 172.21.15.82:0/1258097124 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7faf680a2b90 0x7faf680a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:11:54.377 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.369+0000 7faf81da9640 1 -- 172.21.15.82:0/1730247399 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d7dc1bc590 msgr2=0x55d7dc5848f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:11:54.377 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.369+0000 7faf81da9640 1 --2- 172.21.15.82:0/1730247399 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d7dc1bc590 0x55d7dc5848f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:54.377 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.369+0000 7faf81da9640 1 -- 172.21.15.82:0/1730247399 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d7dc1dd270 msgr2=0x55d7dc57d470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:11:54.377 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.369+0000 7faf81da9640 1 --2- 172.21.15.82:0/1730247399 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d7dc1dd270 0x55d7dc57d470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:54.377 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.369+0000 7faf72ffd640 1 -- 172.21.15.82:0/1258097124 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8802+0+0 (secure 0 0 0) 0x7faf780efa70 con 0x7faf7c1019b0 2024-05-20T16:11:54.377 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.369+0000 7faf81da9640 1 -- 172.21.15.82:0/1730247399 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d7dc1f52d0 con 0x55d7dc1b43b0 2024-05-20T16:11:54.377 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.369+0000 7faf81da9640 1 --2- 172.21.15.82:0/1730247399 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d7dc1b43b0 0x55d7dc57d9f0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7faf780bbd30 tx=0x7faf7803b920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdd4b503381fed6c server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:11:54.377 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.369+0000 7faf825aa640 1 --2- 172.21.15.82:0/1258097124 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7faf680a2b90 0x7faf680a50a0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x55d7dc57e640 tx=0x7faf74004bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:11:54.377 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.369+0000 7faf67fff640 1 -- 172.21.15.82:0/1730247399 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf780efe60 con 0x55d7dc1b43b0 2024-05-20T16:11:54.377 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.369+0000 7faf67fff640 1 -- 172.21.15.82:0/1730247399 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faf7803be50 con 0x55d7dc1b43b0 2024-05-20T16:11:54.377 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.369+0000 7faf67fff640 1 -- 172.21.15.82:0/1730247399 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf78049070 con 0x55d7dc1b43b0 2024-05-20T16:11:54.378 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.370+0000 7faf8af8e740 1 -- 172.21.15.82:0/1730247399 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d7dc1b43b0 msgr2=0x55d7dc57d9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:11:54.378 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.370+0000 7faf8af8e740 1 --2- 172.21.15.82:0/1730247399 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d7dc1b43b0 0x55d7dc57d9f0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7faf780bbd30 tx=0x7faf7803b920 comp rx=0 tx=0).stop 2024-05-20T16:11:54.378 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.370+0000 7faf8af8e740 1 -- 172.21.15.82:0/1730247399 shutdown_connections 2024-05-20T16:11:54.378 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.370+0000 7faf8af8e740 1 --2- 172.21.15.82:0/1730247399 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d7dc1b43b0 0x55d7dc57d9f0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:54.378 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.370+0000 7faf8af8e740 1 --2- 172.21.15.82:0/1730247399 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d7dc1bc590 0x55d7dc5848f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:54.378 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.370+0000 7faf8af8e740 1 --2- 172.21.15.82:0/1730247399 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d7dc1dd270 0x55d7dc57d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:54.378 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.370+0000 7faf8af8e740 1 -- 172.21.15.82:0/1730247399 >> 172.21.15.82:0/1730247399 conn(0x55d7dc1d13c0 msgr2=0x55d7dc578e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:11:54.378 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.370+0000 7faf8af8e740 1 -- 172.21.15.82:0/1730247399 shutdown_connections 2024-05-20T16:11:54.378 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.370+0000 7faf8af8e740 1 -- 172.21.15.82:0/1730247399 wait complete. 2024-05-20T16:11:54.378 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.371+0000 7faf8af8e740 1 -- start start 2024-05-20T16:11:54.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.371+0000 7faf8af8e740 1 Processor -- start 2024-05-20T16:11:54.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.371+0000 7faf8af8e740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d7dc1b43b0 0x55d7dc586f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:11:54.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.371+0000 7faf8af8e740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d7dc1bc590 0x55d7dc5874a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:11:54.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.371+0000 7faf8af8e740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d7dc1dd270 0x55d7dc58e400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:11:54.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.371+0000 7faf8af8e740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55d7dc4e14d0 con 0x55d7dc1bc590 2024-05-20T16:11:54.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.371+0000 7faf8af8e740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x55d7dc49c8e0 con 0x55d7dc1dd270 2024-05-20T16:11:54.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.371+0000 7faf8af8e740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55d7dc585100 con 0x55d7dc1b43b0 2024-05-20T16:11:54.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.371+0000 7faf8af8e740 10 client.0 _ll_register_callbacks cb 0x55d7dc4699c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T16:11:54.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.371+0000 7faf815a8640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d7dc1bc590 0x55d7dc5874a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:11:54.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.371+0000 7faf815a8640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d7dc1bc590 0x55d7dc5874a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:56358/0 (socket says 172.21.15.82:56358) 2024-05-20T16:11:54.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.371+0000 7faf815a8640 1 -- 172.21.15.82:0/162028994 learned_addr learned my addr 172.21.15.82:0/162028994 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:11:54.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.371+0000 7faf825aa640 1 --2- 172.21.15.82:0/162028994 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d7dc1dd270 0x55d7dc58e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:11:54.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.372+0000 7faf81da9640 1 --2- 172.21.15.82:0/162028994 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d7dc1b43b0 0x55d7dc586f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:11:54.380 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.372+0000 7faf815a8640 1 -- 172.21.15.82:0/162028994 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d7dc1b43b0 msgr2=0x55d7dc586f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:11:54.380 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.372+0000 7faf815a8640 1 --2- 172.21.15.82:0/162028994 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d7dc1b43b0 0x55d7dc586f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:54.380 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.372+0000 7faf815a8640 1 -- 172.21.15.82:0/162028994 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d7dc1dd270 msgr2=0x55d7dc58e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:11:54.380 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.372+0000 7faf815a8640 1 --2- 172.21.15.82:0/162028994 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d7dc1dd270 0x55d7dc58e400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:54.380 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.372+0000 7faf815a8640 1 -- 172.21.15.82:0/162028994 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d7dc2bf6b0 con 0x55d7dc1bc590 2024-05-20T16:11:54.380 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.372+0000 7faf81da9640 1 --2- 172.21.15.82:0/162028994 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d7dc1b43b0 0x55d7dc586f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T16:11:54.380 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.372+0000 7faf825aa640 1 --2- 172.21.15.82:0/162028994 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d7dc1dd270 0x55d7dc58e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T16:11:54.380 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.372+0000 7faf815a8640 1 --2- 172.21.15.82:0/162028994 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d7dc1bc590 0x55d7dc5874a0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7faf6c010500 tx=0x7faf6c0109c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c43b355c056eda5e server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:11:54.380 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.372+0000 7faf657fa640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-05-20T16:11:54.380 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.372+0000 7faf657fa640 1 -- 172.21.15.82:0/162028994 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf6c01a020 con 0x55d7dc1bc590 2024-05-20T16:11:54.380 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.372+0000 7faf657fa640 1 -- 172.21.15.82:0/162028994 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8802+0+0 (secure 0 0 0) 0x7faf6c016690 con 0x55d7dc1bc590 2024-05-20T16:11:54.380 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.372+0000 7faf8af8e740 1 -- 172.21.15.82:0/162028994 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faf7803be50 con 0x55d7dc1bc590 2024-05-20T16:11:54.380 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.372+0000 7faf657fa640 1 -- 172.21.15.82:0/162028994 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7faf6c025050 con 0x55d7dc1bc590 2024-05-20T16:11:54.380 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.372+0000 7faf8af8e740 10 client.14757 fetch_fsmap learned FSMap version 17 2024-05-20T16:11:54.380 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.372+0000 7faf8af8e740 1 -- 172.21.15.82:0/162028994 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) -- 0x55d7dc591430 con 0x55d7dc1bc590 2024-05-20T16:11:54.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.372+0000 7faf657fa640 1 -- 172.21.15.82:0/162028994 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faf7803be50 con 0x55d7dc1bc590 2024-05-20T16:11:54.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.373+0000 7faf657fa640 1 -- 172.21.15.82:0/162028994 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 17) ==== 8342+0+0 (secure 0 0 0) 0x7faf6c014b60 con 0x55d7dc1bc590 2024-05-20T16:11:54.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.373+0000 7faf8af8e740 10 client.14757 fetch_fsmap finished waiting for FSMap version 17 2024-05-20T16:11:54.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.373+0000 7faf8af8e740 10 client.14757 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-20T16:11:54.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.373+0000 7faf8af8e740 20 client.14757 populate_metadata read hostname 'smithi082' 2024-05-20T16:11:54.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.373+0000 7faf8af8e740 1 --2- 172.21.15.82:0/162028994 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x55d7dc19be40 0x55d7dc594fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:11:54.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.373+0000 7faf8af8e740 4 client.14757 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T16:11:54.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.373+0000 7faf8af8e740 1 -- 172.21.15.82:0/162028994 --> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d7dc4eec00 con 0x55d7dc19be40 2024-05-20T16:11:54.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.373+0000 7faf81da9640 1 --2- 172.21.15.82:0/162028994 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x55d7dc19be40 0x55d7dc594fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:11:54.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.373+0000 7faf657fa640 10 client.14757 ms_handle_connect on v2:172.21.15.160:6838/2765554482 2024-05-20T16:11:54.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.373+0000 7faf81da9640 1 --2- 172.21.15.82:0/162028994 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x55d7dc19be40 0x55d7dc594fd0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1abaeb93e09531da server_cookie=119320a55e3c7405 in_seq=0 out_seq=0 2024-05-20T16:11:54.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.374+0000 7faf657fa640 1 -- 172.21.15.82:0/162028994 <== mds.0 v2:172.21.15.160:6838/2765554482 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55d7dc4eec00 con 0x55d7dc19be40 2024-05-20T16:11:54.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.374+0000 7faf657fa640 10 client.14757 handle_command_reply: tid=0 2024-05-20T16:11:54.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.374+0000 7faf8af8e740 1 client.14757 shutdown 2024-05-20T16:11:54.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.374+0000 7faf8af8e740 1 -- 172.21.15.82:0/162028994 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d7dc1bc590 msgr2=0x55d7dc5874a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:11:54.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.374+0000 7faf8af8e740 1 --2- 172.21.15.82:0/162028994 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d7dc1bc590 0x55d7dc5874a0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7faf6c010500 tx=0x7faf6c0109c0 comp rx=0 tx=0).stop 2024-05-20T16:11:54.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.375+0000 7faf8af8e740 1 -- 172.21.15.82:0/162028994 shutdown_connections 2024-05-20T16:11:54.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.375+0000 7faf8af8e740 1 --2- 172.21.15.82:0/162028994 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x55d7dc19be40 0x55d7dc594fd0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:54.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.375+0000 7faf8af8e740 1 --2- 172.21.15.82:0/162028994 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d7dc1bc590 0x55d7dc5874a0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:54.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.375+0000 7faf8af8e740 1 --2- 172.21.15.82:0/162028994 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d7dc1dd270 0x55d7dc58e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:54.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.375+0000 7faf8af8e740 1 --2- 172.21.15.82:0/162028994 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d7dc1b43b0 0x55d7dc586f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:54.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.375+0000 7faf657fa640 0 client.14757 ms_handle_reset on v2:172.21.15.160:6838/2765554482 2024-05-20T16:11:54.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.375+0000 7faf8af8e740 1 -- 172.21.15.82:0/162028994 >> 172.21.15.82:0/162028994 conn(0x55d7dc1d13c0 msgr2=0x55d7dc577ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:11:54.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.375+0000 7faf8af8e740 1 -- 172.21.15.82:0/162028994 shutdown_connections 2024-05-20T16:11:54.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.375+0000 7faf8af8e740 1 -- 172.21.15.82:0/162028994 wait complete. 2024-05-20T16:11:54.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.375+0000 7faf8af8e740 20 client.14757 trim_cache size 0 max 16384 2024-05-20T16:11:54.394 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.393+0000 7faf8af8e740 1 Processor -- start 2024-05-20T16:11:54.394 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.393+0000 7faf8af8e740 1 -- start start 2024-05-20T16:11:54.394 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.393+0000 7faf8af8e740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d7dc1b43b0 0x55d7dc5787f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:11:54.395 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.393+0000 7faf8af8e740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d7dc1bc590 0x55d7dc578d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:11:54.395 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.393+0000 7faf8af8e740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d7dc1dd270 0x55d7dc5816f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:11:54.395 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.393+0000 7faf8af8e740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55d7dc593310 con 0x55d7dc1bc590 2024-05-20T16:11:54.395 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.393+0000 7faf8af8e740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x55d7dc5939e0 con 0x55d7dc1b43b0 2024-05-20T16:11:54.395 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.393+0000 7faf8af8e740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55d7dc5940b0 con 0x55d7dc1dd270 2024-05-20T16:11:54.395 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.393+0000 7faf815a8640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d7dc1bc590 0x55d7dc578d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:11:54.395 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.393+0000 7faf815a8640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d7dc1bc590 0x55d7dc578d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:56360/0 (socket says 172.21.15.82:56360) 2024-05-20T16:11:54.395 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.394+0000 7faf815a8640 1 -- 172.21.15.82:0/4042052292 learned_addr learned my addr 172.21.15.82:0/4042052292 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:11:54.395 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.394+0000 7faf825aa640 1 --2- 172.21.15.82:0/4042052292 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d7dc1dd270 0x55d7dc5816f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:11:54.395 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.394+0000 7faf81da9640 1 --2- 172.21.15.82:0/4042052292 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d7dc1b43b0 0x55d7dc5787f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:11:54.395 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.394+0000 7faf815a8640 1 -- 172.21.15.82:0/4042052292 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d7dc1dd270 msgr2=0x55d7dc5816f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:11:54.395 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.394+0000 7faf815a8640 1 --2- 172.21.15.82:0/4042052292 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d7dc1dd270 0x55d7dc5816f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:54.395 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.394+0000 7faf815a8640 1 -- 172.21.15.82:0/4042052292 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d7dc1b43b0 msgr2=0x55d7dc5787f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:11:54.395 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.394+0000 7faf815a8640 1 --2- 172.21.15.82:0/4042052292 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d7dc1b43b0 0x55d7dc5787f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:54.396 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.394+0000 7faf815a8640 1 -- 172.21.15.82:0/4042052292 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d7dc57d020 con 0x55d7dc1bc590 2024-05-20T16:11:54.396 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.394+0000 7faf815a8640 1 --2- 172.21.15.82:0/4042052292 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d7dc1bc590 0x55d7dc578d30 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7faf6c00b2d0 tx=0x7faf6c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80f3c39f039338c7 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:11:54.396 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.394+0000 7faf677fe640 1 -- 172.21.15.82:0/4042052292 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf6c02d020 con 0x55d7dc1bc590 2024-05-20T16:11:54.396 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.394+0000 7faf677fe640 1 -- 172.21.15.82:0/4042052292 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faf6c0210d0 con 0x55d7dc1bc590 2024-05-20T16:11:54.396 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.394+0000 7faf677fe640 1 -- 172.21.15.82:0/4042052292 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf6c01a040 con 0x55d7dc1bc590 2024-05-20T16:11:54.400 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.395+0000 7faf8af8e740 1 -- 172.21.15.82:0/4042052292 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d7dc1bc590 msgr2=0x55d7dc578d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:11:54.400 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.395+0000 7faf8af8e740 1 --2- 172.21.15.82:0/4042052292 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d7dc1bc590 0x55d7dc578d30 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7faf6c00b2d0 tx=0x7faf6c00be20 comp rx=0 tx=0).stop 2024-05-20T16:11:54.400 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.395+0000 7faf8af8e740 1 -- 172.21.15.82:0/4042052292 shutdown_connections 2024-05-20T16:11:54.400 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.395+0000 7faf8af8e740 1 --2- 172.21.15.82:0/4042052292 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d7dc1bc590 0x55d7dc578d30 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:54.400 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.395+0000 7faf8af8e740 1 --2- 172.21.15.82:0/4042052292 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d7dc1dd270 0x55d7dc5816f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:54.400 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.395+0000 7faf8af8e740 1 --2- 172.21.15.82:0/4042052292 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d7dc1b43b0 0x55d7dc5787f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:54.400 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.395+0000 7faf8af8e740 1 -- 172.21.15.82:0/4042052292 >> 172.21.15.82:0/4042052292 conn(0x55d7dc1d13c0 msgr2=0x55d7dc586f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:11:54.400 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.395+0000 7faf8af8e740 1 -- 172.21.15.82:0/4042052292 shutdown_connections 2024-05-20T16:11:54.400 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.395+0000 7faf8af8e740 1 -- 172.21.15.82:0/4042052292 wait complete. 2024-05-20T16:11:54.400 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.395+0000 7faf8af8e740 1 -- start start 2024-05-20T16:11:54.401 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.395+0000 7faf8af8e740 1 Processor -- start 2024-05-20T16:11:54.401 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.396+0000 7faf8af8e740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d7dc1b43b0 0x55d7dc588850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:11:54.401 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.396+0000 7faf8af8e740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d7dc1bc590 0x55d7dc588d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:11:54.401 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.396+0000 7faf8af8e740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d7dc1dd270 0x55d7dc5fbe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:11:54.401 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.396+0000 7faf8af8e740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55d7dc594e50 con 0x55d7dc1dd270 2024-05-20T16:11:54.401 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.396+0000 7faf8af8e740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x55d7dc594780 con 0x55d7dc1b43b0 2024-05-20T16:11:54.401 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.396+0000 7faf8af8e740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55d7dc595520 con 0x55d7dc1bc590 2024-05-20T16:11:54.401 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.396+0000 7faf8af8e740 10 client.0 _ll_register_callbacks cb 0x7faf6c00b520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T16:11:54.401 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.396+0000 7faf825aa640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d7dc1dd270 0x55d7dc5fbe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:11:54.401 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.396+0000 7faf825aa640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d7dc1dd270 0x55d7dc5fbe40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:56364/0 (socket says 172.21.15.82:56364) 2024-05-20T16:11:54.401 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.396+0000 7faf825aa640 1 -- 172.21.15.82:0/3452822028 learned_addr learned my addr 172.21.15.82:0/3452822028 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:11:54.401 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.396+0000 7faf81da9640 1 --2- 172.21.15.82:0/3452822028 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d7dc1b43b0 0x55d7dc588850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:11:54.401 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.396+0000 7faf815a8640 1 --2- 172.21.15.82:0/3452822028 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d7dc1bc590 0x55d7dc588d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:11:54.401 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.396+0000 7faf825aa640 1 -- 172.21.15.82:0/3452822028 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d7dc1bc590 msgr2=0x55d7dc588d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:11:54.402 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.396+0000 7faf825aa640 1 --2- 172.21.15.82:0/3452822028 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d7dc1bc590 0x55d7dc588d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:54.402 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.396+0000 7faf825aa640 1 -- 172.21.15.82:0/3452822028 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d7dc1b43b0 msgr2=0x55d7dc588850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:11:54.402 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.396+0000 7faf825aa640 1 --2- 172.21.15.82:0/3452822028 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d7dc1b43b0 0x55d7dc588850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:54.402 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.396+0000 7faf825aa640 1 -- 172.21.15.82:0/3452822028 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d7dc58dcf0 con 0x55d7dc1dd270 2024-05-20T16:11:54.402 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.397+0000 7faf825aa640 1 --2- 172.21.15.82:0/3452822028 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d7dc1dd270 0x55d7dc5fbe40 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7faf74017e00 tx=0x7faf74004b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39d9a0759605411a server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:11:54.402 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.397+0000 7faf657fa640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-05-20T16:11:54.402 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.397+0000 7faf657fa640 1 -- 172.21.15.82:0/3452822028 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf74008030 con 0x55d7dc1dd270 2024-05-20T16:11:54.402 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.397+0000 7faf657fa640 1 -- 172.21.15.82:0/3452822028 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8802+0+0 (secure 0 0 0) 0x7faf74016d20 con 0x55d7dc1dd270 2024-05-20T16:11:54.402 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.397+0000 7faf8af8e740 1 -- 172.21.15.82:0/3452822028 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faf6c0210d0 con 0x55d7dc1dd270 2024-05-20T16:11:54.402 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.397+0000 7faf657fa640 1 -- 172.21.15.82:0/3452822028 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7faf74030050 con 0x55d7dc1dd270 2024-05-20T16:11:54.402 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.397+0000 7faf8af8e740 10 client.14763 fetch_fsmap learned FSMap version 17 2024-05-20T16:11:54.402 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.397+0000 7faf8af8e740 1 -- 172.21.15.82:0/3452822028 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) -- 0x55d7dc5fd430 con 0x55d7dc1dd270 2024-05-20T16:11:54.402 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.397+0000 7faf657fa640 1 -- 172.21.15.82:0/3452822028 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faf6c0210d0 con 0x55d7dc1dd270 2024-05-20T16:11:54.403 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.397+0000 7faf657fa640 1 -- 172.21.15.82:0/3452822028 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 17) ==== 8342+0+0 (secure 0 0 0) 0x7faf7402b4b0 con 0x55d7dc1dd270 2024-05-20T16:11:54.403 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.398+0000 7faf8af8e740 10 client.14763 fetch_fsmap finished waiting for FSMap version 17 2024-05-20T16:11:54.403 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.398+0000 7faf8af8e740 10 client.14763 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-20T16:11:54.403 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.398+0000 7faf8af8e740 20 client.14763 populate_metadata read hostname 'smithi082' 2024-05-20T16:11:54.403 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.398+0000 7faf8af8e740 1 --2- 172.21.15.82:0/3452822028 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x55d7dc171280 0x55d7dc5fe540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:11:54.403 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.398+0000 7faf8af8e740 4 client.14763 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-20T16:11:54.403 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.398+0000 7faf8af8e740 1 -- 172.21.15.82:0/3452822028 --> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] -- command(tid 0: {"prefix": "status"}) -- 0x55d7dc4e8f50 con 0x55d7dc171280 2024-05-20T16:11:54.403 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.398+0000 7faf81da9640 1 --2- 172.21.15.82:0/3452822028 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x55d7dc171280 0x55d7dc5fe540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:11:54.403 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.398+0000 7faf657fa640 10 client.14763 ms_handle_connect on v2:172.21.15.160:6838/2765554482 2024-05-20T16:11:54.403 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.398+0000 7faf81da9640 1 --2- 172.21.15.82:0/3452822028 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x55d7dc171280 0x55d7dc5fe540 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=105f525882b16987 server_cookie=6397c54373d23e6a in_seq=0 out_seq=0 2024-05-20T16:11:54.403 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.398+0000 7faf657fa640 1 -- 172.21.15.82:0/3452822028 <== mds.0 v2:172.21.15.160:6838/2765554482 1 ==== command_reply(tid 0: 0 ) ==== 8+0+326 (crc 0 0 0) 0x55d7dc4e8f50 con 0x55d7dc171280 2024-05-20T16:11:54.403 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.398+0000 7faf657fa640 10 client.14763 handle_command_reply: tid=0 2024-05-20T16:11:54.403 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.399+0000 7faf8af8e740 1 client.14763 shutdown 2024-05-20T16:11:54.404 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.399+0000 7faf8af8e740 1 -- 172.21.15.82:0/3452822028 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d7dc1dd270 msgr2=0x55d7dc5fbe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:11:54.404 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.399+0000 7faf8af8e740 1 --2- 172.21.15.82:0/3452822028 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d7dc1dd270 0x55d7dc5fbe40 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7faf74017e00 tx=0x7faf74004b50 comp rx=0 tx=0).stop 2024-05-20T16:11:54.404 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.399+0000 7faf8af8e740 1 -- 172.21.15.82:0/3452822028 shutdown_connections 2024-05-20T16:11:54.404 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.399+0000 7faf8af8e740 1 --2- 172.21.15.82:0/3452822028 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x55d7dc171280 0x55d7dc5fe540 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:54.404 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.399+0000 7faf8af8e740 1 --2- 172.21.15.82:0/3452822028 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d7dc1dd270 0x55d7dc5fbe40 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:54.404 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.399+0000 7faf8af8e740 1 --2- 172.21.15.82:0/3452822028 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d7dc1bc590 0x55d7dc588d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:54.404 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.399+0000 7faf657fa640 0 client.14763 ms_handle_reset on v2:172.21.15.160:6838/2765554482 2024-05-20T16:11:54.404 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.399+0000 7faf8af8e740 1 --2- 172.21.15.82:0/3452822028 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d7dc1b43b0 0x55d7dc588850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:54.404 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.399+0000 7faf8af8e740 1 -- 172.21.15.82:0/3452822028 >> 172.21.15.82:0/3452822028 conn(0x55d7dc1d13c0 msgr2=0x55d7dc5816f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:11:54.404 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.399+0000 7faf8af8e740 1 -- 172.21.15.82:0/3452822028 shutdown_connections 2024-05-20T16:11:54.404 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.399+0000 7faf8af8e740 1 -- 172.21.15.82:0/3452822028 wait complete. 2024-05-20T16:11:54.404 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.399+0000 7faf8af8e740 20 client.14763 trim_cache size 0 max 16384 2024-05-20T16:11:54.405 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-05-20T16:11:54.405 INFO:teuthology.orchestra.run.smithi082.stdout: "cluster_fsid": "59d1600a-16c2-11ef-bc9a-c7b262605968", 2024-05-20T16:11:54.405 INFO:teuthology.orchestra.run.smithi082.stdout: "whoami": 0, 2024-05-20T16:11:54.405 INFO:teuthology.orchestra.run.smithi082.stdout: "id": 24548, 2024-05-20T16:11:54.405 INFO:teuthology.orchestra.run.smithi082.stdout: "want_state": "up:active", 2024-05-20T16:11:54.405 INFO:teuthology.orchestra.run.smithi082.stdout: "state": "up:active", 2024-05-20T16:11:54.405 INFO:teuthology.orchestra.run.smithi082.stdout: "fs_name": "cephfs", 2024-05-20T16:11:54.405 INFO:teuthology.orchestra.run.smithi082.stdout: "rank_uptime": 7.0178170309999999, 2024-05-20T16:11:54.405 INFO:teuthology.orchestra.run.smithi082.stdout: "mdsmap_epoch": 17, 2024-05-20T16:11:54.405 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_epoch": 91, 2024-05-20T16:11:54.405 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_epoch_barrier": 90, 2024-05-20T16:11:54.405 INFO:teuthology.orchestra.run.smithi082.stdout: "uptime": 11.150237425 2024-05-20T16:11:54.405 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-05-20T16:11:54.406 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.399+0000 7faf8400d640 1 -- 172.21.15.82:0/1258097124 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7faf680a2b90 msgr2=0x7faf680a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:11:54.406 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.399+0000 7faf8400d640 1 --2- 172.21.15.82:0/1258097124 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7faf680a2b90 0x7faf680a50a0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x55d7dc57e640 tx=0x7faf74004bc0 comp rx=0 tx=0).stop 2024-05-20T16:11:54.406 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.399+0000 7faf8400d640 1 -- 172.21.15.82:0/1258097124 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faf7c1019b0 msgr2=0x7faf7c199420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:11:54.406 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.399+0000 7faf8400d640 1 --2- 172.21.15.82:0/1258097124 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faf7c1019b0 0x7faf7c199420 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7faf78006010 tx=0x7faf780043b0 comp rx=0 tx=0).stop 2024-05-20T16:11:54.406 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.399+0000 7faf8400d640 1 -- 172.21.15.82:0/1258097124 shutdown_connections 2024-05-20T16:11:54.407 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.399+0000 7faf8400d640 1 --2- 172.21.15.82:0/1258097124 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7faf680a2b90 0x7faf680a50a0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:54.407 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.400+0000 7faf8400d640 1 --2- 172.21.15.82:0/1258097124 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faf7c1078f0 0x7faf7c1a08f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:54.407 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.400+0000 7faf8400d640 1 --2- 172.21.15.82:0/1258097124 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faf7c102bf0 0x7faf7c199960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:54.407 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.400+0000 7faf8400d640 1 --2- 172.21.15.82:0/1258097124 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faf7c1019b0 0x7faf7c199420 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:54.407 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.400+0000 7faf8400d640 1 -- 172.21.15.82:0/1258097124 >> 172.21.15.82:0/1258097124 conn(0x7faf7c0fcba0 msgr2=0x7faf7c1038d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:11:54.407 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.400+0000 7faf8400d640 1 -- 172.21.15.82:0/1258097124 shutdown_connections 2024-05-20T16:11:54.407 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:54.400+0000 7faf8400d640 1 -- 172.21.15.82:0/1258097124 wait complete. 2024-05-20T16:11:54.411 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-20T16:11:54.411 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-05-20T16:11:54.411 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell --fsid 59d1600a-16c2-11ef-bc9a-c7b262605968 -- ceph fs ls 2024-05-20T16:11:54.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:54 smithi082 ceph-mon[32238]: osdmap e91: 12 total, 12 up, 12 in 2024-05-20T16:11:54.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:54 smithi082 ceph-mon[32238]: from='client.? 172.21.15.82:0/3084002116' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-20T16:11:54.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:54 smithi082 ceph-mon[32238]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-05-20T16:11:54.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:54 smithi082 ceph-mon[32238]: Cluster is now healthy 2024-05-20T16:11:54.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:54 smithi082 ceph-mon[32238]: from='client.? 172.21.15.82:0/3285430907' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T16:11:54.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:54 smithi082 ceph-mon[32238]: from='client.? 172.21.15.82:0/3661896714' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-20T16:11:54.648 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T16:11:54.787 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:54 smithi184 ceph-mon[36209]: osdmap e91: 12 total, 12 up, 12 in 2024-05-20T16:11:54.789 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:54 smithi184 ceph-mon[36209]: from='client.? 172.21.15.82:0/3084002116' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-20T16:11:54.789 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:54 smithi184 ceph-mon[36209]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-05-20T16:11:54.790 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:54 smithi184 ceph-mon[36209]: Cluster is now healthy 2024-05-20T16:11:54.790 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:54 smithi184 ceph-mon[36209]: from='client.? 172.21.15.82:0/3285430907' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T16:11:54.790 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:54 smithi184 ceph-mon[36209]: from='client.? 172.21.15.82:0/3661896714' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-20T16:11:54.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:54 smithi160 ceph-mon[36347]: osdmap e91: 12 total, 12 up, 12 in 2024-05-20T16:11:54.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:54 smithi160 ceph-mon[36347]: from='client.? 172.21.15.82:0/3084002116' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-20T16:11:54.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:54 smithi160 ceph-mon[36347]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-05-20T16:11:54.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:54 smithi160 ceph-mon[36347]: Cluster is now healthy 2024-05-20T16:11:54.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:54 smithi160 ceph-mon[36347]: from='client.? 172.21.15.82:0/3285430907' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T16:11:54.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:54 smithi160 ceph-mon[36347]: from='client.? 172.21.15.82:0/3661896714' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-20T16:11:55.700 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:55 smithi082 ceph-mon[32238]: pgmap v296: 129 pgs: 5 unknown, 124 active+clean; 579 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 211 B/s wr, 1 op/s 2024-05-20T16:11:55.803 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:55 smithi184 ceph-mon[36209]: pgmap v296: 129 pgs: 5 unknown, 124 active+clean; 579 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 211 B/s wr, 1 op/s 2024-05-20T16:11:55.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:55 smithi160 ceph-mon[36347]: pgmap v296: 129 pgs: 5 unknown, 124 active+clean; 579 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 211 B/s wr, 1 op/s 2024-05-20T16:11:56.227 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:56.224+0000 7fa25dbc4700 1 -- 172.21.15.82:0/827406120 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa258107da0 msgr2=0x7fa2581081b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:11:56.227 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:56.224+0000 7fa25dbc4700 1 --2- 172.21.15.82:0/827406120 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa258107da0 0x7fa2581081b0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fa2400099c0 tx=0x7fa240009cd0 comp rx=0 tx=0).stop 2024-05-20T16:11:56.227 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:56.225+0000 7fa25dbc4700 1 -- 172.21.15.82:0/827406120 shutdown_connections 2024-05-20T16:11:56.227 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:56.225+0000 7fa25dbc4700 1 --2- 172.21.15.82:0/827406120 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa258103470 0x7fa258103880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:56.227 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:56.225+0000 7fa25dbc4700 1 --2- 172.21.15.82:0/827406120 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa2581086f0 0x7fa258110050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:56.227 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:56.225+0000 7fa25dbc4700 1 --2- 172.21.15.82:0/827406120 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa258107da0 0x7fa2581081b0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:56.227 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:56.225+0000 7fa25dbc4700 1 -- 172.21.15.82:0/827406120 >> 172.21.15.82:0/827406120 conn(0x7fa2580feec0 msgr2=0x7fa2581012e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:11:56.227 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:56.225+0000 7fa25dbc4700 1 -- 172.21.15.82:0/827406120 shutdown_connections 2024-05-20T16:11:56.227 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:56.225+0000 7fa25dbc4700 1 -- 172.21.15.82:0/827406120 wait complete. 2024-05-20T16:11:56.228 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:56.225+0000 7fa25dbc4700 1 Processor -- start 2024-05-20T16:11:56.228 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:56.225+0000 7fa25dbc4700 1 -- start start 2024-05-20T16:11:56.228 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:56.226+0000 7fa25dbc4700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa258103470 0x7fa258199df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:11:56.228 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:56.226+0000 7fa25dbc4700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa258107da0 0x7fa25819a390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:11:56.228 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:56.226+0000 7fa25dbc4700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa2581086f0 0x7fa25819e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:11:56.228 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:56.226+0000 7fa25dbc4700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fa2581134c0 con 0x7fa258103470 2024-05-20T16:11:56.228 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:56.226+0000 7fa25dbc4700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fa258113340 con 0x7fa258107da0 2024-05-20T16:11:56.228 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:56.226+0000 7fa25dbc4700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa258113640 con 0x7fa2581086f0 2024-05-20T16:11:56.228 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:56.226+0000 7fa2577fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa258103470 0x7fa258199df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:11:56.228 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:56.226+0000 7fa2577fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa258103470 0x7fa258199df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:56376/0 (socket says 172.21.15.82:56376) 2024-05-20T16:11:56.228 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:56.226+0000 7fa2577fe700 1 -- 172.21.15.82:0/510144500 learned_addr learned my addr 172.21.15.82:0/510144500 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:11:56.228 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:56.226+0000 7fa24ffff700 1 --2- 172.21.15.82:0/510144500 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa258107da0 0x7fa25819a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:11:56.228 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:56.226+0000 7fa257fff700 1 --2- 172.21.15.82:0/510144500 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa2581086f0 0x7fa25819e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:11:56.228 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:56.226+0000 7fa2577fe700 1 -- 172.21.15.82:0/510144500 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa2581086f0 msgr2=0x7fa25819e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:11:56.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:56.226+0000 7fa2577fe700 1 --2- 172.21.15.82:0/510144500 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa2581086f0 0x7fa25819e620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:56.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:56.226+0000 7fa2577fe700 1 -- 172.21.15.82:0/510144500 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa258107da0 msgr2=0x7fa25819a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:11:56.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:56.226+0000 7fa2577fe700 1 --2- 172.21.15.82:0/510144500 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa258107da0 0x7fa25819a390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:56.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:56.226+0000 7fa2577fe700 1 -- 172.21.15.82:0/510144500 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa25819ecc0 con 0x7fa258103470 2024-05-20T16:11:56.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:56.226+0000 7fa2577fe700 1 --2- 172.21.15.82:0/510144500 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa258103470 0x7fa258199df0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fa24800b530 tx=0x7fa24800b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:11:56.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:56.227+0000 7fa2557fa700 1 -- 172.21.15.82:0/510144500 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa248015020 con 0x7fa258103470 2024-05-20T16:11:56.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:56.227+0000 7fa2557fa700 1 -- 172.21.15.82:0/510144500 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa248011d10 con 0x7fa258103470 2024-05-20T16:11:56.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:56.227+0000 7fa25dbc4700 1 -- 172.21.15.82:0/510144500 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa25819efb0 con 0x7fa258103470 2024-05-20T16:11:56.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:56.227+0000 7fa2557fa700 1 -- 172.21.15.82:0/510144500 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa24800f2e0 con 0x7fa258103470 2024-05-20T16:11:56.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:56.227+0000 7fa25dbc4700 1 -- 172.21.15.82:0/510144500 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2581a68d0 con 0x7fa258103470 2024-05-20T16:11:56.230 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:56.228+0000 7fa25dbc4700 1 -- 172.21.15.82:0/510144500 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa258050170 con 0x7fa258103470 2024-05-20T16:11:56.230 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:56.229+0000 7fa2557fa700 1 -- 172.21.15.82:0/510144500 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fa248018020 con 0x7fa258103470 2024-05-20T16:11:56.230 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:56.230+0000 7fa2557fa700 1 --2- 172.21.15.82:0/510144500 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fa2440a2c20 0x7fa2440a5130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:11:56.230 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:56.230+0000 7fa24ffff700 1 --2- 172.21.15.82:0/510144500 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fa2440a2c20 0x7fa2440a5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:11:56.230 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:56.230+0000 7fa2557fa700 1 -- 172.21.15.82:0/510144500 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8802+0+0 (secure 0 0 0) 0x7fa2480ceda0 con 0x7fa258103470 2024-05-20T16:11:56.230 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:56.230+0000 7fa24ffff700 1 --2- 172.21.15.82:0/510144500 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fa2440a2c20 0x7fa2440a5130 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fa240000c00 tx=0x7fa24001a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:11:56.233 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:56.234+0000 7fa2557fa700 1 -- 172.21.15.82:0/510144500 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fa24801f020 con 0x7fa258103470 2024-05-20T16:11:56.473 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:56.473+0000 7fa25dbc4700 1 -- 172.21.15.82:0/510144500 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fa25819b150 con 0x7fa258103470 2024-05-20T16:11:56.473 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:56.473+0000 7fa2557fa700 1 -- 172.21.15.82:0/510144500 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) ==== 53+0+88 (secure 0 0 0) 0x7fa24809bac0 con 0x7fa258103470 2024-05-20T16:11:56.473 INFO:teuthology.orchestra.run.smithi082.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-05-20T16:11:56.476 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:56.475+0000 7fa25dbc4700 1 -- 172.21.15.82:0/510144500 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fa2440a2c20 msgr2=0x7fa2440a5130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:11:56.476 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:56.475+0000 7fa25dbc4700 1 --2- 172.21.15.82:0/510144500 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fa2440a2c20 0x7fa2440a5130 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fa240000c00 tx=0x7fa24001a040 comp rx=0 tx=0).stop 2024-05-20T16:11:56.476 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:56.475+0000 7fa25dbc4700 1 -- 172.21.15.82:0/510144500 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa258103470 msgr2=0x7fa258199df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:11:56.476 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:56.475+0000 7fa25dbc4700 1 --2- 172.21.15.82:0/510144500 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa258103470 0x7fa258199df0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fa24800b530 tx=0x7fa24800b840 comp rx=0 tx=0).stop 2024-05-20T16:11:56.476 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:56.476+0000 7fa25dbc4700 1 -- 172.21.15.82:0/510144500 shutdown_connections 2024-05-20T16:11:56.476 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:56.476+0000 7fa25dbc4700 1 --2- 172.21.15.82:0/510144500 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fa2440a2c20 0x7fa2440a5130 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:56.476 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:56.476+0000 7fa25dbc4700 1 --2- 172.21.15.82:0/510144500 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa2581086f0 0x7fa25819e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:56.476 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:56.476+0000 7fa25dbc4700 1 --2- 172.21.15.82:0/510144500 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa258103470 0x7fa258199df0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:56.476 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:56.476+0000 7fa25dbc4700 1 --2- 172.21.15.82:0/510144500 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa258107da0 0x7fa25819a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:56.476 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:56.476+0000 7fa25dbc4700 1 -- 172.21.15.82:0/510144500 >> 172.21.15.82:0/510144500 conn(0x7fa2580feec0 msgr2=0x7fa258107480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:11:56.476 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:56.476+0000 7fa25dbc4700 1 -- 172.21.15.82:0/510144500 shutdown_connections 2024-05-20T16:11:56.476 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:56.476+0000 7fa25dbc4700 1 -- 172.21.15.82:0/510144500 wait complete. 2024-05-20T16:11:56.800 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:56 smithi082 ceph-mon[32238]: from='client.? 172.21.15.82:0/510144500' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-20T16:11:56.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:56 smithi160 ceph-mon[36347]: from='client.? 172.21.15.82:0/510144500' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-20T16:11:56.885 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:56 smithi184 ceph-mon[36209]: from='client.? 172.21.15.82:0/510144500' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-20T16:11:57.155 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-05-20T16:11:57.155 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-20T16:11:57.155 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi082.front.sepia.ceph.com 2024-05-20T16:11:57.155 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-05-20T16:11:57.155 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-05-20T16:11:57.155 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-05-20T16:11:57.155 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-20T16:11:57.155 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-20T16:11:57.155 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-05-20T16:11:57.155 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T16:11:57.155 DEBUG:teuthology.orchestra.run.smithi082:> ip addr 2024-05-20T16:11:57.172 INFO:teuthology.orchestra.run.smithi082.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-20T16:11:57.172 INFO:teuthology.orchestra.run.smithi082.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-20T16:11:57.172 INFO:teuthology.orchestra.run.smithi082.stdout: inet 127.0.0.1/8 scope host lo 2024-05-20T16:11:57.172 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-05-20T16:11:57.172 INFO:teuthology.orchestra.run.smithi082.stdout: inet6 ::1/128 scope host 2024-05-20T16:11:57.173 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-05-20T16:11:57.173 INFO:teuthology.orchestra.run.smithi082.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-20T16:11:57.173 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 0c:c4:7a:d8:da:c6 brd ff:ff:ff:ff:ff:ff 2024-05-20T16:11:57.173 INFO:teuthology.orchestra.run.smithi082.stdout: altname enp7s0f0 2024-05-20T16:11:57.173 INFO:teuthology.orchestra.run.smithi082.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-20T16:11:57.173 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 0c:c4:7a:d8:da:c7 brd ff:ff:ff:ff:ff:ff 2024-05-20T16:11:57.173 INFO:teuthology.orchestra.run.smithi082.stdout: altname enp7s0f1 2024-05-20T16:11:57.173 INFO:teuthology.orchestra.run.smithi082.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-20T16:11:57.173 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 0c:c4:7a:8f:d3:fe brd ff:ff:ff:ff:ff:ff 2024-05-20T16:11:57.173 INFO:teuthology.orchestra.run.smithi082.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-20T16:11:57.173 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 0c:c4:7a:8f:d3:ff brd ff:ff:ff:ff:ff:ff 2024-05-20T16:11:57.173 INFO:teuthology.orchestra.run.smithi082.stdout: inet 172.21.15.82/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-05-20T16:11:57.174 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft 42205sec preferred_lft 42205sec 2024-05-20T16:11:57.174 INFO:teuthology.orchestra.run.smithi082.stdout: inet6 fe80::ec4:7aff:fe8f:d3ff/64 scope link 2024-05-20T16:11:57.174 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-05-20T16:11:57.174 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-05-20T16:11:57.174 DEBUG:teuthology.orchestra.run.smithi082:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-20T16:11:57.175 DEBUG:teuthology.orchestra.run.smithi082:> set -e 2024-05-20T16:11:57.175 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip link add name ceph-brx type bridge 2024-05-20T16:11:57.175 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip addr flush dev ceph-brx 2024-05-20T16:11:57.175 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip link set ceph-brx up 2024-05-20T16:11:57.175 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-05-20T16:11:57.175 DEBUG:teuthology.orchestra.run.smithi082:> ') 2024-05-20T16:11:57.263 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:57 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-20T16:11:57.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-20T16:11:57.368 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T16:11:57.369 DEBUG:teuthology.orchestra.run.smithi082:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-20T16:11:57.394 INFO:teuthology.orchestra.run.smithi082.stdout:1 2024-05-20T16:11:57.396 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T16:11:57.396 DEBUG:teuthology.orchestra.run.smithi082:> route 2024-05-20T16:11:57.460 INFO:teuthology.orchestra.run.smithi082.stdout:Kernel IP routing table 2024-05-20T16:11:57.460 INFO:teuthology.orchestra.run.smithi082.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-20T16:11:57.460 INFO:teuthology.orchestra.run.smithi082.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-20T16:11:57.461 INFO:teuthology.orchestra.run.smithi082.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-20T16:11:57.461 INFO:teuthology.orchestra.run.smithi082.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-20T16:11:57.461 DEBUG:teuthology.orchestra.run.smithi082:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-20T16:11:57.461 DEBUG:teuthology.orchestra.run.smithi082:> set -e 2024-05-20T16:11:57.461 DEBUG:teuthology.orchestra.run.smithi082:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-20T16:11:57.461 DEBUG:teuthology.orchestra.run.smithi082:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-20T16:11:57.461 DEBUG:teuthology.orchestra.run.smithi082:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-20T16:11:57.461 DEBUG:teuthology.orchestra.run.smithi082:> ') 2024-05-20T16:11:57.550 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:57 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-20T16:11:57.650 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:57 smithi082 ceph-mon[32238]: pgmap v297: 129 pgs: 129 active+clean; 579 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-05-20T16:11:57.732 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-20T16:11:57.739 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T16:11:57.739 DEBUG:teuthology.orchestra.run.smithi082:> ip netns list 2024-05-20T16:11:57.754 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T16:11:57.754 DEBUG:teuthology.orchestra.run.smithi082:> ip netns list-id 2024-05-20T16:11:57.810 DEBUG:teuthology.orchestra.run.smithi082:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-20T16:11:57.810 DEBUG:teuthology.orchestra.run.smithi082:> set -e 2024-05-20T16:11:57.810 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-20T16:11:57.810 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-05-20T16:11:57.810 DEBUG:teuthology.orchestra.run.smithi082:> ') 2024-05-20T16:11:57.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:57 smithi160 ceph-mon[36347]: pgmap v297: 129 pgs: 129 active+clean; 579 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-05-20T16:11:57.885 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:57 smithi184 ceph-mon[36209]: pgmap v297: 129 pgs: 129 active+clean; 579 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-05-20T16:11:57.899 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:57 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-20T16:11:57.923 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-20T16:11:57.929 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-05-20T16:11:57.930 DEBUG:teuthology.orchestra.run.smithi082:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-20T16:11:57.930 DEBUG:teuthology.orchestra.run.smithi082:> set -e 2024-05-20T16:11:57.930 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-05-20T16:11:57.930 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-05-20T16:11:57.930 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-05-20T16:11:57.930 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-05-20T16:11:57.930 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-05-20T16:11:57.930 DEBUG:teuthology.orchestra.run.smithi082:> ') 2024-05-20T16:11:57.979 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:57 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-20T16:11:58.052 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-20T16:11:58.059 DEBUG:teuthology.orchestra.run.smithi082:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-20T16:11:58.059 DEBUG:teuthology.orchestra.run.smithi082:> set -e 2024-05-20T16:11:58.059 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip link set brx.0 up 2024-05-20T16:11:58.059 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip link set dev brx.0 master ceph-brx 2024-05-20T16:11:58.059 DEBUG:teuthology.orchestra.run.smithi082:> ') 2024-05-20T16:11:58.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:58 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-20T16:11:58.131 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-20T16:11:58.137 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-05-20T16:11:58.137 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-20T16:11:58.137 DEBUG:teuthology.orchestra.run.smithi082:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-05-20T16:11:58.152 INFO:teuthology.orchestra.run.smithi082.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-05-20T16:11:58.152 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-20T16:11:58.152 DEBUG:teuthology.orchestra.run.smithi082:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-05-20T16:11:58.202 DEBUG:teuthology.orchestra.run.smithi082:> sudo modprobe fuse 2024-05-20T16:11:58.268 DEBUG:teuthology.orchestra.run.smithi082:> cat /proc/self/mounts | awk '{print $2}' 2024-05-20T16:11:58.325 INFO:teuthology.orchestra.run.smithi082.stdout:/proc 2024-05-20T16:11:58.325 INFO:teuthology.orchestra.run.smithi082.stdout:/sys 2024-05-20T16:11:58.325 INFO:teuthology.orchestra.run.smithi082.stdout:/dev 2024-05-20T16:11:58.325 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/security 2024-05-20T16:11:58.325 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/shm 2024-05-20T16:11:58.325 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/pts 2024-05-20T16:11:58.325 INFO:teuthology.orchestra.run.smithi082.stdout:/run 2024-05-20T16:11:58.325 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/cgroup 2024-05-20T16:11:58.325 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/pstore 2024-05-20T16:11:58.325 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/bpf 2024-05-20T16:11:58.326 INFO:teuthology.orchestra.run.smithi082.stdout:/ 2024-05-20T16:11:58.326 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/selinux 2024-05-20T16:11:58.326 INFO:teuthology.orchestra.run.smithi082.stdout:/proc/sys/fs/binfmt_misc 2024-05-20T16:11:58.326 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/hugepages 2024-05-20T16:11:58.326 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/mqueue 2024-05-20T16:11:58.326 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/debug 2024-05-20T16:11:58.326 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/tracing 2024-05-20T16:11:58.326 INFO:teuthology.orchestra.run.smithi082.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-20T16:11:58.326 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/fuse/connections 2024-05-20T16:11:58.326 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/config 2024-05-20T16:11:58.326 INFO:teuthology.orchestra.run.smithi082.stdout:/run/credentials/systemd-sysctl.service 2024-05-20T16:11:58.326 INFO:teuthology.orchestra.run.smithi082.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-20T16:11:58.326 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/nfs/rpc_pipefs 2024-05-20T16:11:58.326 INFO:teuthology.orchestra.run.smithi082.stdout:/run/user/1000 2024-05-20T16:11:58.327 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph 2024-05-20T16:11:58.327 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/containers/storage/overlay 2024-05-20T16:11:58.327 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/containers/storage/overlay/ef8932cbbc046ea18df403712510a29285f43606494cbc64c0b94f2147c344ed/merged 2024-05-20T16:11:58.327 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/containers/storage/overlay/3822d964e3a06abb40ae6f674d23d8d62e9ed5f4780244cea64b83789e4f0427/merged 2024-05-20T16:11:58.327 INFO:teuthology.orchestra.run.smithi082.stdout:/run/user/0 2024-05-20T16:11:58.327 INFO:teuthology.orchestra.run.smithi082.stdout:/proc/sys/fs/binfmt_misc 2024-05-20T16:11:58.327 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/containers/storage/overlay/54211883a426bc1b68bdeaa3ab155b2c8a4538cc0517adb6b91b5866dd4307d1/merged 2024-05-20T16:11:58.327 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/containers/storage/overlay/9f26ca92f84a343d05c924135e38f848355bb9bc71bc568dfe4440a7f0c9aa47/merged 2024-05-20T16:11:58.327 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/containers/storage/overlay/e6aa55d8c02101522400b22ba82e5c537ea386eb6b426cee5244250528903658/merged 2024-05-20T16:11:58.327 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/containers/storage/overlay/89867740c746c351be39797ec011a127ae09d709fcc838f7c3d9e02576a6a3b3/merged 2024-05-20T16:11:58.327 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/containers/storage/overlay/bf4fa0f6ef83f6acc7115ccc2e038b6573c683501ff78d3dcd4192d62e167e30/merged 2024-05-20T16:11:58.327 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/containers/storage/overlay/777a12b0bdd168010f1173b01ee07601ab4988ee63b62b6e3dc8a77a4f5db08a/merged 2024-05-20T16:11:58.327 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/containers/storage/overlay/0d0ef845bb5c1d9d619ae929a2717ecdec380a4064c032b9b155194b54a89f5a/merged 2024-05-20T16:11:58.327 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/containers/storage/overlay/6cb3cfaff07d34a2f64a2169d29727c2fba9a36fd1243f641e4f8b12c8f35c65/merged 2024-05-20T16:11:58.328 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns 2024-05-20T16:11:58.328 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-20T16:11:58.328 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-20T16:11:58.328 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T16:11:58.328 DEBUG:teuthology.orchestra.run.smithi082:> ls /sys/fs/fuse/connections 2024-05-20T16:11:58.380 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-05-20T16:11:58.380 DEBUG:teuthology.orchestra.run.smithi082:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-05-20T16:11:58.423 DEBUG:teuthology.orchestra.run.smithi082:> sudo modprobe fuse 2024-05-20T16:11:58.447 DEBUG:teuthology.orchestra.run.smithi082:> cat /proc/self/mounts | awk '{print $2}' 2024-05-20T16:11:58.502 INFO:teuthology.orchestra.run.smithi082.stdout:/proc 2024-05-20T16:11:58.503 INFO:teuthology.orchestra.run.smithi082.stdout:/sys 2024-05-20T16:11:58.503 INFO:teuthology.orchestra.run.smithi082.stdout:/dev 2024-05-20T16:11:58.503 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/security 2024-05-20T16:11:58.503 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/shm 2024-05-20T16:11:58.503 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/pts 2024-05-20T16:11:58.503 INFO:teuthology.orchestra.run.smithi082.stdout:/run 2024-05-20T16:11:58.503 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/cgroup 2024-05-20T16:11:58.503 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/pstore 2024-05-20T16:11:58.503 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/bpf 2024-05-20T16:11:58.503 INFO:teuthology.orchestra.run.smithi082.stdout:/ 2024-05-20T16:11:58.503 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/selinux 2024-05-20T16:11:58.504 INFO:teuthology.orchestra.run.smithi082.stdout:/proc/sys/fs/binfmt_misc 2024-05-20T16:11:58.504 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/hugepages 2024-05-20T16:11:58.504 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/mqueue 2024-05-20T16:11:58.504 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/debug 2024-05-20T16:11:58.504 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/tracing 2024-05-20T16:11:58.504 INFO:teuthology.orchestra.run.smithi082.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-20T16:11:58.504 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/fuse/connections 2024-05-20T16:11:58.504 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/config 2024-05-20T16:11:58.504 INFO:teuthology.orchestra.run.smithi082.stdout:/run/credentials/systemd-sysctl.service 2024-05-20T16:11:58.504 INFO:teuthology.orchestra.run.smithi082.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-20T16:11:58.504 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/nfs/rpc_pipefs 2024-05-20T16:11:58.504 INFO:teuthology.orchestra.run.smithi082.stdout:/run/user/1000 2024-05-20T16:11:58.504 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph 2024-05-20T16:11:58.504 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/containers/storage/overlay 2024-05-20T16:11:58.505 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/containers/storage/overlay/ef8932cbbc046ea18df403712510a29285f43606494cbc64c0b94f2147c344ed/merged 2024-05-20T16:11:58.505 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/containers/storage/overlay/3822d964e3a06abb40ae6f674d23d8d62e9ed5f4780244cea64b83789e4f0427/merged 2024-05-20T16:11:58.505 INFO:teuthology.orchestra.run.smithi082.stdout:/run/user/0 2024-05-20T16:11:58.505 INFO:teuthology.orchestra.run.smithi082.stdout:/proc/sys/fs/binfmt_misc 2024-05-20T16:11:58.505 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/containers/storage/overlay/54211883a426bc1b68bdeaa3ab155b2c8a4538cc0517adb6b91b5866dd4307d1/merged 2024-05-20T16:11:58.505 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/containers/storage/overlay/9f26ca92f84a343d05c924135e38f848355bb9bc71bc568dfe4440a7f0c9aa47/merged 2024-05-20T16:11:58.505 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/containers/storage/overlay/e6aa55d8c02101522400b22ba82e5c537ea386eb6b426cee5244250528903658/merged 2024-05-20T16:11:58.505 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/containers/storage/overlay/89867740c746c351be39797ec011a127ae09d709fcc838f7c3d9e02576a6a3b3/merged 2024-05-20T16:11:58.505 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/containers/storage/overlay/bf4fa0f6ef83f6acc7115ccc2e038b6573c683501ff78d3dcd4192d62e167e30/merged 2024-05-20T16:11:58.505 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/containers/storage/overlay/777a12b0bdd168010f1173b01ee07601ab4988ee63b62b6e3dc8a77a4f5db08a/merged 2024-05-20T16:11:58.505 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/containers/storage/overlay/0d0ef845bb5c1d9d619ae929a2717ecdec380a4064c032b9b155194b54a89f5a/merged 2024-05-20T16:11:58.505 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/containers/storage/overlay/6cb3cfaff07d34a2f64a2169d29727c2fba9a36fd1243f641e4f8b12c8f35c65/merged 2024-05-20T16:11:58.505 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns 2024-05-20T16:11:58.505 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-20T16:11:58.505 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-20T16:11:58.506 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T16:11:58.506 DEBUG:teuthology.orchestra.run.smithi082:> ls /sys/fs/fuse/connections 2024-05-20T16:11:58.529 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi082.stderr:2024-05-20T16:11:58.529+0000 7f9f16f5e480 -1 init, newargv = 0x7f9ef80046a0 newargc=15 2024-05-20T16:11:58.529 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi082.stderr:ceph-fuse[75520]: starting ceph client 2024-05-20T16:11:58.535 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi082.stderr:ceph-fuse[75520]: starting fuse 2024-05-20T16:11:58.544 INFO:teuthology.orchestra.run.smithi082.stdout:119 2024-05-20T16:11:58.544 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [119] 2024-05-20T16:11:58.545 DEBUG:teuthology.orchestra.run.smithi082:> sudo stdin-killer -- python3 -c ' 2024-05-20T16:11:58.545 DEBUG:teuthology.orchestra.run.smithi082:> import glob 2024-05-20T16:11:58.545 DEBUG:teuthology.orchestra.run.smithi082:> import re 2024-05-20T16:11:58.545 DEBUG:teuthology.orchestra.run.smithi082:> import os 2024-05-20T16:11:58.545 DEBUG:teuthology.orchestra.run.smithi082:> import subprocess 2024-05-20T16:11:58.545 DEBUG:teuthology.orchestra.run.smithi082:> 2024-05-20T16:11:58.545 DEBUG:teuthology.orchestra.run.smithi082:> def _find_admin_socket(client_name): 2024-05-20T16:11:58.545 DEBUG:teuthology.orchestra.run.smithi082:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-05-20T16:11:58.545 DEBUG:teuthology.orchestra.run.smithi082:> files = glob.glob(asok_path) 2024-05-20T16:11:58.545 DEBUG:teuthology.orchestra.run.smithi082:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-05-20T16:11:58.545 DEBUG:teuthology.orchestra.run.smithi082:> 2024-05-20T16:11:58.545 DEBUG:teuthology.orchestra.run.smithi082:> # Given a non-glob path, it better be there 2024-05-20T16:11:58.545 DEBUG:teuthology.orchestra.run.smithi082:> if "*" not in asok_path: 2024-05-20T16:11:58.546 DEBUG:teuthology.orchestra.run.smithi082:> assert(len(files) == 1) 2024-05-20T16:11:58.546 DEBUG:teuthology.orchestra.run.smithi082:> return files[0] 2024-05-20T16:11:58.546 DEBUG:teuthology.orchestra.run.smithi082:> 2024-05-20T16:11:58.546 DEBUG:teuthology.orchestra.run.smithi082:> for f in files: 2024-05-20T16:11:58.546 DEBUG:teuthology.orchestra.run.smithi082:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-05-20T16:11:58.546 DEBUG:teuthology.orchestra.run.smithi082:> if os.path.exists("/proc/{0}".format(pid)): 2024-05-20T16:11:58.546 DEBUG:teuthology.orchestra.run.smithi082:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-05-20T16:11:58.546 DEBUG:teuthology.orchestra.run.smithi082:> contents = proc_f.read() 2024-05-20T16:11:58.546 DEBUG:teuthology.orchestra.run.smithi082:> if mountpoint in contents: 2024-05-20T16:11:58.547 DEBUG:teuthology.orchestra.run.smithi082:> return f 2024-05-20T16:11:58.547 DEBUG:teuthology.orchestra.run.smithi082:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-05-20T16:11:58.547 DEBUG:teuthology.orchestra.run.smithi082:> 2024-05-20T16:11:58.547 DEBUG:teuthology.orchestra.run.smithi082:> print(_find_admin_socket("client.admin")) 2024-05-20T16:11:58.547 DEBUG:teuthology.orchestra.run.smithi082:> ' 2024-05-20T16:11:58.664 INFO:teuthology.orchestra.run.smithi082.stdout:/var/run/ceph/ceph-client.admin.75520.asok 2024-05-20T16:11:58.667 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-20T16:11:58.674 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.75520.asok 2024-05-20T16:11:58.674 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T16:11:58.675 DEBUG:teuthology.orchestra.run.smithi082:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.75520.asok status 2024-05-20T16:11:58.819 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-05-20T16:11:58.819 INFO:teuthology.orchestra.run.smithi082.stdout: "metadata": { 2024-05-20T16:11:58.819 INFO:teuthology.orchestra.run.smithi082.stdout: "ceph_sha1": "d2b507c724ef6c8295ee0a35d185cdab167fd61d", 2024-05-20T16:11:58.819 INFO:teuthology.orchestra.run.smithi082.stdout: "ceph_version": "ceph version 19.0.0-2744-gd2b507c7 (d2b507c724ef6c8295ee0a35d185cdab167fd61d) squid (dev)", 2024-05-20T16:11:58.819 INFO:teuthology.orchestra.run.smithi082.stdout: "entity_id": "admin", 2024-05-20T16:11:58.819 INFO:teuthology.orchestra.run.smithi082.stdout: "hostname": "smithi082", 2024-05-20T16:11:58.819 INFO:teuthology.orchestra.run.smithi082.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-05-20T16:11:58.819 INFO:teuthology.orchestra.run.smithi082.stdout: "pid": "75520", 2024-05-20T16:11:58.819 INFO:teuthology.orchestra.run.smithi082.stdout: "root": "/" 2024-05-20T16:11:58.820 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:11:58.820 INFO:teuthology.orchestra.run.smithi082.stdout: "dentry_count": 0, 2024-05-20T16:11:58.820 INFO:teuthology.orchestra.run.smithi082.stdout: "dentry_pinned_count": 0, 2024-05-20T16:11:58.820 INFO:teuthology.orchestra.run.smithi082.stdout: "id": 14775, 2024-05-20T16:11:58.820 INFO:teuthology.orchestra.run.smithi082.stdout: "inst": { 2024-05-20T16:11:58.820 INFO:teuthology.orchestra.run.smithi082.stdout: "name": { 2024-05-20T16:11:58.820 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "client", 2024-05-20T16:11:58.820 INFO:teuthology.orchestra.run.smithi082.stdout: "num": 14775 2024-05-20T16:11:58.820 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:11:58.820 INFO:teuthology.orchestra.run.smithi082.stdout: "addr": { 2024-05-20T16:11:58.820 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "v1", 2024-05-20T16:11:58.820 INFO:teuthology.orchestra.run.smithi082.stdout: "addr": "192.168.0.1:0", 2024-05-20T16:11:58.820 INFO:teuthology.orchestra.run.smithi082.stdout: "nonce": 1869788468 2024-05-20T16:11:58.821 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-20T16:11:58.821 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:11:58.821 INFO:teuthology.orchestra.run.smithi082.stdout: "addr": { 2024-05-20T16:11:58.821 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "v1", 2024-05-20T16:11:58.821 INFO:teuthology.orchestra.run.smithi082.stdout: "addr": "192.168.0.1:0", 2024-05-20T16:11:58.821 INFO:teuthology.orchestra.run.smithi082.stdout: "nonce": 1869788468 2024-05-20T16:11:58.821 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:11:58.821 INFO:teuthology.orchestra.run.smithi082.stdout: "inst_str": "client.14775 192.168.0.1:0/1869788468", 2024-05-20T16:11:58.821 INFO:teuthology.orchestra.run.smithi082.stdout: "addr_str": "192.168.0.1:0/1869788468", 2024-05-20T16:11:58.821 INFO:teuthology.orchestra.run.smithi082.stdout: "inode_count": 1, 2024-05-20T16:11:58.821 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_epoch": 17, 2024-05-20T16:11:58.821 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_epoch": 91, 2024-05-20T16:11:58.821 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_epoch_barrier": 0, 2024-05-20T16:11:58.821 INFO:teuthology.orchestra.run.smithi082.stdout: "blocklisted": false, 2024-05-20T16:11:58.822 INFO:teuthology.orchestra.run.smithi082.stdout: "fs_name": "cephfs" 2024-05-20T16:11:58.822 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-05-20T16:11:58.828 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T16:11:58.828 DEBUG:teuthology.orchestra.run.smithi082:> stat --file-system '--printf=%T 2024-05-20T16:11:58.828 DEBUG:teuthology.orchestra.run.smithi082:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-05-20T16:11:58.843 INFO:teuthology.orchestra.run.smithi082.stdout:fuseblk 2024-05-20T16:11:58.844 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-05-20T16:11:58.844 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T16:11:58.844 DEBUG:teuthology.orchestra.run.smithi082:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-05-20T16:11:58.909 DEBUG:teuthology.orchestra.run.smithi082:> (cd /home/ubuntu/cephtest/mnt.admin && exec stdin-killer --timeout=900 -- bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-05-20T16:11:59.001 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:59 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-20T16:11:59.021 INFO:teuthology.orchestra.run.smithi082.stdout:# file: . 2024-05-20T16:11:59.021 INFO:teuthology.orchestra.run.smithi082.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-05-20T16:11:59.021 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-05-20T16:11:59.021 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:59 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-20T16:11:59.027 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-05-20T16:11:59.027 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-20T16:11:59.044 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-20T16:11:59.196 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:59.194+0000 7fcedcd79640 1 -- 172.21.15.82:0/12326303 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fced8105b90 msgr2=0x7fced8108070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:11:59.196 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:59.194+0000 7fcedcd79640 1 --2- 172.21.15.82:0/12326303 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fced8105b90 0x7fced8108070 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fcec0009900 tx=0x7fcec002a410 comp rx=0 tx=0).stop 2024-05-20T16:11:59.197 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:59.194+0000 7fcedcd79640 1 -- 172.21.15.82:0/12326303 shutdown_connections 2024-05-20T16:11:59.197 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:59.194+0000 7fcedcd79640 1 --2- 172.21.15.82:0/12326303 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fced8105b90 0x7fced8108070 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:59.197 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:59.194+0000 7fcedcd79640 1 --2- 172.21.15.82:0/12326303 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fced8108700 0x7fced810ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:59.197 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:59.194+0000 7fcedcd79640 1 --2- 172.21.15.82:0/12326303 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fced81031a0 0x7fced81055c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:59.197 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:59.194+0000 7fcedcd79640 1 -- 172.21.15.82:0/12326303 >> 172.21.15.82:0/12326303 conn(0x7fced80fcb80 msgr2=0x7fced80ff010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:11:59.197 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:59.194+0000 7fcedcd79640 1 -- 172.21.15.82:0/12326303 shutdown_connections 2024-05-20T16:11:59.197 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:59.194+0000 7fcedcd79640 1 -- 172.21.15.82:0/12326303 wait complete. 2024-05-20T16:11:59.197 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:59.195+0000 7fcedcd79640 1 Processor -- start 2024-05-20T16:11:59.197 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:59.195+0000 7fcedcd79640 1 -- start start 2024-05-20T16:11:59.197 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:59.195+0000 7fcedcd79640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fced81031a0 0x7fced8199310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:11:59.197 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:59.195+0000 7fcedcd79640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fced8105b90 0x7fced8199850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:11:59.197 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:59.195+0000 7fcedcd79640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fced8108700 0x7fced81a0770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:11:59.197 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:59.195+0000 7fcedcd79640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fced810e730 con 0x7fced8108700 2024-05-20T16:11:59.198 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:59.195+0000 7fcedcd79640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fced810e5b0 con 0x7fced8105b90 2024-05-20T16:11:59.198 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:59.195+0000 7fcedcd79640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fced810e8b0 con 0x7fced81031a0 2024-05-20T16:11:59.198 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:59.195+0000 7fced6d9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fced8108700 0x7fced81a0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:11:59.198 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:59.195+0000 7fced6d9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fced8108700 0x7fced81a0770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:56400/0 (socket says 172.21.15.82:56400) 2024-05-20T16:11:59.198 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:59.195+0000 7fced6d9d640 1 -- 172.21.15.82:0/1553773811 learned_addr learned my addr 172.21.15.82:0/1553773811 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:11:59.198 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:59.195+0000 7fced5d9b640 1 --2- 172.21.15.82:0/1553773811 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fced8105b90 0x7fced8199850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:11:59.198 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:59.195+0000 7fced659c640 1 --2- 172.21.15.82:0/1553773811 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fced81031a0 0x7fced8199310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:11:59.198 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:59.196+0000 7fced6d9d640 1 -- 172.21.15.82:0/1553773811 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fced81031a0 msgr2=0x7fced8199310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:11:59.198 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:59.196+0000 7fced6d9d640 1 --2- 172.21.15.82:0/1553773811 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fced81031a0 0x7fced8199310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:59.198 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:59.196+0000 7fced6d9d640 1 -- 172.21.15.82:0/1553773811 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fced8105b90 msgr2=0x7fced8199850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:11:59.198 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:59.196+0000 7fced6d9d640 1 --2- 172.21.15.82:0/1553773811 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fced8105b90 0x7fced8199850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:59.198 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:59.196+0000 7fced6d9d640 1 -- 172.21.15.82:0/1553773811 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fced81a0de0 con 0x7fced8108700 2024-05-20T16:11:59.199 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:59.196+0000 7fced6d9d640 1 --2- 172.21.15.82:0/1553773811 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fced8108700 0x7fced81a0770 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fced0009fd0 tx=0x7fced000b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:11:59.199 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:59.196+0000 7fcecf7fe640 1 -- 172.21.15.82:0/1553773811 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fced0011830 con 0x7fced8108700 2024-05-20T16:11:59.199 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:59.196+0000 7fcecf7fe640 1 -- 172.21.15.82:0/1553773811 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fced0016070 con 0x7fced8108700 2024-05-20T16:11:59.199 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:59.196+0000 7fcedcd79640 1 -- 172.21.15.82:0/1553773811 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fced81a10d0 con 0x7fced8108700 2024-05-20T16:11:59.199 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:59.196+0000 7fcecf7fe640 1 -- 172.21.15.82:0/1553773811 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fced000f2e0 con 0x7fced8108700 2024-05-20T16:11:59.199 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:59.196+0000 7fcedcd79640 1 -- 172.21.15.82:0/1553773811 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fced81a1650 con 0x7fced8108700 2024-05-20T16:11:59.200 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:59.197+0000 7fcedcd79640 1 -- 172.21.15.82:0/1553773811 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fced80fe6a0 con 0x7fced8108700 2024-05-20T16:11:59.200 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:59.198+0000 7fcecf7fe640 1 -- 172.21.15.82:0/1553773811 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fced00119d0 con 0x7fced8108700 2024-05-20T16:11:59.200 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:59.199+0000 7fcecf7fe640 1 --2- 172.21.15.82:0/1553773811 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fcec40a2b90 0x7fcec40a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:11:59.200 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:59.199+0000 7fced659c640 1 --2- 172.21.15.82:0/1553773811 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fcec40a2b90 0x7fcec40a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:11:59.200 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:59.199+0000 7fcecf7fe640 1 -- 172.21.15.82:0/1553773811 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8802+0+0 (secure 0 0 0) 0x7fced00cef10 con 0x7fced8108700 2024-05-20T16:11:59.200 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:59.200+0000 7fced659c640 1 --2- 172.21.15.82:0/1553773811 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fcec40a2b90 0x7fcec40a50a0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fced8069800 tx=0x7fcec8009680 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:11:59.202 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:59.203+0000 7fcecf7fe640 1 -- 172.21.15.82:0/1553773811 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fced009bc30 con 0x7fced8108700 2024-05-20T16:11:59.384 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:59.384+0000 7fcedcd79640 1 -- 172.21.15.82:0/1553773811 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fced8062750 con 0x7fced8108700 2024-05-20T16:11:59.385 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:59.385+0000 7fcecf7fe640 1 -- 172.21.15.82:0/1553773811 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+24153 (secure 0 0 0) 0x7fced009b530 con 0x7fced8108700 2024-05-20T16:11:59.386 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-05-20T16:11:59.386 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":91,"fsid":"59d1600a-16c2-11ef-bc9a-c7b262605968","created":"2024-05-20T16:03:46.611556+0000","modified":"2024-05-20T16:11:53.416085+0000","last_up_change":"2024-05-20T16:10:59.303479+0000","last_in_change":"2024-05-20T16:10:39.063561+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-20T16:07:09.493727+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-20T16:11:45.058035+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-20T16:11:45.943862+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-20T16:11:49.637579+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"bae842a9-03ec-490e-b56d-f7957235cdc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":1484330167},{"type":"v1","addr":"172.21.15.82:6803","nonce":1484330167}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":1484330167},{"type":"v1","addr":"172.21.15.82:6805","nonce":1484330167}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1484330167},{"type":"v1","addr":"172.21.15.82:6809","nonce":1484330167}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":1484330167},{"type":"v1","addr":"172.21.15.82:6807","nonce":1484330167}]},"public_addr":"172.21.15.82:6803/1484330167","cluster_addr":"172.21.15.82:6805/1484330167","heartbeat_back_addr":"172.21.15.82:6809/1484330167","heartbeat_front_addr":"172.21.15.82:6807/1484330167","state":["exists","up"]},{"osd":1,"uuid":"59f6a2ba-b5de-4161-81c5-92a417abacdf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":1457003568},{"type":"v1","addr":"172.21.15.160:6801","nonce":1457003568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":1457003568},{"type":"v1","addr":"172.21.15.160:6803","nonce":1457003568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6806","nonce":1457003568},{"type":"v1","addr":"172.21.15.160:6807","nonce":1457003568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":1457003568},{"type":"v1","addr":"172.21.15.160:6805","nonce":1457003568}]},"public_addr":"172.21.15.160:6801/1457003568","cluster_addr":"172.21.15.160:6803/1457003568","heartbeat_back_addr":"172.21.15.160:6807/1457003568","heartbeat_front_addr":"172.21.15.160:6805/1457003568","state":["exists","up"]},{"osd":2,"uuid":"3bbb5587-b449-41f3-bd26-67ec3d6afdfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":4051293934},{"type":"v1","addr":"172.21.15.184:6801","nonce":4051293934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":4051293934},{"type":"v1","addr":"172.21.15.184:6803","nonce":4051293934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":4051293934},{"type":"v1","addr":"172.21.15.184:6807","nonce":4051293934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":4051293934},{"type":"v1","addr":"172.21.15.184:6805","nonce":4051293934}]},"public_addr":"172.21.15.184:6801/4051293934","cluster_addr":"172.21.15.184:6803/4051293934","heartbeat_back_addr":"172.21.15.184:6807/4051293934","heartbeat_front_addr":"172.21.15.184:6805/4051293934","state":["exists","up"]},{"osd":3,"uuid":"ca651f90-09b8-4081-a6f9-40b31268a837","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":3947485599},{"type":"v1","addr":"172.21.15.82:6811","nonce":3947485599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":3947485599},{"type":"v1","addr":"172.21.15.82:6813","nonce":3947485599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":3947485599},{"type":"v1","addr":"172.21.15.82:6817","nonce":3947485599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":3947485599},{"type":"v1","addr":"172.21.15.82:6815","nonce":3947485599}]},"public_addr":"172.21.15.82:6811/3947485599","cluster_addr":"172.21.15.82:6813/3947485599","heartbeat_back_addr":"172.21.15.82:6817/3947485599","heartbeat_front_addr":"172.21.15.82:6815/3947485599","state":["exists","up"]},{"osd":4,"uuid":"cb3dfd56-f677-40cb-9c96-8319ea5ac788","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":4160830014},{"type":"v1","addr":"172.21.15.160:6809","nonce":4160830014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":4160830014},{"type":"v1","addr":"172.21.15.160:6811","nonce":4160830014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":4160830014},{"type":"v1","addr":"172.21.15.160:6815","nonce":4160830014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6812","nonce":4160830014},{"type":"v1","addr":"172.21.15.160:6813","nonce":4160830014}]},"public_addr":"172.21.15.160:6809/4160830014","cluster_addr":"172.21.15.160:6811/4160830014","heartbeat_back_addr":"172.21.15.160:6815/4160830014","heartbeat_front_addr":"172.21.15.160:6813/4160830014","state":["exists","up"]},{"osd":5,"uuid":"c70cfcd1-a4ff-461f-8c73-479b061200fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":1046282672},{"type":"v1","addr":"172.21.15.184:6809","nonce":1046282672}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":1046282672},{"type":"v1","addr":"172.21.15.184:6811","nonce":1046282672}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":1046282672},{"type":"v1","addr":"172.21.15.184:6815","nonce":1046282672}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":1046282672},{"type":"v1","addr":"172.21.15.184:6813","nonce":1046282672}]},"public_addr":"172.21.15.184:6809/1046282672","cluster_addr":"172.21.15.184:6811/1046282672","heartbeat_back_addr":"172.21.15.184:6815/1046282672","heartbeat_front_addr":"172.21.15.184:6813/1046282672","state":["exists","up"]},{"osd":6,"uuid":"d47ce001-9dd1-4830-bcc9-888601187c6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":2789736499},{"type":"v1","addr":"172.21.15.82:6819","nonce":2789736499}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":2789736499},{"type":"v1","addr":"172.21.15.82:6821","nonce":2789736499}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":2789736499},{"type":"v1","addr":"172.21.15.82:6825","nonce":2789736499}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":2789736499},{"type":"v1","addr":"172.21.15.82:6823","nonce":2789736499}]},"public_addr":"172.21.15.82:6819/2789736499","cluster_addr":"172.21.15.82:6821/2789736499","heartbeat_back_addr":"172.21.15.82:6825/2789736499","heartbeat_front_addr":"172.21.15.82:6823/2789736499","state":["exists","up"]},{"osd":7,"uuid":"25c54f43-18cb-41e4-83b4-2732b5a93889","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6816","nonce":2340683378},{"type":"v1","addr":"172.21.15.160:6817","nonce":2340683378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":2340683378},{"type":"v1","addr":"172.21.15.160:6819","nonce":2340683378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":2340683378},{"type":"v1","addr":"172.21.15.160:6823","nonce":2340683378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6820","nonce":2340683378},{"type":"v1","addr":"172.21.15.160:6821","nonce":2340683378}]},"public_addr":"172.21.15.160:6817/2340683378","cluster_addr":"172.21.15.160:6819/2340683378","heartbeat_back_addr":"172.21.15.160:6823/2340683378","heartbeat_front_addr":"172.21.15.160:6821/2340683378","state":["exists","up"]},{"osd":8,"uuid":"1f8b478d-287f-45f5-8526-d1a58c5bc07c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":3831576372},{"type":"v1","addr":"172.21.15.184:6817","nonce":3831576372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":3831576372},{"type":"v1","addr":"172.21.15.184:6819","nonce":3831576372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":3831576372},{"type":"v1","addr":"172.21.15.184:6823","nonce":3831576372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":3831576372},{"type":"v1","addr":"172.21.15.184:6821","nonce":3831576372}]},"public_addr":"172.21.15.184:6817/3831576372","cluster_addr":"172.21.15.184:6819/3831576372","heartbeat_back_addr":"172.21.15.184:6823/3831576372","heartbeat_front_addr":"172.21.15.184:6821/3831576372","state":["exists","up"]},{"osd":9,"uuid":"0d69e319-954f-4b18-95ea-31794f18dc3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":716330782},{"type":"v1","addr":"172.21.15.82:6827","nonce":716330782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":716330782},{"type":"v1","addr":"172.21.15.82:6829","nonce":716330782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":716330782},{"type":"v1","addr":"172.21.15.82:6833","nonce":716330782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":716330782},{"type":"v1","addr":"172.21.15.82:6831","nonce":716330782}]},"public_addr":"172.21.15.82:6827/716330782","cluster_addr":"172.21.15.82:6829/716330782","heartbeat_back_addr":"172.21.15.82:6833/716330782","heartbeat_front_addr":"172.21.15.82:6831/716330782","state":["exists","up"]},{"osd":10,"uuid":"39dc1b78-61cb-42ca-b5b6-1e44ac2b6500","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":2327428014},{"type":"v1","addr":"172.21.15.160:6825","nonce":2327428014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":2327428014},{"type":"v1","addr":"172.21.15.160:6827","nonce":2327428014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":2327428014},{"type":"v1","addr":"172.21.15.160:6831","nonce":2327428014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":2327428014},{"type":"v1","addr":"172.21.15.160:6829","nonce":2327428014}]},"public_addr":"172.21.15.160:6825/2327428014","cluster_addr":"172.21.15.160:6827/2327428014","heartbeat_back_addr":"172.21.15.160:6831/2327428014","heartbeat_front_addr":"172.21.15.160:6829/2327428014","state":["exists","up"]},{"osd":11,"uuid":"4306b3c1-707a-4010-987c-209ea0e948e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":3594364650},{"type":"v1","addr":"172.21.15.184:6825","nonce":3594364650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":3594364650},{"type":"v1","addr":"172.21.15.184:6827","nonce":3594364650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3594364650},{"type":"v1","addr":"172.21.15.184:6831","nonce":3594364650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3594364650},{"type":"v1","addr":"172.21.15.184:6829","nonce":3594364650}]},"public_addr":"172.21.15.184:6825/3594364650","cluster_addr":"172.21.15.184:6827/3594364650","heartbeat_back_addr":"172.21.15.184:6831/3594364650","heartbeat_front_addr":"172.21.15.184:6829/3594364650","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:06:17.855998+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:06:41.473270+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:07:05.304108+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:07:29.887472+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:07:55.387286+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:08:20.579669+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:08:46.333051+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:09:12.289668+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:09:38.180696+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:10:04.471459+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:10:31.041912+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:10:57.477266+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.82:0/2005424509":"2024-05-21T16:04:45.161949+0000","172.21.15.82:6801/1763049308":"2024-05-21T16:04:45.161949+0000","172.21.15.82:6801/1959459348":"2024-05-21T16:04:19.208291+0000","172.21.15.82:0/3131904556":"2024-05-21T16:04:19.208291+0000","172.21.15.82:6800/1959459348":"2024-05-21T16:04:19.208291+0000","172.21.15.82:0/2456966342":"2024-05-21T16:04:45.161949+0000","172.21.15.82:0/4179978833":"2024-05-21T16:04:19.208291+0000","172.21.15.82:0/1602862225":"2024-05-21T16:04:45.161949+0000","172.21.15.82:0/106101180":"2024-05-21T16:04:19.208291+0000","172.21.15.82:6800/1763049308":"2024-05-21T16:04:45.161949+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-20T16:11:59.388 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:59.388+0000 7fcedcd79640 1 -- 172.21.15.82:0/1553773811 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fcec40a2b90 msgr2=0x7fcec40a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:11:59.388 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:59.388+0000 7fcedcd79640 1 --2- 172.21.15.82:0/1553773811 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fcec40a2b90 0x7fcec40a50a0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fced8069800 tx=0x7fcec8009680 comp rx=0 tx=0).stop 2024-05-20T16:11:59.388 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:59.388+0000 7fcedcd79640 1 -- 172.21.15.82:0/1553773811 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fced8108700 msgr2=0x7fced81a0770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:11:59.388 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:59.388+0000 7fcedcd79640 1 --2- 172.21.15.82:0/1553773811 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fced8108700 0x7fced81a0770 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fced0009fd0 tx=0x7fced000b8e0 comp rx=0 tx=0).stop 2024-05-20T16:11:59.388 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:59.388+0000 7fcedcd79640 1 -- 172.21.15.82:0/1553773811 shutdown_connections 2024-05-20T16:11:59.388 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:59.388+0000 7fcedcd79640 1 --2- 172.21.15.82:0/1553773811 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fcec40a2b90 0x7fcec40a50a0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:59.388 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:59.388+0000 7fcedcd79640 1 --2- 172.21.15.82:0/1553773811 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fced8108700 0x7fced81a0770 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:59.388 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:59.388+0000 7fcedcd79640 1 --2- 172.21.15.82:0/1553773811 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fced8105b90 0x7fced8199850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:59.388 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:59.388+0000 7fcedcd79640 1 --2- 172.21.15.82:0/1553773811 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fced81031a0 0x7fced8199310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:59.389 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:59.388+0000 7fcedcd79640 1 -- 172.21.15.82:0/1553773811 >> 172.21.15.82:0/1553773811 conn(0x7fced80fcb80 msgr2=0x7fced81040a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:11:59.389 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:59.388+0000 7fcedcd79640 1 -- 172.21.15.82:0/1553773811 shutdown_connections 2024-05-20T16:11:59.389 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:59.388+0000 7fcedcd79640 1 -- 172.21.15.82:0/1553773811 wait complete. 2024-05-20T16:11:59.399 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-20T16:07:09.493727+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-20T16:11:45.058035+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-20T16:11:45.943862+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-20T16:11:49.637579+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '91', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-20T16:11:59.400 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-20T16:11:59.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:59.517+0000 7f9fffbc7640 1 -- 172.21.15.82:0/1824157482 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9ff81078d0 msgr2=0x7f9ff8109cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:11:59.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:59.517+0000 7f9fffbc7640 1 --2- 172.21.15.82:0/1824157482 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9ff81078d0 0x7f9ff8109cf0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f9fe8009ca0 tx=0x7f9fe802a8c0 comp rx=0 tx=0).stop 2024-05-20T16:11:59.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:59.517+0000 7f9fffbc7640 1 -- 172.21.15.82:0/1824157482 shutdown_connections 2024-05-20T16:11:59.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:59.517+0000 7f9fffbc7640 1 --2- 172.21.15.82:0/1824157482 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9ff81078d0 0x7f9ff8109cf0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:59.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:59.517+0000 7f9fffbc7640 1 --2- 172.21.15.82:0/1824157482 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9ff8102bd0 0x7f9ff8102fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:59.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:59.517+0000 7f9fffbc7640 1 --2- 172.21.15.82:0/1824157482 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9ff8101990 0x7f9ff8102690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:59.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:59.517+0000 7f9fffbc7640 1 -- 172.21.15.82:0/1824157482 >> 172.21.15.82:0/1824157482 conn(0x7f9ff80fcb80 msgr2=0x7f9ff80ff030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:11:59.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:59.517+0000 7f9fffbc7640 1 -- 172.21.15.82:0/1824157482 shutdown_connections 2024-05-20T16:11:59.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:59.517+0000 7f9fffbc7640 1 -- 172.21.15.82:0/1824157482 wait complete. 2024-05-20T16:11:59.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:59.518+0000 7f9fffbc7640 1 Processor -- start 2024-05-20T16:11:59.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:59.518+0000 7f9fffbc7640 1 -- start start 2024-05-20T16:11:59.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:59.518+0000 7f9fffbc7640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9ff8101990 0x7f9ff80698d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:11:59.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:59.518+0000 7f9fffbc7640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9ff8102bd0 0x7f9ff819e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:11:59.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:59.518+0000 7f9fffbc7640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9ff81078d0 0x7f9ff81a0830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:11:59.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:59.518+0000 7f9fffbc7640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f9ff810d610 con 0x7f9ff81078d0 2024-05-20T16:11:59.521 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:59.518+0000 7f9fffbc7640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f9ff810d490 con 0x7f9ff8102bd0 2024-05-20T16:11:59.521 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:59.518+0000 7f9fffbc7640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f9ff810d790 con 0x7f9ff8101990 2024-05-20T16:11:59.521 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:59.518+0000 7f9ffe164640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9ff81078d0 0x7f9ff81a0830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:11:59.521 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:59.518+0000 7f9ffe164640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9ff81078d0 0x7f9ff81a0830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:56412/0 (socket says 172.21.15.82:56412) 2024-05-20T16:11:59.521 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:59.518+0000 7f9ffe164640 1 -- 172.21.15.82:0/2502860420 learned_addr learned my addr 172.21.15.82:0/2502860420 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:11:59.521 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:59.518+0000 7f9ffd162640 1 --2- 172.21.15.82:0/2502860420 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9ff8102bd0 0x7f9ff819e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:11:59.521 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:59.518+0000 7f9ffd963640 1 --2- 172.21.15.82:0/2502860420 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9ff8101990 0x7f9ff80698d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:11:59.521 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:59.518+0000 7f9ffe164640 1 -- 172.21.15.82:0/2502860420 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9ff8101990 msgr2=0x7f9ff80698d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:11:59.521 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:59.518+0000 7f9ffe164640 1 --2- 172.21.15.82:0/2502860420 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9ff8101990 0x7f9ff80698d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:59.521 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:59.518+0000 7f9ffe164640 1 -- 172.21.15.82:0/2502860420 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9ff8102bd0 msgr2=0x7f9ff819e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:11:59.521 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:59.518+0000 7f9ffe164640 1 --2- 172.21.15.82:0/2502860420 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9ff8102bd0 0x7f9ff819e2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:59.521 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:59.518+0000 7f9ffe164640 1 -- 172.21.15.82:0/2502860420 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ff81a1050 con 0x7f9ff81078d0 2024-05-20T16:11:59.521 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:59.518+0000 7f9ffd162640 1 --2- 172.21.15.82:0/2502860420 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9ff8102bd0 0x7f9ff819e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T16:11:59.521 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:59.518+0000 7f9ffd963640 1 --2- 172.21.15.82:0/2502860420 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9ff8101990 0x7f9ff80698d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T16:11:59.522 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:59.519+0000 7f9ffe164640 1 --2- 172.21.15.82:0/2502860420 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9ff81078d0 0x7f9ff81a0830 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f9fe8005cc0 tx=0x7f9fe8004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:11:59.522 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:59.519+0000 7f9ff6ffd640 1 -- 172.21.15.82:0/2502860420 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fe8003b70 con 0x7f9ff81078d0 2024-05-20T16:11:59.522 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:59.519+0000 7f9ff6ffd640 1 -- 172.21.15.82:0/2502860420 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9fe8003d10 con 0x7f9ff81078d0 2024-05-20T16:11:59.522 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:59.519+0000 7f9fffbc7640 1 -- 172.21.15.82:0/2502860420 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ff81a12e0 con 0x7f9ff81078d0 2024-05-20T16:11:59.522 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:59.519+0000 7f9ff6ffd640 1 -- 172.21.15.82:0/2502860420 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fe802fbd0 con 0x7f9ff81078d0 2024-05-20T16:11:59.522 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:59.519+0000 7f9fffbc7640 1 -- 172.21.15.82:0/2502860420 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ff81a1830 con 0x7f9ff81078d0 2024-05-20T16:11:59.523 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:59.520+0000 7f9fffbc7640 1 -- 172.21.15.82:0/2502860420 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ff81066e0 con 0x7f9ff81078d0 2024-05-20T16:11:59.523 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:59.522+0000 7f9ff6ffd640 1 -- 172.21.15.82:0/2502860420 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f9fe802fd70 con 0x7f9ff81078d0 2024-05-20T16:11:59.523 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:59.523+0000 7f9ff6ffd640 1 --2- 172.21.15.82:0/2502860420 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f9fe00a2b90 0x7f9fe00a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:11:59.523 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:59.523+0000 7f9ffd963640 1 --2- 172.21.15.82:0/2502860420 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f9fe00a2b90 0x7f9fe00a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:11:59.523 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:59.523+0000 7f9ff6ffd640 1 -- 172.21.15.82:0/2502860420 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8802+0+0 (secure 0 0 0) 0x7f9fe80f0730 con 0x7f9ff81078d0 2024-05-20T16:11:59.523 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:59.523+0000 7f9ffd963640 1 --2- 172.21.15.82:0/2502860420 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f9fe00a2b90 0x7f9fe00a50a0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f9ff8102590 tx=0x7f9ff0005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:11:59.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:59.526+0000 7f9ff6ffd640 1 -- 172.21.15.82:0/2502860420 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f9fe80bd480 con 0x7f9ff81078d0 2024-05-20T16:11:59.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:11:59 smithi082 ceph-mon[32238]: pgmap v298: 129 pgs: 129 active+clean; 579 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 130 B/s wr, 0 op/s 2024-05-20T16:11:59.635 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:11:59 smithi184 ceph-mon[36209]: pgmap v298: 129 pgs: 129 active+clean; 579 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 130 B/s wr, 0 op/s 2024-05-20T16:11:59.704 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:59.704+0000 7f9fffbc7640 1 -- 172.21.15.82:0/2502860420 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f9ff802d3c0 con 0x7f9ff81078d0 2024-05-20T16:11:59.704 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:59.704+0000 7f9ff6ffd640 1 -- 172.21.15.82:0/2502860420 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v91) ==== 93+0+10 (secure 0 0 0) 0x7f9fe80bcd80 con 0x7f9ff81078d0 2024-05-20T16:11:59.704 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 1 2024-05-20T16:11:59.706 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:59.706+0000 7f9fffbc7640 1 -- 172.21.15.82:0/2502860420 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f9fe00a2b90 msgr2=0x7f9fe00a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:11:59.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:59.706+0000 7f9fffbc7640 1 --2- 172.21.15.82:0/2502860420 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f9fe00a2b90 0x7f9fe00a50a0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f9ff8102590 tx=0x7f9ff0005f90 comp rx=0 tx=0).stop 2024-05-20T16:11:59.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:59.706+0000 7f9fffbc7640 1 -- 172.21.15.82:0/2502860420 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9ff81078d0 msgr2=0x7f9ff81a0830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:11:59.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:59.706+0000 7f9fffbc7640 1 --2- 172.21.15.82:0/2502860420 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9ff81078d0 0x7f9ff81a0830 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f9fe8005cc0 tx=0x7f9fe8004ab0 comp rx=0 tx=0).stop 2024-05-20T16:11:59.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:59.706+0000 7f9fffbc7640 1 -- 172.21.15.82:0/2502860420 shutdown_connections 2024-05-20T16:11:59.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:59.706+0000 7f9fffbc7640 1 --2- 172.21.15.82:0/2502860420 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f9fe00a2b90 0x7f9fe00a50a0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:59.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:59.706+0000 7f9fffbc7640 1 --2- 172.21.15.82:0/2502860420 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9ff81078d0 0x7f9ff81a0830 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:59.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:59.706+0000 7f9fffbc7640 1 --2- 172.21.15.82:0/2502860420 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9ff8102bd0 0x7f9ff819e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:59.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:59.706+0000 7f9fffbc7640 1 --2- 172.21.15.82:0/2502860420 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9ff8101990 0x7f9ff80698d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:59.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:59.706+0000 7f9fffbc7640 1 -- 172.21.15.82:0/2502860420 >> 172.21.15.82:0/2502860420 conn(0x7f9ff80fcb80 msgr2=0x7f9ff81038b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:11:59.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:59.706+0000 7f9fffbc7640 1 -- 172.21.15.82:0/2502860420 shutdown_connections 2024-05-20T16:11:59.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:59.706+0000 7f9fffbc7640 1 -- 172.21.15.82:0/2502860420 wait complete. 2024-05-20T16:11:59.717 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-20T16:11:59.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:11:59 smithi160 ceph-mon[36347]: pgmap v298: 129 pgs: 129 active+clean; 579 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 130 B/s wr, 0 op/s 2024-05-20T16:11:59.831 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:59.831+0000 7f411c6ca640 1 -- 172.21.15.82:0/1422658508 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4114108750 msgr2=0x7f411410ac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:11:59.832 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:59.831+0000 7f411c6ca640 1 --2- 172.21.15.82:0/1422658508 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4114108750 0x7f411410ac30 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f410c009ca0 tx=0x7f410c02a8c0 comp rx=0 tx=0).stop 2024-05-20T16:11:59.832 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:59.831+0000 7f411c6ca640 1 -- 172.21.15.82:0/1422658508 shutdown_connections 2024-05-20T16:11:59.832 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:59.831+0000 7f411c6ca640 1 --2- 172.21.15.82:0/1422658508 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4114108750 0x7f411410ac30 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:59.832 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:59.831+0000 7f411c6ca640 1 --2- 172.21.15.82:0/1422658508 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4114105be0 0x7f41141080c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:59.832 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:59.831+0000 7f411c6ca640 1 --2- 172.21.15.82:0/1422658508 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f41141031f0 0x7f4114105610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:59.832 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:59.831+0000 7f411c6ca640 1 -- 172.21.15.82:0/1422658508 >> 172.21.15.82:0/1422658508 conn(0x7f41140fcb70 msgr2=0x7f41140ff040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:11:59.832 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:59.831+0000 7f411c6ca640 1 -- 172.21.15.82:0/1422658508 shutdown_connections 2024-05-20T16:11:59.832 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:59.832+0000 7f411c6ca640 1 -- 172.21.15.82:0/1422658508 wait complete. 2024-05-20T16:11:59.834 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:59.832+0000 7f411c6ca640 1 Processor -- start 2024-05-20T16:11:59.834 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:59.832+0000 7f411c6ca640 1 -- start start 2024-05-20T16:11:59.834 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:59.832+0000 7f411c6ca640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f41141031f0 0x7f41141991e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:11:59.834 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:59.832+0000 7f411c6ca640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4114105be0 0x7f4114199720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:11:59.834 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:59.832+0000 7f411c6ca640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4114108750 0x7f41141a06b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:11:59.834 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:59.832+0000 7f411c6ca640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f411410e6c0 con 0x7f41141031f0 2024-05-20T16:11:59.835 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:59.832+0000 7f411c6ca640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f411410e540 con 0x7f4114108750 2024-05-20T16:11:59.835 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:59.832+0000 7f411c6ca640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f411410e840 con 0x7f4114105be0 2024-05-20T16:11:59.835 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:59.832+0000 7f411a466640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f41141031f0 0x7f41141991e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:11:59.835 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:59.832+0000 7f411a466640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f41141031f0 0x7f41141991e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43336/0 (socket says 172.21.15.82:43336) 2024-05-20T16:11:59.835 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:59.832+0000 7f411a466640 1 -- 172.21.15.82:0/3864685909 learned_addr learned my addr 172.21.15.82:0/3864685909 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:11:59.835 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:59.833+0000 7f4119c65640 1 --2- 172.21.15.82:0/3864685909 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4114105be0 0x7f4114199720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:11:59.835 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:59.833+0000 7f411ac67640 1 --2- 172.21.15.82:0/3864685909 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4114108750 0x7f41141a06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:11:59.835 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:59.833+0000 7f411a466640 1 -- 172.21.15.82:0/3864685909 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4114105be0 msgr2=0x7f4114199720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:11:59.835 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:59.833+0000 7f411a466640 1 --2- 172.21.15.82:0/3864685909 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4114105be0 0x7f4114199720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:59.835 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:59.833+0000 7f411a466640 1 -- 172.21.15.82:0/3864685909 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4114108750 msgr2=0x7f41141a06b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:11:59.835 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:59.833+0000 7f411a466640 1 --2- 172.21.15.82:0/3864685909 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4114108750 0x7f41141a06b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:11:59.835 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:59.833+0000 7f411a466640 1 -- 172.21.15.82:0/3864685909 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41141a0d20 con 0x7f41141031f0 2024-05-20T16:11:59.835 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:59.833+0000 7f411a466640 1 --2- 172.21.15.82:0/3864685909 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f41141031f0 0x7f41141991e0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f411000d950 tx=0x7f411000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:11:59.835 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:59.833+0000 7f410b7fe640 1 -- 172.21.15.82:0/3864685909 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4110017030 con 0x7f41141031f0 2024-05-20T16:11:59.836 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:59.833+0000 7f410b7fe640 1 -- 172.21.15.82:0/3864685909 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4110004020 con 0x7f41141031f0 2024-05-20T16:11:59.836 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:59.833+0000 7f411c6ca640 1 -- 172.21.15.82:0/3864685909 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41141a0fb0 con 0x7f41141031f0 2024-05-20T16:11:59.836 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:59.833+0000 7f410b7fe640 1 -- 172.21.15.82:0/3864685909 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41100047a0 con 0x7f41141031f0 2024-05-20T16:11:59.836 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:59.833+0000 7f411c6ca640 1 -- 172.21.15.82:0/3864685909 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41141a1470 con 0x7f41141031f0 2024-05-20T16:11:59.836 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:59.834+0000 7f411c6ca640 1 -- 172.21.15.82:0/3864685909 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41140fe690 con 0x7f41141031f0 2024-05-20T16:11:59.837 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:59.835+0000 7f410b7fe640 1 -- 172.21.15.82:0/3864685909 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f41100041d0 con 0x7f41141031f0 2024-05-20T16:11:59.837 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:59.836+0000 7f410b7fe640 1 --2- 172.21.15.82:0/3864685909 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f41000a2b90 0x7f41000a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:11:59.837 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:59.836+0000 7f4119c65640 1 --2- 172.21.15.82:0/3864685909 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f41000a2b90 0x7f41000a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:11:59.837 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:59.836+0000 7f410b7fe640 1 -- 172.21.15.82:0/3864685909 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8802+0+0 (secure 0 0 0) 0x7f41100cf230 con 0x7f41141031f0 2024-05-20T16:11:59.837 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:59.837+0000 7f4119c65640 1 --2- 172.21.15.82:0/3864685909 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f41000a2b90 0x7f41000a50a0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f411419a280 tx=0x7f4104005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:11:59.839 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:11:59.840+0000 7f410b7fe640 1 -- 172.21.15.82:0/3864685909 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f411009bff0 con 0x7f41141031f0 2024-05-20T16:12:00.017 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:00.017+0000 7f411c6ca640 1 -- 172.21.15.82:0/3864685909 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f4114062750 con 0x7f41141031f0 2024-05-20T16:12:00.017 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:00.017+0000 7f410b7fe640 1 -- 172.21.15.82:0/3864685909 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v91) ==== 104+0+11 (secure 0 0 0) 0x7f411009b8f0 con 0x7f41141031f0 2024-05-20T16:12:00.017 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 32 2024-05-20T16:12:00.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:00.019+0000 7f411c6ca640 1 -- 172.21.15.82:0/3864685909 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f41000a2b90 msgr2=0x7f41000a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:12:00.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:00.019+0000 7f411c6ca640 1 --2- 172.21.15.82:0/3864685909 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f41000a2b90 0x7f41000a50a0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f411419a280 tx=0x7f4104005f90 comp rx=0 tx=0).stop 2024-05-20T16:12:00.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:00.019+0000 7f411c6ca640 1 -- 172.21.15.82:0/3864685909 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f41141031f0 msgr2=0x7f41141991e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:12:00.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:00.019+0000 7f411c6ca640 1 --2- 172.21.15.82:0/3864685909 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f41141031f0 0x7f41141991e0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f411000d950 tx=0x7f411000de10 comp rx=0 tx=0).stop 2024-05-20T16:12:00.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:00.019+0000 7f411c6ca640 1 -- 172.21.15.82:0/3864685909 shutdown_connections 2024-05-20T16:12:00.020 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:00.019+0000 7f411c6ca640 1 --2- 172.21.15.82:0/3864685909 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f41000a2b90 0x7f41000a50a0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:00.020 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:00.019+0000 7f411c6ca640 1 --2- 172.21.15.82:0/3864685909 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4114108750 0x7f41141a06b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:00.020 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:00.019+0000 7f411c6ca640 1 --2- 172.21.15.82:0/3864685909 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4114105be0 0x7f4114199720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:00.020 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:00.019+0000 7f411c6ca640 1 --2- 172.21.15.82:0/3864685909 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f41141031f0 0x7f41141991e0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:00.020 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:00.019+0000 7f411c6ca640 1 -- 172.21.15.82:0/3864685909 >> 172.21.15.82:0/3864685909 conn(0x7f41140fcb70 msgr2=0x7f41141040f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:12:00.020 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:00.019+0000 7f411c6ca640 1 -- 172.21.15.82:0/3864685909 shutdown_connections 2024-05-20T16:12:00.020 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:00.019+0000 7f411c6ca640 1 -- 172.21.15.82:0/3864685909 wait complete. 2024-05-20T16:12:00.031 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-20T16:12:00.144 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:00.144+0000 7f7037509640 1 -- 172.21.15.82:0/1930038199 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7030104060 msgr2=0x7f7030104470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:12:00.144 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:00.144+0000 7f7037509640 1 --2- 172.21.15.82:0/1930038199 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7030104060 0x7f7030104470 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f702c009900 tx=0x7f702c02a410 comp rx=0 tx=0).stop 2024-05-20T16:12:00.144 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:00.144+0000 7f7037509640 1 -- 172.21.15.82:0/1930038199 shutdown_connections 2024-05-20T16:12:00.145 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:00.144+0000 7f7037509640 1 --2- 172.21.15.82:0/1930038199 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7030101210 0x7f703010ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:00.145 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:00.144+0000 7f7037509640 1 --2- 172.21.15.82:0/1930038199 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7030100710 0x7f7030100b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:00.145 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:00.144+0000 7f7037509640 1 --2- 172.21.15.82:0/1930038199 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7030104060 0x7f7030104470 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:00.145 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:00.144+0000 7f7037509640 1 -- 172.21.15.82:0/1930038199 >> 172.21.15.82:0/1930038199 conn(0x7f70300fc330 msgr2=0x7f70300fe7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:12:00.145 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:00.144+0000 7f7037509640 1 -- 172.21.15.82:0/1930038199 shutdown_connections 2024-05-20T16:12:00.145 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:00.144+0000 7f7037509640 1 -- 172.21.15.82:0/1930038199 wait complete. 2024-05-20T16:12:00.147 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:00.145+0000 7f7037509640 1 Processor -- start 2024-05-20T16:12:00.147 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:00.145+0000 7f7037509640 1 -- start start 2024-05-20T16:12:00.147 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:00.145+0000 7f7037509640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7030100710 0x7f70301992c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:12:00.147 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:00.145+0000 7f7037509640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7030101210 0x7f7030199800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:12:00.147 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:00.145+0000 7f7037509640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7030104060 0x7f70301a0790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:12:00.148 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:00.145+0000 7f7037509640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f703010e700 con 0x7f7030104060 2024-05-20T16:12:00.148 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:00.145+0000 7f7037509640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f703010e580 con 0x7f7030101210 2024-05-20T16:12:00.148 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:00.145+0000 7f7037509640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f703010e880 con 0x7f7030100710 2024-05-20T16:12:00.148 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:00.145+0000 7f7035aa6640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7030104060 0x7f70301a0790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:12:00.148 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:00.145+0000 7f7035aa6640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7030104060 0x7f70301a0790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43352/0 (socket says 172.21.15.82:43352) 2024-05-20T16:12:00.148 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:00.145+0000 7f7035aa6640 1 -- 172.21.15.82:0/4064022853 learned_addr learned my addr 172.21.15.82:0/4064022853 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:12:00.148 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:00.145+0000 7f7034aa4640 1 --2- 172.21.15.82:0/4064022853 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7030101210 0x7f7030199800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:12:00.148 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:00.146+0000 7f70352a5640 1 --2- 172.21.15.82:0/4064022853 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7030100710 0x7f70301992c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:12:00.149 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:00.146+0000 7f7035aa6640 1 -- 172.21.15.82:0/4064022853 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7030100710 msgr2=0x7f70301992c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:12:00.149 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:00.146+0000 7f7035aa6640 1 --2- 172.21.15.82:0/4064022853 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7030100710 0x7f70301992c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:00.149 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:00.146+0000 7f7035aa6640 1 -- 172.21.15.82:0/4064022853 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7030101210 msgr2=0x7f7030199800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:12:00.149 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:00.146+0000 7f7035aa6640 1 --2- 172.21.15.82:0/4064022853 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7030101210 0x7f7030199800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:00.149 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:00.146+0000 7f7035aa6640 1 -- 172.21.15.82:0/4064022853 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70301a0e00 con 0x7f7030104060 2024-05-20T16:12:00.149 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:00.146+0000 7f7034aa4640 1 --2- 172.21.15.82:0/4064022853 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7030101210 0x7f7030199800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T16:12:00.149 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:00.146+0000 7f70352a5640 1 --2- 172.21.15.82:0/4064022853 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7030100710 0x7f70301992c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T16:12:00.149 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:00.146+0000 7f7035aa6640 1 --2- 172.21.15.82:0/4064022853 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7030104060 0x7f70301a0790 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f702800eb10 tx=0x7f702800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:12:00.149 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:00.146+0000 7f70267fc640 1 -- 172.21.15.82:0/4064022853 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7028004b20 con 0x7f7030104060 2024-05-20T16:12:00.149 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:00.146+0000 7f70267fc640 1 -- 172.21.15.82:0/4064022853 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7028004cc0 con 0x7f7030104060 2024-05-20T16:12:00.149 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:00.146+0000 7f7037509640 1 -- 172.21.15.82:0/4064022853 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70301a10f0 con 0x7f7030104060 2024-05-20T16:12:00.149 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:00.146+0000 7f70267fc640 1 -- 172.21.15.82:0/4064022853 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7028010760 con 0x7f7030104060 2024-05-20T16:12:00.149 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:00.146+0000 7f7037509640 1 -- 172.21.15.82:0/4064022853 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70301a15f0 con 0x7f7030104060 2024-05-20T16:12:00.149 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:00.147+0000 7f7037509640 1 -- 172.21.15.82:0/4064022853 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70301079f0 con 0x7f7030104060 2024-05-20T16:12:00.150 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:00.148+0000 7f70267fc640 1 -- 172.21.15.82:0/4064022853 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f7028005000 con 0x7f7030104060 2024-05-20T16:12:00.150 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:00.149+0000 7f70267fc640 1 --2- 172.21.15.82:0/4064022853 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f701c0a2b90 0x7f701c0a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:12:00.150 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:00.149+0000 7f70352a5640 1 --2- 172.21.15.82:0/4064022853 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f701c0a2b90 0x7f701c0a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:12:00.150 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:00.149+0000 7f70267fc640 1 -- 172.21.15.82:0/4064022853 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8802+0+0 (secure 0 0 0) 0x7f70280cfbb0 con 0x7f7030104060 2024-05-20T16:12:00.151 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:00.150+0000 7f70352a5640 1 --2- 172.21.15.82:0/4064022853 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f701c0a2b90 0x7f701c0a50a0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f702c000c00 tx=0x7f702c031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:12:00.152 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:00.153+0000 7f70267fc640 1 -- 172.21.15.82:0/4064022853 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f7028053850 con 0x7f7030104060 2024-05-20T16:12:00.330 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:00.330+0000 7f7037509640 1 -- 172.21.15.82:0/4064022853 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f7030062750 con 0x7f7030104060 2024-05-20T16:12:00.330 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:00.330+0000 7f70267fc640 1 -- 172.21.15.82:0/4064022853 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v91) ==== 100+0+11 (secure 0 0 0) 0x7f702809c8d0 con 0x7f7030104060 2024-05-20T16:12:00.330 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 64 2024-05-20T16:12:00.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:00.332+0000 7f7037509640 1 -- 172.21.15.82:0/4064022853 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f701c0a2b90 msgr2=0x7f701c0a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:12:00.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:00.332+0000 7f7037509640 1 --2- 172.21.15.82:0/4064022853 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f701c0a2b90 0x7f701c0a50a0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f702c000c00 tx=0x7f702c031400 comp rx=0 tx=0).stop 2024-05-20T16:12:00.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:00.332+0000 7f7037509640 1 -- 172.21.15.82:0/4064022853 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7030104060 msgr2=0x7f70301a0790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:12:00.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:00.332+0000 7f7037509640 1 --2- 172.21.15.82:0/4064022853 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7030104060 0x7f70301a0790 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f702800eb10 tx=0x7f702800c2d0 comp rx=0 tx=0).stop 2024-05-20T16:12:00.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:00.332+0000 7f7037509640 1 -- 172.21.15.82:0/4064022853 shutdown_connections 2024-05-20T16:12:00.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:00.332+0000 7f7037509640 1 --2- 172.21.15.82:0/4064022853 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f701c0a2b90 0x7f701c0a50a0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:00.333 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:00.332+0000 7f7037509640 1 --2- 172.21.15.82:0/4064022853 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7030104060 0x7f70301a0790 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:00.333 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:00.332+0000 7f7037509640 1 --2- 172.21.15.82:0/4064022853 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7030101210 0x7f7030199800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:00.333 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:00.332+0000 7f7037509640 1 --2- 172.21.15.82:0/4064022853 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7030100710 0x7f70301992c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:00.333 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:00.332+0000 7f7037509640 1 -- 172.21.15.82:0/4064022853 >> 172.21.15.82:0/4064022853 conn(0x7f70300fc330 msgr2=0x7f7030104bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:12:00.333 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:00.332+0000 7f7037509640 1 -- 172.21.15.82:0/4064022853 shutdown_connections 2024-05-20T16:12:00.333 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:00.332+0000 7f7037509640 1 -- 172.21.15.82:0/4064022853 wait complete. 2024-05-20T16:12:00.343 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-20T16:12:00.460 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:00.458+0000 7fba5739d640 1 -- 172.21.15.82:0/1983898770 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fba501029f0 msgr2=0x7fba50102e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:12:00.460 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:00.458+0000 7fba5739d640 1 --2- 172.21.15.82:0/1983898770 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fba501029f0 0x7fba50102e00 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fba40009900 tx=0x7fba4002a410 comp rx=0 tx=0).stop 2024-05-20T16:12:00.461 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:00.458+0000 7fba5739d640 1 -- 172.21.15.82:0/1983898770 shutdown_connections 2024-05-20T16:12:00.461 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:00.458+0000 7fba5739d640 1 --2- 172.21.15.82:0/1983898770 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fba501029f0 0x7fba50102e00 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:00.461 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:00.458+0000 7fba5739d640 1 --2- 172.21.15.82:0/1983898770 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fba501076f0 0x7fba50109b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:00.461 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:00.458+0000 7fba5739d640 1 --2- 172.21.15.82:0/1983898770 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fba501017b0 0x7fba501024b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:00.461 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:00.458+0000 7fba5739d640 1 -- 172.21.15.82:0/1983898770 >> 172.21.15.82:0/1983898770 conn(0x7fba500fc9a0 msgr2=0x7fba500fee50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:12:00.461 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:00.458+0000 7fba5739d640 1 -- 172.21.15.82:0/1983898770 shutdown_connections 2024-05-20T16:12:00.461 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:00.458+0000 7fba5739d640 1 -- 172.21.15.82:0/1983898770 wait complete. 2024-05-20T16:12:00.461 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:00.459+0000 7fba5739d640 1 Processor -- start 2024-05-20T16:12:00.461 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:00.459+0000 7fba5739d640 1 -- start start 2024-05-20T16:12:00.461 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:00.459+0000 7fba5739d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fba501017b0 0x7fba50069860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:12:00.461 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:00.459+0000 7fba5739d640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fba501029f0 0x7fba5019de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:12:00.461 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:00.459+0000 7fba5739d640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fba501076f0 0x7fba501a03e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:12:00.462 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:00.459+0000 7fba5739d640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fba5010d430 con 0x7fba501017b0 2024-05-20T16:12:00.462 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:00.459+0000 7fba5739d640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fba5010d2b0 con 0x7fba501029f0 2024-05-20T16:12:00.462 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:00.459+0000 7fba5739d640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fba5010d5b0 con 0x7fba501076f0 2024-05-20T16:12:00.462 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:00.459+0000 7fba5639b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fba501017b0 0x7fba50069860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:12:00.462 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:00.459+0000 7fba5639b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fba501017b0 0x7fba50069860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43366/0 (socket says 172.21.15.82:43366) 2024-05-20T16:12:00.462 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:00.459+0000 7fba5639b640 1 -- 172.21.15.82:0/2366327769 learned_addr learned my addr 172.21.15.82:0/2366327769 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:12:00.462 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:00.459+0000 7fba56b9c640 1 --2- 172.21.15.82:0/2366327769 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fba501076f0 0x7fba501a03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:12:00.462 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:00.459+0000 7fba5639b640 1 -- 172.21.15.82:0/2366327769 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fba501076f0 msgr2=0x7fba501a03e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:12:00.462 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:00.459+0000 7fba5639b640 1 --2- 172.21.15.82:0/2366327769 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fba501076f0 0x7fba501a03e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:00.462 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:00.459+0000 7fba5639b640 1 -- 172.21.15.82:0/2366327769 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fba501029f0 msgr2=0x7fba5019de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:12:00.462 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:00.459+0000 7fba5639b640 1 --2- 172.21.15.82:0/2366327769 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fba501029f0 0x7fba5019de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:00.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:00.459+0000 7fba5639b640 1 -- 172.21.15.82:0/2366327769 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba501a0bb0 con 0x7fba501017b0 2024-05-20T16:12:00.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:00.459+0000 7fba56b9c640 1 --2- 172.21.15.82:0/2366327769 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fba501076f0 0x7fba501a03e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T16:12:00.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:00.459+0000 7fba5639b640 1 --2- 172.21.15.82:0/2366327769 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fba501017b0 0x7fba50069860 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fba4c00d740 tx=0x7fba4c00dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:12:00.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:00.459+0000 7fba477fe640 1 -- 172.21.15.82:0/2366327769 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba4c017070 con 0x7fba501017b0 2024-05-20T16:12:00.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:00.460+0000 7fba5739d640 1 -- 172.21.15.82:0/2366327769 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba501a0ea0 con 0x7fba501017b0 2024-05-20T16:12:00.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:00.460+0000 7fba477fe640 1 -- 172.21.15.82:0/2366327769 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fba4c00de50 con 0x7fba501017b0 2024-05-20T16:12:00.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:00.460+0000 7fba477fe640 1 -- 172.21.15.82:0/2366327769 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba4c003e00 con 0x7fba501017b0 2024-05-20T16:12:00.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:00.460+0000 7fba5739d640 1 -- 172.21.15.82:0/2366327769 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba501a13b0 con 0x7fba501017b0 2024-05-20T16:12:00.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:00.461+0000 7fba5739d640 1 -- 172.21.15.82:0/2366327769 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba50106500 con 0x7fba501017b0 2024-05-20T16:12:00.464 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:00.462+0000 7fba477fe640 1 -- 172.21.15.82:0/2366327769 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fba4c003fa0 con 0x7fba501017b0 2024-05-20T16:12:00.464 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:00.462+0000 7fba477fe640 1 --2- 172.21.15.82:0/2366327769 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fba3c0a2b80 0x7fba3c0a5090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:12:00.464 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:00.463+0000 7fba55b9a640 1 --2- 172.21.15.82:0/2366327769 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fba3c0a2b80 0x7fba3c0a5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:12:00.465 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:00.463+0000 7fba477fe640 1 -- 172.21.15.82:0/2366327769 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8802+0+0 (secure 0 0 0) 0x7fba4c0cecf0 con 0x7fba501017b0 2024-05-20T16:12:00.465 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:00.463+0000 7fba55b9a640 1 --2- 172.21.15.82:0/2366327769 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fba3c0a2b80 0x7fba3c0a5090 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fba400096c0 tx=0x7fba40031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:12:00.467 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:00.467+0000 7fba477fe640 1 -- 172.21.15.82:0/2366327769 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fba4c0d3070 con 0x7fba501017b0 2024-05-20T16:12:00.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:12:00 smithi082 ceph-mon[32238]: from='client.? 172.21.15.82:0/1553773811' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-20T16:12:00.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:12:00 smithi082 ceph-mon[32238]: from='client.? 172.21.15.82:0/2502860420' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-20T16:12:00.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:12:00 smithi082 ceph-mon[32238]: from='client.? 172.21.15.82:0/3864685909' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-20T16:12:00.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:12:00 smithi082 ceph-mon[32238]: from='client.? 172.21.15.82:0/4064022853' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-20T16:12:00.635 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:12:00 smithi184 ceph-mon[36209]: from='client.? 172.21.15.82:0/1553773811' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-20T16:12:00.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:12:00 smithi184 ceph-mon[36209]: from='client.? 172.21.15.82:0/2502860420' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-20T16:12:00.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:12:00 smithi184 ceph-mon[36209]: from='client.? 172.21.15.82:0/3864685909' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-20T16:12:00.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:12:00 smithi184 ceph-mon[36209]: from='client.? 172.21.15.82:0/4064022853' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-20T16:12:00.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:00.643+0000 7fba5739d640 1 -- 172.21.15.82:0/2366327769 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fba50062750 con 0x7fba501017b0 2024-05-20T16:12:00.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:00.643+0000 7fba477fe640 1 -- 172.21.15.82:0/2366327769 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v91) ==== 103+0+11 (secure 0 0 0) 0x7fba4c09ba10 con 0x7fba501017b0 2024-05-20T16:12:00.643 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 32 2024-05-20T16:12:00.646 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:00.645+0000 7fba5739d640 1 -- 172.21.15.82:0/2366327769 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fba3c0a2b80 msgr2=0x7fba3c0a5090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:12:00.646 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:00.645+0000 7fba5739d640 1 --2- 172.21.15.82:0/2366327769 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fba3c0a2b80 0x7fba3c0a5090 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fba400096c0 tx=0x7fba40031400 comp rx=0 tx=0).stop 2024-05-20T16:12:00.646 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:00.645+0000 7fba5739d640 1 -- 172.21.15.82:0/2366327769 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fba501017b0 msgr2=0x7fba50069860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:12:00.646 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:00.645+0000 7fba5739d640 1 --2- 172.21.15.82:0/2366327769 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fba501017b0 0x7fba50069860 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fba4c00d740 tx=0x7fba4c00dc00 comp rx=0 tx=0).stop 2024-05-20T16:12:00.646 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:00.645+0000 7fba5739d640 1 -- 172.21.15.82:0/2366327769 shutdown_connections 2024-05-20T16:12:00.646 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:00.645+0000 7fba5739d640 1 --2- 172.21.15.82:0/2366327769 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fba3c0a2b80 0x7fba3c0a5090 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:00.646 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:00.645+0000 7fba5739d640 1 --2- 172.21.15.82:0/2366327769 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fba501076f0 0x7fba501a03e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:00.646 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:00.645+0000 7fba5739d640 1 --2- 172.21.15.82:0/2366327769 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fba501029f0 0x7fba5019de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:00.646 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:00.645+0000 7fba5739d640 1 --2- 172.21.15.82:0/2366327769 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fba501017b0 0x7fba50069860 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:00.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:00.645+0000 7fba5739d640 1 -- 172.21.15.82:0/2366327769 >> 172.21.15.82:0/2366327769 conn(0x7fba500fc9a0 msgr2=0x7fba501036d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:12:00.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:00.645+0000 7fba5739d640 1 -- 172.21.15.82:0/2366327769 shutdown_connections 2024-05-20T16:12:00.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:00.645+0000 7fba5739d640 1 -- 172.21.15.82:0/2366327769 wait complete. 2024-05-20T16:12:00.657 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-20T16:12:00.770 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:00.769+0000 7fb24cbf0640 1 -- 172.21.15.82:0/1240524050 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb248101f20 msgr2=0x7fb248102330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:12:00.770 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:00.769+0000 7fb24cbf0640 1 --2- 172.21.15.82:0/1240524050 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb248101f20 0x7fb248102330 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fb238009900 tx=0x7fb23802a410 comp rx=0 tx=0).stop 2024-05-20T16:12:00.770 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:00.770+0000 7fb24cbf0640 1 -- 172.21.15.82:0/1240524050 shutdown_connections 2024-05-20T16:12:00.770 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:00.770+0000 7fb24cbf0640 1 --2- 172.21.15.82:0/1240524050 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb248103420 0x7fb248109b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:00.770 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:00.770+0000 7fb24cbf0640 1 --2- 172.21.15.82:0/1240524050 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb248102900 0x7fb248102d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:00.770 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:00.770+0000 7fb24cbf0640 1 --2- 172.21.15.82:0/1240524050 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb248101f20 0x7fb248102330 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:00.770 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:00.770+0000 7fb24cbf0640 1 -- 172.21.15.82:0/1240524050 >> 172.21.15.82:0/1240524050 conn(0x7fb2480fc300 msgr2=0x7fb2480fe7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:12:00.770 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:00.770+0000 7fb24cbf0640 1 -- 172.21.15.82:0/1240524050 shutdown_connections 2024-05-20T16:12:00.770 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:00.770+0000 7fb24cbf0640 1 -- 172.21.15.82:0/1240524050 wait complete. 2024-05-20T16:12:00.772 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:00.770+0000 7fb24cbf0640 1 Processor -- start 2024-05-20T16:12:00.772 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:00.770+0000 7fb24cbf0640 1 -- start start 2024-05-20T16:12:00.772 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:00.771+0000 7fb24cbf0640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb248101f20 0x7fb2481011e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:12:00.773 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:00.771+0000 7fb24cbf0640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb248102900 0x7fb24819e020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:12:00.773 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:00.771+0000 7fb24cbf0640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb248103420 0x7fb2481a0590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:12:00.773 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:00.771+0000 7fb24cbf0640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fb24810d610 con 0x7fb248102900 2024-05-20T16:12:00.773 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:00.771+0000 7fb24cbf0640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fb24810d490 con 0x7fb248101f20 2024-05-20T16:12:00.773 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:00.771+0000 7fb24cbf0640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fb24810d790 con 0x7fb248103420 2024-05-20T16:12:00.773 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:00.771+0000 7fb23ffff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb248102900 0x7fb24819e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:12:00.773 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:00.771+0000 7fb23ffff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb248102900 0x7fb24819e020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43386/0 (socket says 172.21.15.82:43386) 2024-05-20T16:12:00.773 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:00.771+0000 7fb23ffff640 1 -- 172.21.15.82:0/4251431073 learned_addr learned my addr 172.21.15.82:0/4251431073 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:12:00.773 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:00.771+0000 7fb246d9d640 1 --2- 172.21.15.82:0/4251431073 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb248103420 0x7fb2481a0590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:12:00.773 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:00.771+0000 7fb24659c640 1 --2- 172.21.15.82:0/4251431073 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb248101f20 0x7fb2481011e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:12:00.773 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:00.771+0000 7fb23ffff640 1 -- 172.21.15.82:0/4251431073 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb248103420 msgr2=0x7fb2481a0590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:12:00.773 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:00.771+0000 7fb23ffff640 1 --2- 172.21.15.82:0/4251431073 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb248103420 0x7fb2481a0590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:00.774 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:00.771+0000 7fb23ffff640 1 -- 172.21.15.82:0/4251431073 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb248101f20 msgr2=0x7fb2481011e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:12:00.774 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:00.771+0000 7fb23ffff640 1 --2- 172.21.15.82:0/4251431073 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb248101f20 0x7fb2481011e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:00.774 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:00.771+0000 7fb23ffff640 1 -- 172.21.15.82:0/4251431073 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2481a0db0 con 0x7fb248102900 2024-05-20T16:12:00.774 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:00.771+0000 7fb23ffff640 1 --2- 172.21.15.82:0/4251431073 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb248102900 0x7fb24819e020 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fb23000d770 tx=0x7fb23000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:12:00.774 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:00.771+0000 7fb23f7fe640 1 -- 172.21.15.82:0/4251431073 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb230017070 con 0x7fb248102900 2024-05-20T16:12:00.774 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:00.771+0000 7fb23f7fe640 1 -- 172.21.15.82:0/4251431073 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb230005500 con 0x7fb248102900 2024-05-20T16:12:00.774 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:00.771+0000 7fb23f7fe640 1 -- 172.21.15.82:0/4251431073 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb230004e00 con 0x7fb248102900 2024-05-20T16:12:00.774 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:00.771+0000 7fb24cbf0640 1 -- 172.21.15.82:0/4251431073 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2481a10a0 con 0x7fb248102900 2024-05-20T16:12:00.774 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:00.771+0000 7fb24cbf0640 1 -- 172.21.15.82:0/4251431073 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2481a15f0 con 0x7fb248102900 2024-05-20T16:12:00.774 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:00.772+0000 7fb24cbf0640 1 -- 172.21.15.82:0/4251431073 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2481068c0 con 0x7fb248102900 2024-05-20T16:12:00.775 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:00.774+0000 7fb23f7fe640 1 -- 172.21.15.82:0/4251431073 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fb230004fa0 con 0x7fb248102900 2024-05-20T16:12:00.775 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:00.775+0000 7fb23f7fe640 1 --2- 172.21.15.82:0/4251431073 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fb2340a2c10 0x7fb2340a5120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:12:00.775 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:00.775+0000 7fb23f7fe640 1 -- 172.21.15.82:0/4251431073 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8802+0+0 (secure 0 0 0) 0x7fb2300cf0c0 con 0x7fb248102900 2024-05-20T16:12:00.775 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:00.775+0000 7fb24659c640 1 --2- 172.21.15.82:0/4251431073 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fb2340a2c10 0x7fb2340a5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:12:00.775 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:00.775+0000 7fb24659c640 1 --2- 172.21.15.82:0/4251431073 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fb2340a2c10 0x7fb2340a5120 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fb238000c00 tx=0x7fb238031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:12:00.778 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:00.778+0000 7fb23f7fe640 1 -- 172.21.15.82:0/4251431073 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fb23009be60 con 0x7fb248102900 2024-05-20T16:12:00.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:12:00 smithi160 ceph-mon[36347]: from='client.? 172.21.15.82:0/1553773811' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-20T16:12:00.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:12:00 smithi160 ceph-mon[36347]: from='client.? 172.21.15.82:0/2502860420' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-20T16:12:00.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:12:00 smithi160 ceph-mon[36347]: from='client.? 172.21.15.82:0/3864685909' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-20T16:12:00.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:12:00 smithi160 ceph-mon[36347]: from='client.? 172.21.15.82:0/4064022853' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-20T16:12:00.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:00.968+0000 7fb24cbf0640 1 -- 172.21.15.82:0/4251431073 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fb24802d2a0 con 0x7fb248102900 2024-05-20T16:12:00.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:00.968+0000 7fb23f7fe640 1 -- 172.21.15.82:0/4251431073 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v91) ==== 81+0+591 (secure 0 0 0) 0x7fb23009b760 con 0x7fb248102900 2024-05-20T16:12:00.968 INFO:teuthology.orchestra.run.smithi082.stdout:172.21.15.82:0/2005424509 2024-05-21T16:04:45.161949+0000 2024-05-20T16:12:00.968 INFO:teuthology.orchestra.run.smithi082.stdout:172.21.15.82:6801/1763049308 2024-05-21T16:04:45.161949+0000 2024-05-20T16:12:00.968 INFO:teuthology.orchestra.run.smithi082.stdout:172.21.15.82:6801/1959459348 2024-05-21T16:04:19.208291+0000 2024-05-20T16:12:00.968 INFO:teuthology.orchestra.run.smithi082.stdout:172.21.15.82:0/3131904556 2024-05-21T16:04:19.208291+0000 2024-05-20T16:12:00.969 INFO:teuthology.orchestra.run.smithi082.stdout:172.21.15.82:6800/1959459348 2024-05-21T16:04:19.208291+0000 2024-05-20T16:12:00.969 INFO:teuthology.orchestra.run.smithi082.stdout:172.21.15.82:0/2456966342 2024-05-21T16:04:45.161949+0000 2024-05-20T16:12:00.969 INFO:teuthology.orchestra.run.smithi082.stdout:172.21.15.82:0/4179978833 2024-05-21T16:04:19.208291+0000 2024-05-20T16:12:00.969 INFO:teuthology.orchestra.run.smithi082.stdout:172.21.15.82:0/1602862225 2024-05-21T16:04:45.161949+0000 2024-05-20T16:12:00.969 INFO:teuthology.orchestra.run.smithi082.stdout:172.21.15.82:0/106101180 2024-05-21T16:04:19.208291+0000 2024-05-20T16:12:00.969 INFO:teuthology.orchestra.run.smithi082.stdout:172.21.15.82:6800/1763049308 2024-05-21T16:04:45.161949+0000 2024-05-20T16:12:00.969 INFO:teuthology.orchestra.run.smithi082.stderr:listed 10 entries 2024-05-20T16:12:00.970 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:00.970+0000 7fb24cbf0640 1 -- 172.21.15.82:0/4251431073 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fb2340a2c10 msgr2=0x7fb2340a5120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:12:00.970 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:00.970+0000 7fb24cbf0640 1 --2- 172.21.15.82:0/4251431073 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fb2340a2c10 0x7fb2340a5120 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fb238000c00 tx=0x7fb238031400 comp rx=0 tx=0).stop 2024-05-20T16:12:00.970 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:00.970+0000 7fb24cbf0640 1 -- 172.21.15.82:0/4251431073 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb248102900 msgr2=0x7fb24819e020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:12:00.970 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:00.970+0000 7fb24cbf0640 1 --2- 172.21.15.82:0/4251431073 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb248102900 0x7fb24819e020 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fb23000d770 tx=0x7fb23000dc30 comp rx=0 tx=0).stop 2024-05-20T16:12:00.970 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:00.970+0000 7fb24cbf0640 1 -- 172.21.15.82:0/4251431073 shutdown_connections 2024-05-20T16:12:00.971 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:00.970+0000 7fb24cbf0640 1 --2- 172.21.15.82:0/4251431073 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fb2340a2c10 0x7fb2340a5120 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:00.971 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:00.970+0000 7fb24cbf0640 1 --2- 172.21.15.82:0/4251431073 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb248102900 0x7fb24819e020 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:00.971 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:00.970+0000 7fb24cbf0640 1 --2- 172.21.15.82:0/4251431073 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb248103420 0x7fb2481a0590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:00.971 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:00.970+0000 7fb24cbf0640 1 --2- 172.21.15.82:0/4251431073 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb248101f20 0x7fb2481011e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:00.971 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:00.970+0000 7fb24cbf0640 1 -- 172.21.15.82:0/4251431073 >> 172.21.15.82:0/4251431073 conn(0x7fb2480fc300 msgr2=0x7fb248103a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:12:00.971 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:00.970+0000 7fb24cbf0640 1 -- 172.21.15.82:0/4251431073 shutdown_connections 2024-05-20T16:12:00.971 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:00.970+0000 7fb24cbf0640 1 -- 172.21.15.82:0/4251431073 wait complete. 2024-05-20T16:12:00.982 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-05-20T16:12:00.982 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-20T16:12:00.999 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-20T16:12:01.151 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:01.149+0000 7f2a4fb38640 1 -- 172.21.15.82:0/31867797 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2a48103cf0 msgr2=0x7f2a48104100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:12:01.151 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:01.149+0000 7f2a4fb38640 1 --2- 172.21.15.82:0/31867797 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2a48103cf0 0x7f2a48104100 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f2a3c009900 tx=0x7f2a3c02a410 comp rx=0 tx=0).stop 2024-05-20T16:12:01.151 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:01.149+0000 7f2a4fb38640 1 -- 172.21.15.82:0/31867797 shutdown_connections 2024-05-20T16:12:01.152 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:01.149+0000 7f2a4fb38640 1 --2- 172.21.15.82:0/31867797 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2a48103cf0 0x7f2a48104100 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:01.152 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:01.149+0000 7f2a4fb38640 1 --2- 172.21.15.82:0/31867797 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2a481089f0 0x7f2a4810ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:01.152 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:01.149+0000 7f2a4fb38640 1 --2- 172.21.15.82:0/31867797 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2a48102ab0 0x7f2a481037b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:01.152 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:01.149+0000 7f2a4fb38640 1 -- 172.21.15.82:0/31867797 >> 172.21.15.82:0/31867797 conn(0x7f2a480fdcc0 msgr2=0x7f2a48100150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:12:01.152 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:01.149+0000 7f2a4fb38640 1 -- 172.21.15.82:0/31867797 shutdown_connections 2024-05-20T16:12:01.152 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:01.149+0000 7f2a4fb38640 1 -- 172.21.15.82:0/31867797 wait complete. 2024-05-20T16:12:01.152 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:01.150+0000 7f2a4fb38640 1 Processor -- start 2024-05-20T16:12:01.152 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:01.150+0000 7f2a4fb38640 1 -- start start 2024-05-20T16:12:01.152 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:01.150+0000 7f2a4fb38640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2a48102ab0 0x7f2a481992f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:12:01.152 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:01.150+0000 7f2a4fb38640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2a48103cf0 0x7f2a48199830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:12:01.152 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:01.150+0000 7f2a4fb38640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2a481089f0 0x7f2a481a07c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:12:01.152 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:01.150+0000 7f2a4fb38640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f2a4810e720 con 0x7f2a48103cf0 2024-05-20T16:12:01.153 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:01.150+0000 7f2a4fb38640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f2a4810e5a0 con 0x7f2a481089f0 2024-05-20T16:12:01.153 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:01.150+0000 7f2a4fb38640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f2a4810e8a0 con 0x7f2a48102ab0 2024-05-20T16:12:01.153 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:01.150+0000 7f2a4d0d3640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2a48103cf0 0x7f2a48199830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:12:01.153 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:01.150+0000 7f2a4d0d3640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2a48103cf0 0x7f2a48199830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43404/0 (socket says 172.21.15.82:43404) 2024-05-20T16:12:01.153 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:01.150+0000 7f2a4d0d3640 1 -- 172.21.15.82:0/3095289311 learned_addr learned my addr 172.21.15.82:0/3095289311 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:12:01.153 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:01.150+0000 7f2a4e0d5640 1 --2- 172.21.15.82:0/3095289311 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2a481089f0 0x7f2a481a07c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:12:01.153 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:01.150+0000 7f2a4d8d4640 1 --2- 172.21.15.82:0/3095289311 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2a48102ab0 0x7f2a481992f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:12:01.153 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:01.151+0000 7f2a4d0d3640 1 -- 172.21.15.82:0/3095289311 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2a48102ab0 msgr2=0x7f2a481992f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:12:01.153 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:01.151+0000 7f2a4d0d3640 1 --2- 172.21.15.82:0/3095289311 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2a48102ab0 0x7f2a481992f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:01.153 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:01.151+0000 7f2a4d0d3640 1 -- 172.21.15.82:0/3095289311 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2a481089f0 msgr2=0x7f2a481a07c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:12:01.153 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:01.151+0000 7f2a4d0d3640 1 --2- 172.21.15.82:0/3095289311 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2a481089f0 0x7f2a481a07c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:01.153 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:01.151+0000 7f2a4d0d3640 1 -- 172.21.15.82:0/3095289311 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a481a0e30 con 0x7f2a48103cf0 2024-05-20T16:12:01.153 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:01.151+0000 7f2a4d0d3640 1 --2- 172.21.15.82:0/3095289311 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2a48103cf0 0x7f2a48199830 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f2a3c006010 tx=0x7f2a3c004360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:12:01.153 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:01.151+0000 7f2a3affd640 1 -- 172.21.15.82:0/3095289311 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a3c03f030 con 0x7f2a48103cf0 2024-05-20T16:12:01.154 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:01.151+0000 7f2a4fb38640 1 -- 172.21.15.82:0/3095289311 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a481a10c0 con 0x7f2a48103cf0 2024-05-20T16:12:01.154 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:01.151+0000 7f2a3affd640 1 -- 172.21.15.82:0/3095289311 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a3c003ce0 con 0x7f2a48103cf0 2024-05-20T16:12:01.154 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:01.151+0000 7f2a3affd640 1 -- 172.21.15.82:0/3095289311 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a3c035070 con 0x7f2a48103cf0 2024-05-20T16:12:01.154 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:01.151+0000 7f2a4fb38640 1 -- 172.21.15.82:0/3095289311 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a481a1600 con 0x7f2a48103cf0 2024-05-20T16:12:01.154 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:01.152+0000 7f2a4fb38640 1 -- 172.21.15.82:0/3095289311 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a48107800 con 0x7f2a48103cf0 2024-05-20T16:12:01.155 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:01.154+0000 7f2a3affd640 1 -- 172.21.15.82:0/3095289311 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f2a3c042030 con 0x7f2a48103cf0 2024-05-20T16:12:01.155 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:01.155+0000 7f2a3affd640 1 --2- 172.21.15.82:0/3095289311 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f2a340a2b90 0x7f2a340a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:12:01.155 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:01.155+0000 7f2a4d8d4640 1 --2- 172.21.15.82:0/3095289311 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f2a340a2b90 0x7f2a340a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:12:01.155 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:01.155+0000 7f2a3affd640 1 -- 172.21.15.82:0/3095289311 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8802+0+0 (secure 0 0 0) 0x7f2a3c0f0020 con 0x7f2a48103cf0 2024-05-20T16:12:01.155 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:01.155+0000 7f2a4d8d4640 1 --2- 172.21.15.82:0/3095289311 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f2a340a2b90 0x7f2a340a50a0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f2a440058c0 tx=0x7f2a4400a4c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:12:01.158 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:01.158+0000 7f2a3affd640 1 -- 172.21.15.82:0/3095289311 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f2a3c0bcd40 con 0x7f2a48103cf0 2024-05-20T16:12:01.335 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:01.335+0000 7f2a4fb38640 1 -- 172.21.15.82:0/3095289311 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2a4802d470 con 0x7f2a48103cf0 2024-05-20T16:12:01.336 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:01.337+0000 7f2a3affd640 1 -- 172.21.15.82:0/3095289311 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+24153 (secure 0 0 0) 0x7f2a3c0b8030 con 0x7f2a48103cf0 2024-05-20T16:12:01.337 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-05-20T16:12:01.337 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":91,"fsid":"59d1600a-16c2-11ef-bc9a-c7b262605968","created":"2024-05-20T16:03:46.611556+0000","modified":"2024-05-20T16:11:53.416085+0000","last_up_change":"2024-05-20T16:10:59.303479+0000","last_in_change":"2024-05-20T16:10:39.063561+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-20T16:07:09.493727+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-20T16:11:45.058035+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-20T16:11:45.943862+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-20T16:11:49.637579+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"bae842a9-03ec-490e-b56d-f7957235cdc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":1484330167},{"type":"v1","addr":"172.21.15.82:6803","nonce":1484330167}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":1484330167},{"type":"v1","addr":"172.21.15.82:6805","nonce":1484330167}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1484330167},{"type":"v1","addr":"172.21.15.82:6809","nonce":1484330167}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":1484330167},{"type":"v1","addr":"172.21.15.82:6807","nonce":1484330167}]},"public_addr":"172.21.15.82:6803/1484330167","cluster_addr":"172.21.15.82:6805/1484330167","heartbeat_back_addr":"172.21.15.82:6809/1484330167","heartbeat_front_addr":"172.21.15.82:6807/1484330167","state":["exists","up"]},{"osd":1,"uuid":"59f6a2ba-b5de-4161-81c5-92a417abacdf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":1457003568},{"type":"v1","addr":"172.21.15.160:6801","nonce":1457003568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":1457003568},{"type":"v1","addr":"172.21.15.160:6803","nonce":1457003568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6806","nonce":1457003568},{"type":"v1","addr":"172.21.15.160:6807","nonce":1457003568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":1457003568},{"type":"v1","addr":"172.21.15.160:6805","nonce":1457003568}]},"public_addr":"172.21.15.160:6801/1457003568","cluster_addr":"172.21.15.160:6803/1457003568","heartbeat_back_addr":"172.21.15.160:6807/1457003568","heartbeat_front_addr":"172.21.15.160:6805/1457003568","state":["exists","up"]},{"osd":2,"uuid":"3bbb5587-b449-41f3-bd26-67ec3d6afdfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":4051293934},{"type":"v1","addr":"172.21.15.184:6801","nonce":4051293934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":4051293934},{"type":"v1","addr":"172.21.15.184:6803","nonce":4051293934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":4051293934},{"type":"v1","addr":"172.21.15.184:6807","nonce":4051293934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":4051293934},{"type":"v1","addr":"172.21.15.184:6805","nonce":4051293934}]},"public_addr":"172.21.15.184:6801/4051293934","cluster_addr":"172.21.15.184:6803/4051293934","heartbeat_back_addr":"172.21.15.184:6807/4051293934","heartbeat_front_addr":"172.21.15.184:6805/4051293934","state":["exists","up"]},{"osd":3,"uuid":"ca651f90-09b8-4081-a6f9-40b31268a837","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":3947485599},{"type":"v1","addr":"172.21.15.82:6811","nonce":3947485599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":3947485599},{"type":"v1","addr":"172.21.15.82:6813","nonce":3947485599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":3947485599},{"type":"v1","addr":"172.21.15.82:6817","nonce":3947485599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":3947485599},{"type":"v1","addr":"172.21.15.82:6815","nonce":3947485599}]},"public_addr":"172.21.15.82:6811/3947485599","cluster_addr":"172.21.15.82:6813/3947485599","heartbeat_back_addr":"172.21.15.82:6817/3947485599","heartbeat_front_addr":"172.21.15.82:6815/3947485599","state":["exists","up"]},{"osd":4,"uuid":"cb3dfd56-f677-40cb-9c96-8319ea5ac788","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":4160830014},{"type":"v1","addr":"172.21.15.160:6809","nonce":4160830014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":4160830014},{"type":"v1","addr":"172.21.15.160:6811","nonce":4160830014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":4160830014},{"type":"v1","addr":"172.21.15.160:6815","nonce":4160830014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6812","nonce":4160830014},{"type":"v1","addr":"172.21.15.160:6813","nonce":4160830014}]},"public_addr":"172.21.15.160:6809/4160830014","cluster_addr":"172.21.15.160:6811/4160830014","heartbeat_back_addr":"172.21.15.160:6815/4160830014","heartbeat_front_addr":"172.21.15.160:6813/4160830014","state":["exists","up"]},{"osd":5,"uuid":"c70cfcd1-a4ff-461f-8c73-479b061200fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":1046282672},{"type":"v1","addr":"172.21.15.184:6809","nonce":1046282672}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":1046282672},{"type":"v1","addr":"172.21.15.184:6811","nonce":1046282672}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":1046282672},{"type":"v1","addr":"172.21.15.184:6815","nonce":1046282672}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":1046282672},{"type":"v1","addr":"172.21.15.184:6813","nonce":1046282672}]},"public_addr":"172.21.15.184:6809/1046282672","cluster_addr":"172.21.15.184:6811/1046282672","heartbeat_back_addr":"172.21.15.184:6815/1046282672","heartbeat_front_addr":"172.21.15.184:6813/1046282672","state":["exists","up"]},{"osd":6,"uuid":"d47ce001-9dd1-4830-bcc9-888601187c6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":2789736499},{"type":"v1","addr":"172.21.15.82:6819","nonce":2789736499}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":2789736499},{"type":"v1","addr":"172.21.15.82:6821","nonce":2789736499}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":2789736499},{"type":"v1","addr":"172.21.15.82:6825","nonce":2789736499}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":2789736499},{"type":"v1","addr":"172.21.15.82:6823","nonce":2789736499}]},"public_addr":"172.21.15.82:6819/2789736499","cluster_addr":"172.21.15.82:6821/2789736499","heartbeat_back_addr":"172.21.15.82:6825/2789736499","heartbeat_front_addr":"172.21.15.82:6823/2789736499","state":["exists","up"]},{"osd":7,"uuid":"25c54f43-18cb-41e4-83b4-2732b5a93889","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6816","nonce":2340683378},{"type":"v1","addr":"172.21.15.160:6817","nonce":2340683378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":2340683378},{"type":"v1","addr":"172.21.15.160:6819","nonce":2340683378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":2340683378},{"type":"v1","addr":"172.21.15.160:6823","nonce":2340683378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6820","nonce":2340683378},{"type":"v1","addr":"172.21.15.160:6821","nonce":2340683378}]},"public_addr":"172.21.15.160:6817/2340683378","cluster_addr":"172.21.15.160:6819/2340683378","heartbeat_back_addr":"172.21.15.160:6823/2340683378","heartbeat_front_addr":"172.21.15.160:6821/2340683378","state":["exists","up"]},{"osd":8,"uuid":"1f8b478d-287f-45f5-8526-d1a58c5bc07c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":3831576372},{"type":"v1","addr":"172.21.15.184:6817","nonce":3831576372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":3831576372},{"type":"v1","addr":"172.21.15.184:6819","nonce":3831576372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":3831576372},{"type":"v1","addr":"172.21.15.184:6823","nonce":3831576372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":3831576372},{"type":"v1","addr":"172.21.15.184:6821","nonce":3831576372}]},"public_addr":"172.21.15.184:6817/3831576372","cluster_addr":"172.21.15.184:6819/3831576372","heartbeat_back_addr":"172.21.15.184:6823/3831576372","heartbeat_front_addr":"172.21.15.184:6821/3831576372","state":["exists","up"]},{"osd":9,"uuid":"0d69e319-954f-4b18-95ea-31794f18dc3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":716330782},{"type":"v1","addr":"172.21.15.82:6827","nonce":716330782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":716330782},{"type":"v1","addr":"172.21.15.82:6829","nonce":716330782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":716330782},{"type":"v1","addr":"172.21.15.82:6833","nonce":716330782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":716330782},{"type":"v1","addr":"172.21.15.82:6831","nonce":716330782}]},"public_addr":"172.21.15.82:6827/716330782","cluster_addr":"172.21.15.82:6829/716330782","heartbeat_back_addr":"172.21.15.82:6833/716330782","heartbeat_front_addr":"172.21.15.82:6831/716330782","state":["exists","up"]},{"osd":10,"uuid":"39dc1b78-61cb-42ca-b5b6-1e44ac2b6500","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":2327428014},{"type":"v1","addr":"172.21.15.160:6825","nonce":2327428014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":2327428014},{"type":"v1","addr":"172.21.15.160:6827","nonce":2327428014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":2327428014},{"type":"v1","addr":"172.21.15.160:6831","nonce":2327428014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":2327428014},{"type":"v1","addr":"172.21.15.160:6829","nonce":2327428014}]},"public_addr":"172.21.15.160:6825/2327428014","cluster_addr":"172.21.15.160:6827/2327428014","heartbeat_back_addr":"172.21.15.160:6831/2327428014","heartbeat_front_addr":"172.21.15.160:6829/2327428014","state":["exists","up"]},{"osd":11,"uuid":"4306b3c1-707a-4010-987c-209ea0e948e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":3594364650},{"type":"v1","addr":"172.21.15.184:6825","nonce":3594364650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":3594364650},{"type":"v1","addr":"172.21.15.184:6827","nonce":3594364650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3594364650},{"type":"v1","addr":"172.21.15.184:6831","nonce":3594364650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3594364650},{"type":"v1","addr":"172.21.15.184:6829","nonce":3594364650}]},"public_addr":"172.21.15.184:6825/3594364650","cluster_addr":"172.21.15.184:6827/3594364650","heartbeat_back_addr":"172.21.15.184:6831/3594364650","heartbeat_front_addr":"172.21.15.184:6829/3594364650","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:06:17.855998+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:06:41.473270+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:07:05.304108+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:07:29.887472+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:07:55.387286+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:08:20.579669+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:08:46.333051+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:09:12.289668+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:09:38.180696+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:10:04.471459+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:10:31.041912+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:10:57.477266+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.82:0/2005424509":"2024-05-21T16:04:45.161949+0000","172.21.15.82:6801/1763049308":"2024-05-21T16:04:45.161949+0000","172.21.15.82:6801/1959459348":"2024-05-21T16:04:19.208291+0000","172.21.15.82:0/3131904556":"2024-05-21T16:04:19.208291+0000","172.21.15.82:6800/1959459348":"2024-05-21T16:04:19.208291+0000","172.21.15.82:0/2456966342":"2024-05-21T16:04:45.161949+0000","172.21.15.82:0/4179978833":"2024-05-21T16:04:19.208291+0000","172.21.15.82:0/1602862225":"2024-05-21T16:04:45.161949+0000","172.21.15.82:0/106101180":"2024-05-21T16:04:19.208291+0000","172.21.15.82:6800/1763049308":"2024-05-21T16:04:45.161949+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-20T16:12:01.339 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:01.339+0000 7f2a4fb38640 1 -- 172.21.15.82:0/3095289311 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f2a340a2b90 msgr2=0x7f2a340a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:12:01.339 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:01.339+0000 7f2a4fb38640 1 --2- 172.21.15.82:0/3095289311 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f2a340a2b90 0x7f2a340a50a0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f2a440058c0 tx=0x7f2a4400a4c0 comp rx=0 tx=0).stop 2024-05-20T16:12:01.339 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:01.339+0000 7f2a4fb38640 1 -- 172.21.15.82:0/3095289311 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2a48103cf0 msgr2=0x7f2a48199830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:12:01.339 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:01.339+0000 7f2a4fb38640 1 --2- 172.21.15.82:0/3095289311 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2a48103cf0 0x7f2a48199830 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f2a3c006010 tx=0x7f2a3c004360 comp rx=0 tx=0).stop 2024-05-20T16:12:01.339 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:01.339+0000 7f2a4fb38640 1 -- 172.21.15.82:0/3095289311 shutdown_connections 2024-05-20T16:12:01.339 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:01.339+0000 7f2a4fb38640 1 --2- 172.21.15.82:0/3095289311 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f2a340a2b90 0x7f2a340a50a0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:01.339 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:01.339+0000 7f2a4fb38640 1 --2- 172.21.15.82:0/3095289311 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2a48103cf0 0x7f2a48199830 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:01.339 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:01.339+0000 7f2a4fb38640 1 --2- 172.21.15.82:0/3095289311 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2a481089f0 0x7f2a481a07c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:01.339 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:01.339+0000 7f2a4fb38640 1 --2- 172.21.15.82:0/3095289311 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2a48102ab0 0x7f2a481992f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:01.339 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:01.339+0000 7f2a4fb38640 1 -- 172.21.15.82:0/3095289311 >> 172.21.15.82:0/3095289311 conn(0x7f2a480fdcc0 msgr2=0x7f2a481049d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:12:01.339 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:01.339+0000 7f2a4fb38640 1 -- 172.21.15.82:0/3095289311 shutdown_connections 2024-05-20T16:12:01.340 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:01.339+0000 7f2a4fb38640 1 -- 172.21.15.82:0/3095289311 wait complete. 2024-05-20T16:12:01.350 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-20T16:07:09.493727+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-20T16:11:45.058035+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-20T16:11:45.943862+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-20T16:11:49.637579+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '91', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-20T16:12:01.351 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-20T16:12:01.468 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:01.466+0000 7ff75e0b6640 1 -- 172.21.15.82:0/2429386422 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff758102bd0 msgr2=0x7ff758102fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:12:01.468 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:01.466+0000 7ff75e0b6640 1 --2- 172.21.15.82:0/2429386422 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff758102bd0 0x7ff758102fe0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7ff740009900 tx=0x7ff74002a410 comp rx=0 tx=0).stop 2024-05-20T16:12:01.468 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:01.466+0000 7ff75e0b6640 1 -- 172.21.15.82:0/2429386422 shutdown_connections 2024-05-20T16:12:01.468 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:01.466+0000 7ff75e0b6640 1 --2- 172.21.15.82:0/2429386422 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff758102bd0 0x7ff758102fe0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:01.469 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:01.466+0000 7ff75e0b6640 1 --2- 172.21.15.82:0/2429386422 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff7581078d0 0x7ff758109cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:01.469 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:01.466+0000 7ff75e0b6640 1 --2- 172.21.15.82:0/2429386422 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff758101990 0x7ff758102690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:01.469 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:01.466+0000 7ff75e0b6640 1 -- 172.21.15.82:0/2429386422 >> 172.21.15.82:0/2429386422 conn(0x7ff7580fcb80 msgr2=0x7ff7580ff030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:12:01.469 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:01.466+0000 7ff75e0b6640 1 -- 172.21.15.82:0/2429386422 shutdown_connections 2024-05-20T16:12:01.469 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:01.466+0000 7ff75e0b6640 1 -- 172.21.15.82:0/2429386422 wait complete. 2024-05-20T16:12:01.469 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:01.466+0000 7ff75e0b6640 1 Processor -- start 2024-05-20T16:12:01.469 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:01.466+0000 7ff75e0b6640 1 -- start start 2024-05-20T16:12:01.469 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:01.467+0000 7ff75e0b6640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff758101990 0x7ff7580698b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:12:01.469 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:01.467+0000 7ff75e0b6640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff758102bd0 0x7ff75819e380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:12:01.469 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:01.467+0000 7ff75e0b6640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff7581078d0 0x7ff7581a08d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:12:01.469 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:01.467+0000 7ff75e0b6640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7ff75810d600 con 0x7ff758102bd0 2024-05-20T16:12:01.469 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:01.467+0000 7ff75e0b6640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7ff75810d480 con 0x7ff758101990 2024-05-20T16:12:01.469 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:01.467+0000 7ff75e0b6640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ff75810d780 con 0x7ff7581078d0 2024-05-20T16:12:01.469 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:01.467+0000 7ff74effd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff758102bd0 0x7ff75819e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:12:01.470 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:01.467+0000 7ff74effd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff758102bd0 0x7ff75819e380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43418/0 (socket says 172.21.15.82:43418) 2024-05-20T16:12:01.470 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:01.467+0000 7ff74effd640 1 -- 172.21.15.82:0/422695650 learned_addr learned my addr 172.21.15.82:0/422695650 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:12:01.470 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:01.467+0000 7ff757fff640 1 --2- 172.21.15.82:0/422695650 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff7581078d0 0x7ff7581a08d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:12:01.470 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:01.467+0000 7ff7577fe640 1 --2- 172.21.15.82:0/422695650 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff758101990 0x7ff7580698b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:12:01.470 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:01.467+0000 7ff74effd640 1 -- 172.21.15.82:0/422695650 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff7581078d0 msgr2=0x7ff7581a08d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:12:01.470 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:01.467+0000 7ff74effd640 1 --2- 172.21.15.82:0/422695650 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff7581078d0 0x7ff7581a08d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:01.470 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:01.467+0000 7ff74effd640 1 -- 172.21.15.82:0/422695650 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff758101990 msgr2=0x7ff7580698b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:12:01.470 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:01.467+0000 7ff74effd640 1 --2- 172.21.15.82:0/422695650 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff758101990 0x7ff7580698b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:01.470 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:01.467+0000 7ff74effd640 1 -- 172.21.15.82:0/422695650 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7581a10f0 con 0x7ff758102bd0 2024-05-20T16:12:01.470 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:01.467+0000 7ff7577fe640 1 --2- 172.21.15.82:0/422695650 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff758101990 0x7ff7580698b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T16:12:01.470 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:01.467+0000 7ff74effd640 1 --2- 172.21.15.82:0/422695650 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff758102bd0 0x7ff75819e380 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7ff740006010 tx=0x7ff740004360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:12:01.470 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:01.468+0000 7ff7557fa640 1 -- 172.21.15.82:0/422695650 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff74003f030 con 0x7ff758102bd0 2024-05-20T16:12:01.471 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:01.468+0000 7ff7557fa640 1 -- 172.21.15.82:0/422695650 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff740003ce0 con 0x7ff758102bd0 2024-05-20T16:12:01.471 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:01.468+0000 7ff75e0b6640 1 -- 172.21.15.82:0/422695650 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7581a1380 con 0x7ff758102bd0 2024-05-20T16:12:01.471 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:01.468+0000 7ff7557fa640 1 -- 172.21.15.82:0/422695650 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff740035070 con 0x7ff758102bd0 2024-05-20T16:12:01.471 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:01.468+0000 7ff75e0b6640 1 -- 172.21.15.82:0/422695650 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7581a1870 con 0x7ff758102bd0 2024-05-20T16:12:01.471 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:01.469+0000 7ff75e0b6640 1 -- 172.21.15.82:0/422695650 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7581066e0 con 0x7ff758102bd0 2024-05-20T16:12:01.472 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:01.470+0000 7ff7557fa640 1 -- 172.21.15.82:0/422695650 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7ff740042030 con 0x7ff758102bd0 2024-05-20T16:12:01.472 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:01.471+0000 7ff7557fa640 1 --2- 172.21.15.82:0/422695650 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7ff7440a2b90 0x7ff7440a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:12:01.472 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:01.471+0000 7ff7577fe640 1 --2- 172.21.15.82:0/422695650 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7ff7440a2b90 0x7ff7440a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:12:01.472 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:01.471+0000 7ff7557fa640 1 -- 172.21.15.82:0/422695650 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8802+0+0 (secure 0 0 0) 0x7ff7400efd30 con 0x7ff758102bd0 2024-05-20T16:12:01.472 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:01.471+0000 7ff7577fe640 1 --2- 172.21.15.82:0/422695650 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7ff7440a2b90 0x7ff7440a50a0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7ff7480058c0 tx=0x7ff748005d50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:12:01.475 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:01.476+0000 7ff7557fa640 1 -- 172.21.15.82:0/422695650 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7ff7400bca50 con 0x7ff758102bd0 2024-05-20T16:12:01.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:12:01 smithi082 ceph-mon[32238]: from='client.? 172.21.15.82:0/2366327769' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-20T16:12:01.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:12:01 smithi082 ceph-mon[32238]: from='client.? 172.21.15.82:0/4251431073' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-20T16:12:01.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:12:01 smithi082 ceph-mon[32238]: pgmap v299: 129 pgs: 129 active+clean; 579 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-20T16:12:01.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:12:01 smithi082 ceph-mon[32238]: from='client.? 172.21.15.82:0/3095289311' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-20T16:12:01.635 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:12:01 smithi184 ceph-mon[36209]: from='client.? 172.21.15.82:0/2366327769' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-20T16:12:01.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:12:01 smithi184 ceph-mon[36209]: from='client.? 172.21.15.82:0/4251431073' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-20T16:12:01.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:12:01 smithi184 ceph-mon[36209]: pgmap v299: 129 pgs: 129 active+clean; 579 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-20T16:12:01.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:12:01 smithi184 ceph-mon[36209]: from='client.? 172.21.15.82:0/3095289311' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-20T16:12:01.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:01.652+0000 7ff75e0b6640 1 -- 172.21.15.82:0/422695650 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7ff75806b250 con 0x7ff758102bd0 2024-05-20T16:12:01.652 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:01.652+0000 7ff7557fa640 1 -- 172.21.15.82:0/422695650 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v91) ==== 93+0+10 (secure 0 0 0) 0x7ff75806b250 con 0x7ff758102bd0 2024-05-20T16:12:01.652 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 1 2024-05-20T16:12:01.654 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:01.654+0000 7ff75e0b6640 1 -- 172.21.15.82:0/422695650 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7ff7440a2b90 msgr2=0x7ff7440a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:12:01.654 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:01.654+0000 7ff75e0b6640 1 --2- 172.21.15.82:0/422695650 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7ff7440a2b90 0x7ff7440a50a0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7ff7480058c0 tx=0x7ff748005d50 comp rx=0 tx=0).stop 2024-05-20T16:12:01.654 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:01.654+0000 7ff75e0b6640 1 -- 172.21.15.82:0/422695650 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff758102bd0 msgr2=0x7ff75819e380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:12:01.655 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:01.654+0000 7ff75e0b6640 1 --2- 172.21.15.82:0/422695650 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff758102bd0 0x7ff75819e380 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7ff740006010 tx=0x7ff740004360 comp rx=0 tx=0).stop 2024-05-20T16:12:01.655 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:01.654+0000 7ff75e0b6640 1 -- 172.21.15.82:0/422695650 shutdown_connections 2024-05-20T16:12:01.655 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:01.654+0000 7ff75e0b6640 1 --2- 172.21.15.82:0/422695650 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7ff7440a2b90 0x7ff7440a50a0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:01.655 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:01.654+0000 7ff75e0b6640 1 --2- 172.21.15.82:0/422695650 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff758102bd0 0x7ff75819e380 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:01.655 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:01.654+0000 7ff75e0b6640 1 --2- 172.21.15.82:0/422695650 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff7581078d0 0x7ff7581a08d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:01.655 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:01.654+0000 7ff75e0b6640 1 --2- 172.21.15.82:0/422695650 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff758101990 0x7ff7580698b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:01.655 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:01.654+0000 7ff75e0b6640 1 -- 172.21.15.82:0/422695650 >> 172.21.15.82:0/422695650 conn(0x7ff7580fcb80 msgr2=0x7ff7581038b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:12:01.655 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:01.654+0000 7ff75e0b6640 1 -- 172.21.15.82:0/422695650 shutdown_connections 2024-05-20T16:12:01.655 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:01.654+0000 7ff75e0b6640 1 -- 172.21.15.82:0/422695650 wait complete. 2024-05-20T16:12:01.666 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-20T16:12:01.781 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:01.779+0000 7fe02b1aa640 1 -- 172.21.15.82:0/1028278191 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe024102bd0 msgr2=0x7fe024102fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:12:01.781 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:01.779+0000 7fe02b1aa640 1 --2- 172.21.15.82:0/1028278191 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe024102bd0 0x7fe024102fe0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fe014009900 tx=0x7fe01402a410 comp rx=0 tx=0).stop 2024-05-20T16:12:01.781 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:01.779+0000 7fe02b1aa640 1 -- 172.21.15.82:0/1028278191 shutdown_connections 2024-05-20T16:12:01.781 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:01.779+0000 7fe02b1aa640 1 --2- 172.21.15.82:0/1028278191 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe024102bd0 0x7fe024102fe0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:01.782 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:01.779+0000 7fe02b1aa640 1 --2- 172.21.15.82:0/1028278191 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe0241078d0 0x7fe024109cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:01.782 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:01.779+0000 7fe02b1aa640 1 --2- 172.21.15.82:0/1028278191 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe024101990 0x7fe024102690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:01.782 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:01.779+0000 7fe02b1aa640 1 -- 172.21.15.82:0/1028278191 >> 172.21.15.82:0/1028278191 conn(0x7fe0240fcb80 msgr2=0x7fe0240ff030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:12:01.782 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:01.779+0000 7fe02b1aa640 1 -- 172.21.15.82:0/1028278191 shutdown_connections 2024-05-20T16:12:01.782 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:01.779+0000 7fe02b1aa640 1 -- 172.21.15.82:0/1028278191 wait complete. 2024-05-20T16:12:01.782 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:01.780+0000 7fe02b1aa640 1 Processor -- start 2024-05-20T16:12:01.782 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:01.780+0000 7fe02b1aa640 1 -- start start 2024-05-20T16:12:01.782 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:01.780+0000 7fe02b1aa640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe024101990 0x7fe0241995d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:12:01.782 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:01.780+0000 7fe02b1aa640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe024102bd0 0x7fe024199b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:12:01.783 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:01.780+0000 7fe02b1aa640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe0241078d0 0x7fe0241a0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:12:01.783 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:01.780+0000 7fe02b1aa640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fe02410d610 con 0x7fe024101990 2024-05-20T16:12:01.783 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:01.780+0000 7fe02b1aa640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fe02410d490 con 0x7fe0241078d0 2024-05-20T16:12:01.783 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:01.780+0000 7fe02b1aa640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fe02410d790 con 0x7fe024102bd0 2024-05-20T16:12:01.783 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:01.780+0000 7fe028f46640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe024101990 0x7fe0241995d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:12:01.783 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:01.780+0000 7fe028f46640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe024101990 0x7fe0241995d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43422/0 (socket says 172.21.15.82:43422) 2024-05-20T16:12:01.783 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:01.780+0000 7fe028f46640 1 -- 172.21.15.82:0/1132676645 learned_addr learned my addr 172.21.15.82:0/1132676645 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:12:01.783 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:01.781+0000 7fe028f46640 1 -- 172.21.15.82:0/1132676645 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe024102bd0 msgr2=0x7fe024199b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:12:01.783 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:01.781+0000 7fe028f46640 1 --2- 172.21.15.82:0/1132676645 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe024102bd0 0x7fe024199b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:01.783 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:01.781+0000 7fe028f46640 1 -- 172.21.15.82:0/1132676645 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe0241078d0 msgr2=0x7fe0241a0aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:12:01.783 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:01.781+0000 7fe028f46640 1 --2- 172.21.15.82:0/1132676645 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe0241078d0 0x7fe0241a0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:01.783 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:01.781+0000 7fe028f46640 1 -- 172.21.15.82:0/1132676645 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0241a1110 con 0x7fe024101990 2024-05-20T16:12:01.784 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:01.781+0000 7fe028f46640 1 --2- 172.21.15.82:0/1132676645 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe024101990 0x7fe0241995d0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fe02000e9f0 tx=0x7fe02000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:12:01.784 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:01.781+0000 7fe019ffb640 1 -- 172.21.15.82:0/1132676645 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe02000ccc0 con 0x7fe024101990 2024-05-20T16:12:01.784 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:01.781+0000 7fe019ffb640 1 -- 172.21.15.82:0/1132676645 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe0200035f0 con 0x7fe024101990 2024-05-20T16:12:01.784 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:01.781+0000 7fe02b1aa640 1 -- 172.21.15.82:0/1132676645 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0241a1400 con 0x7fe024101990 2024-05-20T16:12:01.784 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:01.781+0000 7fe019ffb640 1 -- 172.21.15.82:0/1132676645 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe02000ce60 con 0x7fe024101990 2024-05-20T16:12:01.784 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:01.781+0000 7fe02b1aa640 1 -- 172.21.15.82:0/1132676645 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0241a1950 con 0x7fe024101990 2024-05-20T16:12:01.785 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:01.782+0000 7fe02b1aa640 1 -- 172.21.15.82:0/1132676645 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe02406b250 con 0x7fe024101990 2024-05-20T16:12:01.785 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:01.783+0000 7fe019ffb640 1 -- 172.21.15.82:0/1132676645 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fe0200037a0 con 0x7fe024101990 2024-05-20T16:12:01.785 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:01.784+0000 7fe019ffb640 1 --2- 172.21.15.82:0/1132676645 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fe00c0a2b90 0x7fe00c0a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:12:01.785 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:01.784+0000 7fe01bfff640 1 --2- 172.21.15.82:0/1132676645 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fe00c0a2b90 0x7fe00c0a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:12:01.785 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:01.784+0000 7fe019ffb640 1 -- 172.21.15.82:0/1132676645 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8802+0+0 (secure 0 0 0) 0x7fe0200cf700 con 0x7fe024101990 2024-05-20T16:12:01.785 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:01.784+0000 7fe01bfff640 1 --2- 172.21.15.82:0/1132676645 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fe00c0a2b90 0x7fe00c0a50a0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fe0140096c0 tx=0x7fe014031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:12:01.788 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:01.789+0000 7fe019ffb640 1 -- 172.21.15.82:0/1132676645 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fe02009c420 con 0x7fe024101990 2024-05-20T16:12:01.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:12:01 smithi160 ceph-mon[36347]: from='client.? 172.21.15.82:0/2366327769' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-20T16:12:01.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:12:01 smithi160 ceph-mon[36347]: from='client.? 172.21.15.82:0/4251431073' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-20T16:12:01.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:12:01 smithi160 ceph-mon[36347]: pgmap v299: 129 pgs: 129 active+clean; 579 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-20T16:12:01.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:12:01 smithi160 ceph-mon[36347]: from='client.? 172.21.15.82:0/3095289311' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-20T16:12:01.966 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:01.966+0000 7fe02b1aa640 1 -- 172.21.15.82:0/1132676645 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fe0241a1c10 con 0x7fe024101990 2024-05-20T16:12:01.966 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:01.966+0000 7fe019ffb640 1 -- 172.21.15.82:0/1132676645 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v91) ==== 104+0+11 (secure 0 0 0) 0x7fe02009bd20 con 0x7fe024101990 2024-05-20T16:12:01.967 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 32 2024-05-20T16:12:01.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:01.968+0000 7fe02b1aa640 1 -- 172.21.15.82:0/1132676645 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fe00c0a2b90 msgr2=0x7fe00c0a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:12:01.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:01.968+0000 7fe02b1aa640 1 --2- 172.21.15.82:0/1132676645 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fe00c0a2b90 0x7fe00c0a50a0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fe0140096c0 tx=0x7fe014031400 comp rx=0 tx=0).stop 2024-05-20T16:12:01.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:01.968+0000 7fe02b1aa640 1 -- 172.21.15.82:0/1132676645 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe024101990 msgr2=0x7fe0241995d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:12:01.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:01.968+0000 7fe02b1aa640 1 --2- 172.21.15.82:0/1132676645 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe024101990 0x7fe0241995d0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fe02000e9f0 tx=0x7fe02000eeb0 comp rx=0 tx=0).stop 2024-05-20T16:12:01.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:01.968+0000 7fe02b1aa640 1 -- 172.21.15.82:0/1132676645 shutdown_connections 2024-05-20T16:12:01.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:01.968+0000 7fe02b1aa640 1 --2- 172.21.15.82:0/1132676645 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fe00c0a2b90 0x7fe00c0a50a0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:01.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:01.968+0000 7fe02b1aa640 1 --2- 172.21.15.82:0/1132676645 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe0241078d0 0x7fe0241a0aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:01.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:01.968+0000 7fe02b1aa640 1 --2- 172.21.15.82:0/1132676645 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe024102bd0 0x7fe024199b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:01.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:01.968+0000 7fe02b1aa640 1 --2- 172.21.15.82:0/1132676645 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe024101990 0x7fe0241995d0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:01.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:01.968+0000 7fe02b1aa640 1 -- 172.21.15.82:0/1132676645 >> 172.21.15.82:0/1132676645 conn(0x7fe0240fcb80 msgr2=0x7fe024068e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:12:01.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:01.969+0000 7fe02b1aa640 1 -- 172.21.15.82:0/1132676645 shutdown_connections 2024-05-20T16:12:01.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:01.969+0000 7fe02b1aa640 1 -- 172.21.15.82:0/1132676645 wait complete. 2024-05-20T16:12:01.980 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-20T16:12:02.094 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:02.094+0000 7f436dd38640 1 -- 172.21.15.82:0/880607477 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f43681078d0 msgr2=0x7f4368109cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:12:02.094 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:02.094+0000 7f436dd38640 1 --2- 172.21.15.82:0/880607477 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f43681078d0 0x7f4368109cf0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f4360009ca0 tx=0x7f436002a8c0 comp rx=0 tx=0).stop 2024-05-20T16:12:02.094 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:02.094+0000 7f436dd38640 1 -- 172.21.15.82:0/880607477 shutdown_connections 2024-05-20T16:12:02.094 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:02.094+0000 7f436dd38640 1 --2- 172.21.15.82:0/880607477 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f43681078d0 0x7f4368109cf0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:02.094 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:02.094+0000 7f436dd38640 1 --2- 172.21.15.82:0/880607477 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4368102bd0 0x7f4368102fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:02.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:02.094+0000 7f436dd38640 1 --2- 172.21.15.82:0/880607477 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4368101990 0x7f4368102690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:02.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:02.094+0000 7f436dd38640 1 -- 172.21.15.82:0/880607477 >> 172.21.15.82:0/880607477 conn(0x7f43680fcb80 msgr2=0x7f43680ff030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:12:02.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:02.094+0000 7f436dd38640 1 -- 172.21.15.82:0/880607477 shutdown_connections 2024-05-20T16:12:02.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:02.094+0000 7f436dd38640 1 -- 172.21.15.82:0/880607477 wait complete. 2024-05-20T16:12:02.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:02.094+0000 7f436dd38640 1 Processor -- start 2024-05-20T16:12:02.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:02.095+0000 7f436dd38640 1 -- start start 2024-05-20T16:12:02.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:02.095+0000 7f436dd38640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4368101990 0x7f43680698b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:12:02.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:02.095+0000 7f436dd38640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4368102bd0 0x7f436819e310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:12:02.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:02.095+0000 7f436dd38640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f43681078d0 0x7f43681a0860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:12:02.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:02.095+0000 7f436dd38640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f436810d5f0 con 0x7f4368102bd0 2024-05-20T16:12:02.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:02.095+0000 7f436dd38640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f436810d470 con 0x7f4368101990 2024-05-20T16:12:02.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:02.095+0000 7f436dd38640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f436810d770 con 0x7f43681078d0 2024-05-20T16:12:02.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:02.095+0000 7f4366ffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4368102bd0 0x7f436819e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:12:02.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:02.095+0000 7f4366ffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4368102bd0 0x7f436819e310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43452/0 (socket says 172.21.15.82:43452) 2024-05-20T16:12:02.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:02.095+0000 7f4366ffd640 1 -- 172.21.15.82:0/1878467659 learned_addr learned my addr 172.21.15.82:0/1878467659 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:12:02.098 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:02.095+0000 7f4367fff640 1 --2- 172.21.15.82:0/1878467659 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f43681078d0 0x7f43681a0860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:12:02.098 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:02.095+0000 7f43677fe640 1 --2- 172.21.15.82:0/1878467659 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4368101990 0x7f43680698b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:12:02.098 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:02.095+0000 7f4366ffd640 1 -- 172.21.15.82:0/1878467659 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f43681078d0 msgr2=0x7f43681a0860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:12:02.098 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:02.095+0000 7f4366ffd640 1 --2- 172.21.15.82:0/1878467659 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f43681078d0 0x7f43681a0860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:02.098 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:02.095+0000 7f4366ffd640 1 -- 172.21.15.82:0/1878467659 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4368101990 msgr2=0x7f43680698b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:12:02.098 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:02.095+0000 7f4366ffd640 1 --2- 172.21.15.82:0/1878467659 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4368101990 0x7f43680698b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:02.098 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:02.095+0000 7f4366ffd640 1 -- 172.21.15.82:0/1878467659 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43681a0ff0 con 0x7f4368102bd0 2024-05-20T16:12:02.098 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:02.095+0000 7f4367fff640 1 --2- 172.21.15.82:0/1878467659 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f43681078d0 0x7f43681a0860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T16:12:02.098 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:02.095+0000 7f43677fe640 1 --2- 172.21.15.82:0/1878467659 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4368101990 0x7f43680698b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T16:12:02.098 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:02.095+0000 7f4366ffd640 1 --2- 172.21.15.82:0/1878467659 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4368102bd0 0x7f436819e310 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f435c00d950 tx=0x7f435c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:12:02.098 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:02.096+0000 7f4364ff9640 1 -- 172.21.15.82:0/1878467659 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f435c017030 con 0x7f4368102bd0 2024-05-20T16:12:02.098 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:02.096+0000 7f436dd38640 1 -- 172.21.15.82:0/1878467659 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43681a12e0 con 0x7f4368102bd0 2024-05-20T16:12:02.098 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:02.096+0000 7f4364ff9640 1 -- 172.21.15.82:0/1878467659 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f435c004020 con 0x7f4368102bd0 2024-05-20T16:12:02.099 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:02.096+0000 7f4364ff9640 1 -- 172.21.15.82:0/1878467659 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f435c0047a0 con 0x7f4368102bd0 2024-05-20T16:12:02.099 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:02.096+0000 7f436dd38640 1 -- 172.21.15.82:0/1878467659 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43681a1830 con 0x7f4368102bd0 2024-05-20T16:12:02.099 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:02.097+0000 7f436dd38640 1 -- 172.21.15.82:0/1878467659 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43681066e0 con 0x7f4368102bd0 2024-05-20T16:12:02.099 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:02.098+0000 7f4364ff9640 1 -- 172.21.15.82:0/1878467659 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f435c0041d0 con 0x7f4368102bd0 2024-05-20T16:12:02.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:02.099+0000 7f4364ff9640 1 --2- 172.21.15.82:0/1878467659 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f43500a2b90 0x7f43500a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:12:02.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:02.099+0000 7f43677fe640 1 --2- 172.21.15.82:0/1878467659 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f43500a2b90 0x7f43500a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:12:02.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:02.099+0000 7f4364ff9640 1 -- 172.21.15.82:0/1878467659 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8802+0+0 (secure 0 0 0) 0x7f435c0cf390 con 0x7f4368102bd0 2024-05-20T16:12:02.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:02.099+0000 7f43677fe640 1 --2- 172.21.15.82:0/1878467659 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f43500a2b90 0x7f43500a50a0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f4368102590 tx=0x7f4358005db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:12:02.102 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:02.102+0000 7f4364ff9640 1 -- 172.21.15.82:0/1878467659 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f435c09c0b0 con 0x7f4368102bd0 2024-05-20T16:12:02.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:02.283+0000 7f436dd38640 1 -- 172.21.15.82:0/1878467659 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f436802d350 con 0x7f4368102bd0 2024-05-20T16:12:02.284 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:02.284+0000 7f4364ff9640 1 -- 172.21.15.82:0/1878467659 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v91) ==== 100+0+11 (secure 0 0 0) 0x7f435c09b9b0 con 0x7f4368102bd0 2024-05-20T16:12:02.284 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 64 2024-05-20T16:12:02.286 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:02.286+0000 7f436dd38640 1 -- 172.21.15.82:0/1878467659 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f43500a2b90 msgr2=0x7f43500a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:12:02.286 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:02.286+0000 7f436dd38640 1 --2- 172.21.15.82:0/1878467659 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f43500a2b90 0x7f43500a50a0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f4368102590 tx=0x7f4358005db0 comp rx=0 tx=0).stop 2024-05-20T16:12:02.287 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:02.286+0000 7f436dd38640 1 -- 172.21.15.82:0/1878467659 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4368102bd0 msgr2=0x7f436819e310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:12:02.287 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:02.286+0000 7f436dd38640 1 --2- 172.21.15.82:0/1878467659 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4368102bd0 0x7f436819e310 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f435c00d950 tx=0x7f435c00de10 comp rx=0 tx=0).stop 2024-05-20T16:12:02.287 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:02.286+0000 7f436dd38640 1 -- 172.21.15.82:0/1878467659 shutdown_connections 2024-05-20T16:12:02.287 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:02.286+0000 7f436dd38640 1 --2- 172.21.15.82:0/1878467659 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f43500a2b90 0x7f43500a50a0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:02.287 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:02.286+0000 7f436dd38640 1 --2- 172.21.15.82:0/1878467659 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4368102bd0 0x7f436819e310 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:02.287 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:02.286+0000 7f436dd38640 1 --2- 172.21.15.82:0/1878467659 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f43681078d0 0x7f43681a0860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:02.287 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:02.286+0000 7f436dd38640 1 --2- 172.21.15.82:0/1878467659 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4368101990 0x7f43680698b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:02.287 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:02.286+0000 7f436dd38640 1 -- 172.21.15.82:0/1878467659 >> 172.21.15.82:0/1878467659 conn(0x7f43680fcb80 msgr2=0x7f43681038b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:12:02.287 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:02.286+0000 7f436dd38640 1 -- 172.21.15.82:0/1878467659 shutdown_connections 2024-05-20T16:12:02.287 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:02.287+0000 7f436dd38640 1 -- 172.21.15.82:0/1878467659 wait complete. 2024-05-20T16:12:02.298 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-20T16:12:02.417 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:02.414+0000 7fe91b938640 1 -- 172.21.15.82:0/461341919 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe914102bd0 msgr2=0x7fe914102fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:12:02.417 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:02.414+0000 7fe91b938640 1 --2- 172.21.15.82:0/461341919 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe914102bd0 0x7fe914102fe0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fe904009900 tx=0x7fe90402a410 comp rx=0 tx=0).stop 2024-05-20T16:12:02.417 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:02.414+0000 7fe91b938640 1 -- 172.21.15.82:0/461341919 shutdown_connections 2024-05-20T16:12:02.417 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:02.414+0000 7fe91b938640 1 --2- 172.21.15.82:0/461341919 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe914102bd0 0x7fe914102fe0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:02.417 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:02.414+0000 7fe91b938640 1 --2- 172.21.15.82:0/461341919 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe9141078d0 0x7fe914109cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:02.417 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:02.414+0000 7fe91b938640 1 --2- 172.21.15.82:0/461341919 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe914101990 0x7fe914102690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:02.417 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:02.414+0000 7fe91b938640 1 -- 172.21.15.82:0/461341919 >> 172.21.15.82:0/461341919 conn(0x7fe9140fcb80 msgr2=0x7fe9140ff030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:12:02.417 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:02.414+0000 7fe91b938640 1 -- 172.21.15.82:0/461341919 shutdown_connections 2024-05-20T16:12:02.417 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:02.414+0000 7fe91b938640 1 -- 172.21.15.82:0/461341919 wait complete. 2024-05-20T16:12:02.417 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:02.415+0000 7fe91b938640 1 Processor -- start 2024-05-20T16:12:02.418 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:02.415+0000 7fe91b938640 1 -- start start 2024-05-20T16:12:02.418 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:02.415+0000 7fe91b938640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe914101990 0x7fe914074850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:12:02.418 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:02.415+0000 7fe91b938640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe914102bd0 0x7fe91419e030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:12:02.418 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:02.415+0000 7fe91b938640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe9141078d0 0x7fe9141a05e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:12:02.418 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:02.415+0000 7fe91b938640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fe91410d610 con 0x7fe9141078d0 2024-05-20T16:12:02.418 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:02.415+0000 7fe91b938640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fe91410d490 con 0x7fe914102bd0 2024-05-20T16:12:02.418 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:02.415+0000 7fe91b938640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fe91410d790 con 0x7fe914101990 2024-05-20T16:12:02.418 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:02.415+0000 7fe919ed5640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe9141078d0 0x7fe9141a05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:12:02.418 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:02.415+0000 7fe919ed5640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe9141078d0 0x7fe9141a05e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43470/0 (socket says 172.21.15.82:43470) 2024-05-20T16:12:02.419 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:02.415+0000 7fe919ed5640 1 -- 172.21.15.82:0/3944625266 learned_addr learned my addr 172.21.15.82:0/3944625266 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:12:02.419 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:02.415+0000 7fe918ed3640 1 --2- 172.21.15.82:0/3944625266 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe914102bd0 0x7fe91419e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:12:02.419 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:02.415+0000 7fe9196d4640 1 --2- 172.21.15.82:0/3944625266 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe914101990 0x7fe914074850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:12:02.419 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:02.415+0000 7fe919ed5640 1 -- 172.21.15.82:0/3944625266 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe914101990 msgr2=0x7fe914074850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:12:02.419 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:02.415+0000 7fe919ed5640 1 --2- 172.21.15.82:0/3944625266 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe914101990 0x7fe914074850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:02.419 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:02.415+0000 7fe919ed5640 1 -- 172.21.15.82:0/3944625266 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe914102bd0 msgr2=0x7fe91419e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:12:02.419 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:02.415+0000 7fe919ed5640 1 --2- 172.21.15.82:0/3944625266 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe914102bd0 0x7fe91419e030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:02.419 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:02.415+0000 7fe919ed5640 1 -- 172.21.15.82:0/3944625266 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9141a0e00 con 0x7fe9141078d0 2024-05-20T16:12:02.419 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:02.415+0000 7fe918ed3640 1 --2- 172.21.15.82:0/3944625266 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe914102bd0 0x7fe91419e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T16:12:02.419 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:02.416+0000 7fe919ed5640 1 --2- 172.21.15.82:0/3944625266 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe9141078d0 0x7fe9141a05e0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fe90c00eb10 tx=0x7fe90c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:12:02.420 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:02.416+0000 7fe90a7fc640 1 -- 172.21.15.82:0/3944625266 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe90c004b20 con 0x7fe9141078d0 2024-05-20T16:12:02.420 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:02.416+0000 7fe90a7fc640 1 -- 172.21.15.82:0/3944625266 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe90c004cc0 con 0x7fe9141078d0 2024-05-20T16:12:02.420 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:02.416+0000 7fe91b938640 1 -- 172.21.15.82:0/3944625266 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9141a10f0 con 0x7fe9141078d0 2024-05-20T16:12:02.420 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:02.416+0000 7fe90a7fc640 1 -- 172.21.15.82:0/3944625266 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe90c010760 con 0x7fe9141078d0 2024-05-20T16:12:02.420 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:02.416+0000 7fe91b938640 1 -- 172.21.15.82:0/3944625266 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9141a1640 con 0x7fe9141078d0 2024-05-20T16:12:02.420 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:02.417+0000 7fe91b938640 1 -- 172.21.15.82:0/3944625266 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9141066e0 con 0x7fe9141078d0 2024-05-20T16:12:02.421 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:02.419+0000 7fe90a7fc640 1 -- 172.21.15.82:0/3944625266 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fe90c005000 con 0x7fe9141078d0 2024-05-20T16:12:02.421 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:02.419+0000 7fe90a7fc640 1 --2- 172.21.15.82:0/3944625266 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fe9000a2b90 0x7fe9000a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:12:02.421 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:02.420+0000 7fe9196d4640 1 --2- 172.21.15.82:0/3944625266 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fe9000a2b90 0x7fe9000a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:12:02.421 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:02.420+0000 7fe90a7fc640 1 -- 172.21.15.82:0/3944625266 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8802+0+0 (secure 0 0 0) 0x7fe90c0cfdf0 con 0x7fe9141078d0 2024-05-20T16:12:02.421 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:02.420+0000 7fe9196d4640 1 --2- 172.21.15.82:0/3944625266 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fe9000a2b90 0x7fe9000a50a0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fe914102590 tx=0x7fe910005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:12:02.423 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:02.423+0000 7fe90a7fc640 1 -- 172.21.15.82:0/3944625266 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fe90c09cb10 con 0x7fe9141078d0 2024-05-20T16:12:02.597 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:02.597+0000 7fe91b938640 1 -- 172.21.15.82:0/3944625266 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fe91402d430 con 0x7fe9141078d0 2024-05-20T16:12:02.597 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:02.598+0000 7fe90a7fc640 1 -- 172.21.15.82:0/3944625266 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v91) ==== 103+0+11 (secure 0 0 0) 0x7fe90c09c410 con 0x7fe9141078d0 2024-05-20T16:12:02.598 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 32 2024-05-20T16:12:02.599 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:02.599+0000 7fe91b938640 1 -- 172.21.15.82:0/3944625266 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fe9000a2b90 msgr2=0x7fe9000a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:12:02.600 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:02.599+0000 7fe91b938640 1 --2- 172.21.15.82:0/3944625266 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fe9000a2b90 0x7fe9000a50a0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fe914102590 tx=0x7fe910005f90 comp rx=0 tx=0).stop 2024-05-20T16:12:02.600 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:02.599+0000 7fe91b938640 1 -- 172.21.15.82:0/3944625266 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe9141078d0 msgr2=0x7fe9141a05e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:12:02.600 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:02.599+0000 7fe91b938640 1 --2- 172.21.15.82:0/3944625266 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe9141078d0 0x7fe9141a05e0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fe90c00eb10 tx=0x7fe90c00c2d0 comp rx=0 tx=0).stop 2024-05-20T16:12:02.600 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:02.599+0000 7fe91b938640 1 -- 172.21.15.82:0/3944625266 shutdown_connections 2024-05-20T16:12:02.600 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:02.600+0000 7fe91b938640 1 --2- 172.21.15.82:0/3944625266 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fe9000a2b90 0x7fe9000a50a0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:02.600 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:02.600+0000 7fe91b938640 1 --2- 172.21.15.82:0/3944625266 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe9141078d0 0x7fe9141a05e0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:02.600 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:02.600+0000 7fe91b938640 1 --2- 172.21.15.82:0/3944625266 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe914102bd0 0x7fe91419e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:02.600 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:02.600+0000 7fe91b938640 1 --2- 172.21.15.82:0/3944625266 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe914101990 0x7fe914074850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:02.600 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:02.600+0000 7fe91b938640 1 -- 172.21.15.82:0/3944625266 >> 172.21.15.82:0/3944625266 conn(0x7fe9140fcb80 msgr2=0x7fe9141038b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:12:02.600 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:02.600+0000 7fe91b938640 1 -- 172.21.15.82:0/3944625266 shutdown_connections 2024-05-20T16:12:02.600 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:02.600+0000 7fe91b938640 1 -- 172.21.15.82:0/3944625266 wait complete. 2024-05-20T16:12:02.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:12:02 smithi082 ceph-mon[32238]: from='client.? 172.21.15.82:0/422695650' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-20T16:12:02.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:12:02 smithi082 ceph-mon[32238]: from='client.? 172.21.15.82:0/1132676645' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-20T16:12:02.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:12:02 smithi082 ceph-mon[32238]: from='client.? 172.21.15.82:0/1878467659' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-20T16:12:02.611 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-20T16:12:02.635 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:12:02 smithi184 ceph-mon[36209]: from='client.? 172.21.15.82:0/422695650' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-20T16:12:02.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:12:02 smithi184 ceph-mon[36209]: from='client.? 172.21.15.82:0/1132676645' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-20T16:12:02.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:12:02 smithi184 ceph-mon[36209]: from='client.? 172.21.15.82:0/1878467659' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-20T16:12:02.722 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:02.722+0000 7fe8e8f4c640 1 -- 172.21.15.82:0/3674788921 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe8e4102bd0 msgr2=0x7fe8e4102fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:12:02.722 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:02.722+0000 7fe8e8f4c640 1 --2- 172.21.15.82:0/3674788921 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe8e4102bd0 0x7fe8e4102fe0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fe8d8009900 tx=0x7fe8d802a410 comp rx=0 tx=0).stop 2024-05-20T16:12:02.722 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:02.722+0000 7fe8e8f4c640 1 -- 172.21.15.82:0/3674788921 shutdown_connections 2024-05-20T16:12:02.722 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:02.722+0000 7fe8e8f4c640 1 --2- 172.21.15.82:0/3674788921 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe8e4102bd0 0x7fe8e4102fe0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:02.722 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:02.722+0000 7fe8e8f4c640 1 --2- 172.21.15.82:0/3674788921 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe8e41078d0 0x7fe8e4109cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:02.722 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:02.722+0000 7fe8e8f4c640 1 --2- 172.21.15.82:0/3674788921 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe8e4101990 0x7fe8e4102690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:02.722 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:02.722+0000 7fe8e8f4c640 1 -- 172.21.15.82:0/3674788921 >> 172.21.15.82:0/3674788921 conn(0x7fe8e40fcb80 msgr2=0x7fe8e40ff030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:12:02.723 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:02.722+0000 7fe8e8f4c640 1 -- 172.21.15.82:0/3674788921 shutdown_connections 2024-05-20T16:12:02.723 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:02.722+0000 7fe8e8f4c640 1 -- 172.21.15.82:0/3674788921 wait complete. 2024-05-20T16:12:02.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:02.723+0000 7fe8e8f4c640 1 Processor -- start 2024-05-20T16:12:02.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:02.723+0000 7fe8e8f4c640 1 -- start start 2024-05-20T16:12:02.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:02.723+0000 7fe8e8f4c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe8e4101990 0x7fe8e41984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:12:02.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:02.723+0000 7fe8e8f4c640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe8e4102bd0 0x7fe8e41989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:12:02.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:02.723+0000 7fe8e8f4c640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe8e41078d0 0x7fe8e419f980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:12:02.726 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:02.723+0000 7fe8e8f4c640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fe8e410d5e0 con 0x7fe8e4101990 2024-05-20T16:12:02.726 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:02.723+0000 7fe8e8f4c640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fe8e410d460 con 0x7fe8e41078d0 2024-05-20T16:12:02.726 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:02.723+0000 7fe8e8f4c640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fe8e410d760 con 0x7fe8e4102bd0 2024-05-20T16:12:02.726 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:02.723+0000 7fe8e259c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe8e4101990 0x7fe8e41984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:12:02.726 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:02.723+0000 7fe8e259c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe8e4101990 0x7fe8e41984b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43486/0 (socket says 172.21.15.82:43486) 2024-05-20T16:12:02.726 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:02.723+0000 7fe8e259c640 1 -- 172.21.15.82:0/1205293137 learned_addr learned my addr 172.21.15.82:0/1205293137 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:12:02.726 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:02.723+0000 7fe8e1d9b640 1 --2- 172.21.15.82:0/1205293137 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe8e4102bd0 0x7fe8e41989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:12:02.726 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:02.723+0000 7fe8e259c640 1 -- 172.21.15.82:0/1205293137 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe8e4102bd0 msgr2=0x7fe8e41989f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:12:02.726 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:02.723+0000 7fe8e259c640 1 --2- 172.21.15.82:0/1205293137 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe8e4102bd0 0x7fe8e41989f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:02.726 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:02.723+0000 7fe8e259c640 1 -- 172.21.15.82:0/1205293137 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe8e41078d0 msgr2=0x7fe8e419f980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:12:02.726 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:02.723+0000 7fe8e2d9d640 1 --2- 172.21.15.82:0/1205293137 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe8e41078d0 0x7fe8e419f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:12:02.726 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:02.723+0000 7fe8e259c640 1 --2- 172.21.15.82:0/1205293137 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe8e41078d0 0x7fe8e419f980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:02.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:02.723+0000 7fe8e259c640 1 -- 172.21.15.82:0/1205293137 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8e419fff0 con 0x7fe8e4101990 2024-05-20T16:12:02.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:02.723+0000 7fe8e2d9d640 1 --2- 172.21.15.82:0/1205293137 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe8e41078d0 0x7fe8e419f980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T16:12:02.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:02.723+0000 7fe8e259c640 1 --2- 172.21.15.82:0/1205293137 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe8e4101990 0x7fe8e41984b0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fe8d4009830 tx=0x7fe8d4009cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:12:02.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:02.724+0000 7fe8d37fe640 1 -- 172.21.15.82:0/1205293137 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8d4004020 con 0x7fe8e4101990 2024-05-20T16:12:02.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:02.724+0000 7fe8d37fe640 1 -- 172.21.15.82:0/1205293137 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe8d4007c80 con 0x7fe8e4101990 2024-05-20T16:12:02.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:02.724+0000 7fe8e8f4c640 1 -- 172.21.15.82:0/1205293137 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8e4068ea0 con 0x7fe8e4101990 2024-05-20T16:12:02.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:02.724+0000 7fe8d37fe640 1 -- 172.21.15.82:0/1205293137 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8d4003ab0 con 0x7fe8e4101990 2024-05-20T16:12:02.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:02.724+0000 7fe8e8f4c640 1 -- 172.21.15.82:0/1205293137 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8e4069440 con 0x7fe8e4101990 2024-05-20T16:12:02.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:02.725+0000 7fe8e8f4c640 1 -- 172.21.15.82:0/1205293137 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8e4069f80 con 0x7fe8e4101990 2024-05-20T16:12:02.728 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:02.726+0000 7fe8d37fe640 1 -- 172.21.15.82:0/1205293137 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fe8d4003c70 con 0x7fe8e4101990 2024-05-20T16:12:02.728 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:02.726+0000 7fe8d37fe640 1 --2- 172.21.15.82:0/1205293137 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fe8cc0a2c10 0x7fe8cc0a5120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:12:02.728 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:02.727+0000 7fe8e1d9b640 1 --2- 172.21.15.82:0/1205293137 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fe8cc0a2c10 0x7fe8cc0a5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:12:02.728 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:02.727+0000 7fe8d37fe640 1 -- 172.21.15.82:0/1205293137 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8802+0+0 (secure 0 0 0) 0x7fe8d40d07b0 con 0x7fe8e4101990 2024-05-20T16:12:02.728 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:02.727+0000 7fe8e1d9b640 1 --2- 172.21.15.82:0/1205293137 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fe8cc0a2c10 0x7fe8cc0a5120 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fe8d8000c00 tx=0x7fe8d8031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:12:02.731 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:02.731+0000 7fe8d37fe640 1 -- 172.21.15.82:0/1205293137 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fe8e4069f80 con 0x7fe8e4101990 2024-05-20T16:12:02.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:12:02 smithi160 ceph-mon[36347]: from='client.? 172.21.15.82:0/422695650' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-20T16:12:02.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:12:02 smithi160 ceph-mon[36347]: from='client.? 172.21.15.82:0/1132676645' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-20T16:12:02.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:12:02 smithi160 ceph-mon[36347]: from='client.? 172.21.15.82:0/1878467659' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-20T16:12:02.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:02.921+0000 7fe8e8f4c640 1 -- 172.21.15.82:0/1205293137 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fe8e41063a0 con 0x7fe8e4101990 2024-05-20T16:12:02.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:02.922+0000 7fe8d37fe640 1 -- 172.21.15.82:0/1205293137 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v91) ==== 81+0+591 (secure 0 0 0) 0x7fe8d409d4d0 con 0x7fe8e4101990 2024-05-20T16:12:02.922 INFO:teuthology.orchestra.run.smithi082.stdout:172.21.15.82:0/2005424509 2024-05-21T16:04:45.161949+0000 2024-05-20T16:12:02.922 INFO:teuthology.orchestra.run.smithi082.stdout:172.21.15.82:6801/1763049308 2024-05-21T16:04:45.161949+0000 2024-05-20T16:12:02.922 INFO:teuthology.orchestra.run.smithi082.stdout:172.21.15.82:6801/1959459348 2024-05-21T16:04:19.208291+0000 2024-05-20T16:12:02.922 INFO:teuthology.orchestra.run.smithi082.stdout:172.21.15.82:0/3131904556 2024-05-21T16:04:19.208291+0000 2024-05-20T16:12:02.923 INFO:teuthology.orchestra.run.smithi082.stdout:172.21.15.82:6800/1959459348 2024-05-21T16:04:19.208291+0000 2024-05-20T16:12:02.923 INFO:teuthology.orchestra.run.smithi082.stdout:172.21.15.82:0/2456966342 2024-05-21T16:04:45.161949+0000 2024-05-20T16:12:02.923 INFO:teuthology.orchestra.run.smithi082.stdout:172.21.15.82:0/4179978833 2024-05-21T16:04:19.208291+0000 2024-05-20T16:12:02.923 INFO:teuthology.orchestra.run.smithi082.stdout:172.21.15.82:0/1602862225 2024-05-21T16:04:45.161949+0000 2024-05-20T16:12:02.923 INFO:teuthology.orchestra.run.smithi082.stdout:172.21.15.82:0/106101180 2024-05-21T16:04:19.208291+0000 2024-05-20T16:12:02.923 INFO:teuthology.orchestra.run.smithi082.stdout:172.21.15.82:6800/1763049308 2024-05-21T16:04:45.161949+0000 2024-05-20T16:12:02.923 INFO:teuthology.orchestra.run.smithi082.stderr:listed 10 entries 2024-05-20T16:12:02.924 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:02.924+0000 7fe8e8f4c640 1 -- 172.21.15.82:0/1205293137 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fe8cc0a2c10 msgr2=0x7fe8cc0a5120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:12:02.924 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:02.924+0000 7fe8e8f4c640 1 --2- 172.21.15.82:0/1205293137 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fe8cc0a2c10 0x7fe8cc0a5120 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fe8d8000c00 tx=0x7fe8d8031400 comp rx=0 tx=0).stop 2024-05-20T16:12:02.924 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:02.924+0000 7fe8e8f4c640 1 -- 172.21.15.82:0/1205293137 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe8e4101990 msgr2=0x7fe8e41984b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:12:02.924 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:02.924+0000 7fe8e8f4c640 1 --2- 172.21.15.82:0/1205293137 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe8e4101990 0x7fe8e41984b0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fe8d4009830 tx=0x7fe8d4009cf0 comp rx=0 tx=0).stop 2024-05-20T16:12:02.924 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:02.924+0000 7fe8e8f4c640 1 -- 172.21.15.82:0/1205293137 shutdown_connections 2024-05-20T16:12:02.924 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:02.924+0000 7fe8e8f4c640 1 --2- 172.21.15.82:0/1205293137 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fe8cc0a2c10 0x7fe8cc0a5120 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:02.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:02.924+0000 7fe8e8f4c640 1 --2- 172.21.15.82:0/1205293137 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe8e41078d0 0x7fe8e419f980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:02.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:02.924+0000 7fe8e8f4c640 1 --2- 172.21.15.82:0/1205293137 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe8e4102bd0 0x7fe8e41989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:02.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:02.924+0000 7fe8e8f4c640 1 --2- 172.21.15.82:0/1205293137 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe8e4101990 0x7fe8e41984b0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:02.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:02.924+0000 7fe8e8f4c640 1 -- 172.21.15.82:0/1205293137 >> 172.21.15.82:0/1205293137 conn(0x7fe8e40fcb80 msgr2=0x7fe8e41038b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:12:02.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:02.924+0000 7fe8e8f4c640 1 -- 172.21.15.82:0/1205293137 shutdown_connections 2024-05-20T16:12:02.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:02.924+0000 7fe8e8f4c640 1 -- 172.21.15.82:0/1205293137 wait complete. 2024-05-20T16:12:02.935 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi082.front.sepia.ceph.com... 2024-05-20T16:12:02.935 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T16:12:02.936 DEBUG:teuthology.orchestra.run.smithi082:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-05-20T16:12:02.959 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi082.stderr:ceph-fuse[75520]: fuse finished with error 0 and tester_r 0 2024-05-20T16:12:02.960 INFO:teuthology.orchestra.run:waiting for 300 2024-05-20T16:12:03.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:12:03 smithi184 ceph-mon[36209]: from='client.? 172.21.15.82:0/3944625266' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-20T16:12:03.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:12:03 smithi184 ceph-mon[36209]: from='client.? 172.21.15.82:0/1205293137' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-20T16:12:03.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:12:03 smithi184 ceph-mon[36209]: pgmap v300: 129 pgs: 129 active+clean; 581 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 307 B/s wr, 0 op/s 2024-05-20T16:12:03.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:12:03 smithi160 ceph-mon[36347]: from='client.? 172.21.15.82:0/3944625266' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-20T16:12:03.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:12:03 smithi160 ceph-mon[36347]: from='client.? 172.21.15.82:0/1205293137' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-20T16:12:03.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:12:03 smithi160 ceph-mon[36347]: pgmap v300: 129 pgs: 129 active+clean; 581 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 307 B/s wr, 0 op/s 2024-05-20T16:12:03.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:12:03 smithi082 ceph-mon[32238]: from='client.? 172.21.15.82:0/3944625266' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-20T16:12:03.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:12:03 smithi082 ceph-mon[32238]: from='client.? 172.21.15.82:0/1205293137' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-20T16:12:03.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:12:03 smithi082 ceph-mon[32238]: pgmap v300: 129 pgs: 129 active+clean; 581 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 307 B/s wr, 0 op/s 2024-05-20T16:12:05.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:12:05 smithi082 ceph-mon[32238]: pgmap v301: 129 pgs: 129 active+clean; 581 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 259 B/s wr, 0 op/s 2024-05-20T16:12:05.635 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:12:05 smithi184 ceph-mon[36209]: pgmap v301: 129 pgs: 129 active+clean; 581 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 259 B/s wr, 0 op/s 2024-05-20T16:12:05.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:12:05 smithi160 ceph-mon[36347]: pgmap v301: 129 pgs: 129 active+clean; 581 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 259 B/s wr, 0 op/s 2024-05-20T16:12:07.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:12:07 smithi082 ceph-mon[32238]: pgmap v302: 129 pgs: 129 active+clean; 581 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-05-20T16:12:07.635 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:12:07 smithi184 ceph-mon[36209]: pgmap v302: 129 pgs: 129 active+clean; 581 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-05-20T16:12:07.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:12:07 smithi160 ceph-mon[36347]: pgmap v302: 129 pgs: 129 active+clean; 581 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-05-20T16:12:09.061 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi082.front.sepia.ceph.com 2024-05-20T16:12:09.062 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T16:12:09.062 DEBUG:teuthology.orchestra.run.smithi082:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-05-20T16:12:09.087 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-20T16:12:09.254 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:09.252+0000 7f3ebbf37640 1 -- 172.21.15.82:0/3178380020 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3eb4105ec0 msgr2=0x7f3eb41082e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:12:09.254 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:09.252+0000 7f3ebbf37640 1 --2- 172.21.15.82:0/3178380020 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3eb4105ec0 0x7f3eb41082e0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f3ea4009900 tx=0x7f3ea402a410 comp rx=0 tx=0).stop 2024-05-20T16:12:09.254 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:09.252+0000 7f3ebbf37640 1 -- 172.21.15.82:0/3178380020 shutdown_connections 2024-05-20T16:12:09.254 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:09.252+0000 7f3ebbf37640 1 --2- 172.21.15.82:0/3178380020 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3eb4105ec0 0x7f3eb41082e0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:09.254 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:09.252+0000 7f3ebbf37640 1 --2- 172.21.15.82:0/3178380020 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3eb4108820 0x7f3eb410ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:09.254 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:09.252+0000 7f3ebbf37640 1 --2- 172.21.15.82:0/3178380020 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3eb4103560 0x7f3eb4105980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:09.255 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:09.252+0000 7f3ebbf37640 1 -- 172.21.15.82:0/3178380020 >> 172.21.15.82:0/3178380020 conn(0x7f3eb40fcb80 msgr2=0x7f3eb40ff010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:12:09.255 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:09.252+0000 7f3ebbf37640 1 -- 172.21.15.82:0/3178380020 shutdown_connections 2024-05-20T16:12:09.255 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:09.252+0000 7f3ebbf37640 1 -- 172.21.15.82:0/3178380020 wait complete. 2024-05-20T16:12:09.255 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:09.252+0000 7f3ebbf37640 1 Processor -- start 2024-05-20T16:12:09.255 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:09.252+0000 7f3ebbf37640 1 -- start start 2024-05-20T16:12:09.255 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:09.253+0000 7f3ebbf37640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3eb4103560 0x7f3eb41992d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:12:09.255 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:09.253+0000 7f3ebbf37640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3eb4105ec0 0x7f3eb4199810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:12:09.255 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:09.253+0000 7f3ebbf37640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3eb4108820 0x7f3eb41a07a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:12:09.255 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:09.253+0000 7f3ebbf37640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f3eb410e6f0 con 0x7f3eb4105ec0 2024-05-20T16:12:09.255 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:09.253+0000 7f3ebbf37640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f3eb410e570 con 0x7f3eb4103560 2024-05-20T16:12:09.255 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:09.253+0000 7f3ebbf37640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f3eb410e870 con 0x7f3eb4108820 2024-05-20T16:12:09.255 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:09.253+0000 7f3eb94d2640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3eb4105ec0 0x7f3eb4199810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:12:09.255 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:09.253+0000 7f3eb94d2640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3eb4105ec0 0x7f3eb4199810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43500/0 (socket says 172.21.15.82:43500) 2024-05-20T16:12:09.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:09.253+0000 7f3eb94d2640 1 -- 172.21.15.82:0/1629290717 learned_addr learned my addr 172.21.15.82:0/1629290717 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:12:09.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:09.253+0000 7f3eba4d4640 1 --2- 172.21.15.82:0/1629290717 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3eb4108820 0x7f3eb41a07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:12:09.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:09.253+0000 7f3eb9cd3640 1 --2- 172.21.15.82:0/1629290717 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3eb4103560 0x7f3eb41992d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:12:09.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:09.253+0000 7f3eb94d2640 1 -- 172.21.15.82:0/1629290717 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3eb4108820 msgr2=0x7f3eb41a07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:12:09.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:09.253+0000 7f3eb94d2640 1 --2- 172.21.15.82:0/1629290717 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3eb4108820 0x7f3eb41a07a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:09.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:09.253+0000 7f3eb94d2640 1 -- 172.21.15.82:0/1629290717 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3eb4103560 msgr2=0x7f3eb41992d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:12:09.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:09.253+0000 7f3eb94d2640 1 --2- 172.21.15.82:0/1629290717 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3eb4103560 0x7f3eb41992d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:09.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:09.253+0000 7f3eb94d2640 1 -- 172.21.15.82:0/1629290717 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3eb41a0e10 con 0x7f3eb4105ec0 2024-05-20T16:12:09.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:09.253+0000 7f3eba4d4640 1 --2- 172.21.15.82:0/1629290717 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3eb4108820 0x7f3eb41a07a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T16:12:09.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:09.253+0000 7f3eb9cd3640 1 --2- 172.21.15.82:0/1629290717 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3eb4103560 0x7f3eb41992d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T16:12:09.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:09.253+0000 7f3eb94d2640 1 --2- 172.21.15.82:0/1629290717 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3eb4105ec0 0x7f3eb4199810 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f3ea4006010 tx=0x7f3ea4004360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:12:09.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:09.253+0000 7f3eaaffd640 1 -- 172.21.15.82:0/1629290717 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ea403f030 con 0x7f3eb4105ec0 2024-05-20T16:12:09.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:09.253+0000 7f3ebbf37640 1 -- 172.21.15.82:0/1629290717 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3eb41a10a0 con 0x7f3eb4105ec0 2024-05-20T16:12:09.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:09.254+0000 7f3eaaffd640 1 -- 172.21.15.82:0/1629290717 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ea4003ce0 con 0x7f3eb4105ec0 2024-05-20T16:12:09.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:09.254+0000 7f3eaaffd640 1 -- 172.21.15.82:0/1629290717 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ea4035070 con 0x7f3eb4105ec0 2024-05-20T16:12:09.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:09.254+0000 7f3ebbf37640 1 -- 172.21.15.82:0/1629290717 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3eb41a15e0 con 0x7f3eb4105ec0 2024-05-20T16:12:09.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:09.255+0000 7f3ebbf37640 1 -- 172.21.15.82:0/1629290717 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3eb40fe6a0 con 0x7f3eb4105ec0 2024-05-20T16:12:09.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:09.256+0000 7f3eaaffd640 1 -- 172.21.15.82:0/1629290717 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f3ea4042030 con 0x7f3eb4105ec0 2024-05-20T16:12:09.258 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:09.257+0000 7f3eaaffd640 1 --2- 172.21.15.82:0/1629290717 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f3ea00a2b90 0x7f3ea00a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:12:09.258 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:09.257+0000 7f3eaaffd640 1 -- 172.21.15.82:0/1629290717 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8802+0+0 (secure 0 0 0) 0x7f3ea40efcd0 con 0x7f3eb4105ec0 2024-05-20T16:12:09.258 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:09.257+0000 7f3eb9cd3640 1 --2- 172.21.15.82:0/1629290717 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f3ea00a2b90 0x7f3ea00a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:12:09.258 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:09.257+0000 7f3eb9cd3640 1 --2- 172.21.15.82:0/1629290717 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f3ea00a2b90 0x7f3ea00a50a0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f3eb4069860 tx=0x7f3eb0005e50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:12:09.261 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:09.262+0000 7f3eaaffd640 1 -- 172.21.15.82:0/1629290717 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f3ea40bc9f0 con 0x7f3eb4105ec0 2024-05-20T16:12:09.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:09.443+0000 7f3ebbf37640 1 -- 172.21.15.82:0/1629290717 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3eb4062750 con 0x7f3eb4105ec0 2024-05-20T16:12:09.445 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:09.445+0000 7f3eaaffd640 1 -- 172.21.15.82:0/1629290717 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+24153 (secure 0 0 0) 0x7f3ea40bc2f0 con 0x7f3eb4105ec0 2024-05-20T16:12:09.445 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-05-20T16:12:09.445 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":91,"fsid":"59d1600a-16c2-11ef-bc9a-c7b262605968","created":"2024-05-20T16:03:46.611556+0000","modified":"2024-05-20T16:11:53.416085+0000","last_up_change":"2024-05-20T16:10:59.303479+0000","last_in_change":"2024-05-20T16:10:39.063561+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-20T16:07:09.493727+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-20T16:11:45.058035+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-20T16:11:45.943862+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-20T16:11:49.637579+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"bae842a9-03ec-490e-b56d-f7957235cdc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":1484330167},{"type":"v1","addr":"172.21.15.82:6803","nonce":1484330167}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":1484330167},{"type":"v1","addr":"172.21.15.82:6805","nonce":1484330167}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1484330167},{"type":"v1","addr":"172.21.15.82:6809","nonce":1484330167}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":1484330167},{"type":"v1","addr":"172.21.15.82:6807","nonce":1484330167}]},"public_addr":"172.21.15.82:6803/1484330167","cluster_addr":"172.21.15.82:6805/1484330167","heartbeat_back_addr":"172.21.15.82:6809/1484330167","heartbeat_front_addr":"172.21.15.82:6807/1484330167","state":["exists","up"]},{"osd":1,"uuid":"59f6a2ba-b5de-4161-81c5-92a417abacdf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":1457003568},{"type":"v1","addr":"172.21.15.160:6801","nonce":1457003568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":1457003568},{"type":"v1","addr":"172.21.15.160:6803","nonce":1457003568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6806","nonce":1457003568},{"type":"v1","addr":"172.21.15.160:6807","nonce":1457003568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":1457003568},{"type":"v1","addr":"172.21.15.160:6805","nonce":1457003568}]},"public_addr":"172.21.15.160:6801/1457003568","cluster_addr":"172.21.15.160:6803/1457003568","heartbeat_back_addr":"172.21.15.160:6807/1457003568","heartbeat_front_addr":"172.21.15.160:6805/1457003568","state":["exists","up"]},{"osd":2,"uuid":"3bbb5587-b449-41f3-bd26-67ec3d6afdfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":4051293934},{"type":"v1","addr":"172.21.15.184:6801","nonce":4051293934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":4051293934},{"type":"v1","addr":"172.21.15.184:6803","nonce":4051293934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":4051293934},{"type":"v1","addr":"172.21.15.184:6807","nonce":4051293934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":4051293934},{"type":"v1","addr":"172.21.15.184:6805","nonce":4051293934}]},"public_addr":"172.21.15.184:6801/4051293934","cluster_addr":"172.21.15.184:6803/4051293934","heartbeat_back_addr":"172.21.15.184:6807/4051293934","heartbeat_front_addr":"172.21.15.184:6805/4051293934","state":["exists","up"]},{"osd":3,"uuid":"ca651f90-09b8-4081-a6f9-40b31268a837","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":3947485599},{"type":"v1","addr":"172.21.15.82:6811","nonce":3947485599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":3947485599},{"type":"v1","addr":"172.21.15.82:6813","nonce":3947485599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":3947485599},{"type":"v1","addr":"172.21.15.82:6817","nonce":3947485599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":3947485599},{"type":"v1","addr":"172.21.15.82:6815","nonce":3947485599}]},"public_addr":"172.21.15.82:6811/3947485599","cluster_addr":"172.21.15.82:6813/3947485599","heartbeat_back_addr":"172.21.15.82:6817/3947485599","heartbeat_front_addr":"172.21.15.82:6815/3947485599","state":["exists","up"]},{"osd":4,"uuid":"cb3dfd56-f677-40cb-9c96-8319ea5ac788","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":4160830014},{"type":"v1","addr":"172.21.15.160:6809","nonce":4160830014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":4160830014},{"type":"v1","addr":"172.21.15.160:6811","nonce":4160830014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":4160830014},{"type":"v1","addr":"172.21.15.160:6815","nonce":4160830014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6812","nonce":4160830014},{"type":"v1","addr":"172.21.15.160:6813","nonce":4160830014}]},"public_addr":"172.21.15.160:6809/4160830014","cluster_addr":"172.21.15.160:6811/4160830014","heartbeat_back_addr":"172.21.15.160:6815/4160830014","heartbeat_front_addr":"172.21.15.160:6813/4160830014","state":["exists","up"]},{"osd":5,"uuid":"c70cfcd1-a4ff-461f-8c73-479b061200fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":1046282672},{"type":"v1","addr":"172.21.15.184:6809","nonce":1046282672}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":1046282672},{"type":"v1","addr":"172.21.15.184:6811","nonce":1046282672}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":1046282672},{"type":"v1","addr":"172.21.15.184:6815","nonce":1046282672}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":1046282672},{"type":"v1","addr":"172.21.15.184:6813","nonce":1046282672}]},"public_addr":"172.21.15.184:6809/1046282672","cluster_addr":"172.21.15.184:6811/1046282672","heartbeat_back_addr":"172.21.15.184:6815/1046282672","heartbeat_front_addr":"172.21.15.184:6813/1046282672","state":["exists","up"]},{"osd":6,"uuid":"d47ce001-9dd1-4830-bcc9-888601187c6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":2789736499},{"type":"v1","addr":"172.21.15.82:6819","nonce":2789736499}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":2789736499},{"type":"v1","addr":"172.21.15.82:6821","nonce":2789736499}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":2789736499},{"type":"v1","addr":"172.21.15.82:6825","nonce":2789736499}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":2789736499},{"type":"v1","addr":"172.21.15.82:6823","nonce":2789736499}]},"public_addr":"172.21.15.82:6819/2789736499","cluster_addr":"172.21.15.82:6821/2789736499","heartbeat_back_addr":"172.21.15.82:6825/2789736499","heartbeat_front_addr":"172.21.15.82:6823/2789736499","state":["exists","up"]},{"osd":7,"uuid":"25c54f43-18cb-41e4-83b4-2732b5a93889","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6816","nonce":2340683378},{"type":"v1","addr":"172.21.15.160:6817","nonce":2340683378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":2340683378},{"type":"v1","addr":"172.21.15.160:6819","nonce":2340683378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":2340683378},{"type":"v1","addr":"172.21.15.160:6823","nonce":2340683378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6820","nonce":2340683378},{"type":"v1","addr":"172.21.15.160:6821","nonce":2340683378}]},"public_addr":"172.21.15.160:6817/2340683378","cluster_addr":"172.21.15.160:6819/2340683378","heartbeat_back_addr":"172.21.15.160:6823/2340683378","heartbeat_front_addr":"172.21.15.160:6821/2340683378","state":["exists","up"]},{"osd":8,"uuid":"1f8b478d-287f-45f5-8526-d1a58c5bc07c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":3831576372},{"type":"v1","addr":"172.21.15.184:6817","nonce":3831576372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":3831576372},{"type":"v1","addr":"172.21.15.184:6819","nonce":3831576372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":3831576372},{"type":"v1","addr":"172.21.15.184:6823","nonce":3831576372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":3831576372},{"type":"v1","addr":"172.21.15.184:6821","nonce":3831576372}]},"public_addr":"172.21.15.184:6817/3831576372","cluster_addr":"172.21.15.184:6819/3831576372","heartbeat_back_addr":"172.21.15.184:6823/3831576372","heartbeat_front_addr":"172.21.15.184:6821/3831576372","state":["exists","up"]},{"osd":9,"uuid":"0d69e319-954f-4b18-95ea-31794f18dc3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":716330782},{"type":"v1","addr":"172.21.15.82:6827","nonce":716330782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":716330782},{"type":"v1","addr":"172.21.15.82:6829","nonce":716330782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":716330782},{"type":"v1","addr":"172.21.15.82:6833","nonce":716330782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":716330782},{"type":"v1","addr":"172.21.15.82:6831","nonce":716330782}]},"public_addr":"172.21.15.82:6827/716330782","cluster_addr":"172.21.15.82:6829/716330782","heartbeat_back_addr":"172.21.15.82:6833/716330782","heartbeat_front_addr":"172.21.15.82:6831/716330782","state":["exists","up"]},{"osd":10,"uuid":"39dc1b78-61cb-42ca-b5b6-1e44ac2b6500","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":2327428014},{"type":"v1","addr":"172.21.15.160:6825","nonce":2327428014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":2327428014},{"type":"v1","addr":"172.21.15.160:6827","nonce":2327428014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":2327428014},{"type":"v1","addr":"172.21.15.160:6831","nonce":2327428014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":2327428014},{"type":"v1","addr":"172.21.15.160:6829","nonce":2327428014}]},"public_addr":"172.21.15.160:6825/2327428014","cluster_addr":"172.21.15.160:6827/2327428014","heartbeat_back_addr":"172.21.15.160:6831/2327428014","heartbeat_front_addr":"172.21.15.160:6829/2327428014","state":["exists","up"]},{"osd":11,"uuid":"4306b3c1-707a-4010-987c-209ea0e948e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":3594364650},{"type":"v1","addr":"172.21.15.184:6825","nonce":3594364650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":3594364650},{"type":"v1","addr":"172.21.15.184:6827","nonce":3594364650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3594364650},{"type":"v1","addr":"172.21.15.184:6831","nonce":3594364650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3594364650},{"type":"v1","addr":"172.21.15.184:6829","nonce":3594364650}]},"public_addr":"172.21.15.184:6825/3594364650","cluster_addr":"172.21.15.184:6827/3594364650","heartbeat_back_addr":"172.21.15.184:6831/3594364650","heartbeat_front_addr":"172.21.15.184:6829/3594364650","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:06:17.855998+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:06:41.473270+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:07:05.304108+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:07:29.887472+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:07:55.387286+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:08:20.579669+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:08:46.333051+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:09:12.289668+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:09:38.180696+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:10:04.471459+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:10:31.041912+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:10:57.477266+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.82:0/2005424509":"2024-05-21T16:04:45.161949+0000","172.21.15.82:6801/1763049308":"2024-05-21T16:04:45.161949+0000","172.21.15.82:6801/1959459348":"2024-05-21T16:04:19.208291+0000","172.21.15.82:0/3131904556":"2024-05-21T16:04:19.208291+0000","172.21.15.82:6800/1959459348":"2024-05-21T16:04:19.208291+0000","172.21.15.82:0/2456966342":"2024-05-21T16:04:45.161949+0000","172.21.15.82:0/4179978833":"2024-05-21T16:04:19.208291+0000","172.21.15.82:0/1602862225":"2024-05-21T16:04:45.161949+0000","172.21.15.82:0/106101180":"2024-05-21T16:04:19.208291+0000","172.21.15.82:6800/1763049308":"2024-05-21T16:04:45.161949+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-20T16:12:09.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:09.447+0000 7f3ebbf37640 1 -- 172.21.15.82:0/1629290717 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f3ea00a2b90 msgr2=0x7f3ea00a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:12:09.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:09.447+0000 7f3ebbf37640 1 --2- 172.21.15.82:0/1629290717 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f3ea00a2b90 0x7f3ea00a50a0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f3eb4069860 tx=0x7f3eb0005e50 comp rx=0 tx=0).stop 2024-05-20T16:12:09.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:09.447+0000 7f3ebbf37640 1 -- 172.21.15.82:0/1629290717 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3eb4105ec0 msgr2=0x7f3eb4199810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:12:09.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:09.447+0000 7f3ebbf37640 1 --2- 172.21.15.82:0/1629290717 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3eb4105ec0 0x7f3eb4199810 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f3ea4006010 tx=0x7f3ea4004360 comp rx=0 tx=0).stop 2024-05-20T16:12:09.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:09.447+0000 7f3ebbf37640 1 -- 172.21.15.82:0/1629290717 shutdown_connections 2024-05-20T16:12:09.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:09.447+0000 7f3ebbf37640 1 --2- 172.21.15.82:0/1629290717 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f3ea00a2b90 0x7f3ea00a50a0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:09.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:09.447+0000 7f3ebbf37640 1 --2- 172.21.15.82:0/1629290717 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3eb4105ec0 0x7f3eb4199810 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:09.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:09.447+0000 7f3ebbf37640 1 --2- 172.21.15.82:0/1629290717 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3eb4108820 0x7f3eb41a07a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:09.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:09.447+0000 7f3ebbf37640 1 --2- 172.21.15.82:0/1629290717 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3eb4103560 0x7f3eb41992d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:09.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:09.447+0000 7f3ebbf37640 1 -- 172.21.15.82:0/1629290717 >> 172.21.15.82:0/1629290717 conn(0x7f3eb40fcb80 msgr2=0x7f3eb4104050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:12:09.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:09.447+0000 7f3ebbf37640 1 -- 172.21.15.82:0/1629290717 shutdown_connections 2024-05-20T16:12:09.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:09.447+0000 7f3ebbf37640 1 -- 172.21.15.82:0/1629290717 wait complete. 2024-05-20T16:12:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:12:09 smithi082 ceph-mon[32238]: pgmap v303: 129 pgs: 129 active+clean; 581 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-05-20T16:12:09.459 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-20T16:12:09.572 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:09.570+0000 7f5ae8af7640 1 -- 172.21.15.82:0/717294071 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5ae4101200 msgr2=0x7f5ae410ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:12:09.572 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:09.570+0000 7f5ae8af7640 1 --2- 172.21.15.82:0/717294071 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5ae4101200 0x7f5ae410ac60 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f5adc009ca0 tx=0x7f5adc02a8c0 comp rx=0 tx=0).stop 2024-05-20T16:12:09.573 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:09.570+0000 7f5ae8af7640 1 -- 172.21.15.82:0/717294071 shutdown_connections 2024-05-20T16:12:09.573 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:09.570+0000 7f5ae8af7640 1 --2- 172.21.15.82:0/717294071 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5ae4101200 0x7f5ae410ac60 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:09.573 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:09.570+0000 7f5ae8af7640 1 --2- 172.21.15.82:0/717294071 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5ae4100700 0x7f5ae4100b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:09.573 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:09.570+0000 7f5ae8af7640 1 --2- 172.21.15.82:0/717294071 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5ae4104050 0x7f5ae4104460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:09.573 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:09.570+0000 7f5ae8af7640 1 -- 172.21.15.82:0/717294071 >> 172.21.15.82:0/717294071 conn(0x7f5ae40fc320 msgr2=0x7f5ae40fe7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:12:09.573 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:09.570+0000 7f5ae8af7640 1 -- 172.21.15.82:0/717294071 shutdown_connections 2024-05-20T16:12:09.573 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:09.570+0000 7f5ae8af7640 1 -- 172.21.15.82:0/717294071 wait complete. 2024-05-20T16:12:09.573 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:09.571+0000 7f5ae8af7640 1 Processor -- start 2024-05-20T16:12:09.573 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:09.571+0000 7f5ae8af7640 1 -- start start 2024-05-20T16:12:09.574 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:09.571+0000 7f5ae8af7640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5ae4100700 0x7f5ae4199220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:12:09.574 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:09.571+0000 7f5ae8af7640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5ae4101200 0x7f5ae4199760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:12:09.574 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:09.571+0000 7f5ae8af7640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5ae4104050 0x7f5ae41a06f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:12:09.574 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:09.571+0000 7f5ae8af7640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f5ae410e710 con 0x7f5ae4100700 2024-05-20T16:12:09.574 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:09.571+0000 7f5ae8af7640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f5ae410e590 con 0x7f5ae4101200 2024-05-20T16:12:09.574 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:09.571+0000 7f5ae8af7640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f5ae410e890 con 0x7f5ae4104050 2024-05-20T16:12:09.574 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:09.571+0000 7f5ae259c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5ae4100700 0x7f5ae4199220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:12:09.574 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:09.571+0000 7f5ae259c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5ae4100700 0x7f5ae4199220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:47052/0 (socket says 172.21.15.82:47052) 2024-05-20T16:12:09.574 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:09.571+0000 7f5ae259c640 1 -- 172.21.15.82:0/2250707894 learned_addr learned my addr 172.21.15.82:0/2250707894 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:12:09.574 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:09.572+0000 7f5ae1d9b640 1 --2- 172.21.15.82:0/2250707894 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5ae4101200 0x7f5ae4199760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:12:09.574 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:09.572+0000 7f5ae259c640 1 -- 172.21.15.82:0/2250707894 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5ae4104050 msgr2=0x7f5ae41a06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:12:09.574 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:09.572+0000 7f5ae2d9d640 1 --2- 172.21.15.82:0/2250707894 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5ae4104050 0x7f5ae41a06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:12:09.574 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:09.572+0000 7f5ae259c640 1 --2- 172.21.15.82:0/2250707894 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5ae4104050 0x7f5ae41a06f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:09.575 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:09.572+0000 7f5ae259c640 1 -- 172.21.15.82:0/2250707894 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5ae4101200 msgr2=0x7f5ae4199760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:12:09.575 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:09.572+0000 7f5ae259c640 1 --2- 172.21.15.82:0/2250707894 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5ae4101200 0x7f5ae4199760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:09.575 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:09.572+0000 7f5ae259c640 1 -- 172.21.15.82:0/2250707894 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ae41a0cb0 con 0x7f5ae4100700 2024-05-20T16:12:09.575 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:09.572+0000 7f5ae1d9b640 1 --2- 172.21.15.82:0/2250707894 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5ae4101200 0x7f5ae4199760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T16:12:09.575 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:09.572+0000 7f5ae259c640 1 --2- 172.21.15.82:0/2250707894 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5ae4100700 0x7f5ae4199220 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f5acc00c970 tx=0x7f5acc00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:12:09.575 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:09.572+0000 7f5adb7fe640 1 -- 172.21.15.82:0/2250707894 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5acc007c70 con 0x7f5ae4100700 2024-05-20T16:12:09.575 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:09.572+0000 7f5adb7fe640 1 -- 172.21.15.82:0/2250707894 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5acc007e10 con 0x7f5ae4100700 2024-05-20T16:12:09.575 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:09.572+0000 7f5ae8af7640 1 -- 172.21.15.82:0/2250707894 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ae41a0fa0 con 0x7f5ae4100700 2024-05-20T16:12:09.575 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:09.572+0000 7f5adb7fe640 1 -- 172.21.15.82:0/2250707894 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5acc0188c0 con 0x7f5ae4100700 2024-05-20T16:12:09.575 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:09.572+0000 7f5ae8af7640 1 -- 172.21.15.82:0/2250707894 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ae41a1400 con 0x7f5ae4100700 2024-05-20T16:12:09.576 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:09.573+0000 7f5ae8af7640 1 -- 172.21.15.82:0/2250707894 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ae4069170 con 0x7f5ae4100700 2024-05-20T16:12:09.576 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:09.575+0000 7f5adb7fe640 1 -- 172.21.15.82:0/2250707894 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f5acc00f450 con 0x7f5ae4100700 2024-05-20T16:12:09.576 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:09.575+0000 7f5adb7fe640 1 --2- 172.21.15.82:0/2250707894 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f5ad00a2b90 0x7f5ad00a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:12:09.576 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:09.575+0000 7f5ae1d9b640 1 --2- 172.21.15.82:0/2250707894 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f5ad00a2b90 0x7f5ad00a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:12:09.577 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:09.575+0000 7f5adb7fe640 1 -- 172.21.15.82:0/2250707894 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8802+0+0 (secure 0 0 0) 0x7f5acc0cf590 con 0x7f5ae4100700 2024-05-20T16:12:09.577 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:09.576+0000 7f5ae1d9b640 1 --2- 172.21.15.82:0/2250707894 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f5ad00a2b90 0x7f5ad00a50a0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f5ad4005fd0 tx=0x7f5ad4005ec0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:12:09.578 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:09.579+0000 7f5adb7fe640 1 -- 172.21.15.82:0/2250707894 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f5ae4069170 con 0x7f5ae4100700 2024-05-20T16:12:09.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:12:09 smithi184 ceph-mon[36209]: pgmap v303: 129 pgs: 129 active+clean; 581 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-05-20T16:12:09.755 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:09.756+0000 7f5ae8af7640 1 -- 172.21.15.82:0/2250707894 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5ae402d380 con 0x7f5ae4100700 2024-05-20T16:12:09.757 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:09.757+0000 7f5adb7fe640 1 -- 172.21.15.82:0/2250707894 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+24153 (secure 0 0 0) 0x7f5acc014030 con 0x7f5ae4100700 2024-05-20T16:12:09.758 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-05-20T16:12:09.758 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":91,"fsid":"59d1600a-16c2-11ef-bc9a-c7b262605968","created":"2024-05-20T16:03:46.611556+0000","modified":"2024-05-20T16:11:53.416085+0000","last_up_change":"2024-05-20T16:10:59.303479+0000","last_in_change":"2024-05-20T16:10:39.063561+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-20T16:07:09.493727+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-20T16:11:45.058035+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-20T16:11:45.943862+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-20T16:11:49.637579+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"bae842a9-03ec-490e-b56d-f7957235cdc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":1484330167},{"type":"v1","addr":"172.21.15.82:6803","nonce":1484330167}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":1484330167},{"type":"v1","addr":"172.21.15.82:6805","nonce":1484330167}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1484330167},{"type":"v1","addr":"172.21.15.82:6809","nonce":1484330167}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":1484330167},{"type":"v1","addr":"172.21.15.82:6807","nonce":1484330167}]},"public_addr":"172.21.15.82:6803/1484330167","cluster_addr":"172.21.15.82:6805/1484330167","heartbeat_back_addr":"172.21.15.82:6809/1484330167","heartbeat_front_addr":"172.21.15.82:6807/1484330167","state":["exists","up"]},{"osd":1,"uuid":"59f6a2ba-b5de-4161-81c5-92a417abacdf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":1457003568},{"type":"v1","addr":"172.21.15.160:6801","nonce":1457003568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":1457003568},{"type":"v1","addr":"172.21.15.160:6803","nonce":1457003568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6806","nonce":1457003568},{"type":"v1","addr":"172.21.15.160:6807","nonce":1457003568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":1457003568},{"type":"v1","addr":"172.21.15.160:6805","nonce":1457003568}]},"public_addr":"172.21.15.160:6801/1457003568","cluster_addr":"172.21.15.160:6803/1457003568","heartbeat_back_addr":"172.21.15.160:6807/1457003568","heartbeat_front_addr":"172.21.15.160:6805/1457003568","state":["exists","up"]},{"osd":2,"uuid":"3bbb5587-b449-41f3-bd26-67ec3d6afdfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":4051293934},{"type":"v1","addr":"172.21.15.184:6801","nonce":4051293934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":4051293934},{"type":"v1","addr":"172.21.15.184:6803","nonce":4051293934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":4051293934},{"type":"v1","addr":"172.21.15.184:6807","nonce":4051293934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":4051293934},{"type":"v1","addr":"172.21.15.184:6805","nonce":4051293934}]},"public_addr":"172.21.15.184:6801/4051293934","cluster_addr":"172.21.15.184:6803/4051293934","heartbeat_back_addr":"172.21.15.184:6807/4051293934","heartbeat_front_addr":"172.21.15.184:6805/4051293934","state":["exists","up"]},{"osd":3,"uuid":"ca651f90-09b8-4081-a6f9-40b31268a837","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":3947485599},{"type":"v1","addr":"172.21.15.82:6811","nonce":3947485599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":3947485599},{"type":"v1","addr":"172.21.15.82:6813","nonce":3947485599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":3947485599},{"type":"v1","addr":"172.21.15.82:6817","nonce":3947485599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":3947485599},{"type":"v1","addr":"172.21.15.82:6815","nonce":3947485599}]},"public_addr":"172.21.15.82:6811/3947485599","cluster_addr":"172.21.15.82:6813/3947485599","heartbeat_back_addr":"172.21.15.82:6817/3947485599","heartbeat_front_addr":"172.21.15.82:6815/3947485599","state":["exists","up"]},{"osd":4,"uuid":"cb3dfd56-f677-40cb-9c96-8319ea5ac788","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":4160830014},{"type":"v1","addr":"172.21.15.160:6809","nonce":4160830014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":4160830014},{"type":"v1","addr":"172.21.15.160:6811","nonce":4160830014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":4160830014},{"type":"v1","addr":"172.21.15.160:6815","nonce":4160830014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6812","nonce":4160830014},{"type":"v1","addr":"172.21.15.160:6813","nonce":4160830014}]},"public_addr":"172.21.15.160:6809/4160830014","cluster_addr":"172.21.15.160:6811/4160830014","heartbeat_back_addr":"172.21.15.160:6815/4160830014","heartbeat_front_addr":"172.21.15.160:6813/4160830014","state":["exists","up"]},{"osd":5,"uuid":"c70cfcd1-a4ff-461f-8c73-479b061200fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":1046282672},{"type":"v1","addr":"172.21.15.184:6809","nonce":1046282672}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":1046282672},{"type":"v1","addr":"172.21.15.184:6811","nonce":1046282672}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":1046282672},{"type":"v1","addr":"172.21.15.184:6815","nonce":1046282672}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":1046282672},{"type":"v1","addr":"172.21.15.184:6813","nonce":1046282672}]},"public_addr":"172.21.15.184:6809/1046282672","cluster_addr":"172.21.15.184:6811/1046282672","heartbeat_back_addr":"172.21.15.184:6815/1046282672","heartbeat_front_addr":"172.21.15.184:6813/1046282672","state":["exists","up"]},{"osd":6,"uuid":"d47ce001-9dd1-4830-bcc9-888601187c6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":2789736499},{"type":"v1","addr":"172.21.15.82:6819","nonce":2789736499}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":2789736499},{"type":"v1","addr":"172.21.15.82:6821","nonce":2789736499}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":2789736499},{"type":"v1","addr":"172.21.15.82:6825","nonce":2789736499}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":2789736499},{"type":"v1","addr":"172.21.15.82:6823","nonce":2789736499}]},"public_addr":"172.21.15.82:6819/2789736499","cluster_addr":"172.21.15.82:6821/2789736499","heartbeat_back_addr":"172.21.15.82:6825/2789736499","heartbeat_front_addr":"172.21.15.82:6823/2789736499","state":["exists","up"]},{"osd":7,"uuid":"25c54f43-18cb-41e4-83b4-2732b5a93889","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6816","nonce":2340683378},{"type":"v1","addr":"172.21.15.160:6817","nonce":2340683378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":2340683378},{"type":"v1","addr":"172.21.15.160:6819","nonce":2340683378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":2340683378},{"type":"v1","addr":"172.21.15.160:6823","nonce":2340683378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6820","nonce":2340683378},{"type":"v1","addr":"172.21.15.160:6821","nonce":2340683378}]},"public_addr":"172.21.15.160:6817/2340683378","cluster_addr":"172.21.15.160:6819/2340683378","heartbeat_back_addr":"172.21.15.160:6823/2340683378","heartbeat_front_addr":"172.21.15.160:6821/2340683378","state":["exists","up"]},{"osd":8,"uuid":"1f8b478d-287f-45f5-8526-d1a58c5bc07c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":3831576372},{"type":"v1","addr":"172.21.15.184:6817","nonce":3831576372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":3831576372},{"type":"v1","addr":"172.21.15.184:6819","nonce":3831576372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":3831576372},{"type":"v1","addr":"172.21.15.184:6823","nonce":3831576372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":3831576372},{"type":"v1","addr":"172.21.15.184:6821","nonce":3831576372}]},"public_addr":"172.21.15.184:6817/3831576372","cluster_addr":"172.21.15.184:6819/3831576372","heartbeat_back_addr":"172.21.15.184:6823/3831576372","heartbeat_front_addr":"172.21.15.184:6821/3831576372","state":["exists","up"]},{"osd":9,"uuid":"0d69e319-954f-4b18-95ea-31794f18dc3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":716330782},{"type":"v1","addr":"172.21.15.82:6827","nonce":716330782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":716330782},{"type":"v1","addr":"172.21.15.82:6829","nonce":716330782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":716330782},{"type":"v1","addr":"172.21.15.82:6833","nonce":716330782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":716330782},{"type":"v1","addr":"172.21.15.82:6831","nonce":716330782}]},"public_addr":"172.21.15.82:6827/716330782","cluster_addr":"172.21.15.82:6829/716330782","heartbeat_back_addr":"172.21.15.82:6833/716330782","heartbeat_front_addr":"172.21.15.82:6831/716330782","state":["exists","up"]},{"osd":10,"uuid":"39dc1b78-61cb-42ca-b5b6-1e44ac2b6500","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":2327428014},{"type":"v1","addr":"172.21.15.160:6825","nonce":2327428014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":2327428014},{"type":"v1","addr":"172.21.15.160:6827","nonce":2327428014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":2327428014},{"type":"v1","addr":"172.21.15.160:6831","nonce":2327428014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":2327428014},{"type":"v1","addr":"172.21.15.160:6829","nonce":2327428014}]},"public_addr":"172.21.15.160:6825/2327428014","cluster_addr":"172.21.15.160:6827/2327428014","heartbeat_back_addr":"172.21.15.160:6831/2327428014","heartbeat_front_addr":"172.21.15.160:6829/2327428014","state":["exists","up"]},{"osd":11,"uuid":"4306b3c1-707a-4010-987c-209ea0e948e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":3594364650},{"type":"v1","addr":"172.21.15.184:6825","nonce":3594364650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":3594364650},{"type":"v1","addr":"172.21.15.184:6827","nonce":3594364650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3594364650},{"type":"v1","addr":"172.21.15.184:6831","nonce":3594364650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3594364650},{"type":"v1","addr":"172.21.15.184:6829","nonce":3594364650}]},"public_addr":"172.21.15.184:6825/3594364650","cluster_addr":"172.21.15.184:6827/3594364650","heartbeat_back_addr":"172.21.15.184:6831/3594364650","heartbeat_front_addr":"172.21.15.184:6829/3594364650","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:06:17.855998+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:06:41.473270+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:07:05.304108+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:07:29.887472+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:07:55.387286+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:08:20.579669+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:08:46.333051+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:09:12.289668+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:09:38.180696+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:10:04.471459+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:10:31.041912+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:10:57.477266+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.82:0/2005424509":"2024-05-21T16:04:45.161949+0000","172.21.15.82:6801/1763049308":"2024-05-21T16:04:45.161949+0000","172.21.15.82:6801/1959459348":"2024-05-21T16:04:19.208291+0000","172.21.15.82:0/3131904556":"2024-05-21T16:04:19.208291+0000","172.21.15.82:6800/1959459348":"2024-05-21T16:04:19.208291+0000","172.21.15.82:0/2456966342":"2024-05-21T16:04:45.161949+0000","172.21.15.82:0/4179978833":"2024-05-21T16:04:19.208291+0000","172.21.15.82:0/1602862225":"2024-05-21T16:04:45.161949+0000","172.21.15.82:0/106101180":"2024-05-21T16:04:19.208291+0000","172.21.15.82:6800/1763049308":"2024-05-21T16:04:45.161949+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-20T16:12:09.760 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:09.759+0000 7f5ae8af7640 1 -- 172.21.15.82:0/2250707894 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f5ad00a2b90 msgr2=0x7f5ad00a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:12:09.760 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:09.759+0000 7f5ae8af7640 1 --2- 172.21.15.82:0/2250707894 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f5ad00a2b90 0x7f5ad00a50a0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f5ad4005fd0 tx=0x7f5ad4005ec0 comp rx=0 tx=0).stop 2024-05-20T16:12:09.760 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:09.759+0000 7f5ae8af7640 1 -- 172.21.15.82:0/2250707894 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5ae4100700 msgr2=0x7f5ae4199220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:12:09.760 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:09.759+0000 7f5ae8af7640 1 --2- 172.21.15.82:0/2250707894 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5ae4100700 0x7f5ae4199220 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f5acc00c970 tx=0x7f5acc00ce30 comp rx=0 tx=0).stop 2024-05-20T16:12:09.760 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:09.759+0000 7f5ae8af7640 1 -- 172.21.15.82:0/2250707894 shutdown_connections 2024-05-20T16:12:09.760 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:09.759+0000 7f5ae8af7640 1 --2- 172.21.15.82:0/2250707894 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f5ad00a2b90 0x7f5ad00a50a0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:09.760 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:09.759+0000 7f5ae8af7640 1 --2- 172.21.15.82:0/2250707894 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5ae4104050 0x7f5ae41a06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:09.760 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:09.759+0000 7f5ae8af7640 1 --2- 172.21.15.82:0/2250707894 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5ae4101200 0x7f5ae4199760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:09.760 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:09.759+0000 7f5ae8af7640 1 --2- 172.21.15.82:0/2250707894 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5ae4100700 0x7f5ae4199220 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:09.760 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:09.759+0000 7f5ae8af7640 1 -- 172.21.15.82:0/2250707894 >> 172.21.15.82:0/2250707894 conn(0x7f5ae40fc320 msgr2=0x7f5ae4106bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:12:09.760 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:09.760+0000 7f5ae8af7640 1 -- 172.21.15.82:0/2250707894 shutdown_connections 2024-05-20T16:12:09.760 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:09.760+0000 7f5ae8af7640 1 -- 172.21.15.82:0/2250707894 wait complete. 2024-05-20T16:12:09.771 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-05-20T16:12:09.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:12:09 smithi160 ceph-mon[36347]: pgmap v303: 129 pgs: 129 active+clean; 581 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-05-20T16:12:09.887 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:09.885+0000 7f62777b5640 1 -- 172.21.15.82:0/1939122354 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f62701078d0 msgr2=0x7f6270109cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:12:09.887 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:09.885+0000 7f62777b5640 1 --2- 172.21.15.82:0/1939122354 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f62701078d0 0x7f6270109cf0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f6268009900 tx=0x7f626802a410 comp rx=0 tx=0).stop 2024-05-20T16:12:09.887 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:09.885+0000 7f62777b5640 1 -- 172.21.15.82:0/1939122354 shutdown_connections 2024-05-20T16:12:09.887 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:09.885+0000 7f62777b5640 1 --2- 172.21.15.82:0/1939122354 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f62701078d0 0x7f6270109cf0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:09.887 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:09.885+0000 7f62777b5640 1 --2- 172.21.15.82:0/1939122354 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6270102bd0 0x7f6270102fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:09.887 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:09.885+0000 7f62777b5640 1 --2- 172.21.15.82:0/1939122354 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6270101990 0x7f6270102690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:09.887 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:09.885+0000 7f62777b5640 1 -- 172.21.15.82:0/1939122354 >> 172.21.15.82:0/1939122354 conn(0x7f62700fcb80 msgr2=0x7f62700ff030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:12:09.888 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:09.885+0000 7f62777b5640 1 -- 172.21.15.82:0/1939122354 shutdown_connections 2024-05-20T16:12:09.888 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:09.885+0000 7f62777b5640 1 -- 172.21.15.82:0/1939122354 wait complete. 2024-05-20T16:12:09.888 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:09.885+0000 7f62777b5640 1 Processor -- start 2024-05-20T16:12:09.888 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:09.886+0000 7f62777b5640 1 -- start start 2024-05-20T16:12:09.888 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:09.886+0000 7f62777b5640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6270101990 0x7f6270198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:12:09.888 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:09.886+0000 7f62777b5640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6270102bd0 0x7f6270198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:12:09.888 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:09.886+0000 7f62777b5640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f62701078d0 0x7f627019f900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:12:09.888 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:09.886+0000 7f62777b5640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f627010d5f0 con 0x7f6270102bd0 2024-05-20T16:12:09.888 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:09.886+0000 7f62777b5640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f627010d470 con 0x7f6270101990 2024-05-20T16:12:09.888 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:09.886+0000 7f62777b5640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f627010d770 con 0x7f62701078d0 2024-05-20T16:12:09.888 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:09.886+0000 7f6274d50640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6270102bd0 0x7f6270198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:12:09.889 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:09.886+0000 7f6274d50640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6270102bd0 0x7f6270198970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:47066/0 (socket says 172.21.15.82:47066) 2024-05-20T16:12:09.889 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:09.886+0000 7f6274d50640 1 -- 172.21.15.82:0/3617055753 learned_addr learned my addr 172.21.15.82:0/3617055753 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:12:09.889 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:09.886+0000 7f6275d52640 1 --2- 172.21.15.82:0/3617055753 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f62701078d0 0x7f627019f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:12:09.889 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:09.886+0000 7f6275551640 1 --2- 172.21.15.82:0/3617055753 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6270101990 0x7f6270198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:12:09.889 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:09.886+0000 7f6274d50640 1 -- 172.21.15.82:0/3617055753 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f62701078d0 msgr2=0x7f627019f900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:12:09.889 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:09.886+0000 7f6274d50640 1 --2- 172.21.15.82:0/3617055753 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f62701078d0 0x7f627019f900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:09.889 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:09.886+0000 7f6274d50640 1 -- 172.21.15.82:0/3617055753 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6270101990 msgr2=0x7f6270198430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:12:09.889 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:09.886+0000 7f6274d50640 1 --2- 172.21.15.82:0/3617055753 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6270101990 0x7f6270198430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:09.889 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:09.886+0000 7f6274d50640 1 -- 172.21.15.82:0/3617055753 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f627019ff70 con 0x7f6270102bd0 2024-05-20T16:12:09.889 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:09.886+0000 7f6274d50640 1 --2- 172.21.15.82:0/3617055753 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6270102bd0 0x7f6270198970 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f626400d950 tx=0x7f626400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:12:09.889 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:09.887+0000 7f626e7fc640 1 -- 172.21.15.82:0/3617055753 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6264017070 con 0x7f6270102bd0 2024-05-20T16:12:09.889 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:09.887+0000 7f62777b5640 1 -- 172.21.15.82:0/3617055753 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62701a0260 con 0x7f6270102bd0 2024-05-20T16:12:09.889 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:09.887+0000 7f626e7fc640 1 -- 172.21.15.82:0/3617055753 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6264009c30 con 0x7f6270102bd0 2024-05-20T16:12:09.890 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:09.887+0000 7f626e7fc640 1 -- 172.21.15.82:0/3617055753 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f626400f6b0 con 0x7f6270102bd0 2024-05-20T16:12:09.890 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:09.887+0000 7f62777b5640 1 -- 172.21.15.82:0/3617055753 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62701a07b0 con 0x7f6270102bd0 2024-05-20T16:12:09.891 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:09.888+0000 7f62777b5640 1 -- 172.21.15.82:0/3617055753 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62701066e0 con 0x7f6270102bd0 2024-05-20T16:12:09.891 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:09.889+0000 7f626e7fc640 1 -- 172.21.15.82:0/3617055753 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f62640109d0 con 0x7f6270102bd0 2024-05-20T16:12:09.891 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:09.889+0000 7f626e7fc640 1 --2- 172.21.15.82:0/3617055753 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f62580a2b90 0x7f62580a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:12:09.891 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:09.890+0000 7f6275551640 1 --2- 172.21.15.82:0/3617055753 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f62580a2b90 0x7f62580a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:12:09.891 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:09.890+0000 7f626e7fc640 1 -- 172.21.15.82:0/3617055753 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8802+0+0 (secure 0 0 0) 0x7f62640cee30 con 0x7f6270102bd0 2024-05-20T16:12:09.891 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:09.890+0000 7f6275551640 1 --2- 172.21.15.82:0/3617055753 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f62580a2b90 0x7f62580a50a0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f6270102590 tx=0x7f62600098f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:12:09.893 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:09.893+0000 7f626e7fc640 1 -- 172.21.15.82:0/3617055753 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f6264009de0 con 0x7f6270102bd0 2024-05-20T16:12:10.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:10.108+0000 7f62777b5640 1 -- 172.21.15.82:0/3617055753 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f627006b250 con 0x7f6270102bd0 2024-05-20T16:12:10.358 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:10.358+0000 7f626e7fc640 1 -- 172.21.15.82:0/3617055753 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v18) ==== 118+0+0 (secure 0 0 0) 0x7f626409bb50 con 0x7f6270102bd0 2024-05-20T16:12:10.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:10.361+0000 7f62777b5640 1 -- 172.21.15.82:0/3617055753 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f62580a2b90 msgr2=0x7f62580a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:12:10.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:10.361+0000 7f62777b5640 1 --2- 172.21.15.82:0/3617055753 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f62580a2b90 0x7f62580a50a0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f6270102590 tx=0x7f62600098f0 comp rx=0 tx=0).stop 2024-05-20T16:12:10.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:10.361+0000 7f62777b5640 1 -- 172.21.15.82:0/3617055753 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6270102bd0 msgr2=0x7f6270198970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:12:10.362 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:10.361+0000 7f62777b5640 1 --2- 172.21.15.82:0/3617055753 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6270102bd0 0x7f6270198970 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f626400d950 tx=0x7f626400de10 comp rx=0 tx=0).stop 2024-05-20T16:12:10.362 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:10.361+0000 7f62777b5640 1 -- 172.21.15.82:0/3617055753 shutdown_connections 2024-05-20T16:12:10.362 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:10.361+0000 7f62777b5640 1 --2- 172.21.15.82:0/3617055753 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f62580a2b90 0x7f62580a50a0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:10.362 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:10.361+0000 7f62777b5640 1 --2- 172.21.15.82:0/3617055753 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6270102bd0 0x7f6270198970 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:10.362 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:10.361+0000 7f62777b5640 1 --2- 172.21.15.82:0/3617055753 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f62701078d0 0x7f627019f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:10.362 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:10.361+0000 7f62777b5640 1 --2- 172.21.15.82:0/3617055753 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6270101990 0x7f6270198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:10.362 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:10.361+0000 7f62777b5640 1 -- 172.21.15.82:0/3617055753 >> 172.21.15.82:0/3617055753 conn(0x7f62700fcb80 msgr2=0x7f62701038b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:12:10.362 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:10.361+0000 7f62777b5640 1 -- 172.21.15.82:0/3617055753 shutdown_connections 2024-05-20T16:12:10.362 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:10.361+0000 7f62777b5640 1 -- 172.21.15.82:0/3617055753 wait complete. 2024-05-20T16:12:10.373 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 3, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated'}} 2024-05-20T16:12:10.374 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 3 2024-05-20T16:12:10.490 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:10.488+0000 7f0e1812a640 1 -- 172.21.15.82:0/2833480492 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0e10103cd0 msgr2=0x7f0e101060f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:12:10.490 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:10.488+0000 7f0e1812a640 1 --2- 172.21.15.82:0/2833480492 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0e10103cd0 0x7f0e101060f0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f0e04009900 tx=0x7f0e0402a410 comp rx=0 tx=0).stop 2024-05-20T16:12:10.490 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:10.488+0000 7f0e1812a640 1 -- 172.21.15.82:0/2833480492 shutdown_connections 2024-05-20T16:12:10.491 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:10.488+0000 7f0e1812a640 1 --2- 172.21.15.82:0/2833480492 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0e10103cd0 0x7f0e101060f0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:10.491 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:10.488+0000 7f0e1812a640 1 --2- 172.21.15.82:0/2833480492 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0e10106630 0x7f0e10106a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:10.491 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:10.488+0000 7f0e1812a640 1 --2- 172.21.15.82:0/2833480492 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0e10101370 0x7f0e10103790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:10.491 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:10.488+0000 7f0e1812a640 1 -- 172.21.15.82:0/2833480492 >> 172.21.15.82:0/2833480492 conn(0x7f0e100fcb80 msgr2=0x7f0e100ff030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:12:10.491 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:10.488+0000 7f0e1812a640 1 -- 172.21.15.82:0/2833480492 shutdown_connections 2024-05-20T16:12:10.491 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:10.488+0000 7f0e1812a640 1 -- 172.21.15.82:0/2833480492 wait complete. 2024-05-20T16:12:10.491 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:10.489+0000 7f0e1812a640 1 Processor -- start 2024-05-20T16:12:10.491 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:10.489+0000 7f0e1812a640 1 -- start start 2024-05-20T16:12:10.491 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:10.489+0000 7f0e1812a640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0e10101370 0x7f0e1019fa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:12:10.491 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:10.489+0000 7f0e1812a640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0e10103cd0 0x7f0e1019ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:12:10.491 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:10.489+0000 7f0e1812a640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0e10106630 0x7f0e101a6f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:12:10.491 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:10.489+0000 7f0e1812a640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f0e10114e00 con 0x7f0e10101370 2024-05-20T16:12:10.491 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:10.489+0000 7f0e1812a640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f0e10114c80 con 0x7f0e10103cd0 2024-05-20T16:12:10.491 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:10.489+0000 7f0e1812a640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f0e10114f80 con 0x7f0e10106630 2024-05-20T16:12:10.492 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:10.489+0000 7f0e15ec6640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0e10101370 0x7f0e1019fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:12:10.492 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:10.489+0000 7f0e15ec6640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0e10101370 0x7f0e1019fa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:47094/0 (socket says 172.21.15.82:47094) 2024-05-20T16:12:10.492 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:10.489+0000 7f0e15ec6640 1 -- 172.21.15.82:0/3917408108 learned_addr learned my addr 172.21.15.82:0/3917408108 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:12:10.492 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:10.489+0000 7f0e156c5640 1 --2- 172.21.15.82:0/3917408108 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0e10103cd0 0x7f0e1019ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:12:10.492 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:10.489+0000 7f0e166c7640 1 --2- 172.21.15.82:0/3917408108 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0e10106630 0x7f0e101a6f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:12:10.492 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:10.490+0000 7f0e15ec6640 1 -- 172.21.15.82:0/3917408108 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0e10106630 msgr2=0x7f0e101a6f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:12:10.492 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:10.490+0000 7f0e15ec6640 1 --2- 172.21.15.82:0/3917408108 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0e10106630 0x7f0e101a6f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:10.492 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:10.490+0000 7f0e15ec6640 1 -- 172.21.15.82:0/3917408108 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0e10103cd0 msgr2=0x7f0e1019ffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:12:10.492 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:10.490+0000 7f0e15ec6640 1 --2- 172.21.15.82:0/3917408108 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0e10103cd0 0x7f0e1019ffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:10.492 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:10.490+0000 7f0e15ec6640 1 -- 172.21.15.82:0/3917408108 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e101a75d0 con 0x7f0e10101370 2024-05-20T16:12:10.492 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:10.490+0000 7f0e166c7640 1 --2- 172.21.15.82:0/3917408108 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0e10106630 0x7f0e101a6f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T16:12:10.492 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:10.490+0000 7f0e15ec6640 1 --2- 172.21.15.82:0/3917408108 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0e10101370 0x7f0e1019fa90 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f0e0c00b450 tx=0x7f0e0c00b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:12:10.492 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:10.490+0000 7f0e02ffd640 1 -- 172.21.15.82:0/3917408108 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e0c015020 con 0x7f0e10101370 2024-05-20T16:12:10.493 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:10.490+0000 7f0e02ffd640 1 -- 172.21.15.82:0/3917408108 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0e0c011d70 con 0x7f0e10101370 2024-05-20T16:12:10.493 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:10.490+0000 7f0e1812a640 1 -- 172.21.15.82:0/3917408108 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e101a78c0 con 0x7f0e10101370 2024-05-20T16:12:10.493 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:10.490+0000 7f0e02ffd640 1 -- 172.21.15.82:0/3917408108 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e0c00f340 con 0x7f0e10101370 2024-05-20T16:12:10.493 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:10.490+0000 7f0e1812a640 1 -- 172.21.15.82:0/3917408108 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e101a7e40 con 0x7f0e10101370 2024-05-20T16:12:10.494 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:10.491+0000 7f0e1812a640 1 -- 172.21.15.82:0/3917408108 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e100fe6a0 con 0x7f0e10101370 2024-05-20T16:12:10.494 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:10.492+0000 7f0e02ffd640 1 -- 172.21.15.82:0/3917408108 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f0e0c018020 con 0x7f0e10101370 2024-05-20T16:12:10.494 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:10.493+0000 7f0e02ffd640 1 --2- 172.21.15.82:0/3917408108 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f0dfc0a2b90 0x7f0dfc0a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:12:10.494 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:10.493+0000 7f0e156c5640 1 --2- 172.21.15.82:0/3917408108 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f0dfc0a2b90 0x7f0dfc0a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:12:10.494 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:10.493+0000 7f0e02ffd640 1 -- 172.21.15.82:0/3917408108 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8802+0+0 (secure 0 0 0) 0x7f0e0c0cee10 con 0x7f0e10101370 2024-05-20T16:12:10.494 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:10.493+0000 7f0e156c5640 1 --2- 172.21.15.82:0/3917408108 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f0dfc0a2b90 0x7f0dfc0a50a0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f0e04000c00 tx=0x7f0e04031510 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:12:10.497 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:10.497+0000 7f0e02ffd640 1 -- 172.21.15.82:0/3917408108 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f0e0c01f020 con 0x7f0e10101370 2024-05-20T16:12:10.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:12:10 smithi082 ceph-mon[32238]: from='client.? 172.21.15.82:0/1629290717' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-20T16:12:10.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:12:10 smithi082 ceph-mon[32238]: from='client.? 172.21.15.82:0/2250707894' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-20T16:12:10.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:12:10 smithi082 ceph-mon[32238]: from='client.? 172.21.15.82:0/3617055753' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-05-20T16:12:10.635 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:12:10 smithi184 ceph-mon[36209]: from='client.? 172.21.15.82:0/1629290717' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-20T16:12:10.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:12:10 smithi184 ceph-mon[36209]: from='client.? 172.21.15.82:0/2250707894' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-20T16:12:10.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:12:10 smithi184 ceph-mon[36209]: from='client.? 172.21.15.82:0/3617055753' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-05-20T16:12:10.709 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:10.710+0000 7f0e1812a640 1 -- 172.21.15.82:0/3917408108 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) -- 0x7f0e10062750 con 0x7f0e10101370 2024-05-20T16:12:10.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:12:10 smithi160 ceph-mon[36347]: from='client.? 172.21.15.82:0/1629290717' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-20T16:12:10.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:12:10 smithi160 ceph-mon[36347]: from='client.? 172.21.15.82:0/2250707894' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-20T16:12:10.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:12:10 smithi160 ceph-mon[36347]: from='client.? 172.21.15.82:0/3617055753' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-05-20T16:12:11.366 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:11.366+0000 7f0e02ffd640 1 -- 172.21.15.82:0/3917408108 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v19) ==== 105+0+0 (secure 0 0 0) 0x7f0e0c00d040 con 0x7f0e10101370 2024-05-20T16:12:11.370 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:11.368+0000 7f0e1812a640 1 -- 172.21.15.82:0/3917408108 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f0dfc0a2b90 msgr2=0x7f0dfc0a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:12:11.370 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:11.368+0000 7f0e1812a640 1 --2- 172.21.15.82:0/3917408108 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f0dfc0a2b90 0x7f0dfc0a50a0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f0e04000c00 tx=0x7f0e04031510 comp rx=0 tx=0).stop 2024-05-20T16:12:11.370 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:11.368+0000 7f0e1812a640 1 -- 172.21.15.82:0/3917408108 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0e10101370 msgr2=0x7f0e1019fa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:12:11.370 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:11.369+0000 7f0e1812a640 1 --2- 172.21.15.82:0/3917408108 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0e10101370 0x7f0e1019fa90 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f0e0c00b450 tx=0x7f0e0c00b910 comp rx=0 tx=0).stop 2024-05-20T16:12:11.370 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:11.369+0000 7f0e1812a640 1 -- 172.21.15.82:0/3917408108 shutdown_connections 2024-05-20T16:12:11.371 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:11.369+0000 7f0e1812a640 1 --2- 172.21.15.82:0/3917408108 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f0dfc0a2b90 0x7f0dfc0a50a0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:11.371 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:11.369+0000 7f0e1812a640 1 --2- 172.21.15.82:0/3917408108 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0e10106630 0x7f0e101a6f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:11.371 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:11.369+0000 7f0e1812a640 1 --2- 172.21.15.82:0/3917408108 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0e10103cd0 0x7f0e1019ffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:11.371 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:11.369+0000 7f0e1812a640 1 --2- 172.21.15.82:0/3917408108 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0e10101370 0x7f0e1019fa90 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:11.371 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:11.369+0000 7f0e1812a640 1 -- 172.21.15.82:0/3917408108 >> 172.21.15.82:0/3917408108 conn(0x7f0e100fcb80 msgr2=0x7f0e101022f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:12:11.371 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:11.369+0000 7f0e1812a640 1 -- 172.21.15.82:0/3917408108 shutdown_connections 2024-05-20T16:12:11.371 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:11.369+0000 7f0e1812a640 1 -- 172.21.15.82:0/3917408108 wait complete. 2024-05-20T16:12:11.382 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay true 2024-05-20T16:12:11.498 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:11.498+0000 7f5d7dd77640 1 -- 172.21.15.82:0/3687022796 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5d78105bf0 msgr2=0x7f5d78108090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:12:11.498 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:11.498+0000 7f5d7dd77640 1 --2- 172.21.15.82:0/3687022796 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5d78105bf0 0x7f5d78108090 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f5d60009900 tx=0x7f5d6002a410 comp rx=0 tx=0).stop 2024-05-20T16:12:11.498 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:11.498+0000 7f5d7dd77640 1 -- 172.21.15.82:0/3687022796 shutdown_connections 2024-05-20T16:12:11.499 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:11.498+0000 7f5d7dd77640 1 --2- 172.21.15.82:0/3687022796 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5d78105bf0 0x7f5d78108090 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:11.499 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:11.498+0000 7f5d7dd77640 1 --2- 172.21.15.82:0/3687022796 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5d78108720 0x7f5d7810ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:11.499 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:11.498+0000 7f5d7dd77640 1 --2- 172.21.15.82:0/3687022796 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5d78103200 0x7f5d78105620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:11.499 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:11.498+0000 7f5d7dd77640 1 -- 172.21.15.82:0/3687022796 >> 172.21.15.82:0/3687022796 conn(0x7f5d780fcb80 msgr2=0x7f5d780ff030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:12:11.499 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:11.498+0000 7f5d7dd77640 1 -- 172.21.15.82:0/3687022796 shutdown_connections 2024-05-20T16:12:11.499 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:11.498+0000 7f5d7dd77640 1 -- 172.21.15.82:0/3687022796 wait complete. 2024-05-20T16:12:11.499 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:11.499+0000 7f5d7dd77640 1 Processor -- start 2024-05-20T16:12:11.499 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:11.499+0000 7f5d7dd77640 1 -- start start 2024-05-20T16:12:11.501 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:11.499+0000 7f5d7dd77640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5d78103200 0x7f5d78199610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:12:11.501 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:11.499+0000 7f5d7dd77640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5d78105bf0 0x7f5d78199b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:12:11.501 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:11.499+0000 7f5d7dd77640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5d78108720 0x7f5d781a0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:12:11.501 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:11.499+0000 7f5d7dd77640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f5d7810e750 con 0x7f5d78103200 2024-05-20T16:12:11.501 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:11.499+0000 7f5d7dd77640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f5d7810e5d0 con 0x7f5d78108720 2024-05-20T16:12:11.501 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:11.499+0000 7f5d7dd77640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f5d7810e8d0 con 0x7f5d78105bf0 2024-05-20T16:12:11.501 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:11.499+0000 7f5d777fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5d78103200 0x7f5d78199610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:12:11.501 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:11.499+0000 7f5d777fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5d78103200 0x7f5d78199610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:47118/0 (socket says 172.21.15.82:47118) 2024-05-20T16:12:11.501 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:11.499+0000 7f5d777fe640 1 -- 172.21.15.82:0/2041083724 learned_addr learned my addr 172.21.15.82:0/2041083724 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:12:11.502 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:11.499+0000 7f5d6effd640 1 --2- 172.21.15.82:0/2041083724 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5d78105bf0 0x7f5d78199b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:12:11.502 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:11.499+0000 7f5d77fff640 1 --2- 172.21.15.82:0/2041083724 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5d78108720 0x7f5d781a0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:12:11.502 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:11.499+0000 7f5d777fe640 1 -- 172.21.15.82:0/2041083724 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5d78105bf0 msgr2=0x7f5d78199b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:12:11.502 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:11.499+0000 7f5d777fe640 1 --2- 172.21.15.82:0/2041083724 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5d78105bf0 0x7f5d78199b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:11.502 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:11.499+0000 7f5d777fe640 1 -- 172.21.15.82:0/2041083724 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5d78108720 msgr2=0x7f5d781a0ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:12:11.502 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:11.499+0000 7f5d777fe640 1 --2- 172.21.15.82:0/2041083724 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5d78108720 0x7f5d781a0ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:11.502 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:11.499+0000 7f5d777fe640 1 -- 172.21.15.82:0/2041083724 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d781a1150 con 0x7f5d78103200 2024-05-20T16:12:11.502 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:11.500+0000 7f5d777fe640 1 --2- 172.21.15.82:0/2041083724 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5d78103200 0x7f5d78199610 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f5d6800d740 tx=0x7f5d6800dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:12:11.502 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:11.500+0000 7f5d757fa640 1 -- 172.21.15.82:0/2041083724 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d68017070 con 0x7f5d78103200 2024-05-20T16:12:11.502 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:11.500+0000 7f5d757fa640 1 -- 172.21.15.82:0/2041083724 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d6800de50 con 0x7f5d78103200 2024-05-20T16:12:11.502 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:11.500+0000 7f5d7dd77640 1 -- 172.21.15.82:0/2041083724 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d781a1440 con 0x7f5d78103200 2024-05-20T16:12:11.502 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:11.500+0000 7f5d757fa640 1 -- 172.21.15.82:0/2041083724 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d68004e00 con 0x7f5d78103200 2024-05-20T16:12:11.503 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:11.500+0000 7f5d7dd77640 1 -- 172.21.15.82:0/2041083724 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d781a1910 con 0x7f5d78103200 2024-05-20T16:12:11.503 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:11.501+0000 7f5d7dd77640 1 -- 172.21.15.82:0/2041083724 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d780fe6a0 con 0x7f5d78103200 2024-05-20T16:12:11.503 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:11.502+0000 7f5d757fa640 1 -- 172.21.15.82:0/2041083724 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f5d68004fa0 con 0x7f5d78103200 2024-05-20T16:12:11.503 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:11.503+0000 7f5d757fa640 1 --2- 172.21.15.82:0/2041083724 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f5d640a2b90 0x7f5d640a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:12:11.504 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:11.503+0000 7f5d6effd640 1 --2- 172.21.15.82:0/2041083724 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f5d640a2b90 0x7f5d640a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:12:11.504 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:11.503+0000 7f5d757fa640 1 -- 172.21.15.82:0/2041083724 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8802+0+0 (secure 0 0 0) 0x7f5d680cf060 con 0x7f5d78103200 2024-05-20T16:12:11.504 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:11.503+0000 7f5d6effd640 1 --2- 172.21.15.82:0/2041083724 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f5d640a2b90 0x7f5d640a50a0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f5d60005ab0 tx=0x7f5d600315b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:12:11.506 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:11.506+0000 7f5d757fa640 1 -- 172.21.15.82:0/2041083724 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f5d6809bd80 con 0x7f5d78103200 2024-05-20T16:12:11.635 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:12:11 smithi184 ceph-mon[36209]: from='client.? 172.21.15.82:0/3617055753' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-05-20T16:12:11.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:12:11 smithi184 ceph-mon[36209]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-20T16:12:11.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:12:11 smithi184 ceph-mon[36209]: from='client.? 172.21.15.82:0/3917408108' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-05-20T16:12:11.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:12:11 smithi184 ceph-mon[36209]: pgmap v304: 129 pgs: 129 active+clean; 581 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-05-20T16:12:11.720 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:11.720+0000 7f5d7dd77640 1 -- 172.21.15.82:0/2041083724 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7f5d7806b250 con 0x7f5d78103200 2024-05-20T16:12:11.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:12:11 smithi160 ceph-mon[36347]: from='client.? 172.21.15.82:0/3617055753' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-05-20T16:12:11.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:12:11 smithi160 ceph-mon[36347]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-20T16:12:11.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:12:11 smithi160 ceph-mon[36347]: from='client.? 172.21.15.82:0/3917408108' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-05-20T16:12:11.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:12:11 smithi160 ceph-mon[36347]: pgmap v304: 129 pgs: 129 active+clean; 581 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-05-20T16:12:11.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:12:11 smithi082 ceph-mon[32238]: from='client.? 172.21.15.82:0/3617055753' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-05-20T16:12:11.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:12:11 smithi082 ceph-mon[32238]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-20T16:12:11.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:12:11 smithi082 ceph-mon[32238]: from='client.? 172.21.15.82:0/3917408108' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-05-20T16:12:11.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:12:11 smithi082 ceph-mon[32238]: pgmap v304: 129 pgs: 129 active+clean; 581 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-05-20T16:12:12.377 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:12.377+0000 7f5d757fa640 1 -- 172.21.15.82:0/2041083724 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v21) ==== 121+0+0 (secure 0 0 0) 0x7f5d6809b680 con 0x7f5d78103200 2024-05-20T16:12:12.380 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:12.380+0000 7f5d7dd77640 1 -- 172.21.15.82:0/2041083724 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f5d640a2b90 msgr2=0x7f5d640a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:12:12.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:12.380+0000 7f5d7dd77640 1 --2- 172.21.15.82:0/2041083724 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f5d640a2b90 0x7f5d640a50a0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f5d60005ab0 tx=0x7f5d600315b0 comp rx=0 tx=0).stop 2024-05-20T16:12:12.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:12.380+0000 7f5d7dd77640 1 -- 172.21.15.82:0/2041083724 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5d78103200 msgr2=0x7f5d78199610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:12:12.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:12.380+0000 7f5d7dd77640 1 --2- 172.21.15.82:0/2041083724 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5d78103200 0x7f5d78199610 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f5d6800d740 tx=0x7f5d6800dc00 comp rx=0 tx=0).stop 2024-05-20T16:12:12.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:12.381+0000 7f5d7dd77640 1 -- 172.21.15.82:0/2041083724 shutdown_connections 2024-05-20T16:12:12.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:12.381+0000 7f5d7dd77640 1 --2- 172.21.15.82:0/2041083724 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f5d640a2b90 0x7f5d640a50a0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:12.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:12.381+0000 7f5d7dd77640 1 --2- 172.21.15.82:0/2041083724 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5d78108720 0x7f5d781a0ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:12.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:12.381+0000 7f5d7dd77640 1 --2- 172.21.15.82:0/2041083724 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5d78105bf0 0x7f5d78199b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:12.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:12.381+0000 7f5d7dd77640 1 --2- 172.21.15.82:0/2041083724 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5d78103200 0x7f5d78199610 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:12.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:12.381+0000 7f5d7dd77640 1 -- 172.21.15.82:0/2041083724 >> 172.21.15.82:0/2041083724 conn(0x7f5d780fcb80 msgr2=0x7f5d781043c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:12:12.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:12.381+0000 7f5d7dd77640 1 -- 172.21.15.82:0/2041083724 shutdown_connections 2024-05-20T16:12:12.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:12.381+0000 7f5d7dd77640 1 -- 172.21.15.82:0/2041083724 wait complete. 2024-05-20T16:12:12.397 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2024-05-20T16:12:12.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:12.516+0000 7f2c1f7ea640 1 -- 172.21.15.82:0/1917004992 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2c18103cf0 msgr2=0x7f2c18104100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:12:12.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:12.516+0000 7f2c1f7ea640 1 --2- 172.21.15.82:0/1917004992 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2c18103cf0 0x7f2c18104100 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f2c08009900 tx=0x7f2c0802a410 comp rx=0 tx=0).stop 2024-05-20T16:12:12.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:12.516+0000 7f2c1f7ea640 1 -- 172.21.15.82:0/1917004992 shutdown_connections 2024-05-20T16:12:12.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:12.516+0000 7f2c1f7ea640 1 --2- 172.21.15.82:0/1917004992 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2c18103cf0 0x7f2c18104100 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:12.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:12.516+0000 7f2c1f7ea640 1 --2- 172.21.15.82:0/1917004992 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2c181089f0 0x7f2c1810ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:12.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:12.516+0000 7f2c1f7ea640 1 --2- 172.21.15.82:0/1917004992 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2c18102ab0 0x7f2c181037b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:12.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:12.516+0000 7f2c1f7ea640 1 -- 172.21.15.82:0/1917004992 >> 172.21.15.82:0/1917004992 conn(0x7f2c180fdcc0 msgr2=0x7f2c18100150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:12:12.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:12.516+0000 7f2c1f7ea640 1 -- 172.21.15.82:0/1917004992 shutdown_connections 2024-05-20T16:12:12.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:12.516+0000 7f2c1f7ea640 1 -- 172.21.15.82:0/1917004992 wait complete. 2024-05-20T16:12:12.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:12.516+0000 7f2c1f7ea640 1 Processor -- start 2024-05-20T16:12:12.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:12.516+0000 7f2c1f7ea640 1 -- start start 2024-05-20T16:12:12.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:12.517+0000 7f2c1f7ea640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2c18102ab0 0x7f2c18199320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:12:12.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:12.517+0000 7f2c1f7ea640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2c18103cf0 0x7f2c18199860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:12:12.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:12.517+0000 7f2c1f7ea640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2c181089f0 0x7f2c181a07f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:12:12.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:12.517+0000 7f2c1f7ea640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f2c1810e720 con 0x7f2c181089f0 2024-05-20T16:12:12.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:12.517+0000 7f2c1f7ea640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f2c1810e5a0 con 0x7f2c18102ab0 2024-05-20T16:12:12.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:12.517+0000 7f2c1f7ea640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f2c1810e8a0 con 0x7f2c18103cf0 2024-05-20T16:12:12.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:12.517+0000 7f2c1dd87640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2c181089f0 0x7f2c181a07f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:12:12.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:12.517+0000 7f2c1dd87640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2c181089f0 0x7f2c181a07f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:47136/0 (socket says 172.21.15.82:47136) 2024-05-20T16:12:12.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:12.517+0000 7f2c1dd87640 1 -- 172.21.15.82:0/682875220 learned_addr learned my addr 172.21.15.82:0/682875220 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:12:12.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:12.517+0000 7f2c1d586640 1 --2- 172.21.15.82:0/682875220 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2c18102ab0 0x7f2c18199320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:12:12.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:12.517+0000 7f2c1cd85640 1 --2- 172.21.15.82:0/682875220 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2c18103cf0 0x7f2c18199860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:12:12.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:12.517+0000 7f2c1dd87640 1 -- 172.21.15.82:0/682875220 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2c18103cf0 msgr2=0x7f2c18199860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:12:12.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:12.517+0000 7f2c1dd87640 1 --2- 172.21.15.82:0/682875220 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2c18103cf0 0x7f2c18199860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:12.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:12.517+0000 7f2c1dd87640 1 -- 172.21.15.82:0/682875220 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2c18102ab0 msgr2=0x7f2c18199320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:12:12.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:12.517+0000 7f2c1dd87640 1 --2- 172.21.15.82:0/682875220 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2c18102ab0 0x7f2c18199320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:12.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:12.517+0000 7f2c1dd87640 1 -- 172.21.15.82:0/682875220 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c181a0e60 con 0x7f2c181089f0 2024-05-20T16:12:12.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:12.517+0000 7f2c1cd85640 1 --2- 172.21.15.82:0/682875220 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2c18103cf0 0x7f2c18199860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T16:12:12.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:12.517+0000 7f2c1dd87640 1 --2- 172.21.15.82:0/682875220 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2c181089f0 0x7f2c181a07f0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f2c1000eb10 tx=0x7f2c1000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:12:12.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:12.517+0000 7f2c0e7fc640 1 -- 172.21.15.82:0/682875220 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c10004b20 con 0x7f2c181089f0 2024-05-20T16:12:12.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:12.518+0000 7f2c0e7fc640 1 -- 172.21.15.82:0/682875220 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2c10004cc0 con 0x7f2c181089f0 2024-05-20T16:12:12.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:12.518+0000 7f2c1f7ea640 1 -- 172.21.15.82:0/682875220 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c181a1150 con 0x7f2c181089f0 2024-05-20T16:12:12.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:12.518+0000 7f2c0e7fc640 1 -- 172.21.15.82:0/682875220 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c10010760 con 0x7f2c181089f0 2024-05-20T16:12:12.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:12.518+0000 7f2c1f7ea640 1 -- 172.21.15.82:0/682875220 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c181a1650 con 0x7f2c181089f0 2024-05-20T16:12:12.521 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:12.518+0000 7f2c1f7ea640 1 -- 172.21.15.82:0/682875220 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c18107800 con 0x7f2c181089f0 2024-05-20T16:12:12.521 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:12.520+0000 7f2c0e7fc640 1 -- 172.21.15.82:0/682875220 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f2c10005000 con 0x7f2c181089f0 2024-05-20T16:12:12.521 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:12.520+0000 7f2c0e7fc640 1 --2- 172.21.15.82:0/682875220 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f2c040a2b90 0x7f2c040a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:12:12.522 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:12.521+0000 7f2c1d586640 1 --2- 172.21.15.82:0/682875220 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f2c040a2b90 0x7f2c040a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:12:12.522 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:12.521+0000 7f2c0e7fc640 1 -- 172.21.15.82:0/682875220 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8802+0+0 (secure 0 0 0) 0x7f2c100cfad0 con 0x7f2c181089f0 2024-05-20T16:12:12.522 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:12.521+0000 7f2c1d586640 1 --2- 172.21.15.82:0/682875220 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f2c040a2b90 0x7f2c040a50a0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f2c140058c0 tx=0x7f2c1400a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:12:12.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:12.524+0000 7f2c0e7fc640 1 -- 172.21.15.82:0/682875220 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f2c10053850 con 0x7f2c181089f0 2024-05-20T16:12:12.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:12:12 smithi184 ceph-mon[36209]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-05-20T16:12:12.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:12:12 smithi184 ceph-mon[36209]: from='client.? 172.21.15.82:0/3917408108' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-05-20T16:12:12.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:12:12 smithi184 ceph-mon[36209]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-05-20T16:12:12.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:12:12 smithi184 ceph-mon[36209]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-20T16:12:12.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:12:12 smithi184 ceph-mon[36209]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-05-20T16:12:12.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:12:12 smithi184 ceph-mon[36209]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-05-20T16:12:12.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:12:12 smithi184 ceph-mon[36209]: from='client.? 172.21.15.82:0/2041083724' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-20T16:12:12.737 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:12.737+0000 7f2c1f7ea640 1 -- 172.21.15.82:0/682875220 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7f2c1802d2a0 con 0x7f2c181089f0 2024-05-20T16:12:12.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:12:12 smithi160 ceph-mon[36347]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-05-20T16:12:12.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:12:12 smithi160 ceph-mon[36347]: from='client.? 172.21.15.82:0/3917408108' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-05-20T16:12:12.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:12:12 smithi160 ceph-mon[36347]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-05-20T16:12:12.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:12:12 smithi160 ceph-mon[36347]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-20T16:12:12.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:12:12 smithi160 ceph-mon[36347]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-05-20T16:12:12.813 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:12:12 smithi160 ceph-mon[36347]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-05-20T16:12:12.813 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:12:12 smithi160 ceph-mon[36347]: from='client.? 172.21.15.82:0/2041083724' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-20T16:12:12.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:12:12 smithi082 ceph-mon[32238]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-05-20T16:12:12.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:12:12 smithi082 ceph-mon[32238]: from='client.? 172.21.15.82:0/3917408108' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-05-20T16:12:12.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:12:12 smithi082 ceph-mon[32238]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-05-20T16:12:12.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:12:12 smithi082 ceph-mon[32238]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-20T16:12:12.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:12:12 smithi082 ceph-mon[32238]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-05-20T16:12:12.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:12:12 smithi082 ceph-mon[32238]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-05-20T16:12:12.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:12:12 smithi082 ceph-mon[32238]: from='client.? 172.21.15.82:0/2041083724' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-20T16:12:13.389 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:13.390+0000 7f2c0e7fc640 1 -- 172.21.15.82:0/682875220 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v23) ==== 115+0+0 (secure 0 0 0) 0x7f2c1009c7f0 con 0x7f2c181089f0 2024-05-20T16:12:13.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:13.392+0000 7f2c1f7ea640 1 -- 172.21.15.82:0/682875220 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f2c040a2b90 msgr2=0x7f2c040a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:12:13.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:13.392+0000 7f2c1f7ea640 1 --2- 172.21.15.82:0/682875220 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f2c040a2b90 0x7f2c040a50a0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f2c140058c0 tx=0x7f2c1400a2d0 comp rx=0 tx=0).stop 2024-05-20T16:12:13.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:13.392+0000 7f2c1f7ea640 1 -- 172.21.15.82:0/682875220 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2c181089f0 msgr2=0x7f2c181a07f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:12:13.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:13.392+0000 7f2c1f7ea640 1 --2- 172.21.15.82:0/682875220 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2c181089f0 0x7f2c181a07f0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f2c1000eb10 tx=0x7f2c1000c2d0 comp rx=0 tx=0).stop 2024-05-20T16:12:13.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:13.392+0000 7f2c1f7ea640 1 -- 172.21.15.82:0/682875220 shutdown_connections 2024-05-20T16:12:13.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:13.392+0000 7f2c1f7ea640 1 --2- 172.21.15.82:0/682875220 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f2c040a2b90 0x7f2c040a50a0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:13.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:13.392+0000 7f2c1f7ea640 1 --2- 172.21.15.82:0/682875220 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2c181089f0 0x7f2c181a07f0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:13.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:13.392+0000 7f2c1f7ea640 1 --2- 172.21.15.82:0/682875220 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2c18103cf0 0x7f2c18199860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:13.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:13.392+0000 7f2c1f7ea640 1 --2- 172.21.15.82:0/682875220 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2c18102ab0 0x7f2c18199320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:13.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:13.392+0000 7f2c1f7ea640 1 -- 172.21.15.82:0/682875220 >> 172.21.15.82:0/682875220 conn(0x7f2c180fdcc0 msgr2=0x7f2c181049d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:12:13.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:13.393+0000 7f2c1f7ea640 1 -- 172.21.15.82:0/682875220 shutdown_connections 2024-05-20T16:12:13.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:13.393+0000 7f2c1f7ea640 1 -- 172.21.15.82:0/682875220 wait complete. 2024-05-20T16:12:13.405 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--namespace-isolated'} subvols for filesystem 'cephfs' 2024-05-20T16:12:13.406 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create cephfs qa 2024-05-20T16:12:13.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:13.523+0000 7f31c65b5640 1 -- 172.21.15.82:0/3070164800 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f31c0102be0 msgr2=0x7f31c0102ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:12:13.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:13.523+0000 7f31c65b5640 1 --2- 172.21.15.82:0/3070164800 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f31c0102be0 0x7f31c0102ff0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f31a8009900 tx=0x7f31a802a410 comp rx=0 tx=0).stop 2024-05-20T16:12:13.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:13.523+0000 7f31c65b5640 1 -- 172.21.15.82:0/3070164800 shutdown_connections 2024-05-20T16:12:13.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:13.523+0000 7f31c65b5640 1 --2- 172.21.15.82:0/3070164800 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f31c0102be0 0x7f31c0102ff0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:13.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:13.523+0000 7f31c65b5640 1 --2- 172.21.15.82:0/3070164800 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f31c01078e0 0x7f31c0109d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:13.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:13.523+0000 7f31c65b5640 1 --2- 172.21.15.82:0/3070164800 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f31c01019a0 0x7f31c01026a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:13.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:13.523+0000 7f31c65b5640 1 -- 172.21.15.82:0/3070164800 >> 172.21.15.82:0/3070164800 conn(0x7f31c00fcb70 msgr2=0x7f31c00ff040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:12:13.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:13.523+0000 7f31c65b5640 1 -- 172.21.15.82:0/3070164800 shutdown_connections 2024-05-20T16:12:13.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:13.523+0000 7f31c65b5640 1 -- 172.21.15.82:0/3070164800 wait complete. 2024-05-20T16:12:13.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:13.523+0000 7f31c65b5640 1 Processor -- start 2024-05-20T16:12:13.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:13.524+0000 7f31c65b5640 1 -- start start 2024-05-20T16:12:13.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:13.524+0000 7f31c65b5640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f31c01019a0 0x7f31c0074810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:12:13.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:13.524+0000 7f31c65b5640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f31c0102be0 0x7f31c019e040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:12:13.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:13.524+0000 7f31c65b5640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f31c01078e0 0x7f31c01a05b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:12:13.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:13.524+0000 7f31c65b5640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f31c010d610 con 0x7f31c01078e0 2024-05-20T16:12:13.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:13.524+0000 7f31c65b5640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f31c010d490 con 0x7f31c0102be0 2024-05-20T16:12:13.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:13.524+0000 7f31c65b5640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f31c010d790 con 0x7f31c01019a0 2024-05-20T16:12:13.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:13.524+0000 7f31c4b52640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f31c01078e0 0x7f31c01a05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:12:13.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:13.524+0000 7f31c4b52640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f31c01078e0 0x7f31c01a05b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:47150/0 (socket says 172.21.15.82:47150) 2024-05-20T16:12:13.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:13.524+0000 7f31c4b52640 1 -- 172.21.15.82:0/3498323414 learned_addr learned my addr 172.21.15.82:0/3498323414 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:12:13.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:13.524+0000 7f31b7fff640 1 --2- 172.21.15.82:0/3498323414 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f31c0102be0 0x7f31c019e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:12:13.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:13.524+0000 7f31bffff640 1 --2- 172.21.15.82:0/3498323414 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f31c01019a0 0x7f31c0074810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:12:13.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:13.524+0000 7f31c4b52640 1 -- 172.21.15.82:0/3498323414 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f31c01019a0 msgr2=0x7f31c0074810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:12:13.527 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:13.524+0000 7f31c4b52640 1 --2- 172.21.15.82:0/3498323414 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f31c01019a0 0x7f31c0074810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:13.527 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:13.524+0000 7f31c4b52640 1 -- 172.21.15.82:0/3498323414 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f31c0102be0 msgr2=0x7f31c019e040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:12:13.527 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:13.524+0000 7f31c4b52640 1 --2- 172.21.15.82:0/3498323414 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f31c0102be0 0x7f31c019e040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:13.527 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:13.524+0000 7f31c4b52640 1 -- 172.21.15.82:0/3498323414 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31c01a0dd0 con 0x7f31c01078e0 2024-05-20T16:12:13.527 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:13.524+0000 7f31b7fff640 1 --2- 172.21.15.82:0/3498323414 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f31c0102be0 0x7f31c019e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T16:12:13.527 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:13.524+0000 7f31c4b52640 1 --2- 172.21.15.82:0/3498323414 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f31c01078e0 0x7f31c01a05b0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f31b800eb10 tx=0x7f31b800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:12:13.527 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:13.524+0000 7f31bdffb640 1 -- 172.21.15.82:0/3498323414 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31b8004b20 con 0x7f31c01078e0 2024-05-20T16:12:13.527 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:13.524+0000 7f31bdffb640 1 -- 172.21.15.82:0/3498323414 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f31b8004cc0 con 0x7f31c01078e0 2024-05-20T16:12:13.527 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:13.524+0000 7f31bdffb640 1 -- 172.21.15.82:0/3498323414 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31b8010760 con 0x7f31c01078e0 2024-05-20T16:12:13.527 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:13.524+0000 7f31c65b5640 1 -- 172.21.15.82:0/3498323414 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31c01a10c0 con 0x7f31c01078e0 2024-05-20T16:12:13.527 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:13.525+0000 7f31c65b5640 1 -- 172.21.15.82:0/3498323414 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31c01a15c0 con 0x7f31c01078e0 2024-05-20T16:12:13.528 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:13.525+0000 7f31c65b5640 1 -- 172.21.15.82:0/3498323414 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31c01066f0 con 0x7f31c01078e0 2024-05-20T16:12:13.528 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:13.528+0000 7f31bdffb640 1 -- 172.21.15.82:0/3498323414 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f31b8005000 con 0x7f31c01078e0 2024-05-20T16:12:13.528 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:13.528+0000 7f31bdffb640 1 --2- 172.21.15.82:0/3498323414 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f31ac0a2b90 0x7f31ac0a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:12:13.529 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:13.528+0000 7f31bffff640 1 --2- 172.21.15.82:0/3498323414 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f31ac0a2b90 0x7f31ac0a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:12:13.529 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:13.528+0000 7f31bdffb640 1 -- 172.21.15.82:0/3498323414 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8802+0+0 (secure 0 0 0) 0x7f31b80cfbb0 con 0x7f31c01078e0 2024-05-20T16:12:13.529 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:13.529+0000 7f31bffff640 1 --2- 172.21.15.82:0/3498323414 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f31ac0a2b90 0x7f31ac0a50a0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f31b00058c0 tx=0x7f31b0005c80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:12:13.532 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:13.532+0000 7f31bdffb640 1 -- 172.21.15.82:0/3498323414 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f31b8053850 con 0x7f31c01078e0 2024-05-20T16:12:13.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:12:13 smithi184 ceph-mon[36209]: from='client.? 172.21.15.82:0/2041083724' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-05-20T16:12:13.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:12:13 smithi184 ceph-mon[36209]: mds.? [v2:172.21.15.184:6838/2199955067,v1:172.21.15.184:6839/2199955067] up:active 2024-05-20T16:12:13.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:12:13 smithi184 ceph-mon[36209]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-05-20T16:12:13.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:12:13 smithi184 ceph-mon[36209]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-05-20T16:12:13.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:12:13 smithi184 ceph-mon[36209]: Cluster is now healthy 2024-05-20T16:12:13.637 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:12:13 smithi184 ceph-mon[36209]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-05-20T16:12:13.637 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:12:13 smithi184 ceph-mon[36209]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2024-05-20T16:12:13.637 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:12:13 smithi184 ceph-mon[36209]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-05-20T16:12:13.637 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:12:13 smithi184 ceph-mon[36209]: from='client.? 172.21.15.82:0/682875220' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-05-20T16:12:13.637 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:12:13 smithi184 ceph-mon[36209]: pgmap v305: 129 pgs: 129 active+clean; 582 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s wr, 1 op/s 2024-05-20T16:12:13.733 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:13.733+0000 7f31c65b5640 1 -- 172.21.15.82:0/3498323414 --> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f31c0061f90 con 0x7f31ac0a2b90 2024-05-20T16:12:13.740 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:12:13 smithi082 ceph-mon[32238]: from='client.? 172.21.15.82:0/2041083724' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-05-20T16:12:13.740 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:12:13 smithi082 ceph-mon[32238]: mds.? [v2:172.21.15.184:6838/2199955067,v1:172.21.15.184:6839/2199955067] up:active 2024-05-20T16:12:13.740 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:12:13 smithi082 ceph-mon[32238]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-05-20T16:12:13.740 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:12:13 smithi082 ceph-mon[32238]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-05-20T16:12:13.740 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:12:13 smithi082 ceph-mon[32238]: Cluster is now healthy 2024-05-20T16:12:13.740 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:12:13 smithi082 ceph-mon[32238]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-05-20T16:12:13.740 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:12:13 smithi082 ceph-mon[32238]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2024-05-20T16:12:13.740 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:12:13 smithi082 ceph-mon[32238]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-05-20T16:12:13.740 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:12:13 smithi082 ceph-mon[32238]: from='client.? 172.21.15.82:0/682875220' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-05-20T16:12:13.740 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:12:13 smithi082 ceph-mon[32238]: pgmap v305: 129 pgs: 129 active+clean; 582 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s wr, 1 op/s 2024-05-20T16:12:13.756 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:13.756+0000 7f31bdffb640 1 -- 172.21.15.82:0/3498323414 <== mgr.14150 v2:172.21.15.82:6800/379530823 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f31c0061f90 con 0x7f31ac0a2b90 2024-05-20T16:12:13.759 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:13.759+0000 7f31c65b5640 1 -- 172.21.15.82:0/3498323414 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f31ac0a2b90 msgr2=0x7f31ac0a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:12:13.759 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:13.759+0000 7f31c65b5640 1 --2- 172.21.15.82:0/3498323414 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f31ac0a2b90 0x7f31ac0a50a0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f31b00058c0 tx=0x7f31b0005c80 comp rx=0 tx=0).stop 2024-05-20T16:12:13.759 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:13.759+0000 7f31c65b5640 1 -- 172.21.15.82:0/3498323414 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f31c01078e0 msgr2=0x7f31c01a05b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:12:13.759 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:13.759+0000 7f31c65b5640 1 --2- 172.21.15.82:0/3498323414 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f31c01078e0 0x7f31c01a05b0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f31b800eb10 tx=0x7f31b800c2d0 comp rx=0 tx=0).stop 2024-05-20T16:12:13.759 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:13.759+0000 7f31c65b5640 1 -- 172.21.15.82:0/3498323414 shutdown_connections 2024-05-20T16:12:13.759 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:13.759+0000 7f31c65b5640 1 --2- 172.21.15.82:0/3498323414 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f31ac0a2b90 0x7f31ac0a50a0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:13.759 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:13.759+0000 7f31c65b5640 1 --2- 172.21.15.82:0/3498323414 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f31c01078e0 0x7f31c01a05b0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:13.760 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:13.759+0000 7f31c65b5640 1 --2- 172.21.15.82:0/3498323414 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f31c0102be0 0x7f31c019e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:13.760 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:13.759+0000 7f31c65b5640 1 --2- 172.21.15.82:0/3498323414 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f31c01019a0 0x7f31c0074810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:13.760 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:13.759+0000 7f31c65b5640 1 -- 172.21.15.82:0/3498323414 >> 172.21.15.82:0/3498323414 conn(0x7f31c00fcb70 msgr2=0x7f31c01038c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:12:13.760 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:13.759+0000 7f31c65b5640 1 -- 172.21.15.82:0/3498323414 shutdown_connections 2024-05-20T16:12:13.760 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:13.759+0000 7f31c65b5640 1 -- 172.21.15.82:0/3498323414 wait complete. 2024-05-20T16:12:13.770 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_0 --group_name qa 2024-05-20T16:12:13.812 INFO:journalctl@ceph.mds.b.smithi160.stdout:May 20 16:12:13 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-b[66944]: 2024-05-20T16:12:13.311+0000 7ff20f5dd700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-05-20T16:12:13.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:12:13 smithi160 ceph-mon[36347]: from='client.? 172.21.15.82:0/2041083724' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-05-20T16:12:13.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:12:13 smithi160 ceph-mon[36347]: mds.? [v2:172.21.15.184:6838/2199955067,v1:172.21.15.184:6839/2199955067] up:active 2024-05-20T16:12:13.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:12:13 smithi160 ceph-mon[36347]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-05-20T16:12:13.813 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:12:13 smithi160 ceph-mon[36347]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-05-20T16:12:13.813 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:12:13 smithi160 ceph-mon[36347]: Cluster is now healthy 2024-05-20T16:12:13.813 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:12:13 smithi160 ceph-mon[36347]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-05-20T16:12:13.813 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:12:13 smithi160 ceph-mon[36347]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2024-05-20T16:12:13.813 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:12:13 smithi160 ceph-mon[36347]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-05-20T16:12:13.813 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:12:13 smithi160 ceph-mon[36347]: from='client.? 172.21.15.82:0/682875220' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-05-20T16:12:13.813 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:12:13 smithi160 ceph-mon[36347]: pgmap v305: 129 pgs: 129 active+clean; 582 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s wr, 1 op/s 2024-05-20T16:12:13.890 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:13.889+0000 7fcc58b1a640 1 -- 172.21.15.82:0/3701852275 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcc54101200 msgr2=0x7fcc5410ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:12:13.890 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:13.889+0000 7fcc58b1a640 1 --2- 172.21.15.82:0/3701852275 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcc54101200 0x7fcc5410ac60 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fcc4c009ca0 tx=0x7fcc4c02a8c0 comp rx=0 tx=0).stop 2024-05-20T16:12:13.890 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:13.889+0000 7fcc58b1a640 1 -- 172.21.15.82:0/3701852275 shutdown_connections 2024-05-20T16:12:13.890 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:13.889+0000 7fcc58b1a640 1 --2- 172.21.15.82:0/3701852275 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcc54101200 0x7fcc5410ac60 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:13.890 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:13.889+0000 7fcc58b1a640 1 --2- 172.21.15.82:0/3701852275 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcc54100700 0x7fcc54100b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:13.890 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:13.889+0000 7fcc58b1a640 1 --2- 172.21.15.82:0/3701852275 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcc54104050 0x7fcc54104460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:13.890 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:13.889+0000 7fcc58b1a640 1 -- 172.21.15.82:0/3701852275 >> 172.21.15.82:0/3701852275 conn(0x7fcc540fc300 msgr2=0x7fcc540fe7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:12:13.890 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:13.889+0000 7fcc58b1a640 1 -- 172.21.15.82:0/3701852275 shutdown_connections 2024-05-20T16:12:13.890 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:13.889+0000 7fcc58b1a640 1 -- 172.21.15.82:0/3701852275 wait complete. 2024-05-20T16:12:13.892 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:13.890+0000 7fcc58b1a640 1 Processor -- start 2024-05-20T16:12:13.893 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:13.890+0000 7fcc58b1a640 1 -- start start 2024-05-20T16:12:13.893 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:13.890+0000 7fcc58b1a640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcc54100700 0x7fcc54199220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:12:13.893 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:13.891+0000 7fcc58b1a640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcc54101200 0x7fcc54199760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:12:13.893 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:13.891+0000 7fcc58b1a640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcc54104050 0x7fcc541a06f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:12:13.893 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:13.891+0000 7fcc58b1a640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fcc5410e730 con 0x7fcc54101200 2024-05-20T16:12:13.893 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:13.891+0000 7fcc58b1a640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fcc5410e5b0 con 0x7fcc54100700 2024-05-20T16:12:13.893 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:13.891+0000 7fcc58b1a640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fcc5410e8b0 con 0x7fcc54104050 2024-05-20T16:12:13.893 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:13.891+0000 7fcc51d9b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcc54101200 0x7fcc54199760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:12:13.893 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:13.891+0000 7fcc51d9b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcc54101200 0x7fcc54199760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:47178/0 (socket says 172.21.15.82:47178) 2024-05-20T16:12:13.893 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:13.891+0000 7fcc51d9b640 1 -- 172.21.15.82:0/4181946218 learned_addr learned my addr 172.21.15.82:0/4181946218 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:12:13.893 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:13.891+0000 7fcc5259c640 1 --2- 172.21.15.82:0/4181946218 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcc54100700 0x7fcc54199220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:12:13.893 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:13.891+0000 7fcc52d9d640 1 --2- 172.21.15.82:0/4181946218 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcc54104050 0x7fcc541a06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:12:13.894 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:13.891+0000 7fcc51d9b640 1 -- 172.21.15.82:0/4181946218 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcc54104050 msgr2=0x7fcc541a06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:12:13.894 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:13.891+0000 7fcc51d9b640 1 --2- 172.21.15.82:0/4181946218 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcc54104050 0x7fcc541a06f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:13.894 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:13.891+0000 7fcc51d9b640 1 -- 172.21.15.82:0/4181946218 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcc54100700 msgr2=0x7fcc54199220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:12:13.894 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:13.891+0000 7fcc51d9b640 1 --2- 172.21.15.82:0/4181946218 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcc54100700 0x7fcc54199220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:13.894 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:13.891+0000 7fcc51d9b640 1 -- 172.21.15.82:0/4181946218 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc541a0d60 con 0x7fcc54101200 2024-05-20T16:12:13.894 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:13.891+0000 7fcc5259c640 1 --2- 172.21.15.82:0/4181946218 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcc54100700 0x7fcc54199220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T16:12:13.894 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:13.891+0000 7fcc51d9b640 1 --2- 172.21.15.82:0/4181946218 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcc54101200 0x7fcc54199760 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fcc4800d770 tx=0x7fcc4800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:12:13.894 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:13.891+0000 7fcc437fe640 1 -- 172.21.15.82:0/4181946218 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc48017070 con 0x7fcc54101200 2024-05-20T16:12:13.894 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:13.891+0000 7fcc437fe640 1 -- 172.21.15.82:0/4181946218 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcc48005500 con 0x7fcc54101200 2024-05-20T16:12:13.894 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:13.891+0000 7fcc58b1a640 1 -- 172.21.15.82:0/4181946218 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc541a0ff0 con 0x7fcc54101200 2024-05-20T16:12:13.894 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:13.891+0000 7fcc437fe640 1 -- 172.21.15.82:0/4181946218 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc48004e00 con 0x7fcc54101200 2024-05-20T16:12:13.894 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:13.891+0000 7fcc58b1a640 1 -- 172.21.15.82:0/4181946218 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc541a1490 con 0x7fcc54101200 2024-05-20T16:12:13.894 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:13.892+0000 7fcc58b1a640 1 -- 172.21.15.82:0/4181946218 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc541099f0 con 0x7fcc54101200 2024-05-20T16:12:13.895 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:13.894+0000 7fcc437fe640 1 -- 172.21.15.82:0/4181946218 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fcc48004fa0 con 0x7fcc54101200 2024-05-20T16:12:13.895 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:13.894+0000 7fcc437fe640 1 --2- 172.21.15.82:0/4181946218 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fcc3c0a2b90 0x7fcc3c0a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:12:13.895 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:13.895+0000 7fcc5259c640 1 --2- 172.21.15.82:0/4181946218 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fcc3c0a2b90 0x7fcc3c0a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:12:13.895 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:13.895+0000 7fcc437fe640 1 -- 172.21.15.82:0/4181946218 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8802+0+0 (secure 0 0 0) 0x7fcc480cf130 con 0x7fcc54101200 2024-05-20T16:12:13.896 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:13.895+0000 7fcc5259c640 1 --2- 172.21.15.82:0/4181946218 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fcc3c0a2b90 0x7fcc3c0a50a0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fcc5406a2a0 tx=0x7fcc44005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:12:13.902 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:13.903+0000 7fcc437fe640 1 -- 172.21.15.82:0/4181946218 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fcc4809be50 con 0x7fcc54101200 2024-05-20T16:12:14.052 INFO:journalctl@ceph.mgr.x.smithi082.stdout:May 20 16:12:13 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mgr-x[32467]: 2024-05-20T16:12:13.739+0000 7f1a2d4ae700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-20T16:12:14.053 INFO:journalctl@ceph.mgr.x.smithi082.stdout:May 20 16:12:13 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mgr-x[32467]: 2024-05-20T16:12:13.739+0000 7f1a2d4ae700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-20T16:12:14.053 INFO:journalctl@ceph.mgr.x.smithi082.stdout:May 20 16:12:13 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mgr-x[32467]: 2024-05-20T16:12:13.739+0000 7f1a2d4ae700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-20T16:12:14.053 INFO:journalctl@ceph.mgr.x.smithi082.stdout:May 20 16:12:13 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mgr-x[32467]: 2024-05-20T16:12:13.739+0000 7f1a2d4ae700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-20T16:12:14.053 INFO:journalctl@ceph.mgr.x.smithi082.stdout:May 20 16:12:13 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mgr-x[32467]: 2024-05-20T16:12:13.739+0000 7f1a2d4ae700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-20T16:12:14.114 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:14.114+0000 7fcc58b1a640 1 -- 172.21.15.82:0/4181946218 --> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fcc54062020 con 0x7fcc3c0a2b90 2024-05-20T16:12:14.138 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:14.138+0000 7fcc437fe640 1 -- 172.21.15.82:0/4181946218 <== mgr.14150 v2:172.21.15.82:6800/379530823 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fcc54062020 con 0x7fcc3c0a2b90 2024-05-20T16:12:14.140 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:14.140+0000 7fcc58b1a640 1 -- 172.21.15.82:0/4181946218 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fcc3c0a2b90 msgr2=0x7fcc3c0a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:12:14.140 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:14.140+0000 7fcc58b1a640 1 --2- 172.21.15.82:0/4181946218 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fcc3c0a2b90 0x7fcc3c0a50a0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fcc5406a2a0 tx=0x7fcc44005f90 comp rx=0 tx=0).stop 2024-05-20T16:12:14.140 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:14.140+0000 7fcc58b1a640 1 -- 172.21.15.82:0/4181946218 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcc54101200 msgr2=0x7fcc54199760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:12:14.140 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:14.140+0000 7fcc58b1a640 1 --2- 172.21.15.82:0/4181946218 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcc54101200 0x7fcc54199760 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fcc4800d770 tx=0x7fcc4800dc30 comp rx=0 tx=0).stop 2024-05-20T16:12:14.140 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:14.140+0000 7fcc58b1a640 1 -- 172.21.15.82:0/4181946218 shutdown_connections 2024-05-20T16:12:14.140 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:14.140+0000 7fcc58b1a640 1 --2- 172.21.15.82:0/4181946218 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fcc3c0a2b90 0x7fcc3c0a50a0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:14.140 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:14.140+0000 7fcc58b1a640 1 --2- 172.21.15.82:0/4181946218 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcc54101200 0x7fcc54199760 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:14.140 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:14.140+0000 7fcc58b1a640 1 --2- 172.21.15.82:0/4181946218 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcc54104050 0x7fcc541a06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:14.141 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:14.140+0000 7fcc58b1a640 1 --2- 172.21.15.82:0/4181946218 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcc54100700 0x7fcc54199220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:14.141 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:14.140+0000 7fcc58b1a640 1 -- 172.21.15.82:0/4181946218 >> 172.21.15.82:0/4181946218 conn(0x7fcc540fc300 msgr2=0x7fcc54106bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:12:14.141 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:14.140+0000 7fcc58b1a640 1 -- 172.21.15.82:0/4181946218 shutdown_connections 2024-05-20T16:12:14.141 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:14.140+0000 7fcc58b1a640 1 -- 172.21.15.82:0/4181946218 wait complete. 2024-05-20T16:12:14.153 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-05-20T16:12:14.274 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:14.272+0000 7f4dd02eb640 1 -- 172.21.15.82:0/1890006967 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4dc8101ee0 msgr2=0x7f4dc81022f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:12:14.274 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:14.272+0000 7f4dd02eb640 1 --2- 172.21.15.82:0/1890006967 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4dc8101ee0 0x7f4dc81022f0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f4db8009900 tx=0x7f4db802a410 comp rx=0 tx=0).stop 2024-05-20T16:12:14.274 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:14.272+0000 7f4dd02eb640 1 -- 172.21.15.82:0/1890006967 shutdown_connections 2024-05-20T16:12:14.274 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:14.272+0000 7f4dd02eb640 1 --2- 172.21.15.82:0/1890006967 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4dc8103400 0x7f4dc8109b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:14.274 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:14.272+0000 7f4dd02eb640 1 --2- 172.21.15.82:0/1890006967 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4dc81028c0 0x7f4dc8102d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:14.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:14.272+0000 7f4dd02eb640 1 --2- 172.21.15.82:0/1890006967 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4dc8101ee0 0x7f4dc81022f0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:14.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:14.272+0000 7f4dd02eb640 1 -- 172.21.15.82:0/1890006967 >> 172.21.15.82:0/1890006967 conn(0x7f4dc80fc300 msgr2=0x7f4dc80fe7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:12:14.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:14.272+0000 7f4dd02eb640 1 -- 172.21.15.82:0/1890006967 shutdown_connections 2024-05-20T16:12:14.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:14.272+0000 7f4dd02eb640 1 -- 172.21.15.82:0/1890006967 wait complete. 2024-05-20T16:12:14.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:14.273+0000 7f4dd02eb640 1 Processor -- start 2024-05-20T16:12:14.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:14.273+0000 7f4dd02eb640 1 -- start start 2024-05-20T16:12:14.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:14.273+0000 7f4dd02eb640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4dc8101ee0 0x7f4dc819b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:12:14.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:14.273+0000 7f4dd02eb640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4dc81028c0 0x7f4dc819ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:12:14.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:14.273+0000 7f4dd02eb640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4dc8103400 0x7f4dc81a29b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:12:14.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:14.273+0000 7f4dd02eb640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f4dc810d5e0 con 0x7f4dc81028c0 2024-05-20T16:12:14.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:14.273+0000 7f4dd02eb640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f4dc810d460 con 0x7f4dc8101ee0 2024-05-20T16:12:14.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:14.273+0000 7f4dd02eb640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f4dc810d760 con 0x7f4dc8103400 2024-05-20T16:12:14.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:14.273+0000 7f4dcd886640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4dc81028c0 0x7f4dc819ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:12:14.276 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:14.273+0000 7f4dcd886640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4dc81028c0 0x7f4dc819ba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:47194/0 (socket says 172.21.15.82:47194) 2024-05-20T16:12:14.276 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:14.273+0000 7f4dcd886640 1 -- 172.21.15.82:0/3517370494 learned_addr learned my addr 172.21.15.82:0/3517370494 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:12:14.276 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:14.273+0000 7f4dce087640 1 --2- 172.21.15.82:0/3517370494 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4dc8101ee0 0x7f4dc819b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:12:14.276 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:14.273+0000 7f4dce888640 1 --2- 172.21.15.82:0/3517370494 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4dc8103400 0x7f4dc81a29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:12:14.276 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:14.273+0000 7f4dcd886640 1 -- 172.21.15.82:0/3517370494 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4dc8103400 msgr2=0x7f4dc81a29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:12:14.276 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:14.273+0000 7f4dcd886640 1 --2- 172.21.15.82:0/3517370494 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4dc8103400 0x7f4dc81a29b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:14.276 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:14.273+0000 7f4dcd886640 1 -- 172.21.15.82:0/3517370494 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4dc8101ee0 msgr2=0x7f4dc819b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:12:14.276 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:14.273+0000 7f4dcd886640 1 --2- 172.21.15.82:0/3517370494 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4dc8101ee0 0x7f4dc819b4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:14.276 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:14.273+0000 7f4dcd886640 1 -- 172.21.15.82:0/3517370494 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4dc81a3020 con 0x7f4dc81028c0 2024-05-20T16:12:14.276 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:14.273+0000 7f4dce888640 1 --2- 172.21.15.82:0/3517370494 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4dc8103400 0x7f4dc81a29b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T16:12:14.276 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:14.274+0000 7f4dcd886640 1 --2- 172.21.15.82:0/3517370494 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4dc81028c0 0x7f4dc819ba20 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f4dc400d770 tx=0x7f4dc400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:12:14.276 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:14.274+0000 7f4dbf7fe640 1 -- 172.21.15.82:0/3517370494 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dc4017070 con 0x7f4dc81028c0 2024-05-20T16:12:14.276 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:14.274+0000 7f4dbf7fe640 1 -- 172.21.15.82:0/3517370494 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4dc4005500 con 0x7f4dc81028c0 2024-05-20T16:12:14.276 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:14.274+0000 7f4dd02eb640 1 -- 172.21.15.82:0/3517370494 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4dc81a3310 con 0x7f4dc81028c0 2024-05-20T16:12:14.277 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:14.274+0000 7f4dbf7fe640 1 -- 172.21.15.82:0/3517370494 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dc4004e00 con 0x7f4dc81028c0 2024-05-20T16:12:14.277 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:14.274+0000 7f4dd02eb640 1 -- 172.21.15.82:0/3517370494 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4dc81a3810 con 0x7f4dc81028c0 2024-05-20T16:12:14.277 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:14.275+0000 7f4dd02eb640 1 -- 172.21.15.82:0/3517370494 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4dc8105160 con 0x7f4dc81028c0 2024-05-20T16:12:14.278 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:14.276+0000 7f4dbf7fe640 1 -- 172.21.15.82:0/3517370494 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f4dc4004fa0 con 0x7f4dc81028c0 2024-05-20T16:12:14.278 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:14.277+0000 7f4dbf7fe640 1 --2- 172.21.15.82:0/3517370494 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f4db40a2b90 0x7f4db40a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:12:14.278 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:14.277+0000 7f4dce087640 1 --2- 172.21.15.82:0/3517370494 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f4db40a2b90 0x7f4db40a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:12:14.278 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:14.277+0000 7f4dbf7fe640 1 -- 172.21.15.82:0/3517370494 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8802+0+0 (secure 0 0 0) 0x7f4dc40ced30 con 0x7f4dc81028c0 2024-05-20T16:12:14.278 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:14.277+0000 7f4dce087640 1 --2- 172.21.15.82:0/3517370494 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f4db40a2b90 0x7f4db40a50a0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f4db8005ab0 tx=0x7f4db8031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:12:14.281 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:14.282+0000 7f4dbf7fe640 1 -- 172.21.15.82:0/3517370494 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f4dc409ba50 con 0x7f4dc81028c0 2024-05-20T16:12:14.494 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:14.494+0000 7f4dd02eb640 1 -- 172.21.15.82:0/3517370494 --> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7f4dc8062020 con 0x7f4db40a2b90 2024-05-20T16:12:14.500 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:14.500+0000 7f4dbf7fe640 1 -- 172.21.15.82:0/3517370494 <== mgr.14150 v2:172.21.15.82:6800/379530823 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f4dc8062020 con 0x7f4db40a2b90 2024-05-20T16:12:14.500 INFO:teuthology.orchestra.run.smithi082.stdout:/volumes/qa/sv_0/47c42bfc-ec0b-4eae-878c-54feed2e6b3e 2024-05-20T16:12:14.502 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:14.502+0000 7f4dd02eb640 1 -- 172.21.15.82:0/3517370494 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f4db40a2b90 msgr2=0x7f4db40a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:12:14.502 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:14.502+0000 7f4dd02eb640 1 --2- 172.21.15.82:0/3517370494 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f4db40a2b90 0x7f4db40a50a0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f4db8005ab0 tx=0x7f4db8031400 comp rx=0 tx=0).stop 2024-05-20T16:12:14.502 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:14.502+0000 7f4dd02eb640 1 -- 172.21.15.82:0/3517370494 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4dc81028c0 msgr2=0x7f4dc819ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:12:14.502 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:14.502+0000 7f4dd02eb640 1 --2- 172.21.15.82:0/3517370494 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4dc81028c0 0x7f4dc819ba20 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f4dc400d770 tx=0x7f4dc400dc30 comp rx=0 tx=0).stop 2024-05-20T16:12:14.502 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:14.502+0000 7f4dd02eb640 1 -- 172.21.15.82:0/3517370494 shutdown_connections 2024-05-20T16:12:14.502 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:14.502+0000 7f4dd02eb640 1 --2- 172.21.15.82:0/3517370494 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f4db40a2b90 0x7f4db40a50a0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:14.502 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:14.502+0000 7f4dd02eb640 1 --2- 172.21.15.82:0/3517370494 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4dc81028c0 0x7f4dc819ba20 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:14.503 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:14.502+0000 7f4dd02eb640 1 --2- 172.21.15.82:0/3517370494 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4dc8103400 0x7f4dc81a29b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:14.503 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:14.502+0000 7f4dd02eb640 1 --2- 172.21.15.82:0/3517370494 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4dc8101ee0 0x7f4dc819b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:14.503 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:14.502+0000 7f4dd02eb640 1 -- 172.21.15.82:0/3517370494 >> 172.21.15.82:0/3517370494 conn(0x7f4dc80fc300 msgr2=0x7f4dc80fe720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:12:14.503 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:14.502+0000 7f4dd02eb640 1 -- 172.21.15.82:0/3517370494 shutdown_connections 2024-05-20T16:12:14.503 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:14.502+0000 7f4dd02eb640 1 -- 172.21.15.82:0/3517370494 wait complete. 2024-05-20T16:12:14.513 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_1 --group_name qa 2024-05-20T16:12:14.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:14.631+0000 7f97cb3e0640 1 -- 172.21.15.82:0/2781607509 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f97c4103590 msgr2=0x7f97c41059b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:12:14.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:14.631+0000 7f97cb3e0640 1 --2- 172.21.15.82:0/2781607509 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f97c4103590 0x7f97c41059b0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f97b8009900 tx=0x7f97b802a410 comp rx=0 tx=0).stop 2024-05-20T16:12:14.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:14.631+0000 7f97cb3e0640 1 -- 172.21.15.82:0/2781607509 shutdown_connections 2024-05-20T16:12:14.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:14.631+0000 7f97cb3e0640 1 --2- 172.21.15.82:0/2781607509 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f97c4108850 0x7f97c410ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:14.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:14.631+0000 7f97cb3e0640 1 --2- 172.21.15.82:0/2781607509 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f97c4105ef0 0x7f97c4108310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:14.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:14.631+0000 7f97cb3e0640 1 --2- 172.21.15.82:0/2781607509 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f97c4103590 0x7f97c41059b0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:14.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:14.631+0000 7f97cb3e0640 1 -- 172.21.15.82:0/2781607509 >> 172.21.15.82:0/2781607509 conn(0x7f97c40fcbb0 msgr2=0x7f97c40ff040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:12:14.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:14.631+0000 7f97cb3e0640 1 -- 172.21.15.82:0/2781607509 shutdown_connections 2024-05-20T16:12:14.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:14.631+0000 7f97cb3e0640 1 -- 172.21.15.82:0/2781607509 wait complete. 2024-05-20T16:12:14.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:14.631+0000 7f97cb3e0640 1 Processor -- start 2024-05-20T16:12:14.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:14.632+0000 7f97cb3e0640 1 -- start start 2024-05-20T16:12:14.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:14.632+0000 7f97cb3e0640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f97c4103590 0x7f97c41029e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:12:14.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:14.632+0000 7f97cb3e0640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f97c4105ef0 0x7f97c4101080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:12:14.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:14.632+0000 7f97cb3e0640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f97c4108850 0x7f97c41015c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:12:14.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:14.632+0000 7f97cb3e0640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f97c410e770 con 0x7f97c4108850 2024-05-20T16:12:14.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:14.632+0000 7f97cb3e0640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f97c410e5f0 con 0x7f97c4103590 2024-05-20T16:12:14.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:14.632+0000 7f97cb3e0640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f97c410e8f0 con 0x7f97c4105ef0 2024-05-20T16:12:14.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:14.632+0000 7f97c997d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f97c4108850 0x7f97c41015c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:12:14.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:14.632+0000 7f97c917c640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f97c4103590 0x7f97c41029e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:12:14.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:14.632+0000 7f97c997d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f97c4108850 0x7f97c41015c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:47208/0 (socket says 172.21.15.82:47208) 2024-05-20T16:12:14.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:14.632+0000 7f97c997d640 1 -- 172.21.15.82:0/3798536797 learned_addr learned my addr 172.21.15.82:0/3798536797 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:12:14.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:14.632+0000 7f97c897b640 1 --2- 172.21.15.82:0/3798536797 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f97c4105ef0 0x7f97c4101080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:12:14.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:14.632+0000 7f97c997d640 1 -- 172.21.15.82:0/3798536797 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f97c4105ef0 msgr2=0x7f97c4101080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:12:14.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:14.632+0000 7f97c997d640 1 --2- 172.21.15.82:0/3798536797 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f97c4105ef0 0x7f97c4101080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:14.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:14.632+0000 7f97c997d640 1 -- 172.21.15.82:0/3798536797 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f97c4103590 msgr2=0x7f97c41029e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:12:14.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:14.632+0000 7f97c997d640 1 --2- 172.21.15.82:0/3798536797 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f97c4103590 0x7f97c41029e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:14.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:14.632+0000 7f97c997d640 1 -- 172.21.15.82:0/3798536797 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97c4101cf0 con 0x7f97c4108850 2024-05-20T16:12:14.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:14.632+0000 7f97c997d640 1 --2- 172.21.15.82:0/3798536797 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f97c4108850 0x7f97c41015c0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f97bc00b9e0 tx=0x7f97bc00bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:12:14.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:14.633+0000 7f97b67fc640 1 -- 172.21.15.82:0/3798536797 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97bc019070 con 0x7f97c4108850 2024-05-20T16:12:14.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:14.633+0000 7f97b67fc640 1 -- 172.21.15.82:0/3798536797 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f97bc014440 con 0x7f97c4108850 2024-05-20T16:12:14.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:14.633+0000 7f97b67fc640 1 -- 172.21.15.82:0/3798536797 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97bc0125c0 con 0x7f97c4108850 2024-05-20T16:12:14.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:14.633+0000 7f97cb3e0640 1 -- 172.21.15.82:0/3798536797 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97c41a3410 con 0x7f97c4108850 2024-05-20T16:12:14.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:14.633+0000 7f97cb3e0640 1 -- 172.21.15.82:0/3798536797 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97c41a3950 con 0x7f97c4108850 2024-05-20T16:12:14.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:14.634+0000 7f97cb3e0640 1 -- 172.21.15.82:0/3798536797 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97c40fe6d0 con 0x7f97c4108850 2024-05-20T16:12:14.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:14.635+0000 7f97b67fc640 1 -- 172.21.15.82:0/3798536797 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f97bc01c070 con 0x7f97c4108850 2024-05-20T16:12:14.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:14.636+0000 7f97b67fc640 1 --2- 172.21.15.82:0/3798536797 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f97b00a2b90 0x7f97b00a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:12:14.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:14.636+0000 7f97c917c640 1 --2- 172.21.15.82:0/3798536797 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f97b00a2b90 0x7f97b00a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:12:14.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:14.636+0000 7f97b67fc640 1 -- 172.21.15.82:0/3798536797 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8802+0+0 (secure 0 0 0) 0x7f97bc0cedb0 con 0x7f97c4108850 2024-05-20T16:12:14.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:14.636+0000 7f97c917c640 1 --2- 172.21.15.82:0/3798536797 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f97b00a2b90 0x7f97b00a50a0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f97b8009230 tx=0x7f97b8031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:12:14.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:14.640+0000 7f97b67fc640 1 -- 172.21.15.82:0/3798536797 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f97bc09b480 con 0x7f97c4108850 2024-05-20T16:12:14.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:12:14 smithi160 ceph-mon[36347]: from='client.? 172.21.15.82:0/682875220' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-05-20T16:12:14.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:12:14 smithi160 ceph-mon[36347]: mds.? [v2:172.21.15.160:6836/2846188680,v1:172.21.15.160:6837/2846188680] up:active 2024-05-20T16:12:14.813 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:12:14 smithi160 ceph-mon[36347]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 9 up:standby 2024-05-20T16:12:14.813 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:12:14 smithi160 ceph-mon[36347]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-05-20T16:12:14.813 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:12:14 smithi160 ceph-mon[36347]: from='client.14889 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-20T16:12:14.813 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:12:14 smithi160 ceph-mon[36347]: from='client.14901 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-20T16:12:14.847 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:14.847+0000 7f97cb3e0640 1 -- 172.21.15.82:0/3798536797 --> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f97c4061f90 con 0x7f97b00a2b90 2024-05-20T16:12:14.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:12:14 smithi082 ceph-mon[32238]: from='client.? 172.21.15.82:0/682875220' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-05-20T16:12:14.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:12:14 smithi082 ceph-mon[32238]: mds.? [v2:172.21.15.160:6836/2846188680,v1:172.21.15.160:6837/2846188680] up:active 2024-05-20T16:12:14.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:12:14 smithi082 ceph-mon[32238]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 9 up:standby 2024-05-20T16:12:14.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:12:14 smithi082 ceph-mon[32238]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-05-20T16:12:14.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:12:14 smithi082 ceph-mon[32238]: from='client.14889 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-20T16:12:14.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:12:14 smithi082 ceph-mon[32238]: from='client.14901 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-20T16:12:14.876 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:14.877+0000 7f97b67fc640 1 -- 172.21.15.82:0/3798536797 <== mgr.14150 v2:172.21.15.82:6800/379530823 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f97c4061f90 con 0x7f97b00a2b90 2024-05-20T16:12:14.879 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:14.879+0000 7f97cb3e0640 1 -- 172.21.15.82:0/3798536797 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f97b00a2b90 msgr2=0x7f97b00a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:12:14.879 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:14.879+0000 7f97cb3e0640 1 --2- 172.21.15.82:0/3798536797 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f97b00a2b90 0x7f97b00a50a0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f97b8009230 tx=0x7f97b8031400 comp rx=0 tx=0).stop 2024-05-20T16:12:14.879 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:14.879+0000 7f97cb3e0640 1 -- 172.21.15.82:0/3798536797 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f97c4108850 msgr2=0x7f97c41015c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:12:14.879 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:14.879+0000 7f97cb3e0640 1 --2- 172.21.15.82:0/3798536797 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f97c4108850 0x7f97c41015c0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f97bc00b9e0 tx=0x7f97bc00bea0 comp rx=0 tx=0).stop 2024-05-20T16:12:14.879 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:14.879+0000 7f97cb3e0640 1 -- 172.21.15.82:0/3798536797 shutdown_connections 2024-05-20T16:12:14.879 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:14.879+0000 7f97cb3e0640 1 --2- 172.21.15.82:0/3798536797 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f97b00a2b90 0x7f97b00a50a0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:14.879 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:14.879+0000 7f97cb3e0640 1 --2- 172.21.15.82:0/3798536797 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f97c4108850 0x7f97c41015c0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:14.879 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:14.879+0000 7f97cb3e0640 1 --2- 172.21.15.82:0/3798536797 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f97c4105ef0 0x7f97c4101080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:14.879 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:14.879+0000 7f97cb3e0640 1 --2- 172.21.15.82:0/3798536797 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f97c4103590 0x7f97c41029e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:14.879 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:14.879+0000 7f97cb3e0640 1 -- 172.21.15.82:0/3798536797 >> 172.21.15.82:0/3798536797 conn(0x7f97c40fcbb0 msgr2=0x7f97c41044d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:12:14.879 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:14.879+0000 7f97cb3e0640 1 -- 172.21.15.82:0/3798536797 shutdown_connections 2024-05-20T16:12:14.879 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:14.879+0000 7f97cb3e0640 1 -- 172.21.15.82:0/3798536797 wait complete. 2024-05-20T16:12:14.885 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:12:14 smithi184 ceph-mon[36209]: from='client.? 172.21.15.82:0/682875220' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-05-20T16:12:14.886 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:12:14 smithi184 ceph-mon[36209]: mds.? [v2:172.21.15.160:6836/2846188680,v1:172.21.15.160:6837/2846188680] up:active 2024-05-20T16:12:14.886 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:12:14 smithi184 ceph-mon[36209]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 9 up:standby 2024-05-20T16:12:14.886 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:12:14 smithi184 ceph-mon[36209]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-05-20T16:12:14.886 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:12:14 smithi184 ceph-mon[36209]: from='client.14889 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-20T16:12:14.886 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:12:14 smithi184 ceph-mon[36209]: from='client.14901 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-20T16:12:14.890 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-05-20T16:12:15.011 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:15.007+0000 7f8ab8461640 1 -- 172.21.15.82:0/1188628556 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8ab0102bd0 msgr2=0x7f8ab0102fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:12:15.012 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:15.007+0000 7f8ab8461640 1 --2- 172.21.15.82:0/1188628556 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8ab0102bd0 0x7f8ab0102fe0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f8aac009900 tx=0x7f8aac02a410 comp rx=0 tx=0).stop 2024-05-20T16:12:15.012 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:15.007+0000 7f8ab8461640 1 -- 172.21.15.82:0/1188628556 shutdown_connections 2024-05-20T16:12:15.012 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:15.007+0000 7f8ab8461640 1 --2- 172.21.15.82:0/1188628556 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8ab0102bd0 0x7f8ab0102fe0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:15.012 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:15.007+0000 7f8ab8461640 1 --2- 172.21.15.82:0/1188628556 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8ab01078d0 0x7f8ab0109cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:15.012 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:15.007+0000 7f8ab8461640 1 --2- 172.21.15.82:0/1188628556 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8ab0101990 0x7f8ab0102690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:15.013 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:15.007+0000 7f8ab8461640 1 -- 172.21.15.82:0/1188628556 >> 172.21.15.82:0/1188628556 conn(0x7f8ab00fcb80 msgr2=0x7f8ab00ff030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:12:15.013 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:15.007+0000 7f8ab8461640 1 -- 172.21.15.82:0/1188628556 shutdown_connections 2024-05-20T16:12:15.013 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:15.007+0000 7f8ab8461640 1 -- 172.21.15.82:0/1188628556 wait complete. 2024-05-20T16:12:15.013 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:15.008+0000 7f8ab8461640 1 Processor -- start 2024-05-20T16:12:15.013 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:15.008+0000 7f8ab8461640 1 -- start start 2024-05-20T16:12:15.013 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:15.008+0000 7f8ab8461640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8ab0101990 0x7f8ab0069910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:12:15.013 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:15.008+0000 7f8ab8461640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8ab0102bd0 0x7f8ab019e300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:12:15.014 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:15.008+0000 7f8ab8461640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8ab01078d0 0x7f8ab01a08b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:12:15.014 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:15.008+0000 7f8ab8461640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f8ab010d610 con 0x7f8ab0102bd0 2024-05-20T16:12:15.014 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:15.008+0000 7f8ab8461640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f8ab010d490 con 0x7f8ab0101990 2024-05-20T16:12:15.014 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:15.008+0000 7f8ab8461640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f8ab010d790 con 0x7f8ab01078d0 2024-05-20T16:12:15.014 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:15.008+0000 7f8ab59fc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8ab0102bd0 0x7f8ab019e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:12:15.014 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:15.008+0000 7f8ab59fc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8ab0102bd0 0x7f8ab019e300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:47222/0 (socket says 172.21.15.82:47222) 2024-05-20T16:12:15.015 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:15.008+0000 7f8ab59fc640 1 -- 172.21.15.82:0/1635659928 learned_addr learned my addr 172.21.15.82:0/1635659928 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:12:15.015 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:15.008+0000 7f8ab61fd640 1 --2- 172.21.15.82:0/1635659928 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8ab0101990 0x7f8ab0069910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:12:15.015 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:15.008+0000 7f8ab69fe640 1 --2- 172.21.15.82:0/1635659928 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8ab01078d0 0x7f8ab01a08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:12:15.015 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:15.008+0000 7f8ab59fc640 1 -- 172.21.15.82:0/1635659928 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8ab01078d0 msgr2=0x7f8ab01a08b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:12:15.015 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:15.008+0000 7f8ab59fc640 1 --2- 172.21.15.82:0/1635659928 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8ab01078d0 0x7f8ab01a08b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:15.015 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:15.008+0000 7f8ab59fc640 1 -- 172.21.15.82:0/1635659928 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8ab0101990 msgr2=0x7f8ab0069910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:12:15.015 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:15.008+0000 7f8ab59fc640 1 --2- 172.21.15.82:0/1635659928 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8ab0101990 0x7f8ab0069910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:15.015 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:15.008+0000 7f8ab59fc640 1 -- 172.21.15.82:0/1635659928 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ab01a10d0 con 0x7f8ab0102bd0 2024-05-20T16:12:15.015 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:15.008+0000 7f8ab69fe640 1 --2- 172.21.15.82:0/1635659928 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8ab01078d0 0x7f8ab01a08b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T16:12:15.015 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:15.009+0000 7f8ab59fc640 1 --2- 172.21.15.82:0/1635659928 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8ab0102bd0 0x7f8ab019e300 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f8aac006010 tx=0x7f8aac004360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:12:15.015 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:15.009+0000 7f8aa37fe640 1 -- 172.21.15.82:0/1635659928 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8aac03f030 con 0x7f8ab0102bd0 2024-05-20T16:12:15.016 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:15.009+0000 7f8aa37fe640 1 -- 172.21.15.82:0/1635659928 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8aac003ce0 con 0x7f8ab0102bd0 2024-05-20T16:12:15.016 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:15.009+0000 7f8ab8461640 1 -- 172.21.15.82:0/1635659928 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ab01a1360 con 0x7f8ab0102bd0 2024-05-20T16:12:15.016 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:15.009+0000 7f8aa37fe640 1 -- 172.21.15.82:0/1635659928 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8aac035070 con 0x7f8ab0102bd0 2024-05-20T16:12:15.016 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:15.009+0000 7f8ab8461640 1 -- 172.21.15.82:0/1635659928 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ab01a1850 con 0x7f8ab0102bd0 2024-05-20T16:12:15.016 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:15.010+0000 7f8ab8461640 1 -- 172.21.15.82:0/1635659928 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ab01066e0 con 0x7f8ab0102bd0 2024-05-20T16:12:15.016 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:15.011+0000 7f8aa37fe640 1 -- 172.21.15.82:0/1635659928 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f8aac042030 con 0x7f8ab0102bd0 2024-05-20T16:12:15.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:15.012+0000 7f8aa37fe640 1 --2- 172.21.15.82:0/1635659928 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f8a9c0a2b90 0x7f8a9c0a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:12:15.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:15.012+0000 7f8ab61fd640 1 --2- 172.21.15.82:0/1635659928 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f8a9c0a2b90 0x7f8a9c0a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:12:15.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:15.012+0000 7f8aa37fe640 1 -- 172.21.15.82:0/1635659928 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8802+0+0 (secure 0 0 0) 0x7f8aac0efc80 con 0x7f8ab0102bd0 2024-05-20T16:12:15.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:15.013+0000 7f8ab61fd640 1 --2- 172.21.15.82:0/1635659928 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f8a9c0a2b90 0x7f8a9c0a50a0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f8ab0102590 tx=0x7f8aa400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:12:15.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:15.017+0000 7f8aa37fe640 1 -- 172.21.15.82:0/1635659928 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f8aac0bc9a0 con 0x7f8ab0102bd0 2024-05-20T16:12:15.222 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:15.222+0000 7f8ab8461640 1 -- 172.21.15.82:0/1635659928 --> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7f8ab002d3e0 con 0x7f8a9c0a2b90 2024-05-20T16:12:15.226 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:15.227+0000 7f8aa37fe640 1 -- 172.21.15.82:0/1635659928 <== mgr.14150 v2:172.21.15.82:6800/379530823 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f8ab002d3e0 con 0x7f8a9c0a2b90 2024-05-20T16:12:15.227 INFO:teuthology.orchestra.run.smithi082.stdout:/volumes/qa/sv_1/02a0ab78-ccb9-4d90-ba91-3f79600eded7 2024-05-20T16:12:15.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:15.229+0000 7f8ab8461640 1 -- 172.21.15.82:0/1635659928 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f8a9c0a2b90 msgr2=0x7f8a9c0a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:12:15.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:15.229+0000 7f8ab8461640 1 --2- 172.21.15.82:0/1635659928 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f8a9c0a2b90 0x7f8a9c0a50a0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f8ab0102590 tx=0x7f8aa400b390 comp rx=0 tx=0).stop 2024-05-20T16:12:15.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:15.229+0000 7f8ab8461640 1 -- 172.21.15.82:0/1635659928 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8ab0102bd0 msgr2=0x7f8ab019e300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:12:15.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:15.229+0000 7f8ab8461640 1 --2- 172.21.15.82:0/1635659928 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8ab0102bd0 0x7f8ab019e300 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f8aac006010 tx=0x7f8aac004360 comp rx=0 tx=0).stop 2024-05-20T16:12:15.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:15.229+0000 7f8ab8461640 1 -- 172.21.15.82:0/1635659928 shutdown_connections 2024-05-20T16:12:15.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:15.229+0000 7f8ab8461640 1 --2- 172.21.15.82:0/1635659928 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f8a9c0a2b90 0x7f8a9c0a50a0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:15.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:15.229+0000 7f8ab8461640 1 --2- 172.21.15.82:0/1635659928 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8ab0102bd0 0x7f8ab019e300 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:15.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:15.229+0000 7f8ab8461640 1 --2- 172.21.15.82:0/1635659928 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8ab01078d0 0x7f8ab01a08b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:15.230 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:15.229+0000 7f8ab8461640 1 --2- 172.21.15.82:0/1635659928 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8ab0101990 0x7f8ab0069910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:15.230 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:15.229+0000 7f8ab8461640 1 -- 172.21.15.82:0/1635659928 >> 172.21.15.82:0/1635659928 conn(0x7f8ab00fcb80 msgr2=0x7f8ab01038b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:12:15.230 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:15.229+0000 7f8ab8461640 1 -- 172.21.15.82:0/1635659928 shutdown_connections 2024-05-20T16:12:15.230 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:15.229+0000 7f8ab8461640 1 -- 172.21.15.82:0/1635659928 wait complete. 2024-05-20T16:12:15.240 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-20T16:12:15.351 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:15.351+0000 7f8c82164640 1 -- 172.21.15.82:0/2449980566 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8c7c103560 msgr2=0x7f8c7c105980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:12:15.352 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:15.351+0000 7f8c82164640 1 --2- 172.21.15.82:0/2449980566 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8c7c103560 0x7f8c7c105980 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f8c6c009900 tx=0x7f8c6c02a410 comp rx=0 tx=0).stop 2024-05-20T16:12:15.352 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:15.351+0000 7f8c82164640 1 -- 172.21.15.82:0/2449980566 shutdown_connections 2024-05-20T16:12:15.352 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:15.351+0000 7f8c82164640 1 --2- 172.21.15.82:0/2449980566 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8c7c108820 0x7f8c7c10ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:15.352 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:15.351+0000 7f8c82164640 1 --2- 172.21.15.82:0/2449980566 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8c7c105ec0 0x7f8c7c1082e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:15.352 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:15.351+0000 7f8c82164640 1 --2- 172.21.15.82:0/2449980566 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8c7c103560 0x7f8c7c105980 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:15.352 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:15.351+0000 7f8c82164640 1 -- 172.21.15.82:0/2449980566 >> 172.21.15.82:0/2449980566 conn(0x7f8c7c0fcb80 msgr2=0x7f8c7c0ff010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:12:15.352 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:15.351+0000 7f8c82164640 1 -- 172.21.15.82:0/2449980566 shutdown_connections 2024-05-20T16:12:15.352 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:15.351+0000 7f8c82164640 1 -- 172.21.15.82:0/2449980566 wait complete. 2024-05-20T16:12:15.352 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:15.352+0000 7f8c82164640 1 Processor -- start 2024-05-20T16:12:15.354 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:15.352+0000 7f8c82164640 1 -- start start 2024-05-20T16:12:15.355 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:15.352+0000 7f8c82164640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8c7c103560 0x7f8c7c199270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:12:15.355 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:15.352+0000 7f8c82164640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8c7c105ec0 0x7f8c7c1997b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:12:15.355 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:15.352+0000 7f8c82164640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8c7c108820 0x7f8c7c1a0740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:12:15.355 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:15.352+0000 7f8c82164640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f8c7c10e6a0 con 0x7f8c7c105ec0 2024-05-20T16:12:15.355 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:15.352+0000 7f8c82164640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f8c7c10e520 con 0x7f8c7c108820 2024-05-20T16:12:15.355 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:15.352+0000 7f8c82164640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f8c7c10e820 con 0x7f8c7c103560 2024-05-20T16:12:15.355 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:15.352+0000 7f8c7affd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8c7c105ec0 0x7f8c7c1997b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:12:15.355 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:15.353+0000 7f8c7affd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8c7c105ec0 0x7f8c7c1997b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:47232/0 (socket says 172.21.15.82:47232) 2024-05-20T16:12:15.355 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:15.353+0000 7f8c7affd640 1 -- 172.21.15.82:0/1454241362 learned_addr learned my addr 172.21.15.82:0/1454241362 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:12:15.355 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:15.353+0000 7f8c7bfff640 1 --2- 172.21.15.82:0/1454241362 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8c7c108820 0x7f8c7c1a0740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:12:15.356 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:15.353+0000 7f8c7b7fe640 1 --2- 172.21.15.82:0/1454241362 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8c7c103560 0x7f8c7c199270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:12:15.356 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:15.353+0000 7f8c7affd640 1 -- 172.21.15.82:0/1454241362 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8c7c103560 msgr2=0x7f8c7c199270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:12:15.356 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:15.353+0000 7f8c7affd640 1 --2- 172.21.15.82:0/1454241362 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8c7c103560 0x7f8c7c199270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:15.356 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:15.353+0000 7f8c7affd640 1 -- 172.21.15.82:0/1454241362 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8c7c108820 msgr2=0x7f8c7c1a0740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:12:15.356 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:15.353+0000 7f8c7affd640 1 --2- 172.21.15.82:0/1454241362 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8c7c108820 0x7f8c7c1a0740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:15.356 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:15.353+0000 7f8c7affd640 1 -- 172.21.15.82:0/1454241362 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c7c1a0d00 con 0x7f8c7c105ec0 2024-05-20T16:12:15.356 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:15.353+0000 7f8c7affd640 1 --2- 172.21.15.82:0/1454241362 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8c7c105ec0 0x7f8c7c1997b0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f8c7000c920 tx=0x7f8c7000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:12:15.356 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:15.353+0000 7f8c78ff9640 1 -- 172.21.15.82:0/1454241362 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c70007c70 con 0x7f8c7c105ec0 2024-05-20T16:12:15.356 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:15.353+0000 7f8c78ff9640 1 -- 172.21.15.82:0/1454241362 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c70007e10 con 0x7f8c7c105ec0 2024-05-20T16:12:15.357 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:15.353+0000 7f8c82164640 1 -- 172.21.15.82:0/1454241362 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c7c1a0ff0 con 0x7f8c7c105ec0 2024-05-20T16:12:15.357 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:15.353+0000 7f8c78ff9640 1 -- 172.21.15.82:0/1454241362 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c70018890 con 0x7f8c7c105ec0 2024-05-20T16:12:15.357 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:15.353+0000 7f8c82164640 1 -- 172.21.15.82:0/1454241362 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c7c1a1460 con 0x7f8c7c105ec0 2024-05-20T16:12:15.357 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:15.354+0000 7f8c82164640 1 -- 172.21.15.82:0/1454241362 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c7c0fe6a0 con 0x7f8c7c105ec0 2024-05-20T16:12:15.357 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:15.356+0000 7f8c78ff9640 1 -- 172.21.15.82:0/1454241362 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f8c70018a50 con 0x7f8c7c105ec0 2024-05-20T16:12:15.358 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:15.357+0000 7f8c78ff9640 1 --2- 172.21.15.82:0/1454241362 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f8c640a2b90 0x7f8c640a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:12:15.358 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:15.357+0000 7f8c7b7fe640 1 --2- 172.21.15.82:0/1454241362 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f8c640a2b90 0x7f8c640a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:12:15.358 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:15.357+0000 7f8c78ff9640 1 -- 172.21.15.82:0/1454241362 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8802+0+0 (secure 0 0 0) 0x7f8c700cf720 con 0x7f8c7c105ec0 2024-05-20T16:12:15.358 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:15.357+0000 7f8c7b7fe640 1 --2- 172.21.15.82:0/1454241362 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f8c640a2b90 0x7f8c640a50a0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f8c6c02a930 tx=0x7f8c6c000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:12:15.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:15.361+0000 7f8c78ff9640 1 -- 172.21.15.82:0/1454241362 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f8c70014030 con 0x7f8c7c105ec0 2024-05-20T16:12:15.414 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:15.415+0000 7f8c78ff9640 1 -- 172.21.15.82:0/1454241362 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f8c70098440 con 0x7f8c7c105ec0 2024-05-20T16:12:15.587 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:15.587+0000 7f8c82164640 1 -- 172.21.15.82:0/1454241362 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8c7c02d2a0 con 0x7f8c7c105ec0 2024-05-20T16:12:15.588 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:15.588+0000 7f8c78ff9640 1 -- 172.21.15.82:0/1454241362 <== mon.0 v2:172.21.15.82:3300/0 8 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11879 (secure 0 0 0) 0x7f8c700c7320 con 0x7f8c7c105ec0 2024-05-20T16:12:15.588 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-05-20T16:12:15.588 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/2044534217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":2044534217},{"type":"v1","addr":"172.21.15.82:6835","nonce":2044534217}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/2635635097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":2635635097},{"type":"v1","addr":"172.21.15.82:6837","nonce":2635635097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14625,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6839/2118233541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6838","nonce":2118233541},{"type":"v1","addr":"172.21.15.82:6839","nonce":2118233541}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14637,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6841/2119517973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6840","nonce":2119517973},{"type":"v1","addr":"172.21.15.82:6841","nonce":2119517973}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6833/4015144007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":4015144007},{"type":"v1","addr":"172.21.15.184:6833","nonce":4015144007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6833/975243372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6832","nonce":975243372},{"type":"v1","addr":"172.21.15.160:6833","nonce":975243372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-20T16:11:46.751632+0000","modified":"2024-05-20T16:12:13.394770+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24548,"mds_1":24511,"mds_2":24506},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24470":{"gid":24470,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.160:6835/664068363","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":664068363},{"type":"v1","addr":"172.21.15.160:6835","nonce":664068363}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6835/4242156090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":4242156090},{"type":"v1","addr":"172.21.15.184:6835","nonce":4242156090}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6837/403262242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":403262242},{"type":"v1","addr":"172.21.15.184:6837","nonce":403262242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.160:6837/2846188680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6836","nonce":2846188680},{"type":"v1","addr":"172.21.15.160:6837","nonce":2846188680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.184:6839/2199955067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":2199955067},{"type":"v1","addr":"172.21.15.184:6839","nonce":2199955067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24548":{"gid":24548,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.160:6839/2765554482","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6838","nonce":2765554482},{"type":"v1","addr":"172.21.15.160:6839","nonce":2765554482}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24548,"qdb_cluster":[24511,24506,24548]},"id":1}]} 2024-05-20T16:12:15.588 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 24 2024-05-20T16:12:15.590 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:15.590+0000 7f8c82164640 1 -- 172.21.15.82:0/1454241362 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f8c640a2b90 msgr2=0x7f8c640a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:12:15.590 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:15.590+0000 7f8c82164640 1 --2- 172.21.15.82:0/1454241362 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f8c640a2b90 0x7f8c640a50a0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f8c6c02a930 tx=0x7f8c6c000bc0 comp rx=0 tx=0).stop 2024-05-20T16:12:15.590 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:15.590+0000 7f8c82164640 1 -- 172.21.15.82:0/1454241362 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8c7c105ec0 msgr2=0x7f8c7c1997b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:12:15.590 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:15.590+0000 7f8c82164640 1 --2- 172.21.15.82:0/1454241362 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8c7c105ec0 0x7f8c7c1997b0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f8c7000c920 tx=0x7f8c7000cde0 comp rx=0 tx=0).stop 2024-05-20T16:12:15.590 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:15.590+0000 7f8c82164640 1 -- 172.21.15.82:0/1454241362 shutdown_connections 2024-05-20T16:12:15.590 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:15.590+0000 7f8c82164640 1 --2- 172.21.15.82:0/1454241362 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f8c640a2b90 0x7f8c640a50a0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:15.590 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:15.590+0000 7f8c82164640 1 --2- 172.21.15.82:0/1454241362 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8c7c105ec0 0x7f8c7c1997b0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:15.590 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:15.590+0000 7f8c82164640 1 --2- 172.21.15.82:0/1454241362 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8c7c108820 0x7f8c7c1a0740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:15.591 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:15.590+0000 7f8c82164640 1 --2- 172.21.15.82:0/1454241362 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8c7c103560 0x7f8c7c199270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:15.591 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:15.590+0000 7f8c82164640 1 -- 172.21.15.82:0/1454241362 >> 172.21.15.82:0/1454241362 conn(0x7f8c7c0fcb80 msgr2=0x7f8c7c1041f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:12:15.591 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:15.590+0000 7f8c82164640 1 -- 172.21.15.82:0/1454241362 shutdown_connections 2024-05-20T16:12:15.591 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:15.590+0000 7f8c82164640 1 -- 172.21.15.82:0/1454241362 wait complete. 2024-05-20T16:12:15.600 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:12:15 smithi082 ceph-mon[32238]: from='client.14907 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-05-20T16:12:15.600 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:12:15 smithi082 ceph-mon[32238]: from='client.14913 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-20T16:12:15.600 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:12:15 smithi082 ceph-mon[32238]: from='client.14919 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-20T16:12:15.600 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:12:15 smithi082 ceph-mon[32238]: pgmap v306: 129 pgs: 129 active+clean; 584 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s rd, 2.2 KiB/s wr, 6 op/s 2024-05-20T16:12:15.602 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-20T16:12:15.715 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:15.713+0000 7fddfa3f5640 1 -- 172.21.15.82:0/1058690071 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fddf4101980 msgr2=0x7fddf4102680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:12:15.715 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:15.713+0000 7fddfa3f5640 1 --2- 172.21.15.82:0/1058690071 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fddf4101980 0x7fddf4102680 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fdde4009ca0 tx=0x7fdde402a8c0 comp rx=0 tx=0).stop 2024-05-20T16:12:15.715 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:15.713+0000 7fddfa3f5640 1 -- 172.21.15.82:0/1058690071 shutdown_connections 2024-05-20T16:12:15.715 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:15.713+0000 7fddfa3f5640 1 --2- 172.21.15.82:0/1058690071 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fddf41078c0 0x7fddf4109ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:15.715 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:15.713+0000 7fddfa3f5640 1 --2- 172.21.15.82:0/1058690071 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fddf4102bc0 0x7fddf4102fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:15.715 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:15.713+0000 7fddfa3f5640 1 --2- 172.21.15.82:0/1058690071 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fddf4101980 0x7fddf4102680 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:15.715 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:15.713+0000 7fddfa3f5640 1 -- 172.21.15.82:0/1058690071 >> 172.21.15.82:0/1058690071 conn(0x7fddf40fcb50 msgr2=0x7fddf40ff020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:12:15.716 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:15.713+0000 7fddfa3f5640 1 -- 172.21.15.82:0/1058690071 shutdown_connections 2024-05-20T16:12:15.716 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:15.713+0000 7fddfa3f5640 1 -- 172.21.15.82:0/1058690071 wait complete. 2024-05-20T16:12:15.716 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:15.714+0000 7fddfa3f5640 1 Processor -- start 2024-05-20T16:12:15.716 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:15.714+0000 7fddfa3f5640 1 -- start start 2024-05-20T16:12:15.716 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:15.714+0000 7fddfa3f5640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fddf4101980 0x7fddf406acf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:12:15.716 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:15.714+0000 7fddfa3f5640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fddf4102bc0 0x7fddf419e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:12:15.716 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:15.714+0000 7fddfa3f5640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fddf41078c0 0x7fddf41a0610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:12:15.716 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:15.714+0000 7fddfa3f5640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fddf410d630 con 0x7fddf41078c0 2024-05-20T16:12:15.716 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:15.714+0000 7fddfa3f5640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fddf410d4b0 con 0x7fddf4101980 2024-05-20T16:12:15.716 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:15.714+0000 7fddfa3f5640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fddf410d7b0 con 0x7fddf4102bc0 2024-05-20T16:12:15.716 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:15.714+0000 7fddf8992640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fddf41078c0 0x7fddf41a0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:12:15.716 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:15.714+0000 7fddf8992640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fddf41078c0 0x7fddf41a0610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:47250/0 (socket says 172.21.15.82:47250) 2024-05-20T16:12:15.716 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:15.714+0000 7fddf8992640 1 -- 172.21.15.82:0/1445240663 learned_addr learned my addr 172.21.15.82:0/1445240663 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:12:15.716 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:15.714+0000 7fddf3fff640 1 --2- 172.21.15.82:0/1445240663 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fddf4101980 0x7fddf406acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:12:15.717 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:15.714+0000 7fdde3fff640 1 --2- 172.21.15.82:0/1445240663 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fddf4102bc0 0x7fddf419e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:12:15.717 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:15.714+0000 7fddf8992640 1 -- 172.21.15.82:0/1445240663 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fddf4102bc0 msgr2=0x7fddf419e0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:12:15.717 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:15.714+0000 7fddf8992640 1 --2- 172.21.15.82:0/1445240663 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fddf4102bc0 0x7fddf419e0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:15.717 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:15.714+0000 7fddf8992640 1 -- 172.21.15.82:0/1445240663 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fddf4101980 msgr2=0x7fddf406acf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:12:15.717 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:15.714+0000 7fddf8992640 1 --2- 172.21.15.82:0/1445240663 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fddf4101980 0x7fddf406acf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:15.717 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:15.714+0000 7fddf8992640 1 -- 172.21.15.82:0/1445240663 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddf41a0e30 con 0x7fddf41078c0 2024-05-20T16:12:15.717 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:15.714+0000 7fddf8992640 1 --2- 172.21.15.82:0/1445240663 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fddf41078c0 0x7fddf41a0610 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fddec00d750 tx=0x7fddec00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:12:15.717 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:15.715+0000 7fddf1ffb640 1 -- 172.21.15.82:0/1445240663 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddec017070 con 0x7fddf41078c0 2024-05-20T16:12:15.717 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:15.715+0000 7fddf1ffb640 1 -- 172.21.15.82:0/1445240663 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fddec005500 con 0x7fddf41078c0 2024-05-20T16:12:15.717 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:15.715+0000 7fddfa3f5640 1 -- 172.21.15.82:0/1445240663 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fddf41a1120 con 0x7fddf41078c0 2024-05-20T16:12:15.717 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:15.715+0000 7fddf1ffb640 1 -- 172.21.15.82:0/1445240663 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddec004d90 con 0x7fddf41078c0 2024-05-20T16:12:15.717 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:15.715+0000 7fddfa3f5640 1 -- 172.21.15.82:0/1445240663 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fddf41a16a0 con 0x7fddf41078c0 2024-05-20T16:12:15.718 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:15.716+0000 7fddfa3f5640 1 -- 172.21.15.82:0/1445240663 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fddf41066d0 con 0x7fddf41078c0 2024-05-20T16:12:15.718 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:15.717+0000 7fddf1ffb640 1 -- 172.21.15.82:0/1445240663 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fddec004f30 con 0x7fddf41078c0 2024-05-20T16:12:15.718 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:15.718+0000 7fddf1ffb640 1 --2- 172.21.15.82:0/1445240663 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fdddc0a2c20 0x7fdddc0a5130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:12:15.718 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:15.718+0000 7fddf1ffb640 1 -- 172.21.15.82:0/1445240663 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8802+0+0 (secure 0 0 0) 0x7fddec0cf200 con 0x7fddf41078c0 2024-05-20T16:12:15.718 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:15.718+0000 7fddf3fff640 1 --2- 172.21.15.82:0/1445240663 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fdddc0a2c20 0x7fdddc0a5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:12:15.719 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:15.719+0000 7fddf3fff640 1 --2- 172.21.15.82:0/1445240663 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fdddc0a2c20 0x7fdddc0a5130 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fdde4000f80 tx=0x7fdde4005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:12:15.721 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:15.722+0000 7fddf1ffb640 1 -- 172.21.15.82:0/1445240663 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fddec09bdc0 con 0x7fddf41078c0 2024-05-20T16:12:15.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:12:15 smithi160 ceph-mon[36347]: from='client.14907 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-05-20T16:12:15.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:12:15 smithi160 ceph-mon[36347]: from='client.14913 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-20T16:12:15.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:12:15 smithi160 ceph-mon[36347]: from='client.14919 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-20T16:12:15.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:12:15 smithi160 ceph-mon[36347]: pgmap v306: 129 pgs: 129 active+clean; 584 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s rd, 2.2 KiB/s wr, 6 op/s 2024-05-20T16:12:15.886 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:12:15 smithi184 ceph-mon[36209]: from='client.14907 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-05-20T16:12:15.886 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:12:15 smithi184 ceph-mon[36209]: from='client.14913 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-20T16:12:15.886 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:12:15 smithi184 ceph-mon[36209]: from='client.14919 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-20T16:12:15.886 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:12:15 smithi184 ceph-mon[36209]: pgmap v306: 129 pgs: 129 active+clean; 584 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s rd, 2.2 KiB/s wr, 6 op/s 2024-05-20T16:12:15.898 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:15.899+0000 7fddfa3f5640 1 -- 172.21.15.82:0/1445240663 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fddf402d470 con 0x7fddf41078c0 2024-05-20T16:12:15.900 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:15.900+0000 7fddf1ffb640 1 -- 172.21.15.82:0/1445240663 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+24153 (secure 0 0 0) 0x7fddec09b6c0 con 0x7fddf41078c0 2024-05-20T16:12:15.901 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-05-20T16:12:15.901 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":91,"fsid":"59d1600a-16c2-11ef-bc9a-c7b262605968","created":"2024-05-20T16:03:46.611556+0000","modified":"2024-05-20T16:11:53.416085+0000","last_up_change":"2024-05-20T16:10:59.303479+0000","last_in_change":"2024-05-20T16:10:39.063561+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-20T16:07:09.493727+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-20T16:11:45.058035+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-20T16:11:45.943862+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-20T16:11:49.637579+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"bae842a9-03ec-490e-b56d-f7957235cdc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":1484330167},{"type":"v1","addr":"172.21.15.82:6803","nonce":1484330167}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":1484330167},{"type":"v1","addr":"172.21.15.82:6805","nonce":1484330167}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1484330167},{"type":"v1","addr":"172.21.15.82:6809","nonce":1484330167}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":1484330167},{"type":"v1","addr":"172.21.15.82:6807","nonce":1484330167}]},"public_addr":"172.21.15.82:6803/1484330167","cluster_addr":"172.21.15.82:6805/1484330167","heartbeat_back_addr":"172.21.15.82:6809/1484330167","heartbeat_front_addr":"172.21.15.82:6807/1484330167","state":["exists","up"]},{"osd":1,"uuid":"59f6a2ba-b5de-4161-81c5-92a417abacdf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":1457003568},{"type":"v1","addr":"172.21.15.160:6801","nonce":1457003568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":1457003568},{"type":"v1","addr":"172.21.15.160:6803","nonce":1457003568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6806","nonce":1457003568},{"type":"v1","addr":"172.21.15.160:6807","nonce":1457003568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":1457003568},{"type":"v1","addr":"172.21.15.160:6805","nonce":1457003568}]},"public_addr":"172.21.15.160:6801/1457003568","cluster_addr":"172.21.15.160:6803/1457003568","heartbeat_back_addr":"172.21.15.160:6807/1457003568","heartbeat_front_addr":"172.21.15.160:6805/1457003568","state":["exists","up"]},{"osd":2,"uuid":"3bbb5587-b449-41f3-bd26-67ec3d6afdfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":4051293934},{"type":"v1","addr":"172.21.15.184:6801","nonce":4051293934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":4051293934},{"type":"v1","addr":"172.21.15.184:6803","nonce":4051293934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":4051293934},{"type":"v1","addr":"172.21.15.184:6807","nonce":4051293934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":4051293934},{"type":"v1","addr":"172.21.15.184:6805","nonce":4051293934}]},"public_addr":"172.21.15.184:6801/4051293934","cluster_addr":"172.21.15.184:6803/4051293934","heartbeat_back_addr":"172.21.15.184:6807/4051293934","heartbeat_front_addr":"172.21.15.184:6805/4051293934","state":["exists","up"]},{"osd":3,"uuid":"ca651f90-09b8-4081-a6f9-40b31268a837","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":3947485599},{"type":"v1","addr":"172.21.15.82:6811","nonce":3947485599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":3947485599},{"type":"v1","addr":"172.21.15.82:6813","nonce":3947485599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":3947485599},{"type":"v1","addr":"172.21.15.82:6817","nonce":3947485599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":3947485599},{"type":"v1","addr":"172.21.15.82:6815","nonce":3947485599}]},"public_addr":"172.21.15.82:6811/3947485599","cluster_addr":"172.21.15.82:6813/3947485599","heartbeat_back_addr":"172.21.15.82:6817/3947485599","heartbeat_front_addr":"172.21.15.82:6815/3947485599","state":["exists","up"]},{"osd":4,"uuid":"cb3dfd56-f677-40cb-9c96-8319ea5ac788","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":4160830014},{"type":"v1","addr":"172.21.15.160:6809","nonce":4160830014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":4160830014},{"type":"v1","addr":"172.21.15.160:6811","nonce":4160830014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":4160830014},{"type":"v1","addr":"172.21.15.160:6815","nonce":4160830014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6812","nonce":4160830014},{"type":"v1","addr":"172.21.15.160:6813","nonce":4160830014}]},"public_addr":"172.21.15.160:6809/4160830014","cluster_addr":"172.21.15.160:6811/4160830014","heartbeat_back_addr":"172.21.15.160:6815/4160830014","heartbeat_front_addr":"172.21.15.160:6813/4160830014","state":["exists","up"]},{"osd":5,"uuid":"c70cfcd1-a4ff-461f-8c73-479b061200fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":1046282672},{"type":"v1","addr":"172.21.15.184:6809","nonce":1046282672}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":1046282672},{"type":"v1","addr":"172.21.15.184:6811","nonce":1046282672}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":1046282672},{"type":"v1","addr":"172.21.15.184:6815","nonce":1046282672}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":1046282672},{"type":"v1","addr":"172.21.15.184:6813","nonce":1046282672}]},"public_addr":"172.21.15.184:6809/1046282672","cluster_addr":"172.21.15.184:6811/1046282672","heartbeat_back_addr":"172.21.15.184:6815/1046282672","heartbeat_front_addr":"172.21.15.184:6813/1046282672","state":["exists","up"]},{"osd":6,"uuid":"d47ce001-9dd1-4830-bcc9-888601187c6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":2789736499},{"type":"v1","addr":"172.21.15.82:6819","nonce":2789736499}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":2789736499},{"type":"v1","addr":"172.21.15.82:6821","nonce":2789736499}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":2789736499},{"type":"v1","addr":"172.21.15.82:6825","nonce":2789736499}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":2789736499},{"type":"v1","addr":"172.21.15.82:6823","nonce":2789736499}]},"public_addr":"172.21.15.82:6819/2789736499","cluster_addr":"172.21.15.82:6821/2789736499","heartbeat_back_addr":"172.21.15.82:6825/2789736499","heartbeat_front_addr":"172.21.15.82:6823/2789736499","state":["exists","up"]},{"osd":7,"uuid":"25c54f43-18cb-41e4-83b4-2732b5a93889","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6816","nonce":2340683378},{"type":"v1","addr":"172.21.15.160:6817","nonce":2340683378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":2340683378},{"type":"v1","addr":"172.21.15.160:6819","nonce":2340683378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":2340683378},{"type":"v1","addr":"172.21.15.160:6823","nonce":2340683378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6820","nonce":2340683378},{"type":"v1","addr":"172.21.15.160:6821","nonce":2340683378}]},"public_addr":"172.21.15.160:6817/2340683378","cluster_addr":"172.21.15.160:6819/2340683378","heartbeat_back_addr":"172.21.15.160:6823/2340683378","heartbeat_front_addr":"172.21.15.160:6821/2340683378","state":["exists","up"]},{"osd":8,"uuid":"1f8b478d-287f-45f5-8526-d1a58c5bc07c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":3831576372},{"type":"v1","addr":"172.21.15.184:6817","nonce":3831576372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":3831576372},{"type":"v1","addr":"172.21.15.184:6819","nonce":3831576372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":3831576372},{"type":"v1","addr":"172.21.15.184:6823","nonce":3831576372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":3831576372},{"type":"v1","addr":"172.21.15.184:6821","nonce":3831576372}]},"public_addr":"172.21.15.184:6817/3831576372","cluster_addr":"172.21.15.184:6819/3831576372","heartbeat_back_addr":"172.21.15.184:6823/3831576372","heartbeat_front_addr":"172.21.15.184:6821/3831576372","state":["exists","up"]},{"osd":9,"uuid":"0d69e319-954f-4b18-95ea-31794f18dc3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":716330782},{"type":"v1","addr":"172.21.15.82:6827","nonce":716330782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":716330782},{"type":"v1","addr":"172.21.15.82:6829","nonce":716330782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":716330782},{"type":"v1","addr":"172.21.15.82:6833","nonce":716330782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":716330782},{"type":"v1","addr":"172.21.15.82:6831","nonce":716330782}]},"public_addr":"172.21.15.82:6827/716330782","cluster_addr":"172.21.15.82:6829/716330782","heartbeat_back_addr":"172.21.15.82:6833/716330782","heartbeat_front_addr":"172.21.15.82:6831/716330782","state":["exists","up"]},{"osd":10,"uuid":"39dc1b78-61cb-42ca-b5b6-1e44ac2b6500","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":2327428014},{"type":"v1","addr":"172.21.15.160:6825","nonce":2327428014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":2327428014},{"type":"v1","addr":"172.21.15.160:6827","nonce":2327428014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":2327428014},{"type":"v1","addr":"172.21.15.160:6831","nonce":2327428014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":2327428014},{"type":"v1","addr":"172.21.15.160:6829","nonce":2327428014}]},"public_addr":"172.21.15.160:6825/2327428014","cluster_addr":"172.21.15.160:6827/2327428014","heartbeat_back_addr":"172.21.15.160:6831/2327428014","heartbeat_front_addr":"172.21.15.160:6829/2327428014","state":["exists","up"]},{"osd":11,"uuid":"4306b3c1-707a-4010-987c-209ea0e948e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":3594364650},{"type":"v1","addr":"172.21.15.184:6825","nonce":3594364650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":3594364650},{"type":"v1","addr":"172.21.15.184:6827","nonce":3594364650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3594364650},{"type":"v1","addr":"172.21.15.184:6831","nonce":3594364650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3594364650},{"type":"v1","addr":"172.21.15.184:6829","nonce":3594364650}]},"public_addr":"172.21.15.184:6825/3594364650","cluster_addr":"172.21.15.184:6827/3594364650","heartbeat_back_addr":"172.21.15.184:6831/3594364650","heartbeat_front_addr":"172.21.15.184:6829/3594364650","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:06:17.855998+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:06:41.473270+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:07:05.304108+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:07:29.887472+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:07:55.387286+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:08:20.579669+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:08:46.333051+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:09:12.289668+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:09:38.180696+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:10:04.471459+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:10:31.041912+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:10:57.477266+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.82:0/2005424509":"2024-05-21T16:04:45.161949+0000","172.21.15.82:6801/1763049308":"2024-05-21T16:04:45.161949+0000","172.21.15.82:6801/1959459348":"2024-05-21T16:04:19.208291+0000","172.21.15.82:0/3131904556":"2024-05-21T16:04:19.208291+0000","172.21.15.82:6800/1959459348":"2024-05-21T16:04:19.208291+0000","172.21.15.82:0/2456966342":"2024-05-21T16:04:45.161949+0000","172.21.15.82:0/4179978833":"2024-05-21T16:04:19.208291+0000","172.21.15.82:0/1602862225":"2024-05-21T16:04:45.161949+0000","172.21.15.82:0/106101180":"2024-05-21T16:04:19.208291+0000","172.21.15.82:6800/1763049308":"2024-05-21T16:04:45.161949+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-20T16:12:15.903 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:15.902+0000 7fddfa3f5640 1 -- 172.21.15.82:0/1445240663 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fdddc0a2c20 msgr2=0x7fdddc0a5130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:12:15.903 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:15.902+0000 7fddfa3f5640 1 --2- 172.21.15.82:0/1445240663 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fdddc0a2c20 0x7fdddc0a5130 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fdde4000f80 tx=0x7fdde4005fd0 comp rx=0 tx=0).stop 2024-05-20T16:12:15.903 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:15.902+0000 7fddfa3f5640 1 -- 172.21.15.82:0/1445240663 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fddf41078c0 msgr2=0x7fddf41a0610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:12:15.903 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:15.902+0000 7fddfa3f5640 1 --2- 172.21.15.82:0/1445240663 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fddf41078c0 0x7fddf41a0610 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fddec00d750 tx=0x7fddec00dc10 comp rx=0 tx=0).stop 2024-05-20T16:12:15.903 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:15.902+0000 7fddfa3f5640 1 -- 172.21.15.82:0/1445240663 shutdown_connections 2024-05-20T16:12:15.903 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:15.902+0000 7fddfa3f5640 1 --2- 172.21.15.82:0/1445240663 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fdddc0a2c20 0x7fdddc0a5130 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:15.903 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:15.902+0000 7fddfa3f5640 1 --2- 172.21.15.82:0/1445240663 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fddf41078c0 0x7fddf41a0610 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:15.903 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:15.902+0000 7fddfa3f5640 1 --2- 172.21.15.82:0/1445240663 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fddf4102bc0 0x7fddf419e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:15.904 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:15.902+0000 7fddfa3f5640 1 --2- 172.21.15.82:0/1445240663 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fddf4101980 0x7fddf406acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:15.904 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:15.903+0000 7fddfa3f5640 1 -- 172.21.15.82:0/1445240663 >> 172.21.15.82:0/1445240663 conn(0x7fddf40fcb50 msgr2=0x7fddf41038a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:12:15.904 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:15.903+0000 7fddfa3f5640 1 -- 172.21.15.82:0/1445240663 shutdown_connections 2024-05-20T16:12:15.904 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:15.903+0000 7fddfa3f5640 1 -- 172.21.15.82:0/1445240663 wait complete. 2024-05-20T16:12:15.914 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-05-20T16:12:15.915 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-20T16:12:16.026 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:16.025+0000 7fa6e481b640 1 -- 172.21.15.82:0/1225465347 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa6e0100ea0 msgr2=0x7fa6e0105b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:12:16.026 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:16.025+0000 7fa6e481b640 1 --2- 172.21.15.82:0/1225465347 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa6e0100ea0 0x7fa6e0105b10 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fa6d0009900 tx=0x7fa6d002a410 comp rx=0 tx=0).stop 2024-05-20T16:12:16.026 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:16.025+0000 7fa6e481b640 1 -- 172.21.15.82:0/1225465347 shutdown_connections 2024-05-20T16:12:16.026 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:16.025+0000 7fa6e481b640 1 --2- 172.21.15.82:0/1225465347 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa6e01089b0 0x7fa6e010add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:16.026 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:16.025+0000 7fa6e481b640 1 --2- 172.21.15.82:0/1225465347 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa6e0106050 0x7fa6e0108470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:16.027 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:16.025+0000 7fa6e481b640 1 --2- 172.21.15.82:0/1225465347 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa6e0100ea0 0x7fa6e0105b10 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:16.027 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:16.025+0000 7fa6e481b640 1 -- 172.21.15.82:0/1225465347 >> 172.21.15.82:0/1225465347 conn(0x7fa6e00fc2e0 msgr2=0x7fa6e00fe7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:12:16.027 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:16.025+0000 7fa6e481b640 1 -- 172.21.15.82:0/1225465347 shutdown_connections 2024-05-20T16:12:16.027 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:16.026+0000 7fa6e481b640 1 -- 172.21.15.82:0/1225465347 wait complete. 2024-05-20T16:12:16.027 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:16.026+0000 7fa6e481b640 1 Processor -- start 2024-05-20T16:12:16.027 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:16.026+0000 7fa6e481b640 1 -- start start 2024-05-20T16:12:16.029 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:16.027+0000 7fa6e481b640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa6e0100ea0 0x7fa6e0199310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:12:16.029 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:16.027+0000 7fa6e481b640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa6e0106050 0x7fa6e0199850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:12:16.030 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:16.027+0000 7fa6e481b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa6e01089b0 0x7fa6e01a07e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:12:16.030 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:16.027+0000 7fa6e481b640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fa6e010e770 con 0x7fa6e01089b0 2024-05-20T16:12:16.030 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:16.027+0000 7fa6e481b640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fa6e010e5f0 con 0x7fa6e0100ea0 2024-05-20T16:12:16.030 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:16.027+0000 7fa6e481b640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa6e010e8f0 con 0x7fa6e0106050 2024-05-20T16:12:16.030 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:16.027+0000 7fa6ded9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa6e01089b0 0x7fa6e01a07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:12:16.030 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:16.027+0000 7fa6ded9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa6e01089b0 0x7fa6e01a07e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:47266/0 (socket says 172.21.15.82:47266) 2024-05-20T16:12:16.030 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:16.027+0000 7fa6ded9d640 1 -- 172.21.15.82:0/3967631852 learned_addr learned my addr 172.21.15.82:0/3967631852 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:12:16.030 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:16.027+0000 7fa6d5d9b640 1 --2- 172.21.15.82:0/3967631852 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa6e0106050 0x7fa6e0199850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:12:16.030 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:16.027+0000 7fa6de59c640 1 --2- 172.21.15.82:0/3967631852 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa6e0100ea0 0x7fa6e0199310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:12:16.030 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:16.027+0000 7fa6ded9d640 1 -- 172.21.15.82:0/3967631852 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa6e0106050 msgr2=0x7fa6e0199850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:12:16.030 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:16.027+0000 7fa6ded9d640 1 --2- 172.21.15.82:0/3967631852 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa6e0106050 0x7fa6e0199850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:16.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:16.027+0000 7fa6ded9d640 1 -- 172.21.15.82:0/3967631852 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa6e0100ea0 msgr2=0x7fa6e0199310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:12:16.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:16.027+0000 7fa6ded9d640 1 --2- 172.21.15.82:0/3967631852 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa6e0100ea0 0x7fa6e0199310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:16.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:16.027+0000 7fa6ded9d640 1 -- 172.21.15.82:0/3967631852 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6e01a0e50 con 0x7fa6e01089b0 2024-05-20T16:12:16.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:16.027+0000 7fa6de59c640 1 --2- 172.21.15.82:0/3967631852 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa6e0100ea0 0x7fa6e0199310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T16:12:16.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:16.027+0000 7fa6ded9d640 1 --2- 172.21.15.82:0/3967631852 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa6e01089b0 0x7fa6e01a07e0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fa6d800eaa0 tx=0x7fa6d800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:12:16.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:16.027+0000 7fa6d7fff640 1 -- 172.21.15.82:0/3967631852 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6d800cdb0 con 0x7fa6e01089b0 2024-05-20T16:12:16.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:16.028+0000 7fa6d7fff640 1 -- 172.21.15.82:0/3967631852 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa6d8005020 con 0x7fa6e01089b0 2024-05-20T16:12:16.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:16.028+0000 7fa6e481b640 1 -- 172.21.15.82:0/3967631852 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6e01a1140 con 0x7fa6e01089b0 2024-05-20T16:12:16.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:16.028+0000 7fa6d7fff640 1 -- 172.21.15.82:0/3967631852 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6d80116b0 con 0x7fa6e01089b0 2024-05-20T16:12:16.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:16.028+0000 7fa6e481b640 1 -- 172.21.15.82:0/3967631852 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6e01a1690 con 0x7fa6e01089b0 2024-05-20T16:12:16.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:16.029+0000 7fa6e481b640 1 -- 172.21.15.82:0/3967631852 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6e00fde00 con 0x7fa6e01089b0 2024-05-20T16:12:16.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:16.030+0000 7fa6d7fff640 1 -- 172.21.15.82:0/3967631852 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fa6d8011850 con 0x7fa6e01089b0 2024-05-20T16:12:16.032 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:16.030+0000 7fa6d7fff640 1 --2- 172.21.15.82:0/3967631852 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fa6cc0a2c20 0x7fa6cc0a5130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:12:16.032 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:16.031+0000 7fa6d7fff640 1 -- 172.21.15.82:0/3967631852 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8802+0+0 (secure 0 0 0) 0x7fa6d80cf660 con 0x7fa6e01089b0 2024-05-20T16:12:16.032 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:16.031+0000 7fa6de59c640 1 --2- 172.21.15.82:0/3967631852 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fa6cc0a2c20 0x7fa6cc0a5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:12:16.032 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:16.031+0000 7fa6de59c640 1 --2- 172.21.15.82:0/3967631852 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fa6cc0a2c20 0x7fa6cc0a5130 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fa6d0000c00 tx=0x7fa6d0031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:12:16.035 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:16.035+0000 7fa6d7fff640 1 -- 172.21.15.82:0/3967631852 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fa6d8016030 con 0x7fa6e01089b0 2024-05-20T16:12:16.208 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:16.208+0000 7fa6e481b640 1 -- 172.21.15.82:0/3967631852 --> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa6e0061f90 con 0x7fa6cc0a2c20 2024-05-20T16:12:16.218 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:16.218+0000 7fa6d7fff640 1 -- 172.21.15.82:0/3967631852 <== mgr.14150 v2:172.21.15.82:6800/379530823 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340559 (secure 0 0 0) 0x7fa6e0061f90 con 0x7fa6cc0a2c20 2024-05-20T16:12:16.223 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-05-20T16:12:16.225 INFO:teuthology.orchestra.run.smithi082.stderr:dumped all 2024-05-20T16:12:16.227 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:16.222+0000 7fa6e481b640 1 -- 172.21.15.82:0/3967631852 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fa6cc0a2c20 msgr2=0x7fa6cc0a5130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:12:16.227 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:16.222+0000 7fa6e481b640 1 --2- 172.21.15.82:0/3967631852 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fa6cc0a2c20 0x7fa6cc0a5130 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fa6d0000c00 tx=0x7fa6d0031400 comp rx=0 tx=0).stop 2024-05-20T16:12:16.227 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:16.222+0000 7fa6e481b640 1 -- 172.21.15.82:0/3967631852 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa6e01089b0 msgr2=0x7fa6e01a07e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:12:16.228 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:16.222+0000 7fa6e481b640 1 --2- 172.21.15.82:0/3967631852 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa6e01089b0 0x7fa6e01a07e0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fa6d800eaa0 tx=0x7fa6d800ef60 comp rx=0 tx=0).stop 2024-05-20T16:12:16.228 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:16.222+0000 7fa6e481b640 1 -- 172.21.15.82:0/3967631852 shutdown_connections 2024-05-20T16:12:16.228 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:16.222+0000 7fa6e481b640 1 --2- 172.21.15.82:0/3967631852 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fa6cc0a2c20 0x7fa6cc0a5130 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:16.228 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:16.222+0000 7fa6e481b640 1 --2- 172.21.15.82:0/3967631852 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa6e01089b0 0x7fa6e01a07e0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:16.228 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:16.222+0000 7fa6e481b640 1 --2- 172.21.15.82:0/3967631852 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa6e0106050 0x7fa6e0199850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:16.228 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:16.222+0000 7fa6e481b640 1 --2- 172.21.15.82:0/3967631852 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa6e0100ea0 0x7fa6e0199310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:16.228 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:16.222+0000 7fa6e481b640 1 -- 172.21.15.82:0/3967631852 >> 172.21.15.82:0/3967631852 conn(0x7fa6e00fc2e0 msgr2=0x7fa6e01045b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:12:16.228 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:16.222+0000 7fa6e481b640 1 -- 172.21.15.82:0/3967631852 shutdown_connections 2024-05-20T16:12:16.228 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:16.222+0000 7fa6e481b640 1 -- 172.21.15.82:0/3967631852 wait complete. 2024-05-20T16:12:16.233 INFO:teuthology.orchestra.run.smithi082.stdout:{"pg_ready":true,"pg_map":{"version":306,"stamp":"2024-05-20T16:12:15.256713+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":597937,"num_objects":59,"num_object_clones":0,"num_object_copies":178,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":0,"num_read":132,"num_read_kb":115,"num_write":301,"num_write_kb":2982,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":36,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":242,"ondisk_log_size":242,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":348544,"kb_used_data":11852,"kb_used_omap":18,"kb_used_meta":336557,"kb_avail":1124593280,"statfs":{"total":1151940427776,"available":1151583518720,"internally_reserved":0,"allocated":12136448,"data_stored":7673128,"data_compressed":318294,"data_compressed_allocated":430080,"data_compressed_original":2359296,"omap_allocated":19141,"internal_metadata":344634683},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":3070,"num_objects":35,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":73,"num_write_kb":27,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":22,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004139"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:11:53.846675+0000","last_change":"2024-05-20T16:11:47.187104+0000","last_active":"2024-05-20T16:11:53.846675+0000","last_peered":"2024-05-20T16:11:53.846675+0000","last_clean":"2024-05-20T16:11:53.846675+0000","last_became_active":"2024-05-20T16:11:47.186791+0000","last_became_peered":"2024-05-20T16:11:47.186791+0000","last_unstale":"2024-05-20T16:11:53.846675+0000","last_undegraded":"2024-05-20T16:11:53.846675+0000","last_fullsized":"2024-05-20T16:11:53.846675+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:33:38.914389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:11:53.786549+0000","last_change":"2024-05-20T16:11:47.187846+0000","last_active":"2024-05-20T16:11:53.786549+0000","last_peered":"2024-05-20T16:11:53.786549+0000","last_clean":"2024-05-20T16:11:53.786549+0000","last_became_active":"2024-05-20T16:11:47.187540+0000","last_became_peered":"2024-05-20T16:11:47.187540+0000","last_unstale":"2024-05-20T16:11:53.786549+0000","last_undegraded":"2024-05-20T16:11:53.786549+0000","last_fullsized":"2024-05-20T16:11:53.786549+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:19:59.784437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:11:53.786527+0000","last_change":"2024-05-20T16:11:47.185215+0000","last_active":"2024-05-20T16:11:53.786527+0000","last_peered":"2024-05-20T16:11:53.786527+0000","last_clean":"2024-05-20T16:11:53.786527+0000","last_became_active":"2024-05-20T16:11:47.184880+0000","last_became_peered":"2024-05-20T16:11:47.184880+0000","last_unstale":"2024-05-20T16:11:53.786527+0000","last_undegraded":"2024-05-20T16:11:53.786527+0000","last_fullsized":"2024-05-20T16:11:53.786527+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T18:43:32.125643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:11:53.423464+0000","last_change":"2024-05-20T16:11:47.175849+0000","last_active":"2024-05-20T16:11:53.423464+0000","last_peered":"2024-05-20T16:11:53.423464+0000","last_clean":"2024-05-20T16:11:53.423464+0000","last_became_active":"2024-05-20T16:11:47.175342+0000","last_became_peered":"2024-05-20T16:11:47.175342+0000","last_unstale":"2024-05-20T16:11:53.423464+0000","last_undegraded":"2024-05-20T16:11:53.423464+0000","last_fullsized":"2024-05-20T16:11:53.423464+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:02:43.798388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:11:53.846865+0000","last_change":"2024-05-20T16:11:47.174798+0000","last_active":"2024-05-20T16:11:53.846865+0000","last_peered":"2024-05-20T16:11:53.846865+0000","last_clean":"2024-05-20T16:11:53.846865+0000","last_became_active":"2024-05-20T16:11:47.174474+0000","last_became_peered":"2024-05-20T16:11:47.174474+0000","last_unstale":"2024-05-20T16:11:53.846865+0000","last_undegraded":"2024-05-20T16:11:53.846865+0000","last_fullsized":"2024-05-20T16:11:53.846865+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:41:24.404901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:11:53.845635+0000","last_change":"2024-05-20T16:11:47.185731+0000","last_active":"2024-05-20T16:11:53.845635+0000","last_peered":"2024-05-20T16:11:53.845635+0000","last_clean":"2024-05-20T16:11:53.845635+0000","last_became_active":"2024-05-20T16:11:47.185380+0000","last_became_peered":"2024-05-20T16:11:47.185380+0000","last_unstale":"2024-05-20T16:11:53.845635+0000","last_undegraded":"2024-05-20T16:11:53.845635+0000","last_fullsized":"2024-05-20T16:11:53.845635+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T16:33:28.854205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:11:53.589081+0000","last_change":"2024-05-20T16:11:47.183594+0000","last_active":"2024-05-20T16:11:53.589081+0000","last_peered":"2024-05-20T16:11:53.589081+0000","last_clean":"2024-05-20T16:11:53.589081+0000","last_became_active":"2024-05-20T16:11:47.183260+0000","last_became_peered":"2024-05-20T16:11:47.183260+0000","last_unstale":"2024-05-20T16:11:53.589081+0000","last_undegraded":"2024-05-20T16:11:53.589081+0000","last_fullsized":"2024-05-20T16:11:53.589081+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T17:12:22.781236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:11:53.844654+0000","last_change":"2024-05-20T16:11:47.191784+0000","last_active":"2024-05-20T16:11:53.844654+0000","last_peered":"2024-05-20T16:11:53.844654+0000","last_clean":"2024-05-20T16:11:53.844654+0000","last_became_active":"2024-05-20T16:11:47.190553+0000","last_became_peered":"2024-05-20T16:11:47.190553+0000","last_unstale":"2024-05-20T16:11:53.844654+0000","last_undegraded":"2024-05-20T16:11:53.844654+0000","last_fullsized":"2024-05-20T16:11:53.844654+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T20:17:29.346688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:11:53.771929+0000","last_change":"2024-05-20T16:11:47.174817+0000","last_active":"2024-05-20T16:11:53.771929+0000","last_peered":"2024-05-20T16:11:53.771929+0000","last_clean":"2024-05-20T16:11:53.771929+0000","last_became_active":"2024-05-20T16:11:47.174483+0000","last_became_peered":"2024-05-20T16:11:47.174483+0000","last_unstale":"2024-05-20T16:11:53.771929+0000","last_undegraded":"2024-05-20T16:11:53.771929+0000","last_fullsized":"2024-05-20T16:11:53.771929+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T20:32:30.284912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:11:53.846029+0000","last_change":"2024-05-20T16:11:47.186899+0000","last_active":"2024-05-20T16:11:53.846029+0000","last_peered":"2024-05-20T16:11:53.846029+0000","last_clean":"2024-05-20T16:11:53.846029+0000","last_became_active":"2024-05-20T16:11:47.186564+0000","last_became_peered":"2024-05-20T16:11:47.186564+0000","last_unstale":"2024-05-20T16:11:53.846029+0000","last_undegraded":"2024-05-20T16:11:53.846029+0000","last_fullsized":"2024-05-20T16:11:53.846029+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:19:57.110107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:11:53.589667+0000","last_change":"2024-05-20T16:11:47.182696+0000","last_active":"2024-05-20T16:11:53.589667+0000","last_peered":"2024-05-20T16:11:53.589667+0000","last_clean":"2024-05-20T16:11:53.589667+0000","last_became_active":"2024-05-20T16:11:47.182169+0000","last_became_peered":"2024-05-20T16:11:47.182169+0000","last_unstale":"2024-05-20T16:11:53.589667+0000","last_undegraded":"2024-05-20T16:11:53.589667+0000","last_fullsized":"2024-05-20T16:11:53.589667+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:55:16.413493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:11:53.423405+0000","last_change":"2024-05-20T16:11:47.181716+0000","last_active":"2024-05-20T16:11:53.423405+0000","last_peered":"2024-05-20T16:11:53.423405+0000","last_clean":"2024-05-20T16:11:53.423405+0000","last_became_active":"2024-05-20T16:11:47.181403+0000","last_became_peered":"2024-05-20T16:11:47.181403+0000","last_unstale":"2024-05-20T16:11:53.423405+0000","last_undegraded":"2024-05-20T16:11:53.423405+0000","last_fullsized":"2024-05-20T16:11:53.423405+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:38:04.938882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:11:54.266744+0000","last_change":"2024-05-20T16:11:47.183320+0000","last_active":"2024-05-20T16:11:54.266744+0000","last_peered":"2024-05-20T16:11:54.266744+0000","last_clean":"2024-05-20T16:11:54.266744+0000","last_became_active":"2024-05-20T16:11:47.183028+0000","last_became_peered":"2024-05-20T16:11:47.183028+0000","last_unstale":"2024-05-20T16:11:54.266744+0000","last_undegraded":"2024-05-20T16:11:54.266744+0000","last_fullsized":"2024-05-20T16:11:54.266744+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T17:28:33.614173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:11:53.786235+0000","last_change":"2024-05-20T16:11:47.183718+0000","last_active":"2024-05-20T16:11:53.786235+0000","last_peered":"2024-05-20T16:11:53.786235+0000","last_clean":"2024-05-20T16:11:53.786235+0000","last_became_active":"2024-05-20T16:11:47.183378+0000","last_became_peered":"2024-05-20T16:11:47.183378+0000","last_unstale":"2024-05-20T16:11:53.786235+0000","last_undegraded":"2024-05-20T16:11:53.786235+0000","last_fullsized":"2024-05-20T16:11:53.786235+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:08:32.066924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:11:53.786231+0000","last_change":"2024-05-20T16:11:47.174163+0000","last_active":"2024-05-20T16:11:53.786231+0000","last_peered":"2024-05-20T16:11:53.786231+0000","last_clean":"2024-05-20T16:11:53.786231+0000","last_became_active":"2024-05-20T16:11:47.173798+0000","last_became_peered":"2024-05-20T16:11:47.173798+0000","last_unstale":"2024-05-20T16:11:53.786231+0000","last_undegraded":"2024-05-20T16:11:53.786231+0000","last_fullsized":"2024-05-20T16:11:53.786231+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T20:51:20.646273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:11:53.771220+0000","last_change":"2024-05-20T16:11:47.181809+0000","last_active":"2024-05-20T16:11:53.771220+0000","last_peered":"2024-05-20T16:11:53.771220+0000","last_clean":"2024-05-20T16:11:53.771220+0000","last_became_active":"2024-05-20T16:11:47.181473+0000","last_became_peered":"2024-05-20T16:11:47.181473+0000","last_unstale":"2024-05-20T16:11:53.771220+0000","last_undegraded":"2024-05-20T16:11:53.771220+0000","last_fullsized":"2024-05-20T16:11:53.771220+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:20:45.302995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:11:53.846554+0000","last_change":"2024-05-20T16:11:47.187947+0000","last_active":"2024-05-20T16:11:53.846554+0000","last_peered":"2024-05-20T16:11:53.846554+0000","last_clean":"2024-05-20T16:11:53.846554+0000","last_became_active":"2024-05-20T16:11:47.187623+0000","last_became_peered":"2024-05-20T16:11:47.187623+0000","last_unstale":"2024-05-20T16:11:53.846554+0000","last_undegraded":"2024-05-20T16:11:53.846554+0000","last_fullsized":"2024-05-20T16:11:53.846554+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:52:08.674888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:11:53.420181+0000","last_change":"2024-05-20T16:11:47.188599+0000","last_active":"2024-05-20T16:11:53.420181+0000","last_peered":"2024-05-20T16:11:53.420181+0000","last_clean":"2024-05-20T16:11:53.420181+0000","last_became_active":"2024-05-20T16:11:47.188278+0000","last_became_peered":"2024-05-20T16:11:47.188278+0000","last_unstale":"2024-05-20T16:11:53.420181+0000","last_undegraded":"2024-05-20T16:11:53.420181+0000","last_fullsized":"2024-05-20T16:11:53.420181+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T17:45:42.003546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:11:53.420823+0000","last_change":"2024-05-20T16:11:47.176250+0000","last_active":"2024-05-20T16:11:53.420823+0000","last_peered":"2024-05-20T16:11:53.420823+0000","last_clean":"2024-05-20T16:11:53.420823+0000","last_became_active":"2024-05-20T16:11:47.175903+0000","last_became_peered":"2024-05-20T16:11:47.175903+0000","last_unstale":"2024-05-20T16:11:53.420823+0000","last_undegraded":"2024-05-20T16:11:53.420823+0000","last_fullsized":"2024-05-20T16:11:53.420823+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T18:11:26.045745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:11:53.420414+0000","last_change":"2024-05-20T16:11:51.438824+0000","last_active":"2024-05-20T16:11:53.420414+0000","last_peered":"2024-05-20T16:11:53.420414+0000","last_clean":"2024-05-20T16:11:53.420414+0000","last_became_active":"2024-05-20T16:11:51.438279+0000","last_became_peered":"2024-05-20T16:11:51.438279+0000","last_unstale":"2024-05-20T16:11:53.420414+0000","last_undegraded":"2024-05-20T16:11:53.420414+0000","last_fullsized":"2024-05-20T16:11:53.420414+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_clean_scrub_stamp":"2024-05-20T16:11:50.398948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:30:50.680109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"85'4","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:11:53.420004+0000","last_change":"2024-05-20T16:11:46.397871+0000","last_active":"2024-05-20T16:11:53.420004+0000","last_peered":"2024-05-20T16:11:53.420004+0000","last_clean":"2024-05-20T16:11:53.420004+0000","last_became_active":"2024-05-20T16:11:46.397569+0000","last_became_peered":"2024-05-20T16:11:46.397569+0000","last_unstale":"2024-05-20T16:11:53.420004+0000","last_undegraded":"2024-05-20T16:11:53.420004+0000","last_fullsized":"2024-05-20T16:11:53.420004+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_clean_scrub_stamp":"2024-05-20T16:11:45.363714+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:32:24.329823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:11:53.845169+0000","last_change":"2024-05-20T16:11:47.183935+0000","last_active":"2024-05-20T16:11:53.845169+0000","last_peered":"2024-05-20T16:11:53.845169+0000","last_clean":"2024-05-20T16:11:53.845169+0000","last_became_active":"2024-05-20T16:11:47.183604+0000","last_became_peered":"2024-05-20T16:11:47.183604+0000","last_unstale":"2024-05-20T16:11:53.845169+0000","last_undegraded":"2024-05-20T16:11:53.845169+0000","last_fullsized":"2024-05-20T16:11:53.845169+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T16:59:36.924396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:11:53.422665+0000","last_change":"2024-05-20T16:11:51.441603+0000","last_active":"2024-05-20T16:11:53.422665+0000","last_peered":"2024-05-20T16:11:53.422665+0000","last_clean":"2024-05-20T16:11:53.422665+0000","last_became_active":"2024-05-20T16:11:51.441106+0000","last_became_peered":"2024-05-20T16:11:51.441106+0000","last_unstale":"2024-05-20T16:11:53.422665+0000","last_undegraded":"2024-05-20T16:11:53.422665+0000","last_fullsized":"2024-05-20T16:11:53.422665+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_clean_scrub_stamp":"2024-05-20T16:11:50.398948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:07:29.221992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"85'1","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:11:53.845201+0000","last_change":"2024-05-20T16:11:46.401617+0000","last_active":"2024-05-20T16:11:53.845201+0000","last_peered":"2024-05-20T16:11:53.845201+0000","last_clean":"2024-05-20T16:11:53.845201+0000","last_became_active":"2024-05-20T16:11:46.401295+0000","last_became_peered":"2024-05-20T16:11:46.401295+0000","last_unstale":"2024-05-20T16:11:53.845201+0000","last_undegraded":"2024-05-20T16:11:53.845201+0000","last_fullsized":"2024-05-20T16:11:53.845201+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_clean_scrub_stamp":"2024-05-20T16:11:45.363714+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:42:21.039088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:11:53.786434+0000","last_change":"2024-05-20T16:11:47.187157+0000","last_active":"2024-05-20T16:11:53.786434+0000","last_peered":"2024-05-20T16:11:53.786434+0000","last_clean":"2024-05-20T16:11:53.786434+0000","last_became_active":"2024-05-20T16:11:47.186765+0000","last_became_peered":"2024-05-20T16:11:47.186765+0000","last_unstale":"2024-05-20T16:11:53.786434+0000","last_undegraded":"2024-05-20T16:11:53.786434+0000","last_fullsized":"2024-05-20T16:11:53.786434+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T17:09:56.687541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:11:53.423137+0000","last_change":"2024-05-20T16:11:51.442251+0000","last_active":"2024-05-20T16:11:53.423137+0000","last_peered":"2024-05-20T16:11:53.423137+0000","last_clean":"2024-05-20T16:11:53.423137+0000","last_became_active":"2024-05-20T16:11:51.441427+0000","last_became_peered":"2024-05-20T16:11:51.441427+0000","last_unstale":"2024-05-20T16:11:53.423137+0000","last_undegraded":"2024-05-20T16:11:53.423137+0000","last_fullsized":"2024-05-20T16:11:53.423137+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_clean_scrub_stamp":"2024-05-20T16:11:50.398948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:14:08.108310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"91'4","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:12:12.397731+0000","last_change":"2024-05-20T16:11:46.404722+0000","last_active":"2024-05-20T16:12:12.397731+0000","last_peered":"2024-05-20T16:12:12.397731+0000","last_clean":"2024-05-20T16:12:12.397731+0000","last_became_active":"2024-05-20T16:11:46.403626+0000","last_became_peered":"2024-05-20T16:11:46.403626+0000","last_unstale":"2024-05-20T16:12:12.397731+0000","last_undegraded":"2024-05-20T16:12:12.397731+0000","last_fullsized":"2024-05-20T16:12:12.397731+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_clean_scrub_stamp":"2024-05-20T16:11:45.363714+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T18:46:11.283344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:11:53.771585+0000","last_change":"2024-05-20T16:11:47.180014+0000","last_active":"2024-05-20T16:11:53.771585+0000","last_peered":"2024-05-20T16:11:53.771585+0000","last_clean":"2024-05-20T16:11:53.771585+0000","last_became_active":"2024-05-20T16:11:47.179660+0000","last_became_peered":"2024-05-20T16:11:47.179660+0000","last_unstale":"2024-05-20T16:11:53.771585+0000","last_undegraded":"2024-05-20T16:11:53.771585+0000","last_fullsized":"2024-05-20T16:11:53.771585+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:07:26.925346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:11:53.422707+0000","last_change":"2024-05-20T16:11:51.439268+0000","last_active":"2024-05-20T16:11:53.422707+0000","last_peered":"2024-05-20T16:11:53.422707+0000","last_clean":"2024-05-20T16:11:53.422707+0000","last_became_active":"2024-05-20T16:11:51.438767+0000","last_became_peered":"2024-05-20T16:11:51.438767+0000","last_unstale":"2024-05-20T16:11:53.422707+0000","last_undegraded":"2024-05-20T16:11:53.422707+0000","last_fullsized":"2024-05-20T16:11:53.422707+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_clean_scrub_stamp":"2024-05-20T16:11:50.398948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:31:22.394666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:11:53.847517+0000","last_change":"2024-05-20T16:11:46.398922+0000","last_active":"2024-05-20T16:11:53.847517+0000","last_peered":"2024-05-20T16:11:53.847517+0000","last_clean":"2024-05-20T16:11:53.847517+0000","last_became_active":"2024-05-20T16:11:46.398434+0000","last_became_peered":"2024-05-20T16:11:46.398434+0000","last_unstale":"2024-05-20T16:11:53.847517+0000","last_undegraded":"2024-05-20T16:11:53.847517+0000","last_fullsized":"2024-05-20T16:11:53.847517+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_clean_scrub_stamp":"2024-05-20T16:11:45.363714+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T17:38:12.120113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:11:53.847427+0000","last_change":"2024-05-20T16:11:47.179879+0000","last_active":"2024-05-20T16:11:53.847427+0000","last_peered":"2024-05-20T16:11:53.847427+0000","last_clean":"2024-05-20T16:11:53.847427+0000","last_became_active":"2024-05-20T16:11:47.179545+0000","last_became_peered":"2024-05-20T16:11:47.179545+0000","last_unstale":"2024-05-20T16:11:53.847427+0000","last_undegraded":"2024-05-20T16:11:53.847427+0000","last_fullsized":"2024-05-20T16:11:53.847427+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:00:15.966911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:11:53.420660+0000","last_change":"2024-05-20T16:11:51.435981+0000","last_active":"2024-05-20T16:11:53.420660+0000","last_peered":"2024-05-20T16:11:53.420660+0000","last_clean":"2024-05-20T16:11:53.420660+0000","last_became_active":"2024-05-20T16:11:51.435387+0000","last_became_peered":"2024-05-20T16:11:51.435387+0000","last_unstale":"2024-05-20T16:11:53.420660+0000","last_undegraded":"2024-05-20T16:11:53.420660+0000","last_fullsized":"2024-05-20T16:11:53.420660+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_clean_scrub_stamp":"2024-05-20T16:11:50.398948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T16:14:14.877384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"91'5","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:12:11.387883+0000","last_change":"2024-05-20T16:11:46.402453+0000","last_active":"2024-05-20T16:12:11.387883+0000","last_peered":"2024-05-20T16:12:11.387883+0000","last_clean":"2024-05-20T16:12:11.387883+0000","last_became_active":"2024-05-20T16:11:46.400824+0000","last_became_peered":"2024-05-20T16:11:46.400824+0000","last_unstale":"2024-05-20T16:12:11.387883+0000","last_undegraded":"2024-05-20T16:12:11.387883+0000","last_fullsized":"2024-05-20T16:12:11.387883+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_clean_scrub_stamp":"2024-05-20T16:11:45.363714+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T17:39:47.431485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:11:53.846351+0000","last_change":"2024-05-20T16:11:47.180885+0000","last_active":"2024-05-20T16:11:53.846351+0000","last_peered":"2024-05-20T16:11:53.846351+0000","last_clean":"2024-05-20T16:11:53.846351+0000","last_became_active":"2024-05-20T16:11:47.178680+0000","last_became_peered":"2024-05-20T16:11:47.178680+0000","last_unstale":"2024-05-20T16:11:53.846351+0000","last_undegraded":"2024-05-20T16:11:53.846351+0000","last_fullsized":"2024-05-20T16:11:53.846351+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T19:28:00.540561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:11:53.420527+0000","last_change":"2024-05-20T16:11:51.436093+0000","last_active":"2024-05-20T16:11:53.420527+0000","last_peered":"2024-05-20T16:11:53.420527+0000","last_clean":"2024-05-20T16:11:53.420527+0000","last_became_active":"2024-05-20T16:11:51.435448+0000","last_became_peered":"2024-05-20T16:11:51.435448+0000","last_unstale":"2024-05-20T16:11:53.420527+0000","last_undegraded":"2024-05-20T16:11:53.420527+0000","last_fullsized":"2024-05-20T16:11:53.420527+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_clean_scrub_stamp":"2024-05-20T16:11:50.398948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:43:52.365055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"91'3","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:12:12.397695+0000","last_change":"2024-05-20T16:11:46.401695+0000","last_active":"2024-05-20T16:12:12.397695+0000","last_peered":"2024-05-20T16:12:12.397695+0000","last_clean":"2024-05-20T16:12:12.397695+0000","last_became_active":"2024-05-20T16:11:46.401352+0000","last_became_peered":"2024-05-20T16:11:46.401352+0000","last_unstale":"2024-05-20T16:12:12.397695+0000","last_undegraded":"2024-05-20T16:12:12.397695+0000","last_fullsized":"2024-05-20T16:12:12.397695+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_clean_scrub_stamp":"2024-05-20T16:11:45.363714+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:50:58.714842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:11:53.845816+0000","last_change":"2024-05-20T16:11:47.185820+0000","last_active":"2024-05-20T16:11:53.845816+0000","last_peered":"2024-05-20T16:11:53.845816+0000","last_clean":"2024-05-20T16:11:53.845816+0000","last_became_active":"2024-05-20T16:11:47.185491+0000","last_became_peered":"2024-05-20T16:11:47.185491+0000","last_unstale":"2024-05-20T16:11:53.845816+0000","last_undegraded":"2024-05-20T16:11:53.845816+0000","last_fullsized":"2024-05-20T16:11:53.845816+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:23:27.065634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:11:53.846833+0000","last_change":"2024-05-20T16:11:51.432409+0000","last_active":"2024-05-20T16:11:53.846833+0000","last_peered":"2024-05-20T16:11:53.846833+0000","last_clean":"2024-05-20T16:11:53.846833+0000","last_became_active":"2024-05-20T16:11:51.431993+0000","last_became_peered":"2024-05-20T16:11:51.431993+0000","last_unstale":"2024-05-20T16:11:53.846833+0000","last_undegraded":"2024-05-20T16:11:53.846833+0000","last_fullsized":"2024-05-20T16:11:53.846833+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_clean_scrub_stamp":"2024-05-20T16:11:50.398948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:41:08.301120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"91'2","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:12:12.398563+0000","last_change":"2024-05-20T16:11:46.397358+0000","last_active":"2024-05-20T16:12:12.398563+0000","last_peered":"2024-05-20T16:12:12.398563+0000","last_clean":"2024-05-20T16:12:12.398563+0000","last_became_active":"2024-05-20T16:11:46.396989+0000","last_became_peered":"2024-05-20T16:11:46.396989+0000","last_unstale":"2024-05-20T16:12:12.398563+0000","last_undegraded":"2024-05-20T16:12:12.398563+0000","last_fullsized":"2024-05-20T16:12:12.398563+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_clean_scrub_stamp":"2024-05-20T16:11:45.363714+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:31:19.737223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:11:53.844781+0000","last_change":"2024-05-20T16:11:47.175117+0000","last_active":"2024-05-20T16:11:53.844781+0000","last_peered":"2024-05-20T16:11:53.844781+0000","last_clean":"2024-05-20T16:11:53.844781+0000","last_became_active":"2024-05-20T16:11:47.174885+0000","last_became_peered":"2024-05-20T16:11:47.174885+0000","last_unstale":"2024-05-20T16:11:53.844781+0000","last_undegraded":"2024-05-20T16:11:53.844781+0000","last_fullsized":"2024-05-20T16:11:53.844781+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:38:35.765034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:11:53.786630+0000","last_change":"2024-05-20T16:11:51.440650+0000","last_active":"2024-05-20T16:11:53.786630+0000","last_peered":"2024-05-20T16:11:53.786630+0000","last_clean":"2024-05-20T16:11:53.786630+0000","last_became_active":"2024-05-20T16:11:51.440208+0000","last_became_peered":"2024-05-20T16:11:51.440208+0000","last_unstale":"2024-05-20T16:11:53.786630+0000","last_undegraded":"2024-05-20T16:11:53.786630+0000","last_fullsized":"2024-05-20T16:11:53.786630+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_clean_scrub_stamp":"2024-05-20T16:11:50.398948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:02:02.925911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"91'1","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:11:53.845684+0000","last_change":"2024-05-20T16:11:46.402078+0000","last_active":"2024-05-20T16:11:53.845684+0000","last_peered":"2024-05-20T16:11:53.845684+0000","last_clean":"2024-05-20T16:11:53.845684+0000","last_became_active":"2024-05-20T16:11:46.401680+0000","last_became_peered":"2024-05-20T16:11:46.401680+0000","last_unstale":"2024-05-20T16:11:53.845684+0000","last_undegraded":"2024-05-20T16:11:53.845684+0000","last_fullsized":"2024-05-20T16:11:53.845684+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_clean_scrub_stamp":"2024-05-20T16:11:45.363714+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:32:06.118688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:11:53.419730+0000","last_change":"2024-05-20T16:11:47.186933+0000","last_active":"2024-05-20T16:11:53.419730+0000","last_peered":"2024-05-20T16:11:53.419730+0000","last_clean":"2024-05-20T16:11:53.419730+0000","last_became_active":"2024-05-20T16:11:47.186554+0000","last_became_peered":"2024-05-20T16:11:47.186554+0000","last_unstale":"2024-05-20T16:11:53.419730+0000","last_undegraded":"2024-05-20T16:11:53.419730+0000","last_fullsized":"2024-05-20T16:11:53.419730+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T16:24:34.634974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:11:53.844922+0000","last_change":"2024-05-20T16:11:51.442535+0000","last_active":"2024-05-20T16:11:53.844922+0000","last_peered":"2024-05-20T16:11:53.844922+0000","last_clean":"2024-05-20T16:11:53.844922+0000","last_became_active":"2024-05-20T16:11:51.442143+0000","last_became_peered":"2024-05-20T16:11:51.442143+0000","last_unstale":"2024-05-20T16:11:53.844922+0000","last_undegraded":"2024-05-20T16:11:53.844922+0000","last_fullsized":"2024-05-20T16:11:53.844922+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_clean_scrub_stamp":"2024-05-20T16:11:50.398948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T19:31:41.640330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"91'3","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:12:11.386560+0000","last_change":"2024-05-20T16:11:46.394319+0000","last_active":"2024-05-20T16:12:11.386560+0000","last_peered":"2024-05-20T16:12:11.386560+0000","last_clean":"2024-05-20T16:12:11.386560+0000","last_became_active":"2024-05-20T16:11:46.393975+0000","last_became_peered":"2024-05-20T16:11:46.393975+0000","last_unstale":"2024-05-20T16:12:11.386560+0000","last_undegraded":"2024-05-20T16:12:11.386560+0000","last_fullsized":"2024-05-20T16:12:11.386560+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_clean_scrub_stamp":"2024-05-20T16:11:45.363714+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:13:18.706944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:11:53.771899+0000","last_change":"2024-05-20T16:11:47.177382+0000","last_active":"2024-05-20T16:11:53.771899+0000","last_peered":"2024-05-20T16:11:53.771899+0000","last_clean":"2024-05-20T16:11:53.771899+0000","last_became_active":"2024-05-20T16:11:47.176949+0000","last_became_peered":"2024-05-20T16:11:47.176949+0000","last_unstale":"2024-05-20T16:11:53.771899+0000","last_undegraded":"2024-05-20T16:11:53.771899+0000","last_fullsized":"2024-05-20T16:11:53.771899+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:20:36.689937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:11:53.419807+0000","last_change":"2024-05-20T16:11:51.438622+0000","last_active":"2024-05-20T16:11:53.419807+0000","last_peered":"2024-05-20T16:11:53.419807+0000","last_clean":"2024-05-20T16:11:53.419807+0000","last_became_active":"2024-05-20T16:11:51.438191+0000","last_became_peered":"2024-05-20T16:11:51.438191+0000","last_unstale":"2024-05-20T16:11:53.419807+0000","last_undegraded":"2024-05-20T16:11:53.419807+0000","last_fullsized":"2024-05-20T16:11:53.419807+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_clean_scrub_stamp":"2024-05-20T16:11:50.398948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T16:39:56.327423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"91'2","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:12:11.386811+0000","last_change":"2024-05-20T16:11:46.399436+0000","last_active":"2024-05-20T16:12:11.386811+0000","last_peered":"2024-05-20T16:12:11.386811+0000","last_clean":"2024-05-20T16:12:11.386811+0000","last_became_active":"2024-05-20T16:11:46.398591+0000","last_became_peered":"2024-05-20T16:11:46.398591+0000","last_unstale":"2024-05-20T16:12:11.386811+0000","last_undegraded":"2024-05-20T16:12:11.386811+0000","last_fullsized":"2024-05-20T16:12:11.386811+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_clean_scrub_stamp":"2024-05-20T16:11:45.363714+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:18:26.775585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:11:53.786586+0000","last_change":"2024-05-20T16:11:47.190246+0000","last_active":"2024-05-20T16:11:53.786586+0000","last_peered":"2024-05-20T16:11:53.786586+0000","last_clean":"2024-05-20T16:11:53.786586+0000","last_became_active":"2024-05-20T16:11:47.189490+0000","last_became_peered":"2024-05-20T16:11:47.189490+0000","last_unstale":"2024-05-20T16:11:53.786586+0000","last_undegraded":"2024-05-20T16:11:53.786586+0000","last_fullsized":"2024-05-20T16:11:53.786586+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:31:05.934097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:11:53.423061+0000","last_change":"2024-05-20T16:11:51.442169+0000","last_active":"2024-05-20T16:11:53.423061+0000","last_peered":"2024-05-20T16:11:53.423061+0000","last_clean":"2024-05-20T16:11:53.423061+0000","last_became_active":"2024-05-20T16:11:51.441114+0000","last_became_peered":"2024-05-20T16:11:51.441114+0000","last_unstale":"2024-05-20T16:11:53.423061+0000","last_undegraded":"2024-05-20T16:11:53.423061+0000","last_fullsized":"2024-05-20T16:11:53.423061+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_clean_scrub_stamp":"2024-05-20T16:11:50.398948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:01:50.833632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"91'4","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:11:53.846924+0000","last_change":"2024-05-20T16:11:46.403424+0000","last_active":"2024-05-20T16:11:53.846924+0000","last_peered":"2024-05-20T16:11:53.846924+0000","last_clean":"2024-05-20T16:11:53.846924+0000","last_became_active":"2024-05-20T16:11:46.400786+0000","last_became_peered":"2024-05-20T16:11:46.400786+0000","last_unstale":"2024-05-20T16:11:53.846924+0000","last_undegraded":"2024-05-20T16:11:53.846924+0000","last_fullsized":"2024-05-20T16:11:53.846924+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_clean_scrub_stamp":"2024-05-20T16:11:45.363714+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T16:29:33.300267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:11:53.771758+0000","last_change":"2024-05-20T16:11:47.176372+0000","last_active":"2024-05-20T16:11:53.771758+0000","last_peered":"2024-05-20T16:11:53.771758+0000","last_clean":"2024-05-20T16:11:53.771758+0000","last_became_active":"2024-05-20T16:11:47.175535+0000","last_became_peered":"2024-05-20T16:11:47.175535+0000","last_unstale":"2024-05-20T16:11:53.771758+0000","last_undegraded":"2024-05-20T16:11:53.771758+0000","last_fullsized":"2024-05-20T16:11:53.771758+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:40:22.293771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:11:53.419875+0000","last_change":"2024-05-20T16:11:51.442678+0000","last_active":"2024-05-20T16:11:53.419875+0000","last_peered":"2024-05-20T16:11:53.419875+0000","last_clean":"2024-05-20T16:11:53.419875+0000","last_became_active":"2024-05-20T16:11:51.442300+0000","last_became_peered":"2024-05-20T16:11:51.442300+0000","last_unstale":"2024-05-20T16:11:53.419875+0000","last_undegraded":"2024-05-20T16:11:53.419875+0000","last_fullsized":"2024-05-20T16:11:53.419875+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_clean_scrub_stamp":"2024-05-20T16:11:50.398948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:29:14.668640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"91'12","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:12:11.386666+0000","last_change":"2024-05-20T16:11:46.398550+0000","last_active":"2024-05-20T16:12:11.386666+0000","last_peered":"2024-05-20T16:12:11.386666+0000","last_clean":"2024-05-20T16:12:11.386666+0000","last_became_active":"2024-05-20T16:11:46.398216+0000","last_became_peered":"2024-05-20T16:11:46.398216+0000","last_unstale":"2024-05-20T16:12:11.386666+0000","last_undegraded":"2024-05-20T16:12:11.386666+0000","last_fullsized":"2024-05-20T16:12:11.386666+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_clean_scrub_stamp":"2024-05-20T16:11:45.363714+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:24:53.292837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:11:53.847039+0000","last_change":"2024-05-20T16:11:47.177641+0000","last_active":"2024-05-20T16:11:53.847039+0000","last_peered":"2024-05-20T16:11:53.847039+0000","last_clean":"2024-05-20T16:11:53.847039+0000","last_became_active":"2024-05-20T16:11:47.177333+0000","last_became_peered":"2024-05-20T16:11:47.177333+0000","last_unstale":"2024-05-20T16:11:53.847039+0000","last_undegraded":"2024-05-20T16:11:53.847039+0000","last_fullsized":"2024-05-20T16:11:53.847039+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T17:42:01.674485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:11:53.847473+0000","last_change":"2024-05-20T16:11:51.445804+0000","last_active":"2024-05-20T16:11:53.847473+0000","last_peered":"2024-05-20T16:11:53.847473+0000","last_clean":"2024-05-20T16:11:53.847473+0000","last_became_active":"2024-05-20T16:11:51.445223+0000","last_became_peered":"2024-05-20T16:11:51.445223+0000","last_unstale":"2024-05-20T16:11:53.847473+0000","last_undegraded":"2024-05-20T16:11:53.847473+0000","last_fullsized":"2024-05-20T16:11:53.847473+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_clean_scrub_stamp":"2024-05-20T16:11:50.398948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:45:19.185992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"91'6","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:12:12.398116+0000","last_change":"2024-05-20T16:11:46.398349+0000","last_active":"2024-05-20T16:12:12.398116+0000","last_peered":"2024-05-20T16:12:12.398116+0000","last_clean":"2024-05-20T16:12:12.398116+0000","last_became_active":"2024-05-20T16:11:46.397878+0000","last_became_peered":"2024-05-20T16:11:46.397878+0000","last_unstale":"2024-05-20T16:12:12.398116+0000","last_undegraded":"2024-05-20T16:12:12.398116+0000","last_fullsized":"2024-05-20T16:12:12.398116+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_clean_scrub_stamp":"2024-05-20T16:11:45.363714+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:39:07.374098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:11:54.266598+0000","last_change":"2024-05-20T16:11:47.179322+0000","last_active":"2024-05-20T16:11:54.266598+0000","last_peered":"2024-05-20T16:11:54.266598+0000","last_clean":"2024-05-20T16:11:54.266598+0000","last_became_active":"2024-05-20T16:11:47.178897+0000","last_became_peered":"2024-05-20T16:11:47.178897+0000","last_unstale":"2024-05-20T16:11:54.266598+0000","last_undegraded":"2024-05-20T16:11:54.266598+0000","last_fullsized":"2024-05-20T16:11:54.266598+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T20:01:04.096952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:11:53.846308+0000","last_change":"2024-05-20T16:11:51.439577+0000","last_active":"2024-05-20T16:11:53.846308+0000","last_peered":"2024-05-20T16:11:53.846308+0000","last_clean":"2024-05-20T16:11:53.846308+0000","last_became_active":"2024-05-20T16:11:51.439043+0000","last_became_peered":"2024-05-20T16:11:51.439043+0000","last_unstale":"2024-05-20T16:11:53.846308+0000","last_undegraded":"2024-05-20T16:11:53.846308+0000","last_fullsized":"2024-05-20T16:11:53.846308+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_clean_scrub_stamp":"2024-05-20T16:11:50.398948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:23:18.861715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"91'3","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:12:11.387273+0000","last_change":"2024-05-20T16:11:46.402276+0000","last_active":"2024-05-20T16:12:11.387273+0000","last_peered":"2024-05-20T16:12:11.387273+0000","last_clean":"2024-05-20T16:12:11.387273+0000","last_became_active":"2024-05-20T16:11:46.401923+0000","last_became_peered":"2024-05-20T16:11:46.401923+0000","last_unstale":"2024-05-20T16:12:11.387273+0000","last_undegraded":"2024-05-20T16:12:11.387273+0000","last_fullsized":"2024-05-20T16:12:11.387273+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_clean_scrub_stamp":"2024-05-20T16:11:45.363714+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T17:57:00.320492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:11:53.589485+0000","last_change":"2024-05-20T16:11:47.174448+0000","last_active":"2024-05-20T16:11:53.589485+0000","last_peered":"2024-05-20T16:11:53.589485+0000","last_clean":"2024-05-20T16:11:53.589485+0000","last_became_active":"2024-05-20T16:11:47.174099+0000","last_became_peered":"2024-05-20T16:11:47.174099+0000","last_unstale":"2024-05-20T16:11:53.589485+0000","last_undegraded":"2024-05-20T16:11:53.589485+0000","last_fullsized":"2024-05-20T16:11:53.589485+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T20:53:30.392099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:11:53.419693+0000","last_change":"2024-05-20T16:11:51.446231+0000","last_active":"2024-05-20T16:11:53.419693+0000","last_peered":"2024-05-20T16:11:53.419693+0000","last_clean":"2024-05-20T16:11:53.419693+0000","last_became_active":"2024-05-20T16:11:51.445805+0000","last_became_peered":"2024-05-20T16:11:51.445805+0000","last_unstale":"2024-05-20T16:11:53.419693+0000","last_undegraded":"2024-05-20T16:11:53.419693+0000","last_fullsized":"2024-05-20T16:11:53.419693+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_clean_scrub_stamp":"2024-05-20T16:11:50.398948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:51:29.505139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"91'3","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:12:12.397003+0000","last_change":"2024-05-20T16:11:46.392677+0000","last_active":"2024-05-20T16:12:12.397003+0000","last_peered":"2024-05-20T16:12:12.397003+0000","last_clean":"2024-05-20T16:12:12.397003+0000","last_became_active":"2024-05-20T16:11:46.392315+0000","last_became_peered":"2024-05-20T16:11:46.392315+0000","last_unstale":"2024-05-20T16:12:12.397003+0000","last_undegraded":"2024-05-20T16:12:12.397003+0000","last_fullsized":"2024-05-20T16:12:12.397003+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_clean_scrub_stamp":"2024-05-20T16:11:45.363714+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T19:04:14.191754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:11:53.589114+0000","last_change":"2024-05-20T16:11:47.187104+0000","last_active":"2024-05-20T16:11:53.589114+0000","last_peered":"2024-05-20T16:11:53.589114+0000","last_clean":"2024-05-20T16:11:53.589114+0000","last_became_active":"2024-05-20T16:11:47.186375+0000","last_became_peered":"2024-05-20T16:11:47.186375+0000","last_unstale":"2024-05-20T16:11:53.589114+0000","last_undegraded":"2024-05-20T16:11:53.589114+0000","last_fullsized":"2024-05-20T16:11:53.589114+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:27:32.478899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:11:54.266942+0000","last_change":"2024-05-20T16:11:51.444872+0000","last_active":"2024-05-20T16:11:54.266942+0000","last_peered":"2024-05-20T16:11:54.266942+0000","last_clean":"2024-05-20T16:11:54.266942+0000","last_became_active":"2024-05-20T16:11:51.444472+0000","last_became_peered":"2024-05-20T16:11:51.444472+0000","last_unstale":"2024-05-20T16:11:54.266942+0000","last_undegraded":"2024-05-20T16:11:54.266942+0000","last_fullsized":"2024-05-20T16:11:54.266942+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_clean_scrub_stamp":"2024-05-20T16:11:50.398948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:37:57.117101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"91'15","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:12:12.398665+0000","last_change":"2024-05-20T16:11:46.407244+0000","last_active":"2024-05-20T16:12:12.398665+0000","last_peered":"2024-05-20T16:12:12.398665+0000","last_clean":"2024-05-20T16:12:12.398665+0000","last_became_active":"2024-05-20T16:11:46.406905+0000","last_became_peered":"2024-05-20T16:11:46.406905+0000","last_unstale":"2024-05-20T16:12:12.398665+0000","last_undegraded":"2024-05-20T16:12:12.398665+0000","last_fullsized":"2024-05-20T16:12:12.398665+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_clean_scrub_stamp":"2024-05-20T16:11:45.363714+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:45:40.497442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:11:53.845955+0000","last_change":"2024-05-20T16:11:47.180326+0000","last_active":"2024-05-20T16:11:53.845955+0000","last_peered":"2024-05-20T16:11:53.845955+0000","last_clean":"2024-05-20T16:11:53.845955+0000","last_became_active":"2024-05-20T16:11:47.179937+0000","last_became_peered":"2024-05-20T16:11:47.179937+0000","last_unstale":"2024-05-20T16:11:53.845955+0000","last_undegraded":"2024-05-20T16:11:53.845955+0000","last_fullsized":"2024-05-20T16:11:53.845955+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:59:59.431365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:11:53.845779+0000","last_change":"2024-05-20T16:11:51.444495+0000","last_active":"2024-05-20T16:11:53.845779+0000","last_peered":"2024-05-20T16:11:53.845779+0000","last_clean":"2024-05-20T16:11:53.845779+0000","last_became_active":"2024-05-20T16:11:51.444136+0000","last_became_peered":"2024-05-20T16:11:51.444136+0000","last_unstale":"2024-05-20T16:11:53.845779+0000","last_undegraded":"2024-05-20T16:11:53.845779+0000","last_fullsized":"2024-05-20T16:11:53.845779+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_clean_scrub_stamp":"2024-05-20T16:11:50.398948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:27:46.260340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"85'1","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:11:53.847676+0000","last_change":"2024-05-20T16:11:46.407252+0000","last_active":"2024-05-20T16:11:53.847676+0000","last_peered":"2024-05-20T16:11:53.847676+0000","last_clean":"2024-05-20T16:11:53.847676+0000","last_became_active":"2024-05-20T16:11:46.406867+0000","last_became_peered":"2024-05-20T16:11:46.406867+0000","last_unstale":"2024-05-20T16:11:53.847676+0000","last_undegraded":"2024-05-20T16:11:53.847676+0000","last_fullsized":"2024-05-20T16:11:53.847676+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_clean_scrub_stamp":"2024-05-20T16:11:45.363714+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T17:53:46.638460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:11:53.846490+0000","last_change":"2024-05-20T16:11:47.179046+0000","last_active":"2024-05-20T16:11:53.846490+0000","last_peered":"2024-05-20T16:11:53.846490+0000","last_clean":"2024-05-20T16:11:53.846490+0000","last_became_active":"2024-05-20T16:11:47.178722+0000","last_became_peered":"2024-05-20T16:11:47.178722+0000","last_unstale":"2024-05-20T16:11:53.846490+0000","last_undegraded":"2024-05-20T16:11:53.846490+0000","last_fullsized":"2024-05-20T16:11:53.846490+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T17:38:15.355878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:11:53.846213+0000","last_change":"2024-05-20T16:11:51.915537+0000","last_active":"2024-05-20T16:11:53.846213+0000","last_peered":"2024-05-20T16:11:53.846213+0000","last_clean":"2024-05-20T16:11:53.846213+0000","last_became_active":"2024-05-20T16:11:51.914973+0000","last_became_peered":"2024-05-20T16:11:51.914973+0000","last_unstale":"2024-05-20T16:11:53.846213+0000","last_undegraded":"2024-05-20T16:11:53.846213+0000","last_fullsized":"2024-05-20T16:11:53.846213+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_clean_scrub_stamp":"2024-05-20T16:11:50.398948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:53:19.724086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:11:53.419724+0000","last_change":"2024-05-20T16:11:46.402585+0000","last_active":"2024-05-20T16:11:53.419724+0000","last_peered":"2024-05-20T16:11:53.419724+0000","last_clean":"2024-05-20T16:11:53.419724+0000","last_became_active":"2024-05-20T16:11:46.402163+0000","last_became_peered":"2024-05-20T16:11:46.402163+0000","last_unstale":"2024-05-20T16:11:53.419724+0000","last_undegraded":"2024-05-20T16:11:53.419724+0000","last_fullsized":"2024-05-20T16:11:53.419724+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_clean_scrub_stamp":"2024-05-20T16:11:45.363714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:35:26.036970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:11:53.845878+0000","last_change":"2024-05-20T16:11:47.183088+0000","last_active":"2024-05-20T16:11:53.845878+0000","last_peered":"2024-05-20T16:11:53.845878+0000","last_clean":"2024-05-20T16:11:53.845878+0000","last_became_active":"2024-05-20T16:11:47.182608+0000","last_became_peered":"2024-05-20T16:11:47.182608+0000","last_unstale":"2024-05-20T16:11:53.845878+0000","last_undegraded":"2024-05-20T16:11:53.845878+0000","last_fullsized":"2024-05-20T16:11:53.845878+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T20:23:37.086778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:11:54.266770+0000","last_change":"2024-05-20T16:11:51.437420+0000","last_active":"2024-05-20T16:11:54.266770+0000","last_peered":"2024-05-20T16:11:54.266770+0000","last_clean":"2024-05-20T16:11:54.266770+0000","last_became_active":"2024-05-20T16:11:51.437018+0000","last_became_peered":"2024-05-20T16:11:51.437018+0000","last_unstale":"2024-05-20T16:11:54.266770+0000","last_undegraded":"2024-05-20T16:11:54.266770+0000","last_fullsized":"2024-05-20T16:11:54.266770+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_clean_scrub_stamp":"2024-05-20T16:11:50.398948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:03:30.650894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"91'1","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:11:53.589272+0000","last_change":"2024-05-20T16:11:46.400818+0000","last_active":"2024-05-20T16:11:53.589272+0000","last_peered":"2024-05-20T16:11:53.589272+0000","last_clean":"2024-05-20T16:11:53.589272+0000","last_became_active":"2024-05-20T16:11:46.400482+0000","last_became_peered":"2024-05-20T16:11:46.400482+0000","last_unstale":"2024-05-20T16:11:53.589272+0000","last_undegraded":"2024-05-20T16:11:53.589272+0000","last_fullsized":"2024-05-20T16:11:53.589272+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_clean_scrub_stamp":"2024-05-20T16:11:45.363714+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:13:13.968284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:11:53.845827+0000","last_change":"2024-05-20T16:11:47.187542+0000","last_active":"2024-05-20T16:11:53.845827+0000","last_peered":"2024-05-20T16:11:53.845827+0000","last_clean":"2024-05-20T16:11:53.845827+0000","last_became_active":"2024-05-20T16:11:47.187217+0000","last_became_peered":"2024-05-20T16:11:47.187217+0000","last_unstale":"2024-05-20T16:11:53.845827+0000","last_undegraded":"2024-05-20T16:11:53.845827+0000","last_fullsized":"2024-05-20T16:11:53.845827+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T19:53:54.608555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:11:54.266991+0000","last_change":"2024-05-20T16:11:51.915176+0000","last_active":"2024-05-20T16:11:54.266991+0000","last_peered":"2024-05-20T16:11:54.266991+0000","last_clean":"2024-05-20T16:11:54.266991+0000","last_became_active":"2024-05-20T16:11:51.914766+0000","last_became_peered":"2024-05-20T16:11:51.914766+0000","last_unstale":"2024-05-20T16:11:54.266991+0000","last_undegraded":"2024-05-20T16:11:54.266991+0000","last_fullsized":"2024-05-20T16:11:54.266991+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_clean_scrub_stamp":"2024-05-20T16:11:50.398948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T16:26:02.276646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:11:53.846197+0000","last_change":"2024-05-20T16:11:47.184502+0000","last_active":"2024-05-20T16:11:53.846197+0000","last_peered":"2024-05-20T16:11:53.846197+0000","last_clean":"2024-05-20T16:11:53.846197+0000","last_became_active":"2024-05-20T16:11:47.184123+0000","last_became_peered":"2024-05-20T16:11:47.184123+0000","last_unstale":"2024-05-20T16:11:53.846197+0000","last_undegraded":"2024-05-20T16:11:53.846197+0000","last_fullsized":"2024-05-20T16:11:53.846197+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:55:23.989206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"91'4","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:12:12.398907+0000","last_change":"2024-05-20T16:11:46.399731+0000","last_active":"2024-05-20T16:12:12.398907+0000","last_peered":"2024-05-20T16:12:12.398907+0000","last_clean":"2024-05-20T16:12:12.398907+0000","last_became_active":"2024-05-20T16:11:46.399349+0000","last_became_peered":"2024-05-20T16:11:46.399349+0000","last_unstale":"2024-05-20T16:12:12.398907+0000","last_undegraded":"2024-05-20T16:12:12.398907+0000","last_fullsized":"2024-05-20T16:12:12.398907+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_clean_scrub_stamp":"2024-05-20T16:11:45.363714+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T20:27:39.038116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"91'1","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:11:54.496742+0000","last_change":"2024-05-20T16:11:51.435547+0000","last_active":"2024-05-20T16:11:54.496742+0000","last_peered":"2024-05-20T16:11:54.496742+0000","last_clean":"2024-05-20T16:11:54.496742+0000","last_became_active":"2024-05-20T16:11:51.435224+0000","last_became_peered":"2024-05-20T16:11:51.435224+0000","last_unstale":"2024-05-20T16:11:54.496742+0000","last_undegraded":"2024-05-20T16:11:54.496742+0000","last_fullsized":"2024-05-20T16:11:54.496742+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_clean_scrub_stamp":"2024-05-20T16:11:50.398948+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:09:02.989494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:11:53.844875+0000","last_change":"2024-05-20T16:11:47.185746+0000","last_active":"2024-05-20T16:11:53.844875+0000","last_peered":"2024-05-20T16:11:53.844875+0000","last_clean":"2024-05-20T16:11:53.844875+0000","last_became_active":"2024-05-20T16:11:47.185399+0000","last_became_peered":"2024-05-20T16:11:47.185399+0000","last_unstale":"2024-05-20T16:11:53.844875+0000","last_undegraded":"2024-05-20T16:11:53.844875+0000","last_fullsized":"2024-05-20T16:11:53.844875+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:05:20.241804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"91'3","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:12:11.385524+0000","last_change":"2024-05-20T16:11:46.393867+0000","last_active":"2024-05-20T16:12:11.385524+0000","last_peered":"2024-05-20T16:12:11.385524+0000","last_clean":"2024-05-20T16:12:11.385524+0000","last_became_active":"2024-05-20T16:11:46.393523+0000","last_became_peered":"2024-05-20T16:11:46.393523+0000","last_unstale":"2024-05-20T16:12:11.385524+0000","last_undegraded":"2024-05-20T16:12:11.385524+0000","last_fullsized":"2024-05-20T16:12:11.385524+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_clean_scrub_stamp":"2024-05-20T16:11:45.363714+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T18:36:40.067975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":348,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:11:53.588927+0000","last_change":"2024-05-20T16:10:35.566958+0000","last_active":"2024-05-20T16:11:53.588927+0000","last_peered":"2024-05-20T16:11:53.588927+0000","last_clean":"2024-05-20T16:11:53.588927+0000","last_became_active":"2024-05-20T16:10:35.566551+0000","last_became_peered":"2024-05-20T16:10:35.566551+0000","last_unstale":"2024-05-20T16:11:53.588927+0000","last_undegraded":"2024-05-20T16:11:53.588927+0000","last_fullsized":"2024-05-20T16:11:53.588927+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:07:09.543878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:07:09.543878+0000","last_clean_scrub_stamp":"2024-05-20T16:07:09.543878+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:48:26.484769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:11:53.847419+0000","last_change":"2024-05-20T16:11:51.915534+0000","last_active":"2024-05-20T16:11:53.847419+0000","last_peered":"2024-05-20T16:11:53.847419+0000","last_clean":"2024-05-20T16:11:53.847419+0000","last_became_active":"2024-05-20T16:11:51.914903+0000","last_became_peered":"2024-05-20T16:11:51.914903+0000","last_unstale":"2024-05-20T16:11:53.847419+0000","last_undegraded":"2024-05-20T16:11:53.847419+0000","last_fullsized":"2024-05-20T16:11:53.847419+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_clean_scrub_stamp":"2024-05-20T16:11:50.398948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T18:56:47.255129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:11:54.266326+0000","last_change":"2024-05-20T16:11:47.174101+0000","last_active":"2024-05-20T16:11:54.266326+0000","last_peered":"2024-05-20T16:11:54.266326+0000","last_clean":"2024-05-20T16:11:54.266326+0000","last_became_active":"2024-05-20T16:11:47.173772+0000","last_became_peered":"2024-05-20T16:11:47.173772+0000","last_unstale":"2024-05-20T16:11:54.266326+0000","last_undegraded":"2024-05-20T16:11:54.266326+0000","last_fullsized":"2024-05-20T16:11:54.266326+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:33:33.656757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"91'3","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:12:12.398445+0000","last_change":"2024-05-20T16:11:46.401951+0000","last_active":"2024-05-20T16:12:12.398445+0000","last_peered":"2024-05-20T16:12:12.398445+0000","last_clean":"2024-05-20T16:12:12.398445+0000","last_became_active":"2024-05-20T16:11:46.401377+0000","last_became_peered":"2024-05-20T16:11:46.401377+0000","last_unstale":"2024-05-20T16:12:12.398445+0000","last_undegraded":"2024-05-20T16:12:12.398445+0000","last_fullsized":"2024-05-20T16:12:12.398445+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_clean_scrub_stamp":"2024-05-20T16:11:45.363714+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:00:43.538206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:11:53.846308+0000","last_change":"2024-05-20T16:11:51.436284+0000","last_active":"2024-05-20T16:11:53.846308+0000","last_peered":"2024-05-20T16:11:53.846308+0000","last_clean":"2024-05-20T16:11:53.846308+0000","last_became_active":"2024-05-20T16:11:51.435755+0000","last_became_peered":"2024-05-20T16:11:51.435755+0000","last_unstale":"2024-05-20T16:11:53.846308+0000","last_undegraded":"2024-05-20T16:11:53.846308+0000","last_fullsized":"2024-05-20T16:11:53.846308+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_clean_scrub_stamp":"2024-05-20T16:11:50.398948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T20:51:05.434690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"91'1","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:12:11.384345+0000","last_change":"2024-05-20T16:11:46.408055+0000","last_active":"2024-05-20T16:12:11.384345+0000","last_peered":"2024-05-20T16:12:11.384345+0000","last_clean":"2024-05-20T16:12:11.384345+0000","last_became_active":"2024-05-20T16:11:46.407733+0000","last_became_peered":"2024-05-20T16:11:46.407733+0000","last_unstale":"2024-05-20T16:12:11.384345+0000","last_undegraded":"2024-05-20T16:12:11.384345+0000","last_fullsized":"2024-05-20T16:12:11.384345+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_clean_scrub_stamp":"2024-05-20T16:11:45.363714+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T18:32:32.344334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:11:54.266857+0000","last_change":"2024-05-20T16:11:47.179831+0000","last_active":"2024-05-20T16:11:54.266857+0000","last_peered":"2024-05-20T16:11:54.266857+0000","last_clean":"2024-05-20T16:11:54.266857+0000","last_became_active":"2024-05-20T16:11:47.179501+0000","last_became_peered":"2024-05-20T16:11:47.179501+0000","last_unstale":"2024-05-20T16:11:54.266857+0000","last_undegraded":"2024-05-20T16:11:54.266857+0000","last_fullsized":"2024-05-20T16:11:54.266857+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T17:38:13.651180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:11:53.844736+0000","last_change":"2024-05-20T16:11:51.915210+0000","last_active":"2024-05-20T16:11:53.844736+0000","last_peered":"2024-05-20T16:11:53.844736+0000","last_clean":"2024-05-20T16:11:53.844736+0000","last_became_active":"2024-05-20T16:11:51.914751+0000","last_became_peered":"2024-05-20T16:11:51.914751+0000","last_unstale":"2024-05-20T16:11:53.844736+0000","last_undegraded":"2024-05-20T16:11:53.844736+0000","last_fullsized":"2024-05-20T16:11:53.844736+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_clean_scrub_stamp":"2024-05-20T16:11:50.398948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:07:14.817497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"86'1","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:11:53.420161+0000","last_change":"2024-05-20T16:11:46.403148+0000","last_active":"2024-05-20T16:11:53.420161+0000","last_peered":"2024-05-20T16:11:53.420161+0000","last_clean":"2024-05-20T16:11:53.420161+0000","last_became_active":"2024-05-20T16:11:46.402818+0000","last_became_peered":"2024-05-20T16:11:46.402818+0000","last_unstale":"2024-05-20T16:11:53.420161+0000","last_undegraded":"2024-05-20T16:11:53.420161+0000","last_fullsized":"2024-05-20T16:11:53.420161+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_clean_scrub_stamp":"2024-05-20T16:11:45.363714+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:49:52.095893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:11:53.845169+0000","last_change":"2024-05-20T16:11:47.184550+0000","last_active":"2024-05-20T16:11:53.845169+0000","last_peered":"2024-05-20T16:11:53.845169+0000","last_clean":"2024-05-20T16:11:53.845169+0000","last_became_active":"2024-05-20T16:11:47.184260+0000","last_became_peered":"2024-05-20T16:11:47.184260+0000","last_unstale":"2024-05-20T16:11:53.845169+0000","last_undegraded":"2024-05-20T16:11:53.845169+0000","last_fullsized":"2024-05-20T16:11:53.845169+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T17:21:43.013563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:11:53.420078+0000","last_change":"2024-05-20T16:11:51.431710+0000","last_active":"2024-05-20T16:11:53.420078+0000","last_peered":"2024-05-20T16:11:53.420078+0000","last_clean":"2024-05-20T16:11:53.420078+0000","last_became_active":"2024-05-20T16:11:51.431286+0000","last_became_peered":"2024-05-20T16:11:51.431286+0000","last_unstale":"2024-05-20T16:11:53.420078+0000","last_undegraded":"2024-05-20T16:11:53.420078+0000","last_fullsized":"2024-05-20T16:11:53.420078+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_clean_scrub_stamp":"2024-05-20T16:11:50.398948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:35:19.016412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"91'4","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:12:11.384319+0000","last_change":"2024-05-20T16:11:46.401772+0000","last_active":"2024-05-20T16:12:11.384319+0000","last_peered":"2024-05-20T16:12:11.384319+0000","last_clean":"2024-05-20T16:12:11.384319+0000","last_became_active":"2024-05-20T16:11:46.401414+0000","last_became_peered":"2024-05-20T16:11:46.401414+0000","last_unstale":"2024-05-20T16:12:11.384319+0000","last_undegraded":"2024-05-20T16:12:11.384319+0000","last_fullsized":"2024-05-20T16:12:11.384319+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_clean_scrub_stamp":"2024-05-20T16:11:45.363714+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:23:59.146573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:11:53.786667+0000","last_change":"2024-05-20T16:11:47.177333+0000","last_active":"2024-05-20T16:11:53.786667+0000","last_peered":"2024-05-20T16:11:53.786667+0000","last_clean":"2024-05-20T16:11:53.786667+0000","last_became_active":"2024-05-20T16:11:47.176161+0000","last_became_peered":"2024-05-20T16:11:47.176161+0000","last_unstale":"2024-05-20T16:11:53.786667+0000","last_undegraded":"2024-05-20T16:11:53.786667+0000","last_fullsized":"2024-05-20T16:11:53.786667+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:48:40.846477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:11:53.423145+0000","last_change":"2024-05-20T16:11:51.915348+0000","last_active":"2024-05-20T16:11:53.423145+0000","last_peered":"2024-05-20T16:11:53.423145+0000","last_clean":"2024-05-20T16:11:53.423145+0000","last_became_active":"2024-05-20T16:11:51.914864+0000","last_became_peered":"2024-05-20T16:11:51.914864+0000","last_unstale":"2024-05-20T16:11:53.423145+0000","last_undegraded":"2024-05-20T16:11:53.423145+0000","last_fullsized":"2024-05-20T16:11:53.423145+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_clean_scrub_stamp":"2024-05-20T16:11:50.398948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:02:26.846136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"91'2","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:12:11.387259+0000","last_change":"2024-05-20T16:11:46.407127+0000","last_active":"2024-05-20T16:12:11.387259+0000","last_peered":"2024-05-20T16:12:11.387259+0000","last_clean":"2024-05-20T16:12:11.387259+0000","last_became_active":"2024-05-20T16:11:46.406833+0000","last_became_peered":"2024-05-20T16:11:46.406833+0000","last_unstale":"2024-05-20T16:12:11.387259+0000","last_undegraded":"2024-05-20T16:12:11.387259+0000","last_fullsized":"2024-05-20T16:12:11.387259+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_clean_scrub_stamp":"2024-05-20T16:11:45.363714+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:58:30.464017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:11:53.845653+0000","last_change":"2024-05-20T16:11:47.174434+0000","last_active":"2024-05-20T16:11:53.845653+0000","last_peered":"2024-05-20T16:11:53.845653+0000","last_clean":"2024-05-20T16:11:53.845653+0000","last_became_active":"2024-05-20T16:11:47.174019+0000","last_became_peered":"2024-05-20T16:11:47.174019+0000","last_unstale":"2024-05-20T16:11:53.845653+0000","last_undegraded":"2024-05-20T16:11:53.845653+0000","last_fullsized":"2024-05-20T16:11:53.845653+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T18:32:51.150925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:11:53.420024+0000","last_change":"2024-05-20T16:11:51.446241+0000","last_active":"2024-05-20T16:11:53.420024+0000","last_peered":"2024-05-20T16:11:53.420024+0000","last_clean":"2024-05-20T16:11:53.420024+0000","last_became_active":"2024-05-20T16:11:51.445828+0000","last_became_peered":"2024-05-20T16:11:51.445828+0000","last_unstale":"2024-05-20T16:11:53.420024+0000","last_undegraded":"2024-05-20T16:11:53.420024+0000","last_fullsized":"2024-05-20T16:11:53.420024+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_clean_scrub_stamp":"2024-05-20T16:11:50.398948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:26:00.239338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"91'3","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:12:11.386469+0000","last_change":"2024-05-20T16:11:46.396364+0000","last_active":"2024-05-20T16:12:11.386469+0000","last_peered":"2024-05-20T16:12:11.386469+0000","last_clean":"2024-05-20T16:12:11.386469+0000","last_became_active":"2024-05-20T16:11:46.396014+0000","last_became_peered":"2024-05-20T16:11:46.396014+0000","last_unstale":"2024-05-20T16:12:11.386469+0000","last_undegraded":"2024-05-20T16:12:11.386469+0000","last_fullsized":"2024-05-20T16:12:11.386469+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_clean_scrub_stamp":"2024-05-20T16:11:45.363714+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:02:46.660077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:11:53.846016+0000","last_change":"2024-05-20T16:11:47.184320+0000","last_active":"2024-05-20T16:11:53.846016+0000","last_peered":"2024-05-20T16:11:53.846016+0000","last_clean":"2024-05-20T16:11:53.846016+0000","last_became_active":"2024-05-20T16:11:47.183981+0000","last_became_peered":"2024-05-20T16:11:47.183981+0000","last_unstale":"2024-05-20T16:11:53.846016+0000","last_undegraded":"2024-05-20T16:11:53.846016+0000","last_fullsized":"2024-05-20T16:11:53.846016+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T17:41:47.160563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:11:53.422872+0000","last_change":"2024-05-20T16:11:51.440073+0000","last_active":"2024-05-20T16:11:53.422872+0000","last_peered":"2024-05-20T16:11:53.422872+0000","last_clean":"2024-05-20T16:11:53.422872+0000","last_became_active":"2024-05-20T16:11:51.439560+0000","last_became_peered":"2024-05-20T16:11:51.439560+0000","last_unstale":"2024-05-20T16:11:53.422872+0000","last_undegraded":"2024-05-20T16:11:53.422872+0000","last_fullsized":"2024-05-20T16:11:53.422872+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_clean_scrub_stamp":"2024-05-20T16:11:50.398948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:50:54.429811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"91'1","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:11:53.420164+0000","last_change":"2024-05-20T16:11:46.404866+0000","last_active":"2024-05-20T16:11:53.420164+0000","last_peered":"2024-05-20T16:11:53.420164+0000","last_clean":"2024-05-20T16:11:53.420164+0000","last_became_active":"2024-05-20T16:11:46.404509+0000","last_became_peered":"2024-05-20T16:11:46.404509+0000","last_unstale":"2024-05-20T16:11:53.420164+0000","last_undegraded":"2024-05-20T16:11:53.420164+0000","last_fullsized":"2024-05-20T16:11:53.420164+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_clean_scrub_stamp":"2024-05-20T16:11:45.363714+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:58:40.303826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:11:53.844942+0000","last_change":"2024-05-20T16:11:47.191505+0000","last_active":"2024-05-20T16:11:53.844942+0000","last_peered":"2024-05-20T16:11:53.844942+0000","last_clean":"2024-05-20T16:11:53.844942+0000","last_became_active":"2024-05-20T16:11:47.191181+0000","last_became_peered":"2024-05-20T16:11:47.191181+0000","last_unstale":"2024-05-20T16:11:53.844942+0000","last_undegraded":"2024-05-20T16:11:53.844942+0000","last_fullsized":"2024-05-20T16:11:53.844942+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T19:30:42.561146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:11:53.423029+0000","last_change":"2024-05-20T16:11:51.915313+0000","last_active":"2024-05-20T16:11:53.423029+0000","last_peered":"2024-05-20T16:11:53.423029+0000","last_clean":"2024-05-20T16:11:53.423029+0000","last_became_active":"2024-05-20T16:11:51.914761+0000","last_became_peered":"2024-05-20T16:11:51.914761+0000","last_unstale":"2024-05-20T16:11:53.423029+0000","last_undegraded":"2024-05-20T16:11:53.423029+0000","last_fullsized":"2024-05-20T16:11:53.423029+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_clean_scrub_stamp":"2024-05-20T16:11:50.398948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T17:15:36.673449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"91'4","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:12:11.387439+0000","last_change":"2024-05-20T16:11:46.404513+0000","last_active":"2024-05-20T16:12:11.387439+0000","last_peered":"2024-05-20T16:12:11.387439+0000","last_clean":"2024-05-20T16:12:11.387439+0000","last_became_active":"2024-05-20T16:11:46.404163+0000","last_became_peered":"2024-05-20T16:11:46.404163+0000","last_unstale":"2024-05-20T16:12:11.387439+0000","last_undegraded":"2024-05-20T16:12:11.387439+0000","last_fullsized":"2024-05-20T16:12:11.387439+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_clean_scrub_stamp":"2024-05-20T16:11:45.363714+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:24:07.155558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:11:53.589347+0000","last_change":"2024-05-20T16:11:47.174527+0000","last_active":"2024-05-20T16:11:53.589347+0000","last_peered":"2024-05-20T16:11:53.589347+0000","last_clean":"2024-05-20T16:11:53.589347+0000","last_became_active":"2024-05-20T16:11:47.174165+0000","last_became_peered":"2024-05-20T16:11:47.174165+0000","last_unstale":"2024-05-20T16:11:53.589347+0000","last_undegraded":"2024-05-20T16:11:53.589347+0000","last_fullsized":"2024-05-20T16:11:53.589347+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T17:44:47.614704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:11:53.771534+0000","last_change":"2024-05-20T16:11:51.439013+0000","last_active":"2024-05-20T16:11:53.771534+0000","last_peered":"2024-05-20T16:11:53.771534+0000","last_clean":"2024-05-20T16:11:53.771534+0000","last_became_active":"2024-05-20T16:11:51.438408+0000","last_became_peered":"2024-05-20T16:11:51.438408+0000","last_unstale":"2024-05-20T16:11:53.771534+0000","last_undegraded":"2024-05-20T16:11:53.771534+0000","last_fullsized":"2024-05-20T16:11:53.771534+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_clean_scrub_stamp":"2024-05-20T16:11:50.398948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T19:27:53.723823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"91'3","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:12:12.397985+0000","last_change":"2024-05-20T16:11:46.407283+0000","last_active":"2024-05-20T16:12:12.397985+0000","last_peered":"2024-05-20T16:12:12.397985+0000","last_clean":"2024-05-20T16:12:12.397985+0000","last_became_active":"2024-05-20T16:11:46.406919+0000","last_became_peered":"2024-05-20T16:11:46.406919+0000","last_unstale":"2024-05-20T16:12:12.397985+0000","last_undegraded":"2024-05-20T16:12:12.397985+0000","last_fullsized":"2024-05-20T16:12:12.397985+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_clean_scrub_stamp":"2024-05-20T16:11:45.363714+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:16:09.166422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:11:53.589710+0000","last_change":"2024-05-20T16:11:47.177973+0000","last_active":"2024-05-20T16:11:53.589710+0000","last_peered":"2024-05-20T16:11:53.589710+0000","last_clean":"2024-05-20T16:11:53.589710+0000","last_became_active":"2024-05-20T16:11:47.177658+0000","last_became_peered":"2024-05-20T16:11:47.177658+0000","last_unstale":"2024-05-20T16:11:53.589710+0000","last_undegraded":"2024-05-20T16:11:53.589710+0000","last_fullsized":"2024-05-20T16:11:53.589710+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:34:04.004730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:11:53.786483+0000","last_change":"2024-05-20T16:11:51.441326+0000","last_active":"2024-05-20T16:11:53.786483+0000","last_peered":"2024-05-20T16:11:53.786483+0000","last_clean":"2024-05-20T16:11:53.786483+0000","last_became_active":"2024-05-20T16:11:51.440903+0000","last_became_peered":"2024-05-20T16:11:51.440903+0000","last_unstale":"2024-05-20T16:11:53.786483+0000","last_undegraded":"2024-05-20T16:11:53.786483+0000","last_fullsized":"2024-05-20T16:11:53.786483+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_clean_scrub_stamp":"2024-05-20T16:11:50.398948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:52:54.988916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"85'1","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:11:53.847073+0000","last_change":"2024-05-20T16:11:46.397548+0000","last_active":"2024-05-20T16:11:53.847073+0000","last_peered":"2024-05-20T16:11:53.847073+0000","last_clean":"2024-05-20T16:11:53.847073+0000","last_became_active":"2024-05-20T16:11:46.397207+0000","last_became_peered":"2024-05-20T16:11:46.397207+0000","last_unstale":"2024-05-20T16:11:53.847073+0000","last_undegraded":"2024-05-20T16:11:53.847073+0000","last_fullsized":"2024-05-20T16:11:53.847073+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_clean_scrub_stamp":"2024-05-20T16:11:45.363714+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:25:50.468298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:11:53.589596+0000","last_change":"2024-05-20T16:11:47.184162+0000","last_active":"2024-05-20T16:11:53.589596+0000","last_peered":"2024-05-20T16:11:53.589596+0000","last_clean":"2024-05-20T16:11:53.589596+0000","last_became_active":"2024-05-20T16:11:47.183845+0000","last_became_peered":"2024-05-20T16:11:47.183845+0000","last_unstale":"2024-05-20T16:11:53.589596+0000","last_undegraded":"2024-05-20T16:11:53.589596+0000","last_fullsized":"2024-05-20T16:11:53.589596+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T20:23:14.728178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:11:53.845283+0000","last_change":"2024-05-20T16:11:51.437974+0000","last_active":"2024-05-20T16:11:53.845283+0000","last_peered":"2024-05-20T16:11:53.845283+0000","last_clean":"2024-05-20T16:11:53.845283+0000","last_became_active":"2024-05-20T16:11:51.437053+0000","last_became_peered":"2024-05-20T16:11:51.437053+0000","last_unstale":"2024-05-20T16:11:53.845283+0000","last_undegraded":"2024-05-20T16:11:53.845283+0000","last_fullsized":"2024-05-20T16:11:53.845283+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_clean_scrub_stamp":"2024-05-20T16:11:50.398948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:09:31.314668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:11:54.266887+0000","last_change":"2024-05-20T16:11:46.398461+0000","last_active":"2024-05-20T16:11:54.266887+0000","last_peered":"2024-05-20T16:11:54.266887+0000","last_clean":"2024-05-20T16:11:54.266887+0000","last_became_active":"2024-05-20T16:11:46.398122+0000","last_became_peered":"2024-05-20T16:11:46.398122+0000","last_unstale":"2024-05-20T16:11:54.266887+0000","last_undegraded":"2024-05-20T16:11:54.266887+0000","last_fullsized":"2024-05-20T16:11:54.266887+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_clean_scrub_stamp":"2024-05-20T16:11:45.363714+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:34:45.592470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:11:53.589683+0000","last_change":"2024-05-20T16:11:47.179160+0000","last_active":"2024-05-20T16:11:53.589683+0000","last_peered":"2024-05-20T16:11:53.589683+0000","last_clean":"2024-05-20T16:11:53.589683+0000","last_became_active":"2024-05-20T16:11:47.178843+0000","last_became_peered":"2024-05-20T16:11:47.178843+0000","last_unstale":"2024-05-20T16:11:53.589683+0000","last_undegraded":"2024-05-20T16:11:53.589683+0000","last_fullsized":"2024-05-20T16:11:53.589683+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T20:05:57.013207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:11:53.846191+0000","last_change":"2024-05-20T16:11:47.185401+0000","last_active":"2024-05-20T16:11:53.846191+0000","last_peered":"2024-05-20T16:11:53.846191+0000","last_clean":"2024-05-20T16:11:53.846191+0000","last_became_active":"2024-05-20T16:11:47.185061+0000","last_became_peered":"2024-05-20T16:11:47.185061+0000","last_unstale":"2024-05-20T16:11:53.846191+0000","last_undegraded":"2024-05-20T16:11:53.846191+0000","last_fullsized":"2024-05-20T16:11:53.846191+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:33:32.528968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:11:53.420412+0000","last_change":"2024-05-20T16:11:47.174418+0000","last_active":"2024-05-20T16:11:53.420412+0000","last_peered":"2024-05-20T16:11:53.420412+0000","last_clean":"2024-05-20T16:11:53.420412+0000","last_became_active":"2024-05-20T16:11:47.174078+0000","last_became_peered":"2024-05-20T16:11:47.174078+0000","last_unstale":"2024-05-20T16:11:53.420412+0000","last_undegraded":"2024-05-20T16:11:53.420412+0000","last_fullsized":"2024-05-20T16:11:53.420412+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T16:20:12.268161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:11:53.420140+0000","last_change":"2024-05-20T16:11:47.177686+0000","last_active":"2024-05-20T16:11:53.420140+0000","last_peered":"2024-05-20T16:11:53.420140+0000","last_clean":"2024-05-20T16:11:53.420140+0000","last_became_active":"2024-05-20T16:11:47.177305+0000","last_became_peered":"2024-05-20T16:11:47.177305+0000","last_unstale":"2024-05-20T16:11:53.420140+0000","last_undegraded":"2024-05-20T16:11:53.420140+0000","last_fullsized":"2024-05-20T16:11:53.420140+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T18:36:35.919491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:11:53.786492+0000","last_change":"2024-05-20T16:11:47.175383+0000","last_active":"2024-05-20T16:11:53.786492+0000","last_peered":"2024-05-20T16:11:53.786492+0000","last_clean":"2024-05-20T16:11:53.786492+0000","last_became_active":"2024-05-20T16:11:47.175079+0000","last_became_peered":"2024-05-20T16:11:47.175079+0000","last_unstale":"2024-05-20T16:11:53.786492+0000","last_undegraded":"2024-05-20T16:11:53.786492+0000","last_fullsized":"2024-05-20T16:11:53.786492+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:33:52.072038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:11:54.266936+0000","last_change":"2024-05-20T16:11:47.182717+0000","last_active":"2024-05-20T16:11:54.266936+0000","last_peered":"2024-05-20T16:11:54.266936+0000","last_clean":"2024-05-20T16:11:54.266936+0000","last_became_active":"2024-05-20T16:11:47.182408+0000","last_became_peered":"2024-05-20T16:11:47.182408+0000","last_unstale":"2024-05-20T16:11:54.266936+0000","last_undegraded":"2024-05-20T16:11:54.266936+0000","last_fullsized":"2024-05-20T16:11:54.266936+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T18:25:13.698066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:11:53.846571+0000","last_change":"2024-05-20T16:11:47.182804+0000","last_active":"2024-05-20T16:11:53.846571+0000","last_peered":"2024-05-20T16:11:53.846571+0000","last_clean":"2024-05-20T16:11:53.846571+0000","last_became_active":"2024-05-20T16:11:47.182486+0000","last_became_peered":"2024-05-20T16:11:47.182486+0000","last_unstale":"2024-05-20T16:11:53.846571+0000","last_undegraded":"2024-05-20T16:11:53.846571+0000","last_fullsized":"2024-05-20T16:11:53.846571+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:24:52.592522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:11:53.420848+0000","last_change":"2024-05-20T16:11:47.175689+0000","last_active":"2024-05-20T16:11:53.420848+0000","last_peered":"2024-05-20T16:11:53.420848+0000","last_clean":"2024-05-20T16:11:53.420848+0000","last_became_active":"2024-05-20T16:11:47.168216+0000","last_became_peered":"2024-05-20T16:11:47.168216+0000","last_unstale":"2024-05-20T16:11:53.420848+0000","last_undegraded":"2024-05-20T16:11:53.420848+0000","last_fullsized":"2024-05-20T16:11:53.420848+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:50:52.707138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:11:53.845259+0000","last_change":"2024-05-20T16:11:47.191398+0000","last_active":"2024-05-20T16:11:53.845259+0000","last_peered":"2024-05-20T16:11:53.845259+0000","last_clean":"2024-05-20T16:11:53.845259+0000","last_became_active":"2024-05-20T16:11:47.191036+0000","last_became_peered":"2024-05-20T16:11:47.191036+0000","last_unstale":"2024-05-20T16:11:53.845259+0000","last_undegraded":"2024-05-20T16:11:53.845259+0000","last_fullsized":"2024-05-20T16:11:53.845259+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:27:49.023581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:11:53.845753+0000","last_change":"2024-05-20T16:11:47.184206+0000","last_active":"2024-05-20T16:11:53.845753+0000","last_peered":"2024-05-20T16:11:53.845753+0000","last_clean":"2024-05-20T16:11:53.845753+0000","last_became_active":"2024-05-20T16:11:47.183860+0000","last_became_peered":"2024-05-20T16:11:47.183860+0000","last_unstale":"2024-05-20T16:11:53.845753+0000","last_undegraded":"2024-05-20T16:11:53.845753+0000","last_fullsized":"2024-05-20T16:11:53.845753+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:52:42.507615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:11:53.844883+0000","last_change":"2024-05-20T16:11:47.180298+0000","last_active":"2024-05-20T16:11:53.844883+0000","last_peered":"2024-05-20T16:11:53.844883+0000","last_clean":"2024-05-20T16:11:53.844883+0000","last_became_active":"2024-05-20T16:11:47.179996+0000","last_became_peered":"2024-05-20T16:11:47.179996+0000","last_unstale":"2024-05-20T16:11:53.844883+0000","last_undegraded":"2024-05-20T16:11:53.844883+0000","last_fullsized":"2024-05-20T16:11:53.844883+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:49:53.432240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:11:53.771592+0000","last_change":"2024-05-20T16:11:47.176123+0000","last_active":"2024-05-20T16:11:53.771592+0000","last_peered":"2024-05-20T16:11:53.771592+0000","last_clean":"2024-05-20T16:11:53.771592+0000","last_became_active":"2024-05-20T16:11:47.175019+0000","last_became_peered":"2024-05-20T16:11:47.175019+0000","last_unstale":"2024-05-20T16:11:53.771592+0000","last_undegraded":"2024-05-20T16:11:53.771592+0000","last_fullsized":"2024-05-20T16:11:53.771592+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:23:32.010352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:11:54.266628+0000","last_change":"2024-05-20T16:11:47.180877+0000","last_active":"2024-05-20T16:11:54.266628+0000","last_peered":"2024-05-20T16:11:54.266628+0000","last_clean":"2024-05-20T16:11:54.266628+0000","last_became_active":"2024-05-20T16:11:47.180567+0000","last_became_peered":"2024-05-20T16:11:47.180567+0000","last_unstale":"2024-05-20T16:11:54.266628+0000","last_undegraded":"2024-05-20T16:11:54.266628+0000","last_fullsized":"2024-05-20T16:11:54.266628+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T17:54:31.165693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:11:53.846436+0000","last_change":"2024-05-20T16:11:47.179607+0000","last_active":"2024-05-20T16:11:53.846436+0000","last_peered":"2024-05-20T16:11:53.846436+0000","last_clean":"2024-05-20T16:11:53.846436+0000","last_became_active":"2024-05-20T16:11:47.179279+0000","last_became_peered":"2024-05-20T16:11:47.179279+0000","last_unstale":"2024-05-20T16:11:53.846436+0000","last_undegraded":"2024-05-20T16:11:53.846436+0000","last_fullsized":"2024-05-20T16:11:53.846436+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T18:38:20.788325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":246552,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":7452,"num_objects":56,"num_object_clones":0,"num_object_copies":168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":119,"num_write_kb":43,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":36,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":229376,"data_stored":39952,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":104,"ondisk_log_size":104,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892350992,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":983040,"data_stored":489673,"data_compressed":17602,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612630,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28060,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717092,"statfs":{"total":95995035648,"available":95966302208,"internally_reserved":0,"allocated":1069056,"data_stored":1084839,"data_compressed":53292,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":1598,"internal_metadata":27654594},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776156,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":987136,"data_stored":490255,"data_compressed":17602,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037793,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32156,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712996,"statfs":{"total":95995035648,"available":95962107904,"internally_reserved":0,"allocated":1003520,"data_stored":490411,"data_compressed":17602,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299431,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32200,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712952,"statfs":{"total":95995035648,"available":95962062848,"internally_reserved":0,"allocated":1048576,"data_stored":1081833,"data_compressed":53292,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462956,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32140,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713012,"statfs":{"total":95995035648,"available":95962124288,"internally_reserved":0,"allocated":987136,"data_stored":489763,"data_compressed":17602,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574917,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":1003520,"data_stored":500115,"data_compressed":17602,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738441,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":987136,"data_stored":489763,"data_compressed":17602,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411391,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1019904,"data_stored":494865,"data_compressed":17602,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149754,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":991232,"data_stored":489777,"data_compressed":17602,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888118,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":995328,"data_stored":490317,"data_compressed":17602,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724592,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1060864,"data_stored":1081517,"data_compressed":53292,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34051,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34051,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35690,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35690,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35690,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35690,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35690,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":23141,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":5192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":5420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-20T16:12:16.239 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-20T16:12:16.387 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:16.385+0000 7faeff59e640 1 -- 172.21.15.82:0/1277104388 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faf00102a30 msgr2=0x7faf00102e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:12:16.387 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:16.385+0000 7faeff59e640 1 --2- 172.21.15.82:0/1277104388 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faf00102a30 0x7faf00102e40 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7faef4009900 tx=0x7faef402a410 comp rx=0 tx=0).stop 2024-05-20T16:12:16.387 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:16.385+0000 7faeff59e640 1 -- 172.21.15.82:0/1277104388 shutdown_connections 2024-05-20T16:12:16.387 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:16.385+0000 7faeff59e640 1 --2- 172.21.15.82:0/1277104388 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faf00102a30 0x7faf00102e40 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:16.387 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:16.385+0000 7faeff59e640 1 --2- 172.21.15.82:0/1277104388 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faf00107730 0x7faf00109b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:16.387 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:16.385+0000 7faeff59e640 1 --2- 172.21.15.82:0/1277104388 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faf001017f0 0x7faf001024f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:16.387 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:16.385+0000 7faeff59e640 1 -- 172.21.15.82:0/1277104388 >> 172.21.15.82:0/1277104388 conn(0x7faf000fc9e0 msgr2=0x7faf000fee90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:12:16.387 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:16.385+0000 7faeff59e640 1 -- 172.21.15.82:0/1277104388 shutdown_connections 2024-05-20T16:12:16.388 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:16.385+0000 7faeff59e640 1 -- 172.21.15.82:0/1277104388 wait complete. 2024-05-20T16:12:16.388 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:16.385+0000 7faeff59e640 1 Processor -- start 2024-05-20T16:12:16.388 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:16.386+0000 7faeff59e640 1 -- start start 2024-05-20T16:12:16.388 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:16.386+0000 7faeff59e640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faf001017f0 0x7faf001990b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:12:16.388 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:16.386+0000 7faeff59e640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faf00102a30 0x7faf001995f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:12:16.388 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:16.386+0000 7faeff59e640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faf00107730 0x7faf001a0580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:12:16.388 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:16.386+0000 7faeff59e640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7faf0010d440 con 0x7faf00107730 2024-05-20T16:12:16.388 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:16.386+0000 7faeff59e640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7faf0010d2c0 con 0x7faf001017f0 2024-05-20T16:12:16.388 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:16.386+0000 7faeff59e640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7faf0010d5c0 con 0x7faf00102a30 2024-05-20T16:12:16.388 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:16.386+0000 7faefed9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faf00107730 0x7faf001a0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:12:16.388 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:16.386+0000 7faefed9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faf00107730 0x7faf001a0580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:47282/0 (socket says 172.21.15.82:47282) 2024-05-20T16:12:16.388 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:16.386+0000 7faefed9d640 1 -- 172.21.15.82:0/1607523674 learned_addr learned my addr 172.21.15.82:0/1607523674 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:12:16.388 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:16.386+0000 7faefdd9b640 1 --2- 172.21.15.82:0/1607523674 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faf00102a30 0x7faf001995f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:12:16.389 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:16.386+0000 7faefed9d640 1 -- 172.21.15.82:0/1607523674 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faf00102a30 msgr2=0x7faf001995f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:12:16.389 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:16.386+0000 7faefed9d640 1 --2- 172.21.15.82:0/1607523674 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faf00102a30 0x7faf001995f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:16.389 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:16.386+0000 7faefe59c640 1 --2- 172.21.15.82:0/1607523674 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faf001017f0 0x7faf001990b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:12:16.389 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:16.386+0000 7faefed9d640 1 -- 172.21.15.82:0/1607523674 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faf001017f0 msgr2=0x7faf001990b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:12:16.389 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:16.386+0000 7faefed9d640 1 --2- 172.21.15.82:0/1607523674 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faf001017f0 0x7faf001990b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:16.389 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:16.386+0000 7faefed9d640 1 -- 172.21.15.82:0/1607523674 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf001a0bf0 con 0x7faf00107730 2024-05-20T16:12:16.389 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:16.386+0000 7faefed9d640 1 --2- 172.21.15.82:0/1607523674 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faf00107730 0x7faf001a0580 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7faef800c970 tx=0x7faef800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:12:16.389 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:16.387+0000 7faeef7fe640 1 -- 172.21.15.82:0/1607523674 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faef8007bb0 con 0x7faf00107730 2024-05-20T16:12:16.389 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:16.387+0000 7faeef7fe640 1 -- 172.21.15.82:0/1607523674 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faef8007d50 con 0x7faf00107730 2024-05-20T16:12:16.389 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:16.387+0000 7faeff59e640 1 -- 172.21.15.82:0/1607523674 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf001a0ee0 con 0x7faf00107730 2024-05-20T16:12:16.389 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:16.387+0000 7faeef7fe640 1 -- 172.21.15.82:0/1607523674 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faef800f450 con 0x7faf00107730 2024-05-20T16:12:16.389 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:16.387+0000 7faeff59e640 1 -- 172.21.15.82:0/1607523674 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf001a1340 con 0x7faf00107730 2024-05-20T16:12:16.390 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:16.388+0000 7faeff59e640 1 -- 172.21.15.82:0/1607523674 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf001a15d0 con 0x7faf00107730 2024-05-20T16:12:16.390 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:16.389+0000 7faeef7fe640 1 -- 172.21.15.82:0/1607523674 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7faef8004f00 con 0x7faf00107730 2024-05-20T16:12:16.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:16.390+0000 7faeef7fe640 1 --2- 172.21.15.82:0/1607523674 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7faee80a2c10 0x7faee80a5120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:12:16.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:16.390+0000 7faefe59c640 1 --2- 172.21.15.82:0/1607523674 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7faee80a2c10 0x7faee80a5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:12:16.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:16.390+0000 7faeef7fe640 1 -- 172.21.15.82:0/1607523674 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8802+0+0 (secure 0 0 0) 0x7faef80cf4a0 con 0x7faf00107730 2024-05-20T16:12:16.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:16.390+0000 7faefe59c640 1 --2- 172.21.15.82:0/1607523674 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7faee80a2c10 0x7faee80a5120 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7faf001023f0 tx=0x7faef0008230 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:12:16.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:16.393+0000 7faeef7fe640 1 -- 172.21.15.82:0/1607523674 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7faef8014030 con 0x7faf00107730 2024-05-20T16:12:16.568 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:16.568+0000 7faeff59e640 1 -- 172.21.15.82:0/1607523674 --> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7faf001a17a0 con 0x7faee80a2c10 2024-05-20T16:12:16.577 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:16.577+0000 7faeef7fe640 1 -- 172.21.15.82:0/1607523674 <== mgr.14150 v2:172.21.15.82:6800/379530823 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340559 (secure 0 0 0) 0x7faf001a17a0 con 0x7faee80a2c10 2024-05-20T16:12:16.581 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-05-20T16:12:16.583 INFO:teuthology.orchestra.run.smithi082.stderr:dumped all 2024-05-20T16:12:16.585 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:16.581+0000 7faeff59e640 1 -- 172.21.15.82:0/1607523674 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7faee80a2c10 msgr2=0x7faee80a5120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:12:16.585 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:16.581+0000 7faeff59e640 1 --2- 172.21.15.82:0/1607523674 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7faee80a2c10 0x7faee80a5120 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7faf001023f0 tx=0x7faef0008230 comp rx=0 tx=0).stop 2024-05-20T16:12:16.585 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:16.581+0000 7faeff59e640 1 -- 172.21.15.82:0/1607523674 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faf00107730 msgr2=0x7faf001a0580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:12:16.585 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:16.581+0000 7faeff59e640 1 --2- 172.21.15.82:0/1607523674 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faf00107730 0x7faf001a0580 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7faef800c970 tx=0x7faef800ce30 comp rx=0 tx=0).stop 2024-05-20T16:12:16.585 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:16.581+0000 7faeff59e640 1 -- 172.21.15.82:0/1607523674 shutdown_connections 2024-05-20T16:12:16.585 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:16.581+0000 7faeff59e640 1 --2- 172.21.15.82:0/1607523674 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7faee80a2c10 0x7faee80a5120 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:16.585 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:16.581+0000 7faeff59e640 1 --2- 172.21.15.82:0/1607523674 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faf00107730 0x7faf001a0580 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:16.585 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:16.581+0000 7faeff59e640 1 --2- 172.21.15.82:0/1607523674 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faf00102a30 0x7faf001995f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:16.585 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:16.581+0000 7faeff59e640 1 --2- 172.21.15.82:0/1607523674 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faf001017f0 0x7faf001990b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:16.585 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:16.581+0000 7faeff59e640 1 -- 172.21.15.82:0/1607523674 >> 172.21.15.82:0/1607523674 conn(0x7faf000fc9e0 msgr2=0x7faf000ffeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:12:16.585 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:16.581+0000 7faeff59e640 1 -- 172.21.15.82:0/1607523674 shutdown_connections 2024-05-20T16:12:16.585 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:16.581+0000 7faeff59e640 1 -- 172.21.15.82:0/1607523674 wait complete. 2024-05-20T16:12:16.591 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:12:16 smithi082 ceph-mon[32238]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-05-20T16:12:16.591 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:12:16 smithi082 ceph-mon[32238]: from='client.? 172.21.15.82:0/1454241362' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T16:12:16.591 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:12:16 smithi082 ceph-mon[32238]: from='client.? 172.21.15.82:0/1445240663' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-20T16:12:16.591 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:12:16 smithi082 ceph-mon[32238]: from='client.14937 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-20T16:12:16.592 INFO:teuthology.orchestra.run.smithi082.stdout:{"pg_ready":true,"pg_map":{"version":306,"stamp":"2024-05-20T16:12:15.256713+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":597937,"num_objects":59,"num_object_clones":0,"num_object_copies":178,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":0,"num_read":132,"num_read_kb":115,"num_write":301,"num_write_kb":2982,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":36,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":242,"ondisk_log_size":242,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":348544,"kb_used_data":11852,"kb_used_omap":18,"kb_used_meta":336557,"kb_avail":1124593280,"statfs":{"total":1151940427776,"available":1151583518720,"internally_reserved":0,"allocated":12136448,"data_stored":7673128,"data_compressed":318294,"data_compressed_allocated":430080,"data_compressed_original":2359296,"omap_allocated":19141,"internal_metadata":344634683},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":3070,"num_objects":35,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":73,"num_write_kb":27,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":22,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004139"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:11:53.846675+0000","last_change":"2024-05-20T16:11:47.187104+0000","last_active":"2024-05-20T16:11:53.846675+0000","last_peered":"2024-05-20T16:11:53.846675+0000","last_clean":"2024-05-20T16:11:53.846675+0000","last_became_active":"2024-05-20T16:11:47.186791+0000","last_became_peered":"2024-05-20T16:11:47.186791+0000","last_unstale":"2024-05-20T16:11:53.846675+0000","last_undegraded":"2024-05-20T16:11:53.846675+0000","last_fullsized":"2024-05-20T16:11:53.846675+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:33:38.914389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:11:53.786549+0000","last_change":"2024-05-20T16:11:47.187846+0000","last_active":"2024-05-20T16:11:53.786549+0000","last_peered":"2024-05-20T16:11:53.786549+0000","last_clean":"2024-05-20T16:11:53.786549+0000","last_became_active":"2024-05-20T16:11:47.187540+0000","last_became_peered":"2024-05-20T16:11:47.187540+0000","last_unstale":"2024-05-20T16:11:53.786549+0000","last_undegraded":"2024-05-20T16:11:53.786549+0000","last_fullsized":"2024-05-20T16:11:53.786549+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:19:59.784437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:11:53.786527+0000","last_change":"2024-05-20T16:11:47.185215+0000","last_active":"2024-05-20T16:11:53.786527+0000","last_peered":"2024-05-20T16:11:53.786527+0000","last_clean":"2024-05-20T16:11:53.786527+0000","last_became_active":"2024-05-20T16:11:47.184880+0000","last_became_peered":"2024-05-20T16:11:47.184880+0000","last_unstale":"2024-05-20T16:11:53.786527+0000","last_undegraded":"2024-05-20T16:11:53.786527+0000","last_fullsized":"2024-05-20T16:11:53.786527+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T18:43:32.125643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:11:53.423464+0000","last_change":"2024-05-20T16:11:47.175849+0000","last_active":"2024-05-20T16:11:53.423464+0000","last_peered":"2024-05-20T16:11:53.423464+0000","last_clean":"2024-05-20T16:11:53.423464+0000","last_became_active":"2024-05-20T16:11:47.175342+0000","last_became_peered":"2024-05-20T16:11:47.175342+0000","last_unstale":"2024-05-20T16:11:53.423464+0000","last_undegraded":"2024-05-20T16:11:53.423464+0000","last_fullsized":"2024-05-20T16:11:53.423464+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:02:43.798388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:11:53.846865+0000","last_change":"2024-05-20T16:11:47.174798+0000","last_active":"2024-05-20T16:11:53.846865+0000","last_peered":"2024-05-20T16:11:53.846865+0000","last_clean":"2024-05-20T16:11:53.846865+0000","last_became_active":"2024-05-20T16:11:47.174474+0000","last_became_peered":"2024-05-20T16:11:47.174474+0000","last_unstale":"2024-05-20T16:11:53.846865+0000","last_undegraded":"2024-05-20T16:11:53.846865+0000","last_fullsized":"2024-05-20T16:11:53.846865+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:41:24.404901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:11:53.845635+0000","last_change":"2024-05-20T16:11:47.185731+0000","last_active":"2024-05-20T16:11:53.845635+0000","last_peered":"2024-05-20T16:11:53.845635+0000","last_clean":"2024-05-20T16:11:53.845635+0000","last_became_active":"2024-05-20T16:11:47.185380+0000","last_became_peered":"2024-05-20T16:11:47.185380+0000","last_unstale":"2024-05-20T16:11:53.845635+0000","last_undegraded":"2024-05-20T16:11:53.845635+0000","last_fullsized":"2024-05-20T16:11:53.845635+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T16:33:28.854205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:11:53.589081+0000","last_change":"2024-05-20T16:11:47.183594+0000","last_active":"2024-05-20T16:11:53.589081+0000","last_peered":"2024-05-20T16:11:53.589081+0000","last_clean":"2024-05-20T16:11:53.589081+0000","last_became_active":"2024-05-20T16:11:47.183260+0000","last_became_peered":"2024-05-20T16:11:47.183260+0000","last_unstale":"2024-05-20T16:11:53.589081+0000","last_undegraded":"2024-05-20T16:11:53.589081+0000","last_fullsized":"2024-05-20T16:11:53.589081+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T17:12:22.781236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:11:53.844654+0000","last_change":"2024-05-20T16:11:47.191784+0000","last_active":"2024-05-20T16:11:53.844654+0000","last_peered":"2024-05-20T16:11:53.844654+0000","last_clean":"2024-05-20T16:11:53.844654+0000","last_became_active":"2024-05-20T16:11:47.190553+0000","last_became_peered":"2024-05-20T16:11:47.190553+0000","last_unstale":"2024-05-20T16:11:53.844654+0000","last_undegraded":"2024-05-20T16:11:53.844654+0000","last_fullsized":"2024-05-20T16:11:53.844654+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T20:17:29.346688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:11:53.771929+0000","last_change":"2024-05-20T16:11:47.174817+0000","last_active":"2024-05-20T16:11:53.771929+0000","last_peered":"2024-05-20T16:11:53.771929+0000","last_clean":"2024-05-20T16:11:53.771929+0000","last_became_active":"2024-05-20T16:11:47.174483+0000","last_became_peered":"2024-05-20T16:11:47.174483+0000","last_unstale":"2024-05-20T16:11:53.771929+0000","last_undegraded":"2024-05-20T16:11:53.771929+0000","last_fullsized":"2024-05-20T16:11:53.771929+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T20:32:30.284912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:11:53.846029+0000","last_change":"2024-05-20T16:11:47.186899+0000","last_active":"2024-05-20T16:11:53.846029+0000","last_peered":"2024-05-20T16:11:53.846029+0000","last_clean":"2024-05-20T16:11:53.846029+0000","last_became_active":"2024-05-20T16:11:47.186564+0000","last_became_peered":"2024-05-20T16:11:47.186564+0000","last_unstale":"2024-05-20T16:11:53.846029+0000","last_undegraded":"2024-05-20T16:11:53.846029+0000","last_fullsized":"2024-05-20T16:11:53.846029+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:19:57.110107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:11:53.589667+0000","last_change":"2024-05-20T16:11:47.182696+0000","last_active":"2024-05-20T16:11:53.589667+0000","last_peered":"2024-05-20T16:11:53.589667+0000","last_clean":"2024-05-20T16:11:53.589667+0000","last_became_active":"2024-05-20T16:11:47.182169+0000","last_became_peered":"2024-05-20T16:11:47.182169+0000","last_unstale":"2024-05-20T16:11:53.589667+0000","last_undegraded":"2024-05-20T16:11:53.589667+0000","last_fullsized":"2024-05-20T16:11:53.589667+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:55:16.413493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:11:53.423405+0000","last_change":"2024-05-20T16:11:47.181716+0000","last_active":"2024-05-20T16:11:53.423405+0000","last_peered":"2024-05-20T16:11:53.423405+0000","last_clean":"2024-05-20T16:11:53.423405+0000","last_became_active":"2024-05-20T16:11:47.181403+0000","last_became_peered":"2024-05-20T16:11:47.181403+0000","last_unstale":"2024-05-20T16:11:53.423405+0000","last_undegraded":"2024-05-20T16:11:53.423405+0000","last_fullsized":"2024-05-20T16:11:53.423405+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:38:04.938882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:11:54.266744+0000","last_change":"2024-05-20T16:11:47.183320+0000","last_active":"2024-05-20T16:11:54.266744+0000","last_peered":"2024-05-20T16:11:54.266744+0000","last_clean":"2024-05-20T16:11:54.266744+0000","last_became_active":"2024-05-20T16:11:47.183028+0000","last_became_peered":"2024-05-20T16:11:47.183028+0000","last_unstale":"2024-05-20T16:11:54.266744+0000","last_undegraded":"2024-05-20T16:11:54.266744+0000","last_fullsized":"2024-05-20T16:11:54.266744+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T17:28:33.614173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:11:53.786235+0000","last_change":"2024-05-20T16:11:47.183718+0000","last_active":"2024-05-20T16:11:53.786235+0000","last_peered":"2024-05-20T16:11:53.786235+0000","last_clean":"2024-05-20T16:11:53.786235+0000","last_became_active":"2024-05-20T16:11:47.183378+0000","last_became_peered":"2024-05-20T16:11:47.183378+0000","last_unstale":"2024-05-20T16:11:53.786235+0000","last_undegraded":"2024-05-20T16:11:53.786235+0000","last_fullsized":"2024-05-20T16:11:53.786235+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:08:32.066924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:11:53.786231+0000","last_change":"2024-05-20T16:11:47.174163+0000","last_active":"2024-05-20T16:11:53.786231+0000","last_peered":"2024-05-20T16:11:53.786231+0000","last_clean":"2024-05-20T16:11:53.786231+0000","last_became_active":"2024-05-20T16:11:47.173798+0000","last_became_peered":"2024-05-20T16:11:47.173798+0000","last_unstale":"2024-05-20T16:11:53.786231+0000","last_undegraded":"2024-05-20T16:11:53.786231+0000","last_fullsized":"2024-05-20T16:11:53.786231+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T20:51:20.646273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:11:53.771220+0000","last_change":"2024-05-20T16:11:47.181809+0000","last_active":"2024-05-20T16:11:53.771220+0000","last_peered":"2024-05-20T16:11:53.771220+0000","last_clean":"2024-05-20T16:11:53.771220+0000","last_became_active":"2024-05-20T16:11:47.181473+0000","last_became_peered":"2024-05-20T16:11:47.181473+0000","last_unstale":"2024-05-20T16:11:53.771220+0000","last_undegraded":"2024-05-20T16:11:53.771220+0000","last_fullsized":"2024-05-20T16:11:53.771220+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:20:45.302995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:11:53.846554+0000","last_change":"2024-05-20T16:11:47.187947+0000","last_active":"2024-05-20T16:11:53.846554+0000","last_peered":"2024-05-20T16:11:53.846554+0000","last_clean":"2024-05-20T16:11:53.846554+0000","last_became_active":"2024-05-20T16:11:47.187623+0000","last_became_peered":"2024-05-20T16:11:47.187623+0000","last_unstale":"2024-05-20T16:11:53.846554+0000","last_undegraded":"2024-05-20T16:11:53.846554+0000","last_fullsized":"2024-05-20T16:11:53.846554+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:52:08.674888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:11:53.420181+0000","last_change":"2024-05-20T16:11:47.188599+0000","last_active":"2024-05-20T16:11:53.420181+0000","last_peered":"2024-05-20T16:11:53.420181+0000","last_clean":"2024-05-20T16:11:53.420181+0000","last_became_active":"2024-05-20T16:11:47.188278+0000","last_became_peered":"2024-05-20T16:11:47.188278+0000","last_unstale":"2024-05-20T16:11:53.420181+0000","last_undegraded":"2024-05-20T16:11:53.420181+0000","last_fullsized":"2024-05-20T16:11:53.420181+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T17:45:42.003546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:11:53.420823+0000","last_change":"2024-05-20T16:11:47.176250+0000","last_active":"2024-05-20T16:11:53.420823+0000","last_peered":"2024-05-20T16:11:53.420823+0000","last_clean":"2024-05-20T16:11:53.420823+0000","last_became_active":"2024-05-20T16:11:47.175903+0000","last_became_peered":"2024-05-20T16:11:47.175903+0000","last_unstale":"2024-05-20T16:11:53.420823+0000","last_undegraded":"2024-05-20T16:11:53.420823+0000","last_fullsized":"2024-05-20T16:11:53.420823+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T18:11:26.045745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:11:53.420414+0000","last_change":"2024-05-20T16:11:51.438824+0000","last_active":"2024-05-20T16:11:53.420414+0000","last_peered":"2024-05-20T16:11:53.420414+0000","last_clean":"2024-05-20T16:11:53.420414+0000","last_became_active":"2024-05-20T16:11:51.438279+0000","last_became_peered":"2024-05-20T16:11:51.438279+0000","last_unstale":"2024-05-20T16:11:53.420414+0000","last_undegraded":"2024-05-20T16:11:53.420414+0000","last_fullsized":"2024-05-20T16:11:53.420414+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_clean_scrub_stamp":"2024-05-20T16:11:50.398948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:30:50.680109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"85'4","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:11:53.420004+0000","last_change":"2024-05-20T16:11:46.397871+0000","last_active":"2024-05-20T16:11:53.420004+0000","last_peered":"2024-05-20T16:11:53.420004+0000","last_clean":"2024-05-20T16:11:53.420004+0000","last_became_active":"2024-05-20T16:11:46.397569+0000","last_became_peered":"2024-05-20T16:11:46.397569+0000","last_unstale":"2024-05-20T16:11:53.420004+0000","last_undegraded":"2024-05-20T16:11:53.420004+0000","last_fullsized":"2024-05-20T16:11:53.420004+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_clean_scrub_stamp":"2024-05-20T16:11:45.363714+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:32:24.329823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:11:53.845169+0000","last_change":"2024-05-20T16:11:47.183935+0000","last_active":"2024-05-20T16:11:53.845169+0000","last_peered":"2024-05-20T16:11:53.845169+0000","last_clean":"2024-05-20T16:11:53.845169+0000","last_became_active":"2024-05-20T16:11:47.183604+0000","last_became_peered":"2024-05-20T16:11:47.183604+0000","last_unstale":"2024-05-20T16:11:53.845169+0000","last_undegraded":"2024-05-20T16:11:53.845169+0000","last_fullsized":"2024-05-20T16:11:53.845169+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T16:59:36.924396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:11:53.422665+0000","last_change":"2024-05-20T16:11:51.441603+0000","last_active":"2024-05-20T16:11:53.422665+0000","last_peered":"2024-05-20T16:11:53.422665+0000","last_clean":"2024-05-20T16:11:53.422665+0000","last_became_active":"2024-05-20T16:11:51.441106+0000","last_became_peered":"2024-05-20T16:11:51.441106+0000","last_unstale":"2024-05-20T16:11:53.422665+0000","last_undegraded":"2024-05-20T16:11:53.422665+0000","last_fullsized":"2024-05-20T16:11:53.422665+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_clean_scrub_stamp":"2024-05-20T16:11:50.398948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:07:29.221992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"85'1","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:11:53.845201+0000","last_change":"2024-05-20T16:11:46.401617+0000","last_active":"2024-05-20T16:11:53.845201+0000","last_peered":"2024-05-20T16:11:53.845201+0000","last_clean":"2024-05-20T16:11:53.845201+0000","last_became_active":"2024-05-20T16:11:46.401295+0000","last_became_peered":"2024-05-20T16:11:46.401295+0000","last_unstale":"2024-05-20T16:11:53.845201+0000","last_undegraded":"2024-05-20T16:11:53.845201+0000","last_fullsized":"2024-05-20T16:11:53.845201+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_clean_scrub_stamp":"2024-05-20T16:11:45.363714+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:42:21.039088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:11:53.786434+0000","last_change":"2024-05-20T16:11:47.187157+0000","last_active":"2024-05-20T16:11:53.786434+0000","last_peered":"2024-05-20T16:11:53.786434+0000","last_clean":"2024-05-20T16:11:53.786434+0000","last_became_active":"2024-05-20T16:11:47.186765+0000","last_became_peered":"2024-05-20T16:11:47.186765+0000","last_unstale":"2024-05-20T16:11:53.786434+0000","last_undegraded":"2024-05-20T16:11:53.786434+0000","last_fullsized":"2024-05-20T16:11:53.786434+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T17:09:56.687541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:11:53.423137+0000","last_change":"2024-05-20T16:11:51.442251+0000","last_active":"2024-05-20T16:11:53.423137+0000","last_peered":"2024-05-20T16:11:53.423137+0000","last_clean":"2024-05-20T16:11:53.423137+0000","last_became_active":"2024-05-20T16:11:51.441427+0000","last_became_peered":"2024-05-20T16:11:51.441427+0000","last_unstale":"2024-05-20T16:11:53.423137+0000","last_undegraded":"2024-05-20T16:11:53.423137+0000","last_fullsized":"2024-05-20T16:11:53.423137+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_clean_scrub_stamp":"2024-05-20T16:11:50.398948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:14:08.108310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"91'4","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:12:12.397731+0000","last_change":"2024-05-20T16:11:46.404722+0000","last_active":"2024-05-20T16:12:12.397731+0000","last_peered":"2024-05-20T16:12:12.397731+0000","last_clean":"2024-05-20T16:12:12.397731+0000","last_became_active":"2024-05-20T16:11:46.403626+0000","last_became_peered":"2024-05-20T16:11:46.403626+0000","last_unstale":"2024-05-20T16:12:12.397731+0000","last_undegraded":"2024-05-20T16:12:12.397731+0000","last_fullsized":"2024-05-20T16:12:12.397731+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_clean_scrub_stamp":"2024-05-20T16:11:45.363714+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T18:46:11.283344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:11:53.771585+0000","last_change":"2024-05-20T16:11:47.180014+0000","last_active":"2024-05-20T16:11:53.771585+0000","last_peered":"2024-05-20T16:11:53.771585+0000","last_clean":"2024-05-20T16:11:53.771585+0000","last_became_active":"2024-05-20T16:11:47.179660+0000","last_became_peered":"2024-05-20T16:11:47.179660+0000","last_unstale":"2024-05-20T16:11:53.771585+0000","last_undegraded":"2024-05-20T16:11:53.771585+0000","last_fullsized":"2024-05-20T16:11:53.771585+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:07:26.925346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:11:53.422707+0000","last_change":"2024-05-20T16:11:51.439268+0000","last_active":"2024-05-20T16:11:53.422707+0000","last_peered":"2024-05-20T16:11:53.422707+0000","last_clean":"2024-05-20T16:11:53.422707+0000","last_became_active":"2024-05-20T16:11:51.438767+0000","last_became_peered":"2024-05-20T16:11:51.438767+0000","last_unstale":"2024-05-20T16:11:53.422707+0000","last_undegraded":"2024-05-20T16:11:53.422707+0000","last_fullsized":"2024-05-20T16:11:53.422707+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_clean_scrub_stamp":"2024-05-20T16:11:50.398948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:31:22.394666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:11:53.847517+0000","last_change":"2024-05-20T16:11:46.398922+0000","last_active":"2024-05-20T16:11:53.847517+0000","last_peered":"2024-05-20T16:11:53.847517+0000","last_clean":"2024-05-20T16:11:53.847517+0000","last_became_active":"2024-05-20T16:11:46.398434+0000","last_became_peered":"2024-05-20T16:11:46.398434+0000","last_unstale":"2024-05-20T16:11:53.847517+0000","last_undegraded":"2024-05-20T16:11:53.847517+0000","last_fullsized":"2024-05-20T16:11:53.847517+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_clean_scrub_stamp":"2024-05-20T16:11:45.363714+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T17:38:12.120113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:11:53.847427+0000","last_change":"2024-05-20T16:11:47.179879+0000","last_active":"2024-05-20T16:11:53.847427+0000","last_peered":"2024-05-20T16:11:53.847427+0000","last_clean":"2024-05-20T16:11:53.847427+0000","last_became_active":"2024-05-20T16:11:47.179545+0000","last_became_peered":"2024-05-20T16:11:47.179545+0000","last_unstale":"2024-05-20T16:11:53.847427+0000","last_undegraded":"2024-05-20T16:11:53.847427+0000","last_fullsized":"2024-05-20T16:11:53.847427+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:00:15.966911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:11:53.420660+0000","last_change":"2024-05-20T16:11:51.435981+0000","last_active":"2024-05-20T16:11:53.420660+0000","last_peered":"2024-05-20T16:11:53.420660+0000","last_clean":"2024-05-20T16:11:53.420660+0000","last_became_active":"2024-05-20T16:11:51.435387+0000","last_became_peered":"2024-05-20T16:11:51.435387+0000","last_unstale":"2024-05-20T16:11:53.420660+0000","last_undegraded":"2024-05-20T16:11:53.420660+0000","last_fullsized":"2024-05-20T16:11:53.420660+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_clean_scrub_stamp":"2024-05-20T16:11:50.398948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T16:14:14.877384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"91'5","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:12:11.387883+0000","last_change":"2024-05-20T16:11:46.402453+0000","last_active":"2024-05-20T16:12:11.387883+0000","last_peered":"2024-05-20T16:12:11.387883+0000","last_clean":"2024-05-20T16:12:11.387883+0000","last_became_active":"2024-05-20T16:11:46.400824+0000","last_became_peered":"2024-05-20T16:11:46.400824+0000","last_unstale":"2024-05-20T16:12:11.387883+0000","last_undegraded":"2024-05-20T16:12:11.387883+0000","last_fullsized":"2024-05-20T16:12:11.387883+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_clean_scrub_stamp":"2024-05-20T16:11:45.363714+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T17:39:47.431485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:11:53.846351+0000","last_change":"2024-05-20T16:11:47.180885+0000","last_active":"2024-05-20T16:11:53.846351+0000","last_peered":"2024-05-20T16:11:53.846351+0000","last_clean":"2024-05-20T16:11:53.846351+0000","last_became_active":"2024-05-20T16:11:47.178680+0000","last_became_peered":"2024-05-20T16:11:47.178680+0000","last_unstale":"2024-05-20T16:11:53.846351+0000","last_undegraded":"2024-05-20T16:11:53.846351+0000","last_fullsized":"2024-05-20T16:11:53.846351+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T19:28:00.540561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:11:53.420527+0000","last_change":"2024-05-20T16:11:51.436093+0000","last_active":"2024-05-20T16:11:53.420527+0000","last_peered":"2024-05-20T16:11:53.420527+0000","last_clean":"2024-05-20T16:11:53.420527+0000","last_became_active":"2024-05-20T16:11:51.435448+0000","last_became_peered":"2024-05-20T16:11:51.435448+0000","last_unstale":"2024-05-20T16:11:53.420527+0000","last_undegraded":"2024-05-20T16:11:53.420527+0000","last_fullsized":"2024-05-20T16:11:53.420527+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_clean_scrub_stamp":"2024-05-20T16:11:50.398948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:43:52.365055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"91'3","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:12:12.397695+0000","last_change":"2024-05-20T16:11:46.401695+0000","last_active":"2024-05-20T16:12:12.397695+0000","last_peered":"2024-05-20T16:12:12.397695+0000","last_clean":"2024-05-20T16:12:12.397695+0000","last_became_active":"2024-05-20T16:11:46.401352+0000","last_became_peered":"2024-05-20T16:11:46.401352+0000","last_unstale":"2024-05-20T16:12:12.397695+0000","last_undegraded":"2024-05-20T16:12:12.397695+0000","last_fullsized":"2024-05-20T16:12:12.397695+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_clean_scrub_stamp":"2024-05-20T16:11:45.363714+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:50:58.714842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:11:53.845816+0000","last_change":"2024-05-20T16:11:47.185820+0000","last_active":"2024-05-20T16:11:53.845816+0000","last_peered":"2024-05-20T16:11:53.845816+0000","last_clean":"2024-05-20T16:11:53.845816+0000","last_became_active":"2024-05-20T16:11:47.185491+0000","last_became_peered":"2024-05-20T16:11:47.185491+0000","last_unstale":"2024-05-20T16:11:53.845816+0000","last_undegraded":"2024-05-20T16:11:53.845816+0000","last_fullsized":"2024-05-20T16:11:53.845816+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:23:27.065634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:11:53.846833+0000","last_change":"2024-05-20T16:11:51.432409+0000","last_active":"2024-05-20T16:11:53.846833+0000","last_peered":"2024-05-20T16:11:53.846833+0000","last_clean":"2024-05-20T16:11:53.846833+0000","last_became_active":"2024-05-20T16:11:51.431993+0000","last_became_peered":"2024-05-20T16:11:51.431993+0000","last_unstale":"2024-05-20T16:11:53.846833+0000","last_undegraded":"2024-05-20T16:11:53.846833+0000","last_fullsized":"2024-05-20T16:11:53.846833+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_clean_scrub_stamp":"2024-05-20T16:11:50.398948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:41:08.301120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"91'2","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:12:12.398563+0000","last_change":"2024-05-20T16:11:46.397358+0000","last_active":"2024-05-20T16:12:12.398563+0000","last_peered":"2024-05-20T16:12:12.398563+0000","last_clean":"2024-05-20T16:12:12.398563+0000","last_became_active":"2024-05-20T16:11:46.396989+0000","last_became_peered":"2024-05-20T16:11:46.396989+0000","last_unstale":"2024-05-20T16:12:12.398563+0000","last_undegraded":"2024-05-20T16:12:12.398563+0000","last_fullsized":"2024-05-20T16:12:12.398563+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_clean_scrub_stamp":"2024-05-20T16:11:45.363714+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:31:19.737223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:11:53.844781+0000","last_change":"2024-05-20T16:11:47.175117+0000","last_active":"2024-05-20T16:11:53.844781+0000","last_peered":"2024-05-20T16:11:53.844781+0000","last_clean":"2024-05-20T16:11:53.844781+0000","last_became_active":"2024-05-20T16:11:47.174885+0000","last_became_peered":"2024-05-20T16:11:47.174885+0000","last_unstale":"2024-05-20T16:11:53.844781+0000","last_undegraded":"2024-05-20T16:11:53.844781+0000","last_fullsized":"2024-05-20T16:11:53.844781+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:38:35.765034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:11:53.786630+0000","last_change":"2024-05-20T16:11:51.440650+0000","last_active":"2024-05-20T16:11:53.786630+0000","last_peered":"2024-05-20T16:11:53.786630+0000","last_clean":"2024-05-20T16:11:53.786630+0000","last_became_active":"2024-05-20T16:11:51.440208+0000","last_became_peered":"2024-05-20T16:11:51.440208+0000","last_unstale":"2024-05-20T16:11:53.786630+0000","last_undegraded":"2024-05-20T16:11:53.786630+0000","last_fullsized":"2024-05-20T16:11:53.786630+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_clean_scrub_stamp":"2024-05-20T16:11:50.398948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:02:02.925911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"91'1","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:11:53.845684+0000","last_change":"2024-05-20T16:11:46.402078+0000","last_active":"2024-05-20T16:11:53.845684+0000","last_peered":"2024-05-20T16:11:53.845684+0000","last_clean":"2024-05-20T16:11:53.845684+0000","last_became_active":"2024-05-20T16:11:46.401680+0000","last_became_peered":"2024-05-20T16:11:46.401680+0000","last_unstale":"2024-05-20T16:11:53.845684+0000","last_undegraded":"2024-05-20T16:11:53.845684+0000","last_fullsized":"2024-05-20T16:11:53.845684+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_clean_scrub_stamp":"2024-05-20T16:11:45.363714+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:32:06.118688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:11:53.419730+0000","last_change":"2024-05-20T16:11:47.186933+0000","last_active":"2024-05-20T16:11:53.419730+0000","last_peered":"2024-05-20T16:11:53.419730+0000","last_clean":"2024-05-20T16:11:53.419730+0000","last_became_active":"2024-05-20T16:11:47.186554+0000","last_became_peered":"2024-05-20T16:11:47.186554+0000","last_unstale":"2024-05-20T16:11:53.419730+0000","last_undegraded":"2024-05-20T16:11:53.419730+0000","last_fullsized":"2024-05-20T16:11:53.419730+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T16:24:34.634974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:11:53.844922+0000","last_change":"2024-05-20T16:11:51.442535+0000","last_active":"2024-05-20T16:11:53.844922+0000","last_peered":"2024-05-20T16:11:53.844922+0000","last_clean":"2024-05-20T16:11:53.844922+0000","last_became_active":"2024-05-20T16:11:51.442143+0000","last_became_peered":"2024-05-20T16:11:51.442143+0000","last_unstale":"2024-05-20T16:11:53.844922+0000","last_undegraded":"2024-05-20T16:11:53.844922+0000","last_fullsized":"2024-05-20T16:11:53.844922+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_clean_scrub_stamp":"2024-05-20T16:11:50.398948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T19:31:41.640330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"91'3","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:12:11.386560+0000","last_change":"2024-05-20T16:11:46.394319+0000","last_active":"2024-05-20T16:12:11.386560+0000","last_peered":"2024-05-20T16:12:11.386560+0000","last_clean":"2024-05-20T16:12:11.386560+0000","last_became_active":"2024-05-20T16:11:46.393975+0000","last_became_peered":"2024-05-20T16:11:46.393975+0000","last_unstale":"2024-05-20T16:12:11.386560+0000","last_undegraded":"2024-05-20T16:12:11.386560+0000","last_fullsized":"2024-05-20T16:12:11.386560+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_clean_scrub_stamp":"2024-05-20T16:11:45.363714+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:13:18.706944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:11:53.771899+0000","last_change":"2024-05-20T16:11:47.177382+0000","last_active":"2024-05-20T16:11:53.771899+0000","last_peered":"2024-05-20T16:11:53.771899+0000","last_clean":"2024-05-20T16:11:53.771899+0000","last_became_active":"2024-05-20T16:11:47.176949+0000","last_became_peered":"2024-05-20T16:11:47.176949+0000","last_unstale":"2024-05-20T16:11:53.771899+0000","last_undegraded":"2024-05-20T16:11:53.771899+0000","last_fullsized":"2024-05-20T16:11:53.771899+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:20:36.689937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:11:53.419807+0000","last_change":"2024-05-20T16:11:51.438622+0000","last_active":"2024-05-20T16:11:53.419807+0000","last_peered":"2024-05-20T16:11:53.419807+0000","last_clean":"2024-05-20T16:11:53.419807+0000","last_became_active":"2024-05-20T16:11:51.438191+0000","last_became_peered":"2024-05-20T16:11:51.438191+0000","last_unstale":"2024-05-20T16:11:53.419807+0000","last_undegraded":"2024-05-20T16:11:53.419807+0000","last_fullsized":"2024-05-20T16:11:53.419807+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_clean_scrub_stamp":"2024-05-20T16:11:50.398948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T16:39:56.327423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"91'2","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:12:11.386811+0000","last_change":"2024-05-20T16:11:46.399436+0000","last_active":"2024-05-20T16:12:11.386811+0000","last_peered":"2024-05-20T16:12:11.386811+0000","last_clean":"2024-05-20T16:12:11.386811+0000","last_became_active":"2024-05-20T16:11:46.398591+0000","last_became_peered":"2024-05-20T16:11:46.398591+0000","last_unstale":"2024-05-20T16:12:11.386811+0000","last_undegraded":"2024-05-20T16:12:11.386811+0000","last_fullsized":"2024-05-20T16:12:11.386811+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_clean_scrub_stamp":"2024-05-20T16:11:45.363714+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:18:26.775585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:11:53.786586+0000","last_change":"2024-05-20T16:11:47.190246+0000","last_active":"2024-05-20T16:11:53.786586+0000","last_peered":"2024-05-20T16:11:53.786586+0000","last_clean":"2024-05-20T16:11:53.786586+0000","last_became_active":"2024-05-20T16:11:47.189490+0000","last_became_peered":"2024-05-20T16:11:47.189490+0000","last_unstale":"2024-05-20T16:11:53.786586+0000","last_undegraded":"2024-05-20T16:11:53.786586+0000","last_fullsized":"2024-05-20T16:11:53.786586+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:31:05.934097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:11:53.423061+0000","last_change":"2024-05-20T16:11:51.442169+0000","last_active":"2024-05-20T16:11:53.423061+0000","last_peered":"2024-05-20T16:11:53.423061+0000","last_clean":"2024-05-20T16:11:53.423061+0000","last_became_active":"2024-05-20T16:11:51.441114+0000","last_became_peered":"2024-05-20T16:11:51.441114+0000","last_unstale":"2024-05-20T16:11:53.423061+0000","last_undegraded":"2024-05-20T16:11:53.423061+0000","last_fullsized":"2024-05-20T16:11:53.423061+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_clean_scrub_stamp":"2024-05-20T16:11:50.398948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:01:50.833632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"91'4","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:11:53.846924+0000","last_change":"2024-05-20T16:11:46.403424+0000","last_active":"2024-05-20T16:11:53.846924+0000","last_peered":"2024-05-20T16:11:53.846924+0000","last_clean":"2024-05-20T16:11:53.846924+0000","last_became_active":"2024-05-20T16:11:46.400786+0000","last_became_peered":"2024-05-20T16:11:46.400786+0000","last_unstale":"2024-05-20T16:11:53.846924+0000","last_undegraded":"2024-05-20T16:11:53.846924+0000","last_fullsized":"2024-05-20T16:11:53.846924+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_clean_scrub_stamp":"2024-05-20T16:11:45.363714+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T16:29:33.300267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:11:53.771758+0000","last_change":"2024-05-20T16:11:47.176372+0000","last_active":"2024-05-20T16:11:53.771758+0000","last_peered":"2024-05-20T16:11:53.771758+0000","last_clean":"2024-05-20T16:11:53.771758+0000","last_became_active":"2024-05-20T16:11:47.175535+0000","last_became_peered":"2024-05-20T16:11:47.175535+0000","last_unstale":"2024-05-20T16:11:53.771758+0000","last_undegraded":"2024-05-20T16:11:53.771758+0000","last_fullsized":"2024-05-20T16:11:53.771758+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:40:22.293771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:11:53.419875+0000","last_change":"2024-05-20T16:11:51.442678+0000","last_active":"2024-05-20T16:11:53.419875+0000","last_peered":"2024-05-20T16:11:53.419875+0000","last_clean":"2024-05-20T16:11:53.419875+0000","last_became_active":"2024-05-20T16:11:51.442300+0000","last_became_peered":"2024-05-20T16:11:51.442300+0000","last_unstale":"2024-05-20T16:11:53.419875+0000","last_undegraded":"2024-05-20T16:11:53.419875+0000","last_fullsized":"2024-05-20T16:11:53.419875+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_clean_scrub_stamp":"2024-05-20T16:11:50.398948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:29:14.668640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"91'12","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:12:11.386666+0000","last_change":"2024-05-20T16:11:46.398550+0000","last_active":"2024-05-20T16:12:11.386666+0000","last_peered":"2024-05-20T16:12:11.386666+0000","last_clean":"2024-05-20T16:12:11.386666+0000","last_became_active":"2024-05-20T16:11:46.398216+0000","last_became_peered":"2024-05-20T16:11:46.398216+0000","last_unstale":"2024-05-20T16:12:11.386666+0000","last_undegraded":"2024-05-20T16:12:11.386666+0000","last_fullsized":"2024-05-20T16:12:11.386666+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_clean_scrub_stamp":"2024-05-20T16:11:45.363714+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:24:53.292837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:11:53.847039+0000","last_change":"2024-05-20T16:11:47.177641+0000","last_active":"2024-05-20T16:11:53.847039+0000","last_peered":"2024-05-20T16:11:53.847039+0000","last_clean":"2024-05-20T16:11:53.847039+0000","last_became_active":"2024-05-20T16:11:47.177333+0000","last_became_peered":"2024-05-20T16:11:47.177333+0000","last_unstale":"2024-05-20T16:11:53.847039+0000","last_undegraded":"2024-05-20T16:11:53.847039+0000","last_fullsized":"2024-05-20T16:11:53.847039+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T17:42:01.674485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:11:53.847473+0000","last_change":"2024-05-20T16:11:51.445804+0000","last_active":"2024-05-20T16:11:53.847473+0000","last_peered":"2024-05-20T16:11:53.847473+0000","last_clean":"2024-05-20T16:11:53.847473+0000","last_became_active":"2024-05-20T16:11:51.445223+0000","last_became_peered":"2024-05-20T16:11:51.445223+0000","last_unstale":"2024-05-20T16:11:53.847473+0000","last_undegraded":"2024-05-20T16:11:53.847473+0000","last_fullsized":"2024-05-20T16:11:53.847473+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_clean_scrub_stamp":"2024-05-20T16:11:50.398948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:45:19.185992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"91'6","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:12:12.398116+0000","last_change":"2024-05-20T16:11:46.398349+0000","last_active":"2024-05-20T16:12:12.398116+0000","last_peered":"2024-05-20T16:12:12.398116+0000","last_clean":"2024-05-20T16:12:12.398116+0000","last_became_active":"2024-05-20T16:11:46.397878+0000","last_became_peered":"2024-05-20T16:11:46.397878+0000","last_unstale":"2024-05-20T16:12:12.398116+0000","last_undegraded":"2024-05-20T16:12:12.398116+0000","last_fullsized":"2024-05-20T16:12:12.398116+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_clean_scrub_stamp":"2024-05-20T16:11:45.363714+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:39:07.374098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:11:54.266598+0000","last_change":"2024-05-20T16:11:47.179322+0000","last_active":"2024-05-20T16:11:54.266598+0000","last_peered":"2024-05-20T16:11:54.266598+0000","last_clean":"2024-05-20T16:11:54.266598+0000","last_became_active":"2024-05-20T16:11:47.178897+0000","last_became_peered":"2024-05-20T16:11:47.178897+0000","last_unstale":"2024-05-20T16:11:54.266598+0000","last_undegraded":"2024-05-20T16:11:54.266598+0000","last_fullsized":"2024-05-20T16:11:54.266598+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T20:01:04.096952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:11:53.846308+0000","last_change":"2024-05-20T16:11:51.439577+0000","last_active":"2024-05-20T16:11:53.846308+0000","last_peered":"2024-05-20T16:11:53.846308+0000","last_clean":"2024-05-20T16:11:53.846308+0000","last_became_active":"2024-05-20T16:11:51.439043+0000","last_became_peered":"2024-05-20T16:11:51.439043+0000","last_unstale":"2024-05-20T16:11:53.846308+0000","last_undegraded":"2024-05-20T16:11:53.846308+0000","last_fullsized":"2024-05-20T16:11:53.846308+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_clean_scrub_stamp":"2024-05-20T16:11:50.398948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:23:18.861715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"91'3","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:12:11.387273+0000","last_change":"2024-05-20T16:11:46.402276+0000","last_active":"2024-05-20T16:12:11.387273+0000","last_peered":"2024-05-20T16:12:11.387273+0000","last_clean":"2024-05-20T16:12:11.387273+0000","last_became_active":"2024-05-20T16:11:46.401923+0000","last_became_peered":"2024-05-20T16:11:46.401923+0000","last_unstale":"2024-05-20T16:12:11.387273+0000","last_undegraded":"2024-05-20T16:12:11.387273+0000","last_fullsized":"2024-05-20T16:12:11.387273+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_clean_scrub_stamp":"2024-05-20T16:11:45.363714+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T17:57:00.320492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:11:53.589485+0000","last_change":"2024-05-20T16:11:47.174448+0000","last_active":"2024-05-20T16:11:53.589485+0000","last_peered":"2024-05-20T16:11:53.589485+0000","last_clean":"2024-05-20T16:11:53.589485+0000","last_became_active":"2024-05-20T16:11:47.174099+0000","last_became_peered":"2024-05-20T16:11:47.174099+0000","last_unstale":"2024-05-20T16:11:53.589485+0000","last_undegraded":"2024-05-20T16:11:53.589485+0000","last_fullsized":"2024-05-20T16:11:53.589485+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T20:53:30.392099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:11:53.419693+0000","last_change":"2024-05-20T16:11:51.446231+0000","last_active":"2024-05-20T16:11:53.419693+0000","last_peered":"2024-05-20T16:11:53.419693+0000","last_clean":"2024-05-20T16:11:53.419693+0000","last_became_active":"2024-05-20T16:11:51.445805+0000","last_became_peered":"2024-05-20T16:11:51.445805+0000","last_unstale":"2024-05-20T16:11:53.419693+0000","last_undegraded":"2024-05-20T16:11:53.419693+0000","last_fullsized":"2024-05-20T16:11:53.419693+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_clean_scrub_stamp":"2024-05-20T16:11:50.398948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:51:29.505139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"91'3","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:12:12.397003+0000","last_change":"2024-05-20T16:11:46.392677+0000","last_active":"2024-05-20T16:12:12.397003+0000","last_peered":"2024-05-20T16:12:12.397003+0000","last_clean":"2024-05-20T16:12:12.397003+0000","last_became_active":"2024-05-20T16:11:46.392315+0000","last_became_peered":"2024-05-20T16:11:46.392315+0000","last_unstale":"2024-05-20T16:12:12.397003+0000","last_undegraded":"2024-05-20T16:12:12.397003+0000","last_fullsized":"2024-05-20T16:12:12.397003+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_clean_scrub_stamp":"2024-05-20T16:11:45.363714+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T19:04:14.191754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:11:53.589114+0000","last_change":"2024-05-20T16:11:47.187104+0000","last_active":"2024-05-20T16:11:53.589114+0000","last_peered":"2024-05-20T16:11:53.589114+0000","last_clean":"2024-05-20T16:11:53.589114+0000","last_became_active":"2024-05-20T16:11:47.186375+0000","last_became_peered":"2024-05-20T16:11:47.186375+0000","last_unstale":"2024-05-20T16:11:53.589114+0000","last_undegraded":"2024-05-20T16:11:53.589114+0000","last_fullsized":"2024-05-20T16:11:53.589114+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:27:32.478899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:11:54.266942+0000","last_change":"2024-05-20T16:11:51.444872+0000","last_active":"2024-05-20T16:11:54.266942+0000","last_peered":"2024-05-20T16:11:54.266942+0000","last_clean":"2024-05-20T16:11:54.266942+0000","last_became_active":"2024-05-20T16:11:51.444472+0000","last_became_peered":"2024-05-20T16:11:51.444472+0000","last_unstale":"2024-05-20T16:11:54.266942+0000","last_undegraded":"2024-05-20T16:11:54.266942+0000","last_fullsized":"2024-05-20T16:11:54.266942+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_clean_scrub_stamp":"2024-05-20T16:11:50.398948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:37:57.117101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"91'15","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:12:12.398665+0000","last_change":"2024-05-20T16:11:46.407244+0000","last_active":"2024-05-20T16:12:12.398665+0000","last_peered":"2024-05-20T16:12:12.398665+0000","last_clean":"2024-05-20T16:12:12.398665+0000","last_became_active":"2024-05-20T16:11:46.406905+0000","last_became_peered":"2024-05-20T16:11:46.406905+0000","last_unstale":"2024-05-20T16:12:12.398665+0000","last_undegraded":"2024-05-20T16:12:12.398665+0000","last_fullsized":"2024-05-20T16:12:12.398665+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_clean_scrub_stamp":"2024-05-20T16:11:45.363714+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:45:40.497442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:11:53.845955+0000","last_change":"2024-05-20T16:11:47.180326+0000","last_active":"2024-05-20T16:11:53.845955+0000","last_peered":"2024-05-20T16:11:53.845955+0000","last_clean":"2024-05-20T16:11:53.845955+0000","last_became_active":"2024-05-20T16:11:47.179937+0000","last_became_peered":"2024-05-20T16:11:47.179937+0000","last_unstale":"2024-05-20T16:11:53.845955+0000","last_undegraded":"2024-05-20T16:11:53.845955+0000","last_fullsized":"2024-05-20T16:11:53.845955+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:59:59.431365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:11:53.845779+0000","last_change":"2024-05-20T16:11:51.444495+0000","last_active":"2024-05-20T16:11:53.845779+0000","last_peered":"2024-05-20T16:11:53.845779+0000","last_clean":"2024-05-20T16:11:53.845779+0000","last_became_active":"2024-05-20T16:11:51.444136+0000","last_became_peered":"2024-05-20T16:11:51.444136+0000","last_unstale":"2024-05-20T16:11:53.845779+0000","last_undegraded":"2024-05-20T16:11:53.845779+0000","last_fullsized":"2024-05-20T16:11:53.845779+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_clean_scrub_stamp":"2024-05-20T16:11:50.398948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:27:46.260340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"85'1","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:11:53.847676+0000","last_change":"2024-05-20T16:11:46.407252+0000","last_active":"2024-05-20T16:11:53.847676+0000","last_peered":"2024-05-20T16:11:53.847676+0000","last_clean":"2024-05-20T16:11:53.847676+0000","last_became_active":"2024-05-20T16:11:46.406867+0000","last_became_peered":"2024-05-20T16:11:46.406867+0000","last_unstale":"2024-05-20T16:11:53.847676+0000","last_undegraded":"2024-05-20T16:11:53.847676+0000","last_fullsized":"2024-05-20T16:11:53.847676+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_clean_scrub_stamp":"2024-05-20T16:11:45.363714+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T17:53:46.638460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:11:53.846490+0000","last_change":"2024-05-20T16:11:47.179046+0000","last_active":"2024-05-20T16:11:53.846490+0000","last_peered":"2024-05-20T16:11:53.846490+0000","last_clean":"2024-05-20T16:11:53.846490+0000","last_became_active":"2024-05-20T16:11:47.178722+0000","last_became_peered":"2024-05-20T16:11:47.178722+0000","last_unstale":"2024-05-20T16:11:53.846490+0000","last_undegraded":"2024-05-20T16:11:53.846490+0000","last_fullsized":"2024-05-20T16:11:53.846490+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T17:38:15.355878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:11:53.846213+0000","last_change":"2024-05-20T16:11:51.915537+0000","last_active":"2024-05-20T16:11:53.846213+0000","last_peered":"2024-05-20T16:11:53.846213+0000","last_clean":"2024-05-20T16:11:53.846213+0000","last_became_active":"2024-05-20T16:11:51.914973+0000","last_became_peered":"2024-05-20T16:11:51.914973+0000","last_unstale":"2024-05-20T16:11:53.846213+0000","last_undegraded":"2024-05-20T16:11:53.846213+0000","last_fullsized":"2024-05-20T16:11:53.846213+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_clean_scrub_stamp":"2024-05-20T16:11:50.398948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:53:19.724086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:11:53.419724+0000","last_change":"2024-05-20T16:11:46.402585+0000","last_active":"2024-05-20T16:11:53.419724+0000","last_peered":"2024-05-20T16:11:53.419724+0000","last_clean":"2024-05-20T16:11:53.419724+0000","last_became_active":"2024-05-20T16:11:46.402163+0000","last_became_peered":"2024-05-20T16:11:46.402163+0000","last_unstale":"2024-05-20T16:11:53.419724+0000","last_undegraded":"2024-05-20T16:11:53.419724+0000","last_fullsized":"2024-05-20T16:11:53.419724+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_clean_scrub_stamp":"2024-05-20T16:11:45.363714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:35:26.036970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:11:53.845878+0000","last_change":"2024-05-20T16:11:47.183088+0000","last_active":"2024-05-20T16:11:53.845878+0000","last_peered":"2024-05-20T16:11:53.845878+0000","last_clean":"2024-05-20T16:11:53.845878+0000","last_became_active":"2024-05-20T16:11:47.182608+0000","last_became_peered":"2024-05-20T16:11:47.182608+0000","last_unstale":"2024-05-20T16:11:53.845878+0000","last_undegraded":"2024-05-20T16:11:53.845878+0000","last_fullsized":"2024-05-20T16:11:53.845878+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T20:23:37.086778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:11:54.266770+0000","last_change":"2024-05-20T16:11:51.437420+0000","last_active":"2024-05-20T16:11:54.266770+0000","last_peered":"2024-05-20T16:11:54.266770+0000","last_clean":"2024-05-20T16:11:54.266770+0000","last_became_active":"2024-05-20T16:11:51.437018+0000","last_became_peered":"2024-05-20T16:11:51.437018+0000","last_unstale":"2024-05-20T16:11:54.266770+0000","last_undegraded":"2024-05-20T16:11:54.266770+0000","last_fullsized":"2024-05-20T16:11:54.266770+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_clean_scrub_stamp":"2024-05-20T16:11:50.398948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:03:30.650894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"91'1","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:11:53.589272+0000","last_change":"2024-05-20T16:11:46.400818+0000","last_active":"2024-05-20T16:11:53.589272+0000","last_peered":"2024-05-20T16:11:53.589272+0000","last_clean":"2024-05-20T16:11:53.589272+0000","last_became_active":"2024-05-20T16:11:46.400482+0000","last_became_peered":"2024-05-20T16:11:46.400482+0000","last_unstale":"2024-05-20T16:11:53.589272+0000","last_undegraded":"2024-05-20T16:11:53.589272+0000","last_fullsized":"2024-05-20T16:11:53.589272+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_clean_scrub_stamp":"2024-05-20T16:11:45.363714+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:13:13.968284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:11:53.845827+0000","last_change":"2024-05-20T16:11:47.187542+0000","last_active":"2024-05-20T16:11:53.845827+0000","last_peered":"2024-05-20T16:11:53.845827+0000","last_clean":"2024-05-20T16:11:53.845827+0000","last_became_active":"2024-05-20T16:11:47.187217+0000","last_became_peered":"2024-05-20T16:11:47.187217+0000","last_unstale":"2024-05-20T16:11:53.845827+0000","last_undegraded":"2024-05-20T16:11:53.845827+0000","last_fullsized":"2024-05-20T16:11:53.845827+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T19:53:54.608555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:11:54.266991+0000","last_change":"2024-05-20T16:11:51.915176+0000","last_active":"2024-05-20T16:11:54.266991+0000","last_peered":"2024-05-20T16:11:54.266991+0000","last_clean":"2024-05-20T16:11:54.266991+0000","last_became_active":"2024-05-20T16:11:51.914766+0000","last_became_peered":"2024-05-20T16:11:51.914766+0000","last_unstale":"2024-05-20T16:11:54.266991+0000","last_undegraded":"2024-05-20T16:11:54.266991+0000","last_fullsized":"2024-05-20T16:11:54.266991+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_clean_scrub_stamp":"2024-05-20T16:11:50.398948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T16:26:02.276646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:11:53.846197+0000","last_change":"2024-05-20T16:11:47.184502+0000","last_active":"2024-05-20T16:11:53.846197+0000","last_peered":"2024-05-20T16:11:53.846197+0000","last_clean":"2024-05-20T16:11:53.846197+0000","last_became_active":"2024-05-20T16:11:47.184123+0000","last_became_peered":"2024-05-20T16:11:47.184123+0000","last_unstale":"2024-05-20T16:11:53.846197+0000","last_undegraded":"2024-05-20T16:11:53.846197+0000","last_fullsized":"2024-05-20T16:11:53.846197+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:55:23.989206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"91'4","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:12:12.398907+0000","last_change":"2024-05-20T16:11:46.399731+0000","last_active":"2024-05-20T16:12:12.398907+0000","last_peered":"2024-05-20T16:12:12.398907+0000","last_clean":"2024-05-20T16:12:12.398907+0000","last_became_active":"2024-05-20T16:11:46.399349+0000","last_became_peered":"2024-05-20T16:11:46.399349+0000","last_unstale":"2024-05-20T16:12:12.398907+0000","last_undegraded":"2024-05-20T16:12:12.398907+0000","last_fullsized":"2024-05-20T16:12:12.398907+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_clean_scrub_stamp":"2024-05-20T16:11:45.363714+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T20:27:39.038116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"91'1","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:11:54.496742+0000","last_change":"2024-05-20T16:11:51.435547+0000","last_active":"2024-05-20T16:11:54.496742+0000","last_peered":"2024-05-20T16:11:54.496742+0000","last_clean":"2024-05-20T16:11:54.496742+0000","last_became_active":"2024-05-20T16:11:51.435224+0000","last_became_peered":"2024-05-20T16:11:51.435224+0000","last_unstale":"2024-05-20T16:11:54.496742+0000","last_undegraded":"2024-05-20T16:11:54.496742+0000","last_fullsized":"2024-05-20T16:11:54.496742+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_clean_scrub_stamp":"2024-05-20T16:11:50.398948+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:09:02.989494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:11:53.844875+0000","last_change":"2024-05-20T16:11:47.185746+0000","last_active":"2024-05-20T16:11:53.844875+0000","last_peered":"2024-05-20T16:11:53.844875+0000","last_clean":"2024-05-20T16:11:53.844875+0000","last_became_active":"2024-05-20T16:11:47.185399+0000","last_became_peered":"2024-05-20T16:11:47.185399+0000","last_unstale":"2024-05-20T16:11:53.844875+0000","last_undegraded":"2024-05-20T16:11:53.844875+0000","last_fullsized":"2024-05-20T16:11:53.844875+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:05:20.241804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"91'3","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:12:11.385524+0000","last_change":"2024-05-20T16:11:46.393867+0000","last_active":"2024-05-20T16:12:11.385524+0000","last_peered":"2024-05-20T16:12:11.385524+0000","last_clean":"2024-05-20T16:12:11.385524+0000","last_became_active":"2024-05-20T16:11:46.393523+0000","last_became_peered":"2024-05-20T16:11:46.393523+0000","last_unstale":"2024-05-20T16:12:11.385524+0000","last_undegraded":"2024-05-20T16:12:11.385524+0000","last_fullsized":"2024-05-20T16:12:11.385524+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_clean_scrub_stamp":"2024-05-20T16:11:45.363714+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T18:36:40.067975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":348,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:11:53.588927+0000","last_change":"2024-05-20T16:10:35.566958+0000","last_active":"2024-05-20T16:11:53.588927+0000","last_peered":"2024-05-20T16:11:53.588927+0000","last_clean":"2024-05-20T16:11:53.588927+0000","last_became_active":"2024-05-20T16:10:35.566551+0000","last_became_peered":"2024-05-20T16:10:35.566551+0000","last_unstale":"2024-05-20T16:11:53.588927+0000","last_undegraded":"2024-05-20T16:11:53.588927+0000","last_fullsized":"2024-05-20T16:11:53.588927+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:07:09.543878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:07:09.543878+0000","last_clean_scrub_stamp":"2024-05-20T16:07:09.543878+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:48:26.484769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:11:53.847419+0000","last_change":"2024-05-20T16:11:51.915534+0000","last_active":"2024-05-20T16:11:53.847419+0000","last_peered":"2024-05-20T16:11:53.847419+0000","last_clean":"2024-05-20T16:11:53.847419+0000","last_became_active":"2024-05-20T16:11:51.914903+0000","last_became_peered":"2024-05-20T16:11:51.914903+0000","last_unstale":"2024-05-20T16:11:53.847419+0000","last_undegraded":"2024-05-20T16:11:53.847419+0000","last_fullsized":"2024-05-20T16:11:53.847419+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_clean_scrub_stamp":"2024-05-20T16:11:50.398948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T18:56:47.255129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:11:54.266326+0000","last_change":"2024-05-20T16:11:47.174101+0000","last_active":"2024-05-20T16:11:54.266326+0000","last_peered":"2024-05-20T16:11:54.266326+0000","last_clean":"2024-05-20T16:11:54.266326+0000","last_became_active":"2024-05-20T16:11:47.173772+0000","last_became_peered":"2024-05-20T16:11:47.173772+0000","last_unstale":"2024-05-20T16:11:54.266326+0000","last_undegraded":"2024-05-20T16:11:54.266326+0000","last_fullsized":"2024-05-20T16:11:54.266326+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:33:33.656757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"91'3","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:12:12.398445+0000","last_change":"2024-05-20T16:11:46.401951+0000","last_active":"2024-05-20T16:12:12.398445+0000","last_peered":"2024-05-20T16:12:12.398445+0000","last_clean":"2024-05-20T16:12:12.398445+0000","last_became_active":"2024-05-20T16:11:46.401377+0000","last_became_peered":"2024-05-20T16:11:46.401377+0000","last_unstale":"2024-05-20T16:12:12.398445+0000","last_undegraded":"2024-05-20T16:12:12.398445+0000","last_fullsized":"2024-05-20T16:12:12.398445+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_clean_scrub_stamp":"2024-05-20T16:11:45.363714+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:00:43.538206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:11:53.846308+0000","last_change":"2024-05-20T16:11:51.436284+0000","last_active":"2024-05-20T16:11:53.846308+0000","last_peered":"2024-05-20T16:11:53.846308+0000","last_clean":"2024-05-20T16:11:53.846308+0000","last_became_active":"2024-05-20T16:11:51.435755+0000","last_became_peered":"2024-05-20T16:11:51.435755+0000","last_unstale":"2024-05-20T16:11:53.846308+0000","last_undegraded":"2024-05-20T16:11:53.846308+0000","last_fullsized":"2024-05-20T16:11:53.846308+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_clean_scrub_stamp":"2024-05-20T16:11:50.398948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T20:51:05.434690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"91'1","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:12:11.384345+0000","last_change":"2024-05-20T16:11:46.408055+0000","last_active":"2024-05-20T16:12:11.384345+0000","last_peered":"2024-05-20T16:12:11.384345+0000","last_clean":"2024-05-20T16:12:11.384345+0000","last_became_active":"2024-05-20T16:11:46.407733+0000","last_became_peered":"2024-05-20T16:11:46.407733+0000","last_unstale":"2024-05-20T16:12:11.384345+0000","last_undegraded":"2024-05-20T16:12:11.384345+0000","last_fullsized":"2024-05-20T16:12:11.384345+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_clean_scrub_stamp":"2024-05-20T16:11:45.363714+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T18:32:32.344334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:11:54.266857+0000","last_change":"2024-05-20T16:11:47.179831+0000","last_active":"2024-05-20T16:11:54.266857+0000","last_peered":"2024-05-20T16:11:54.266857+0000","last_clean":"2024-05-20T16:11:54.266857+0000","last_became_active":"2024-05-20T16:11:47.179501+0000","last_became_peered":"2024-05-20T16:11:47.179501+0000","last_unstale":"2024-05-20T16:11:54.266857+0000","last_undegraded":"2024-05-20T16:11:54.266857+0000","last_fullsized":"2024-05-20T16:11:54.266857+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T17:38:13.651180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:11:53.844736+0000","last_change":"2024-05-20T16:11:51.915210+0000","last_active":"2024-05-20T16:11:53.844736+0000","last_peered":"2024-05-20T16:11:53.844736+0000","last_clean":"2024-05-20T16:11:53.844736+0000","last_became_active":"2024-05-20T16:11:51.914751+0000","last_became_peered":"2024-05-20T16:11:51.914751+0000","last_unstale":"2024-05-20T16:11:53.844736+0000","last_undegraded":"2024-05-20T16:11:53.844736+0000","last_fullsized":"2024-05-20T16:11:53.844736+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_clean_scrub_stamp":"2024-05-20T16:11:50.398948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:07:14.817497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"86'1","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:11:53.420161+0000","last_change":"2024-05-20T16:11:46.403148+0000","last_active":"2024-05-20T16:11:53.420161+0000","last_peered":"2024-05-20T16:11:53.420161+0000","last_clean":"2024-05-20T16:11:53.420161+0000","last_became_active":"2024-05-20T16:11:46.402818+0000","last_became_peered":"2024-05-20T16:11:46.402818+0000","last_unstale":"2024-05-20T16:11:53.420161+0000","last_undegraded":"2024-05-20T16:11:53.420161+0000","last_fullsized":"2024-05-20T16:11:53.420161+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_clean_scrub_stamp":"2024-05-20T16:11:45.363714+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:49:52.095893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:11:53.845169+0000","last_change":"2024-05-20T16:11:47.184550+0000","last_active":"2024-05-20T16:11:53.845169+0000","last_peered":"2024-05-20T16:11:53.845169+0000","last_clean":"2024-05-20T16:11:53.845169+0000","last_became_active":"2024-05-20T16:11:47.184260+0000","last_became_peered":"2024-05-20T16:11:47.184260+0000","last_unstale":"2024-05-20T16:11:53.845169+0000","last_undegraded":"2024-05-20T16:11:53.845169+0000","last_fullsized":"2024-05-20T16:11:53.845169+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T17:21:43.013563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:11:53.420078+0000","last_change":"2024-05-20T16:11:51.431710+0000","last_active":"2024-05-20T16:11:53.420078+0000","last_peered":"2024-05-20T16:11:53.420078+0000","last_clean":"2024-05-20T16:11:53.420078+0000","last_became_active":"2024-05-20T16:11:51.431286+0000","last_became_peered":"2024-05-20T16:11:51.431286+0000","last_unstale":"2024-05-20T16:11:53.420078+0000","last_undegraded":"2024-05-20T16:11:53.420078+0000","last_fullsized":"2024-05-20T16:11:53.420078+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_clean_scrub_stamp":"2024-05-20T16:11:50.398948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:35:19.016412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"91'4","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:12:11.384319+0000","last_change":"2024-05-20T16:11:46.401772+0000","last_active":"2024-05-20T16:12:11.384319+0000","last_peered":"2024-05-20T16:12:11.384319+0000","last_clean":"2024-05-20T16:12:11.384319+0000","last_became_active":"2024-05-20T16:11:46.401414+0000","last_became_peered":"2024-05-20T16:11:46.401414+0000","last_unstale":"2024-05-20T16:12:11.384319+0000","last_undegraded":"2024-05-20T16:12:11.384319+0000","last_fullsized":"2024-05-20T16:12:11.384319+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_clean_scrub_stamp":"2024-05-20T16:11:45.363714+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:23:59.146573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:11:53.786667+0000","last_change":"2024-05-20T16:11:47.177333+0000","last_active":"2024-05-20T16:11:53.786667+0000","last_peered":"2024-05-20T16:11:53.786667+0000","last_clean":"2024-05-20T16:11:53.786667+0000","last_became_active":"2024-05-20T16:11:47.176161+0000","last_became_peered":"2024-05-20T16:11:47.176161+0000","last_unstale":"2024-05-20T16:11:53.786667+0000","last_undegraded":"2024-05-20T16:11:53.786667+0000","last_fullsized":"2024-05-20T16:11:53.786667+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:48:40.846477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:11:53.423145+0000","last_change":"2024-05-20T16:11:51.915348+0000","last_active":"2024-05-20T16:11:53.423145+0000","last_peered":"2024-05-20T16:11:53.423145+0000","last_clean":"2024-05-20T16:11:53.423145+0000","last_became_active":"2024-05-20T16:11:51.914864+0000","last_became_peered":"2024-05-20T16:11:51.914864+0000","last_unstale":"2024-05-20T16:11:53.423145+0000","last_undegraded":"2024-05-20T16:11:53.423145+0000","last_fullsized":"2024-05-20T16:11:53.423145+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_clean_scrub_stamp":"2024-05-20T16:11:50.398948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:02:26.846136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"91'2","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:12:11.387259+0000","last_change":"2024-05-20T16:11:46.407127+0000","last_active":"2024-05-20T16:12:11.387259+0000","last_peered":"2024-05-20T16:12:11.387259+0000","last_clean":"2024-05-20T16:12:11.387259+0000","last_became_active":"2024-05-20T16:11:46.406833+0000","last_became_peered":"2024-05-20T16:11:46.406833+0000","last_unstale":"2024-05-20T16:12:11.387259+0000","last_undegraded":"2024-05-20T16:12:11.387259+0000","last_fullsized":"2024-05-20T16:12:11.387259+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_clean_scrub_stamp":"2024-05-20T16:11:45.363714+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:58:30.464017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:11:53.845653+0000","last_change":"2024-05-20T16:11:47.174434+0000","last_active":"2024-05-20T16:11:53.845653+0000","last_peered":"2024-05-20T16:11:53.845653+0000","last_clean":"2024-05-20T16:11:53.845653+0000","last_became_active":"2024-05-20T16:11:47.174019+0000","last_became_peered":"2024-05-20T16:11:47.174019+0000","last_unstale":"2024-05-20T16:11:53.845653+0000","last_undegraded":"2024-05-20T16:11:53.845653+0000","last_fullsized":"2024-05-20T16:11:53.845653+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T18:32:51.150925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:11:53.420024+0000","last_change":"2024-05-20T16:11:51.446241+0000","last_active":"2024-05-20T16:11:53.420024+0000","last_peered":"2024-05-20T16:11:53.420024+0000","last_clean":"2024-05-20T16:11:53.420024+0000","last_became_active":"2024-05-20T16:11:51.445828+0000","last_became_peered":"2024-05-20T16:11:51.445828+0000","last_unstale":"2024-05-20T16:11:53.420024+0000","last_undegraded":"2024-05-20T16:11:53.420024+0000","last_fullsized":"2024-05-20T16:11:53.420024+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_clean_scrub_stamp":"2024-05-20T16:11:50.398948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:26:00.239338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"91'3","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:12:11.386469+0000","last_change":"2024-05-20T16:11:46.396364+0000","last_active":"2024-05-20T16:12:11.386469+0000","last_peered":"2024-05-20T16:12:11.386469+0000","last_clean":"2024-05-20T16:12:11.386469+0000","last_became_active":"2024-05-20T16:11:46.396014+0000","last_became_peered":"2024-05-20T16:11:46.396014+0000","last_unstale":"2024-05-20T16:12:11.386469+0000","last_undegraded":"2024-05-20T16:12:11.386469+0000","last_fullsized":"2024-05-20T16:12:11.386469+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_clean_scrub_stamp":"2024-05-20T16:11:45.363714+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:02:46.660077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:11:53.846016+0000","last_change":"2024-05-20T16:11:47.184320+0000","last_active":"2024-05-20T16:11:53.846016+0000","last_peered":"2024-05-20T16:11:53.846016+0000","last_clean":"2024-05-20T16:11:53.846016+0000","last_became_active":"2024-05-20T16:11:47.183981+0000","last_became_peered":"2024-05-20T16:11:47.183981+0000","last_unstale":"2024-05-20T16:11:53.846016+0000","last_undegraded":"2024-05-20T16:11:53.846016+0000","last_fullsized":"2024-05-20T16:11:53.846016+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T17:41:47.160563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:11:53.422872+0000","last_change":"2024-05-20T16:11:51.440073+0000","last_active":"2024-05-20T16:11:53.422872+0000","last_peered":"2024-05-20T16:11:53.422872+0000","last_clean":"2024-05-20T16:11:53.422872+0000","last_became_active":"2024-05-20T16:11:51.439560+0000","last_became_peered":"2024-05-20T16:11:51.439560+0000","last_unstale":"2024-05-20T16:11:53.422872+0000","last_undegraded":"2024-05-20T16:11:53.422872+0000","last_fullsized":"2024-05-20T16:11:53.422872+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_clean_scrub_stamp":"2024-05-20T16:11:50.398948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:50:54.429811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"91'1","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:11:53.420164+0000","last_change":"2024-05-20T16:11:46.404866+0000","last_active":"2024-05-20T16:11:53.420164+0000","last_peered":"2024-05-20T16:11:53.420164+0000","last_clean":"2024-05-20T16:11:53.420164+0000","last_became_active":"2024-05-20T16:11:46.404509+0000","last_became_peered":"2024-05-20T16:11:46.404509+0000","last_unstale":"2024-05-20T16:11:53.420164+0000","last_undegraded":"2024-05-20T16:11:53.420164+0000","last_fullsized":"2024-05-20T16:11:53.420164+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_clean_scrub_stamp":"2024-05-20T16:11:45.363714+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:58:40.303826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:11:53.844942+0000","last_change":"2024-05-20T16:11:47.191505+0000","last_active":"2024-05-20T16:11:53.844942+0000","last_peered":"2024-05-20T16:11:53.844942+0000","last_clean":"2024-05-20T16:11:53.844942+0000","last_became_active":"2024-05-20T16:11:47.191181+0000","last_became_peered":"2024-05-20T16:11:47.191181+0000","last_unstale":"2024-05-20T16:11:53.844942+0000","last_undegraded":"2024-05-20T16:11:53.844942+0000","last_fullsized":"2024-05-20T16:11:53.844942+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T19:30:42.561146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:11:53.423029+0000","last_change":"2024-05-20T16:11:51.915313+0000","last_active":"2024-05-20T16:11:53.423029+0000","last_peered":"2024-05-20T16:11:53.423029+0000","last_clean":"2024-05-20T16:11:53.423029+0000","last_became_active":"2024-05-20T16:11:51.914761+0000","last_became_peered":"2024-05-20T16:11:51.914761+0000","last_unstale":"2024-05-20T16:11:53.423029+0000","last_undegraded":"2024-05-20T16:11:53.423029+0000","last_fullsized":"2024-05-20T16:11:53.423029+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_clean_scrub_stamp":"2024-05-20T16:11:50.398948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T17:15:36.673449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"91'4","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:12:11.387439+0000","last_change":"2024-05-20T16:11:46.404513+0000","last_active":"2024-05-20T16:12:11.387439+0000","last_peered":"2024-05-20T16:12:11.387439+0000","last_clean":"2024-05-20T16:12:11.387439+0000","last_became_active":"2024-05-20T16:11:46.404163+0000","last_became_peered":"2024-05-20T16:11:46.404163+0000","last_unstale":"2024-05-20T16:12:11.387439+0000","last_undegraded":"2024-05-20T16:12:11.387439+0000","last_fullsized":"2024-05-20T16:12:11.387439+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_clean_scrub_stamp":"2024-05-20T16:11:45.363714+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:24:07.155558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:11:53.589347+0000","last_change":"2024-05-20T16:11:47.174527+0000","last_active":"2024-05-20T16:11:53.589347+0000","last_peered":"2024-05-20T16:11:53.589347+0000","last_clean":"2024-05-20T16:11:53.589347+0000","last_became_active":"2024-05-20T16:11:47.174165+0000","last_became_peered":"2024-05-20T16:11:47.174165+0000","last_unstale":"2024-05-20T16:11:53.589347+0000","last_undegraded":"2024-05-20T16:11:53.589347+0000","last_fullsized":"2024-05-20T16:11:53.589347+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T17:44:47.614704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:11:53.771534+0000","last_change":"2024-05-20T16:11:51.439013+0000","last_active":"2024-05-20T16:11:53.771534+0000","last_peered":"2024-05-20T16:11:53.771534+0000","last_clean":"2024-05-20T16:11:53.771534+0000","last_became_active":"2024-05-20T16:11:51.438408+0000","last_became_peered":"2024-05-20T16:11:51.438408+0000","last_unstale":"2024-05-20T16:11:53.771534+0000","last_undegraded":"2024-05-20T16:11:53.771534+0000","last_fullsized":"2024-05-20T16:11:53.771534+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_clean_scrub_stamp":"2024-05-20T16:11:50.398948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T19:27:53.723823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"91'3","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:12:12.397985+0000","last_change":"2024-05-20T16:11:46.407283+0000","last_active":"2024-05-20T16:12:12.397985+0000","last_peered":"2024-05-20T16:12:12.397985+0000","last_clean":"2024-05-20T16:12:12.397985+0000","last_became_active":"2024-05-20T16:11:46.406919+0000","last_became_peered":"2024-05-20T16:11:46.406919+0000","last_unstale":"2024-05-20T16:12:12.397985+0000","last_undegraded":"2024-05-20T16:12:12.397985+0000","last_fullsized":"2024-05-20T16:12:12.397985+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_clean_scrub_stamp":"2024-05-20T16:11:45.363714+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:16:09.166422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:11:53.589710+0000","last_change":"2024-05-20T16:11:47.177973+0000","last_active":"2024-05-20T16:11:53.589710+0000","last_peered":"2024-05-20T16:11:53.589710+0000","last_clean":"2024-05-20T16:11:53.589710+0000","last_became_active":"2024-05-20T16:11:47.177658+0000","last_became_peered":"2024-05-20T16:11:47.177658+0000","last_unstale":"2024-05-20T16:11:53.589710+0000","last_undegraded":"2024-05-20T16:11:53.589710+0000","last_fullsized":"2024-05-20T16:11:53.589710+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:34:04.004730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:11:53.786483+0000","last_change":"2024-05-20T16:11:51.441326+0000","last_active":"2024-05-20T16:11:53.786483+0000","last_peered":"2024-05-20T16:11:53.786483+0000","last_clean":"2024-05-20T16:11:53.786483+0000","last_became_active":"2024-05-20T16:11:51.440903+0000","last_became_peered":"2024-05-20T16:11:51.440903+0000","last_unstale":"2024-05-20T16:11:53.786483+0000","last_undegraded":"2024-05-20T16:11:53.786483+0000","last_fullsized":"2024-05-20T16:11:53.786483+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_clean_scrub_stamp":"2024-05-20T16:11:50.398948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:52:54.988916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"85'1","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:11:53.847073+0000","last_change":"2024-05-20T16:11:46.397548+0000","last_active":"2024-05-20T16:11:53.847073+0000","last_peered":"2024-05-20T16:11:53.847073+0000","last_clean":"2024-05-20T16:11:53.847073+0000","last_became_active":"2024-05-20T16:11:46.397207+0000","last_became_peered":"2024-05-20T16:11:46.397207+0000","last_unstale":"2024-05-20T16:11:53.847073+0000","last_undegraded":"2024-05-20T16:11:53.847073+0000","last_fullsized":"2024-05-20T16:11:53.847073+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_clean_scrub_stamp":"2024-05-20T16:11:45.363714+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:25:50.468298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:11:53.589596+0000","last_change":"2024-05-20T16:11:47.184162+0000","last_active":"2024-05-20T16:11:53.589596+0000","last_peered":"2024-05-20T16:11:53.589596+0000","last_clean":"2024-05-20T16:11:53.589596+0000","last_became_active":"2024-05-20T16:11:47.183845+0000","last_became_peered":"2024-05-20T16:11:47.183845+0000","last_unstale":"2024-05-20T16:11:53.589596+0000","last_undegraded":"2024-05-20T16:11:53.589596+0000","last_fullsized":"2024-05-20T16:11:53.589596+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T20:23:14.728178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:11:53.845283+0000","last_change":"2024-05-20T16:11:51.437974+0000","last_active":"2024-05-20T16:11:53.845283+0000","last_peered":"2024-05-20T16:11:53.845283+0000","last_clean":"2024-05-20T16:11:53.845283+0000","last_became_active":"2024-05-20T16:11:51.437053+0000","last_became_peered":"2024-05-20T16:11:51.437053+0000","last_unstale":"2024-05-20T16:11:53.845283+0000","last_undegraded":"2024-05-20T16:11:53.845283+0000","last_fullsized":"2024-05-20T16:11:53.845283+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_clean_scrub_stamp":"2024-05-20T16:11:50.398948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:09:31.314668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:11:54.266887+0000","last_change":"2024-05-20T16:11:46.398461+0000","last_active":"2024-05-20T16:11:54.266887+0000","last_peered":"2024-05-20T16:11:54.266887+0000","last_clean":"2024-05-20T16:11:54.266887+0000","last_became_active":"2024-05-20T16:11:46.398122+0000","last_became_peered":"2024-05-20T16:11:46.398122+0000","last_unstale":"2024-05-20T16:11:54.266887+0000","last_undegraded":"2024-05-20T16:11:54.266887+0000","last_fullsized":"2024-05-20T16:11:54.266887+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_clean_scrub_stamp":"2024-05-20T16:11:45.363714+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:34:45.592470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:11:53.589683+0000","last_change":"2024-05-20T16:11:47.179160+0000","last_active":"2024-05-20T16:11:53.589683+0000","last_peered":"2024-05-20T16:11:53.589683+0000","last_clean":"2024-05-20T16:11:53.589683+0000","last_became_active":"2024-05-20T16:11:47.178843+0000","last_became_peered":"2024-05-20T16:11:47.178843+0000","last_unstale":"2024-05-20T16:11:53.589683+0000","last_undegraded":"2024-05-20T16:11:53.589683+0000","last_fullsized":"2024-05-20T16:11:53.589683+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T20:05:57.013207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:11:53.846191+0000","last_change":"2024-05-20T16:11:47.185401+0000","last_active":"2024-05-20T16:11:53.846191+0000","last_peered":"2024-05-20T16:11:53.846191+0000","last_clean":"2024-05-20T16:11:53.846191+0000","last_became_active":"2024-05-20T16:11:47.185061+0000","last_became_peered":"2024-05-20T16:11:47.185061+0000","last_unstale":"2024-05-20T16:11:53.846191+0000","last_undegraded":"2024-05-20T16:11:53.846191+0000","last_fullsized":"2024-05-20T16:11:53.846191+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:33:32.528968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:11:53.420412+0000","last_change":"2024-05-20T16:11:47.174418+0000","last_active":"2024-05-20T16:11:53.420412+0000","last_peered":"2024-05-20T16:11:53.420412+0000","last_clean":"2024-05-20T16:11:53.420412+0000","last_became_active":"2024-05-20T16:11:47.174078+0000","last_became_peered":"2024-05-20T16:11:47.174078+0000","last_unstale":"2024-05-20T16:11:53.420412+0000","last_undegraded":"2024-05-20T16:11:53.420412+0000","last_fullsized":"2024-05-20T16:11:53.420412+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T16:20:12.268161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:11:53.420140+0000","last_change":"2024-05-20T16:11:47.177686+0000","last_active":"2024-05-20T16:11:53.420140+0000","last_peered":"2024-05-20T16:11:53.420140+0000","last_clean":"2024-05-20T16:11:53.420140+0000","last_became_active":"2024-05-20T16:11:47.177305+0000","last_became_peered":"2024-05-20T16:11:47.177305+0000","last_unstale":"2024-05-20T16:11:53.420140+0000","last_undegraded":"2024-05-20T16:11:53.420140+0000","last_fullsized":"2024-05-20T16:11:53.420140+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T18:36:35.919491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:11:53.786492+0000","last_change":"2024-05-20T16:11:47.175383+0000","last_active":"2024-05-20T16:11:53.786492+0000","last_peered":"2024-05-20T16:11:53.786492+0000","last_clean":"2024-05-20T16:11:53.786492+0000","last_became_active":"2024-05-20T16:11:47.175079+0000","last_became_peered":"2024-05-20T16:11:47.175079+0000","last_unstale":"2024-05-20T16:11:53.786492+0000","last_undegraded":"2024-05-20T16:11:53.786492+0000","last_fullsized":"2024-05-20T16:11:53.786492+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:33:52.072038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:11:54.266936+0000","last_change":"2024-05-20T16:11:47.182717+0000","last_active":"2024-05-20T16:11:54.266936+0000","last_peered":"2024-05-20T16:11:54.266936+0000","last_clean":"2024-05-20T16:11:54.266936+0000","last_became_active":"2024-05-20T16:11:47.182408+0000","last_became_peered":"2024-05-20T16:11:47.182408+0000","last_unstale":"2024-05-20T16:11:54.266936+0000","last_undegraded":"2024-05-20T16:11:54.266936+0000","last_fullsized":"2024-05-20T16:11:54.266936+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T18:25:13.698066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:11:53.846571+0000","last_change":"2024-05-20T16:11:47.182804+0000","last_active":"2024-05-20T16:11:53.846571+0000","last_peered":"2024-05-20T16:11:53.846571+0000","last_clean":"2024-05-20T16:11:53.846571+0000","last_became_active":"2024-05-20T16:11:47.182486+0000","last_became_peered":"2024-05-20T16:11:47.182486+0000","last_unstale":"2024-05-20T16:11:53.846571+0000","last_undegraded":"2024-05-20T16:11:53.846571+0000","last_fullsized":"2024-05-20T16:11:53.846571+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:24:52.592522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:11:53.420848+0000","last_change":"2024-05-20T16:11:47.175689+0000","last_active":"2024-05-20T16:11:53.420848+0000","last_peered":"2024-05-20T16:11:53.420848+0000","last_clean":"2024-05-20T16:11:53.420848+0000","last_became_active":"2024-05-20T16:11:47.168216+0000","last_became_peered":"2024-05-20T16:11:47.168216+0000","last_unstale":"2024-05-20T16:11:53.420848+0000","last_undegraded":"2024-05-20T16:11:53.420848+0000","last_fullsized":"2024-05-20T16:11:53.420848+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:50:52.707138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:11:53.845259+0000","last_change":"2024-05-20T16:11:47.191398+0000","last_active":"2024-05-20T16:11:53.845259+0000","last_peered":"2024-05-20T16:11:53.845259+0000","last_clean":"2024-05-20T16:11:53.845259+0000","last_became_active":"2024-05-20T16:11:47.191036+0000","last_became_peered":"2024-05-20T16:11:47.191036+0000","last_unstale":"2024-05-20T16:11:53.845259+0000","last_undegraded":"2024-05-20T16:11:53.845259+0000","last_fullsized":"2024-05-20T16:11:53.845259+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:27:49.023581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:11:53.845753+0000","last_change":"2024-05-20T16:11:47.184206+0000","last_active":"2024-05-20T16:11:53.845753+0000","last_peered":"2024-05-20T16:11:53.845753+0000","last_clean":"2024-05-20T16:11:53.845753+0000","last_became_active":"2024-05-20T16:11:47.183860+0000","last_became_peered":"2024-05-20T16:11:47.183860+0000","last_unstale":"2024-05-20T16:11:53.845753+0000","last_undegraded":"2024-05-20T16:11:53.845753+0000","last_fullsized":"2024-05-20T16:11:53.845753+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:52:42.507615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:11:53.844883+0000","last_change":"2024-05-20T16:11:47.180298+0000","last_active":"2024-05-20T16:11:53.844883+0000","last_peered":"2024-05-20T16:11:53.844883+0000","last_clean":"2024-05-20T16:11:53.844883+0000","last_became_active":"2024-05-20T16:11:47.179996+0000","last_became_peered":"2024-05-20T16:11:47.179996+0000","last_unstale":"2024-05-20T16:11:53.844883+0000","last_undegraded":"2024-05-20T16:11:53.844883+0000","last_fullsized":"2024-05-20T16:11:53.844883+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:49:53.432240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:11:53.771592+0000","last_change":"2024-05-20T16:11:47.176123+0000","last_active":"2024-05-20T16:11:53.771592+0000","last_peered":"2024-05-20T16:11:53.771592+0000","last_clean":"2024-05-20T16:11:53.771592+0000","last_became_active":"2024-05-20T16:11:47.175019+0000","last_became_peered":"2024-05-20T16:11:47.175019+0000","last_unstale":"2024-05-20T16:11:53.771592+0000","last_undegraded":"2024-05-20T16:11:53.771592+0000","last_fullsized":"2024-05-20T16:11:53.771592+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:23:32.010352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:11:54.266628+0000","last_change":"2024-05-20T16:11:47.180877+0000","last_active":"2024-05-20T16:11:54.266628+0000","last_peered":"2024-05-20T16:11:54.266628+0000","last_clean":"2024-05-20T16:11:54.266628+0000","last_became_active":"2024-05-20T16:11:47.180567+0000","last_became_peered":"2024-05-20T16:11:47.180567+0000","last_unstale":"2024-05-20T16:11:54.266628+0000","last_undegraded":"2024-05-20T16:11:54.266628+0000","last_fullsized":"2024-05-20T16:11:54.266628+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T17:54:31.165693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-20T16:11:53.846436+0000","last_change":"2024-05-20T16:11:47.179607+0000","last_active":"2024-05-20T16:11:53.846436+0000","last_peered":"2024-05-20T16:11:53.846436+0000","last_clean":"2024-05-20T16:11:53.846436+0000","last_became_active":"2024-05-20T16:11:47.179279+0000","last_became_peered":"2024-05-20T16:11:47.179279+0000","last_unstale":"2024-05-20T16:11:53.846436+0000","last_undegraded":"2024-05-20T16:11:53.846436+0000","last_fullsized":"2024-05-20T16:11:53.846436+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T18:38:20.788325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":246552,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":7452,"num_objects":56,"num_object_clones":0,"num_object_copies":168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":119,"num_write_kb":43,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":36,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":229376,"data_stored":39952,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":104,"ondisk_log_size":104,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892350992,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":983040,"data_stored":489673,"data_compressed":17602,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612630,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28060,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717092,"statfs":{"total":95995035648,"available":95966302208,"internally_reserved":0,"allocated":1069056,"data_stored":1084839,"data_compressed":53292,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":1598,"internal_metadata":27654594},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776156,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":987136,"data_stored":490255,"data_compressed":17602,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037793,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32156,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712996,"statfs":{"total":95995035648,"available":95962107904,"internally_reserved":0,"allocated":1003520,"data_stored":490411,"data_compressed":17602,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299431,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32200,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712952,"statfs":{"total":95995035648,"available":95962062848,"internally_reserved":0,"allocated":1048576,"data_stored":1081833,"data_compressed":53292,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462956,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32140,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713012,"statfs":{"total":95995035648,"available":95962124288,"internally_reserved":0,"allocated":987136,"data_stored":489763,"data_compressed":17602,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574917,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":1003520,"data_stored":500115,"data_compressed":17602,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738441,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":987136,"data_stored":489763,"data_compressed":17602,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411391,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1019904,"data_stored":494865,"data_compressed":17602,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149754,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":991232,"data_stored":489777,"data_compressed":17602,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888118,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":995328,"data_stored":490317,"data_compressed":17602,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724592,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1060864,"data_stored":1081517,"data_compressed":53292,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34051,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34051,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35690,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35690,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35690,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35690,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35690,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":23141,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":5192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":5420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-20T16:12:16.599 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-05-20T16:12:16.600 INFO:tasks.cephadm:Setting up client nodes... 2024-05-20T16:12:16.600 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 59d1600a-16c2-11ef-bc9a-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-05-20T16:12:16.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:12:16 smithi160 ceph-mon[36347]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-05-20T16:12:16.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:12:16 smithi160 ceph-mon[36347]: from='client.? 172.21.15.82:0/1454241362' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T16:12:16.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:12:16 smithi160 ceph-mon[36347]: from='client.? 172.21.15.82:0/1445240663' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-20T16:12:16.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:12:16 smithi160 ceph-mon[36347]: from='client.14937 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-20T16:12:16.873 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T16:12:16.885 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:12:16 smithi184 ceph-mon[36209]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-05-20T16:12:16.886 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:12:16 smithi184 ceph-mon[36209]: from='client.? 172.21.15.82:0/1454241362' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T16:12:16.886 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:12:16 smithi184 ceph-mon[36209]: from='client.? 172.21.15.82:0/1445240663' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-20T16:12:16.886 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:12:16 smithi184 ceph-mon[36209]: from='client.14937 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-20T16:12:17.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:12:17 smithi082 ceph-mon[32238]: from='client.14943 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-20T16:12:17.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:12:17 smithi082 ceph-mon[32238]: pgmap v307: 129 pgs: 129 active+clean; 584 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 597 B/s rd, 2.4 KiB/s wr, 7 op/s 2024-05-20T16:12:17.813 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:12:17 smithi160 ceph-mon[36347]: from='client.14943 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-20T16:12:17.813 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:12:17 smithi160 ceph-mon[36347]: pgmap v307: 129 pgs: 129 active+clean; 584 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 597 B/s rd, 2.4 KiB/s wr, 7 op/s 2024-05-20T16:12:17.885 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:12:17 smithi184 ceph-mon[36209]: from='client.14943 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-20T16:12:17.886 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:12:17 smithi184 ceph-mon[36209]: pgmap v307: 129 pgs: 129 active+clean; 584 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 597 B/s rd, 2.4 KiB/s wr, 7 op/s 2024-05-20T16:12:18.562 INFO:journalctl@ceph.mds.b.smithi160.stdout:May 20 16:12:18 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-b[66944]: 2024-05-20T16:12:18.311+0000 7ff20f5dd700 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-05-20T16:12:18.721 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:18.720+0000 7f1d92e7a700 1 -- 172.21.15.82:0/4226999798 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1d8c1020d0 msgr2=0x7f1d8c1024e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:12:18.722 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:18.720+0000 7f1d92e7a700 1 --2- 172.21.15.82:0/4226999798 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1d8c1020d0 0x7f1d8c1024e0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f1d80009990 tx=0x7f1d80009ca0 comp rx=0 tx=0).stop 2024-05-20T16:12:18.722 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:18.720+0000 7f1d92e7a700 1 -- 172.21.15.82:0/4226999798 shutdown_connections 2024-05-20T16:12:18.722 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:18.720+0000 7f1d92e7a700 1 --2- 172.21.15.82:0/4226999798 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1d8c103ba0 0x7f1d8c10a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:18.722 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:18.720+0000 7f1d92e7a700 1 --2- 172.21.15.82:0/4226999798 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1d8c1020d0 0x7f1d8c1024e0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:18.722 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:18.720+0000 7f1d92e7a700 1 --2- 172.21.15.82:0/4226999798 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1d8c103200 0x7f1d8c103610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:18.722 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:18.720+0000 7f1d92e7a700 1 -- 172.21.15.82:0/4226999798 >> 172.21.15.82:0/4226999798 conn(0x7f1d8c0fd710 msgr2=0x7f1d8c0ffbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:12:18.722 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:18.720+0000 7f1d92e7a700 1 -- 172.21.15.82:0/4226999798 shutdown_connections 2024-05-20T16:12:18.722 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:18.720+0000 7f1d92e7a700 1 -- 172.21.15.82:0/4226999798 wait complete. 2024-05-20T16:12:18.722 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:18.721+0000 7f1d92e7a700 1 Processor -- start 2024-05-20T16:12:18.722 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:18.721+0000 7f1d92e7a700 1 -- start start 2024-05-20T16:12:18.722 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:18.721+0000 7f1d92e7a700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1d8c1020d0 0x7f1d8c0752f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:12:18.723 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:18.721+0000 7f1d92e7a700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1d8c103200 0x7f1d8c19e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:12:18.723 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:18.721+0000 7f1d92e7a700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1d8c103ba0 0x7f1d8c1a0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:12:18.723 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:18.721+0000 7f1d92e7a700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f1d8c10dd60 con 0x7f1d8c103ba0 2024-05-20T16:12:18.723 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:18.721+0000 7f1d92e7a700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f1d8c10dbe0 con 0x7f1d8c1020d0 2024-05-20T16:12:18.723 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:18.721+0000 7f1d92e7a700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f1d8c10dee0 con 0x7f1d8c103200 2024-05-20T16:12:18.723 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:18.721+0000 7f1d91417700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1d8c103ba0 0x7f1d8c1a0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:12:18.723 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:18.721+0000 7f1d91417700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1d8c103ba0 0x7f1d8c1a0dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:47312/0 (socket says 172.21.15.82:47312) 2024-05-20T16:12:18.723 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:18.721+0000 7f1d91417700 1 -- 172.21.15.82:0/421862895 learned_addr learned my addr 172.21.15.82:0/421862895 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:12:18.723 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:18.721+0000 7f1d8bfff700 1 --2- 172.21.15.82:0/421862895 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1d8c103200 0x7f1d8c19e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:12:18.723 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:18.721+0000 7f1d90c16700 1 --2- 172.21.15.82:0/421862895 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1d8c1020d0 0x7f1d8c0752f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:12:18.723 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:18.721+0000 7f1d91417700 1 -- 172.21.15.82:0/421862895 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1d8c103200 msgr2=0x7f1d8c19e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:12:18.723 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:18.721+0000 7f1d91417700 1 --2- 172.21.15.82:0/421862895 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1d8c103200 0x7f1d8c19e830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:18.723 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:18.721+0000 7f1d91417700 1 -- 172.21.15.82:0/421862895 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1d8c1020d0 msgr2=0x7f1d8c0752f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:12:18.723 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:18.721+0000 7f1d91417700 1 --2- 172.21.15.82:0/421862895 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1d8c1020d0 0x7f1d8c0752f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:18.724 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:18.721+0000 7f1d91417700 1 -- 172.21.15.82:0/421862895 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d8c1a15f0 con 0x7f1d8c103ba0 2024-05-20T16:12:18.724 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:18.722+0000 7f1d91417700 1 --2- 172.21.15.82:0/421862895 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1d8c103ba0 0x7f1d8c1a0dc0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f1d8400b970 tx=0x7f1d8400bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:12:18.724 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:18.722+0000 7f1d89ffb700 1 -- 172.21.15.82:0/421862895 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d8400d860 con 0x7f1d8c103ba0 2024-05-20T16:12:18.724 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:18.722+0000 7f1d89ffb700 1 -- 172.21.15.82:0/421862895 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1d8400e430 con 0x7f1d8c103ba0 2024-05-20T16:12:18.724 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:18.722+0000 7f1d92e7a700 1 -- 172.21.15.82:0/421862895 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d8c1a18e0 con 0x7f1d8c103ba0 2024-05-20T16:12:18.724 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:18.722+0000 7f1d89ffb700 1 -- 172.21.15.82:0/421862895 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d84017660 con 0x7f1d8c103ba0 2024-05-20T16:12:18.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:18.722+0000 7f1d92e7a700 1 -- 172.21.15.82:0/421862895 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d8c1a1dd0 con 0x7f1d8c103ba0 2024-05-20T16:12:18.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:18.723+0000 7f1d92e7a700 1 -- 172.21.15.82:0/421862895 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d8c050170 con 0x7f1d8c103ba0 2024-05-20T16:12:18.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:18.724+0000 7f1d89ffb700 1 -- 172.21.15.82:0/421862895 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f1d8400e5e0 con 0x7f1d8c103ba0 2024-05-20T16:12:18.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:18.725+0000 7f1d89ffb700 1 --2- 172.21.15.82:0/421862895 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f1d7c0a2cb0 0x7f1d7c0a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:12:18.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:18.725+0000 7f1d90c16700 1 --2- 172.21.15.82:0/421862895 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f1d7c0a2cb0 0x7f1d7c0a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:12:18.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:18.725+0000 7f1d89ffb700 1 -- 172.21.15.82:0/421862895 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8802+0+0 (secure 0 0 0) 0x7f1d840cefa0 con 0x7f1d8c103ba0 2024-05-20T16:12:18.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:18.725+0000 7f1d90c16700 1 --2- 172.21.15.82:0/421862895 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f1d7c0a2cb0 0x7f1d7c0a51c0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f1d80006010 tx=0x7f1d8001a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:12:18.729 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:18.729+0000 7f1d89ffb700 1 -- 172.21.15.82:0/421862895 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f1d84013030 con 0x7f1d8c103ba0 2024-05-20T16:12:19.003 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:19.003+0000 7f1d92e7a700 1 -- 172.21.15.82:0/421862895 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7f1d8c02dc60 con 0x7f1d8c103ba0 2024-05-20T16:12:19.006 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:19.006+0000 7f1d89ffb700 1 -- 172.21.15.82:0/421862895 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7f1d8409bb60 con 0x7f1d8c103ba0 2024-05-20T16:12:19.006 INFO:teuthology.orchestra.run.smithi082.stdout:[client.0] 2024-05-20T16:12:19.006 INFO:teuthology.orchestra.run.smithi082.stdout: key = AQBjdktmzOdMABAAoTyV8kPu+6w+zz33z9gFQA== 2024-05-20T16:12:19.008 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:19.008+0000 7f1d92e7a700 1 -- 172.21.15.82:0/421862895 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f1d7c0a2cb0 msgr2=0x7f1d7c0a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:12:19.008 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:19.008+0000 7f1d92e7a700 1 --2- 172.21.15.82:0/421862895 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f1d7c0a2cb0 0x7f1d7c0a51c0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f1d80006010 tx=0x7f1d8001a040 comp rx=0 tx=0).stop 2024-05-20T16:12:19.008 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:19.008+0000 7f1d92e7a700 1 -- 172.21.15.82:0/421862895 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1d8c103ba0 msgr2=0x7f1d8c1a0dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:12:19.008 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:19.008+0000 7f1d92e7a700 1 --2- 172.21.15.82:0/421862895 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1d8c103ba0 0x7f1d8c1a0dc0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f1d8400b970 tx=0x7f1d8400bc80 comp rx=0 tx=0).stop 2024-05-20T16:12:19.008 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:19.008+0000 7f1d92e7a700 1 -- 172.21.15.82:0/421862895 shutdown_connections 2024-05-20T16:12:19.009 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:19.008+0000 7f1d92e7a700 1 --2- 172.21.15.82:0/421862895 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f1d7c0a2cb0 0x7f1d7c0a51c0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:19.009 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:19.008+0000 7f1d92e7a700 1 --2- 172.21.15.82:0/421862895 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1d8c1020d0 0x7f1d8c0752f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:19.009 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:19.008+0000 7f1d92e7a700 1 --2- 172.21.15.82:0/421862895 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1d8c103ba0 0x7f1d8c1a0dc0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:19.009 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:19.008+0000 7f1d92e7a700 1 --2- 172.21.15.82:0/421862895 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1d8c103200 0x7f1d8c19e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:19.009 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:19.008+0000 7f1d92e7a700 1 -- 172.21.15.82:0/421862895 >> 172.21.15.82:0/421862895 conn(0x7f1d8c0fd710 msgr2=0x7f1d8c106430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:12:19.009 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:19.008+0000 7f1d92e7a700 1 -- 172.21.15.82:0/421862895 shutdown_connections 2024-05-20T16:12:19.009 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:19.008+0000 7f1d92e7a700 1 -- 172.21.15.82:0/421862895 wait complete. 2024-05-20T16:12:19.313 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:12:19 smithi160 ceph-mon[36347]: from='client.? 172.21.15.82:0/421862895' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-20T16:12:19.313 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:12:19 smithi160 ceph-mon[36347]: from='client.? 172.21.15.82:0/421862895' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-05-20T16:12:19.329 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:12:19 smithi082 ceph-mon[32238]: from='client.? 172.21.15.82:0/421862895' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-20T16:12:19.329 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:12:19 smithi082 ceph-mon[32238]: from='client.? 172.21.15.82:0/421862895' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-05-20T16:12:19.385 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:12:19 smithi184 ceph-mon[36209]: from='client.? 172.21.15.82:0/421862895' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-20T16:12:19.386 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:12:19 smithi184 ceph-mon[36209]: from='client.? 172.21.15.82:0/421862895' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-05-20T16:12:19.892 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-05-20T16:12:19.892 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-05-20T16:12:19.893 DEBUG:teuthology.orchestra.run.smithi082:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-05-20T16:12:19.931 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-05-20T16:12:19.931 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-05-20T16:12:19.931 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell --fsid 59d1600a-16c2-11ef-bc9a-c7b262605968 -- ceph mgr dump --format=json 2024-05-20T16:12:20.213 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T16:12:20.335 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:12:20 smithi082 ceph-mon[32238]: pgmap v308: 129 pgs: 129 active+clean; 620 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 5.9 KiB/s wr, 12 op/s 2024-05-20T16:12:20.386 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:12:20 smithi184 ceph-mon[36209]: pgmap v308: 129 pgs: 129 active+clean; 620 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 5.9 KiB/s wr, 12 op/s 2024-05-20T16:12:20.562 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:12:20 smithi160 ceph-mon[36347]: pgmap v308: 129 pgs: 129 active+clean; 620 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 5.9 KiB/s wr, 12 op/s 2024-05-20T16:12:21.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:12:21 smithi082 ceph-mon[32238]: pgmap v309: 129 pgs: 129 active+clean; 620 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 5.7 KiB/s wr, 12 op/s 2024-05-20T16:12:21.635 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:12:21 smithi184 ceph-mon[36209]: pgmap v309: 129 pgs: 129 active+clean; 620 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 5.7 KiB/s wr, 12 op/s 2024-05-20T16:12:21.813 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:12:21 smithi160 ceph-mon[36347]: pgmap v309: 129 pgs: 129 active+clean; 620 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 5.7 KiB/s wr, 12 op/s 2024-05-20T16:12:22.017 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:22.017+0000 7fce9801c700 1 -- 172.21.15.82:0/932734669 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fce90106c90 msgr2=0x7fce901070a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:12:22.017 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:22.017+0000 7fce9801c700 1 --2- 172.21.15.82:0/932734669 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fce90106c90 0x7fce901070a0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fce840099c0 tx=0x7fce84009cd0 comp rx=0 tx=0).stop 2024-05-20T16:12:22.017 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:22.017+0000 7fce9801c700 1 -- 172.21.15.82:0/932734669 shutdown_connections 2024-05-20T16:12:22.017 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:22.017+0000 7fce9801c700 1 --2- 172.21.15.82:0/932734669 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fce901075e0 0x7fce9010ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:22.017 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:22.017+0000 7fce9801c700 1 --2- 172.21.15.82:0/932734669 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fce90102360 0x7fce90102770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:22.017 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:22.017+0000 7fce9801c700 1 --2- 172.21.15.82:0/932734669 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fce90106c90 0x7fce901070a0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:22.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:22.017+0000 7fce9801c700 1 -- 172.21.15.82:0/932734669 >> 172.21.15.82:0/932734669 conn(0x7fce900fddb0 msgr2=0x7fce901001d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:12:22.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:22.017+0000 7fce9801c700 1 -- 172.21.15.82:0/932734669 shutdown_connections 2024-05-20T16:12:22.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:22.017+0000 7fce9801c700 1 -- 172.21.15.82:0/932734669 wait complete. 2024-05-20T16:12:22.020 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:22.018+0000 7fce9801c700 1 Processor -- start 2024-05-20T16:12:22.020 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:22.018+0000 7fce9801c700 1 -- start start 2024-05-20T16:12:22.020 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:22.018+0000 7fce9801c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fce90102360 0x7fce90198e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:12:22.020 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:22.018+0000 7fce9801c700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fce90106c90 0x7fce90199420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:12:22.020 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:22.018+0000 7fce9801c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fce901075e0 0x7fce9019d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:12:22.020 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:22.018+0000 7fce9801c700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fce90112430 con 0x7fce90102360 2024-05-20T16:12:22.020 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:22.018+0000 7fce9801c700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fce901122b0 con 0x7fce901075e0 2024-05-20T16:12:22.020 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:22.018+0000 7fce9801c700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fce901125b0 con 0x7fce90106c90 2024-05-20T16:12:22.021 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:22.018+0000 7fce95db8700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fce90102360 0x7fce90198e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:12:22.021 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:22.018+0000 7fce95db8700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fce90102360 0x7fce90198e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:39898/0 (socket says 172.21.15.82:39898) 2024-05-20T16:12:22.021 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:22.018+0000 7fce95db8700 1 -- 172.21.15.82:0/2870363908 learned_addr learned my addr 172.21.15.82:0/2870363908 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:12:22.021 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:22.018+0000 7fce965b9700 1 --2- 172.21.15.82:0/2870363908 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fce901075e0 0x7fce9019d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:12:22.021 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:22.018+0000 7fce95db8700 1 -- 172.21.15.82:0/2870363908 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fce90106c90 msgr2=0x7fce90199420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:12:22.021 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:22.018+0000 7fce955b7700 1 --2- 172.21.15.82:0/2870363908 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fce90106c90 0x7fce90199420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:12:22.021 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:22.019+0000 7fce95db8700 1 --2- 172.21.15.82:0/2870363908 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fce90106c90 0x7fce90199420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:22.021 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:22.019+0000 7fce95db8700 1 -- 172.21.15.82:0/2870363908 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fce901075e0 msgr2=0x7fce9019d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:12:22.021 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:22.019+0000 7fce95db8700 1 --2- 172.21.15.82:0/2870363908 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fce901075e0 0x7fce9019d6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:22.021 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:22.019+0000 7fce95db8700 1 -- 172.21.15.82:0/2870363908 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce9019dd50 con 0x7fce90102360 2024-05-20T16:12:22.021 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:22.019+0000 7fce965b9700 1 --2- 172.21.15.82:0/2870363908 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fce901075e0 0x7fce9019d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T16:12:22.021 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:22.019+0000 7fce955b7700 1 --2- 172.21.15.82:0/2870363908 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fce90106c90 0x7fce90199420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T16:12:22.021 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:22.019+0000 7fce95db8700 1 --2- 172.21.15.82:0/2870363908 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fce90102360 0x7fce90198e80 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fce8c00d9e0 tx=0x7fce8c00dda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:12:22.022 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:22.019+0000 7fce82ffd700 1 -- 172.21.15.82:0/2870363908 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce8c01b070 con 0x7fce90102360 2024-05-20T16:12:22.022 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:22.019+0000 7fce9801c700 1 -- 172.21.15.82:0/2870363908 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce9019e040 con 0x7fce90102360 2024-05-20T16:12:22.022 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:22.019+0000 7fce82ffd700 1 -- 172.21.15.82:0/2870363908 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fce8c00ecd0 con 0x7fce90102360 2024-05-20T16:12:22.022 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:22.019+0000 7fce82ffd700 1 -- 172.21.15.82:0/2870363908 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce8c014760 con 0x7fce90102360 2024-05-20T16:12:22.022 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:22.019+0000 7fce9801c700 1 -- 172.21.15.82:0/2870363908 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce9006aec0 con 0x7fce90102360 2024-05-20T16:12:22.022 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:22.020+0000 7fce9801c700 1 -- 172.21.15.82:0/2870363908 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce90103810 con 0x7fce90102360 2024-05-20T16:12:22.023 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:22.022+0000 7fce82ffd700 1 -- 172.21.15.82:0/2870363908 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fce8c01e070 con 0x7fce90102360 2024-05-20T16:12:22.023 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:22.023+0000 7fce82ffd700 1 --2- 172.21.15.82:0/2870363908 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fce7c0a2cb0 0x7fce7c0a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:12:22.023 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:22.023+0000 7fce955b7700 1 --2- 172.21.15.82:0/2870363908 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fce7c0a2cb0 0x7fce7c0a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:12:22.023 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:22.023+0000 7fce82ffd700 1 -- 172.21.15.82:0/2870363908 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8802+0+0 (secure 0 0 0) 0x7fce8c0d0e60 con 0x7fce90102360 2024-05-20T16:12:22.023 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:22.023+0000 7fce955b7700 1 --2- 172.21.15.82:0/2870363908 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fce7c0a2cb0 0x7fce7c0a51c0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fce84006010 tx=0x7fce84005e80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:12:22.026 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:22.027+0000 7fce82ffd700 1 -- 172.21.15.82:0/2870363908 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fce8c09dbc0 con 0x7fce90102360 2024-05-20T16:12:22.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:22.282+0000 7fce9801c700 1 -- 172.21.15.82:0/2870363908 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7fce90050170 con 0x7fce90102360 2024-05-20T16:12:22.286 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:22.287+0000 7fce82ffd700 1 -- 172.21.15.82:0/2870363908 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+263198 (secure 0 0 0) 0x7fce90050170 con 0x7fce90102360 2024-05-20T16:12:22.290 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-05-20T16:12:22.291 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:22.290+0000 7fce9801c700 1 -- 172.21.15.82:0/2870363908 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fce7c0a2cb0 msgr2=0x7fce7c0a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:12:22.291 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:22.290+0000 7fce9801c700 1 --2- 172.21.15.82:0/2870363908 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fce7c0a2cb0 0x7fce7c0a51c0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fce84006010 tx=0x7fce84005e80 comp rx=0 tx=0).stop 2024-05-20T16:12:22.292 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:22.290+0000 7fce9801c700 1 -- 172.21.15.82:0/2870363908 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fce90102360 msgr2=0x7fce90198e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:12:22.292 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:22.290+0000 7fce9801c700 1 --2- 172.21.15.82:0/2870363908 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fce90102360 0x7fce90198e80 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fce8c00d9e0 tx=0x7fce8c00dda0 comp rx=0 tx=0).stop 2024-05-20T16:12:22.292 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:22.290+0000 7fce9801c700 1 -- 172.21.15.82:0/2870363908 shutdown_connections 2024-05-20T16:12:22.292 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:22.290+0000 7fce9801c700 1 --2- 172.21.15.82:0/2870363908 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fce7c0a2cb0 0x7fce7c0a51c0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:22.292 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:22.290+0000 7fce9801c700 1 --2- 172.21.15.82:0/2870363908 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fce901075e0 0x7fce9019d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:22.292 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:22.290+0000 7fce9801c700 1 --2- 172.21.15.82:0/2870363908 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fce90102360 0x7fce90198e80 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:22.292 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:22.290+0000 7fce9801c700 1 --2- 172.21.15.82:0/2870363908 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fce90106c90 0x7fce90199420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:22.292 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:22.290+0000 7fce9801c700 1 -- 172.21.15.82:0/2870363908 >> 172.21.15.82:0/2870363908 conn(0x7fce900fddb0 msgr2=0x7fce90108540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:12:22.292 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:22.290+0000 7fce9801c700 1 -- 172.21.15.82:0/2870363908 shutdown_connections 2024-05-20T16:12:22.292 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:22.290+0000 7fce9801c700 1 -- 172.21.15.82:0/2870363908 wait complete. 2024-05-20T16:12:22.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:12:22 smithi082 ceph-mon[32238]: from='client.? 172.21.15.82:0/2870363908' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-05-20T16:12:22.635 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:12:22 smithi184 ceph-mon[36209]: from='client.? 172.21.15.82:0/2870363908' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-05-20T16:12:22.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:12:22 smithi160 ceph-mon[36347]: from='client.? 172.21.15.82:0/2870363908' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-05-20T16:12:22.972 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":17,"flags":0,"active_gid":14150,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":379530823},{"type":"v1","addr":"172.21.15.82:6801","nonce":379530823}]},"active_addr":"172.21.15.82:6801/379530823","active_change":"2024-05-20T16:04:45.162085+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24109,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24116,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.82:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.82:0","nonce":3429482372}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.82:0","nonce":2353029080}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.82:0","nonce":3379158184}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.82:0","nonce":1953851727}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.82:0","nonce":3142964075}]}]} 2024-05-20T16:12:22.977 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-05-20T16:12:22.977 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-05-20T16:12:22.977 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell --fsid 59d1600a-16c2-11ef-bc9a-c7b262605968 -- ceph osd dump --format=json 2024-05-20T16:12:23.222 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T16:12:23.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:12:23 smithi082 ceph-mon[32238]: pgmap v310: 129 pgs: 129 active+clean; 625 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 6.4 KiB/s wr, 14 op/s 2024-05-20T16:12:23.635 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:12:23 smithi184 ceph-mon[36209]: pgmap v310: 129 pgs: 129 active+clean; 625 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 6.4 KiB/s wr, 14 op/s 2024-05-20T16:12:23.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:12:23 smithi160 ceph-mon[36347]: pgmap v310: 129 pgs: 129 active+clean; 625 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 6.4 KiB/s wr, 14 op/s 2024-05-20T16:12:24.961 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:24.961+0000 7fe4e8f7d700 1 -- 172.21.15.82:0/3536383179 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe4e4075af0 msgr2=0x7fe4e4075f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:12:24.962 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:24.961+0000 7fe4e8f7d700 1 --2- 172.21.15.82:0/3536383179 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe4e4075af0 0x7fe4e4075f00 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fe4d4009990 tx=0x7fe4d4009ca0 comp rx=0 tx=0).stop 2024-05-20T16:12:24.962 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:24.961+0000 7fe4e8f7d700 1 -- 172.21.15.82:0/3536383179 shutdown_connections 2024-05-20T16:12:24.962 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:24.961+0000 7fe4e8f7d700 1 --2- 172.21.15.82:0/3536383179 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe4e41141d0 0x7fe4e41165f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:24.962 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:24.961+0000 7fe4e8f7d700 1 --2- 172.21.15.82:0/3536383179 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe4e4075af0 0x7fe4e4075f00 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:24.962 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:24.961+0000 7fe4e8f7d700 1 --2- 172.21.15.82:0/3536383179 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe4e4076440 0x7fe4e4113c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:24.962 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:24.961+0000 7fe4e8f7d700 1 -- 172.21.15.82:0/3536383179 >> 172.21.15.82:0/3536383179 conn(0x7fe4e40fed20 msgr2=0x7fe4e4101170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:12:24.962 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:24.961+0000 7fe4e8f7d700 1 -- 172.21.15.82:0/3536383179 shutdown_connections 2024-05-20T16:12:24.962 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:24.961+0000 7fe4e8f7d700 1 -- 172.21.15.82:0/3536383179 wait complete. 2024-05-20T16:12:24.964 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:24.962+0000 7fe4e8f7d700 1 Processor -- start 2024-05-20T16:12:24.964 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:24.962+0000 7fe4e8f7d700 1 -- start start 2024-05-20T16:12:24.964 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:24.962+0000 7fe4e8f7d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe4e4075af0 0x7fe4e41a0320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:12:24.964 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:24.962+0000 7fe4e8f7d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe4e4076440 0x7fe4e41a08c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:12:24.964 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:24.962+0000 7fe4e8f7d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe4e41141d0 0x7fe4e41a4b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:12:24.964 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:24.962+0000 7fe4e8f7d700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fe4e4119a30 con 0x7fe4e41141d0 2024-05-20T16:12:24.965 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:24.962+0000 7fe4e8f7d700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fe4e41198b0 con 0x7fe4e4075af0 2024-05-20T16:12:24.965 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:24.962+0000 7fe4e8f7d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fe4e4119bb0 con 0x7fe4e4076440 2024-05-20T16:12:24.965 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:24.963+0000 7fe4e2d9d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe4e41141d0 0x7fe4e41a4b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:12:24.965 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:24.963+0000 7fe4e2d9d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe4e41141d0 0x7fe4e41a4b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:39920/0 (socket says 172.21.15.82:39920) 2024-05-20T16:12:24.965 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:24.963+0000 7fe4e2d9d700 1 -- 172.21.15.82:0/3950852226 learned_addr learned my addr 172.21.15.82:0/3950852226 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:12:24.965 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:24.963+0000 7fe4e1d9b700 1 --2- 172.21.15.82:0/3950852226 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe4e4076440 0x7fe4e41a08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:12:24.965 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:24.963+0000 7fe4e259c700 1 --2- 172.21.15.82:0/3950852226 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe4e4075af0 0x7fe4e41a0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:12:24.965 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:24.963+0000 7fe4e2d9d700 1 -- 172.21.15.82:0/3950852226 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe4e4076440 msgr2=0x7fe4e41a08c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:12:24.965 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:24.963+0000 7fe4e2d9d700 1 --2- 172.21.15.82:0/3950852226 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe4e4076440 0x7fe4e41a08c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:24.965 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:24.963+0000 7fe4e2d9d700 1 -- 172.21.15.82:0/3950852226 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe4e4075af0 msgr2=0x7fe4e41a0320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:12:24.965 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:24.963+0000 7fe4e2d9d700 1 --2- 172.21.15.82:0/3950852226 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe4e4075af0 0x7fe4e41a0320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:24.965 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:24.963+0000 7fe4e2d9d700 1 -- 172.21.15.82:0/3950852226 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4e41a51f0 con 0x7fe4e41141d0 2024-05-20T16:12:24.965 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:24.963+0000 7fe4e2d9d700 1 --2- 172.21.15.82:0/3950852226 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe4e41141d0 0x7fe4e41a4b50 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fe4dc007ee0 tx=0x7fe4dc00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:12:24.965 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:24.963+0000 7fe4d37fe700 1 -- 172.21.15.82:0/3950852226 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4dc00dbd0 con 0x7fe4e41141d0 2024-05-20T16:12:24.966 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:24.963+0000 7fe4d37fe700 1 -- 172.21.15.82:0/3950852226 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe4dc00dd70 con 0x7fe4e41141d0 2024-05-20T16:12:24.966 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:24.963+0000 7fe4d37fe700 1 -- 172.21.15.82:0/3950852226 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4dc005a60 con 0x7fe4e41141d0 2024-05-20T16:12:24.966 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:24.963+0000 7fe4e8f7d700 1 -- 172.21.15.82:0/3950852226 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4e41a54e0 con 0x7fe4e41141d0 2024-05-20T16:12:24.966 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:24.963+0000 7fe4e8f7d700 1 -- 172.21.15.82:0/3950852226 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4e41ace00 con 0x7fe4e41141d0 2024-05-20T16:12:24.966 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:24.964+0000 7fe4e8f7d700 1 -- 172.21.15.82:0/3950852226 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4e4050170 con 0x7fe4e41141d0 2024-05-20T16:12:24.966 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:24.967+0000 7fe4d37fe700 1 -- 172.21.15.82:0/3950852226 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fe4dc01a020 con 0x7fe4e41141d0 2024-05-20T16:12:24.967 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:24.967+0000 7fe4d37fe700 1 --2- 172.21.15.82:0/3950852226 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fe4cc0a2d30 0x7fe4cc0a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:12:24.967 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:24.967+0000 7fe4e259c700 1 --2- 172.21.15.82:0/3950852226 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fe4cc0a2d30 0x7fe4cc0a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:12:24.967 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:24.967+0000 7fe4d37fe700 1 -- 172.21.15.82:0/3950852226 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8802+0+0 (secure 0 0 0) 0x7fe4dc0cf0a0 con 0x7fe4e41141d0 2024-05-20T16:12:24.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:24.968+0000 7fe4e259c700 1 --2- 172.21.15.82:0/3950852226 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fe4cc0a2d30 0x7fe4cc0a5240 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fe4d4006010 tx=0x7fe4d401a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:12:24.971 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:24.971+0000 7fe4d37fe700 1 -- 172.21.15.82:0/3950852226 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fe4dc010070 con 0x7fe4e41141d0 2024-05-20T16:12:25.166 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:25.167+0000 7fe4e8f7d700 1 -- 172.21.15.82:0/3950852226 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe4e41acfd0 con 0x7fe4e41141d0 2024-05-20T16:12:25.168 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:25.168+0000 7fe4d37fe700 1 -- 172.21.15.82:0/3950852226 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+24153 (secure 0 0 0) 0x7fe4dc0cce20 con 0x7fe4e41141d0 2024-05-20T16:12:25.169 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-05-20T16:12:25.169 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":91,"fsid":"59d1600a-16c2-11ef-bc9a-c7b262605968","created":"2024-05-20T16:03:46.611556+0000","modified":"2024-05-20T16:11:53.416085+0000","last_up_change":"2024-05-20T16:10:59.303479+0000","last_in_change":"2024-05-20T16:10:39.063561+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-20T16:07:09.493727+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-20T16:11:45.058035+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-20T16:11:45.943862+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-20T16:11:49.637579+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"bae842a9-03ec-490e-b56d-f7957235cdc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":1484330167},{"type":"v1","addr":"172.21.15.82:6803","nonce":1484330167}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":1484330167},{"type":"v1","addr":"172.21.15.82:6805","nonce":1484330167}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1484330167},{"type":"v1","addr":"172.21.15.82:6809","nonce":1484330167}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":1484330167},{"type":"v1","addr":"172.21.15.82:6807","nonce":1484330167}]},"public_addr":"172.21.15.82:6803/1484330167","cluster_addr":"172.21.15.82:6805/1484330167","heartbeat_back_addr":"172.21.15.82:6809/1484330167","heartbeat_front_addr":"172.21.15.82:6807/1484330167","state":["exists","up"]},{"osd":1,"uuid":"59f6a2ba-b5de-4161-81c5-92a417abacdf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":1457003568},{"type":"v1","addr":"172.21.15.160:6801","nonce":1457003568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":1457003568},{"type":"v1","addr":"172.21.15.160:6803","nonce":1457003568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6806","nonce":1457003568},{"type":"v1","addr":"172.21.15.160:6807","nonce":1457003568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":1457003568},{"type":"v1","addr":"172.21.15.160:6805","nonce":1457003568}]},"public_addr":"172.21.15.160:6801/1457003568","cluster_addr":"172.21.15.160:6803/1457003568","heartbeat_back_addr":"172.21.15.160:6807/1457003568","heartbeat_front_addr":"172.21.15.160:6805/1457003568","state":["exists","up"]},{"osd":2,"uuid":"3bbb5587-b449-41f3-bd26-67ec3d6afdfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":4051293934},{"type":"v1","addr":"172.21.15.184:6801","nonce":4051293934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":4051293934},{"type":"v1","addr":"172.21.15.184:6803","nonce":4051293934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":4051293934},{"type":"v1","addr":"172.21.15.184:6807","nonce":4051293934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":4051293934},{"type":"v1","addr":"172.21.15.184:6805","nonce":4051293934}]},"public_addr":"172.21.15.184:6801/4051293934","cluster_addr":"172.21.15.184:6803/4051293934","heartbeat_back_addr":"172.21.15.184:6807/4051293934","heartbeat_front_addr":"172.21.15.184:6805/4051293934","state":["exists","up"]},{"osd":3,"uuid":"ca651f90-09b8-4081-a6f9-40b31268a837","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":3947485599},{"type":"v1","addr":"172.21.15.82:6811","nonce":3947485599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":3947485599},{"type":"v1","addr":"172.21.15.82:6813","nonce":3947485599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":3947485599},{"type":"v1","addr":"172.21.15.82:6817","nonce":3947485599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":3947485599},{"type":"v1","addr":"172.21.15.82:6815","nonce":3947485599}]},"public_addr":"172.21.15.82:6811/3947485599","cluster_addr":"172.21.15.82:6813/3947485599","heartbeat_back_addr":"172.21.15.82:6817/3947485599","heartbeat_front_addr":"172.21.15.82:6815/3947485599","state":["exists","up"]},{"osd":4,"uuid":"cb3dfd56-f677-40cb-9c96-8319ea5ac788","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":4160830014},{"type":"v1","addr":"172.21.15.160:6809","nonce":4160830014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":4160830014},{"type":"v1","addr":"172.21.15.160:6811","nonce":4160830014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":4160830014},{"type":"v1","addr":"172.21.15.160:6815","nonce":4160830014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6812","nonce":4160830014},{"type":"v1","addr":"172.21.15.160:6813","nonce":4160830014}]},"public_addr":"172.21.15.160:6809/4160830014","cluster_addr":"172.21.15.160:6811/4160830014","heartbeat_back_addr":"172.21.15.160:6815/4160830014","heartbeat_front_addr":"172.21.15.160:6813/4160830014","state":["exists","up"]},{"osd":5,"uuid":"c70cfcd1-a4ff-461f-8c73-479b061200fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":1046282672},{"type":"v1","addr":"172.21.15.184:6809","nonce":1046282672}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":1046282672},{"type":"v1","addr":"172.21.15.184:6811","nonce":1046282672}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":1046282672},{"type":"v1","addr":"172.21.15.184:6815","nonce":1046282672}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":1046282672},{"type":"v1","addr":"172.21.15.184:6813","nonce":1046282672}]},"public_addr":"172.21.15.184:6809/1046282672","cluster_addr":"172.21.15.184:6811/1046282672","heartbeat_back_addr":"172.21.15.184:6815/1046282672","heartbeat_front_addr":"172.21.15.184:6813/1046282672","state":["exists","up"]},{"osd":6,"uuid":"d47ce001-9dd1-4830-bcc9-888601187c6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":2789736499},{"type":"v1","addr":"172.21.15.82:6819","nonce":2789736499}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":2789736499},{"type":"v1","addr":"172.21.15.82:6821","nonce":2789736499}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":2789736499},{"type":"v1","addr":"172.21.15.82:6825","nonce":2789736499}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":2789736499},{"type":"v1","addr":"172.21.15.82:6823","nonce":2789736499}]},"public_addr":"172.21.15.82:6819/2789736499","cluster_addr":"172.21.15.82:6821/2789736499","heartbeat_back_addr":"172.21.15.82:6825/2789736499","heartbeat_front_addr":"172.21.15.82:6823/2789736499","state":["exists","up"]},{"osd":7,"uuid":"25c54f43-18cb-41e4-83b4-2732b5a93889","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6816","nonce":2340683378},{"type":"v1","addr":"172.21.15.160:6817","nonce":2340683378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":2340683378},{"type":"v1","addr":"172.21.15.160:6819","nonce":2340683378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":2340683378},{"type":"v1","addr":"172.21.15.160:6823","nonce":2340683378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6820","nonce":2340683378},{"type":"v1","addr":"172.21.15.160:6821","nonce":2340683378}]},"public_addr":"172.21.15.160:6817/2340683378","cluster_addr":"172.21.15.160:6819/2340683378","heartbeat_back_addr":"172.21.15.160:6823/2340683378","heartbeat_front_addr":"172.21.15.160:6821/2340683378","state":["exists","up"]},{"osd":8,"uuid":"1f8b478d-287f-45f5-8526-d1a58c5bc07c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":3831576372},{"type":"v1","addr":"172.21.15.184:6817","nonce":3831576372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":3831576372},{"type":"v1","addr":"172.21.15.184:6819","nonce":3831576372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":3831576372},{"type":"v1","addr":"172.21.15.184:6823","nonce":3831576372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":3831576372},{"type":"v1","addr":"172.21.15.184:6821","nonce":3831576372}]},"public_addr":"172.21.15.184:6817/3831576372","cluster_addr":"172.21.15.184:6819/3831576372","heartbeat_back_addr":"172.21.15.184:6823/3831576372","heartbeat_front_addr":"172.21.15.184:6821/3831576372","state":["exists","up"]},{"osd":9,"uuid":"0d69e319-954f-4b18-95ea-31794f18dc3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":716330782},{"type":"v1","addr":"172.21.15.82:6827","nonce":716330782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":716330782},{"type":"v1","addr":"172.21.15.82:6829","nonce":716330782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":716330782},{"type":"v1","addr":"172.21.15.82:6833","nonce":716330782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":716330782},{"type":"v1","addr":"172.21.15.82:6831","nonce":716330782}]},"public_addr":"172.21.15.82:6827/716330782","cluster_addr":"172.21.15.82:6829/716330782","heartbeat_back_addr":"172.21.15.82:6833/716330782","heartbeat_front_addr":"172.21.15.82:6831/716330782","state":["exists","up"]},{"osd":10,"uuid":"39dc1b78-61cb-42ca-b5b6-1e44ac2b6500","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":2327428014},{"type":"v1","addr":"172.21.15.160:6825","nonce":2327428014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":2327428014},{"type":"v1","addr":"172.21.15.160:6827","nonce":2327428014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":2327428014},{"type":"v1","addr":"172.21.15.160:6831","nonce":2327428014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":2327428014},{"type":"v1","addr":"172.21.15.160:6829","nonce":2327428014}]},"public_addr":"172.21.15.160:6825/2327428014","cluster_addr":"172.21.15.160:6827/2327428014","heartbeat_back_addr":"172.21.15.160:6831/2327428014","heartbeat_front_addr":"172.21.15.160:6829/2327428014","state":["exists","up"]},{"osd":11,"uuid":"4306b3c1-707a-4010-987c-209ea0e948e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":3594364650},{"type":"v1","addr":"172.21.15.184:6825","nonce":3594364650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":3594364650},{"type":"v1","addr":"172.21.15.184:6827","nonce":3594364650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3594364650},{"type":"v1","addr":"172.21.15.184:6831","nonce":3594364650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3594364650},{"type":"v1","addr":"172.21.15.184:6829","nonce":3594364650}]},"public_addr":"172.21.15.184:6825/3594364650","cluster_addr":"172.21.15.184:6827/3594364650","heartbeat_back_addr":"172.21.15.184:6831/3594364650","heartbeat_front_addr":"172.21.15.184:6829/3594364650","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:06:17.855998+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:06:41.473270+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:07:05.304108+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:07:29.887472+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:07:55.387286+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:08:20.579669+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:08:46.333051+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:09:12.289668+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:09:38.180696+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:10:04.471459+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:10:31.041912+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:10:57.477266+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.82:0/2005424509":"2024-05-21T16:04:45.161949+0000","172.21.15.82:6801/1763049308":"2024-05-21T16:04:45.161949+0000","172.21.15.82:6801/1959459348":"2024-05-21T16:04:19.208291+0000","172.21.15.82:0/3131904556":"2024-05-21T16:04:19.208291+0000","172.21.15.82:6800/1959459348":"2024-05-21T16:04:19.208291+0000","172.21.15.82:0/2456966342":"2024-05-21T16:04:45.161949+0000","172.21.15.82:0/4179978833":"2024-05-21T16:04:19.208291+0000","172.21.15.82:0/1602862225":"2024-05-21T16:04:45.161949+0000","172.21.15.82:0/106101180":"2024-05-21T16:04:19.208291+0000","172.21.15.82:6800/1763049308":"2024-05-21T16:04:45.161949+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-20T16:12:25.170 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:25.170+0000 7fe4e8f7d700 1 -- 172.21.15.82:0/3950852226 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fe4cc0a2d30 msgr2=0x7fe4cc0a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:12:25.171 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:25.170+0000 7fe4e8f7d700 1 --2- 172.21.15.82:0/3950852226 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fe4cc0a2d30 0x7fe4cc0a5240 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fe4d4006010 tx=0x7fe4d401a040 comp rx=0 tx=0).stop 2024-05-20T16:12:25.171 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:25.170+0000 7fe4e8f7d700 1 -- 172.21.15.82:0/3950852226 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe4e41141d0 msgr2=0x7fe4e41a4b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:12:25.171 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:25.170+0000 7fe4e8f7d700 1 --2- 172.21.15.82:0/3950852226 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe4e41141d0 0x7fe4e41a4b50 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fe4dc007ee0 tx=0x7fe4dc00d3b0 comp rx=0 tx=0).stop 2024-05-20T16:12:25.171 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:25.171+0000 7fe4e8f7d700 1 -- 172.21.15.82:0/3950852226 shutdown_connections 2024-05-20T16:12:25.171 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:25.171+0000 7fe4e8f7d700 1 --2- 172.21.15.82:0/3950852226 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fe4cc0a2d30 0x7fe4cc0a5240 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:25.171 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:25.171+0000 7fe4e8f7d700 1 --2- 172.21.15.82:0/3950852226 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe4e4075af0 0x7fe4e41a0320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:25.171 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:25.171+0000 7fe4e8f7d700 1 --2- 172.21.15.82:0/3950852226 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe4e41141d0 0x7fe4e41a4b50 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:25.171 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:25.171+0000 7fe4e8f7d700 1 --2- 172.21.15.82:0/3950852226 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe4e4076440 0x7fe4e41a08c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:25.171 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:25.171+0000 7fe4e8f7d700 1 -- 172.21.15.82:0/3950852226 >> 172.21.15.82:0/3950852226 conn(0x7fe4e40fed20 msgr2=0x7fe4e4100be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:12:25.171 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:25.171+0000 7fe4e8f7d700 1 -- 172.21.15.82:0/3950852226 shutdown_connections 2024-05-20T16:12:25.171 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:25.171+0000 7fe4e8f7d700 1 -- 172.21.15.82:0/3950852226 wait complete. 2024-05-20T16:12:25.562 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:12:25 smithi160 ceph-mon[36347]: from='client.? 172.21.15.82:0/3950852226' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-20T16:12:25.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:12:25 smithi082 ceph-mon[32238]: from='client.? 172.21.15.82:0/3950852226' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-20T16:12:25.635 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:12:25 smithi184 ceph-mon[36209]: from='client.? 172.21.15.82:0/3950852226' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-20T16:12:25.913 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-05-20T16:12:25.914 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell --fsid 59d1600a-16c2-11ef-bc9a-c7b262605968 -- ceph osd dump --format=json 2024-05-20T16:12:26.161 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T16:12:26.524 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:12:26 smithi082 ceph-mon[32238]: pgmap v311: 129 pgs: 129 active+clean; 625 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 6.0 KiB/s wr, 14 op/s 2024-05-20T16:12:26.562 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:12:26 smithi160 ceph-mon[36347]: pgmap v311: 129 pgs: 129 active+clean; 625 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 6.0 KiB/s wr, 14 op/s 2024-05-20T16:12:26.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:12:26 smithi184 ceph-mon[36209]: pgmap v311: 129 pgs: 129 active+clean; 625 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 6.0 KiB/s wr, 14 op/s 2024-05-20T16:12:27.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:12:27 smithi082 ceph-mon[32238]: pgmap v312: 129 pgs: 129 active+clean; 625 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 4.3 KiB/s wr, 9 op/s 2024-05-20T16:12:27.635 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:12:27 smithi184 ceph-mon[36209]: pgmap v312: 129 pgs: 129 active+clean; 625 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 4.3 KiB/s wr, 9 op/s 2024-05-20T16:12:27.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:12:27 smithi160 ceph-mon[36347]: pgmap v312: 129 pgs: 129 active+clean; 625 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 4.3 KiB/s wr, 9 op/s 2024-05-20T16:12:27.964 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:27.964+0000 7fe7b4928700 1 -- 172.21.15.82:0/744203455 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe7b0107600 msgr2=0x7fe7b010ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:12:27.965 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:27.964+0000 7fe7b4928700 1 --2- 172.21.15.82:0/744203455 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe7b0107600 0x7fe7b010ef60 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fe7a800b0d0 tx=0x7fe7a800b3e0 comp rx=0 tx=0).stop 2024-05-20T16:12:27.965 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:27.964+0000 7fe7b4928700 1 -- 172.21.15.82:0/744203455 shutdown_connections 2024-05-20T16:12:27.965 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:27.964+0000 7fe7b4928700 1 --2- 172.21.15.82:0/744203455 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe7b01023d0 0x7fe7b01027e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:27.965 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:27.964+0000 7fe7b4928700 1 --2- 172.21.15.82:0/744203455 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe7b0107600 0x7fe7b010ef60 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:27.965 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:27.964+0000 7fe7b4928700 1 --2- 172.21.15.82:0/744203455 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe7b0106cb0 0x7fe7b01070c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:27.965 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:27.964+0000 7fe7b4928700 1 -- 172.21.15.82:0/744203455 >> 172.21.15.82:0/744203455 conn(0x7fe7b0077c80 msgr2=0x7fe7b00780c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:12:27.965 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:27.964+0000 7fe7b4928700 1 -- 172.21.15.82:0/744203455 shutdown_connections 2024-05-20T16:12:27.965 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:27.964+0000 7fe7b4928700 1 -- 172.21.15.82:0/744203455 wait complete. 2024-05-20T16:12:27.967 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:27.965+0000 7fe7b4928700 1 Processor -- start 2024-05-20T16:12:27.967 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:27.965+0000 7fe7b4928700 1 -- start start 2024-05-20T16:12:27.967 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:27.965+0000 7fe7b4928700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe7b01023d0 0x7fe7b019ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:12:27.967 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:27.966+0000 7fe7b4928700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe7b0106cb0 0x7fe7b019b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:12:27.967 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:27.966+0000 7fe7b4928700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe7b0107600 0x7fe7b01a5c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:12:27.967 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:27.966+0000 7fe7b4928700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fe7b01123f0 con 0x7fe7b0107600 2024-05-20T16:12:27.967 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:27.966+0000 7fe7b4928700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fe7b0112270 con 0x7fe7b0106cb0 2024-05-20T16:12:27.967 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:27.966+0000 7fe7b4928700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fe7b0112570 con 0x7fe7b01023d0 2024-05-20T16:12:27.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:27.966+0000 7fe7aed9d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe7b0107600 0x7fe7b01a5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:12:27.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:27.966+0000 7fe7aed9d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe7b0107600 0x7fe7b01a5c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:39942/0 (socket says 172.21.15.82:39942) 2024-05-20T16:12:27.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:27.966+0000 7fe7aed9d700 1 -- 172.21.15.82:0/2324164372 learned_addr learned my addr 172.21.15.82:0/2324164372 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:12:27.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:27.966+0000 7fe7a5d9b700 1 --2- 172.21.15.82:0/2324164372 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe7b0106cb0 0x7fe7b019b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:12:27.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:27.966+0000 7fe7ae59c700 1 --2- 172.21.15.82:0/2324164372 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe7b01023d0 0x7fe7b019ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:12:27.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:27.966+0000 7fe7aed9d700 1 -- 172.21.15.82:0/2324164372 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe7b01023d0 msgr2=0x7fe7b019ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:12:27.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:27.966+0000 7fe7aed9d700 1 --2- 172.21.15.82:0/2324164372 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe7b01023d0 0x7fe7b019ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:27.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:27.966+0000 7fe7aed9d700 1 -- 172.21.15.82:0/2324164372 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe7b0106cb0 msgr2=0x7fe7b019b0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:12:27.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:27.966+0000 7fe7aed9d700 1 --2- 172.21.15.82:0/2324164372 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe7b0106cb0 0x7fe7b019b0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:27.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:27.966+0000 7fe7aed9d700 1 -- 172.21.15.82:0/2324164372 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7b019b8f0 con 0x7fe7b0107600 2024-05-20T16:12:27.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:27.966+0000 7fe7aed9d700 1 --2- 172.21.15.82:0/2324164372 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe7b0107600 0x7fe7b01a5c10 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fe7a8006070 tx=0x7fe7a8004e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:12:27.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:27.967+0000 7fe7a7fff700 1 -- 172.21.15.82:0/2324164372 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7a8021070 con 0x7fe7b0107600 2024-05-20T16:12:27.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:27.967+0000 7fe7a7fff700 1 -- 172.21.15.82:0/2324164372 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe7a8007d00 con 0x7fe7b0107600 2024-05-20T16:12:27.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:27.967+0000 7fe7b4928700 1 -- 172.21.15.82:0/2324164372 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7b01a6150 con 0x7fe7b0107600 2024-05-20T16:12:27.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:27.967+0000 7fe7a7fff700 1 -- 172.21.15.82:0/2324164372 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7a8005470 con 0x7fe7b0107600 2024-05-20T16:12:27.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:27.967+0000 7fe7b4928700 1 -- 172.21.15.82:0/2324164372 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7b01a6600 con 0x7fe7b0107600 2024-05-20T16:12:27.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:27.968+0000 7fe7b4928700 1 -- 172.21.15.82:0/2324164372 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7b0050170 con 0x7fe7b0107600 2024-05-20T16:12:27.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:27.969+0000 7fe7a7fff700 1 -- 172.21.15.82:0/2324164372 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fe7a8024070 con 0x7fe7b0107600 2024-05-20T16:12:27.970 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:27.970+0000 7fe7a7fff700 1 --2- 172.21.15.82:0/2324164372 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fe79c0a2cb0 0x7fe79c0a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:12:27.970 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:27.970+0000 7fe7ae59c700 1 --2- 172.21.15.82:0/2324164372 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fe79c0a2cb0 0x7fe79c0a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:12:27.970 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:27.970+0000 7fe7a7fff700 1 -- 172.21.15.82:0/2324164372 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8802+0+0 (secure 0 0 0) 0x7fe7a800f020 con 0x7fe7b0107600 2024-05-20T16:12:27.971 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:27.971+0000 7fe7ae59c700 1 --2- 172.21.15.82:0/2324164372 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fe79c0a2cb0 0x7fe79c0a51c0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fe7a0005ea0 tx=0x7fe7a0005e30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:12:27.974 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:27.974+0000 7fe7a7fff700 1 -- 172.21.15.82:0/2324164372 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fe7a802b020 con 0x7fe7b0107600 2024-05-20T16:12:28.169 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:28.169+0000 7fe7b4928700 1 -- 172.21.15.82:0/2324164372 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe7b01a67d0 con 0x7fe7b0107600 2024-05-20T16:12:28.171 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:28.171+0000 7fe7a7fff700 1 -- 172.21.15.82:0/2324164372 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+24153 (secure 0 0 0) 0x7fe7a809dbd0 con 0x7fe7b0107600 2024-05-20T16:12:28.171 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-05-20T16:12:28.172 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":91,"fsid":"59d1600a-16c2-11ef-bc9a-c7b262605968","created":"2024-05-20T16:03:46.611556+0000","modified":"2024-05-20T16:11:53.416085+0000","last_up_change":"2024-05-20T16:10:59.303479+0000","last_in_change":"2024-05-20T16:10:39.063561+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-20T16:07:09.493727+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-20T16:11:45.058035+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-20T16:11:45.943862+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-20T16:11:49.637579+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"bae842a9-03ec-490e-b56d-f7957235cdc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":1484330167},{"type":"v1","addr":"172.21.15.82:6803","nonce":1484330167}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":1484330167},{"type":"v1","addr":"172.21.15.82:6805","nonce":1484330167}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1484330167},{"type":"v1","addr":"172.21.15.82:6809","nonce":1484330167}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":1484330167},{"type":"v1","addr":"172.21.15.82:6807","nonce":1484330167}]},"public_addr":"172.21.15.82:6803/1484330167","cluster_addr":"172.21.15.82:6805/1484330167","heartbeat_back_addr":"172.21.15.82:6809/1484330167","heartbeat_front_addr":"172.21.15.82:6807/1484330167","state":["exists","up"]},{"osd":1,"uuid":"59f6a2ba-b5de-4161-81c5-92a417abacdf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":1457003568},{"type":"v1","addr":"172.21.15.160:6801","nonce":1457003568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":1457003568},{"type":"v1","addr":"172.21.15.160:6803","nonce":1457003568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6806","nonce":1457003568},{"type":"v1","addr":"172.21.15.160:6807","nonce":1457003568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":1457003568},{"type":"v1","addr":"172.21.15.160:6805","nonce":1457003568}]},"public_addr":"172.21.15.160:6801/1457003568","cluster_addr":"172.21.15.160:6803/1457003568","heartbeat_back_addr":"172.21.15.160:6807/1457003568","heartbeat_front_addr":"172.21.15.160:6805/1457003568","state":["exists","up"]},{"osd":2,"uuid":"3bbb5587-b449-41f3-bd26-67ec3d6afdfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":4051293934},{"type":"v1","addr":"172.21.15.184:6801","nonce":4051293934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":4051293934},{"type":"v1","addr":"172.21.15.184:6803","nonce":4051293934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":4051293934},{"type":"v1","addr":"172.21.15.184:6807","nonce":4051293934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":4051293934},{"type":"v1","addr":"172.21.15.184:6805","nonce":4051293934}]},"public_addr":"172.21.15.184:6801/4051293934","cluster_addr":"172.21.15.184:6803/4051293934","heartbeat_back_addr":"172.21.15.184:6807/4051293934","heartbeat_front_addr":"172.21.15.184:6805/4051293934","state":["exists","up"]},{"osd":3,"uuid":"ca651f90-09b8-4081-a6f9-40b31268a837","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":3947485599},{"type":"v1","addr":"172.21.15.82:6811","nonce":3947485599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":3947485599},{"type":"v1","addr":"172.21.15.82:6813","nonce":3947485599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":3947485599},{"type":"v1","addr":"172.21.15.82:6817","nonce":3947485599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":3947485599},{"type":"v1","addr":"172.21.15.82:6815","nonce":3947485599}]},"public_addr":"172.21.15.82:6811/3947485599","cluster_addr":"172.21.15.82:6813/3947485599","heartbeat_back_addr":"172.21.15.82:6817/3947485599","heartbeat_front_addr":"172.21.15.82:6815/3947485599","state":["exists","up"]},{"osd":4,"uuid":"cb3dfd56-f677-40cb-9c96-8319ea5ac788","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":4160830014},{"type":"v1","addr":"172.21.15.160:6809","nonce":4160830014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":4160830014},{"type":"v1","addr":"172.21.15.160:6811","nonce":4160830014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":4160830014},{"type":"v1","addr":"172.21.15.160:6815","nonce":4160830014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6812","nonce":4160830014},{"type":"v1","addr":"172.21.15.160:6813","nonce":4160830014}]},"public_addr":"172.21.15.160:6809/4160830014","cluster_addr":"172.21.15.160:6811/4160830014","heartbeat_back_addr":"172.21.15.160:6815/4160830014","heartbeat_front_addr":"172.21.15.160:6813/4160830014","state":["exists","up"]},{"osd":5,"uuid":"c70cfcd1-a4ff-461f-8c73-479b061200fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":1046282672},{"type":"v1","addr":"172.21.15.184:6809","nonce":1046282672}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":1046282672},{"type":"v1","addr":"172.21.15.184:6811","nonce":1046282672}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":1046282672},{"type":"v1","addr":"172.21.15.184:6815","nonce":1046282672}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":1046282672},{"type":"v1","addr":"172.21.15.184:6813","nonce":1046282672}]},"public_addr":"172.21.15.184:6809/1046282672","cluster_addr":"172.21.15.184:6811/1046282672","heartbeat_back_addr":"172.21.15.184:6815/1046282672","heartbeat_front_addr":"172.21.15.184:6813/1046282672","state":["exists","up"]},{"osd":6,"uuid":"d47ce001-9dd1-4830-bcc9-888601187c6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":2789736499},{"type":"v1","addr":"172.21.15.82:6819","nonce":2789736499}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":2789736499},{"type":"v1","addr":"172.21.15.82:6821","nonce":2789736499}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":2789736499},{"type":"v1","addr":"172.21.15.82:6825","nonce":2789736499}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":2789736499},{"type":"v1","addr":"172.21.15.82:6823","nonce":2789736499}]},"public_addr":"172.21.15.82:6819/2789736499","cluster_addr":"172.21.15.82:6821/2789736499","heartbeat_back_addr":"172.21.15.82:6825/2789736499","heartbeat_front_addr":"172.21.15.82:6823/2789736499","state":["exists","up"]},{"osd":7,"uuid":"25c54f43-18cb-41e4-83b4-2732b5a93889","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6816","nonce":2340683378},{"type":"v1","addr":"172.21.15.160:6817","nonce":2340683378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":2340683378},{"type":"v1","addr":"172.21.15.160:6819","nonce":2340683378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":2340683378},{"type":"v1","addr":"172.21.15.160:6823","nonce":2340683378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6820","nonce":2340683378},{"type":"v1","addr":"172.21.15.160:6821","nonce":2340683378}]},"public_addr":"172.21.15.160:6817/2340683378","cluster_addr":"172.21.15.160:6819/2340683378","heartbeat_back_addr":"172.21.15.160:6823/2340683378","heartbeat_front_addr":"172.21.15.160:6821/2340683378","state":["exists","up"]},{"osd":8,"uuid":"1f8b478d-287f-45f5-8526-d1a58c5bc07c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":3831576372},{"type":"v1","addr":"172.21.15.184:6817","nonce":3831576372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":3831576372},{"type":"v1","addr":"172.21.15.184:6819","nonce":3831576372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":3831576372},{"type":"v1","addr":"172.21.15.184:6823","nonce":3831576372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":3831576372},{"type":"v1","addr":"172.21.15.184:6821","nonce":3831576372}]},"public_addr":"172.21.15.184:6817/3831576372","cluster_addr":"172.21.15.184:6819/3831576372","heartbeat_back_addr":"172.21.15.184:6823/3831576372","heartbeat_front_addr":"172.21.15.184:6821/3831576372","state":["exists","up"]},{"osd":9,"uuid":"0d69e319-954f-4b18-95ea-31794f18dc3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":716330782},{"type":"v1","addr":"172.21.15.82:6827","nonce":716330782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":716330782},{"type":"v1","addr":"172.21.15.82:6829","nonce":716330782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":716330782},{"type":"v1","addr":"172.21.15.82:6833","nonce":716330782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":716330782},{"type":"v1","addr":"172.21.15.82:6831","nonce":716330782}]},"public_addr":"172.21.15.82:6827/716330782","cluster_addr":"172.21.15.82:6829/716330782","heartbeat_back_addr":"172.21.15.82:6833/716330782","heartbeat_front_addr":"172.21.15.82:6831/716330782","state":["exists","up"]},{"osd":10,"uuid":"39dc1b78-61cb-42ca-b5b6-1e44ac2b6500","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":2327428014},{"type":"v1","addr":"172.21.15.160:6825","nonce":2327428014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":2327428014},{"type":"v1","addr":"172.21.15.160:6827","nonce":2327428014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":2327428014},{"type":"v1","addr":"172.21.15.160:6831","nonce":2327428014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":2327428014},{"type":"v1","addr":"172.21.15.160:6829","nonce":2327428014}]},"public_addr":"172.21.15.160:6825/2327428014","cluster_addr":"172.21.15.160:6827/2327428014","heartbeat_back_addr":"172.21.15.160:6831/2327428014","heartbeat_front_addr":"172.21.15.160:6829/2327428014","state":["exists","up"]},{"osd":11,"uuid":"4306b3c1-707a-4010-987c-209ea0e948e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":3594364650},{"type":"v1","addr":"172.21.15.184:6825","nonce":3594364650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":3594364650},{"type":"v1","addr":"172.21.15.184:6827","nonce":3594364650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3594364650},{"type":"v1","addr":"172.21.15.184:6831","nonce":3594364650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3594364650},{"type":"v1","addr":"172.21.15.184:6829","nonce":3594364650}]},"public_addr":"172.21.15.184:6825/3594364650","cluster_addr":"172.21.15.184:6827/3594364650","heartbeat_back_addr":"172.21.15.184:6831/3594364650","heartbeat_front_addr":"172.21.15.184:6829/3594364650","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:06:17.855998+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:06:41.473270+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:07:05.304108+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:07:29.887472+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:07:55.387286+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:08:20.579669+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:08:46.333051+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:09:12.289668+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:09:38.180696+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:10:04.471459+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:10:31.041912+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:10:57.477266+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.82:0/2005424509":"2024-05-21T16:04:45.161949+0000","172.21.15.82:6801/1763049308":"2024-05-21T16:04:45.161949+0000","172.21.15.82:6801/1959459348":"2024-05-21T16:04:19.208291+0000","172.21.15.82:0/3131904556":"2024-05-21T16:04:19.208291+0000","172.21.15.82:6800/1959459348":"2024-05-21T16:04:19.208291+0000","172.21.15.82:0/2456966342":"2024-05-21T16:04:45.161949+0000","172.21.15.82:0/4179978833":"2024-05-21T16:04:19.208291+0000","172.21.15.82:0/1602862225":"2024-05-21T16:04:45.161949+0000","172.21.15.82:0/106101180":"2024-05-21T16:04:19.208291+0000","172.21.15.82:6800/1763049308":"2024-05-21T16:04:45.161949+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-20T16:12:28.173 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:28.173+0000 7fe7b4928700 1 -- 172.21.15.82:0/2324164372 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fe79c0a2cb0 msgr2=0x7fe79c0a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:12:28.173 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:28.173+0000 7fe7b4928700 1 --2- 172.21.15.82:0/2324164372 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fe79c0a2cb0 0x7fe79c0a51c0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fe7a0005ea0 tx=0x7fe7a0005e30 comp rx=0 tx=0).stop 2024-05-20T16:12:28.173 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:28.173+0000 7fe7b4928700 1 -- 172.21.15.82:0/2324164372 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe7b0107600 msgr2=0x7fe7b01a5c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:12:28.173 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:28.173+0000 7fe7b4928700 1 --2- 172.21.15.82:0/2324164372 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe7b0107600 0x7fe7b01a5c10 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fe7a8006070 tx=0x7fe7a8004e40 comp rx=0 tx=0).stop 2024-05-20T16:12:28.174 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:28.173+0000 7fe7b4928700 1 -- 172.21.15.82:0/2324164372 shutdown_connections 2024-05-20T16:12:28.174 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:28.173+0000 7fe7b4928700 1 --2- 172.21.15.82:0/2324164372 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fe79c0a2cb0 0x7fe79c0a51c0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:28.174 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:28.173+0000 7fe7b4928700 1 --2- 172.21.15.82:0/2324164372 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe7b0107600 0x7fe7b01a5c10 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:28.174 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:28.173+0000 7fe7b4928700 1 --2- 172.21.15.82:0/2324164372 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe7b01023d0 0x7fe7b019ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:28.174 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:28.173+0000 7fe7b4928700 1 --2- 172.21.15.82:0/2324164372 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe7b0106cb0 0x7fe7b019b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:28.174 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:28.173+0000 7fe7b4928700 1 -- 172.21.15.82:0/2324164372 >> 172.21.15.82:0/2324164372 conn(0x7fe7b0077c80 msgr2=0x7fe7b0100bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:12:28.174 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:28.174+0000 7fe7b4928700 1 -- 172.21.15.82:0/2324164372 shutdown_connections 2024-05-20T16:12:28.174 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:28.174+0000 7fe7b4928700 1 -- 172.21.15.82:0/2324164372 wait complete. 2024-05-20T16:12:28.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:12:28 smithi082 ceph-mon[32238]: from='client.? 172.21.15.82:0/2324164372' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-20T16:12:28.635 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:12:28 smithi184 ceph-mon[36209]: from='client.? 172.21.15.82:0/2324164372' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-20T16:12:28.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:12:28 smithi160 ceph-mon[36347]: from='client.? 172.21.15.82:0/2324164372' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-20T16:12:28.968 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell --fsid 59d1600a-16c2-11ef-bc9a-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-05-20T16:12:28.968 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell --fsid 59d1600a-16c2-11ef-bc9a-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-05-20T16:12:28.969 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell --fsid 59d1600a-16c2-11ef-bc9a-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-05-20T16:12:28.969 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell --fsid 59d1600a-16c2-11ef-bc9a-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-05-20T16:12:28.970 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell --fsid 59d1600a-16c2-11ef-bc9a-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-05-20T16:12:28.970 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell --fsid 59d1600a-16c2-11ef-bc9a-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-05-20T16:12:28.971 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell --fsid 59d1600a-16c2-11ef-bc9a-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-05-20T16:12:28.971 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell --fsid 59d1600a-16c2-11ef-bc9a-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-05-20T16:12:28.972 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell --fsid 59d1600a-16c2-11ef-bc9a-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-05-20T16:12:28.972 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell --fsid 59d1600a-16c2-11ef-bc9a-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-05-20T16:12:28.972 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell --fsid 59d1600a-16c2-11ef-bc9a-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-05-20T16:12:28.973 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell --fsid 59d1600a-16c2-11ef-bc9a-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-05-20T16:12:29.635 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:12:29 smithi184 ceph-mon[36209]: pgmap v313: 129 pgs: 129 active+clean; 640 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.6 KiB/s wr, 11 op/s 2024-05-20T16:12:29.658 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T16:12:29.659 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T16:12:29.702 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T16:12:29.778 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T16:12:29.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:12:29 smithi160 ceph-mon[36347]: pgmap v313: 129 pgs: 129 active+clean; 640 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.6 KiB/s wr, 11 op/s 2024-05-20T16:12:29.842 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:12:29 smithi082 ceph-mon[32238]: pgmap v313: 129 pgs: 129 active+clean; 640 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.6 KiB/s wr, 11 op/s 2024-05-20T16:12:30.012 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T16:12:30.845 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T16:12:30.845 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T16:12:30.845 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T16:12:31.505 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:12:31 smithi082 ceph-mon[32238]: pgmap v314: 129 pgs: 129 active+clean; 640 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 2.1 KiB/s wr, 6 op/s 2024-05-20T16:12:31.635 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:12:31 smithi184 ceph-mon[36209]: pgmap v314: 129 pgs: 129 active+clean; 640 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 2.1 KiB/s wr, 6 op/s 2024-05-20T16:12:31.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:12:31 smithi160 ceph-mon[36347]: pgmap v314: 129 pgs: 129 active+clean; 640 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 2.1 KiB/s wr, 6 op/s 2024-05-20T16:12:32.778 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T16:12:32.778 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T16:12:33.609 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T16:12:33.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:12:33 smithi082 ceph-mon[32238]: pgmap v315: 129 pgs: 129 active+clean; 640 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 2.1 KiB/s wr, 7 op/s 2024-05-20T16:12:33.610 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T16:12:33.635 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:12:33 smithi184 ceph-mon[36209]: pgmap v315: 129 pgs: 129 active+clean; 640 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 2.1 KiB/s wr, 7 op/s 2024-05-20T16:12:33.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:12:33 smithi160 ceph-mon[36347]: pgmap v315: 129 pgs: 129 active+clean; 640 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 2.1 KiB/s wr, 7 op/s 2024-05-20T16:12:35.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:12:35 smithi082 ceph-mon[32238]: pgmap v316: 129 pgs: 129 active+clean; 640 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.4 KiB/s wr, 6 op/s 2024-05-20T16:12:35.635 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:12:35 smithi184 ceph-mon[36209]: pgmap v316: 129 pgs: 129 active+clean; 640 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.4 KiB/s wr, 6 op/s 2024-05-20T16:12:35.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:12:35 smithi160 ceph-mon[36347]: pgmap v316: 129 pgs: 129 active+clean; 640 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.4 KiB/s wr, 6 op/s 2024-05-20T16:12:37.026 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:37.006+0000 7f8a5825b700 1 -- 172.21.15.82:0/1347736186 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8a50107da0 msgr2=0x7f8a501081b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:12:37.026 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:37.006+0000 7f8a5825b700 1 --2- 172.21.15.82:0/1347736186 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8a50107da0 0x7f8a501081b0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f8a400099c0 tx=0x7f8a40009cd0 comp rx=0 tx=0).stop 2024-05-20T16:12:37.026 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:37.006+0000 7f8a5825b700 1 -- 172.21.15.82:0/1347736186 shutdown_connections 2024-05-20T16:12:37.026 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:37.006+0000 7f8a5825b700 1 --2- 172.21.15.82:0/1347736186 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8a501086f0 0x7f8a50110050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:37.026 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:37.006+0000 7f8a5825b700 1 --2- 172.21.15.82:0/1347736186 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8a50103470 0x7f8a50103880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:37.026 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:37.006+0000 7f8a5825b700 1 --2- 172.21.15.82:0/1347736186 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8a50107da0 0x7f8a501081b0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:37.026 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:37.006+0000 7f8a5825b700 1 -- 172.21.15.82:0/1347736186 >> 172.21.15.82:0/1347736186 conn(0x7f8a500feec0 msgr2=0x7f8a501012e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:12:37.026 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:37.006+0000 7f8a5825b700 1 -- 172.21.15.82:0/1347736186 shutdown_connections 2024-05-20T16:12:37.026 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:37.006+0000 7f8a5825b700 1 -- 172.21.15.82:0/1347736186 wait complete. 2024-05-20T16:12:37.027 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:37.007+0000 7f8a5825b700 1 Processor -- start 2024-05-20T16:12:37.027 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:37.007+0000 7f8a5825b700 1 -- start start 2024-05-20T16:12:37.027 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:37.008+0000 7f8a5825b700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8a50103470 0x7f8a50199de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:12:37.027 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:37.008+0000 7f8a5825b700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8a50107da0 0x7f8a5019a380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:12:37.027 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:37.008+0000 7f8a5825b700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8a501086f0 0x7f8a5019e610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:12:37.027 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:37.008+0000 7f8a5825b700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f8a50113450 con 0x7f8a50103470 2024-05-20T16:12:37.027 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:37.008+0000 7f8a5825b700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f8a501132d0 con 0x7f8a501086f0 2024-05-20T16:12:37.027 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:37.008+0000 7f8a5825b700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f8a501135d0 con 0x7f8a50107da0 2024-05-20T16:12:37.027 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:37.008+0000 7f8a55ff7700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8a50103470 0x7f8a50199de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:12:37.027 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:37.008+0000 7f8a55ff7700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8a50103470 0x7f8a50199de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41502/0 (socket says 172.21.15.82:41502) 2024-05-20T16:12:37.027 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:37.008+0000 7f8a55ff7700 1 -- 172.21.15.82:0/3222045842 learned_addr learned my addr 172.21.15.82:0/3222045842 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:12:37.027 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:37.008+0000 7f8a557f6700 1 --2- 172.21.15.82:0/3222045842 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8a50107da0 0x7f8a5019a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:12:37.027 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:37.008+0000 7f8a55ff7700 1 -- 172.21.15.82:0/3222045842 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8a50107da0 msgr2=0x7f8a5019a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:12:37.028 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:37.008+0000 7f8a55ff7700 1 --2- 172.21.15.82:0/3222045842 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8a50107da0 0x7f8a5019a380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:37.028 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:37.008+0000 7f8a55ff7700 1 -- 172.21.15.82:0/3222045842 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8a501086f0 msgr2=0x7f8a5019e610 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-20T16:12:37.028 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:37.008+0000 7f8a55ff7700 1 --2- 172.21.15.82:0/3222045842 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8a501086f0 0x7f8a5019e610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:37.028 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:37.008+0000 7f8a55ff7700 1 -- 172.21.15.82:0/3222045842 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a5019ec00 con 0x7f8a50103470 2024-05-20T16:12:37.028 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:37.008+0000 7f8a557f6700 1 --2- 172.21.15.82:0/3222045842 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8a50107da0 0x7f8a5019a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T16:12:37.028 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:37.008+0000 7f8a55ff7700 1 --2- 172.21.15.82:0/3222045842 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8a50103470 0x7f8a50199de0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f8a4c0060b0 tx=0x7f8a4c007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:12:37.028 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:37.008+0000 7f8a46ffd700 1 -- 172.21.15.82:0/3222045842 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a4c01a070 con 0x7f8a50103470 2024-05-20T16:12:37.028 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:37.008+0000 7f8a46ffd700 1 -- 172.21.15.82:0/3222045842 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a4c00de20 con 0x7f8a50103470 2024-05-20T16:12:37.028 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:37.008+0000 7f8a5825b700 1 -- 172.21.15.82:0/3222045842 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a5019eef0 con 0x7f8a50103470 2024-05-20T16:12:37.028 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:37.009+0000 7f8a46ffd700 1 -- 172.21.15.82:0/3222045842 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a4c014790 con 0x7f8a50103470 2024-05-20T16:12:37.028 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:37.009+0000 7f8a5825b700 1 -- 172.21.15.82:0/3222045842 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a5019f450 con 0x7f8a50103470 2024-05-20T16:12:37.028 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:37.009+0000 7f8a5825b700 1 -- 172.21.15.82:0/3222045842 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f8a50062d90 con 0x7f8a50103470 2024-05-20T16:12:37.028 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:37.011+0000 7f8a46ffd700 1 -- 172.21.15.82:0/3222045842 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f8a4c01d070 con 0x7f8a50103470 2024-05-20T16:12:37.028 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:37.011+0000 7f8a46ffd700 1 --2- 172.21.15.82:0/3222045842 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f8a3c0a2d30 0x7f8a3c0a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:12:37.029 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:37.011+0000 7f8a557f6700 1 --2- 172.21.15.82:0/3222045842 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f8a3c0a2d30 0x7f8a3c0a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:12:37.029 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:37.012+0000 7f8a46ffd700 1 -- 172.21.15.82:0/3222045842 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8802+0+0 (secure 0 0 0) 0x7f8a4c0cf440 con 0x7f8a50103470 2024-05-20T16:12:37.029 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:37.012+0000 7f8a46ffd700 1 --2- 172.21.15.82:0/3222045842 >> [v2:172.21.15.82:6826/716330782,v1:172.21.15.82:6827/716330782] conn(0x7f8a3c0ada40 0x7f8a3c0afed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:12:37.029 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:37.012+0000 7f8a46ffd700 1 -- 172.21.15.82:0/3222045842 --> [v2:172.21.15.82:6826/716330782,v1:172.21.15.82:6827/716330782] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8a3c0b05a0 con 0x7f8a3c0ada40 2024-05-20T16:12:37.029 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:37.012+0000 7f8a46ffd700 1 -- 172.21.15.82:0/3222045842 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7f8a4c0cf890 con 0x7f8a50103470 2024-05-20T16:12:37.029 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:37.012+0000 7f8a567f8700 1 --2- 172.21.15.82:0/3222045842 >> [v2:172.21.15.82:6826/716330782,v1:172.21.15.82:6827/716330782] conn(0x7f8a3c0ada40 0x7f8a3c0afed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:12:37.029 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:37.012+0000 7f8a557f6700 1 --2- 172.21.15.82:0/3222045842 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f8a3c0a2d30 0x7f8a3c0a5240 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f8a40000c00 tx=0x7f8a4001a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:12:37.029 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:37.012+0000 7f8a567f8700 1 --2- 172.21.15.82:0/3222045842 >> [v2:172.21.15.82:6826/716330782,v1:172.21.15.82:6827/716330782] conn(0x7f8a3c0ada40 0x7f8a3c0afed0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:12:37.029 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:37.013+0000 7f8a46ffd700 1 -- 172.21.15.82:0/3222045842 <== osd.9 v2:172.21.15.82:6826/716330782 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8a3c0b05a0 con 0x7f8a3c0ada40 2024-05-20T16:12:37.030 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:37.029+0000 7f8a5825b700 1 -- 172.21.15.82:0/3222045842 --> [v2:172.21.15.82:6826/716330782,v1:172.21.15.82:6827/716330782] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f8a50063210 con 0x7f8a3c0ada40 2024-05-20T16:12:37.030 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:37.029+0000 7f8a46ffd700 1 -- 172.21.15.82:0/3222045842 <== osd.9 v2:172.21.15.82:6826/716330782 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f8a50063210 con 0x7f8a3c0ada40 2024-05-20T16:12:37.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:37.030+0000 7f8a5825b700 1 -- 172.21.15.82:0/3222045842 >> [v2:172.21.15.82:6826/716330782,v1:172.21.15.82:6827/716330782] conn(0x7f8a3c0ada40 msgr2=0x7f8a3c0afed0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:12:37.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:37.030+0000 7f8a5825b700 1 --2- 172.21.15.82:0/3222045842 >> [v2:172.21.15.82:6826/716330782,v1:172.21.15.82:6827/716330782] conn(0x7f8a3c0ada40 0x7f8a3c0afed0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:37.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:37.030+0000 7f8a5825b700 1 -- 172.21.15.82:0/3222045842 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f8a3c0a2d30 msgr2=0x7f8a3c0a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:12:37.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:37.030+0000 7f8a5825b700 1 --2- 172.21.15.82:0/3222045842 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f8a3c0a2d30 0x7f8a3c0a5240 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f8a40000c00 tx=0x7f8a4001a040 comp rx=0 tx=0).stop 2024-05-20T16:12:37.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:37.030+0000 7f8a5825b700 1 -- 172.21.15.82:0/3222045842 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8a50103470 msgr2=0x7f8a50199de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:12:37.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:37.030+0000 7f8a5825b700 1 --2- 172.21.15.82:0/3222045842 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8a50103470 0x7f8a50199de0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f8a4c0060b0 tx=0x7f8a4c007f90 comp rx=0 tx=0).stop 2024-05-20T16:12:37.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:37.030+0000 7f8a567f8700 1 -- 172.21.15.82:0/3222045842 reap_dead start 2024-05-20T16:12:37.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:37.030+0000 7f8a5825b700 1 -- 172.21.15.82:0/3222045842 shutdown_connections 2024-05-20T16:12:37.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:37.030+0000 7f8a5825b700 1 -- 172.21.15.82:0/3222045842 >> 172.21.15.82:0/3222045842 conn(0x7f8a500feec0 msgr2=0x7f8a50107510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:12:37.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:37.030+0000 7f8a5825b700 1 -- 172.21.15.82:0/3222045842 shutdown_connections 2024-05-20T16:12:37.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:37.030+0000 7f8a5825b700 1 -- 172.21.15.82:0/3222045842 wait complete. 2024-05-20T16:12:37.344 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:37.344+0000 7f5dbeca2700 1 -- 172.21.15.82:0/770983494 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5db8101f60 msgr2=0x7f5db810fff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:12:37.344 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:37.344+0000 7f5dbeca2700 1 --2- 172.21.15.82:0/770983494 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5db8101f60 0x7f5db810fff0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f5db000b0a0 tx=0x7f5db000b3b0 comp rx=0 tx=0).stop 2024-05-20T16:12:37.344 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:37.344+0000 7f5dbeca2700 1 -- 172.21.15.82:0/770983494 shutdown_connections 2024-05-20T16:12:37.345 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:37.344+0000 7f5dbeca2700 1 --2- 172.21.15.82:0/770983494 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5db8101f60 0x7f5db810fff0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:37.345 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:37.344+0000 7f5dbeca2700 1 --2- 172.21.15.82:0/770983494 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5db8108180 0x7f5db8108590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:37.345 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:37.344+0000 7f5dbeca2700 1 --2- 172.21.15.82:0/770983494 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5db8101610 0x7f5db8101a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:37.345 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:37.344+0000 7f5dbeca2700 1 -- 172.21.15.82:0/770983494 >> 172.21.15.82:0/770983494 conn(0x7f5db80fd3a0 msgr2=0x7f5db80ff7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:12:37.345 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:37.344+0000 7f5dbeca2700 1 -- 172.21.15.82:0/770983494 shutdown_connections 2024-05-20T16:12:37.345 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:37.344+0000 7f5dbeca2700 1 -- 172.21.15.82:0/770983494 wait complete. 2024-05-20T16:12:37.346 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:37.345+0000 7f5dbeca2700 1 Processor -- start 2024-05-20T16:12:37.347 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:37.345+0000 7f5dbeca2700 1 -- start start 2024-05-20T16:12:37.347 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:37.345+0000 7f5dbeca2700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5db8101610 0x7f5db81027f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:12:37.347 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:37.345+0000 7f5dbeca2700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5db8101f60 0x7f5db8102d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:12:37.347 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:37.345+0000 7f5dbeca2700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5db8108180 0x7f5db8103270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:12:37.347 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:37.345+0000 7f5dbeca2700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f5db8113570 con 0x7f5db8108180 2024-05-20T16:12:37.347 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:37.345+0000 7f5dbeca2700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f5db81133f0 con 0x7f5db8101610 2024-05-20T16:12:37.347 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:37.345+0000 7f5dbeca2700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f5db81136f0 con 0x7f5db8101f60 2024-05-20T16:12:37.347 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:37.345+0000 7f5dbd23f700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5db8108180 0x7f5db8103270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:12:37.347 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:37.345+0000 7f5dbd23f700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5db8108180 0x7f5db8103270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41510/0 (socket says 172.21.15.82:41510) 2024-05-20T16:12:37.347 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:37.346+0000 7f5dbd23f700 1 -- 172.21.15.82:0/1841017413 learned_addr learned my addr 172.21.15.82:0/1841017413 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:12:37.347 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:37.346+0000 7f5dbd23f700 1 -- 172.21.15.82:0/1841017413 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5db8101f60 msgr2=0x7f5db8102d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:12:37.347 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:37.346+0000 7f5dbd23f700 1 --2- 172.21.15.82:0/1841017413 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5db8101f60 0x7f5db8102d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:37.347 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:37.346+0000 7f5dbd23f700 1 -- 172.21.15.82:0/1841017413 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5db8101610 msgr2=0x7f5db81027f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:12:37.347 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:37.346+0000 7f5dbd23f700 1 --2- 172.21.15.82:0/1841017413 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5db8101610 0x7f5db81027f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:37.348 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:37.346+0000 7f5dbd23f700 1 -- 172.21.15.82:0/1841017413 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5db81a8070 con 0x7f5db8108180 2024-05-20T16:12:37.348 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:37.346+0000 7f5dbd23f700 1 --2- 172.21.15.82:0/1841017413 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5db8108180 0x7f5db8103270 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f5db00062a0 tx=0x7f5db0004e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:12:37.348 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:37.346+0000 7f5db5ffb700 1 -- 172.21.15.82:0/1841017413 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5db0021070 con 0x7f5db8108180 2024-05-20T16:12:37.348 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:37.346+0000 7f5db5ffb700 1 -- 172.21.15.82:0/1841017413 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5db0007c80 con 0x7f5db8108180 2024-05-20T16:12:37.348 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:37.346+0000 7f5dbeca2700 1 -- 172.21.15.82:0/1841017413 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5db81a8300 con 0x7f5db8108180 2024-05-20T16:12:37.348 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:37.346+0000 7f5db5ffb700 1 -- 172.21.15.82:0/1841017413 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5db001d450 con 0x7f5db8108180 2024-05-20T16:12:37.348 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:37.346+0000 7f5dbeca2700 1 -- 172.21.15.82:0/1841017413 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5db81a8800 con 0x7f5db8108180 2024-05-20T16:12:37.348 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:37.347+0000 7f5dbeca2700 1 -- 172.21.15.82:0/1841017413 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f5db8062d50 con 0x7f5db8108180 2024-05-20T16:12:37.348 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:37.349+0000 7f5db5ffb700 1 -- 172.21.15.82:0/1841017413 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f5db0004000 con 0x7f5db8108180 2024-05-20T16:12:37.349 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:37.349+0000 7f5db5ffb700 1 --2- 172.21.15.82:0/1841017413 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f5da80a2d30 0x7f5da80a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:12:37.350 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:37.349+0000 7f5dbca3e700 1 --2- 172.21.15.82:0/1841017413 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f5da80a2d30 0x7f5da80a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:12:37.350 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:37.349+0000 7f5db5ffb700 1 -- 172.21.15.82:0/1841017413 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8802+0+0 (secure 0 0 0) 0x7f5db000f020 con 0x7f5db8108180 2024-05-20T16:12:37.350 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:37.350+0000 7f5db5ffb700 1 --2- 172.21.15.82:0/1841017413 >> [v2:172.21.15.160:6824/2327428014,v1:172.21.15.160:6825/2327428014] conn(0x7f5da80ad9c0 0x7f5da80afe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:12:37.350 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:37.350+0000 7f5db5ffb700 1 -- 172.21.15.82:0/1841017413 --> [v2:172.21.15.160:6824/2327428014,v1:172.21.15.160:6825/2327428014] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5db0008ea0 con 0x7f5da80ad9c0 2024-05-20T16:12:37.350 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:37.350+0000 7f5db5ffb700 1 -- 172.21.15.82:0/1841017413 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7f5db00d1080 con 0x7f5db8108180 2024-05-20T16:12:37.350 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:37.350+0000 7f5dbca3e700 1 --2- 172.21.15.82:0/1841017413 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f5da80a2d30 0x7f5da80a5240 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f5dac005fd0 tx=0x7f5dac005f00 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:12:37.350 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:37.350+0000 7f5db7fff700 1 --2- 172.21.15.82:0/1841017413 >> [v2:172.21.15.160:6824/2327428014,v1:172.21.15.160:6825/2327428014] conn(0x7f5da80ad9c0 0x7f5da80afe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:12:37.350 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:37.350+0000 7f5db7fff700 1 --2- 172.21.15.82:0/1841017413 >> [v2:172.21.15.160:6824/2327428014,v1:172.21.15.160:6825/2327428014] conn(0x7f5da80ad9c0 0x7f5da80afe50 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:12:37.351 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:37.351+0000 7f5db5ffb700 1 -- 172.21.15.82:0/1841017413 <== osd.10 v2:172.21.15.160:6824/2327428014 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5db0008ea0 con 0x7f5da80ad9c0 2024-05-20T16:12:37.368 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:37.368+0000 7f5dbeca2700 1 -- 172.21.15.82:0/1841017413 --> [v2:172.21.15.160:6824/2327428014,v1:172.21.15.160:6825/2327428014] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f5db8063150 con 0x7f5da80ad9c0 2024-05-20T16:12:37.368 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:37.368+0000 7f5db5ffb700 1 -- 172.21.15.82:0/1841017413 <== osd.10 v2:172.21.15.160:6824/2327428014 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f5db8063150 con 0x7f5da80ad9c0 2024-05-20T16:12:37.369 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:37.369+0000 7f5dbeca2700 1 -- 172.21.15.82:0/1841017413 >> [v2:172.21.15.160:6824/2327428014,v1:172.21.15.160:6825/2327428014] conn(0x7f5da80ad9c0 msgr2=0x7f5da80afe50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:12:37.369 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:37.369+0000 7f5dbeca2700 1 --2- 172.21.15.82:0/1841017413 >> [v2:172.21.15.160:6824/2327428014,v1:172.21.15.160:6825/2327428014] conn(0x7f5da80ad9c0 0x7f5da80afe50 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:37.369 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:37.369+0000 7f5dbeca2700 1 -- 172.21.15.82:0/1841017413 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f5da80a2d30 msgr2=0x7f5da80a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:12:37.369 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:37.369+0000 7f5dbeca2700 1 --2- 172.21.15.82:0/1841017413 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f5da80a2d30 0x7f5da80a5240 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f5dac005fd0 tx=0x7f5dac005f00 comp rx=0 tx=0).stop 2024-05-20T16:12:37.369 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:37.369+0000 7f5dbeca2700 1 -- 172.21.15.82:0/1841017413 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5db8108180 msgr2=0x7f5db8103270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:12:37.369 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:37.369+0000 7f5dbeca2700 1 --2- 172.21.15.82:0/1841017413 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5db8108180 0x7f5db8103270 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f5db00062a0 tx=0x7f5db0004e20 comp rx=0 tx=0).stop 2024-05-20T16:12:37.369 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:37.369+0000 7f5dbd23f700 1 -- 172.21.15.82:0/1841017413 reap_dead start 2024-05-20T16:12:37.370 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:37.369+0000 7f5dbeca2700 1 -- 172.21.15.82:0/1841017413 shutdown_connections 2024-05-20T16:12:37.370 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:37.369+0000 7f5dbeca2700 1 -- 172.21.15.82:0/1841017413 >> 172.21.15.82:0/1841017413 conn(0x7f5db80fd3a0 msgr2=0x7f5db8105d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:12:37.370 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:37.369+0000 7f5dbeca2700 1 -- 172.21.15.82:0/1841017413 shutdown_connections 2024-05-20T16:12:37.370 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:37.369+0000 7f5dbeca2700 1 -- 172.21.15.82:0/1841017413 wait complete. 2024-05-20T16:12:37.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:12:37 smithi082 ceph-mon[32238]: pgmap v317: 129 pgs: 129 active+clean; 640 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.4 KiB/s wr, 5 op/s 2024-05-20T16:12:37.635 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:12:37 smithi184 ceph-mon[36209]: pgmap v317: 129 pgs: 129 active+clean; 640 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.4 KiB/s wr, 5 op/s 2024-05-20T16:12:37.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:12:37 smithi160 ceph-mon[36347]: pgmap v317: 129 pgs: 129 active+clean; 640 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.4 KiB/s wr, 5 op/s 2024-05-20T16:12:39.212 INFO:teuthology.orchestra.run.smithi082.stdout:313532612635 2024-05-20T16:12:39.213 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell --fsid 59d1600a-16c2-11ef-bc9a-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-05-20T16:12:39.470 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:12:39 smithi082 ceph-mon[32238]: pgmap v318: 129 pgs: 129 active+clean; 641 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-05-20T16:12:39.507 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:39.507+0000 7f99e3acd700 1 -- 172.21.15.82:0/1711048481 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f99dc102340 msgr2=0x7f99dc102750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:12:39.507 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:39.507+0000 7f99e3acd700 1 --2- 172.21.15.82:0/1711048481 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f99dc102340 0x7f99dc102750 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f99d00099c0 tx=0x7f99d0009cd0 comp rx=0 tx=0).stop 2024-05-20T16:12:39.507 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:39.507+0000 7f99e3acd700 1 -- 172.21.15.82:0/1711048481 shutdown_connections 2024-05-20T16:12:39.507 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:39.507+0000 7f99e3acd700 1 --2- 172.21.15.82:0/1711048481 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f99dc1075c0 0x7f99dc10ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:39.507 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:39.507+0000 7f99e3acd700 1 --2- 172.21.15.82:0/1711048481 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f99dc102340 0x7f99dc102750 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:39.507 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:39.507+0000 7f99e3acd700 1 --2- 172.21.15.82:0/1711048481 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f99dc106c70 0x7f99dc107080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:39.507 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:39.507+0000 7f99e3acd700 1 -- 172.21.15.82:0/1711048481 >> 172.21.15.82:0/1711048481 conn(0x7f99dc0fdc70 msgr2=0x7f99dc100090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:12:39.508 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:39.507+0000 7f99e3acd700 1 -- 172.21.15.82:0/1711048481 shutdown_connections 2024-05-20T16:12:39.508 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:39.507+0000 7f99e3acd700 1 -- 172.21.15.82:0/1711048481 wait complete. 2024-05-20T16:12:39.509 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:39.507+0000 7f99e3acd700 1 Processor -- start 2024-05-20T16:12:39.509 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:39.508+0000 7f99e3acd700 1 -- start start 2024-05-20T16:12:39.509 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:39.508+0000 7f99e3acd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f99dc102340 0x7f99dc19ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:12:39.510 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:39.508+0000 7f99e3acd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f99dc106c70 0x7f99dc19b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:12:39.510 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:39.508+0000 7f99e3acd700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f99dc1075c0 0x7f99dc19b710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:12:39.510 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:39.508+0000 7f99e3acd700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f99dc1123b0 con 0x7f99dc106c70 2024-05-20T16:12:39.510 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:39.508+0000 7f99e3acd700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f99dc112230 con 0x7f99dc102340 2024-05-20T16:12:39.510 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:39.508+0000 7f99e3acd700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f99dc112530 con 0x7f99dc1075c0 2024-05-20T16:12:39.510 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:39.508+0000 7f99e1068700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f99dc106c70 0x7f99dc19b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:12:39.510 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:39.508+0000 7f99e1068700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f99dc106c70 0x7f99dc19b1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41524/0 (socket says 172.21.15.82:41524) 2024-05-20T16:12:39.510 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:39.508+0000 7f99e1068700 1 -- 172.21.15.82:0/3099350107 learned_addr learned my addr 172.21.15.82:0/3099350107 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:12:39.510 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:39.508+0000 7f99e1869700 1 --2- 172.21.15.82:0/3099350107 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f99dc102340 0x7f99dc19ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:12:39.510 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:39.508+0000 7f99e1068700 1 -- 172.21.15.82:0/3099350107 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f99dc1075c0 msgr2=0x7f99dc19b710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:12:39.510 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:39.508+0000 7f99e1068700 1 --2- 172.21.15.82:0/3099350107 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f99dc1075c0 0x7f99dc19b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:39.510 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:39.508+0000 7f99e1068700 1 -- 172.21.15.82:0/3099350107 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f99dc102340 msgr2=0x7f99dc19ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:12:39.511 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:39.508+0000 7f99e1068700 1 --2- 172.21.15.82:0/3099350107 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f99dc102340 0x7f99dc19ac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:39.511 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:39.508+0000 7f99e1068700 1 -- 172.21.15.82:0/3099350107 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99dc1a5f30 con 0x7f99dc106c70 2024-05-20T16:12:39.511 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:39.508+0000 7f99e1869700 1 --2- 172.21.15.82:0/3099350107 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f99dc102340 0x7f99dc19ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T16:12:39.511 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:39.508+0000 7f99e1068700 1 --2- 172.21.15.82:0/3099350107 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f99dc106c70 0x7f99dc19b1b0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f99d800b500 tx=0x7f99d800b8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:12:39.511 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:39.509+0000 7f99ceffd700 1 -- 172.21.15.82:0/3099350107 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99d8015020 con 0x7f99dc106c70 2024-05-20T16:12:39.511 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:39.509+0000 7f99ceffd700 1 -- 172.21.15.82:0/3099350107 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f99d8011d10 con 0x7f99dc106c70 2024-05-20T16:12:39.511 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:39.509+0000 7f99e3acd700 1 -- 172.21.15.82:0/3099350107 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99dc1a6220 con 0x7f99dc106c70 2024-05-20T16:12:39.511 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:39.509+0000 7f99ceffd700 1 -- 172.21.15.82:0/3099350107 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99d800f2e0 con 0x7f99dc106c70 2024-05-20T16:12:39.511 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:39.509+0000 7f99e3acd700 1 -- 172.21.15.82:0/3099350107 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99dc1a67b0 con 0x7f99dc106c70 2024-05-20T16:12:39.511 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:39.509+0000 7f99e3acd700 1 -- 172.21.15.82:0/3099350107 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f99dc062d50 con 0x7f99dc106c70 2024-05-20T16:12:39.512 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:39.511+0000 7f99ceffd700 1 -- 172.21.15.82:0/3099350107 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f99d8018020 con 0x7f99dc106c70 2024-05-20T16:12:39.512 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:39.512+0000 7f99ceffd700 1 --2- 172.21.15.82:0/3099350107 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f99c80a2d30 0x7f99c80a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:12:39.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:39.512+0000 7f99e1869700 1 --2- 172.21.15.82:0/3099350107 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f99c80a2d30 0x7f99c80a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:12:39.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:39.512+0000 7f99ceffd700 1 -- 172.21.15.82:0/3099350107 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8802+0+0 (secure 0 0 0) 0x7f99d80cf060 con 0x7f99dc106c70 2024-05-20T16:12:39.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:39.512+0000 7f99ceffd700 1 --2- 172.21.15.82:0/3099350107 >> [v2:172.21.15.184:6816/3831576372,v1:172.21.15.184:6817/3831576372] conn(0x7f99c80ada40 0x7f99c80afed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:12:39.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:39.512+0000 7f99ceffd700 1 -- 172.21.15.82:0/3099350107 --> [v2:172.21.15.184:6816/3831576372,v1:172.21.15.184:6817/3831576372] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f99c80b05a0 con 0x7f99c80ada40 2024-05-20T16:12:39.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:39.512+0000 7f99ceffd700 1 -- 172.21.15.82:0/3099350107 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7f99d80cf4b0 con 0x7f99dc106c70 2024-05-20T16:12:39.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:39.512+0000 7f99e1869700 1 --2- 172.21.15.82:0/3099350107 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f99c80a2d30 0x7f99c80a5240 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f99d000b580 tx=0x7f99d0005ea0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:12:39.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:39.512+0000 7f99e206a700 1 --2- 172.21.15.82:0/3099350107 >> [v2:172.21.15.184:6816/3831576372,v1:172.21.15.184:6817/3831576372] conn(0x7f99c80ada40 0x7f99c80afed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:12:39.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:39.513+0000 7f99e206a700 1 --2- 172.21.15.82:0/3099350107 >> [v2:172.21.15.184:6816/3831576372,v1:172.21.15.184:6817/3831576372] conn(0x7f99c80ada40 0x7f99c80afed0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:12:39.514 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:39.514+0000 7f99ceffd700 1 -- 172.21.15.82:0/3099350107 <== osd.8 v2:172.21.15.184:6816/3831576372 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f99c80b05a0 con 0x7f99c80ada40 2024-05-20T16:12:39.530 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:39.530+0000 7f99e3acd700 1 -- 172.21.15.82:0/3099350107 --> [v2:172.21.15.184:6816/3831576372,v1:172.21.15.184:6817/3831576372] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f99dc063150 con 0x7f99c80ada40 2024-05-20T16:12:39.530 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:39.531+0000 7f99ceffd700 1 -- 172.21.15.82:0/3099350107 <== osd.8 v2:172.21.15.184:6816/3831576372 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f99dc063150 con 0x7f99c80ada40 2024-05-20T16:12:39.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:39.531+0000 7f99e3acd700 1 -- 172.21.15.82:0/3099350107 >> [v2:172.21.15.184:6816/3831576372,v1:172.21.15.184:6817/3831576372] conn(0x7f99c80ada40 msgr2=0x7f99c80afed0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:12:39.532 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:39.531+0000 7f99e3acd700 1 --2- 172.21.15.82:0/3099350107 >> [v2:172.21.15.184:6816/3831576372,v1:172.21.15.184:6817/3831576372] conn(0x7f99c80ada40 0x7f99c80afed0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:39.532 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:39.531+0000 7f99e3acd700 1 -- 172.21.15.82:0/3099350107 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f99c80a2d30 msgr2=0x7f99c80a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:12:39.532 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:39.531+0000 7f99e3acd700 1 --2- 172.21.15.82:0/3099350107 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f99c80a2d30 0x7f99c80a5240 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f99d000b580 tx=0x7f99d0005ea0 comp rx=0 tx=0).stop 2024-05-20T16:12:39.532 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:39.531+0000 7f99e3acd700 1 -- 172.21.15.82:0/3099350107 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f99dc106c70 msgr2=0x7f99dc19b1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:12:39.532 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:39.531+0000 7f99e3acd700 1 --2- 172.21.15.82:0/3099350107 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f99dc106c70 0x7f99dc19b1b0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f99d800b500 tx=0x7f99d800b8c0 comp rx=0 tx=0).stop 2024-05-20T16:12:39.532 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:39.531+0000 7f99e206a700 1 -- 172.21.15.82:0/3099350107 reap_dead start 2024-05-20T16:12:39.532 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:39.531+0000 7f99e3acd700 1 -- 172.21.15.82:0/3099350107 shutdown_connections 2024-05-20T16:12:39.532 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:39.531+0000 7f99e3acd700 1 -- 172.21.15.82:0/3099350107 >> 172.21.15.82:0/3099350107 conn(0x7f99dc0fdc70 msgr2=0x7f99dc108520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:12:39.532 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:39.531+0000 7f99e3acd700 1 -- 172.21.15.82:0/3099350107 shutdown_connections 2024-05-20T16:12:39.532 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:39.531+0000 7f99e3acd700 1 -- 172.21.15.82:0/3099350107 wait complete. 2024-05-20T16:12:39.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:12:39 smithi184 ceph-mon[36209]: pgmap v318: 129 pgs: 129 active+clean; 641 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-05-20T16:12:39.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:12:39 smithi160 ceph-mon[36347]: pgmap v318: 129 pgs: 129 active+clean; 641 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-05-20T16:12:40.194 INFO:teuthology.orchestra.run.smithi082.stdout:292057776162 2024-05-20T16:12:40.194 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell --fsid 59d1600a-16c2-11ef-bc9a-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-05-20T16:12:41.555 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:40.947+0000 7fa98904e700 1 -- 172.21.15.82:0/2969193629 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa984102340 msgr2=0x7fa984102750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:12:41.556 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:40.947+0000 7fa98904e700 1 --2- 172.21.15.82:0/2969193629 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa984102340 0x7fa984102750 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fa9740099c0 tx=0x7fa974009cd0 comp rx=0 tx=0).stop 2024-05-20T16:12:41.556 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:40.947+0000 7fa98904e700 1 -- 172.21.15.82:0/2969193629 shutdown_connections 2024-05-20T16:12:41.556 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:40.947+0000 7fa98904e700 1 --2- 172.21.15.82:0/2969193629 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa9841075c0 0x7fa98410ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:41.556 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:40.947+0000 7fa98904e700 1 --2- 172.21.15.82:0/2969193629 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa984102340 0x7fa984102750 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:41.556 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:40.947+0000 7fa98904e700 1 --2- 172.21.15.82:0/2969193629 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa984106c70 0x7fa984107080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:41.556 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:40.947+0000 7fa98904e700 1 -- 172.21.15.82:0/2969193629 >> 172.21.15.82:0/2969193629 conn(0x7fa9840fdc40 msgr2=0x7fa984100060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:12:41.556 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:40.947+0000 7fa98904e700 1 -- 172.21.15.82:0/2969193629 shutdown_connections 2024-05-20T16:12:41.556 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:40.947+0000 7fa98904e700 1 -- 172.21.15.82:0/2969193629 wait complete. 2024-05-20T16:12:41.556 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:40.948+0000 7fa98904e700 1 Processor -- start 2024-05-20T16:12:41.556 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:40.948+0000 7fa98904e700 1 -- start start 2024-05-20T16:12:41.557 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:40.948+0000 7fa98904e700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa984102340 0x7fa98419e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:12:41.557 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:40.948+0000 7fa98904e700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa984106c70 0x7fa98419ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:12:41.557 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:40.948+0000 7fa98904e700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa9841075c0 0x7fa984198a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:12:41.557 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:40.948+0000 7fa98904e700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fa984112340 con 0x7fa984102340 2024-05-20T16:12:41.557 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:40.948+0000 7fa98904e700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fa9841121c0 con 0x7fa984106c70 2024-05-20T16:12:41.557 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:40.948+0000 7fa98904e700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa9841124c0 con 0x7fa9841075c0 2024-05-20T16:12:41.557 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:40.948+0000 7fa982d9d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa984102340 0x7fa98419e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:12:41.557 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:40.948+0000 7fa982d9d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa984102340 0x7fa98419e840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58196/0 (socket says 172.21.15.82:58196) 2024-05-20T16:12:41.557 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:40.948+0000 7fa982d9d700 1 -- 172.21.15.82:0/405389756 learned_addr learned my addr 172.21.15.82:0/405389756 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:12:41.557 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:40.949+0000 7fa98359e700 1 --2- 172.21.15.82:0/405389756 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa9841075c0 0x7fa984198a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:12:41.558 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:40.949+0000 7fa982d9d700 1 -- 172.21.15.82:0/405389756 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa9841075c0 msgr2=0x7fa984198a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:12:41.558 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:40.949+0000 7fa98259c700 1 --2- 172.21.15.82:0/405389756 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa984106c70 0x7fa98419ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:12:41.558 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:40.949+0000 7fa982d9d700 1 --2- 172.21.15.82:0/405389756 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa9841075c0 0x7fa984198a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:41.558 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:40.949+0000 7fa982d9d700 1 -- 172.21.15.82:0/405389756 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa984106c70 msgr2=0x7fa98419ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:12:41.558 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:40.949+0000 7fa982d9d700 1 --2- 172.21.15.82:0/405389756 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa984106c70 0x7fa98419ed80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:41.559 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:40.949+0000 7fa982d9d700 1 -- 172.21.15.82:0/405389756 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9841990a0 con 0x7fa984102340 2024-05-20T16:12:41.559 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:40.949+0000 7fa98359e700 1 --2- 172.21.15.82:0/405389756 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa9841075c0 0x7fa984198a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T16:12:41.559 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:40.949+0000 7fa982d9d700 1 --2- 172.21.15.82:0/405389756 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa984102340 0x7fa98419e840 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fa974006010 tx=0x7fa974009f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:12:41.559 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:40.949+0000 7fa96bfff700 1 -- 172.21.15.82:0/405389756 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa974029070 con 0x7fa984102340 2024-05-20T16:12:41.559 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:40.949+0000 7fa98904e700 1 -- 172.21.15.82:0/405389756 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa984199330 con 0x7fa984102340 2024-05-20T16:12:41.559 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:40.949+0000 7fa96bfff700 1 -- 172.21.15.82:0/405389756 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa974019cf0 con 0x7fa984102340 2024-05-20T16:12:41.559 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:40.949+0000 7fa96bfff700 1 -- 172.21.15.82:0/405389756 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa974024470 con 0x7fa984102340 2024-05-20T16:12:41.559 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:40.949+0000 7fa98904e700 1 -- 172.21.15.82:0/405389756 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa984199810 con 0x7fa984102340 2024-05-20T16:12:41.559 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:40.950+0000 7fa98904e700 1 -- 172.21.15.82:0/405389756 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa98406a7e0 con 0x7fa984102340 2024-05-20T16:12:41.559 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:40.951+0000 7fa96bfff700 1 -- 172.21.15.82:0/405389756 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fa97402c070 con 0x7fa984102340 2024-05-20T16:12:41.559 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:40.952+0000 7fa96bfff700 1 --2- 172.21.15.82:0/405389756 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fa96c0a2cb0 0x7fa96c0a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:12:41.560 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:40.952+0000 7fa98259c700 1 --2- 172.21.15.82:0/405389756 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fa96c0a2cb0 0x7fa96c0a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:12:41.560 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:40.952+0000 7fa96bfff700 1 -- 172.21.15.82:0/405389756 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8802+0+0 (secure 0 0 0) 0x7fa9740d2a70 con 0x7fa984102340 2024-05-20T16:12:41.560 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:40.952+0000 7fa96bfff700 1 --2- 172.21.15.82:0/405389756 >> [v2:172.21.15.82:6802/1484330167,v1:172.21.15.82:6803/1484330167] conn(0x7fa96c0ad9c0 0x7fa96c0afe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:12:41.560 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:40.952+0000 7fa96bfff700 1 -- 172.21.15.82:0/405389756 --> [v2:172.21.15.82:6802/1484330167,v1:172.21.15.82:6803/1484330167] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa97401ce50 con 0x7fa96c0ad9c0 2024-05-20T16:12:41.560 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:40.952+0000 7fa96bfff700 1 -- 172.21.15.82:0/405389756 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7fa9740d2ec0 con 0x7fa984102340 2024-05-20T16:12:41.560 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:40.952+0000 7fa98359e700 1 --2- 172.21.15.82:0/405389756 >> [v2:172.21.15.82:6802/1484330167,v1:172.21.15.82:6803/1484330167] conn(0x7fa96c0ad9c0 0x7fa96c0afe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:12:41.560 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:40.953+0000 7fa98259c700 1 --2- 172.21.15.82:0/405389756 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fa96c0a2cb0 0x7fa96c0a51c0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fa978005950 tx=0x7fa9780058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:12:41.560 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:40.953+0000 7fa98359e700 1 --2- 172.21.15.82:0/405389756 >> [v2:172.21.15.82:6802/1484330167,v1:172.21.15.82:6803/1484330167] conn(0x7fa96c0ad9c0 0x7fa96c0afe50 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:12:41.560 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:40.953+0000 7fa96bfff700 1 -- 172.21.15.82:0/405389756 <== osd.0 v2:172.21.15.82:6802/1484330167 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa97401ce50 con 0x7fa96c0ad9c0 2024-05-20T16:12:41.560 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:40.970+0000 7fa98904e700 1 -- 172.21.15.82:0/405389756 --> [v2:172.21.15.82:6802/1484330167,v1:172.21.15.82:6803/1484330167] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fa98406ac60 con 0x7fa96c0ad9c0 2024-05-20T16:12:41.560 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:40.970+0000 7fa96bfff700 1 -- 172.21.15.82:0/405389756 <== osd.0 v2:172.21.15.82:6802/1484330167 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fa98406ac60 con 0x7fa96c0ad9c0 2024-05-20T16:12:41.560 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:40.970+0000 7fa98904e700 1 -- 172.21.15.82:0/405389756 >> [v2:172.21.15.82:6802/1484330167,v1:172.21.15.82:6803/1484330167] conn(0x7fa96c0ad9c0 msgr2=0x7fa96c0afe50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:12:41.560 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:40.970+0000 7fa98904e700 1 --2- 172.21.15.82:0/405389756 >> [v2:172.21.15.82:6802/1484330167,v1:172.21.15.82:6803/1484330167] conn(0x7fa96c0ad9c0 0x7fa96c0afe50 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:41.561 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:40.970+0000 7fa98904e700 1 -- 172.21.15.82:0/405389756 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fa96c0a2cb0 msgr2=0x7fa96c0a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:12:41.561 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:40.970+0000 7fa98904e700 1 --2- 172.21.15.82:0/405389756 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fa96c0a2cb0 0x7fa96c0a51c0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fa978005950 tx=0x7fa9780058e0 comp rx=0 tx=0).stop 2024-05-20T16:12:41.561 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:40.970+0000 7fa98904e700 1 -- 172.21.15.82:0/405389756 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa984102340 msgr2=0x7fa98419e840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:12:41.561 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:40.970+0000 7fa98904e700 1 --2- 172.21.15.82:0/405389756 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa984102340 0x7fa98419e840 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fa974006010 tx=0x7fa974009f80 comp rx=0 tx=0).stop 2024-05-20T16:12:41.561 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:40.970+0000 7fa98359e700 1 -- 172.21.15.82:0/405389756 reap_dead start 2024-05-20T16:12:41.561 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:40.970+0000 7fa98904e700 1 -- 172.21.15.82:0/405389756 shutdown_connections 2024-05-20T16:12:41.561 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:40.970+0000 7fa98904e700 1 -- 172.21.15.82:0/405389756 >> 172.21.15.82:0/405389756 conn(0x7fa9840fdc40 msgr2=0x7fa984100a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:12:41.561 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:40.971+0000 7fa98904e700 1 -- 172.21.15.82:0/405389756 shutdown_connections 2024-05-20T16:12:41.561 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:40.971+0000 7fa98904e700 1 -- 172.21.15.82:0/405389756 wait complete. 2024-05-20T16:12:41.561 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:12:41 smithi082 ceph-mon[32238]: pgmap v319: 129 pgs: 129 active+clean; 641 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-20T16:12:41.638 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:12:41 smithi184 ceph-mon[36209]: pgmap v319: 129 pgs: 129 active+clean; 641 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-20T16:12:41.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:12:41 smithi160 ceph-mon[36347]: pgmap v319: 129 pgs: 129 active+clean; 641 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-20T16:12:42.816 INFO:teuthology.orchestra.run.smithi082.stdout:34359738448 2024-05-20T16:12:42.817 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell --fsid 59d1600a-16c2-11ef-bc9a-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-05-20T16:12:43.489 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:12:43 smithi082 ceph-mon[32238]: pgmap v320: 129 pgs: 129 active+clean; 641 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-20T16:12:43.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:12:43 smithi184 ceph-mon[36209]: pgmap v320: 129 pgs: 129 active+clean; 641 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-20T16:12:43.807 INFO:teuthology.orchestra.run.smithi082.stdout:257698037799 2024-05-20T16:12:43.807 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell --fsid 59d1600a-16c2-11ef-bc9a-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-05-20T16:12:43.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:12:43 smithi160 ceph-mon[36347]: pgmap v320: 129 pgs: 129 active+clean; 641 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-20T16:12:44.010 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:44.010+0000 7f00d3f62700 1 -- 172.21.15.82:0/4279646449 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f00cc106c70 msgr2=0x7f00cc107080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:12:44.011 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:44.010+0000 7f00d3f62700 1 --2- 172.21.15.82:0/4279646449 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f00cc106c70 0x7f00cc107080 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f00bc0099c0 tx=0x7f00bc009cd0 comp rx=0 tx=0).stop 2024-05-20T16:12:44.011 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:44.010+0000 7f00d3f62700 1 -- 172.21.15.82:0/4279646449 shutdown_connections 2024-05-20T16:12:44.011 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:44.010+0000 7f00d3f62700 1 --2- 172.21.15.82:0/4279646449 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f00cc1075c0 0x7f00cc10ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:44.011 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:44.010+0000 7f00d3f62700 1 --2- 172.21.15.82:0/4279646449 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f00cc102340 0x7f00cc102750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:44.011 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:44.010+0000 7f00d3f62700 1 --2- 172.21.15.82:0/4279646449 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f00cc106c70 0x7f00cc107080 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:44.011 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:44.010+0000 7f00d3f62700 1 -- 172.21.15.82:0/4279646449 >> 172.21.15.82:0/4279646449 conn(0x7f00cc0fdc40 msgr2=0x7f00cc100060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:12:44.011 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:44.010+0000 7f00d3f62700 1 -- 172.21.15.82:0/4279646449 shutdown_connections 2024-05-20T16:12:44.011 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:44.011+0000 7f00d3f62700 1 -- 172.21.15.82:0/4279646449 wait complete. 2024-05-20T16:12:44.013 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:44.011+0000 7f00d3f62700 1 Processor -- start 2024-05-20T16:12:44.013 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:44.011+0000 7f00d3f62700 1 -- start start 2024-05-20T16:12:44.013 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:44.012+0000 7f00d3f62700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f00cc102340 0x7f00cc19abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:12:44.014 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:44.012+0000 7f00d3f62700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f00cc106c70 0x7f00cc19b120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:12:44.014 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:44.012+0000 7f00d3f62700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f00cc1075c0 0x7f00cc19b680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:12:44.014 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:44.012+0000 7f00d3f62700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f00cc112320 con 0x7f00cc102340 2024-05-20T16:12:44.014 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:44.012+0000 7f00d3f62700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f00cc1121a0 con 0x7f00cc1075c0 2024-05-20T16:12:44.014 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:44.012+0000 7f00d3f62700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f00cc1124a0 con 0x7f00cc106c70 2024-05-20T16:12:44.014 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:44.012+0000 7f00d1cfe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f00cc102340 0x7f00cc19abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:12:44.014 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:44.012+0000 7f00d1cfe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f00cc102340 0x7f00cc19abe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58204/0 (socket says 172.21.15.82:58204) 2024-05-20T16:12:44.014 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:44.012+0000 7f00d1cfe700 1 -- 172.21.15.82:0/3529163907 learned_addr learned my addr 172.21.15.82:0/3529163907 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:12:44.014 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:44.012+0000 7f00d14fd700 1 --2- 172.21.15.82:0/3529163907 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f00cc106c70 0x7f00cc19b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:12:44.014 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:44.012+0000 7f00d24ff700 1 --2- 172.21.15.82:0/3529163907 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f00cc1075c0 0x7f00cc19b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:12:44.015 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:44.012+0000 7f00d1cfe700 1 -- 172.21.15.82:0/3529163907 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f00cc106c70 msgr2=0x7f00cc19b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:12:44.015 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:44.012+0000 7f00d1cfe700 1 --2- 172.21.15.82:0/3529163907 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f00cc106c70 0x7f00cc19b120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:44.015 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:44.012+0000 7f00d1cfe700 1 -- 172.21.15.82:0/3529163907 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f00cc1075c0 msgr2=0x7f00cc19b680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:12:44.015 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:44.012+0000 7f00d1cfe700 1 --2- 172.21.15.82:0/3529163907 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f00cc1075c0 0x7f00cc19b680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:44.015 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:44.012+0000 7f00d1cfe700 1 -- 172.21.15.82:0/3529163907 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00cc1a5f30 con 0x7f00cc102340 2024-05-20T16:12:44.015 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:44.013+0000 7f00d1cfe700 1 --2- 172.21.15.82:0/3529163907 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f00cc102340 0x7f00cc19abe0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f00c800e9a0 tx=0x7f00c800ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:12:44.015 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:44.013+0000 7f00d14fd700 1 --2- 172.21.15.82:0/3529163907 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f00cc106c70 0x7f00cc19b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T16:12:44.015 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:44.013+0000 7f00d24ff700 1 --2- 172.21.15.82:0/3529163907 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f00cc1075c0 0x7f00cc19b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T16:12:44.015 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:44.013+0000 7f00c2ffd700 1 -- 172.21.15.82:0/3529163907 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00c800cbd0 con 0x7f00cc102340 2024-05-20T16:12:44.015 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:44.013+0000 7f00d3f62700 1 -- 172.21.15.82:0/3529163907 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00cc198a30 con 0x7f00cc102340 2024-05-20T16:12:44.015 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:44.013+0000 7f00d3f62700 1 -- 172.21.15.82:0/3529163907 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00cc198f90 con 0x7f00cc102340 2024-05-20T16:12:44.015 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:44.013+0000 7f00c2ffd700 1 -- 172.21.15.82:0/3529163907 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f00c800cd70 con 0x7f00cc102340 2024-05-20T16:12:44.015 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:44.013+0000 7f00c2ffd700 1 -- 172.21.15.82:0/3529163907 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00c8026060 con 0x7f00cc102340 2024-05-20T16:12:44.017 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:44.014+0000 7f00d3f62700 1 -- 172.21.15.82:0/3529163907 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f00b0001030 con 0x7f00cc102340 2024-05-20T16:12:44.017 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:44.015+0000 7f00c2ffd700 1 -- 172.21.15.82:0/3529163907 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f00c8010430 con 0x7f00cc102340 2024-05-20T16:12:44.017 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:44.016+0000 7f00c2ffd700 1 --2- 172.21.15.82:0/3529163907 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f00b80a30b0 0x7f00b80a55c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:12:44.017 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:44.016+0000 7f00d14fd700 1 --2- 172.21.15.82:0/3529163907 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f00b80a30b0 0x7f00b80a55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:12:44.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:44.016+0000 7f00c2ffd700 1 -- 172.21.15.82:0/3529163907 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8802+0+0 (secure 0 0 0) 0x7f00c800c9d0 con 0x7f00cc102340 2024-05-20T16:12:44.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:44.016+0000 7f00c2ffd700 1 --2- 172.21.15.82:0/3529163907 >> [v2:172.21.15.160:6808/4160830014,v1:172.21.15.160:6809/4160830014] conn(0x7f00b80add40 0x7f00b80b01d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:12:44.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:44.016+0000 7f00c2ffd700 1 -- 172.21.15.82:0/3529163907 --> [v2:172.21.15.160:6808/4160830014,v1:172.21.15.160:6809/4160830014] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f00c80056b0 con 0x7f00b80add40 2024-05-20T16:12:44.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:44.016+0000 7f00c2ffd700 1 -- 172.21.15.82:0/3529163907 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7f00c809b330 con 0x7f00cc102340 2024-05-20T16:12:44.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:44.016+0000 7f00d14fd700 1 --2- 172.21.15.82:0/3529163907 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f00b80a30b0 0x7f00b80a55c0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f00bc000c00 tx=0x7f00bc01a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:12:44.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:44.019+0000 7f00d24ff700 1 --2- 172.21.15.82:0/3529163907 >> [v2:172.21.15.160:6808/4160830014,v1:172.21.15.160:6809/4160830014] conn(0x7f00b80add40 0x7f00b80b01d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:12:44.021 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:44.022+0000 7f00d24ff700 1 --2- 172.21.15.82:0/3529163907 >> [v2:172.21.15.160:6808/4160830014,v1:172.21.15.160:6809/4160830014] conn(0x7f00b80add40 0x7f00b80b01d0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:12:44.023 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:44.023+0000 7f00c2ffd700 1 -- 172.21.15.82:0/3529163907 <== osd.4 v2:172.21.15.160:6808/4160830014 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f00c80056b0 con 0x7f00b80add40 2024-05-20T16:12:44.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:44.039+0000 7f00d3f62700 1 -- 172.21.15.82:0/3529163907 --> [v2:172.21.15.160:6808/4160830014,v1:172.21.15.160:6809/4160830014] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f00b0002e60 con 0x7f00b80add40 2024-05-20T16:12:44.040 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:44.040+0000 7f00c2ffd700 1 -- 172.21.15.82:0/3529163907 <== osd.4 v2:172.21.15.160:6808/4160830014 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f00b0002e60 con 0x7f00b80add40 2024-05-20T16:12:44.041 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:44.040+0000 7f00d3f62700 1 -- 172.21.15.82:0/3529163907 >> [v2:172.21.15.160:6808/4160830014,v1:172.21.15.160:6809/4160830014] conn(0x7f00b80add40 msgr2=0x7f00b80b01d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:12:44.041 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:44.040+0000 7f00d3f62700 1 --2- 172.21.15.82:0/3529163907 >> [v2:172.21.15.160:6808/4160830014,v1:172.21.15.160:6809/4160830014] conn(0x7f00b80add40 0x7f00b80b01d0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:44.041 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:44.040+0000 7f00d3f62700 1 -- 172.21.15.82:0/3529163907 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f00b80a30b0 msgr2=0x7f00b80a55c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:12:44.041 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:44.040+0000 7f00d3f62700 1 --2- 172.21.15.82:0/3529163907 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f00b80a30b0 0x7f00b80a55c0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f00bc000c00 tx=0x7f00bc01a040 comp rx=0 tx=0).stop 2024-05-20T16:12:44.042 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:44.040+0000 7f00d3f62700 1 -- 172.21.15.82:0/3529163907 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f00cc102340 msgr2=0x7f00cc19abe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:12:44.042 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:44.040+0000 7f00d3f62700 1 --2- 172.21.15.82:0/3529163907 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f00cc102340 0x7f00cc19abe0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f00c800e9a0 tx=0x7f00c800ed60 comp rx=0 tx=0).stop 2024-05-20T16:12:44.042 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:44.040+0000 7f00d24ff700 1 -- 172.21.15.82:0/3529163907 reap_dead start 2024-05-20T16:12:44.042 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:44.040+0000 7f00d3f62700 1 -- 172.21.15.82:0/3529163907 shutdown_connections 2024-05-20T16:12:44.042 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:44.040+0000 7f00d3f62700 1 -- 172.21.15.82:0/3529163907 >> 172.21.15.82:0/3529163907 conn(0x7f00cc0fdc40 msgr2=0x7f00cc1063e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:12:44.042 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:44.040+0000 7f00d3f62700 1 -- 172.21.15.82:0/3529163907 shutdown_connections 2024-05-20T16:12:44.042 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:44.040+0000 7f00d3f62700 1 -- 172.21.15.82:0/3529163907 wait complete. 2024-05-20T16:12:45.208 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:45.206+0000 7f1987a6a700 1 -- 172.21.15.82:0/2091718927 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1980102340 msgr2=0x7f1980102750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:12:45.209 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:45.206+0000 7f1987a6a700 1 --2- 172.21.15.82:0/2091718927 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1980102340 0x7f1980102750 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f197c0099c0 tx=0x7f197c009cd0 comp rx=0 tx=0).stop 2024-05-20T16:12:45.209 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:45.206+0000 7f1987a6a700 1 -- 172.21.15.82:0/2091718927 shutdown_connections 2024-05-20T16:12:45.209 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:45.206+0000 7f1987a6a700 1 --2- 172.21.15.82:0/2091718927 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f19801075c0 0x7f198010ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:45.209 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:45.206+0000 7f1987a6a700 1 --2- 172.21.15.82:0/2091718927 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1980102340 0x7f1980102750 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:45.209 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:45.206+0000 7f1987a6a700 1 --2- 172.21.15.82:0/2091718927 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1980106c70 0x7f1980107080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:45.209 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:45.206+0000 7f1987a6a700 1 -- 172.21.15.82:0/2091718927 >> 172.21.15.82:0/2091718927 conn(0x7f19800fdc40 msgr2=0x7f1980100060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:12:45.209 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:45.206+0000 7f1987a6a700 1 -- 172.21.15.82:0/2091718927 shutdown_connections 2024-05-20T16:12:45.209 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:45.206+0000 7f1987a6a700 1 -- 172.21.15.82:0/2091718927 wait complete. 2024-05-20T16:12:45.209 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:45.207+0000 7f1987a6a700 1 Processor -- start 2024-05-20T16:12:45.210 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:45.207+0000 7f1987a6a700 1 -- start start 2024-05-20T16:12:45.210 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:45.207+0000 7f1987a6a700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1980102340 0x7f1980199d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:12:45.210 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:45.207+0000 7f1987a6a700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1980106c70 0x7f198019a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:12:45.210 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:45.207+0000 7f1987a6a700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f19801075c0 0x7f198019e590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:12:45.210 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:45.207+0000 7f1987a6a700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f1980113490 con 0x7f1980106c70 2024-05-20T16:12:45.210 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:45.207+0000 7f1987a6a700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f1980113310 con 0x7f1980102340 2024-05-20T16:12:45.210 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:45.207+0000 7f1987a6a700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f1980113610 con 0x7f19801075c0 2024-05-20T16:12:45.210 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:45.207+0000 7f1985005700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1980106c70 0x7f198019a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:12:45.210 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:45.207+0000 7f1985005700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1980106c70 0x7f198019a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58218/0 (socket says 172.21.15.82:58218) 2024-05-20T16:12:45.210 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:45.207+0000 7f1985005700 1 -- 172.21.15.82:0/1048072813 learned_addr learned my addr 172.21.15.82:0/1048072813 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:12:45.210 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:45.207+0000 7f1985806700 1 --2- 172.21.15.82:0/1048072813 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1980102340 0x7f1980199d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:12:45.210 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:45.208+0000 7f1986007700 1 --2- 172.21.15.82:0/1048072813 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f19801075c0 0x7f198019e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:12:45.211 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:45.208+0000 7f1985005700 1 -- 172.21.15.82:0/1048072813 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f19801075c0 msgr2=0x7f198019e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:12:45.211 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:45.208+0000 7f1985005700 1 --2- 172.21.15.82:0/1048072813 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f19801075c0 0x7f198019e590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:45.211 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:45.208+0000 7f1985005700 1 -- 172.21.15.82:0/1048072813 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1980102340 msgr2=0x7f1980199d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:12:45.211 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:45.208+0000 7f1985005700 1 --2- 172.21.15.82:0/1048072813 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1980102340 0x7f1980199d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:45.211 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:45.208+0000 7f1985005700 1 -- 172.21.15.82:0/1048072813 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f198019eb80 con 0x7f1980106c70 2024-05-20T16:12:45.211 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:45.208+0000 7f1985005700 1 --2- 172.21.15.82:0/1048072813 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1980106c70 0x7f198019a300 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f197000d980 tx=0x7f197000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:12:45.211 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:45.208+0000 7f1976ffd700 1 -- 172.21.15.82:0/1048072813 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1970017070 con 0x7f1980106c70 2024-05-20T16:12:45.211 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:45.208+0000 7f1987a6a700 1 -- 172.21.15.82:0/1048072813 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f198019ee70 con 0x7f1980106c70 2024-05-20T16:12:45.211 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:45.208+0000 7f1976ffd700 1 -- 172.21.15.82:0/1048072813 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1970009e10 con 0x7f1980106c70 2024-05-20T16:12:45.211 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:45.208+0000 7f1976ffd700 1 -- 172.21.15.82:0/1048072813 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f197000f620 con 0x7f1980106c70 2024-05-20T16:12:45.211 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:45.208+0000 7f1987a6a700 1 -- 172.21.15.82:0/1048072813 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19801a6840 con 0x7f1980106c70 2024-05-20T16:12:45.212 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:45.209+0000 7f1987a6a700 1 -- 172.21.15.82:0/1048072813 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f1980062dd0 con 0x7f1980106c70 2024-05-20T16:12:45.213 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:45.211+0000 7f1976ffd700 1 -- 172.21.15.82:0/1048072813 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f1970010460 con 0x7f1980106c70 2024-05-20T16:12:45.213 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:45.211+0000 7f1976ffd700 1 --2- 172.21.15.82:0/1048072813 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f196c0a2cb0 0x7f196c0a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:12:45.213 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:45.211+0000 7f1985806700 1 --2- 172.21.15.82:0/1048072813 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f196c0a2cb0 0x7f196c0a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:12:45.213 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:45.211+0000 7f1976ffd700 1 -- 172.21.15.82:0/1048072813 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8802+0+0 (secure 0 0 0) 0x7f19700cf3a0 con 0x7f1980106c70 2024-05-20T16:12:45.213 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:45.211+0000 7f1976ffd700 1 --2- 172.21.15.82:0/1048072813 >> [v2:172.21.15.82:6818/2789736499,v1:172.21.15.82:6819/2789736499] conn(0x7f196c0ad9c0 0x7f196c0afe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:12:45.213 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:45.212+0000 7f1976ffd700 1 -- 172.21.15.82:0/1048072813 --> [v2:172.21.15.82:6818/2789736499,v1:172.21.15.82:6819/2789736499] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f196c0b0520 con 0x7f196c0ad9c0 2024-05-20T16:12:45.213 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:45.212+0000 7f1976ffd700 1 -- 172.21.15.82:0/1048072813 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7f19700cf750 con 0x7f1980106c70 2024-05-20T16:12:45.213 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:45.212+0000 7f1986007700 1 --2- 172.21.15.82:0/1048072813 >> [v2:172.21.15.82:6818/2789736499,v1:172.21.15.82:6819/2789736499] conn(0x7f196c0ad9c0 0x7f196c0afe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:12:45.213 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:45.212+0000 7f1985806700 1 --2- 172.21.15.82:0/1048072813 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f196c0a2cb0 0x7f196c0a51c0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f197c000c00 tx=0x7f197c01a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:12:45.214 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:45.212+0000 7f1986007700 1 --2- 172.21.15.82:0/1048072813 >> [v2:172.21.15.82:6818/2789736499,v1:172.21.15.82:6819/2789736499] conn(0x7f196c0ad9c0 0x7f196c0afe50 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:12:45.214 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:45.213+0000 7f1976ffd700 1 -- 172.21.15.82:0/1048072813 <== osd.6 v2:172.21.15.82:6818/2789736499 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f196c0b0520 con 0x7f196c0ad9c0 2024-05-20T16:12:45.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:45.229+0000 7f1987a6a700 1 -- 172.21.15.82:0/1048072813 --> [v2:172.21.15.82:6818/2789736499,v1:172.21.15.82:6819/2789736499] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f19800631d0 con 0x7f196c0ad9c0 2024-05-20T16:12:45.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:45.229+0000 7f1976ffd700 1 -- 172.21.15.82:0/1048072813 <== osd.6 v2:172.21.15.82:6818/2789736499 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f19800631d0 con 0x7f196c0ad9c0 2024-05-20T16:12:45.230 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:45.230+0000 7f1987a6a700 1 -- 172.21.15.82:0/1048072813 >> [v2:172.21.15.82:6818/2789736499,v1:172.21.15.82:6819/2789736499] conn(0x7f196c0ad9c0 msgr2=0x7f196c0afe50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:12:45.230 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:45.230+0000 7f1987a6a700 1 --2- 172.21.15.82:0/1048072813 >> [v2:172.21.15.82:6818/2789736499,v1:172.21.15.82:6819/2789736499] conn(0x7f196c0ad9c0 0x7f196c0afe50 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:45.230 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:45.230+0000 7f1987a6a700 1 -- 172.21.15.82:0/1048072813 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f196c0a2cb0 msgr2=0x7f196c0a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:12:45.230 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:45.230+0000 7f1987a6a700 1 --2- 172.21.15.82:0/1048072813 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f196c0a2cb0 0x7f196c0a51c0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f197c000c00 tx=0x7f197c01a040 comp rx=0 tx=0).stop 2024-05-20T16:12:45.231 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:45.230+0000 7f1987a6a700 1 -- 172.21.15.82:0/1048072813 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1980106c70 msgr2=0x7f198019a300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:12:45.231 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:45.230+0000 7f1987a6a700 1 --2- 172.21.15.82:0/1048072813 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1980106c70 0x7f198019a300 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f197000d980 tx=0x7f197000dc90 comp rx=0 tx=0).stop 2024-05-20T16:12:45.231 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:45.230+0000 7f1986007700 1 -- 172.21.15.82:0/1048072813 reap_dead start 2024-05-20T16:12:45.231 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:45.230+0000 7f1987a6a700 1 -- 172.21.15.82:0/1048072813 shutdown_connections 2024-05-20T16:12:45.231 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:45.230+0000 7f1987a6a700 1 -- 172.21.15.82:0/1048072813 >> 172.21.15.82:0/1048072813 conn(0x7f19800fdc40 msgr2=0x7f1980100b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:12:45.231 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:45.230+0000 7f1987a6a700 1 -- 172.21.15.82:0/1048072813 shutdown_connections 2024-05-20T16:12:45.231 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:45.230+0000 7f1987a6a700 1 -- 172.21.15.82:0/1048072813 wait complete. 2024-05-20T16:12:45.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:12:45 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]: dispatch 2024-05-20T16:12:45.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:12:45 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 4]}]: dispatch 2024-05-20T16:12:45.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:12:45 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [3, 9]}]: dispatch 2024-05-20T16:12:45.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:12:45 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]: dispatch 2024-05-20T16:12:45.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:12:45 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 8]}]: dispatch 2024-05-20T16:12:45.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:12:45 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [5, 9]}]: dispatch 2024-05-20T16:12:45.360 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:12:45 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [9, 1]}]: dispatch 2024-05-20T16:12:45.360 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:12:45 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.5", "id": [9, 10]}]: dispatch 2024-05-20T16:12:45.360 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:12:45 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [5, 8]}]: dispatch 2024-05-20T16:12:45.360 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:12:45 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [9, 8]}]: dispatch 2024-05-20T16:12:45.562 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:12:45 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]: dispatch 2024-05-20T16:12:45.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:12:45 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 4]}]: dispatch 2024-05-20T16:12:45.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:12:45 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [3, 9]}]: dispatch 2024-05-20T16:12:45.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:12:45 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]: dispatch 2024-05-20T16:12:45.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:12:45 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 8]}]: dispatch 2024-05-20T16:12:45.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:12:45 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [5, 9]}]: dispatch 2024-05-20T16:12:45.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:12:45 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [9, 1]}]: dispatch 2024-05-20T16:12:45.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:12:45 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.5", "id": [9, 10]}]: dispatch 2024-05-20T16:12:45.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:12:45 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [5, 8]}]: dispatch 2024-05-20T16:12:45.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:12:45 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [9, 8]}]: dispatch 2024-05-20T16:12:45.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:12:45 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]: dispatch 2024-05-20T16:12:45.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:12:45 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 4]}]: dispatch 2024-05-20T16:12:45.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:12:45 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [3, 9]}]: dispatch 2024-05-20T16:12:45.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:12:45 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]: dispatch 2024-05-20T16:12:45.637 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:12:45 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 8]}]: dispatch 2024-05-20T16:12:45.637 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:12:45 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [5, 9]}]: dispatch 2024-05-20T16:12:45.637 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:12:45 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [9, 1]}]: dispatch 2024-05-20T16:12:45.637 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:12:45 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.5", "id": [9, 10]}]: dispatch 2024-05-20T16:12:45.637 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:12:45 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [5, 8]}]: dispatch 2024-05-20T16:12:45.638 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:12:45 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [9, 8]}]: dispatch 2024-05-20T16:12:46.475 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:12:46 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]': finished 2024-05-20T16:12:46.475 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:12:46 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 4]}]': finished 2024-05-20T16:12:46.475 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:12:46 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [3, 9]}]': finished 2024-05-20T16:12:46.475 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:12:46 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]': finished 2024-05-20T16:12:46.475 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:12:46 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 8]}]': finished 2024-05-20T16:12:46.476 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:12:46 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [5, 9]}]': finished 2024-05-20T16:12:46.476 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:12:46 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [9, 1]}]': finished 2024-05-20T16:12:46.476 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:12:46 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.5", "id": [9, 10]}]': finished 2024-05-20T16:12:46.476 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:12:46 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [5, 8]}]': finished 2024-05-20T16:12:46.476 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:12:46 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [9, 8]}]': finished 2024-05-20T16:12:46.476 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:12:46 smithi082 ceph-mon[32238]: osdmap e92: 12 total, 12 up, 12 in 2024-05-20T16:12:46.476 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:12:46 smithi082 ceph-mon[32238]: pgmap v322: 129 pgs: 129 active+clean; 641 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 307 B/s wr, 6 op/s 2024-05-20T16:12:46.477 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:12:46 smithi082 ceph-mon[32238]: osdmap e93: 12 total, 12 up, 12 in 2024-05-20T16:12:46.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:12:46 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]': finished 2024-05-20T16:12:46.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:12:46 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 4]}]': finished 2024-05-20T16:12:46.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:12:46 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [3, 9]}]': finished 2024-05-20T16:12:46.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:12:46 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]': finished 2024-05-20T16:12:46.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:12:46 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 8]}]': finished 2024-05-20T16:12:46.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:12:46 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [5, 9]}]': finished 2024-05-20T16:12:46.564 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:12:46 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [9, 1]}]': finished 2024-05-20T16:12:46.564 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:12:46 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.5", "id": [9, 10]}]': finished 2024-05-20T16:12:46.564 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:12:46 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [5, 8]}]': finished 2024-05-20T16:12:46.564 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:12:46 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [9, 8]}]': finished 2024-05-20T16:12:46.564 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:12:46 smithi160 ceph-mon[36347]: osdmap e92: 12 total, 12 up, 12 in 2024-05-20T16:12:46.564 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:12:46 smithi160 ceph-mon[36347]: pgmap v322: 129 pgs: 129 active+clean; 641 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 307 B/s wr, 6 op/s 2024-05-20T16:12:46.565 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:12:46 smithi160 ceph-mon[36347]: osdmap e93: 12 total, 12 up, 12 in 2024-05-20T16:12:46.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:12:46 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]': finished 2024-05-20T16:12:46.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:12:46 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 4]}]': finished 2024-05-20T16:12:46.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:12:46 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [3, 9]}]': finished 2024-05-20T16:12:46.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:12:46 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]': finished 2024-05-20T16:12:46.637 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:12:46 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 8]}]': finished 2024-05-20T16:12:46.637 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:12:46 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [5, 9]}]': finished 2024-05-20T16:12:46.637 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:12:46 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [9, 1]}]': finished 2024-05-20T16:12:46.637 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:12:46 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.5", "id": [9, 10]}]': finished 2024-05-20T16:12:46.637 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:12:46 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [5, 8]}]': finished 2024-05-20T16:12:46.637 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:12:46 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [9, 8]}]': finished 2024-05-20T16:12:46.638 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:12:46 smithi184 ceph-mon[36209]: osdmap e92: 12 total, 12 up, 12 in 2024-05-20T16:12:46.638 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:12:46 smithi184 ceph-mon[36209]: pgmap v322: 129 pgs: 129 active+clean; 641 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 307 B/s wr, 6 op/s 2024-05-20T16:12:46.638 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:12:46 smithi184 ceph-mon[36209]: osdmap e93: 12 total, 12 up, 12 in 2024-05-20T16:12:47.431 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:47.431+0000 7fa8759dd700 1 -- 172.21.15.82:0/1145325010 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa870106c70 msgr2=0x7fa870107080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:12:47.431 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:47.431+0000 7fa8759dd700 1 --2- 172.21.15.82:0/1145325010 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa870106c70 0x7fa870107080 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fa8640099c0 tx=0x7fa864009cd0 comp rx=0 tx=0).stop 2024-05-20T16:12:47.431 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:47.431+0000 7fa8759dd700 1 -- 172.21.15.82:0/1145325010 shutdown_connections 2024-05-20T16:12:47.431 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:47.431+0000 7fa8759dd700 1 --2- 172.21.15.82:0/1145325010 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa870102340 0x7fa870102750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:47.431 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:47.431+0000 7fa8759dd700 1 --2- 172.21.15.82:0/1145325010 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa8701075c0 0x7fa87010ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:47.432 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:47.431+0000 7fa8759dd700 1 --2- 172.21.15.82:0/1145325010 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa870106c70 0x7fa870107080 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:47.432 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:47.431+0000 7fa8759dd700 1 -- 172.21.15.82:0/1145325010 >> 172.21.15.82:0/1145325010 conn(0x7fa8700fdc40 msgr2=0x7fa870100060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:12:47.432 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:47.431+0000 7fa8759dd700 1 -- 172.21.15.82:0/1145325010 shutdown_connections 2024-05-20T16:12:47.432 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:47.431+0000 7fa8759dd700 1 -- 172.21.15.82:0/1145325010 wait complete. 2024-05-20T16:12:47.432 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:47.432+0000 7fa8759dd700 1 Processor -- start 2024-05-20T16:12:47.433 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:47.432+0000 7fa8759dd700 1 -- start start 2024-05-20T16:12:47.434 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:47.432+0000 7fa8759dd700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa870102340 0x7fa870199d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:12:47.434 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:47.432+0000 7fa8759dd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa870106c70 0x7fa87019a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:12:47.434 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:47.432+0000 7fa8759dd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa8701075c0 0x7fa87019e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:12:47.434 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:47.432+0000 7fa8759dd700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fa870112390 con 0x7fa8701075c0 2024-05-20T16:12:47.434 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:47.432+0000 7fa8759dd700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fa870112210 con 0x7fa870106c70 2024-05-20T16:12:47.434 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:47.432+0000 7fa8759dd700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa870112510 con 0x7fa870102340 2024-05-20T16:12:47.434 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:47.432+0000 7fa86f7fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa8701075c0 0x7fa87019e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:12:47.434 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:47.432+0000 7fa86f7fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa8701075c0 0x7fa87019e5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58234/0 (socket says 172.21.15.82:58234) 2024-05-20T16:12:47.434 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:47.432+0000 7fa86f7fe700 1 -- 172.21.15.82:0/2002021028 learned_addr learned my addr 172.21.15.82:0/2002021028 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:12:47.434 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:47.432+0000 7fa86effd700 1 --2- 172.21.15.82:0/2002021028 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa870102340 0x7fa870199d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:12:47.434 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:47.432+0000 7fa86e7fc700 1 --2- 172.21.15.82:0/2002021028 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa870106c70 0x7fa87019a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:12:47.434 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:47.433+0000 7fa86f7fe700 1 -- 172.21.15.82:0/2002021028 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa870102340 msgr2=0x7fa870199d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:12:47.435 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:47.433+0000 7fa86f7fe700 1 --2- 172.21.15.82:0/2002021028 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa870102340 0x7fa870199d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:47.435 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:47.433+0000 7fa86f7fe700 1 -- 172.21.15.82:0/2002021028 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa870106c70 msgr2=0x7fa87019a330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:12:47.435 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:47.433+0000 7fa86f7fe700 1 --2- 172.21.15.82:0/2002021028 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa870106c70 0x7fa87019a330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:47.435 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:47.433+0000 7fa86f7fe700 1 -- 172.21.15.82:0/2002021028 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa87019ec60 con 0x7fa8701075c0 2024-05-20T16:12:47.435 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:47.433+0000 7fa86f7fe700 1 --2- 172.21.15.82:0/2002021028 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa8701075c0 0x7fa87019e5c0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fa868007ee0 tx=0x7fa86800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:12:47.435 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:47.433+0000 7fa8749db700 1 -- 172.21.15.82:0/2002021028 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa86800dbd0 con 0x7fa8701075c0 2024-05-20T16:12:47.435 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:47.433+0000 7fa8759dd700 1 -- 172.21.15.82:0/2002021028 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa87019ef50 con 0x7fa8701075c0 2024-05-20T16:12:47.435 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:47.433+0000 7fa8749db700 1 -- 172.21.15.82:0/2002021028 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa86800dd70 con 0x7fa8701075c0 2024-05-20T16:12:47.435 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:47.433+0000 7fa8749db700 1 -- 172.21.15.82:0/2002021028 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa868005a60 con 0x7fa8701075c0 2024-05-20T16:12:47.435 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:47.433+0000 7fa8759dd700 1 -- 172.21.15.82:0/2002021028 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8701a6870 con 0x7fa8701075c0 2024-05-20T16:12:47.435 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:47.434+0000 7fa8759dd700 1 -- 172.21.15.82:0/2002021028 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa870062dd0 con 0x7fa8701075c0 2024-05-20T16:12:47.436 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:47.435+0000 7fa8749db700 1 -- 172.21.15.82:0/2002021028 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fa86801a020 con 0x7fa8701075c0 2024-05-20T16:12:47.437 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:47.436+0000 7fa8749db700 1 --2- 172.21.15.82:0/2002021028 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fa8580a2cb0 0x7fa8580a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:12:47.437 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:47.436+0000 7fa86effd700 1 --2- 172.21.15.82:0/2002021028 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fa8580a2cb0 0x7fa8580a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:12:47.437 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:47.436+0000 7fa8749db700 1 -- 172.21.15.82:0/2002021028 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9092+0+0 (secure 0 0 0) 0x7fa8680cf6e0 con 0x7fa8701075c0 2024-05-20T16:12:47.437 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:47.436+0000 7fa8749db700 1 --2- 172.21.15.82:0/2002021028 >> [v2:172.21.15.160:6816/2340683378,v1:172.21.15.160:6817/2340683378] conn(0x7fa8580ade60 0x7fa8580b0350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:12:47.437 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:47.436+0000 7fa8749db700 1 -- 172.21.15.82:0/2002021028 --> [v2:172.21.15.160:6816/2340683378,v1:172.21.15.160:6817/2340683378] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa868025e50 con 0x7fa8580ade60 2024-05-20T16:12:47.437 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:47.436+0000 7fa8749db700 1 -- 172.21.15.82:0/2002021028 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_get_version_reply(handle=1 version=93) ==== 24+0+0 (secure 0 0 0) 0x7fa8680cfb30 con 0x7fa8701075c0 2024-05-20T16:12:47.437 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:47.436+0000 7fa86e7fc700 1 --2- 172.21.15.82:0/2002021028 >> [v2:172.21.15.160:6816/2340683378,v1:172.21.15.160:6817/2340683378] conn(0x7fa8580ade60 0x7fa8580b0350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:12:47.437 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:47.437+0000 7fa86effd700 1 --2- 172.21.15.82:0/2002021028 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fa8580a2cb0 0x7fa8580a51c0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fa860009a10 tx=0x7fa860009190 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:12:47.437 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:47.437+0000 7fa86e7fc700 1 --2- 172.21.15.82:0/2002021028 >> [v2:172.21.15.160:6816/2340683378,v1:172.21.15.160:6817/2340683378] conn(0x7fa8580ade60 0x7fa8580b0350 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:12:47.438 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:47.438+0000 7fa8749db700 1 -- 172.21.15.82:0/2002021028 <== osd.7 v2:172.21.15.160:6816/2340683378 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa868025e50 con 0x7fa8580ade60 2024-05-20T16:12:47.461 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:47.461+0000 7fa8759dd700 1 -- 172.21.15.82:0/2002021028 --> [v2:172.21.15.160:6816/2340683378,v1:172.21.15.160:6817/2340683378] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fa8700631d0 con 0x7fa8580ade60 2024-05-20T16:12:47.461 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:47.462+0000 7fa8749db700 1 -- 172.21.15.82:0/2002021028 <== osd.7 v2:172.21.15.160:6816/2340683378 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fa8700631d0 con 0x7fa8580ade60 2024-05-20T16:12:47.462 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:47.462+0000 7fa8759dd700 1 -- 172.21.15.82:0/2002021028 >> [v2:172.21.15.160:6816/2340683378,v1:172.21.15.160:6817/2340683378] conn(0x7fa8580ade60 msgr2=0x7fa8580b0350 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:12:47.462 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:47.462+0000 7fa8759dd700 1 --2- 172.21.15.82:0/2002021028 >> [v2:172.21.15.160:6816/2340683378,v1:172.21.15.160:6817/2340683378] conn(0x7fa8580ade60 0x7fa8580b0350 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:47.462 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:47.462+0000 7fa8759dd700 1 -- 172.21.15.82:0/2002021028 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fa8580a2cb0 msgr2=0x7fa8580a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:12:47.462 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:47.462+0000 7fa8759dd700 1 --2- 172.21.15.82:0/2002021028 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fa8580a2cb0 0x7fa8580a51c0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fa860009a10 tx=0x7fa860009190 comp rx=0 tx=0).stop 2024-05-20T16:12:47.462 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:47.462+0000 7fa8759dd700 1 -- 172.21.15.82:0/2002021028 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa8701075c0 msgr2=0x7fa87019e5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:12:47.462 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:47.462+0000 7fa8759dd700 1 --2- 172.21.15.82:0/2002021028 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa8701075c0 0x7fa87019e5c0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fa868007ee0 tx=0x7fa86800d3b0 comp rx=0 tx=0).stop 2024-05-20T16:12:47.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:47.462+0000 7fa86f7fe700 1 -- 172.21.15.82:0/2002021028 reap_dead start 2024-05-20T16:12:47.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:47.462+0000 7fa8759dd700 1 -- 172.21.15.82:0/2002021028 shutdown_connections 2024-05-20T16:12:47.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:47.462+0000 7fa8759dd700 1 -- 172.21.15.82:0/2002021028 >> 172.21.15.82:0/2002021028 conn(0x7fa8700fdc40 msgr2=0x7fa87006a710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:12:47.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:47.462+0000 7fa8759dd700 1 -- 172.21.15.82:0/2002021028 shutdown_connections 2024-05-20T16:12:47.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:47.462+0000 7fa8759dd700 1 -- 172.21.15.82:0/2002021028 wait complete. 2024-05-20T16:12:47.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:12:47 smithi082 ceph-mon[32238]: pgmap v324: 129 pgs: 1 peering, 128 active+clean; 641 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 3 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-05-20T16:12:47.635 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:12:47 smithi184 ceph-mon[36209]: pgmap v324: 129 pgs: 1 peering, 128 active+clean; 641 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 3 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-05-20T16:12:47.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:12:47 smithi160 ceph-mon[36347]: pgmap v324: 129 pgs: 1 peering, 128 active+clean; 641 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 3 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-05-20T16:12:47.863 INFO:teuthology.orchestra.run.smithi082.stdout:201863462963 2024-05-20T16:12:47.864 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell --fsid 59d1600a-16c2-11ef-bc9a-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-05-20T16:12:48.524 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:12:48 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T16:12:48.562 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:12:48 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T16:12:48.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:12:48 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T16:12:48.657 INFO:teuthology.orchestra.run.smithi082.stdout:146028888125 2024-05-20T16:12:48.658 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell --fsid 59d1600a-16c2-11ef-bc9a-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-05-20T16:12:49.881 INFO:teuthology.orchestra.run.smithi082.stdout:223338299438 2024-05-20T16:12:49.881 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell --fsid 59d1600a-16c2-11ef-bc9a-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-05-20T16:12:50.492 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:12:50 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:12:50.492 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:12:50 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:12:50.492 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:12:50 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:12:50.492 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:12:50 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:12:50.492 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:12:50 smithi082 ceph-mon[32238]: pgmap v325: 129 pgs: 1 peering, 128 active+clean; 641 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 7 op/s; 14 B/s, 0 keys/s, 0 objects/s recovering 2024-05-20T16:12:50.525 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T16:12:50.562 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:12:50 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:12:50.562 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:12:50 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:12:50.562 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:12:50 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:12:50.562 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:12:50 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:12:50.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:12:50 smithi160 ceph-mon[36347]: pgmap v325: 129 pgs: 1 peering, 128 active+clean; 641 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 7 op/s; 14 B/s, 0 keys/s, 0 objects/s recovering 2024-05-20T16:12:50.635 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:12:50 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:12:50.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:12:50 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:12:50.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:12:50 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:12:50.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:12:50 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:12:50.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:12:50 smithi184 ceph-mon[36209]: pgmap v325: 129 pgs: 1 peering, 128 active+clean; 641 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 7 op/s; 14 B/s, 0 keys/s, 0 objects/s recovering 2024-05-20T16:12:51.591 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:12:51 smithi082 ceph-mon[32238]: pgmap v326: 129 pgs: 129 active+clean; 641 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s; 14 B/s, 2 keys/s, 0 objects/s recovering 2024-05-20T16:12:51.635 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:12:51 smithi184 ceph-mon[36209]: pgmap v326: 129 pgs: 129 active+clean; 641 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s; 14 B/s, 2 keys/s, 0 objects/s recovering 2024-05-20T16:12:51.776 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:51.776+0000 7f1f3a337700 1 -- 172.21.15.82:0/960301365 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1f34101840 msgr2=0x7f1f34101c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:12:51.776 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:51.776+0000 7f1f3a337700 1 --2- 172.21.15.82:0/960301365 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1f34101840 0x7f1f34101c50 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f1f1c009990 tx=0x7f1f1c009ca0 comp rx=0 tx=0).stop 2024-05-20T16:12:51.776 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:51.776+0000 7f1f3a337700 1 -- 172.21.15.82:0/960301365 shutdown_connections 2024-05-20T16:12:51.776 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:51.776+0000 7f1f3a337700 1 --2- 172.21.15.82:0/960301365 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1f34102190 0x7f1f3410e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:51.776 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:51.776+0000 7f1f3a337700 1 --2- 172.21.15.82:0/960301365 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1f34107870 0x7f1f34107c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:51.776 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:51.776+0000 7f1f3a337700 1 --2- 172.21.15.82:0/960301365 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1f34101840 0x7f1f34101c50 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:51.776 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:51.776+0000 7f1f3a337700 1 -- 172.21.15.82:0/960301365 >> 172.21.15.82:0/960301365 conn(0x7f1f340fd3a0 msgr2=0x7f1f340ff7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:12:51.777 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:51.776+0000 7f1f3a337700 1 -- 172.21.15.82:0/960301365 shutdown_connections 2024-05-20T16:12:51.777 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:51.776+0000 7f1f3a337700 1 -- 172.21.15.82:0/960301365 wait complete. 2024-05-20T16:12:51.778 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:51.777+0000 7f1f3a337700 1 Processor -- start 2024-05-20T16:12:51.778 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:51.777+0000 7f1f3a337700 1 -- start start 2024-05-20T16:12:51.778 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:51.777+0000 7f1f3a337700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1f34101840 0x7f1f34199dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:12:51.778 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:51.777+0000 7f1f3a337700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1f34102190 0x7f1f3419a360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:12:51.779 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:51.777+0000 7f1f3a337700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1f34107870 0x7f1f3419e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:12:51.779 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:51.777+0000 7f1f3a337700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f1f341112c0 con 0x7f1f34107870 2024-05-20T16:12:51.779 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:51.777+0000 7f1f3a337700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f1f34111140 con 0x7f1f34102190 2024-05-20T16:12:51.779 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:51.777+0000 7f1f3a337700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f1f34111440 con 0x7f1f34101840 2024-05-20T16:12:51.779 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:51.777+0000 7f1f388d4700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1f34107870 0x7f1f3419e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:12:51.779 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:51.777+0000 7f1f388d4700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1f34107870 0x7f1f3419e5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:47352/0 (socket says 172.21.15.82:47352) 2024-05-20T16:12:51.779 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:51.777+0000 7f1f388d4700 1 -- 172.21.15.82:0/3804317257 learned_addr learned my addr 172.21.15.82:0/3804317257 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:12:51.779 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:51.777+0000 7f1f33fff700 1 --2- 172.21.15.82:0/3804317257 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1f34101840 0x7f1f34199dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:12:51.779 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:51.777+0000 7f1f2bfff700 1 --2- 172.21.15.82:0/3804317257 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1f34102190 0x7f1f3419a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:12:51.779 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:51.777+0000 7f1f388d4700 1 -- 172.21.15.82:0/3804317257 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1f34101840 msgr2=0x7f1f34199dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:12:51.779 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:51.777+0000 7f1f388d4700 1 --2- 172.21.15.82:0/3804317257 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1f34101840 0x7f1f34199dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:51.779 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:51.777+0000 7f1f388d4700 1 -- 172.21.15.82:0/3804317257 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1f34102190 msgr2=0x7f1f3419a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:12:51.779 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:51.777+0000 7f1f388d4700 1 --2- 172.21.15.82:0/3804317257 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1f34102190 0x7f1f3419a360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:51.780 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:51.777+0000 7f1f388d4700 1 -- 172.21.15.82:0/3804317257 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f3419ec90 con 0x7f1f34107870 2024-05-20T16:12:51.780 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:51.778+0000 7f1f388d4700 1 --2- 172.21.15.82:0/3804317257 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1f34107870 0x7f1f3419e5f0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f1f2c00d3a0 tx=0x7f1f2c00d6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:12:51.780 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:51.778+0000 7f1f31ffb700 1 -- 172.21.15.82:0/3804317257 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f2c017020 con 0x7f1f34107870 2024-05-20T16:12:51.780 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:51.778+0000 7f1f31ffb700 1 -- 172.21.15.82:0/3804317257 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1f2c004db0 con 0x7f1f34107870 2024-05-20T16:12:51.780 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:51.778+0000 7f1f3a337700 1 -- 172.21.15.82:0/3804317257 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f3419ef80 con 0x7f1f34107870 2024-05-20T16:12:51.780 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:51.778+0000 7f1f31ffb700 1 -- 172.21.15.82:0/3804317257 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f2c008b20 con 0x7f1f34107870 2024-05-20T16:12:51.780 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:51.778+0000 7f1f3a337700 1 -- 172.21.15.82:0/3804317257 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f341a68a0 con 0x7f1f34107870 2024-05-20T16:12:51.780 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:51.779+0000 7f1f3a337700 1 -- 172.21.15.82:0/3804317257 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f1f34062dd0 con 0x7f1f34107870 2024-05-20T16:12:51.782 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:51.780+0000 7f1f31ffb700 1 -- 172.21.15.82:0/3804317257 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f1f2c012070 con 0x7f1f34107870 2024-05-20T16:12:51.782 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:51.781+0000 7f1f31ffb700 1 --2- 172.21.15.82:0/3804317257 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f1f200a2cb0 0x7f1f200a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:12:51.782 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:51.781+0000 7f1f33fff700 1 --2- 172.21.15.82:0/3804317257 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f1f200a2cb0 0x7f1f200a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:12:51.782 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:51.781+0000 7f1f31ffb700 1 -- 172.21.15.82:0/3804317257 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9092+0+0 (secure 0 0 0) 0x7f1f2c0cfdb0 con 0x7f1f34107870 2024-05-20T16:12:51.782 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:51.781+0000 7f1f31ffb700 1 --2- 172.21.15.82:0/3804317257 >> [v2:172.21.15.160:6800/1457003568,v1:172.21.15.160:6801/1457003568] conn(0x7f1f200ade60 0x7f1f200b0350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:12:51.782 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:51.781+0000 7f1f31ffb700 1 -- 172.21.15.82:0/3804317257 --> [v2:172.21.15.160:6800/1457003568,v1:172.21.15.160:6801/1457003568] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f1f200b0a20 con 0x7f1f200ade60 2024-05-20T16:12:51.782 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:51.781+0000 7f1f31ffb700 1 -- 172.21.15.82:0/3804317257 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_get_version_reply(handle=1 version=93) ==== 24+0+0 (secure 0 0 0) 0x7f1f2c0d0200 con 0x7f1f34107870 2024-05-20T16:12:51.782 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:51.781+0000 7f1f33fff700 1 --2- 172.21.15.82:0/3804317257 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f1f200a2cb0 0x7f1f200a51c0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f1f34102980 tx=0x7f1f24009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:12:51.782 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:51.781+0000 7f1f2bfff700 1 --2- 172.21.15.82:0/3804317257 >> [v2:172.21.15.160:6800/1457003568,v1:172.21.15.160:6801/1457003568] conn(0x7f1f200ade60 0x7f1f200b0350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:12:51.782 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:51.782+0000 7f1f2bfff700 1 --2- 172.21.15.82:0/3804317257 >> [v2:172.21.15.160:6800/1457003568,v1:172.21.15.160:6801/1457003568] conn(0x7f1f200ade60 0x7f1f200b0350 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:12:51.783 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:51.783+0000 7f1f31ffb700 1 -- 172.21.15.82:0/3804317257 <== osd.1 v2:172.21.15.160:6800/1457003568 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1f200b0a20 con 0x7f1f200ade60 2024-05-20T16:12:51.798 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:51.799+0000 7f1f3a337700 1 -- 172.21.15.82:0/3804317257 --> [v2:172.21.15.160:6800/1457003568,v1:172.21.15.160:6801/1457003568] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f1f340631d0 con 0x7f1f200ade60 2024-05-20T16:12:51.799 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:51.799+0000 7f1f31ffb700 1 -- 172.21.15.82:0/3804317257 <== osd.1 v2:172.21.15.160:6800/1457003568 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f1f340631d0 con 0x7f1f200ade60 2024-05-20T16:12:51.800 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:51.799+0000 7f1f3a337700 1 -- 172.21.15.82:0/3804317257 >> [v2:172.21.15.160:6800/1457003568,v1:172.21.15.160:6801/1457003568] conn(0x7f1f200ade60 msgr2=0x7f1f200b0350 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:12:51.800 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:51.799+0000 7f1f3a337700 1 --2- 172.21.15.82:0/3804317257 >> [v2:172.21.15.160:6800/1457003568,v1:172.21.15.160:6801/1457003568] conn(0x7f1f200ade60 0x7f1f200b0350 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:51.800 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:51.799+0000 7f1f3a337700 1 -- 172.21.15.82:0/3804317257 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f1f200a2cb0 msgr2=0x7f1f200a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:12:51.800 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:51.799+0000 7f1f3a337700 1 --2- 172.21.15.82:0/3804317257 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f1f200a2cb0 0x7f1f200a51c0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f1f34102980 tx=0x7f1f24009260 comp rx=0 tx=0).stop 2024-05-20T16:12:51.800 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:51.799+0000 7f1f3a337700 1 -- 172.21.15.82:0/3804317257 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1f34107870 msgr2=0x7f1f3419e5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:12:51.800 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:51.799+0000 7f1f3a337700 1 --2- 172.21.15.82:0/3804317257 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1f34107870 0x7f1f3419e5f0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f1f2c00d3a0 tx=0x7f1f2c00d6b0 comp rx=0 tx=0).stop 2024-05-20T16:12:51.800 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:51.800+0000 7f1f388d4700 1 -- 172.21.15.82:0/3804317257 reap_dead start 2024-05-20T16:12:51.800 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:51.800+0000 7f1f3a337700 1 -- 172.21.15.82:0/3804317257 shutdown_connections 2024-05-20T16:12:51.800 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:51.800+0000 7f1f3a337700 1 -- 172.21.15.82:0/3804317257 >> 172.21.15.82:0/3804317257 conn(0x7f1f340fd3a0 msgr2=0x7f1f340ff7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:12:51.800 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:51.800+0000 7f1f3a337700 1 -- 172.21.15.82:0/3804317257 shutdown_connections 2024-05-20T16:12:51.800 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:51.800+0000 7f1f3a337700 1 -- 172.21.15.82:0/3804317257 wait complete. 2024-05-20T16:12:51.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:12:51 smithi160 ceph-mon[36347]: pgmap v326: 129 pgs: 129 active+clean; 641 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s; 14 B/s, 2 keys/s, 0 objects/s recovering 2024-05-20T16:12:51.985 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:51.985+0000 7f1d68fbf700 1 -- 172.21.15.82:0/1705710152 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1d64102b20 msgr2=0x7f1d64102f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:12:51.985 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:51.985+0000 7f1d68fbf700 1 --2- 172.21.15.82:0/1705710152 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1d64102b20 0x7f1d64102f30 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f1d540099c0 tx=0x7f1d54009cd0 comp rx=0 tx=0).stop 2024-05-20T16:12:51.985 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:51.985+0000 7f1d68fbf700 1 -- 172.21.15.82:0/1705710152 shutdown_connections 2024-05-20T16:12:51.985 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:51.985+0000 7f1d68fbf700 1 --2- 172.21.15.82:0/1705710152 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1d641075e0 0x7f1d6410efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:51.985 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:51.985+0000 7f1d68fbf700 1 --2- 172.21.15.82:0/1705710152 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1d64102b20 0x7f1d64102f30 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:51.985 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:51.985+0000 7f1d68fbf700 1 --2- 172.21.15.82:0/1705710152 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1d64106c90 0x7f1d641070a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:51.985 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:51.985+0000 7f1d68fbf700 1 -- 172.21.15.82:0/1705710152 >> 172.21.15.82:0/1705710152 conn(0x7f1d640fd570 msgr2=0x7f1d640ff990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:12:51.985 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:51.985+0000 7f1d68fbf700 1 -- 172.21.15.82:0/1705710152 shutdown_connections 2024-05-20T16:12:51.986 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:51.985+0000 7f1d68fbf700 1 -- 172.21.15.82:0/1705710152 wait complete. 2024-05-20T16:12:51.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:51.986+0000 7f1d68fbf700 1 Processor -- start 2024-05-20T16:12:51.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:51.986+0000 7f1d68fbf700 1 -- start start 2024-05-20T16:12:51.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:51.986+0000 7f1d68fbf700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1d64102b20 0x7f1d6419abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:12:51.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:51.986+0000 7f1d68fbf700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1d64106c90 0x7f1d6419b110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:12:51.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:51.986+0000 7f1d68fbf700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1d641075e0 0x7f1d6419b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:12:51.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:51.986+0000 7f1d68fbf700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f1d64112350 con 0x7f1d64102b20 2024-05-20T16:12:51.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:51.986+0000 7f1d68fbf700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f1d641121d0 con 0x7f1d64106c90 2024-05-20T16:12:51.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:51.986+0000 7f1d68fbf700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f1d641124d0 con 0x7f1d641075e0 2024-05-20T16:12:51.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:51.986+0000 7f1d6259c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1d64102b20 0x7f1d6419abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:12:51.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:51.986+0000 7f1d6259c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1d64102b20 0x7f1d6419abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:47364/0 (socket says 172.21.15.82:47364) 2024-05-20T16:12:51.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:51.986+0000 7f1d6259c700 1 -- 172.21.15.82:0/944178056 learned_addr learned my addr 172.21.15.82:0/944178056 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:12:51.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:51.986+0000 7f1d61d9b700 1 --2- 172.21.15.82:0/944178056 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1d64106c90 0x7f1d6419b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:12:51.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:51.986+0000 7f1d62d9d700 1 --2- 172.21.15.82:0/944178056 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1d641075e0 0x7f1d6419b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:12:51.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:51.986+0000 7f1d6259c700 1 -- 172.21.15.82:0/944178056 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1d641075e0 msgr2=0x7f1d6419b670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:12:51.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:51.986+0000 7f1d6259c700 1 --2- 172.21.15.82:0/944178056 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1d641075e0 0x7f1d6419b670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:51.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:51.986+0000 7f1d6259c700 1 -- 172.21.15.82:0/944178056 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1d64106c90 msgr2=0x7f1d6419b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:12:51.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:51.986+0000 7f1d6259c700 1 --2- 172.21.15.82:0/944178056 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1d64106c90 0x7f1d6419b110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:51.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:51.986+0000 7f1d6259c700 1 -- 172.21.15.82:0/944178056 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d641a5f20 con 0x7f1d64102b20 2024-05-20T16:12:51.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:51.987+0000 7f1d6259c700 1 --2- 172.21.15.82:0/944178056 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1d64102b20 0x7f1d6419abd0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f1d54006010 tx=0x7f1d5400f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:12:51.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:51.987+0000 7f1d537fe700 1 -- 172.21.15.82:0/944178056 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d54003680 con 0x7f1d64102b20 2024-05-20T16:12:51.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:51.987+0000 7f1d537fe700 1 -- 172.21.15.82:0/944178056 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1d540055b0 con 0x7f1d64102b20 2024-05-20T16:12:51.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:51.987+0000 7f1d68fbf700 1 -- 172.21.15.82:0/944178056 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d641a61b0 con 0x7f1d64102b20 2024-05-20T16:12:51.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:51.987+0000 7f1d537fe700 1 -- 172.21.15.82:0/944178056 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d540173f0 con 0x7f1d64102b20 2024-05-20T16:12:51.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:51.987+0000 7f1d68fbf700 1 -- 172.21.15.82:0/944178056 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d641a6600 con 0x7f1d64102b20 2024-05-20T16:12:51.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:51.988+0000 7f1d68fbf700 1 -- 172.21.15.82:0/944178056 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f1d64062d50 con 0x7f1d64102b20 2024-05-20T16:12:51.990 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:51.989+0000 7f1d537fe700 1 -- 172.21.15.82:0/944178056 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f1d5400fca0 con 0x7f1d64102b20 2024-05-20T16:12:51.990 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:51.990+0000 7f1d537fe700 1 --2- 172.21.15.82:0/944178056 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f1d4c0a2e50 0x7f1d4c0a5360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:12:51.990 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:51.990+0000 7f1d537fe700 1 -- 172.21.15.82:0/944178056 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9092+0+0 (secure 0 0 0) 0x7f1d540d0cc0 con 0x7f1d64102b20 2024-05-20T16:12:51.990 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:51.990+0000 7f1d61d9b700 1 --2- 172.21.15.82:0/944178056 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f1d4c0a2e50 0x7f1d4c0a5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:12:51.990 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:51.990+0000 7f1d537fe700 1 --2- 172.21.15.82:0/944178056 >> [v2:172.21.15.184:6824/3594364650,v1:172.21.15.184:6825/3594364650] conn(0x7f1d4c0aded0 0x7f1d4c0b03c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:12:51.990 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:51.990+0000 7f1d537fe700 1 -- 172.21.15.82:0/944178056 --> [v2:172.21.15.184:6824/3594364650,v1:172.21.15.184:6825/3594364650] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f1d54021e80 con 0x7f1d4c0aded0 2024-05-20T16:12:51.990 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:51.990+0000 7f1d537fe700 1 -- 172.21.15.82:0/944178056 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_get_version_reply(handle=1 version=93) ==== 24+0+0 (secure 0 0 0) 0x7f1d540d1110 con 0x7f1d64102b20 2024-05-20T16:12:51.991 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:51.990+0000 7f1d61d9b700 1 --2- 172.21.15.82:0/944178056 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f1d4c0a2e50 0x7f1d4c0a5360 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f1d64105560 tx=0x7f1d58009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:12:51.991 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:51.990+0000 7f1d62d9d700 1 --2- 172.21.15.82:0/944178056 >> [v2:172.21.15.184:6824/3594364650,v1:172.21.15.184:6825/3594364650] conn(0x7f1d4c0aded0 0x7f1d4c0b03c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:12:51.991 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:51.991+0000 7f1d62d9d700 1 --2- 172.21.15.82:0/944178056 >> [v2:172.21.15.184:6824/3594364650,v1:172.21.15.184:6825/3594364650] conn(0x7f1d4c0aded0 0x7f1d4c0b03c0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:12:51.991 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:51.992+0000 7f1d537fe700 1 -- 172.21.15.82:0/944178056 <== osd.11 v2:172.21.15.184:6824/3594364650 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1d54021e80 con 0x7f1d4c0aded0 2024-05-20T16:12:52.007 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:52.008+0000 7f1d68fbf700 1 -- 172.21.15.82:0/944178056 --> [v2:172.21.15.184:6824/3594364650,v1:172.21.15.184:6825/3594364650] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f1d64063150 con 0x7f1d4c0aded0 2024-05-20T16:12:52.008 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:52.008+0000 7f1d537fe700 1 -- 172.21.15.82:0/944178056 <== osd.11 v2:172.21.15.184:6824/3594364650 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f1d64063150 con 0x7f1d4c0aded0 2024-05-20T16:12:52.009 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:52.009+0000 7f1d68fbf700 1 -- 172.21.15.82:0/944178056 >> [v2:172.21.15.184:6824/3594364650,v1:172.21.15.184:6825/3594364650] conn(0x7f1d4c0aded0 msgr2=0x7f1d4c0b03c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:12:52.009 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:52.009+0000 7f1d68fbf700 1 --2- 172.21.15.82:0/944178056 >> [v2:172.21.15.184:6824/3594364650,v1:172.21.15.184:6825/3594364650] conn(0x7f1d4c0aded0 0x7f1d4c0b03c0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:52.009 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:52.009+0000 7f1d68fbf700 1 -- 172.21.15.82:0/944178056 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f1d4c0a2e50 msgr2=0x7f1d4c0a5360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:12:52.009 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:52.009+0000 7f1d68fbf700 1 --2- 172.21.15.82:0/944178056 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f1d4c0a2e50 0x7f1d4c0a5360 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f1d64105560 tx=0x7f1d58009260 comp rx=0 tx=0).stop 2024-05-20T16:12:52.009 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:52.009+0000 7f1d68fbf700 1 -- 172.21.15.82:0/944178056 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1d64102b20 msgr2=0x7f1d6419abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:12:52.009 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:52.009+0000 7f1d68fbf700 1 --2- 172.21.15.82:0/944178056 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1d64102b20 0x7f1d6419abd0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f1d54006010 tx=0x7f1d5400f690 comp rx=0 tx=0).stop 2024-05-20T16:12:52.009 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:52.009+0000 7f1d62d9d700 1 -- 172.21.15.82:0/944178056 reap_dead start 2024-05-20T16:12:52.009 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:52.009+0000 7f1d68fbf700 1 -- 172.21.15.82:0/944178056 shutdown_connections 2024-05-20T16:12:52.009 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:52.009+0000 7f1d68fbf700 1 -- 172.21.15.82:0/944178056 >> 172.21.15.82:0/944178056 conn(0x7f1d640fd570 msgr2=0x7f1d64106400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:12:52.010 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:52.009+0000 7f1d68fbf700 1 -- 172.21.15.82:0/944178056 shutdown_connections 2024-05-20T16:12:52.010 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:52.009+0000 7f1d68fbf700 1 -- 172.21.15.82:0/944178056 wait complete. 2024-05-20T16:12:52.206 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:52.204+0000 7fbd807cb700 1 -- 172.21.15.82:0/3282274077 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbd78102340 msgr2=0x7fbd78102750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:12:52.206 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:52.204+0000 7fbd807cb700 1 --2- 172.21.15.82:0/3282274077 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbd78102340 0x7fbd78102750 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fbd740099c0 tx=0x7fbd74009cd0 comp rx=0 tx=0).stop 2024-05-20T16:12:52.206 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:52.204+0000 7fbd807cb700 1 -- 172.21.15.82:0/3282274077 shutdown_connections 2024-05-20T16:12:52.206 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:52.204+0000 7fbd807cb700 1 --2- 172.21.15.82:0/3282274077 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbd781075c0 0x7fbd7810ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:52.206 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:52.204+0000 7fbd807cb700 1 --2- 172.21.15.82:0/3282274077 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbd78102340 0x7fbd78102750 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:52.206 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:52.204+0000 7fbd807cb700 1 --2- 172.21.15.82:0/3282274077 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbd78106c70 0x7fbd78107080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:52.207 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:52.204+0000 7fbd807cb700 1 -- 172.21.15.82:0/3282274077 >> 172.21.15.82:0/3282274077 conn(0x7fbd780fdc70 msgr2=0x7fbd78100090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:12:52.207 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:52.204+0000 7fbd807cb700 1 -- 172.21.15.82:0/3282274077 shutdown_connections 2024-05-20T16:12:52.207 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:52.204+0000 7fbd807cb700 1 -- 172.21.15.82:0/3282274077 wait complete. 2024-05-20T16:12:52.207 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:52.205+0000 7fbd807cb700 1 Processor -- start 2024-05-20T16:12:52.207 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:52.205+0000 7fbd807cb700 1 -- start start 2024-05-20T16:12:52.207 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:52.205+0000 7fbd807cb700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbd78102340 0x7fbd78199db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:12:52.207 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:52.205+0000 7fbd807cb700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbd78106c70 0x7fbd7819a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:12:52.207 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:52.205+0000 7fbd807cb700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbd781075c0 0x7fbd7819e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:12:52.207 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:52.205+0000 7fbd807cb700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fbd781134e0 con 0x7fbd781075c0 2024-05-20T16:12:52.207 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:52.205+0000 7fbd807cb700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fbd78113360 con 0x7fbd78106c70 2024-05-20T16:12:52.207 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:52.205+0000 7fbd807cb700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fbd78113660 con 0x7fbd78102340 2024-05-20T16:12:52.207 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:52.205+0000 7fbd7ed68700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbd781075c0 0x7fbd7819e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:12:52.207 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:52.205+0000 7fbd7ed68700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbd781075c0 0x7fbd7819e5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:47386/0 (socket says 172.21.15.82:47386) 2024-05-20T16:12:52.207 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:52.205+0000 7fbd7ed68700 1 -- 172.21.15.82:0/2040879062 learned_addr learned my addr 172.21.15.82:0/2040879062 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:12:52.208 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:52.205+0000 7fbd7dd66700 1 --2- 172.21.15.82:0/2040879062 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbd78106c70 0x7fbd7819a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:12:52.208 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:52.205+0000 7fbd7e567700 1 --2- 172.21.15.82:0/2040879062 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbd78102340 0x7fbd78199db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:12:52.208 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:52.206+0000 7fbd7ed68700 1 -- 172.21.15.82:0/2040879062 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbd78102340 msgr2=0x7fbd78199db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:12:52.208 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:52.206+0000 7fbd7ed68700 1 --2- 172.21.15.82:0/2040879062 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbd78102340 0x7fbd78199db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:52.208 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:52.206+0000 7fbd7ed68700 1 -- 172.21.15.82:0/2040879062 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbd78106c70 msgr2=0x7fbd7819a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:12:52.208 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:52.206+0000 7fbd7ed68700 1 --2- 172.21.15.82:0/2040879062 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbd78106c70 0x7fbd7819a350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:52.208 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:52.206+0000 7fbd7ed68700 1 -- 172.21.15.82:0/2040879062 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd7819ec80 con 0x7fbd781075c0 2024-05-20T16:12:52.208 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:52.206+0000 7fbd7ed68700 1 --2- 172.21.15.82:0/2040879062 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbd781075c0 0x7fbd7819e5e0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fbd70007ee0 tx=0x7fbd7000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:12:52.208 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:52.206+0000 7fbd6f7fe700 1 -- 172.21.15.82:0/2040879062 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd7000dd20 con 0x7fbd781075c0 2024-05-20T16:12:52.208 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:52.206+0000 7fbd6f7fe700 1 -- 172.21.15.82:0/2040879062 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbd70004020 con 0x7fbd781075c0 2024-05-20T16:12:52.208 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:52.206+0000 7fbd807cb700 1 -- 172.21.15.82:0/2040879062 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd7819ef70 con 0x7fbd781075c0 2024-05-20T16:12:52.208 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:52.206+0000 7fbd6f7fe700 1 -- 172.21.15.82:0/2040879062 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd70015660 con 0x7fbd781075c0 2024-05-20T16:12:52.208 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:52.206+0000 7fbd807cb700 1 -- 172.21.15.82:0/2040879062 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd781a6890 con 0x7fbd781075c0 2024-05-20T16:12:52.209 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:52.207+0000 7fbd807cb700 1 -- 172.21.15.82:0/2040879062 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fbd78062dd0 con 0x7fbd781075c0 2024-05-20T16:12:52.209 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:52.209+0000 7fbd6f7fe700 1 -- 172.21.15.82:0/2040879062 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fbd700041d0 con 0x7fbd781075c0 2024-05-20T16:12:52.209 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:52.210+0000 7fbd6f7fe700 1 --2- 172.21.15.82:0/2040879062 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fbd640a2d30 0x7fbd640a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:12:52.210 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:52.210+0000 7fbd7e567700 1 --2- 172.21.15.82:0/2040879062 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fbd640a2d30 0x7fbd640a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:12:52.210 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:52.210+0000 7fbd6f7fe700 1 -- 172.21.15.82:0/2040879062 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9092+0+0 (secure 0 0 0) 0x7fbd700cfa10 con 0x7fbd781075c0 2024-05-20T16:12:52.210 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:52.210+0000 7fbd6f7fe700 1 --2- 172.21.15.82:0/2040879062 >> [v2:172.21.15.184:6800/4051293934,v1:172.21.15.184:6801/4051293934] conn(0x7fbd640adee0 0x7fbd640b03d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:12:52.210 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:52.210+0000 7fbd6f7fe700 1 -- 172.21.15.82:0/2040879062 --> [v2:172.21.15.184:6800/4051293934,v1:172.21.15.184:6801/4051293934] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fbd70020e50 con 0x7fbd640adee0 2024-05-20T16:12:52.210 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:52.210+0000 7fbd6f7fe700 1 -- 172.21.15.82:0/2040879062 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_get_version_reply(handle=1 version=93) ==== 24+0+0 (secure 0 0 0) 0x7fbd700cfe60 con 0x7fbd781075c0 2024-05-20T16:12:52.210 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:52.210+0000 7fbd7e567700 1 --2- 172.21.15.82:0/2040879062 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fbd640a2d30 0x7fbd640a5240 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fbd74000c00 tx=0x7fbd7401a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:12:52.211 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:52.210+0000 7fbd7dd66700 1 --2- 172.21.15.82:0/2040879062 >> [v2:172.21.15.184:6800/4051293934,v1:172.21.15.184:6801/4051293934] conn(0x7fbd640adee0 0x7fbd640b03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:12:52.211 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:52.211+0000 7fbd7dd66700 1 --2- 172.21.15.82:0/2040879062 >> [v2:172.21.15.184:6800/4051293934,v1:172.21.15.184:6801/4051293934] conn(0x7fbd640adee0 0x7fbd640b03d0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:12:52.211 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:52.212+0000 7fbd6f7fe700 1 -- 172.21.15.82:0/2040879062 <== osd.2 v2:172.21.15.184:6800/4051293934 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fbd70020e50 con 0x7fbd640adee0 2024-05-20T16:12:52.230 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:52.230+0000 7fbd807cb700 1 -- 172.21.15.82:0/2040879062 --> [v2:172.21.15.184:6800/4051293934,v1:172.21.15.184:6801/4051293934] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fbd78063250 con 0x7fbd640adee0 2024-05-20T16:12:52.230 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:52.230+0000 7fbd6f7fe700 1 -- 172.21.15.82:0/2040879062 <== osd.2 v2:172.21.15.184:6800/4051293934 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fbd78063250 con 0x7fbd640adee0 2024-05-20T16:12:52.231 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:52.231+0000 7fbd807cb700 1 -- 172.21.15.82:0/2040879062 >> [v2:172.21.15.184:6800/4051293934,v1:172.21.15.184:6801/4051293934] conn(0x7fbd640adee0 msgr2=0x7fbd640b03d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:12:52.231 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:52.231+0000 7fbd807cb700 1 --2- 172.21.15.82:0/2040879062 >> [v2:172.21.15.184:6800/4051293934,v1:172.21.15.184:6801/4051293934] conn(0x7fbd640adee0 0x7fbd640b03d0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:52.231 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:52.231+0000 7fbd807cb700 1 -- 172.21.15.82:0/2040879062 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fbd640a2d30 msgr2=0x7fbd640a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:12:52.231 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:52.231+0000 7fbd807cb700 1 --2- 172.21.15.82:0/2040879062 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fbd640a2d30 0x7fbd640a5240 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fbd74000c00 tx=0x7fbd7401a040 comp rx=0 tx=0).stop 2024-05-20T16:12:52.231 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:52.231+0000 7fbd807cb700 1 -- 172.21.15.82:0/2040879062 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbd781075c0 msgr2=0x7fbd7819e5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:12:52.231 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:52.231+0000 7fbd807cb700 1 --2- 172.21.15.82:0/2040879062 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbd781075c0 0x7fbd7819e5e0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fbd70007ee0 tx=0x7fbd7000d3b0 comp rx=0 tx=0).stop 2024-05-20T16:12:52.232 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:52.231+0000 7fbd7ed68700 1 -- 172.21.15.82:0/2040879062 reap_dead start 2024-05-20T16:12:52.232 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:52.231+0000 7fbd807cb700 1 -- 172.21.15.82:0/2040879062 shutdown_connections 2024-05-20T16:12:52.232 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:52.231+0000 7fbd807cb700 1 -- 172.21.15.82:0/2040879062 >> 172.21.15.82:0/2040879062 conn(0x7fbd780fdc70 msgr2=0x7fbd781063e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:12:52.232 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:52.231+0000 7fbd807cb700 1 -- 172.21.15.82:0/2040879062 shutdown_connections 2024-05-20T16:12:52.232 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:52.231+0000 7fbd807cb700 1 -- 172.21.15.82:0/2040879062 wait complete. 2024-05-20T16:12:52.715 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:52.714+0000 7f496c5e1700 1 -- 172.21.15.82:0/1143919349 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f49641086d0 msgr2=0x7f4964110030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:12:52.716 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:52.714+0000 7f496c5e1700 1 --2- 172.21.15.82:0/1143919349 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f49641086d0 0x7f4964110030 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f495c0099c0 tx=0x7f495c009cd0 comp rx=0 tx=0).stop 2024-05-20T16:12:52.716 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:52.714+0000 7f496c5e1700 1 -- 172.21.15.82:0/1143919349 shutdown_connections 2024-05-20T16:12:52.716 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:52.714+0000 7f496c5e1700 1 --2- 172.21.15.82:0/1143919349 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4964103450 0x7f4964103860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:52.716 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:52.714+0000 7f496c5e1700 1 --2- 172.21.15.82:0/1143919349 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f49641086d0 0x7f4964110030 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:52.716 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:52.714+0000 7f496c5e1700 1 --2- 172.21.15.82:0/1143919349 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4964107d80 0x7f4964108190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:52.716 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:52.714+0000 7f496c5e1700 1 -- 172.21.15.82:0/1143919349 >> 172.21.15.82:0/1143919349 conn(0x7f49640fed20 msgr2=0x7f4964101170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:12:52.716 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:52.714+0000 7f496c5e1700 1 -- 172.21.15.82:0/1143919349 shutdown_connections 2024-05-20T16:12:52.716 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:52.714+0000 7f496c5e1700 1 -- 172.21.15.82:0/1143919349 wait complete. 2024-05-20T16:12:52.716 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:52.715+0000 7f496c5e1700 1 Processor -- start 2024-05-20T16:12:52.716 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:52.715+0000 7f496c5e1700 1 -- start start 2024-05-20T16:12:52.716 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:52.715+0000 7f496c5e1700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4964103450 0x7f4964199e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:12:52.716 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:52.715+0000 7f496c5e1700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4964107d80 0x7f496419a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:12:52.717 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:52.715+0000 7f496c5e1700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f49641086d0 0x7f496419e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:12:52.717 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:52.715+0000 7f496c5e1700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f4964113540 con 0x7f4964107d80 2024-05-20T16:12:52.717 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:52.715+0000 7f496c5e1700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f49641133c0 con 0x7f4964103450 2024-05-20T16:12:52.717 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:52.715+0000 7f496c5e1700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f49641136c0 con 0x7f49641086d0 2024-05-20T16:12:52.717 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:52.716+0000 7f4969b7c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4964107d80 0x7f496419a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:12:52.717 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:52.716+0000 7f4969b7c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4964107d80 0x7f496419a3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:47396/0 (socket says 172.21.15.82:47396) 2024-05-20T16:12:52.717 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:52.716+0000 7f4969b7c700 1 -- 172.21.15.82:0/2453392280 learned_addr learned my addr 172.21.15.82:0/2453392280 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:12:52.717 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:52.716+0000 7f496a37d700 1 --2- 172.21.15.82:0/2453392280 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4964103450 0x7f4964199e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:12:52.717 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:52.716+0000 7f496ab7e700 1 --2- 172.21.15.82:0/2453392280 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f49641086d0 0x7f496419e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:12:52.717 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:52.716+0000 7f4969b7c700 1 -- 172.21.15.82:0/2453392280 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f49641086d0 msgr2=0x7f496419e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:12:52.717 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:52.716+0000 7f4969b7c700 1 --2- 172.21.15.82:0/2453392280 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f49641086d0 0x7f496419e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:52.718 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:52.716+0000 7f4969b7c700 1 -- 172.21.15.82:0/2453392280 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4964103450 msgr2=0x7f4964199e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:12:52.718 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:52.716+0000 7f4969b7c700 1 --2- 172.21.15.82:0/2453392280 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4964103450 0x7f4964199e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:52.718 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:52.716+0000 7f4969b7c700 1 -- 172.21.15.82:0/2453392280 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f496419ed20 con 0x7f4964107d80 2024-05-20T16:12:52.719 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:52.716+0000 7f4969b7c700 1 --2- 172.21.15.82:0/2453392280 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4964107d80 0x7f496419a3f0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f495400b500 tx=0x7f495400b810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:12:52.720 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:52.716+0000 7f495b7fe700 1 -- 172.21.15.82:0/2453392280 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4954015020 con 0x7f4964107d80 2024-05-20T16:12:52.720 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:52.716+0000 7f496c5e1700 1 -- 172.21.15.82:0/2453392280 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f496419f010 con 0x7f4964107d80 2024-05-20T16:12:52.720 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:52.716+0000 7f495b7fe700 1 -- 172.21.15.82:0/2453392280 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4954011d10 con 0x7f4964107d80 2024-05-20T16:12:52.720 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:52.716+0000 7f495b7fe700 1 -- 172.21.15.82:0/2453392280 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4954010de0 con 0x7f4964107d80 2024-05-20T16:12:52.720 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:52.716+0000 7f496c5e1700 1 -- 172.21.15.82:0/2453392280 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49641a6930 con 0x7f4964107d80 2024-05-20T16:12:52.720 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:52.717+0000 7f496c5e1700 1 -- 172.21.15.82:0/2453392280 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f4964062dd0 con 0x7f4964107d80 2024-05-20T16:12:52.720 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:52.719+0000 7f495b7fe700 1 -- 172.21.15.82:0/2453392280 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f4954018020 con 0x7f4964107d80 2024-05-20T16:12:52.720 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:52.719+0000 7f495b7fe700 1 --2- 172.21.15.82:0/2453392280 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f49500a2cb0 0x7f49500a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:12:52.720 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:52.720+0000 7f495b7fe700 1 -- 172.21.15.82:0/2453392280 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9092+0+0 (secure 0 0 0) 0x7f49540cf130 con 0x7f4964107d80 2024-05-20T16:12:52.720 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:52.720+0000 7f496a37d700 1 --2- 172.21.15.82:0/2453392280 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f49500a2cb0 0x7f49500a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:12:52.721 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:52.720+0000 7f495b7fe700 1 --2- 172.21.15.82:0/2453392280 >> [v2:172.21.15.82:6810/3947485599,v1:172.21.15.82:6811/3947485599] conn(0x7f49500ade60 0x7f49500b0350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:12:52.721 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:52.720+0000 7f495b7fe700 1 -- 172.21.15.82:0/2453392280 --> [v2:172.21.15.82:6810/3947485599,v1:172.21.15.82:6811/3947485599] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4954018ec0 con 0x7f49500ade60 2024-05-20T16:12:52.721 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:52.720+0000 7f495b7fe700 1 -- 172.21.15.82:0/2453392280 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_get_version_reply(handle=1 version=93) ==== 24+0+0 (secure 0 0 0) 0x7f49540cf580 con 0x7f4964107d80 2024-05-20T16:12:52.721 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:52.720+0000 7f496ab7e700 1 --2- 172.21.15.82:0/2453392280 >> [v2:172.21.15.82:6810/3947485599,v1:172.21.15.82:6811/3947485599] conn(0x7f49500ade60 0x7f49500b0350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:12:52.721 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:52.720+0000 7f496a37d700 1 --2- 172.21.15.82:0/2453392280 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f49500a2cb0 0x7f49500a51c0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f496000a400 tx=0x7f496000d040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:12:52.721 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:52.720+0000 7f496ab7e700 1 --2- 172.21.15.82:0/2453392280 >> [v2:172.21.15.82:6810/3947485599,v1:172.21.15.82:6811/3947485599] conn(0x7f49500ade60 0x7f49500b0350 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:12:52.722 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:52.721+0000 7f495b7fe700 1 -- 172.21.15.82:0/2453392280 <== osd.3 v2:172.21.15.82:6810/3947485599 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4954018ec0 con 0x7f49500ade60 2024-05-20T16:12:52.738 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:52.738+0000 7f496c5e1700 1 -- 172.21.15.82:0/2453392280 --> [v2:172.21.15.82:6810/3947485599,v1:172.21.15.82:6811/3947485599] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f4964063250 con 0x7f49500ade60 2024-05-20T16:12:52.738 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:52.738+0000 7f495b7fe700 1 -- 172.21.15.82:0/2453392280 <== osd.3 v2:172.21.15.82:6810/3947485599 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f4964063250 con 0x7f49500ade60 2024-05-20T16:12:52.739 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:52.739+0000 7f496c5e1700 1 -- 172.21.15.82:0/2453392280 >> [v2:172.21.15.82:6810/3947485599,v1:172.21.15.82:6811/3947485599] conn(0x7f49500ade60 msgr2=0x7f49500b0350 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:12:52.739 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:52.739+0000 7f496c5e1700 1 --2- 172.21.15.82:0/2453392280 >> [v2:172.21.15.82:6810/3947485599,v1:172.21.15.82:6811/3947485599] conn(0x7f49500ade60 0x7f49500b0350 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:52.739 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:52.739+0000 7f496c5e1700 1 -- 172.21.15.82:0/2453392280 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f49500a2cb0 msgr2=0x7f49500a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:12:52.739 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:52.739+0000 7f496c5e1700 1 --2- 172.21.15.82:0/2453392280 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f49500a2cb0 0x7f49500a51c0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f496000a400 tx=0x7f496000d040 comp rx=0 tx=0).stop 2024-05-20T16:12:52.739 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:52.739+0000 7f496c5e1700 1 -- 172.21.15.82:0/2453392280 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4964107d80 msgr2=0x7f496419a3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:12:52.739 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:52.739+0000 7f496c5e1700 1 --2- 172.21.15.82:0/2453392280 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4964107d80 0x7f496419a3f0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f495400b500 tx=0x7f495400b810 comp rx=0 tx=0).stop 2024-05-20T16:12:52.739 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:52.739+0000 7f496ab7e700 1 -- 172.21.15.82:0/2453392280 reap_dead start 2024-05-20T16:12:52.739 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:52.739+0000 7f496c5e1700 1 -- 172.21.15.82:0/2453392280 shutdown_connections 2024-05-20T16:12:52.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:52.739+0000 7f496c5e1700 1 -- 172.21.15.82:0/2453392280 >> 172.21.15.82:0/2453392280 conn(0x7f49640fed20 msgr2=0x7f49641074f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:12:52.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:52.739+0000 7f496c5e1700 1 -- 172.21.15.82:0/2453392280 shutdown_connections 2024-05-20T16:12:52.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:52.739+0000 7f496c5e1700 1 -- 172.21.15.82:0/2453392280 wait complete. 2024-05-20T16:12:53.618 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:12:53 smithi082 ceph-mon[32238]: pgmap v327: 129 pgs: 129 active+clean; 641 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s; 14 B/s, 2 keys/s, 0 objects/s recovering 2024-05-20T16:12:53.635 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:12:53 smithi184 ceph-mon[36209]: pgmap v327: 129 pgs: 129 active+clean; 641 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s; 14 B/s, 2 keys/s, 0 objects/s recovering 2024-05-20T16:12:53.644 INFO:teuthology.orchestra.run.smithi082.stdout:111669149763 2024-05-20T16:12:53.645 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell --fsid 59d1600a-16c2-11ef-bc9a-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-05-20T16:12:53.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:12:53 smithi160 ceph-mon[36347]: pgmap v327: 129 pgs: 129 active+clean; 641 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s; 14 B/s, 2 keys/s, 0 objects/s recovering 2024-05-20T16:12:53.848 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T16:12:54.290 INFO:teuthology.orchestra.run.smithi082.stdout:55834574925 2024-05-20T16:12:54.290 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell --fsid 59d1600a-16c2-11ef-bc9a-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-05-20T16:12:54.775 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:54.773+0000 7fa625c73700 1 -- 172.21.15.82:0/3942430605 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa620107da0 msgr2=0x7fa6201081b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:12:54.776 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:54.773+0000 7fa625c73700 1 --2- 172.21.15.82:0/3942430605 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa620107da0 0x7fa6201081b0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fa6140099c0 tx=0x7fa614009cd0 comp rx=0 tx=0).stop 2024-05-20T16:12:54.776 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:54.773+0000 7fa625c73700 1 -- 172.21.15.82:0/3942430605 shutdown_connections 2024-05-20T16:12:54.776 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:54.773+0000 7fa625c73700 1 --2- 172.21.15.82:0/3942430605 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa620103470 0x7fa620103880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:54.776 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:54.773+0000 7fa625c73700 1 --2- 172.21.15.82:0/3942430605 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa6201086f0 0x7fa620110050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:54.776 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:54.773+0000 7fa625c73700 1 --2- 172.21.15.82:0/3942430605 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa620107da0 0x7fa6201081b0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:54.776 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:54.773+0000 7fa625c73700 1 -- 172.21.15.82:0/3942430605 >> 172.21.15.82:0/3942430605 conn(0x7fa6200fed40 msgr2=0x7fa620101190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:12:54.776 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:54.773+0000 7fa625c73700 1 -- 172.21.15.82:0/3942430605 shutdown_connections 2024-05-20T16:12:54.776 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:54.773+0000 7fa625c73700 1 -- 172.21.15.82:0/3942430605 wait complete. 2024-05-20T16:12:54.776 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:54.773+0000 7fa625c73700 1 Processor -- start 2024-05-20T16:12:54.776 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:54.774+0000 7fa625c73700 1 -- start start 2024-05-20T16:12:54.777 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:54.774+0000 7fa625c73700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa620103470 0x7fa620199eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:12:54.777 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:54.774+0000 7fa625c73700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa620107da0 0x7fa62019a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:12:54.777 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:54.774+0000 7fa625c73700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa6201086f0 0x7fa62019e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:12:54.777 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:54.774+0000 7fa625c73700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fa620113540 con 0x7fa6201086f0 2024-05-20T16:12:54.777 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:54.774+0000 7fa625c73700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fa6201133c0 con 0x7fa620103470 2024-05-20T16:12:54.777 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:54.774+0000 7fa625c73700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa6201136c0 con 0x7fa620107da0 2024-05-20T16:12:54.777 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:54.774+0000 7fa61ffff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa6201086f0 0x7fa62019e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:12:54.777 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:54.774+0000 7fa61ffff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa6201086f0 0x7fa62019e6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:47416/0 (socket says 172.21.15.82:47416) 2024-05-20T16:12:54.777 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:54.774+0000 7fa61ffff700 1 -- 172.21.15.82:0/549259345 learned_addr learned my addr 172.21.15.82:0/549259345 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:12:54.777 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:54.774+0000 7fa61effd700 1 --2- 172.21.15.82:0/549259345 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa620107da0 0x7fa62019a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:12:54.777 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:54.774+0000 7fa61f7fe700 1 --2- 172.21.15.82:0/549259345 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa620103470 0x7fa620199eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:12:54.777 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:54.774+0000 7fa61ffff700 1 -- 172.21.15.82:0/549259345 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa620107da0 msgr2=0x7fa62019a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:12:54.777 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:54.774+0000 7fa61ffff700 1 --2- 172.21.15.82:0/549259345 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa620107da0 0x7fa62019a450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:54.778 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:54.774+0000 7fa61ffff700 1 -- 172.21.15.82:0/549259345 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa620103470 msgr2=0x7fa620199eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:12:54.778 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:54.774+0000 7fa61ffff700 1 --2- 172.21.15.82:0/549259345 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa620103470 0x7fa620199eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:54.778 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:54.774+0000 7fa61ffff700 1 -- 172.21.15.82:0/549259345 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa62019ed80 con 0x7fa6201086f0 2024-05-20T16:12:54.778 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:54.775+0000 7fa61ffff700 1 --2- 172.21.15.82:0/549259345 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa6201086f0 0x7fa62019e6e0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fa61800b970 tx=0x7fa61800bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:12:54.778 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:54.775+0000 7fa61cff9700 1 -- 172.21.15.82:0/549259345 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa61800d860 con 0x7fa6201086f0 2024-05-20T16:12:54.778 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:54.775+0000 7fa61cff9700 1 -- 172.21.15.82:0/549259345 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa61800e430 con 0x7fa6201086f0 2024-05-20T16:12:54.778 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:54.775+0000 7fa625c73700 1 -- 172.21.15.82:0/549259345 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa62019f070 con 0x7fa6201086f0 2024-05-20T16:12:54.778 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:54.775+0000 7fa61cff9700 1 -- 172.21.15.82:0/549259345 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa618017660 con 0x7fa6201086f0 2024-05-20T16:12:54.778 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:54.775+0000 7fa625c73700 1 -- 172.21.15.82:0/549259345 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6201a6990 con 0x7fa6201086f0 2024-05-20T16:12:54.778 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:54.776+0000 7fa625c73700 1 -- 172.21.15.82:0/549259345 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa620062dd0 con 0x7fa6201086f0 2024-05-20T16:12:54.779 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:54.777+0000 7fa61cff9700 1 -- 172.21.15.82:0/549259345 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fa61800e5e0 con 0x7fa6201086f0 2024-05-20T16:12:54.780 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:54.778+0000 7fa61cff9700 1 --2- 172.21.15.82:0/549259345 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fa6080a2cb0 0x7fa6080a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:12:54.780 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:54.778+0000 7fa61cff9700 1 -- 172.21.15.82:0/549259345 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9092+0+0 (secure 0 0 0) 0x7fa6180cf700 con 0x7fa6201086f0 2024-05-20T16:12:54.780 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:54.778+0000 7fa61f7fe700 1 --2- 172.21.15.82:0/549259345 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fa6080a2cb0 0x7fa6080a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:12:54.780 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:54.778+0000 7fa61cff9700 1 --2- 172.21.15.82:0/549259345 >> [v2:172.21.15.184:6808/1046282672,v1:172.21.15.184:6809/1046282672] conn(0x7fa6080ade60 0x7fa6080b0350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:12:54.780 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:54.778+0000 7fa61cff9700 1 -- 172.21.15.82:0/549259345 --> [v2:172.21.15.184:6808/1046282672,v1:172.21.15.184:6809/1046282672] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa6080b0a20 con 0x7fa6080ade60 2024-05-20T16:12:54.780 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:54.778+0000 7fa61cff9700 1 -- 172.21.15.82:0/549259345 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_get_version_reply(handle=1 version=93) ==== 24+0+0 (secure 0 0 0) 0x7fa6180cfb50 con 0x7fa6201086f0 2024-05-20T16:12:54.780 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:54.778+0000 7fa61effd700 1 --2- 172.21.15.82:0/549259345 >> [v2:172.21.15.184:6808/1046282672,v1:172.21.15.184:6809/1046282672] conn(0x7fa6080ade60 0x7fa6080b0350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:12:54.780 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:54.778+0000 7fa61f7fe700 1 --2- 172.21.15.82:0/549259345 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fa6080a2cb0 0x7fa6080a51c0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fa610009a10 tx=0x7fa610009190 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:12:54.780 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:54.779+0000 7fa61effd700 1 --2- 172.21.15.82:0/549259345 >> [v2:172.21.15.184:6808/1046282672,v1:172.21.15.184:6809/1046282672] conn(0x7fa6080ade60 0x7fa6080b0350 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:12:54.780 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:54.780+0000 7fa61cff9700 1 -- 172.21.15.82:0/549259345 <== osd.5 v2:172.21.15.184:6808/1046282672 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa6080b0a20 con 0x7fa6080ade60 2024-05-20T16:12:54.795 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:54.796+0000 7fa625c73700 1 -- 172.21.15.82:0/549259345 --> [v2:172.21.15.184:6808/1046282672,v1:172.21.15.184:6809/1046282672] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fa620063200 con 0x7fa6080ade60 2024-05-20T16:12:54.796 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:54.796+0000 7fa61cff9700 1 -- 172.21.15.82:0/549259345 <== osd.5 v2:172.21.15.184:6808/1046282672 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fa620063200 con 0x7fa6080ade60 2024-05-20T16:12:54.797 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:54.797+0000 7fa625c73700 1 -- 172.21.15.82:0/549259345 >> [v2:172.21.15.184:6808/1046282672,v1:172.21.15.184:6809/1046282672] conn(0x7fa6080ade60 msgr2=0x7fa6080b0350 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:12:54.797 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:54.797+0000 7fa625c73700 1 --2- 172.21.15.82:0/549259345 >> [v2:172.21.15.184:6808/1046282672,v1:172.21.15.184:6809/1046282672] conn(0x7fa6080ade60 0x7fa6080b0350 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:54.797 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:54.797+0000 7fa625c73700 1 -- 172.21.15.82:0/549259345 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fa6080a2cb0 msgr2=0x7fa6080a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:12:54.797 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:54.797+0000 7fa625c73700 1 --2- 172.21.15.82:0/549259345 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fa6080a2cb0 0x7fa6080a51c0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fa610009a10 tx=0x7fa610009190 comp rx=0 tx=0).stop 2024-05-20T16:12:54.797 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:54.797+0000 7fa625c73700 1 -- 172.21.15.82:0/549259345 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa6201086f0 msgr2=0x7fa62019e6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:12:54.797 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:54.797+0000 7fa625c73700 1 --2- 172.21.15.82:0/549259345 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa6201086f0 0x7fa62019e6e0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fa61800b970 tx=0x7fa61800bc80 comp rx=0 tx=0).stop 2024-05-20T16:12:54.798 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:54.797+0000 7fa61ffff700 1 -- 172.21.15.82:0/549259345 reap_dead start 2024-05-20T16:12:54.798 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:54.797+0000 7fa625c73700 1 -- 172.21.15.82:0/549259345 shutdown_connections 2024-05-20T16:12:54.798 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:54.797+0000 7fa625c73700 1 -- 172.21.15.82:0/549259345 >> 172.21.15.82:0/549259345 conn(0x7fa6200fed40 msgr2=0x7fa620101160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:12:54.798 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:54.797+0000 7fa625c73700 1 -- 172.21.15.82:0/549259345 shutdown_connections 2024-05-20T16:12:54.798 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:54.797+0000 7fa625c73700 1 -- 172.21.15.82:0/549259345 wait complete. 2024-05-20T16:12:55.574 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:12:55 smithi082 ceph-mon[32238]: pgmap v328: 129 pgs: 129 active+clean; 641 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s; 11 B/s, 1 keys/s, 0 objects/s recovering 2024-05-20T16:12:55.600 INFO:teuthology.orchestra.run.smithi082.stdout:347892351001 2024-05-20T16:12:55.600 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell --fsid 59d1600a-16c2-11ef-bc9a-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-05-20T16:12:55.635 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:12:55 smithi184 ceph-mon[36209]: pgmap v328: 129 pgs: 129 active+clean; 641 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s; 11 B/s, 1 keys/s, 0 objects/s recovering 2024-05-20T16:12:55.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:12:55 smithi160 ceph-mon[36347]: pgmap v328: 129 pgs: 129 active+clean; 641 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s; 11 B/s, 1 keys/s, 0 objects/s recovering 2024-05-20T16:12:56.208 INFO:teuthology.orchestra.run.smithi082.stdout:77309411399 2024-05-20T16:12:56.208 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell --fsid 59d1600a-16c2-11ef-bc9a-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-05-20T16:12:57.615 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:12:57 smithi160 ceph-mon[36347]: pgmap v329: 129 pgs: 129 active+clean; 641 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s; 10 B/s, 1 keys/s, 0 objects/s recovering 2024-05-20T16:12:57.635 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:12:57 smithi184 ceph-mon[36209]: pgmap v329: 129 pgs: 129 active+clean; 641 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s; 10 B/s, 1 keys/s, 0 objects/s recovering 2024-05-20T16:12:57.640 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:12:57 smithi082 ceph-mon[32238]: pgmap v329: 129 pgs: 129 active+clean; 641 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s; 10 B/s, 1 keys/s, 0 objects/s recovering 2024-05-20T16:12:57.641 INFO:teuthology.orchestra.run.smithi082.stdout:167503724601 2024-05-20T16:12:57.642 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell --fsid 59d1600a-16c2-11ef-bc9a-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-05-20T16:12:58.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:58.105+0000 7fd2f312d700 1 -- 172.21.15.82:0/3506532608 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd2ec102340 msgr2=0x7fd2ec102750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:12:58.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:58.105+0000 7fd2f312d700 1 --2- 172.21.15.82:0/3506532608 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd2ec102340 0x7fd2ec102750 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fd2e00099c0 tx=0x7fd2e0009cd0 comp rx=0 tx=0).stop 2024-05-20T16:12:58.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:58.106+0000 7fd2f312d700 1 -- 172.21.15.82:0/3506532608 shutdown_connections 2024-05-20T16:12:58.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:58.106+0000 7fd2f312d700 1 --2- 172.21.15.82:0/3506532608 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd2ec1075c0 0x7fd2ec10ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:58.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:58.106+0000 7fd2f312d700 1 --2- 172.21.15.82:0/3506532608 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd2ec102340 0x7fd2ec102750 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:58.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:58.106+0000 7fd2f312d700 1 --2- 172.21.15.82:0/3506532608 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd2ec106c70 0x7fd2ec107080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:58.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:58.106+0000 7fd2f312d700 1 -- 172.21.15.82:0/3506532608 >> 172.21.15.82:0/3506532608 conn(0x7fd2ec0fdc40 msgr2=0x7fd2ec100060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:12:58.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:58.106+0000 7fd2f312d700 1 -- 172.21.15.82:0/3506532608 shutdown_connections 2024-05-20T16:12:58.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:58.106+0000 7fd2f312d700 1 -- 172.21.15.82:0/3506532608 wait complete. 2024-05-20T16:12:58.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:58.106+0000 7fd2f312d700 1 Processor -- start 2024-05-20T16:12:58.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:58.106+0000 7fd2f312d700 1 -- start start 2024-05-20T16:12:58.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:58.107+0000 7fd2f312d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd2ec102340 0x7fd2ec19e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:12:58.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:58.107+0000 7fd2f312d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd2ec106c70 0x7fd2ec19ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:12:58.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:58.107+0000 7fd2f312d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd2ec1075c0 0x7fd2ec198a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:12:58.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:58.107+0000 7fd2f312d700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fd2ec1123b0 con 0x7fd2ec106c70 2024-05-20T16:12:58.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:58.107+0000 7fd2f312d700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fd2ec112230 con 0x7fd2ec102340 2024-05-20T16:12:58.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:58.107+0000 7fd2f312d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd2ec112530 con 0x7fd2ec1075c0 2024-05-20T16:12:58.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:58.107+0000 7fd2ebfff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd2ec106c70 0x7fd2ec19ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:12:58.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:58.107+0000 7fd2ebfff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd2ec106c70 0x7fd2ec19ed90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:47434/0 (socket says 172.21.15.82:47434) 2024-05-20T16:12:58.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:58.107+0000 7fd2ebfff700 1 -- 172.21.15.82:0/4220975743 learned_addr learned my addr 172.21.15.82:0/4220975743 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:12:58.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:58.107+0000 7fd2f0ec9700 1 --2- 172.21.15.82:0/4220975743 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd2ec102340 0x7fd2ec19e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:12:58.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:58.107+0000 7fd2f16ca700 1 --2- 172.21.15.82:0/4220975743 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd2ec1075c0 0x7fd2ec198a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:12:58.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:58.107+0000 7fd2ebfff700 1 -- 172.21.15.82:0/4220975743 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd2ec1075c0 msgr2=0x7fd2ec198a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:12:58.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:58.107+0000 7fd2ebfff700 1 --2- 172.21.15.82:0/4220975743 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd2ec1075c0 0x7fd2ec198a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:58.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:58.107+0000 7fd2ebfff700 1 -- 172.21.15.82:0/4220975743 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd2ec102340 msgr2=0x7fd2ec19e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:12:58.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:58.107+0000 7fd2ebfff700 1 --2- 172.21.15.82:0/4220975743 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd2ec102340 0x7fd2ec19e850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:58.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:58.107+0000 7fd2ebfff700 1 -- 172.21.15.82:0/4220975743 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2ec199120 con 0x7fd2ec106c70 2024-05-20T16:12:58.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:58.107+0000 7fd2f0ec9700 1 --2- 172.21.15.82:0/4220975743 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd2ec102340 0x7fd2ec19e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T16:12:58.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:58.107+0000 7fd2ebfff700 1 --2- 172.21.15.82:0/4220975743 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd2ec106c70 0x7fd2ec19ed90 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fd2d800b500 tx=0x7fd2d800b810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:12:58.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:58.108+0000 7fd2e9ffb700 1 -- 172.21.15.82:0/4220975743 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2d8015020 con 0x7fd2ec106c70 2024-05-20T16:12:58.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:58.108+0000 7fd2f312d700 1 -- 172.21.15.82:0/4220975743 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2ec199410 con 0x7fd2ec106c70 2024-05-20T16:12:58.111 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:58.108+0000 7fd2e9ffb700 1 -- 172.21.15.82:0/4220975743 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd2d8011d10 con 0x7fd2ec106c70 2024-05-20T16:12:58.111 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:58.108+0000 7fd2e9ffb700 1 -- 172.21.15.82:0/4220975743 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2d8010de0 con 0x7fd2ec106c70 2024-05-20T16:12:58.111 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:58.108+0000 7fd2f312d700 1 -- 172.21.15.82:0/4220975743 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2ec1a56e0 con 0x7fd2ec106c70 2024-05-20T16:12:58.111 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:58.109+0000 7fd2f312d700 1 -- 172.21.15.82:0/4220975743 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2ec050170 con 0x7fd2ec106c70 2024-05-20T16:12:58.111 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:58.110+0000 7fd2e9ffb700 1 -- 172.21.15.82:0/4220975743 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fd2d8018020 con 0x7fd2ec106c70 2024-05-20T16:12:58.111 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:58.111+0000 7fd2e9ffb700 1 --2- 172.21.15.82:0/4220975743 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fd2dc0a2cb0 0x7fd2dc0a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:12:58.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:58.111+0000 7fd2f0ec9700 1 --2- 172.21.15.82:0/4220975743 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fd2dc0a2cb0 0x7fd2dc0a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:12:58.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:58.111+0000 7fd2e9ffb700 1 -- 172.21.15.82:0/4220975743 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9092+0+0 (secure 0 0 0) 0x7fd2d80ceae0 con 0x7fd2ec106c70 2024-05-20T16:12:58.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:58.111+0000 7fd2f0ec9700 1 --2- 172.21.15.82:0/4220975743 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fd2dc0a2cb0 0x7fd2dc0a51c0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fd2e0000c00 tx=0x7fd2e001a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:12:58.116 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:58.116+0000 7fd2e9ffb700 1 -- 172.21.15.82:0/4220975743 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fd2d801f020 con 0x7fd2ec106c70 2024-05-20T16:12:58.320 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:58.319+0000 7fd2f312d700 1 -- 172.21.15.82:0/4220975743 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7fd2ec199fa0 con 0x7fd2ec106c70 2024-05-20T16:12:58.321 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:58.320+0000 7fd2e9ffb700 1 -- 172.21.15.82:0/4220975743 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7fd2d809b690 con 0x7fd2ec106c70 2024-05-20T16:12:58.321 INFO:teuthology.orchestra.run.smithi082.stdout:313532612639 2024-05-20T16:12:58.323 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:58.322+0000 7fd2f312d700 1 -- 172.21.15.82:0/4220975743 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fd2dc0a2cb0 msgr2=0x7fd2dc0a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:12:58.323 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:58.322+0000 7fd2f312d700 1 --2- 172.21.15.82:0/4220975743 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fd2dc0a2cb0 0x7fd2dc0a51c0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fd2e0000c00 tx=0x7fd2e001a040 comp rx=0 tx=0).stop 2024-05-20T16:12:58.323 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:58.322+0000 7fd2f312d700 1 -- 172.21.15.82:0/4220975743 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd2ec106c70 msgr2=0x7fd2ec19ed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:12:58.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:58.322+0000 7fd2f312d700 1 --2- 172.21.15.82:0/4220975743 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd2ec106c70 0x7fd2ec19ed90 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fd2d800b500 tx=0x7fd2d800b810 comp rx=0 tx=0).stop 2024-05-20T16:12:58.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:58.322+0000 7fd2f312d700 1 -- 172.21.15.82:0/4220975743 shutdown_connections 2024-05-20T16:12:58.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:58.322+0000 7fd2f312d700 1 --2- 172.21.15.82:0/4220975743 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fd2dc0a2cb0 0x7fd2dc0a51c0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:58.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:58.322+0000 7fd2f312d700 1 --2- 172.21.15.82:0/4220975743 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd2ec102340 0x7fd2ec19e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:58.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:58.322+0000 7fd2f312d700 1 --2- 172.21.15.82:0/4220975743 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd2ec1075c0 0x7fd2ec198a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:58.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:58.322+0000 7fd2f312d700 1 --2- 172.21.15.82:0/4220975743 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd2ec106c70 0x7fd2ec19ed90 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:12:58.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:58.322+0000 7fd2f312d700 1 -- 172.21.15.82:0/4220975743 >> 172.21.15.82:0/4220975743 conn(0x7fd2ec0fdc40 msgr2=0x7fd2ec105e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:12:58.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:58.322+0000 7fd2f312d700 1 -- 172.21.15.82:0/4220975743 shutdown_connections 2024-05-20T16:12:58.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:12:58.322+0000 7fd2f312d700 1 -- 172.21.15.82:0/4220975743 wait complete. 2024-05-20T16:12:58.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:12:58 smithi082 ceph-mon[32238]: from='client.? 172.21.15.82:0/4220975743' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-05-20T16:12:58.635 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:12:58 smithi184 ceph-mon[36209]: from='client.? 172.21.15.82:0/4220975743' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-05-20T16:12:58.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:12:58 smithi160 ceph-mon[36347]: from='client.? 172.21.15.82:0/4220975743' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-05-20T16:12:59.203 INFO:tasks.cephadm.ceph_manager.ceph:need seq 313532612635 got 313532612639 for osd.10 2024-05-20T16:12:59.203 DEBUG:teuthology.parallel:result is None 2024-05-20T16:12:59.273 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T16:12:59.273 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T16:12:59.274 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T16:12:59.274 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T16:12:59.483 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:12:59 smithi082 ceph-mon[32238]: pgmap v330: 129 pgs: 129 active+clean; 641 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s; 9 B/s, 0 keys/s, 0 objects/s recovering 2024-05-20T16:12:59.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:12:59 smithi184 ceph-mon[36209]: pgmap v330: 129 pgs: 129 active+clean; 641 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s; 9 B/s, 0 keys/s, 0 objects/s recovering 2024-05-20T16:12:59.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:12:59 smithi160 ceph-mon[36347]: pgmap v330: 129 pgs: 129 active+clean; 641 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s; 9 B/s, 0 keys/s, 0 objects/s recovering 2024-05-20T16:13:00.253 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T16:13:00.254 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T16:13:00.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:00.257+0000 7f51c01ed700 1 -- 172.21.15.82:0/2786977551 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f51b81141f0 msgr2=0x7f51b8116610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:00.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:00.257+0000 7f51c01ed700 1 --2- 172.21.15.82:0/2786977551 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f51b81141f0 0x7f51b8116610 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f51b0009f90 tx=0x7f51b000b330 comp rx=0 tx=0).stop 2024-05-20T16:13:00.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:00.257+0000 7f51c01ed700 1 -- 172.21.15.82:0/2786977551 shutdown_connections 2024-05-20T16:13:00.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:00.257+0000 7f51c01ed700 1 --2- 172.21.15.82:0/2786977551 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f51b81141f0 0x7f51b8116610 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:00.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:00.257+0000 7f51c01ed700 1 --2- 172.21.15.82:0/2786977551 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f51b8075af0 0x7f51b8075f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:00.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:00.257+0000 7f51c01ed700 1 --2- 172.21.15.82:0/2786977551 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f51b8076440 0x7f51b8113cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:00.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:00.257+0000 7f51c01ed700 1 -- 172.21.15.82:0/2786977551 >> 172.21.15.82:0/2786977551 conn(0x7f51b80fed40 msgr2=0x7f51b8101190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:13:00.258 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:00.257+0000 7f51c01ed700 1 -- 172.21.15.82:0/2786977551 shutdown_connections 2024-05-20T16:13:00.258 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:00.257+0000 7f51c01ed700 1 -- 172.21.15.82:0/2786977551 wait complete. 2024-05-20T16:13:00.258 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:00.258+0000 7f51c01ed700 1 Processor -- start 2024-05-20T16:13:00.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:00.258+0000 7f51c01ed700 1 -- start start 2024-05-20T16:13:00.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:00.258+0000 7f51c01ed700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f51b8075af0 0x7f51b81a0340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:00.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:00.258+0000 7f51c01ed700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f51b8076440 0x7f51b81a08e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:00.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:00.258+0000 7f51c01ed700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f51b81141f0 0x7f51b81a4b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:00.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:00.258+0000 7f51c01ed700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f51b8119a90 con 0x7f51b8076440 2024-05-20T16:13:00.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:00.258+0000 7f51c01ed700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f51b8119910 con 0x7f51b8075af0 2024-05-20T16:13:00.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:00.258+0000 7f51c01ed700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f51b8119c10 con 0x7f51b81141f0 2024-05-20T16:13:00.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:00.259+0000 7f51bd788700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f51b8076440 0x7f51b81a08e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:13:00.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:00.259+0000 7f51bd788700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f51b8076440 0x7f51b81a08e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:48816/0 (socket says 172.21.15.82:48816) 2024-05-20T16:13:00.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:00.259+0000 7f51bd788700 1 -- 172.21.15.82:0/392551652 learned_addr learned my addr 172.21.15.82:0/392551652 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:13:00.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:00.259+0000 7f51bdf89700 1 --2- 172.21.15.82:0/392551652 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f51b8075af0 0x7f51b81a0340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:13:00.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:00.259+0000 7f51bd788700 1 -- 172.21.15.82:0/392551652 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f51b81141f0 msgr2=0x7f51b81a4b70 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-20T16:13:00.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:00.259+0000 7f51bd788700 1 --2- 172.21.15.82:0/392551652 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f51b81141f0 0x7f51b81a4b70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:00.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:00.259+0000 7f51bd788700 1 -- 172.21.15.82:0/392551652 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f51b8075af0 msgr2=0x7f51b81a0340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:00.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:00.259+0000 7f51bd788700 1 --2- 172.21.15.82:0/392551652 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f51b8075af0 0x7f51b81a0340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:00.261 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:00.259+0000 7f51bd788700 1 -- 172.21.15.82:0/392551652 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51b81a5210 con 0x7f51b8076440 2024-05-20T16:13:00.261 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:00.259+0000 7f51bd788700 1 --2- 172.21.15.82:0/392551652 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f51b8076440 0x7f51b81a08e0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f51a800e9a0 tx=0x7f51a800ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:13:00.261 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:00.260+0000 7f51aeffd700 1 -- 172.21.15.82:0/392551652 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51a800cbd0 con 0x7f51b8076440 2024-05-20T16:13:00.261 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:00.260+0000 7f51c01ed700 1 -- 172.21.15.82:0/392551652 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51b81a5500 con 0x7f51b8076440 2024-05-20T16:13:00.261 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:00.260+0000 7f51c01ed700 1 -- 172.21.15.82:0/392551652 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51b81ace20 con 0x7f51b8076440 2024-05-20T16:13:00.262 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:00.261+0000 7f51aeffd700 1 -- 172.21.15.82:0/392551652 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f51a800cd70 con 0x7f51b8076440 2024-05-20T16:13:00.263 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:00.263+0000 7f51c01ed700 1 -- 172.21.15.82:0/392551652 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f519c005330 con 0x7f51b8076440 2024-05-20T16:13:00.263 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:00.263+0000 7f51aeffd700 1 -- 172.21.15.82:0/392551652 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51a80187c0 con 0x7f51b8076440 2024-05-20T16:13:00.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:00.264+0000 7f51aeffd700 1 -- 172.21.15.82:0/392551652 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f51a8018a20 con 0x7f51b8076440 2024-05-20T16:13:00.265 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:00.265+0000 7f51aeffd700 1 --2- 172.21.15.82:0/392551652 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f51a40a2c10 0x7f51a40a5120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:00.265 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:00.266+0000 7f51aeffd700 1 -- 172.21.15.82:0/392551652 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9092+0+0 (secure 0 0 0) 0x7f51a80cf6f0 con 0x7f51b8076440 2024-05-20T16:13:00.266 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:00.266+0000 7f51bdf89700 1 --2- 172.21.15.82:0/392551652 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f51a40a2c10 0x7f51a40a5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:13:00.266 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:00.266+0000 7f51bdf89700 1 --2- 172.21.15.82:0/392551652 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f51a40a2c10 0x7f51a40a5120 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f51b4005fd0 tx=0x7f51b4009310 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:13:00.273 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:00.273+0000 7f51aeffd700 1 -- 172.21.15.82:0/392551652 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f51a809c190 con 0x7f51b8076440 2024-05-20T16:13:00.489 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:00.488+0000 7f51c01ed700 1 -- 172.21.15.82:0/392551652 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7f519c0058f0 con 0x7f51b8076440 2024-05-20T16:13:00.489 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:00.489+0000 7f51aeffd700 1 -- 172.21.15.82:0/392551652 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f51a809ba90 con 0x7f51b8076440 2024-05-20T16:13:00.489 INFO:teuthology.orchestra.run.smithi082.stdout:292057776166 2024-05-20T16:13:00.491 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:00.491+0000 7f51c01ed700 1 -- 172.21.15.82:0/392551652 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f51a40a2c10 msgr2=0x7f51a40a5120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:00.491 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:00.491+0000 7f51c01ed700 1 --2- 172.21.15.82:0/392551652 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f51a40a2c10 0x7f51a40a5120 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f51b4005fd0 tx=0x7f51b4009310 comp rx=0 tx=0).stop 2024-05-20T16:13:00.491 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:00.491+0000 7f51c01ed700 1 -- 172.21.15.82:0/392551652 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f51b8076440 msgr2=0x7f51b81a08e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:00.491 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:00.491+0000 7f51c01ed700 1 --2- 172.21.15.82:0/392551652 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f51b8076440 0x7f51b81a08e0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f51a800e9a0 tx=0x7f51a800ed60 comp rx=0 tx=0).stop 2024-05-20T16:13:00.491 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:00.491+0000 7f51c01ed700 1 -- 172.21.15.82:0/392551652 shutdown_connections 2024-05-20T16:13:00.491 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:00.491+0000 7f51c01ed700 1 --2- 172.21.15.82:0/392551652 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f51a40a2c10 0x7f51a40a5120 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:00.491 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:00.491+0000 7f51c01ed700 1 --2- 172.21.15.82:0/392551652 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f51b8075af0 0x7f51b81a0340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:00.491 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:00.491+0000 7f51c01ed700 1 --2- 172.21.15.82:0/392551652 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f51b81141f0 0x7f51b81a4b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:00.492 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:00.491+0000 7f51c01ed700 1 --2- 172.21.15.82:0/392551652 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f51b8076440 0x7f51b81a08e0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:00.492 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:00.491+0000 7f51c01ed700 1 -- 172.21.15.82:0/392551652 >> 172.21.15.82:0/392551652 conn(0x7f51b80fed40 msgr2=0x7f51b8100b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:13:00.492 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:00.491+0000 7f51c01ed700 1 -- 172.21.15.82:0/392551652 shutdown_connections 2024-05-20T16:13:00.492 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:00.491+0000 7f51c01ed700 1 -- 172.21.15.82:0/392551652 wait complete. 2024-05-20T16:13:00.596 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:13:00 smithi082 ceph-mon[32238]: from='client.? 172.21.15.82:0/392551652' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-05-20T16:13:00.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:13:00 smithi160 ceph-mon[36347]: from='client.? 172.21.15.82:0/392551652' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-05-20T16:13:00.886 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:13:00 smithi184 ceph-mon[36209]: from='client.? 172.21.15.82:0/392551652' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-05-20T16:13:01.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:13:01 smithi082 ceph-mon[32238]: pgmap v331: 129 pgs: 129 active+clean; 641 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-05-20T16:13:01.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:13:01 smithi160 ceph-mon[36347]: pgmap v331: 129 pgs: 129 active+clean; 641 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-05-20T16:13:01.886 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:13:01 smithi184 ceph-mon[36209]: pgmap v331: 129 pgs: 129 active+clean; 641 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-05-20T16:13:03.585 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:13:03 smithi082 ceph-mon[32238]: pgmap v332: 129 pgs: 129 active+clean; 641 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-20T16:13:03.635 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:13:03 smithi184 ceph-mon[36209]: pgmap v332: 129 pgs: 129 active+clean; 641 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-20T16:13:03.699 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:03.699+0000 7f6fe2937700 1 -- 172.21.15.82:0/1061150084 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6fdc101610 msgr2=0x7f6fdc101a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:03.699 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:03.699+0000 7f6fe2937700 1 --2- 172.21.15.82:0/1061150084 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6fdc101610 0x7f6fdc101a20 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f6fd0009990 tx=0x7f6fd0009ca0 comp rx=0 tx=0).stop 2024-05-20T16:13:03.699 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:03.699+0000 7f6fe2937700 1 -- 172.21.15.82:0/1061150084 shutdown_connections 2024-05-20T16:13:03.699 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:03.699+0000 7f6fe2937700 1 --2- 172.21.15.82:0/1061150084 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6fdc101f60 0x7f6fdc10fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:03.700 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:03.699+0000 7f6fe2937700 1 --2- 172.21.15.82:0/1061150084 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6fdc108180 0x7f6fdc108590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:03.700 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:03.699+0000 7f6fe2937700 1 --2- 172.21.15.82:0/1061150084 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6fdc101610 0x7f6fdc101a20 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:03.700 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:03.699+0000 7f6fe2937700 1 -- 172.21.15.82:0/1061150084 >> 172.21.15.82:0/1061150084 conn(0x7f6fdc0fd3a0 msgr2=0x7f6fdc0ff7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:13:03.700 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:03.699+0000 7f6fe2937700 1 -- 172.21.15.82:0/1061150084 shutdown_connections 2024-05-20T16:13:03.700 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:03.699+0000 7f6fe2937700 1 -- 172.21.15.82:0/1061150084 wait complete. 2024-05-20T16:13:03.702 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:03.700+0000 7f6fe2937700 1 Processor -- start 2024-05-20T16:13:03.702 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:03.700+0000 7f6fe2937700 1 -- start start 2024-05-20T16:13:03.702 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:03.700+0000 7f6fe2937700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6fdc101610 0x7f6fdc199e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:03.702 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:03.700+0000 7f6fe2937700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6fdc101f60 0x7f6fdc19a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:03.702 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:03.700+0000 7f6fe2937700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6fdc108180 0x7f6fdc19e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:03.702 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:03.700+0000 7f6fe2937700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f6fdc1135c0 con 0x7f6fdc101610 2024-05-20T16:13:03.703 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:03.700+0000 7f6fe2937700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f6fdc113440 con 0x7f6fdc101f60 2024-05-20T16:13:03.703 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:03.700+0000 7f6fe2937700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f6fdc113740 con 0x7f6fdc108180 2024-05-20T16:13:03.703 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:03.700+0000 7f6fdbfff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6fdc101610 0x7f6fdc199e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:13:03.703 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:03.700+0000 7f6fdbfff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6fdc101610 0x7f6fdc199e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:48834/0 (socket says 172.21.15.82:48834) 2024-05-20T16:13:03.703 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:03.700+0000 7f6fdbfff700 1 -- 172.21.15.82:0/2781501118 learned_addr learned my addr 172.21.15.82:0/2781501118 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:13:03.703 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:03.700+0000 7f6fdb7fe700 1 --2- 172.21.15.82:0/2781501118 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6fdc101f60 0x7f6fdc19a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:13:03.703 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:03.701+0000 7f6fe0ed4700 1 --2- 172.21.15.82:0/2781501118 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6fdc108180 0x7f6fdc19e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:13:03.703 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:03.701+0000 7f6fdbfff700 1 -- 172.21.15.82:0/2781501118 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6fdc108180 msgr2=0x7f6fdc19e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:03.703 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:03.701+0000 7f6fdbfff700 1 --2- 172.21.15.82:0/2781501118 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6fdc108180 0x7f6fdc19e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:03.703 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:03.701+0000 7f6fdbfff700 1 -- 172.21.15.82:0/2781501118 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6fdc101f60 msgr2=0x7f6fdc19a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:03.703 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:03.701+0000 7f6fdbfff700 1 --2- 172.21.15.82:0/2781501118 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6fdc101f60 0x7f6fdc19a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:03.703 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:03.701+0000 7f6fdbfff700 1 -- 172.21.15.82:0/2781501118 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fdc19ec70 con 0x7f6fdc101610 2024-05-20T16:13:03.704 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:03.701+0000 7f6fdbfff700 1 --2- 172.21.15.82:0/2781501118 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6fdc101610 0x7f6fdc199e50 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f6fcc00c970 tx=0x7f6fcc00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:13:03.704 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:03.701+0000 7f6fd97fa700 1 -- 172.21.15.82:0/2781501118 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fcc007b00 con 0x7f6fdc101610 2024-05-20T16:13:03.704 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:03.701+0000 7f6fe2937700 1 -- 172.21.15.82:0/2781501118 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6fdc19ef60 con 0x7f6fdc101610 2024-05-20T16:13:03.704 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:03.701+0000 7f6fd97fa700 1 -- 172.21.15.82:0/2781501118 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6fcc007ca0 con 0x7f6fdc101610 2024-05-20T16:13:03.704 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:03.701+0000 7f6fd97fa700 1 -- 172.21.15.82:0/2781501118 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fcc018660 con 0x7f6fdc101610 2024-05-20T16:13:03.704 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:03.701+0000 7f6fe2937700 1 -- 172.21.15.82:0/2781501118 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6fdc19f4c0 con 0x7f6fdc101610 2024-05-20T16:13:03.704 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:03.702+0000 7f6fe2937700 1 -- 172.21.15.82:0/2781501118 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6fdc050170 con 0x7f6fdc101610 2024-05-20T16:13:03.705 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:03.704+0000 7f6fd97fa700 1 -- 172.21.15.82:0/2781501118 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f6fcc004030 con 0x7f6fdc101610 2024-05-20T16:13:03.705 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:03.705+0000 7f6fd97fa700 1 --2- 172.21.15.82:0/2781501118 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f6fc40a2cb0 0x7f6fc40a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:03.705 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:03.705+0000 7f6fdb7fe700 1 --2- 172.21.15.82:0/2781501118 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f6fc40a2cb0 0x7f6fc40a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:13:03.705 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:03.705+0000 7f6fd97fa700 1 -- 172.21.15.82:0/2781501118 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9092+0+0 (secure 0 0 0) 0x7f6fcc0cf5b0 con 0x7f6fdc101610 2024-05-20T16:13:03.705 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:03.706+0000 7f6fdb7fe700 1 --2- 172.21.15.82:0/2781501118 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f6fc40a2cb0 0x7f6fc40a51c0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f6fd000b5c0 tx=0x7f6fd001a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:13:03.708 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:03.709+0000 7f6fd97fa700 1 -- 172.21.15.82:0/2781501118 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f6fcc014030 con 0x7f6fdc101610 2024-05-20T16:13:03.814 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:13:03 smithi160 ceph-mon[36347]: pgmap v332: 129 pgs: 129 active+clean; 641 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-20T16:13:03.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:03.909+0000 7f6fe2937700 1 -- 172.21.15.82:0/2781501118 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f6fdc19b1b0 con 0x7f6fdc101610 2024-05-20T16:13:03.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:03.909+0000 7f6fd97fa700 1 -- 172.21.15.82:0/2781501118 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f6fcc09c080 con 0x7f6fdc101610 2024-05-20T16:13:03.910 INFO:teuthology.orchestra.run.smithi082.stdout:223338299441 2024-05-20T16:13:03.911 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:03.911+0000 7f6fe2937700 1 -- 172.21.15.82:0/2781501118 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f6fc40a2cb0 msgr2=0x7f6fc40a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:03.911 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:03.911+0000 7f6fe2937700 1 --2- 172.21.15.82:0/2781501118 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f6fc40a2cb0 0x7f6fc40a51c0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f6fd000b5c0 tx=0x7f6fd001a040 comp rx=0 tx=0).stop 2024-05-20T16:13:03.911 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:03.911+0000 7f6fe2937700 1 -- 172.21.15.82:0/2781501118 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6fdc101610 msgr2=0x7f6fdc199e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:03.912 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:03.911+0000 7f6fe2937700 1 --2- 172.21.15.82:0/2781501118 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6fdc101610 0x7f6fdc199e50 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f6fcc00c970 tx=0x7f6fcc00cc80 comp rx=0 tx=0).stop 2024-05-20T16:13:03.912 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:03.912+0000 7f6fe2937700 1 -- 172.21.15.82:0/2781501118 shutdown_connections 2024-05-20T16:13:03.912 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:03.912+0000 7f6fe2937700 1 --2- 172.21.15.82:0/2781501118 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f6fc40a2cb0 0x7f6fc40a51c0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:03.912 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:03.912+0000 7f6fe2937700 1 --2- 172.21.15.82:0/2781501118 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6fdc108180 0x7f6fdc19e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:03.912 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:03.912+0000 7f6fe2937700 1 --2- 172.21.15.82:0/2781501118 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6fdc101610 0x7f6fdc199e50 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:03.912 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:03.912+0000 7f6fe2937700 1 --2- 172.21.15.82:0/2781501118 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6fdc101f60 0x7f6fdc19a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:03.912 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:03.912+0000 7f6fe2937700 1 -- 172.21.15.82:0/2781501118 >> 172.21.15.82:0/2781501118 conn(0x7f6fdc0fd3a0 msgr2=0x7f6fdc105e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:13:03.912 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:03.912+0000 7f6fe2937700 1 -- 172.21.15.82:0/2781501118 shutdown_connections 2024-05-20T16:13:03.912 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:03.912+0000 7f6fe2937700 1 -- 172.21.15.82:0/2781501118 wait complete. 2024-05-20T16:13:03.998 INFO:tasks.cephadm.ceph_manager.ceph:need seq 292057776162 got 292057776166 for osd.9 2024-05-20T16:13:03.998 DEBUG:teuthology.parallel:result is None 2024-05-20T16:13:04.601 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:13:04 smithi082 ceph-mon[32238]: from='client.? 172.21.15.82:0/2781501118' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-05-20T16:13:04.629 INFO:tasks.cephadm.ceph_manager.ceph:need seq 223338299438 got 223338299441 for osd.7 2024-05-20T16:13:04.629 DEBUG:teuthology.parallel:result is None 2024-05-20T16:13:04.635 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:13:04 smithi184 ceph-mon[36209]: from='client.? 172.21.15.82:0/2781501118' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-05-20T16:13:04.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:13:04 smithi160 ceph-mon[36347]: from='client.? 172.21.15.82:0/2781501118' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-05-20T16:13:05.567 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:13:05 smithi082 ceph-mon[32238]: pgmap v333: 129 pgs: 129 active+clean; 641 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-20T16:13:05.635 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:13:05 smithi184 ceph-mon[36209]: pgmap v333: 129 pgs: 129 active+clean; 641 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-20T16:13:05.768 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T16:13:05.768 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T16:13:05.769 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T16:13:05.769 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T16:13:05.812 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:05.812+0000 7f945af15700 1 -- 172.21.15.82:0/360141312 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9454106ea0 msgr2=0x7f94541072b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:05.812 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:05.812+0000 7f945af15700 1 --2- 172.21.15.82:0/360141312 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9454106ea0 0x7f94541072b0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f94400099c0 tx=0x7f9440009cd0 comp rx=0 tx=0).stop 2024-05-20T16:13:05.812 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:05.812+0000 7f945af15700 1 -- 172.21.15.82:0/360141312 shutdown_connections 2024-05-20T16:13:05.812 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:05.812+0000 7f945af15700 1 --2- 172.21.15.82:0/360141312 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f94541077f0 0x7f945410f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:05.812 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:05.812+0000 7f945af15700 1 --2- 172.21.15.82:0/360141312 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9454102a90 0x7f9454102ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:05.812 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:05.812+0000 7f945af15700 1 --2- 172.21.15.82:0/360141312 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9454106ea0 0x7f94541072b0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:05.812 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:05.812+0000 7f945af15700 1 -- 172.21.15.82:0/360141312 >> 172.21.15.82:0/360141312 conn(0x7f94540fd380 msgr2=0x7f94540ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:13:05.813 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:05.812+0000 7f945af15700 1 -- 172.21.15.82:0/360141312 shutdown_connections 2024-05-20T16:13:05.813 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:05.812+0000 7f945af15700 1 -- 172.21.15.82:0/360141312 wait complete. 2024-05-20T16:13:05.813 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:13:05 smithi160 ceph-mon[36347]: pgmap v333: 129 pgs: 129 active+clean; 641 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-20T16:13:05.815 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:05.813+0000 7f945af15700 1 Processor -- start 2024-05-20T16:13:05.815 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:05.813+0000 7f945af15700 1 -- start start 2024-05-20T16:13:05.815 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:05.813+0000 7f945af15700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9454102a90 0x7f945419ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:05.815 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:05.813+0000 7f9458cb1700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9454102a90 0x7f945419ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:13:05.815 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:05.813+0000 7f9458cb1700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9454102a90 0x7f945419ae60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:48848/0 (socket says 172.21.15.82:48848) 2024-05-20T16:13:05.815 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:05.813+0000 7f945af15700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9454106ea0 0x7f945419b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:05.815 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:05.813+0000 7f945af15700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f94541077f0 0x7f945419b900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:05.816 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:05.813+0000 7f9458cb1700 1 -- 172.21.15.82:0/3293056156 learned_addr learned my addr 172.21.15.82:0/3293056156 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:13:05.816 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:05.813+0000 7f945af15700 1 -- 172.21.15.82:0/3293056156 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f94541125c0 con 0x7f9454102a90 2024-05-20T16:13:05.816 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:05.813+0000 7f945af15700 1 -- 172.21.15.82:0/3293056156 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f9454112440 con 0x7f94541077f0 2024-05-20T16:13:05.816 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:05.813+0000 7f945af15700 1 -- 172.21.15.82:0/3293056156 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f9454112740 con 0x7f9454106ea0 2024-05-20T16:13:05.816 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:05.813+0000 7f9453fff700 1 --2- 172.21.15.82:0/3293056156 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9454106ea0 0x7f945419b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:13:05.816 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:05.813+0000 7f94594b2700 1 --2- 172.21.15.82:0/3293056156 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f94541077f0 0x7f945419b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:13:05.816 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:05.814+0000 7f9458cb1700 1 -- 172.21.15.82:0/3293056156 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9454106ea0 msgr2=0x7f945419b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:05.816 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:05.814+0000 7f9458cb1700 1 --2- 172.21.15.82:0/3293056156 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9454106ea0 0x7f945419b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:05.816 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:05.814+0000 7f9458cb1700 1 -- 172.21.15.82:0/3293056156 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f94541077f0 msgr2=0x7f945419b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:05.816 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:05.814+0000 7f9458cb1700 1 --2- 172.21.15.82:0/3293056156 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f94541077f0 0x7f945419b900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:05.816 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:05.814+0000 7f9458cb1700 1 -- 172.21.15.82:0/3293056156 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94541a6120 con 0x7f9454102a90 2024-05-20T16:13:05.816 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:05.814+0000 7f94594b2700 1 --2- 172.21.15.82:0/3293056156 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f94541077f0 0x7f945419b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T16:13:05.816 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:05.814+0000 7f9458cb1700 1 --2- 172.21.15.82:0/3293056156 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9454102a90 0x7f945419ae60 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f944800c9b0 tx=0x7f944800cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:13:05.816 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:05.814+0000 7f9451ffb700 1 -- 172.21.15.82:0/3293056156 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9448007b00 con 0x7f9454102a90 2024-05-20T16:13:05.817 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:05.814+0000 7f9451ffb700 1 -- 172.21.15.82:0/3293056156 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9448007ca0 con 0x7f9454102a90 2024-05-20T16:13:05.817 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:05.814+0000 7f945af15700 1 -- 172.21.15.82:0/3293056156 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94541a6410 con 0x7f9454102a90 2024-05-20T16:13:05.817 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:05.814+0000 7f9451ffb700 1 -- 172.21.15.82:0/3293056156 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9448018780 con 0x7f9454102a90 2024-05-20T16:13:05.817 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:05.814+0000 7f945af15700 1 -- 172.21.15.82:0/3293056156 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94541a6970 con 0x7f9454102a90 2024-05-20T16:13:05.817 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:05.815+0000 7f945af15700 1 -- 172.21.15.82:0/3293056156 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9454050170 con 0x7f9454102a90 2024-05-20T16:13:05.817 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:05.817+0000 7f9451ffb700 1 -- 172.21.15.82:0/3293056156 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f944800f450 con 0x7f9454102a90 2024-05-20T16:13:05.817 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:05.817+0000 7f9451ffb700 1 --2- 172.21.15.82:0/3293056156 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f94440a2d30 0x7f94440a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:05.818 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:05.818+0000 7f9453fff700 1 --2- 172.21.15.82:0/3293056156 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f94440a2d30 0x7f94440a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:13:05.818 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:05.818+0000 7f9451ffb700 1 -- 172.21.15.82:0/3293056156 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9092+0+0 (secure 0 0 0) 0x7f94480cf840 con 0x7f9454102a90 2024-05-20T16:13:05.818 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:05.818+0000 7f9453fff700 1 --2- 172.21.15.82:0/3293056156 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f94440a2d30 0x7f94440a5240 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f9440000c00 tx=0x7f944001a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:13:05.821 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:05.821+0000 7f9451ffb700 1 -- 172.21.15.82:0/3293056156 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f9448014030 con 0x7f9454102a90 2024-05-20T16:13:06.025 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:06.026+0000 7f945af15700 1 -- 172.21.15.82:0/3293056156 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f945406af00 con 0x7f9454102a90 2024-05-20T16:13:06.026 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:06.026+0000 7f9451ffb700 1 -- 172.21.15.82:0/3293056156 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f944809c390 con 0x7f9454102a90 2024-05-20T16:13:06.026 INFO:teuthology.orchestra.run.smithi082.stdout:34359738452 2024-05-20T16:13:06.029 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:06.029+0000 7f945af15700 1 -- 172.21.15.82:0/3293056156 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f94440a2d30 msgr2=0x7f94440a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:06.029 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:06.029+0000 7f945af15700 1 --2- 172.21.15.82:0/3293056156 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f94440a2d30 0x7f94440a5240 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f9440000c00 tx=0x7f944001a040 comp rx=0 tx=0).stop 2024-05-20T16:13:06.029 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:06.029+0000 7f945af15700 1 -- 172.21.15.82:0/3293056156 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9454102a90 msgr2=0x7f945419ae60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:06.029 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:06.029+0000 7f945af15700 1 --2- 172.21.15.82:0/3293056156 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9454102a90 0x7f945419ae60 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f944800c9b0 tx=0x7f944800cd70 comp rx=0 tx=0).stop 2024-05-20T16:13:06.029 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:06.029+0000 7f945af15700 1 -- 172.21.15.82:0/3293056156 shutdown_connections 2024-05-20T16:13:06.029 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:06.029+0000 7f945af15700 1 --2- 172.21.15.82:0/3293056156 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f94440a2d30 0x7f94440a5240 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:06.029 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:06.029+0000 7f945af15700 1 --2- 172.21.15.82:0/3293056156 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f94541077f0 0x7f945419b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:06.030 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:06.029+0000 7f945af15700 1 --2- 172.21.15.82:0/3293056156 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9454102a90 0x7f945419ae60 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:06.030 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:06.029+0000 7f945af15700 1 --2- 172.21.15.82:0/3293056156 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9454106ea0 0x7f945419b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:06.030 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:06.029+0000 7f945af15700 1 -- 172.21.15.82:0/3293056156 >> 172.21.15.82:0/3293056156 conn(0x7f94540fd380 msgr2=0x7f9454108750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:13:06.030 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:06.029+0000 7f945af15700 1 -- 172.21.15.82:0/3293056156 shutdown_connections 2024-05-20T16:13:06.030 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:06.029+0000 7f945af15700 1 -- 172.21.15.82:0/3293056156 wait complete. 2024-05-20T16:13:06.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:13:06 smithi082 ceph-mon[32238]: from='client.? 172.21.15.82:0/3293056156' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-05-20T16:13:06.635 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:13:06 smithi184 ceph-mon[36209]: from='client.? 172.21.15.82:0/3293056156' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-05-20T16:13:06.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:13:06 smithi160 ceph-mon[36347]: from='client.? 172.21.15.82:0/3293056156' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-05-20T16:13:07.610 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:07.608+0000 7fd797733700 1 -- 172.21.15.82:0/4262710716 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd790107080 msgr2=0x7fd790107490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:07.610 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:07.608+0000 7fd797733700 1 --2- 172.21.15.82:0/4262710716 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd790107080 0x7fd790107490 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fd78c009990 tx=0x7fd78c009ca0 comp rx=0 tx=0).stop 2024-05-20T16:13:07.610 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:07.608+0000 7fd797733700 1 -- 172.21.15.82:0/4262710716 shutdown_connections 2024-05-20T16:13:07.610 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:07.608+0000 7fd797733700 1 --2- 172.21.15.82:0/4262710716 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd790103170 0x7fd79010eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:07.610 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:07.608+0000 7fd797733700 1 --2- 172.21.15.82:0/4262710716 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd790107080 0x7fd790107490 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:07.610 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:07.608+0000 7fd797733700 1 --2- 172.21.15.82:0/4262710716 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd790102820 0x7fd790102c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:07.610 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:07.608+0000 7fd797733700 1 -- 172.21.15.82:0/4262710716 >> 172.21.15.82:0/4262710716 conn(0x7fd7900fd360 msgr2=0x7fd7900ff7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:13:07.610 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:07.608+0000 7fd797733700 1 -- 172.21.15.82:0/4262710716 shutdown_connections 2024-05-20T16:13:07.610 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:07.608+0000 7fd797733700 1 -- 172.21.15.82:0/4262710716 wait complete. 2024-05-20T16:13:07.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:07.609+0000 7fd797733700 1 Processor -- start 2024-05-20T16:13:07.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:07.609+0000 7fd797733700 1 -- start start 2024-05-20T16:13:07.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:07.609+0000 7fd797733700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd790102820 0x7fd79019e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:07.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:07.609+0000 7fd797733700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd790103170 0x7fd79019ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:07.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:07.609+0000 7fd797733700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd790107080 0x7fd790198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:07.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:07.609+0000 7fd797733700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fd790112340 con 0x7fd790102820 2024-05-20T16:13:07.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:07.609+0000 7fd797733700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fd7901121c0 con 0x7fd790103170 2024-05-20T16:13:07.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:07.609+0000 7fd797733700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd7901124c0 con 0x7fd790107080 2024-05-20T16:13:07.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:07.609+0000 7fd7954cf700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd790102820 0x7fd79019e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:13:07.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:07.609+0000 7fd7954cf700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd790102820 0x7fd79019e820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:48856/0 (socket says 172.21.15.82:48856) 2024-05-20T16:13:07.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:07.609+0000 7fd7954cf700 1 -- 172.21.15.82:0/4171611987 learned_addr learned my addr 172.21.15.82:0/4171611987 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:13:07.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:07.609+0000 7fd794cce700 1 --2- 172.21.15.82:0/4171611987 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd790103170 0x7fd79019ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:13:07.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:07.609+0000 7fd795cd0700 1 --2- 172.21.15.82:0/4171611987 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd790107080 0x7fd790198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:13:07.612 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:07.609+0000 7fd7954cf700 1 -- 172.21.15.82:0/4171611987 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd790107080 msgr2=0x7fd790198a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:07.612 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:07.609+0000 7fd7954cf700 1 --2- 172.21.15.82:0/4171611987 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd790107080 0x7fd790198a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:07.612 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:07.609+0000 7fd7954cf700 1 -- 172.21.15.82:0/4171611987 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd790103170 msgr2=0x7fd79019ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:07.612 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:07.609+0000 7fd7954cf700 1 --2- 172.21.15.82:0/4171611987 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd790103170 0x7fd79019ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:07.612 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:07.609+0000 7fd7954cf700 1 -- 172.21.15.82:0/4171611987 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd790199080 con 0x7fd790102820 2024-05-20T16:13:07.612 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:07.609+0000 7fd795cd0700 1 --2- 172.21.15.82:0/4171611987 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd790107080 0x7fd790198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T16:13:07.612 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:07.610+0000 7fd7954cf700 1 --2- 172.21.15.82:0/4171611987 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd790102820 0x7fd79019e820 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fd78c006010 tx=0x7fd78c0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:13:07.612 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:07.610+0000 7fd7867fc700 1 -- 172.21.15.82:0/4171611987 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd78c027070 con 0x7fd790102820 2024-05-20T16:13:07.612 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:07.610+0000 7fd797733700 1 -- 172.21.15.82:0/4171611987 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd790199310 con 0x7fd790102820 2024-05-20T16:13:07.612 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:07.610+0000 7fd7867fc700 1 -- 172.21.15.82:0/4171611987 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd78c004050 con 0x7fd790102820 2024-05-20T16:13:07.612 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:07.610+0000 7fd7867fc700 1 -- 172.21.15.82:0/4171611987 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd78c022c00 con 0x7fd790102820 2024-05-20T16:13:07.612 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:07.610+0000 7fd797733700 1 -- 172.21.15.82:0/4171611987 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd790199840 con 0x7fd790102820 2024-05-20T16:13:07.613 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:07.611+0000 7fd797733700 1 -- 172.21.15.82:0/4171611987 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd79010a620 con 0x7fd790102820 2024-05-20T16:13:07.613 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:07.613+0000 7fd7867fc700 1 -- 172.21.15.82:0/4171611987 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fd78c02a070 con 0x7fd790102820 2024-05-20T16:13:07.613 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:07.614+0000 7fd7867fc700 1 --2- 172.21.15.82:0/4171611987 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fd77c0a2cb0 0x7fd77c0a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:07.614 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:07.614+0000 7fd794cce700 1 --2- 172.21.15.82:0/4171611987 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fd77c0a2cb0 0x7fd77c0a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:13:07.614 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:07.614+0000 7fd7867fc700 1 -- 172.21.15.82:0/4171611987 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9092+0+0 (secure 0 0 0) 0x7fd78c0d0a10 con 0x7fd790102820 2024-05-20T16:13:07.614 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:07.614+0000 7fd794cce700 1 --2- 172.21.15.82:0/4171611987 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fd77c0a2cb0 0x7fd77c0a51c0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fd790105ab0 tx=0x7fd78000a380 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:13:07.617 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:07.617+0000 7fd7867fc700 1 -- 172.21.15.82:0/4171611987 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fd78c031080 con 0x7fd790102820 2024-05-20T16:13:07.635 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:13:07 smithi184 ceph-mon[36209]: pgmap v334: 129 pgs: 129 active+clean; 641 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-20T16:13:07.700 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:13:07 smithi082 ceph-mon[32238]: pgmap v334: 129 pgs: 129 active+clean; 641 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-20T16:13:07.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:13:07 smithi160 ceph-mon[36347]: pgmap v334: 129 pgs: 129 active+clean; 641 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-20T16:13:07.821 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:07.821+0000 7fd797733700 1 -- 172.21.15.82:0/4171611987 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7fd790199f70 con 0x7fd790102820 2024-05-20T16:13:07.821 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:07.821+0000 7fd7867fc700 1 -- 172.21.15.82:0/4171611987 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fd78c013070 con 0x7fd790102820 2024-05-20T16:13:07.822 INFO:teuthology.orchestra.run.smithi082.stdout:257698037804 2024-05-20T16:13:07.824 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:07.824+0000 7fd797733700 1 -- 172.21.15.82:0/4171611987 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fd77c0a2cb0 msgr2=0x7fd77c0a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:07.824 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:07.824+0000 7fd797733700 1 --2- 172.21.15.82:0/4171611987 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fd77c0a2cb0 0x7fd77c0a51c0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fd790105ab0 tx=0x7fd78000a380 comp rx=0 tx=0).stop 2024-05-20T16:13:07.824 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:07.824+0000 7fd797733700 1 -- 172.21.15.82:0/4171611987 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd790102820 msgr2=0x7fd79019e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:07.824 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:07.824+0000 7fd797733700 1 --2- 172.21.15.82:0/4171611987 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd790102820 0x7fd79019e820 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fd78c006010 tx=0x7fd78c0047b0 comp rx=0 tx=0).stop 2024-05-20T16:13:07.824 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:07.824+0000 7fd797733700 1 -- 172.21.15.82:0/4171611987 shutdown_connections 2024-05-20T16:13:07.824 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:07.824+0000 7fd797733700 1 --2- 172.21.15.82:0/4171611987 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fd77c0a2cb0 0x7fd77c0a51c0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:07.824 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:07.824+0000 7fd797733700 1 --2- 172.21.15.82:0/4171611987 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd790107080 0x7fd790198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:07.824 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:07.824+0000 7fd797733700 1 --2- 172.21.15.82:0/4171611987 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd790102820 0x7fd79019e820 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:07.824 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:07.824+0000 7fd797733700 1 --2- 172.21.15.82:0/4171611987 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd790103170 0x7fd79019ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:07.824 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:07.824+0000 7fd797733700 1 -- 172.21.15.82:0/4171611987 >> 172.21.15.82:0/4171611987 conn(0x7fd7900fd360 msgr2=0x7fd790106470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:13:07.824 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:07.824+0000 7fd797733700 1 -- 172.21.15.82:0/4171611987 shutdown_connections 2024-05-20T16:13:07.824 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:07.824+0000 7fd797733700 1 -- 172.21.15.82:0/4171611987 wait complete. 2024-05-20T16:13:08.549 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:13:08 smithi082 ceph-mon[32238]: from='client.? 172.21.15.82:0/4171611987' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-05-20T16:13:08.635 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:13:08 smithi184 ceph-mon[36209]: from='client.? 172.21.15.82:0/4171611987' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-05-20T16:13:08.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:13:08 smithi160 ceph-mon[36347]: from='client.? 172.21.15.82:0/4171611987' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-05-20T16:13:08.896 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738448 got 34359738452 for osd.0 2024-05-20T16:13:08.896 DEBUG:teuthology.parallel:result is None 2024-05-20T16:13:09.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:13:09 smithi184 ceph-mon[36209]: pgmap v335: 129 pgs: 129 active+clean; 641 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-20T16:13:09.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:13:09 smithi160 ceph-mon[36347]: pgmap v335: 129 pgs: 129 active+clean; 641 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-20T16:13:09.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:13:09 smithi082 ceph-mon[32238]: pgmap v335: 129 pgs: 129 active+clean; 641 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-20T16:13:11.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:13:11 smithi082 ceph-mon[32238]: pgmap v336: 129 pgs: 129 active+clean; 641 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-20T16:13:11.635 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:13:11 smithi184 ceph-mon[36209]: pgmap v336: 129 pgs: 129 active+clean; 641 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-20T16:13:11.777 INFO:tasks.cephadm.ceph_manager.ceph:need seq 257698037799 got 257698037804 for osd.8 2024-05-20T16:13:11.777 DEBUG:teuthology.parallel:result is None 2024-05-20T16:13:11.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:13:11 smithi160 ceph-mon[36347]: pgmap v336: 129 pgs: 129 active+clean; 641 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-20T16:13:13.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:13.259+0000 7ff5e6f9d700 1 -- 172.21.15.82:0/3472051578 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff5e0106a90 msgr2=0x7ff5e0106ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:13.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:13.259+0000 7ff5e6f9d700 1 --2- 172.21.15.82:0/3472051578 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff5e0106a90 0x7ff5e0106ea0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7ff5d00099c0 tx=0x7ff5d0009cd0 comp rx=0 tx=0).stop 2024-05-20T16:13:13.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:13.259+0000 7ff5e6f9d700 1 -- 172.21.15.82:0/3472051578 shutdown_connections 2024-05-20T16:13:13.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:13.259+0000 7ff5e6f9d700 1 --2- 172.21.15.82:0/3472051578 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff5e01073e0 0x7ff5e010ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:13.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:13.259+0000 7ff5e6f9d700 1 --2- 172.21.15.82:0/3472051578 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff5e0102160 0x7ff5e0102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:13.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:13.259+0000 7ff5e6f9d700 1 --2- 172.21.15.82:0/3472051578 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff5e0106a90 0x7ff5e0106ea0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:13.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:13.259+0000 7ff5e6f9d700 1 -- 172.21.15.82:0/3472051578 >> 172.21.15.82:0/3472051578 conn(0x7ff5e0077940 msgr2=0x7ff5e0077da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:13:13.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:13.260+0000 7ff5e6f9d700 1 -- 172.21.15.82:0/3472051578 shutdown_connections 2024-05-20T16:13:13.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:13.260+0000 7ff5e6f9d700 1 -- 172.21.15.82:0/3472051578 wait complete. 2024-05-20T16:13:13.262 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:13.260+0000 7ff5e6f9d700 1 Processor -- start 2024-05-20T16:13:13.262 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:13.260+0000 7ff5e6f9d700 1 -- start start 2024-05-20T16:13:13.262 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:13.261+0000 7ff5e6f9d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff5e0102160 0x7ff5e019e720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:13.262 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:13.261+0000 7ff5e6f9d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff5e0106a90 0x7ff5e019ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:13.262 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:13.261+0000 7ff5e6f9d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff5e01073e0 0x7ff5e0198940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:13.263 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:13.261+0000 7ff5e6f9d700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7ff5e0112160 con 0x7ff5e0102160 2024-05-20T16:13:13.263 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:13.261+0000 7ff5e6f9d700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7ff5e0111fe0 con 0x7ff5e0106a90 2024-05-20T16:13:13.263 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:13.261+0000 7ff5e6f9d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ff5e01122e0 con 0x7ff5e01073e0 2024-05-20T16:13:13.263 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:13.261+0000 7ff5e5f9b700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff5e0102160 0x7ff5e019e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:13:13.263 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:13.261+0000 7ff5e5f9b700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff5e0102160 0x7ff5e019e720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:48496/0 (socket says 172.21.15.82:48496) 2024-05-20T16:13:13.263 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:13.261+0000 7ff5e5f9b700 1 -- 172.21.15.82:0/3782261799 learned_addr learned my addr 172.21.15.82:0/3782261799 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:13:13.263 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:13.261+0000 7ff5e679c700 1 --2- 172.21.15.82:0/3782261799 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff5e01073e0 0x7ff5e0198940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:13:13.263 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:13.261+0000 7ff5e579a700 1 --2- 172.21.15.82:0/3782261799 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff5e0106a90 0x7ff5e019ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:13:13.263 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:13.261+0000 7ff5e5f9b700 1 -- 172.21.15.82:0/3782261799 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff5e01073e0 msgr2=0x7ff5e0198940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:13.263 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:13.261+0000 7ff5e5f9b700 1 --2- 172.21.15.82:0/3782261799 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff5e01073e0 0x7ff5e0198940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:13.263 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:13.261+0000 7ff5e5f9b700 1 -- 172.21.15.82:0/3782261799 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff5e0106a90 msgr2=0x7ff5e019ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:13.263 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:13.261+0000 7ff5e5f9b700 1 --2- 172.21.15.82:0/3782261799 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff5e0106a90 0x7ff5e019ec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:13.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:13.261+0000 7ff5e5f9b700 1 -- 172.21.15.82:0/3782261799 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5e0198f80 con 0x7ff5e0102160 2024-05-20T16:13:13.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:13.261+0000 7ff5e579a700 1 --2- 172.21.15.82:0/3782261799 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff5e0106a90 0x7ff5e019ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T16:13:13.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:13.262+0000 7ff5e5f9b700 1 --2- 172.21.15.82:0/3782261799 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff5e0102160 0x7ff5e019e720 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7ff5dc00c970 tx=0x7ff5dc00cd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:13:13.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:13.262+0000 7ff5d6ffd700 1 -- 172.21.15.82:0/3782261799 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5dc007aa0 con 0x7ff5e0102160 2024-05-20T16:13:13.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:13.262+0000 7ff5e6f9d700 1 -- 172.21.15.82:0/3782261799 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5e0199270 con 0x7ff5e0102160 2024-05-20T16:13:13.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:13.262+0000 7ff5d6ffd700 1 -- 172.21.15.82:0/3782261799 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff5dc004d10 con 0x7ff5e0102160 2024-05-20T16:13:13.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:13.262+0000 7ff5d6ffd700 1 -- 172.21.15.82:0/3782261799 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5dc005690 con 0x7ff5e0102160 2024-05-20T16:13:13.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:13.262+0000 7ff5e6f9d700 1 -- 172.21.15.82:0/3782261799 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5e0199730 con 0x7ff5e0102160 2024-05-20T16:13:13.265 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:13.263+0000 7ff5e6f9d700 1 -- 172.21.15.82:0/3782261799 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5e0050170 con 0x7ff5e0102160 2024-05-20T16:13:13.265 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:13.264+0000 7ff5d6ffd700 1 -- 172.21.15.82:0/3782261799 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7ff5dc004750 con 0x7ff5e0102160 2024-05-20T16:13:13.265 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:13.265+0000 7ff5d6ffd700 1 --2- 172.21.15.82:0/3782261799 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7ff5cc0a2cb0 0x7ff5cc0a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:13.265 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:13.265+0000 7ff5e579a700 1 --2- 172.21.15.82:0/3782261799 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7ff5cc0a2cb0 0x7ff5cc0a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:13:13.265 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:13.265+0000 7ff5d6ffd700 1 -- 172.21.15.82:0/3782261799 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9092+0+0 (secure 0 0 0) 0x7ff5dc0cf300 con 0x7ff5e0102160 2024-05-20T16:13:13.266 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:13.266+0000 7ff5e579a700 1 --2- 172.21.15.82:0/3782261799 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7ff5cc0a2cb0 0x7ff5cc0a51c0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7ff5d0000c00 tx=0x7ff5d001c040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:13:13.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:13.270+0000 7ff5d6ffd700 1 -- 172.21.15.82:0/3782261799 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7ff5dc09be30 con 0x7ff5e0102160 2024-05-20T16:13:13.494 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:13.493+0000 7ff5e6f9d700 1 -- 172.21.15.82:0/3782261799 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7ff5e0109e70 con 0x7ff5e0102160 2024-05-20T16:13:13.494 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:13.494+0000 7ff5d6ffd700 1 -- 172.21.15.82:0/3782261799 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7ff5dc09b730 con 0x7ff5e0102160 2024-05-20T16:13:13.495 INFO:teuthology.orchestra.run.smithi082.stdout:201863462968 2024-05-20T16:13:13.497 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:13.497+0000 7ff5e6f9d700 1 -- 172.21.15.82:0/3782261799 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7ff5cc0a2cb0 msgr2=0x7ff5cc0a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:13.497 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:13.497+0000 7ff5e6f9d700 1 --2- 172.21.15.82:0/3782261799 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7ff5cc0a2cb0 0x7ff5cc0a51c0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7ff5d0000c00 tx=0x7ff5d001c040 comp rx=0 tx=0).stop 2024-05-20T16:13:13.497 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:13.497+0000 7ff5e6f9d700 1 -- 172.21.15.82:0/3782261799 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff5e0102160 msgr2=0x7ff5e019e720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:13.498 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:13.497+0000 7ff5e6f9d700 1 --2- 172.21.15.82:0/3782261799 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff5e0102160 0x7ff5e019e720 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7ff5dc00c970 tx=0x7ff5dc00cd30 comp rx=0 tx=0).stop 2024-05-20T16:13:13.498 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:13.497+0000 7ff5e6f9d700 1 -- 172.21.15.82:0/3782261799 shutdown_connections 2024-05-20T16:13:13.498 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:13.497+0000 7ff5e6f9d700 1 --2- 172.21.15.82:0/3782261799 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7ff5cc0a2cb0 0x7ff5cc0a51c0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:13.498 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:13.497+0000 7ff5e6f9d700 1 --2- 172.21.15.82:0/3782261799 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff5e01073e0 0x7ff5e0198940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:13.498 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:13.497+0000 7ff5e6f9d700 1 --2- 172.21.15.82:0/3782261799 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff5e0102160 0x7ff5e019e720 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:13.498 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:13.497+0000 7ff5e6f9d700 1 --2- 172.21.15.82:0/3782261799 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff5e0106a90 0x7ff5e019ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:13.498 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:13.497+0000 7ff5e6f9d700 1 -- 172.21.15.82:0/3782261799 >> 172.21.15.82:0/3782261799 conn(0x7ff5e0077940 msgr2=0x7ff5e00ff610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:13:13.498 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:13.497+0000 7ff5e6f9d700 1 -- 172.21.15.82:0/3782261799 shutdown_connections 2024-05-20T16:13:13.498 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:13.497+0000 7ff5e6f9d700 1 -- 172.21.15.82:0/3782261799 wait complete. 2024-05-20T16:13:13.587 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:13.159+0000 7f7c2c782700 1 -- 172.21.15.82:0/2586419640 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7c241086d0 msgr2=0x7f7c24110030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:13.587 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:13.159+0000 7f7c2c782700 1 --2- 172.21.15.82:0/2586419640 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7c241086d0 0x7f7c24110030 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f7c1c00b0d0 tx=0x7f7c1c00b3e0 comp rx=0 tx=0).stop 2024-05-20T16:13:13.587 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:13.159+0000 7f7c2c782700 1 -- 172.21.15.82:0/2586419640 shutdown_connections 2024-05-20T16:13:13.587 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:13.159+0000 7f7c2c782700 1 --2- 172.21.15.82:0/2586419640 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7c24103450 0x7f7c24103860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:13.587 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:13.159+0000 7f7c2c782700 1 --2- 172.21.15.82:0/2586419640 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7c241086d0 0x7f7c24110030 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:13.587 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:13.159+0000 7f7c2c782700 1 --2- 172.21.15.82:0/2586419640 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7c24107d80 0x7f7c24108190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:13.587 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:13.159+0000 7f7c2c782700 1 -- 172.21.15.82:0/2586419640 >> 172.21.15.82:0/2586419640 conn(0x7f7c240fed20 msgr2=0x7f7c24101170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:13:13.587 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:13.160+0000 7f7c2c782700 1 -- 172.21.15.82:0/2586419640 shutdown_connections 2024-05-20T16:13:13.588 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:13.160+0000 7f7c2c782700 1 -- 172.21.15.82:0/2586419640 wait complete. 2024-05-20T16:13:13.588 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:13.160+0000 7f7c2c782700 1 Processor -- start 2024-05-20T16:13:13.588 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:13.160+0000 7f7c2c782700 1 -- start start 2024-05-20T16:13:13.588 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:13.161+0000 7f7c2c782700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7c24103450 0x7f7c2419f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:13.588 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:13.161+0000 7f7c2c782700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7c24107d80 0x7f7c2419ff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:13.588 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:13.161+0000 7f7c2c782700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7c241086d0 0x7f7c24199b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:13.588 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:13.161+0000 7f7c2c782700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f7c24113470 con 0x7f7c24107d80 2024-05-20T16:13:13.588 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:13.161+0000 7f7c2c782700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f7c241132f0 con 0x7f7c241086d0 2024-05-20T16:13:13.588 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:13.161+0000 7f7c2c782700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f7c241135f0 con 0x7f7c24103450 2024-05-20T16:13:13.588 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:13.161+0000 7f7c29d1d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7c24107d80 0x7f7c2419ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:13:13.588 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:13.161+0000 7f7c29d1d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7c24107d80 0x7f7c2419ff00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:48470/0 (socket says 172.21.15.82:48470) 2024-05-20T16:13:13.588 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:13.161+0000 7f7c29d1d700 1 -- 172.21.15.82:0/1793765914 learned_addr learned my addr 172.21.15.82:0/1793765914 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:13:13.588 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:13.161+0000 7f7c2a51e700 1 --2- 172.21.15.82:0/1793765914 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7c24103450 0x7f7c2419f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:13:13.588 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:13.161+0000 7f7c2ad1f700 1 --2- 172.21.15.82:0/1793765914 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7c241086d0 0x7f7c24199b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:13:13.589 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:13.161+0000 7f7c29d1d700 1 -- 172.21.15.82:0/1793765914 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7c24103450 msgr2=0x7f7c2419f9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:13.589 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:13.161+0000 7f7c29d1d700 1 --2- 172.21.15.82:0/1793765914 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7c24103450 0x7f7c2419f9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:13.589 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:13.161+0000 7f7c29d1d700 1 -- 172.21.15.82:0/1793765914 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7c241086d0 msgr2=0x7f7c24199b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:13.589 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:13.161+0000 7f7c29d1d700 1 --2- 172.21.15.82:0/1793765914 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7c241086d0 0x7f7c24199b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:13.589 INFO:teuthology.orchestra.run.smithi082.stdout:55834574929 2024-05-20T16:13:13.589 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:13.161+0000 7f7c29d1d700 1 -- 172.21.15.82:0/1793765914 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c2419a280 con 0x7f7c24107d80 2024-05-20T16:13:13.590 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:13.162+0000 7f7c29d1d700 1 --2- 172.21.15.82:0/1793765914 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7c24107d80 0x7f7c2419ff00 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f7c1400e9f0 tx=0x7f7c1400ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:13:13.590 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:13.162+0000 7f7c1b7fe700 1 -- 172.21.15.82:0/1793765914 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c14004a30 con 0x7f7c24107d80 2024-05-20T16:13:13.590 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:13.162+0000 7f7c2c782700 1 -- 172.21.15.82:0/1793765914 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c2419a570 con 0x7f7c24107d80 2024-05-20T16:13:13.590 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:13.162+0000 7f7c2c782700 1 -- 172.21.15.82:0/1793765914 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c241a6770 con 0x7f7c24107d80 2024-05-20T16:13:13.590 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:13.162+0000 7f7c1b7fe700 1 -- 172.21.15.82:0/1793765914 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c1400cca0 con 0x7f7c24107d80 2024-05-20T16:13:13.590 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:13.162+0000 7f7c1b7fe700 1 -- 172.21.15.82:0/1793765914 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c140056c0 con 0x7f7c24107d80 2024-05-20T16:13:13.590 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:13.163+0000 7f7c2c782700 1 -- 172.21.15.82:0/1793765914 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c24104900 con 0x7f7c24107d80 2024-05-20T16:13:13.590 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:13.166+0000 7f7c1b7fe700 1 -- 172.21.15.82:0/1793765914 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f7c1400c7c0 con 0x7f7c24107d80 2024-05-20T16:13:13.590 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:13.167+0000 7f7c1b7fe700 1 --2- 172.21.15.82:0/1793765914 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f7c100a2cb0 0x7f7c100a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:13.590 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:13.167+0000 7f7c2a51e700 1 --2- 172.21.15.82:0/1793765914 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f7c100a2cb0 0x7f7c100a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:13:13.590 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:13.167+0000 7f7c1b7fe700 1 -- 172.21.15.82:0/1793765914 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9092+0+0 (secure 0 0 0) 0x7f7c140cf8a0 con 0x7f7c24107d80 2024-05-20T16:13:13.590 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:13.167+0000 7f7c2a51e700 1 --2- 172.21.15.82:0/1793765914 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f7c100a2cb0 0x7f7c100a51c0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f7c20009c60 tx=0x7f7c20009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:13:13.591 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:13.170+0000 7f7c1b7fe700 1 -- 172.21.15.82:0/1793765914 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f7c14053850 con 0x7f7c24107d80 2024-05-20T16:13:13.591 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:13.422+0000 7f7c2c782700 1 -- 172.21.15.82:0/1793765914 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f7c2402dc60 con 0x7f7c24107d80 2024-05-20T16:13:13.591 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:13.422+0000 7f7c1b7fe700 1 -- 172.21.15.82:0/1793765914 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f7c1409c340 con 0x7f7c24107d80 2024-05-20T16:13:13.591 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:13.424+0000 7f7c2c782700 1 -- 172.21.15.82:0/1793765914 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f7c100a2cb0 msgr2=0x7f7c100a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:13.591 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:13.424+0000 7f7c2c782700 1 --2- 172.21.15.82:0/1793765914 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f7c100a2cb0 0x7f7c100a51c0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f7c20009c60 tx=0x7f7c20009260 comp rx=0 tx=0).stop 2024-05-20T16:13:13.591 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:13.424+0000 7f7c2c782700 1 -- 172.21.15.82:0/1793765914 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7c24107d80 msgr2=0x7f7c2419ff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:13.591 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:13.424+0000 7f7c2c782700 1 --2- 172.21.15.82:0/1793765914 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7c24107d80 0x7f7c2419ff00 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f7c1400e9f0 tx=0x7f7c1400ed00 comp rx=0 tx=0).stop 2024-05-20T16:13:13.591 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:13.424+0000 7f7c2c782700 1 -- 172.21.15.82:0/1793765914 shutdown_connections 2024-05-20T16:13:13.591 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:13.424+0000 7f7c2c782700 1 --2- 172.21.15.82:0/1793765914 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f7c100a2cb0 0x7f7c100a51c0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:13.591 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:13.424+0000 7f7c2c782700 1 --2- 172.21.15.82:0/1793765914 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7c241086d0 0x7f7c24199b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:13.591 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:13.424+0000 7f7c2c782700 1 --2- 172.21.15.82:0/1793765914 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7c24103450 0x7f7c2419f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:13.591 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:13.424+0000 7f7c2c782700 1 --2- 172.21.15.82:0/1793765914 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7c24107d80 0x7f7c2419ff00 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:13.591 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:13.424+0000 7f7c2c782700 1 -- 172.21.15.82:0/1793765914 >> 172.21.15.82:0/1793765914 conn(0x7f7c240fed20 msgr2=0x7f7c241074c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:13:13.591 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:13.424+0000 7f7c2c782700 1 -- 172.21.15.82:0/1793765914 shutdown_connections 2024-05-20T16:13:13.592 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:13.425+0000 7f7c2c782700 1 -- 172.21.15.82:0/1793765914 wait complete. 2024-05-20T16:13:13.592 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:13:13 smithi082 ceph-mon[32238]: pgmap v337: 129 pgs: 129 active+clean; 641 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-20T16:13:13.635 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:13:13 smithi184 ceph-mon[36209]: pgmap v337: 129 pgs: 129 active+clean; 641 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-20T16:13:13.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:13:13 smithi160 ceph-mon[36347]: pgmap v337: 129 pgs: 129 active+clean; 641 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-20T16:13:14.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:13:14 smithi082 ceph-mon[32238]: from='client.? 172.21.15.82:0/1793765914' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-05-20T16:13:14.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:13:14 smithi082 ceph-mon[32238]: from='client.? 172.21.15.82:0/3782261799' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-05-20T16:13:14.635 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:13:14 smithi184 ceph-mon[36209]: from='client.? 172.21.15.82:0/1793765914' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-05-20T16:13:14.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:13:14 smithi184 ceph-mon[36209]: from='client.? 172.21.15.82:0/3782261799' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-05-20T16:13:14.773 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:14.773+0000 7fb5ae822700 1 -- 172.21.15.82:0/1896888486 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb5a8107600 msgr2=0x7fb5a810ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:14.774 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:14.773+0000 7fb5ae822700 1 --2- 172.21.15.82:0/1896888486 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb5a8107600 0x7fb5a810ef60 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fb5a000b0d0 tx=0x7fb5a000b3e0 comp rx=0 tx=0).stop 2024-05-20T16:13:14.774 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:14.773+0000 7fb5ae822700 1 -- 172.21.15.82:0/1896888486 shutdown_connections 2024-05-20T16:13:14.774 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:14.773+0000 7fb5ae822700 1 --2- 172.21.15.82:0/1896888486 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb5a81023d0 0x7fb5a81027e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:14.774 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:14.773+0000 7fb5ae822700 1 --2- 172.21.15.82:0/1896888486 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb5a8107600 0x7fb5a810ef60 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:14.774 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:14.773+0000 7fb5ae822700 1 --2- 172.21.15.82:0/1896888486 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb5a8106cb0 0x7fb5a81070c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:14.774 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:14.773+0000 7fb5ae822700 1 -- 172.21.15.82:0/1896888486 >> 172.21.15.82:0/1896888486 conn(0x7fb5a8077c80 msgr2=0x7fb5a80780c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:13:14.774 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:14.773+0000 7fb5ae822700 1 -- 172.21.15.82:0/1896888486 shutdown_connections 2024-05-20T16:13:14.774 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:14.773+0000 7fb5ae822700 1 -- 172.21.15.82:0/1896888486 wait complete. 2024-05-20T16:13:14.774 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:14.774+0000 7fb5ae822700 1 Processor -- start 2024-05-20T16:13:14.774 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:14.774+0000 7fb5ae822700 1 -- start start 2024-05-20T16:13:14.776 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:14.774+0000 7fb5ae822700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb5a81023d0 0x7fb5a819e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:14.776 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:14.774+0000 7fb5ae822700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb5a8106cb0 0x7fb5a819eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:14.776 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:14.774+0000 7fb5ae822700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb5a8107600 0x7fb5a8198b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:14.776 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:14.774+0000 7fb5ae822700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fb5a8112380 con 0x7fb5a8106cb0 2024-05-20T16:13:14.777 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:14.774+0000 7fb5ae822700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fb5a8112200 con 0x7fb5a8107600 2024-05-20T16:13:14.777 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:14.774+0000 7fb5ae822700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fb5a8112500 con 0x7fb5a81023d0 2024-05-20T16:13:14.777 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:14.774+0000 7fb5a77fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb5a8106cb0 0x7fb5a819eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:13:14.777 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:14.774+0000 7fb5a77fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb5a8106cb0 0x7fb5a819eef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:48526/0 (socket says 172.21.15.82:48526) 2024-05-20T16:13:14.777 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:14.774+0000 7fb5a77fe700 1 -- 172.21.15.82:0/2787431513 learned_addr learned my addr 172.21.15.82:0/2787431513 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:13:14.777 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:14.775+0000 7fb5a7fff700 1 --2- 172.21.15.82:0/2787431513 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb5a81023d0 0x7fb5a819e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:13:14.777 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:14.775+0000 7fb5acdbf700 1 --2- 172.21.15.82:0/2787431513 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb5a8107600 0x7fb5a8198b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:13:14.777 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:14.775+0000 7fb5a77fe700 1 -- 172.21.15.82:0/2787431513 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb5a81023d0 msgr2=0x7fb5a819e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:14.777 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:14.775+0000 7fb5a77fe700 1 --2- 172.21.15.82:0/2787431513 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb5a81023d0 0x7fb5a819e9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:14.777 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:14.775+0000 7fb5a77fe700 1 -- 172.21.15.82:0/2787431513 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb5a8107600 msgr2=0x7fb5a8198b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:14.777 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:14.775+0000 7fb5a77fe700 1 --2- 172.21.15.82:0/2787431513 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb5a8107600 0x7fb5a8198b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:14.777 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:14.775+0000 7fb5a77fe700 1 -- 172.21.15.82:0/2787431513 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5a81991e0 con 0x7fb5a8106cb0 2024-05-20T16:13:14.777 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:14.775+0000 7fb5acdbf700 1 --2- 172.21.15.82:0/2787431513 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb5a8107600 0x7fb5a8198b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T16:13:14.778 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:14.775+0000 7fb5a77fe700 1 --2- 172.21.15.82:0/2787431513 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb5a8106cb0 0x7fb5a819eef0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fb59c00c9e0 tx=0x7fb59c00cda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:13:14.778 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:14.775+0000 7fb5a57fa700 1 -- 172.21.15.82:0/2787431513 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb59c007aa0 con 0x7fb5a8106cb0 2024-05-20T16:13:14.778 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:14.775+0000 7fb5ae822700 1 -- 172.21.15.82:0/2787431513 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5a81994d0 con 0x7fb5a8106cb0 2024-05-20T16:13:14.778 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:14.775+0000 7fb5ae822700 1 -- 172.21.15.82:0/2787431513 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5a81a5760 con 0x7fb5a8106cb0 2024-05-20T16:13:14.778 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:14.775+0000 7fb5a57fa700 1 -- 172.21.15.82:0/2787431513 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb59c007c40 con 0x7fb5a8106cb0 2024-05-20T16:13:14.778 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:14.775+0000 7fb5a57fa700 1 -- 172.21.15.82:0/2787431513 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb59c018780 con 0x7fb5a8106cb0 2024-05-20T16:13:14.778 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:14.776+0000 7fb5ae822700 1 -- 172.21.15.82:0/2787431513 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb594005330 con 0x7fb5a8106cb0 2024-05-20T16:13:14.779 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:14.778+0000 7fb5a57fa700 1 -- 172.21.15.82:0/2787431513 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fb59c018920 con 0x7fb5a8106cb0 2024-05-20T16:13:14.779 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:14.778+0000 7fb5a57fa700 1 --2- 172.21.15.82:0/2787431513 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fb5900a2cb0 0x7fb5900a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:14.779 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:14.778+0000 7fb5a7fff700 1 --2- 172.21.15.82:0/2787431513 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fb5900a2cb0 0x7fb5900a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:13:14.779 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:14.778+0000 7fb5a57fa700 1 -- 172.21.15.82:0/2787431513 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9092+0+0 (secure 0 0 0) 0x7fb59c0cf310 con 0x7fb5a8106cb0 2024-05-20T16:13:14.779 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:14.779+0000 7fb5a7fff700 1 --2- 172.21.15.82:0/2787431513 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fb5900a2cb0 0x7fb5900a51c0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fb598005950 tx=0x7fb5980058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:13:14.783 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:14.783+0000 7fb5a57fa700 1 -- 172.21.15.82:0/2787431513 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fb59c014030 con 0x7fb5a8106cb0 2024-05-20T16:13:14.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:13:14 smithi160 ceph-mon[36347]: from='client.? 172.21.15.82:0/1793765914' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-05-20T16:13:14.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:13:14 smithi160 ceph-mon[36347]: from='client.? 172.21.15.82:0/3782261799' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-05-20T16:13:14.992 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:14.992+0000 7fb5ae822700 1 -- 172.21.15.82:0/2787431513 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7fb5940058f0 con 0x7fb5a8106cb0 2024-05-20T16:13:14.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:14.993+0000 7fb5a57fa700 1 -- 172.21.15.82:0/2787431513 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fb59c09be50 con 0x7fb5a8106cb0 2024-05-20T16:13:14.993 INFO:teuthology.orchestra.run.smithi082.stdout:146028888130 2024-05-20T16:13:14.995 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:14.995+0000 7fb5ae822700 1 -- 172.21.15.82:0/2787431513 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fb5900a2cb0 msgr2=0x7fb5900a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:14.995 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:14.995+0000 7fb5ae822700 1 --2- 172.21.15.82:0/2787431513 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fb5900a2cb0 0x7fb5900a51c0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fb598005950 tx=0x7fb5980058e0 comp rx=0 tx=0).stop 2024-05-20T16:13:14.995 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:14.995+0000 7fb5ae822700 1 -- 172.21.15.82:0/2787431513 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb5a8106cb0 msgr2=0x7fb5a819eef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:14.995 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:14.995+0000 7fb5ae822700 1 --2- 172.21.15.82:0/2787431513 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb5a8106cb0 0x7fb5a819eef0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fb59c00c9e0 tx=0x7fb59c00cda0 comp rx=0 tx=0).stop 2024-05-20T16:13:14.995 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:14.995+0000 7fb5ae822700 1 -- 172.21.15.82:0/2787431513 shutdown_connections 2024-05-20T16:13:14.995 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:14.995+0000 7fb5ae822700 1 --2- 172.21.15.82:0/2787431513 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fb5900a2cb0 0x7fb5900a51c0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:14.995 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:14.995+0000 7fb5ae822700 1 --2- 172.21.15.82:0/2787431513 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb5a8107600 0x7fb5a8198b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:14.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:14.995+0000 7fb5ae822700 1 --2- 172.21.15.82:0/2787431513 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb5a81023d0 0x7fb5a819e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:14.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:14.995+0000 7fb5ae822700 1 --2- 172.21.15.82:0/2787431513 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb5a8106cb0 0x7fb5a819eef0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:14.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:14.995+0000 7fb5ae822700 1 -- 172.21.15.82:0/2787431513 >> 172.21.15.82:0/2787431513 conn(0x7fb5a8077c80 msgr2=0x7fb5a8100c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:13:14.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:14.995+0000 7fb5ae822700 1 -- 172.21.15.82:0/2787431513 shutdown_connections 2024-05-20T16:13:14.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:14.995+0000 7fb5ae822700 1 -- 172.21.15.82:0/2787431513 wait complete. 2024-05-20T16:13:15.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:13:15 smithi184 ceph-mon[36209]: from='client.? 172.21.15.82:0/3437035142' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-05-20T16:13:15.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:13:15 smithi184 ceph-mon[36209]: from='client.? 172.21.15.82:0/2787431513' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-05-20T16:13:15.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:13:15 smithi184 ceph-mon[36209]: pgmap v338: 129 pgs: 129 active+clean; 641 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-20T16:13:15.671 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:14.660+0000 7fac7b5c6700 1 -- 172.21.15.82:0/2223076926 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fac741086d0 msgr2=0x7fac74110030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:15.671 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:14.660+0000 7fac7b5c6700 1 --2- 172.21.15.82:0/2223076926 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fac741086d0 0x7fac74110030 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fac6c00b0d0 tx=0x7fac6c00b3e0 comp rx=0 tx=0).stop 2024-05-20T16:13:15.671 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:14.660+0000 7fac7b5c6700 1 -- 172.21.15.82:0/2223076926 shutdown_connections 2024-05-20T16:13:15.671 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:14.660+0000 7fac7b5c6700 1 --2- 172.21.15.82:0/2223076926 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fac741019d0 0x7fac74101de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:15.671 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:14.660+0000 7fac7b5c6700 1 --2- 172.21.15.82:0/2223076926 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fac741086d0 0x7fac74110030 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:15.671 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:14.660+0000 7fac7b5c6700 1 --2- 172.21.15.82:0/2223076926 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fac74107d80 0x7fac74108190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:15.671 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:14.660+0000 7fac7b5c6700 1 -- 172.21.15.82:0/2223076926 >> 172.21.15.82:0/2223076926 conn(0x7fac740fd570 msgr2=0x7fac740ff990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:13:15.671 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:14.660+0000 7fac7b5c6700 1 -- 172.21.15.82:0/2223076926 shutdown_connections 2024-05-20T16:13:15.671 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:14.660+0000 7fac7b5c6700 1 -- 172.21.15.82:0/2223076926 wait complete. 2024-05-20T16:13:15.672 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:14.661+0000 7fac7b5c6700 1 Processor -- start 2024-05-20T16:13:15.672 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:14.661+0000 7fac7b5c6700 1 -- start start 2024-05-20T16:13:15.672 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:14.661+0000 7fac7b5c6700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fac741019d0 0x7fac74102920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:15.672 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:14.661+0000 7fac7b5c6700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fac74107d80 0x7fac74102e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:15.672 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:14.661+0000 7fac7b5c6700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fac741086d0 0x7fac741033a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:15.672 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:14.661+0000 7fac7b5c6700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fac741134c0 con 0x7fac74107d80 2024-05-20T16:13:15.672 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:14.661+0000 7fac7b5c6700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fac74113340 con 0x7fac741019d0 2024-05-20T16:13:15.672 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:14.661+0000 7fac7b5c6700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fac74113640 con 0x7fac741086d0 2024-05-20T16:13:15.672 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:14.662+0000 7fac78b61700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fac74107d80 0x7fac74102e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:13:15.672 INFO:teuthology.orchestra.run.smithi082.stdout:347892351005 2024-05-20T16:13:15.673 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:14.662+0000 7fac78b61700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fac74107d80 0x7fac74102e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:48510/0 (socket says 172.21.15.82:48510) 2024-05-20T16:13:15.673 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:14.662+0000 7fac78b61700 1 -- 172.21.15.82:0/3437035142 learned_addr learned my addr 172.21.15.82:0/3437035142 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:13:15.673 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:14.662+0000 7fac79362700 1 --2- 172.21.15.82:0/3437035142 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fac741019d0 0x7fac74102920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:13:15.673 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:14.662+0000 7fac79b63700 1 --2- 172.21.15.82:0/3437035142 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fac741086d0 0x7fac741033a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:13:15.673 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:14.662+0000 7fac78b61700 1 -- 172.21.15.82:0/3437035142 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fac741086d0 msgr2=0x7fac741033a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:15.673 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:14.662+0000 7fac78b61700 1 --2- 172.21.15.82:0/3437035142 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fac741086d0 0x7fac741033a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:15.673 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:14.662+0000 7fac78b61700 1 -- 172.21.15.82:0/3437035142 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fac741019d0 msgr2=0x7fac74102920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:15.673 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:14.662+0000 7fac78b61700 1 --2- 172.21.15.82:0/3437035142 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fac741019d0 0x7fac74102920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:15.674 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:14.662+0000 7fac78b61700 1 -- 172.21.15.82:0/3437035142 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac741a80d0 con 0x7fac74107d80 2024-05-20T16:13:15.674 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:14.662+0000 7fac78b61700 1 --2- 172.21.15.82:0/3437035142 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fac74107d80 0x7fac74102e60 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fac6400b500 tx=0x7fac6400b810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:13:15.674 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:14.662+0000 7fac6a7fc700 1 -- 172.21.15.82:0/3437035142 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac64015020 con 0x7fac74107d80 2024-05-20T16:13:15.674 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:14.662+0000 7fac7b5c6700 1 -- 172.21.15.82:0/3437035142 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac741a83c0 con 0x7fac74107d80 2024-05-20T16:13:15.674 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:14.662+0000 7fac6a7fc700 1 -- 172.21.15.82:0/3437035142 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fac64011d10 con 0x7fac74107d80 2024-05-20T16:13:15.674 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:14.662+0000 7fac6a7fc700 1 -- 172.21.15.82:0/3437035142 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac64010de0 con 0x7fac74107d80 2024-05-20T16:13:15.674 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:14.662+0000 7fac7b5c6700 1 -- 172.21.15.82:0/3437035142 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac741a8950 con 0x7fac74107d80 2024-05-20T16:13:15.674 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:14.664+0000 7fac7b5c6700 1 -- 172.21.15.82:0/3437035142 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac74050170 con 0x7fac74107d80 2024-05-20T16:13:15.674 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:14.665+0000 7fac6a7fc700 1 -- 172.21.15.82:0/3437035142 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fac64018020 con 0x7fac74107d80 2024-05-20T16:13:15.674 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:14.665+0000 7fac6a7fc700 1 --2- 172.21.15.82:0/3437035142 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fac600a2cb0 0x7fac600a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:15.674 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:14.665+0000 7fac79362700 1 --2- 172.21.15.82:0/3437035142 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fac600a2cb0 0x7fac600a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:13:15.674 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:14.665+0000 7fac6a7fc700 1 -- 172.21.15.82:0/3437035142 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9092+0+0 (secure 0 0 0) 0x7fac640ceae0 con 0x7fac74107d80 2024-05-20T16:13:15.674 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:14.666+0000 7fac79362700 1 --2- 172.21.15.82:0/3437035142 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fac600a2cb0 0x7fac600a51c0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fac70005950 tx=0x7fac7000b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:13:15.675 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:14.674+0000 7fac6a7fc700 1 -- 172.21.15.82:0/3437035142 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fac6401f020 con 0x7fac74107d80 2024-05-20T16:13:15.675 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:14.881+0000 7fac7b5c6700 1 -- 172.21.15.82:0/3437035142 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7fac74103e50 con 0x7fac74107d80 2024-05-20T16:13:15.675 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:14.882+0000 7fac6a7fc700 1 -- 172.21.15.82:0/3437035142 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7fac6409b690 con 0x7fac74107d80 2024-05-20T16:13:15.675 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:14.884+0000 7fac7b5c6700 1 -- 172.21.15.82:0/3437035142 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fac600a2cb0 msgr2=0x7fac600a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:15.675 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:14.884+0000 7fac7b5c6700 1 --2- 172.21.15.82:0/3437035142 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fac600a2cb0 0x7fac600a51c0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fac70005950 tx=0x7fac7000b410 comp rx=0 tx=0).stop 2024-05-20T16:13:15.675 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:14.884+0000 7fac7b5c6700 1 -- 172.21.15.82:0/3437035142 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fac74107d80 msgr2=0x7fac74102e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:15.675 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:14.884+0000 7fac7b5c6700 1 --2- 172.21.15.82:0/3437035142 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fac74107d80 0x7fac74102e60 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fac6400b500 tx=0x7fac6400b810 comp rx=0 tx=0).stop 2024-05-20T16:13:15.675 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:14.884+0000 7fac7b5c6700 1 -- 172.21.15.82:0/3437035142 shutdown_connections 2024-05-20T16:13:15.675 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:14.884+0000 7fac7b5c6700 1 --2- 172.21.15.82:0/3437035142 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fac600a2cb0 0x7fac600a51c0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:15.675 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:14.884+0000 7fac7b5c6700 1 --2- 172.21.15.82:0/3437035142 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fac741019d0 0x7fac74102920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:15.675 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:14.884+0000 7fac7b5c6700 1 --2- 172.21.15.82:0/3437035142 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fac741086d0 0x7fac741033a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:15.675 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:14.884+0000 7fac7b5c6700 1 --2- 172.21.15.82:0/3437035142 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fac74107d80 0x7fac74102e60 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:15.675 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:14.884+0000 7fac7b5c6700 1 -- 172.21.15.82:0/3437035142 >> 172.21.15.82:0/3437035142 conn(0x7fac740fd570 msgr2=0x7fac74107400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:13:15.675 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:14.884+0000 7fac7b5c6700 1 -- 172.21.15.82:0/3437035142 shutdown_connections 2024-05-20T16:13:15.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:14.884+0000 7fac7b5c6700 1 -- 172.21.15.82:0/3437035142 wait complete. 2024-05-20T16:13:15.676 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:13:15 smithi082 ceph-mon[32238]: from='client.? 172.21.15.82:0/3437035142' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-05-20T16:13:15.676 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:13:15 smithi082 ceph-mon[32238]: from='client.? 172.21.15.82:0/2787431513' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-05-20T16:13:15.676 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:13:15 smithi082 ceph-mon[32238]: pgmap v338: 129 pgs: 129 active+clean; 641 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-20T16:13:15.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:13:15 smithi160 ceph-mon[36347]: from='client.? 172.21.15.82:0/3437035142' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-05-20T16:13:15.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:13:15 smithi160 ceph-mon[36347]: from='client.? 172.21.15.82:0/2787431513' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-05-20T16:13:15.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:13:15 smithi160 ceph-mon[36347]: pgmap v338: 129 pgs: 129 active+clean; 641 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-20T16:13:16.416 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574925 got 55834574929 for osd.1 2024-05-20T16:13:16.416 DEBUG:teuthology.parallel:result is None 2024-05-20T16:13:16.635 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:13:16 smithi184 ceph-mon[36209]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-05-20T16:13:16.646 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:13:16 smithi082 ceph-mon[32238]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-05-20T16:13:16.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:13:16 smithi160 ceph-mon[36347]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-05-20T16:13:17.352 INFO:tasks.cephadm.ceph_manager.ceph:need seq 201863462963 got 201863462968 for osd.6 2024-05-20T16:13:17.352 DEBUG:teuthology.parallel:result is None 2024-05-20T16:13:17.560 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:17.560+0000 7fd0d556a700 1 -- 172.21.15.82:0/1290336542 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd0d01086e0 msgr2=0x7fd0d0110040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:17.560 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:17.560+0000 7fd0d556a700 1 --2- 172.21.15.82:0/1290336542 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd0d01086e0 0x7fd0d0110040 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fd0c800b0d0 tx=0x7fd0c800b3e0 comp rx=0 tx=0).stop 2024-05-20T16:13:17.560 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:17.560+0000 7fd0d556a700 1 -- 172.21.15.82:0/1290336542 shutdown_connections 2024-05-20T16:13:17.560 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:17.560+0000 7fd0d556a700 1 --2- 172.21.15.82:0/1290336542 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd0d0103460 0x7fd0d0103870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:17.560 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:17.560+0000 7fd0d556a700 1 --2- 172.21.15.82:0/1290336542 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd0d01086e0 0x7fd0d0110040 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:17.561 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:17.560+0000 7fd0d556a700 1 --2- 172.21.15.82:0/1290336542 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd0d0107d90 0x7fd0d01081a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:17.561 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:17.560+0000 7fd0d556a700 1 -- 172.21.15.82:0/1290336542 >> 172.21.15.82:0/1290336542 conn(0x7fd0d00fecf0 msgr2=0x7fd0d0101140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:13:17.561 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:17.560+0000 7fd0d556a700 1 -- 172.21.15.82:0/1290336542 shutdown_connections 2024-05-20T16:13:17.561 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:17.560+0000 7fd0d556a700 1 -- 172.21.15.82:0/1290336542 wait complete. 2024-05-20T16:13:17.561 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:17.561+0000 7fd0d556a700 1 Processor -- start 2024-05-20T16:13:17.562 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:17.562+0000 7fd0d556a700 1 -- start start 2024-05-20T16:13:17.562 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:17.562+0000 7fd0d556a700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd0d0103460 0x7fd0d00764e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:17.562 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:17.562+0000 7fd0d556a700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd0d0107d90 0x7fd0d0074940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:17.562 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:17.562+0000 7fd0d556a700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd0d01086e0 0x7fd0d0074f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:17.562 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:17.562+0000 7fd0d556a700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fd0d0113460 con 0x7fd0d01086e0 2024-05-20T16:13:17.562 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:17.562+0000 7fd0d556a700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fd0d01132e0 con 0x7fd0d0103460 2024-05-20T16:13:17.562 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:17.562+0000 7fd0d556a700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd0d01135e0 con 0x7fd0d0107d90 2024-05-20T16:13:17.564 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:17.563+0000 7fd0cf7fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd0d01086e0 0x7fd0d0074f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:13:17.564 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:17.563+0000 7fd0cf7fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd0d01086e0 0x7fd0d0074f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:48536/0 (socket says 172.21.15.82:48536) 2024-05-20T16:13:17.564 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:17.563+0000 7fd0cf7fe700 1 -- 172.21.15.82:0/4250859085 learned_addr learned my addr 172.21.15.82:0/4250859085 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:13:17.564 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:17.563+0000 7fd0ceffd700 1 --2- 172.21.15.82:0/4250859085 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd0d0103460 0x7fd0d00764e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:13:17.564 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:17.563+0000 7fd0ce7fc700 1 --2- 172.21.15.82:0/4250859085 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd0d0107d90 0x7fd0d0074940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:13:17.564 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:17.564+0000 7fd0ce7fc700 1 -- 172.21.15.82:0/4250859085 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd0d0103460 msgr2=0x7fd0d00764e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:17.564 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:17.564+0000 7fd0ce7fc700 1 --2- 172.21.15.82:0/4250859085 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd0d0103460 0x7fd0d00764e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:17.565 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:17.564+0000 7fd0ce7fc700 1 -- 172.21.15.82:0/4250859085 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd0d01086e0 msgr2=0x7fd0d0074f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:17.565 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:17.564+0000 7fd0ce7fc700 1 --2- 172.21.15.82:0/4250859085 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd0d01086e0 0x7fd0d0074f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:17.565 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:17.564+0000 7fd0ce7fc700 1 -- 172.21.15.82:0/4250859085 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0d0075750 con 0x7fd0d0107d90 2024-05-20T16:13:17.565 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:17.564+0000 7fd0cf7fe700 1 --2- 172.21.15.82:0/4250859085 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd0d01086e0 0x7fd0d0074f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T16:13:17.565 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:17.564+0000 7fd0ce7fc700 1 --2- 172.21.15.82:0/4250859085 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd0d0107d90 0x7fd0d0074940 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fd0c400e9c0 tx=0x7fd0c400ed80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:13:17.565 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:17.564+0000 7fd0b7fff700 1 -- 172.21.15.82:0/4250859085 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0c400cbd0 con 0x7fd0d0107d90 2024-05-20T16:13:17.565 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:17.564+0000 7fd0d556a700 1 -- 172.21.15.82:0/4250859085 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0d01ac7c0 con 0x7fd0d0107d90 2024-05-20T16:13:17.565 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:17.564+0000 7fd0b7fff700 1 -- 172.21.15.82:0/4250859085 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd0c400cd70 con 0x7fd0d0107d90 2024-05-20T16:13:17.565 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:17.564+0000 7fd0b7fff700 1 -- 172.21.15.82:0/4250859085 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0c4018750 con 0x7fd0d0107d90 2024-05-20T16:13:17.565 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:17.564+0000 7fd0d556a700 1 -- 172.21.15.82:0/4250859085 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0d01accd0 con 0x7fd0d0107d90 2024-05-20T16:13:17.566 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:17.565+0000 7fd0d556a700 1 -- 172.21.15.82:0/4250859085 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0d0104910 con 0x7fd0d0107d90 2024-05-20T16:13:17.566 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:17.567+0000 7fd0b7fff700 1 -- 172.21.15.82:0/4250859085 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fd0c40188f0 con 0x7fd0d0107d90 2024-05-20T16:13:17.567 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:17.567+0000 7fd0b7fff700 1 --2- 172.21.15.82:0/4250859085 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fd0b80a2c20 0x7fd0b80a5130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:17.567 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:17.568+0000 7fd0ceffd700 1 --2- 172.21.15.82:0/4250859085 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fd0b80a2c20 0x7fd0b80a5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:13:17.567 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:17.568+0000 7fd0b7fff700 1 -- 172.21.15.82:0/4250859085 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9092+0+0 (secure 0 0 0) 0x7fd0c40cf970 con 0x7fd0d0107d90 2024-05-20T16:13:17.568 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:17.568+0000 7fd0ceffd700 1 --2- 172.21.15.82:0/4250859085 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fd0b80a2c20 0x7fd0b80a5130 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fd0c0009b30 tx=0x7fd0c0009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:13:17.571 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:17.571+0000 7fd0b7fff700 1 -- 172.21.15.82:0/4250859085 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fd0c409c510 con 0x7fd0d0107d90 2024-05-20T16:13:17.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:13:17 smithi082 ceph-mon[32238]: pgmap v339: 129 pgs: 129 active+clean; 641 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-05-20T16:13:17.635 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:13:17 smithi184 ceph-mon[36209]: pgmap v339: 129 pgs: 129 active+clean; 641 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-05-20T16:13:17.771 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:17.771+0000 7fd0d556a700 1 -- 172.21.15.82:0/4250859085 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7fd0d002dc60 con 0x7fd0d0107d90 2024-05-20T16:13:17.772 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:17.772+0000 7fd0b7fff700 1 -- 172.21.15.82:0/4250859085 <== mon.1 v2:172.21.15.184:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fd0c409be10 con 0x7fd0d0107d90 2024-05-20T16:13:17.772 INFO:teuthology.orchestra.run.smithi082.stdout:111669149768 2024-05-20T16:13:17.774 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:17.774+0000 7fd0d556a700 1 -- 172.21.15.82:0/4250859085 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fd0b80a2c20 msgr2=0x7fd0b80a5130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:17.775 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:17.774+0000 7fd0d556a700 1 --2- 172.21.15.82:0/4250859085 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fd0b80a2c20 0x7fd0b80a5130 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fd0c0009b30 tx=0x7fd0c0009260 comp rx=0 tx=0).stop 2024-05-20T16:13:17.775 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:17.774+0000 7fd0d556a700 1 -- 172.21.15.82:0/4250859085 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd0d0107d90 msgr2=0x7fd0d0074940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:17.775 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:17.774+0000 7fd0d556a700 1 --2- 172.21.15.82:0/4250859085 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd0d0107d90 0x7fd0d0074940 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fd0c400e9c0 tx=0x7fd0c400ed80 comp rx=0 tx=0).stop 2024-05-20T16:13:17.775 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:17.775+0000 7fd0d556a700 1 -- 172.21.15.82:0/4250859085 shutdown_connections 2024-05-20T16:13:17.775 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:17.775+0000 7fd0d556a700 1 --2- 172.21.15.82:0/4250859085 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fd0b80a2c20 0x7fd0b80a5130 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:17.775 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:17.775+0000 7fd0d556a700 1 --2- 172.21.15.82:0/4250859085 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd0d0103460 0x7fd0d00764e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:17.775 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:17.775+0000 7fd0d556a700 1 --2- 172.21.15.82:0/4250859085 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd0d01086e0 0x7fd0d0074f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:17.775 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:17.775+0000 7fd0d556a700 1 --2- 172.21.15.82:0/4250859085 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd0d0107d90 0x7fd0d0074940 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:17.775 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:17.775+0000 7fd0d556a700 1 -- 172.21.15.82:0/4250859085 >> 172.21.15.82:0/4250859085 conn(0x7fd0d00fecf0 msgr2=0x7fd0d0105c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:13:17.775 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:17.775+0000 7fd0d556a700 1 -- 172.21.15.82:0/4250859085 shutdown_connections 2024-05-20T16:13:17.775 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:17.775+0000 7fd0d556a700 1 -- 172.21.15.82:0/4250859085 wait complete. 2024-05-20T16:13:17.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:13:17 smithi160 ceph-mon[36347]: pgmap v339: 129 pgs: 129 active+clean; 641 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-05-20T16:13:18.007 INFO:tasks.cephadm.ceph_manager.ceph:need seq 347892351001 got 347892351005 for osd.11 2024-05-20T16:13:18.007 DEBUG:teuthology.parallel:result is None 2024-05-20T16:13:18.635 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:13:18 smithi184 ceph-mon[36209]: from='client.? 172.21.15.82:0/4250859085' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-05-20T16:13:18.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:13:18 smithi160 ceph-mon[36347]: from='client.? 172.21.15.82:0/4250859085' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-05-20T16:13:18.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:13:18 smithi082 ceph-mon[32238]: from='client.? 172.21.15.82:0/4250859085' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-05-20T16:13:19.026 INFO:tasks.cephadm.ceph_manager.ceph:need seq 146028888125 got 146028888130 for osd.4 2024-05-20T16:13:19.026 DEBUG:teuthology.parallel:result is None 2024-05-20T16:13:19.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:13:19 smithi082 ceph-mon[32238]: pgmap v340: 129 pgs: 129 active+clean; 641 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 426 B/s wr, 8 op/s 2024-05-20T16:13:19.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:13:19 smithi184 ceph-mon[36209]: pgmap v340: 129 pgs: 129 active+clean; 641 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 426 B/s wr, 8 op/s 2024-05-20T16:13:19.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:13:19 smithi160 ceph-mon[36347]: pgmap v340: 129 pgs: 129 active+clean; 641 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 426 B/s wr, 8 op/s 2024-05-20T16:13:20.206 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:19.492+0000 7f92c853f700 1 -- 172.21.15.82:0/1126936483 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f92c01075e0 msgr2=0x7f92c010ef40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:20.207 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:19.492+0000 7f92c853f700 1 --2- 172.21.15.82:0/1126936483 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f92c01075e0 0x7f92c010ef40 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f92b800b0d0 tx=0x7f92b800b3e0 comp rx=0 tx=0).stop 2024-05-20T16:13:20.207 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:19.492+0000 7f92c853f700 1 -- 172.21.15.82:0/1126936483 shutdown_connections 2024-05-20T16:13:20.207 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:19.492+0000 7f92c853f700 1 --2- 172.21.15.82:0/1126936483 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f92c01075e0 0x7f92c010ef40 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:20.207 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:19.492+0000 7f92c853f700 1 --2- 172.21.15.82:0/1126936483 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f92c0102360 0x7f92c0102770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:20.207 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:19.492+0000 7f92c853f700 1 --2- 172.21.15.82:0/1126936483 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f92c0106c90 0x7f92c01070a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:20.207 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:19.492+0000 7f92c853f700 1 -- 172.21.15.82:0/1126936483 >> 172.21.15.82:0/1126936483 conn(0x7f92c00fddd0 msgr2=0x7f92c01001f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:13:20.207 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:19.492+0000 7f92c853f700 1 -- 172.21.15.82:0/1126936483 shutdown_connections 2024-05-20T16:13:20.207 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:19.492+0000 7f92c853f700 1 -- 172.21.15.82:0/1126936483 wait complete. 2024-05-20T16:13:20.207 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:19.493+0000 7f92c853f700 1 Processor -- start 2024-05-20T16:13:20.207 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:19.493+0000 7f92c853f700 1 -- start start 2024-05-20T16:13:20.207 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:19.493+0000 7f92c853f700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f92c0102360 0x7f92c019aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:20.207 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:19.493+0000 7f92c853f700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f92c0106c90 0x7f92c019b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:20.207 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:19.493+0000 7f92c853f700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f92c01075e0 0x7f92c019b640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:20.208 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:19.493+0000 7f92c853f700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f92c0112360 con 0x7f92c01075e0 2024-05-20T16:13:20.208 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:19.493+0000 7f92c853f700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f92c01121e0 con 0x7f92c0102360 2024-05-20T16:13:20.208 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:19.493+0000 7f92c853f700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f92c01124e0 con 0x7f92c0106c90 2024-05-20T16:13:20.208 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:19.493+0000 7f92c6adc700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f92c01075e0 0x7f92c019b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:13:20.208 INFO:teuthology.orchestra.run.smithi082.stdout:167503724606 2024-05-20T16:13:20.208 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:19.493+0000 7f92c6adc700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f92c01075e0 0x7f92c019b640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:48548/0 (socket says 172.21.15.82:48548) 2024-05-20T16:13:20.209 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:19.493+0000 7f92c6adc700 1 -- 172.21.15.82:0/3858857719 learned_addr learned my addr 172.21.15.82:0/3858857719 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:13:20.209 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:19.493+0000 7f92c62db700 1 --2- 172.21.15.82:0/3858857719 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f92c0102360 0x7f92c019aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:13:20.209 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:19.494+0000 7f92c5ada700 1 --2- 172.21.15.82:0/3858857719 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f92c0106c90 0x7f92c019b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:13:20.209 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:19.494+0000 7f92c6adc700 1 -- 172.21.15.82:0/3858857719 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f92c0106c90 msgr2=0x7f92c019b0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:20.209 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:19.494+0000 7f92c6adc700 1 --2- 172.21.15.82:0/3858857719 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f92c0106c90 0x7f92c019b0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:20.209 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:19.494+0000 7f92c6adc700 1 -- 172.21.15.82:0/3858857719 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f92c0102360 msgr2=0x7f92c019aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:20.209 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:19.494+0000 7f92c6adc700 1 --2- 172.21.15.82:0/3858857719 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f92c0102360 0x7f92c019aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:20.209 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:19.494+0000 7f92c6adc700 1 -- 172.21.15.82:0/3858857719 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92c01a5e60 con 0x7f92c01075e0 2024-05-20T16:13:20.209 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:19.494+0000 7f92c5ada700 1 --2- 172.21.15.82:0/3858857719 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f92c0106c90 0x7f92c019b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T16:13:20.209 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:19.494+0000 7f92c6adc700 1 --2- 172.21.15.82:0/3858857719 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f92c01075e0 0x7f92c019b640 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f92b8006070 tx=0x7f92b8004d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:13:20.209 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:19.494+0000 7f92b77fe700 1 -- 172.21.15.82:0/3858857719 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92b8021070 con 0x7f92c01075e0 2024-05-20T16:13:20.209 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:19.494+0000 7f92c853f700 1 -- 172.21.15.82:0/3858857719 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92c01a60f0 con 0x7f92c01075e0 2024-05-20T16:13:20.209 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:19.494+0000 7f92b77fe700 1 -- 172.21.15.82:0/3858857719 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f92b8007b80 con 0x7f92c01075e0 2024-05-20T16:13:20.210 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:19.494+0000 7f92b77fe700 1 -- 172.21.15.82:0/3858857719 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92b8005330 con 0x7f92c01075e0 2024-05-20T16:13:20.210 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:19.494+0000 7f92c853f700 1 -- 172.21.15.82:0/3858857719 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92c01a65a0 con 0x7f92c01075e0 2024-05-20T16:13:20.210 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:19.495+0000 7f92c853f700 1 -- 172.21.15.82:0/3858857719 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92c0050170 con 0x7f92c01075e0 2024-05-20T16:13:20.210 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:19.497+0000 7f92b77fe700 1 -- 172.21.15.82:0/3858857719 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f92b8024070 con 0x7f92c01075e0 2024-05-20T16:13:20.210 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:19.497+0000 7f92b77fe700 1 --2- 172.21.15.82:0/3858857719 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f92ac0a2c20 0x7f92ac0a5130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:20.210 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:19.497+0000 7f92c62db700 1 --2- 172.21.15.82:0/3858857719 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f92ac0a2c20 0x7f92ac0a5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:13:20.210 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:19.497+0000 7f92b77fe700 1 -- 172.21.15.82:0/3858857719 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9092+0+0 (secure 0 0 0) 0x7f92b800f020 con 0x7f92c01075e0 2024-05-20T16:13:20.210 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:19.498+0000 7f92c62db700 1 --2- 172.21.15.82:0/3858857719 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f92ac0a2c20 0x7f92ac0a5130 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f92bc005950 tx=0x7f92bc00b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:13:20.210 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:19.501+0000 7f92b77fe700 1 -- 172.21.15.82:0/3858857719 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f92b802b020 con 0x7f92c01075e0 2024-05-20T16:13:20.210 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:19.703+0000 7f92c853f700 1 -- 172.21.15.82:0/3858857719 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f92c002de20 con 0x7f92c01075e0 2024-05-20T16:13:20.210 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:19.704+0000 7f92b77fe700 1 -- 172.21.15.82:0/3858857719 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f92b809dab0 con 0x7f92c01075e0 2024-05-20T16:13:20.210 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:19.706+0000 7f92c853f700 1 -- 172.21.15.82:0/3858857719 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f92ac0a2c20 msgr2=0x7f92ac0a5130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:20.211 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:19.706+0000 7f92c853f700 1 --2- 172.21.15.82:0/3858857719 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f92ac0a2c20 0x7f92ac0a5130 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f92bc005950 tx=0x7f92bc00b410 comp rx=0 tx=0).stop 2024-05-20T16:13:20.211 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:19.706+0000 7f92c853f700 1 -- 172.21.15.82:0/3858857719 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f92c01075e0 msgr2=0x7f92c019b640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:20.211 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:19.706+0000 7f92c853f700 1 --2- 172.21.15.82:0/3858857719 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f92c01075e0 0x7f92c019b640 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f92b8006070 tx=0x7f92b8004d00 comp rx=0 tx=0).stop 2024-05-20T16:13:20.211 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:19.706+0000 7f92c853f700 1 -- 172.21.15.82:0/3858857719 shutdown_connections 2024-05-20T16:13:20.211 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:19.706+0000 7f92c853f700 1 --2- 172.21.15.82:0/3858857719 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f92ac0a2c20 0x7f92ac0a5130 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:20.211 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:19.706+0000 7f92c853f700 1 --2- 172.21.15.82:0/3858857719 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f92c0102360 0x7f92c019aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:20.211 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:19.706+0000 7f92c853f700 1 --2- 172.21.15.82:0/3858857719 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f92c01075e0 0x7f92c019b640 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:20.211 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:19.706+0000 7f92c853f700 1 --2- 172.21.15.82:0/3858857719 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f92c0106c90 0x7f92c019b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:20.211 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:19.706+0000 7f92c853f700 1 -- 172.21.15.82:0/3858857719 >> 172.21.15.82:0/3858857719 conn(0x7f92c00fddd0 msgr2=0x7f92c01063d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:13:20.211 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:19.706+0000 7f92c853f700 1 -- 172.21.15.82:0/3858857719 shutdown_connections 2024-05-20T16:13:20.211 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:19.706+0000 7f92c853f700 1 -- 172.21.15.82:0/3858857719 wait complete. 2024-05-20T16:13:20.635 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:13:20 smithi184 ceph-mon[36209]: from='client.? 172.21.15.82:0/3858857719' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-05-20T16:13:20.721 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:13:20 smithi082 ceph-mon[32238]: from='client.? 172.21.15.82:0/3858857719' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-05-20T16:13:20.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:13:20 smithi160 ceph-mon[36347]: from='client.? 172.21.15.82:0/3858857719' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-05-20T16:13:20.834 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:20.833+0000 7fecaea4c700 1 -- 172.21.15.82:0/2985480065 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feca81086e0 msgr2=0x7feca8110040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:20.834 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:20.833+0000 7fecaea4c700 1 --2- 172.21.15.82:0/2985480065 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feca81086e0 0x7feca8110040 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7feca000b0d0 tx=0x7feca000b3e0 comp rx=0 tx=0).stop 2024-05-20T16:13:20.834 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:20.833+0000 7fecaea4c700 1 -- 172.21.15.82:0/2985480065 shutdown_connections 2024-05-20T16:13:20.834 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:20.833+0000 7fecaea4c700 1 --2- 172.21.15.82:0/2985480065 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7feca8103460 0x7feca8103870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:20.834 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:20.833+0000 7fecaea4c700 1 --2- 172.21.15.82:0/2985480065 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feca81086e0 0x7feca8110040 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:20.834 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:20.833+0000 7fecaea4c700 1 --2- 172.21.15.82:0/2985480065 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feca8107d90 0x7feca81081a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:20.834 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:20.833+0000 7fecaea4c700 1 -- 172.21.15.82:0/2985480065 >> 172.21.15.82:0/2985480065 conn(0x7feca80fed50 msgr2=0x7feca81011a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:13:20.834 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:20.833+0000 7fecaea4c700 1 -- 172.21.15.82:0/2985480065 shutdown_connections 2024-05-20T16:13:20.834 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:20.833+0000 7fecaea4c700 1 -- 172.21.15.82:0/2985480065 wait complete. 2024-05-20T16:13:20.836 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:20.834+0000 7fecaea4c700 1 Processor -- start 2024-05-20T16:13:20.836 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:20.834+0000 7fecaea4c700 1 -- start start 2024-05-20T16:13:20.836 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:20.835+0000 7fecaea4c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7feca8103460 0x7feca8199d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:20.836 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:20.835+0000 7fecaea4c700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feca8107d90 0x7feca819a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:20.836 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:20.835+0000 7fecaea4c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feca81086e0 0x7feca819e560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:20.836 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:20.835+0000 7fecaea4c700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7feca8113480 con 0x7feca81086e0 2024-05-20T16:13:20.836 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:20.835+0000 7fecaea4c700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7feca8113300 con 0x7feca8103460 2024-05-20T16:13:20.836 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:20.835+0000 7fecaea4c700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7feca8113600 con 0x7feca8107d90 2024-05-20T16:13:20.836 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:20.835+0000 7fecacfe9700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feca81086e0 0x7feca819e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:13:20.837 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:20.835+0000 7fecacfe9700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feca81086e0 0x7feca819e560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:35868/0 (socket says 172.21.15.82:35868) 2024-05-20T16:13:20.837 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:20.835+0000 7fecacfe9700 1 -- 172.21.15.82:0/4005315824 learned_addr learned my addr 172.21.15.82:0/4005315824 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:13:20.837 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:20.835+0000 7fecacfe9700 1 -- 172.21.15.82:0/4005315824 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feca8107d90 msgr2=0x7feca819a2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:20.837 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:20.835+0000 7fecacfe9700 1 --2- 172.21.15.82:0/4005315824 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feca8107d90 0x7feca819a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:20.837 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:20.835+0000 7feca7fff700 1 --2- 172.21.15.82:0/4005315824 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7feca8103460 0x7feca8199d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:13:20.837 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:20.835+0000 7fecacfe9700 1 -- 172.21.15.82:0/4005315824 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7feca8103460 msgr2=0x7feca8199d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:20.837 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:20.835+0000 7fecacfe9700 1 --2- 172.21.15.82:0/4005315824 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7feca8103460 0x7feca8199d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:20.837 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:20.835+0000 7fecacfe9700 1 -- 172.21.15.82:0/4005315824 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feca819ec00 con 0x7feca81086e0 2024-05-20T16:13:20.837 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:20.835+0000 7feca7fff700 1 --2- 172.21.15.82:0/4005315824 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7feca8103460 0x7feca8199d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T16:13:20.837 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:20.835+0000 7fecacfe9700 1 --2- 172.21.15.82:0/4005315824 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feca81086e0 0x7feca819e560 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7feca0006070 tx=0x7feca0004db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:13:20.837 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:20.835+0000 7feca5ffb700 1 -- 172.21.15.82:0/4005315824 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feca0021070 con 0x7feca81086e0 2024-05-20T16:13:20.837 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:20.835+0000 7feca5ffb700 1 -- 172.21.15.82:0/4005315824 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feca0007c80 con 0x7feca81086e0 2024-05-20T16:13:20.837 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:20.835+0000 7fecaea4c700 1 -- 172.21.15.82:0/4005315824 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feca819ee90 con 0x7feca81086e0 2024-05-20T16:13:20.838 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:20.835+0000 7feca5ffb700 1 -- 172.21.15.82:0/4005315824 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feca00053c0 con 0x7feca81086e0 2024-05-20T16:13:20.838 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:20.836+0000 7fecaea4c700 1 -- 172.21.15.82:0/4005315824 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feca81a6810 con 0x7feca81086e0 2024-05-20T16:13:20.838 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:20.837+0000 7fecaea4c700 1 -- 172.21.15.82:0/4005315824 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feca8104910 con 0x7feca81086e0 2024-05-20T16:13:20.838 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:20.838+0000 7feca5ffb700 1 -- 172.21.15.82:0/4005315824 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7feca0024070 con 0x7feca81086e0 2024-05-20T16:13:20.839 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:20.838+0000 7feca5ffb700 1 --2- 172.21.15.82:0/4005315824 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fec940a2ca0 0x7fec940a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:20.839 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:20.839+0000 7feca7fff700 1 --2- 172.21.15.82:0/4005315824 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fec940a2ca0 0x7fec940a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:13:20.839 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:20.839+0000 7feca5ffb700 1 -- 172.21.15.82:0/4005315824 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9092+0+0 (secure 0 0 0) 0x7feca000f020 con 0x7feca81086e0 2024-05-20T16:13:20.839 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:20.839+0000 7feca7fff700 1 --2- 172.21.15.82:0/4005315824 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fec940a2ca0 0x7fec940a51b0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fec98005d90 tx=0x7fec98005d00 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:13:20.842 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:20.842+0000 7feca5ffb700 1 -- 172.21.15.82:0/4005315824 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7feca002b020 con 0x7feca81086e0 2024-05-20T16:13:21.052 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:21.052+0000 7fecaea4c700 1 -- 172.21.15.82:0/4005315824 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7feca819b090 con 0x7feca81086e0 2024-05-20T16:13:21.052 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:21.052+0000 7feca5ffb700 1 -- 172.21.15.82:0/4005315824 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7feca009da30 con 0x7feca81086e0 2024-05-20T16:13:21.052 INFO:teuthology.orchestra.run.smithi082.stdout:77309411405 2024-05-20T16:13:21.054 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:21.054+0000 7fecaea4c700 1 -- 172.21.15.82:0/4005315824 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fec940a2ca0 msgr2=0x7fec940a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:21.054 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:21.054+0000 7fecaea4c700 1 --2- 172.21.15.82:0/4005315824 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fec940a2ca0 0x7fec940a51b0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fec98005d90 tx=0x7fec98005d00 comp rx=0 tx=0).stop 2024-05-20T16:13:21.055 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:21.054+0000 7fecaea4c700 1 -- 172.21.15.82:0/4005315824 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feca81086e0 msgr2=0x7feca819e560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:21.055 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:21.054+0000 7fecaea4c700 1 --2- 172.21.15.82:0/4005315824 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feca81086e0 0x7feca819e560 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7feca0006070 tx=0x7feca0004db0 comp rx=0 tx=0).stop 2024-05-20T16:13:21.055 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:21.054+0000 7fecaea4c700 1 -- 172.21.15.82:0/4005315824 shutdown_connections 2024-05-20T16:13:21.055 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:21.054+0000 7fecaea4c700 1 --2- 172.21.15.82:0/4005315824 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fec940a2ca0 0x7fec940a51b0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:21.055 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:21.054+0000 7fecaea4c700 1 --2- 172.21.15.82:0/4005315824 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7feca8103460 0x7feca8199d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:21.055 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:21.054+0000 7fecaea4c700 1 --2- 172.21.15.82:0/4005315824 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feca81086e0 0x7feca819e560 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:21.055 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:21.054+0000 7fecaea4c700 1 --2- 172.21.15.82:0/4005315824 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feca8107d90 0x7feca819a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:21.055 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:21.054+0000 7fecaea4c700 1 -- 172.21.15.82:0/4005315824 >> 172.21.15.82:0/4005315824 conn(0x7feca80fed50 msgr2=0x7feca8105d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:13:21.055 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:21.054+0000 7fecaea4c700 1 -- 172.21.15.82:0/4005315824 shutdown_connections 2024-05-20T16:13:21.055 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:21.055+0000 7fecaea4c700 1 -- 172.21.15.82:0/4005315824 wait complete. 2024-05-20T16:13:21.428 INFO:tasks.cephadm.ceph_manager.ceph:need seq 111669149763 got 111669149768 for osd.3 2024-05-20T16:13:21.428 DEBUG:teuthology.parallel:result is None 2024-05-20T16:13:21.728 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:13:21 smithi082 ceph-mon[32238]: from='client.? 172.21.15.82:0/4005315824' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-05-20T16:13:21.728 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:13:21 smithi082 ceph-mon[32238]: pgmap v341: 129 pgs: 129 active+clean; 641 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2024-05-20T16:13:21.754 INFO:tasks.cephadm.ceph_manager.ceph:need seq 167503724601 got 167503724606 for osd.5 2024-05-20T16:13:21.754 DEBUG:teuthology.parallel:result is None 2024-05-20T16:13:21.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:13:21 smithi160 ceph-mon[36347]: from='client.? 172.21.15.82:0/4005315824' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-05-20T16:13:21.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:13:21 smithi160 ceph-mon[36347]: pgmap v341: 129 pgs: 129 active+clean; 641 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2024-05-20T16:13:21.885 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:13:21 smithi184 ceph-mon[36209]: from='client.? 172.21.15.82:0/4005315824' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-05-20T16:13:21.886 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:13:21 smithi184 ceph-mon[36209]: pgmap v341: 129 pgs: 129 active+clean; 641 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2024-05-20T16:13:22.494 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411399 got 77309411405 for osd.2 2024-05-20T16:13:22.494 DEBUG:teuthology.parallel:result is None 2024-05-20T16:13:22.494 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-05-20T16:13:22.494 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell --fsid 59d1600a-16c2-11ef-bc9a-c7b262605968 -- ceph pg dump --format=json 2024-05-20T16:13:22.752 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T16:13:24.035 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:13:23 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:13:24.035 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:13:23 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:13:24.035 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:13:23 smithi082 ceph-mon[32238]: pgmap v342: 129 pgs: 129 active+clean; 641 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 7 op/s 2024-05-20T16:13:24.035 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:13:23 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:13:24.035 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:13:23 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T16:13:24.035 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:13:23 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:13:24.062 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:13:23 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:13:24.062 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:13:23 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:13:24.062 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:13:23 smithi160 ceph-mon[36347]: pgmap v342: 129 pgs: 129 active+clean; 641 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 7 op/s 2024-05-20T16:13:24.062 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:13:23 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:13:24.062 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:13:23 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T16:13:24.062 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:13:23 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:13:24.135 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:13:23 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:13:24.135 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:13:23 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:13:24.136 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:13:23 smithi184 ceph-mon[36209]: pgmap v342: 129 pgs: 129 active+clean; 641 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 7 op/s 2024-05-20T16:13:24.136 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:13:23 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:13:24.136 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:13:23 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T16:13:24.136 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:13:23 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:13:24.739 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:24.682+0000 7fe967e6f700 1 -- 172.21.15.82:0/2369294597 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe9601023d0 msgr2=0x7fe9601027e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:24.739 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:24.682+0000 7fe967e6f700 1 --2- 172.21.15.82:0/2369294597 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe9601023d0 0x7fe9601027e0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fe95c0099c0 tx=0x7fe95c009cd0 comp rx=0 tx=0).stop 2024-05-20T16:13:24.739 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:24.682+0000 7fe967e6f700 1 -- 172.21.15.82:0/2369294597 shutdown_connections 2024-05-20T16:13:24.739 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:24.682+0000 7fe967e6f700 1 --2- 172.21.15.82:0/2369294597 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe960107600 0x7fe96010ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:24.739 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:24.682+0000 7fe967e6f700 1 --2- 172.21.15.82:0/2369294597 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe9601023d0 0x7fe9601027e0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:24.739 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:24.682+0000 7fe967e6f700 1 --2- 172.21.15.82:0/2369294597 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe960106cb0 0x7fe9601070c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:24.739 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:24.682+0000 7fe967e6f700 1 -- 172.21.15.82:0/2369294597 >> 172.21.15.82:0/2369294597 conn(0x7fe960077c80 msgr2=0x7fe9600780c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:13:24.739 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:24.682+0000 7fe967e6f700 1 -- 172.21.15.82:0/2369294597 shutdown_connections 2024-05-20T16:13:24.739 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:24.682+0000 7fe967e6f700 1 -- 172.21.15.82:0/2369294597 wait complete. 2024-05-20T16:13:24.739 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:24.683+0000 7fe967e6f700 1 Processor -- start 2024-05-20T16:13:24.739 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:24.683+0000 7fe967e6f700 1 -- start start 2024-05-20T16:13:24.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:24.683+0000 7fe967e6f700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe9601023d0 0x7fe960198cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:24.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:24.683+0000 7fe967e6f700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe960106cb0 0x7fe960199230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:24.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:24.683+0000 7fe967e6f700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe960107600 0x7fe96019c450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:24.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:24.683+0000 7fe967e6f700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fe960112380 con 0x7fe960106cb0 2024-05-20T16:13:24.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:24.683+0000 7fe967e6f700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fe960112200 con 0x7fe960107600 2024-05-20T16:13:24.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:24.683+0000 7fe967e6f700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fe960112500 con 0x7fe9601023d0 2024-05-20T16:13:24.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:24.683+0000 7fe96540a700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe960106cb0 0x7fe960199230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:13:24.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:24.683+0000 7fe96540a700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe960106cb0 0x7fe960199230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:35888/0 (socket says 172.21.15.82:35888) 2024-05-20T16:13:24.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:24.683+0000 7fe96540a700 1 -- 172.21.15.82:0/111452586 learned_addr learned my addr 172.21.15.82:0/111452586 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:13:24.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:24.683+0000 7fe965c0b700 1 --2- 172.21.15.82:0/111452586 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe9601023d0 0x7fe960198cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:13:24.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:24.683+0000 7fe96640c700 1 --2- 172.21.15.82:0/111452586 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe960107600 0x7fe96019c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:13:24.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:24.684+0000 7fe96540a700 1 -- 172.21.15.82:0/111452586 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe9601023d0 msgr2=0x7fe960198cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:24.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:24.684+0000 7fe96540a700 1 --2- 172.21.15.82:0/111452586 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe9601023d0 0x7fe960198cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:24.741 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:24.684+0000 7fe96540a700 1 -- 172.21.15.82:0/111452586 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe960107600 msgr2=0x7fe96019c450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:24.741 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:24.684+0000 7fe96540a700 1 --2- 172.21.15.82:0/111452586 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe960107600 0x7fe96019c450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:24.741 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:24.684+0000 7fe96540a700 1 -- 172.21.15.82:0/111452586 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe96019ca40 con 0x7fe960106cb0 2024-05-20T16:13:24.741 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:24.684+0000 7fe96540a700 1 --2- 172.21.15.82:0/111452586 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe960106cb0 0x7fe960199230 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fe95000c9e0 tx=0x7fe95000ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:13:24.741 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:24.684+0000 7fe956ffd700 1 -- 172.21.15.82:0/111452586 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe950007aa0 con 0x7fe960106cb0 2024-05-20T16:13:24.741 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:24.684+0000 7fe956ffd700 1 -- 172.21.15.82:0/111452586 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe95000ce20 con 0x7fe960106cb0 2024-05-20T16:13:24.741 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:24.684+0000 7fe967e6f700 1 -- 172.21.15.82:0/111452586 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe96019cd30 con 0x7fe960106cb0 2024-05-20T16:13:24.741 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:24.684+0000 7fe956ffd700 1 -- 172.21.15.82:0/111452586 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe950018660 con 0x7fe960106cb0 2024-05-20T16:13:24.741 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:24.684+0000 7fe967e6f700 1 -- 172.21.15.82:0/111452586 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9601a57e0 con 0x7fe960106cb0 2024-05-20T16:13:24.741 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:24.685+0000 7fe967e6f700 1 -- 172.21.15.82:0/111452586 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe960050170 con 0x7fe960106cb0 2024-05-20T16:13:24.741 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:24.686+0000 7fe956ffd700 1 -- 172.21.15.82:0/111452586 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fe95000f450 con 0x7fe960106cb0 2024-05-20T16:13:24.741 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:24.687+0000 7fe956ffd700 1 --2- 172.21.15.82:0/111452586 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fe94c0a2ca0 0x7fe94c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:24.741 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:24.687+0000 7fe956ffd700 1 -- 172.21.15.82:0/111452586 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9092+0+0 (secure 0 0 0) 0x7fe9500cf4b0 con 0x7fe960106cb0 2024-05-20T16:13:24.741 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:24.687+0000 7fe965c0b700 1 --2- 172.21.15.82:0/111452586 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fe94c0a2ca0 0x7fe94c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:13:24.742 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:24.687+0000 7fe965c0b700 1 --2- 172.21.15.82:0/111452586 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fe94c0a2ca0 0x7fe94c0a51b0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fe95c000c00 tx=0x7fe95c01a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:13:24.742 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:24.691+0000 7fe956ffd700 1 -- 172.21.15.82:0/111452586 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fe950014030 con 0x7fe960106cb0 2024-05-20T16:13:24.883 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:24.883+0000 7fe967e6f700 1 -- 172.21.15.82:0/111452586 --> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe9601013c0 con 0x7fe94c0a2ca0 2024-05-20T16:13:24.892 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:24.893+0000 7fe956ffd700 1 -- 172.21.15.82:0/111452586 <== mgr.14150 v2:172.21.15.82:6800/379530823 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340842 (secure 0 0 0) 0x7fe9601013c0 con 0x7fe94c0a2ca0 2024-05-20T16:13:24.897 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-05-20T16:13:24.900 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:24.896+0000 7fe967e6f700 1 -- 172.21.15.82:0/111452586 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fe94c0a2ca0 msgr2=0x7fe94c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:24.900 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:24.896+0000 7fe967e6f700 1 --2- 172.21.15.82:0/111452586 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fe94c0a2ca0 0x7fe94c0a51b0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fe95c000c00 tx=0x7fe95c01a040 comp rx=0 tx=0).stop 2024-05-20T16:13:24.900 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:24.896+0000 7fe967e6f700 1 -- 172.21.15.82:0/111452586 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe960106cb0 msgr2=0x7fe960199230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:24.900 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:24.896+0000 7fe967e6f700 1 --2- 172.21.15.82:0/111452586 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe960106cb0 0x7fe960199230 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fe95000c9e0 tx=0x7fe95000ccf0 comp rx=0 tx=0).stop 2024-05-20T16:13:24.900 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:24.896+0000 7fe967e6f700 1 -- 172.21.15.82:0/111452586 shutdown_connections 2024-05-20T16:13:24.900 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:24.896+0000 7fe967e6f700 1 --2- 172.21.15.82:0/111452586 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fe94c0a2ca0 0x7fe94c0a51b0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:24.900 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:24.896+0000 7fe967e6f700 1 --2- 172.21.15.82:0/111452586 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe960107600 0x7fe96019c450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:24.900 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:24.896+0000 7fe967e6f700 1 --2- 172.21.15.82:0/111452586 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe9601023d0 0x7fe960198cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:24.901 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:24.896+0000 7fe967e6f700 1 --2- 172.21.15.82:0/111452586 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe960106cb0 0x7fe960199230 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:24.901 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:24.896+0000 7fe967e6f700 1 -- 172.21.15.82:0/111452586 >> 172.21.15.82:0/111452586 conn(0x7fe960077c80 msgr2=0x7fe960100cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:13:24.901 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:24.896+0000 7fe967e6f700 1 -- 172.21.15.82:0/111452586 shutdown_connections 2024-05-20T16:13:24.901 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:24.896+0000 7fe967e6f700 1 -- 172.21.15.82:0/111452586 wait complete. 2024-05-20T16:13:24.901 INFO:teuthology.orchestra.run.smithi082.stderr:dumped all 2024-05-20T16:13:25.544 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:13:25 smithi082 ceph-mon[32238]: from='client.15117 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-20T16:13:25.544 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:13:25 smithi082 ceph-mon[32238]: pgmap v343: 129 pgs: 129 active+clean; 641 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2024-05-20T16:13:25.572 INFO:teuthology.orchestra.run.smithi082.stdout:{"pg_ready":true,"pg_map":{"version":342,"stamp":"2024-05-20T16:13:23.279025+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":656768,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":561,"num_read_kb":398,"num_write":350,"num_write_kb":3059,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":1180853,"num_keys_recovered":17,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":784,"ondisk_log_size":784,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":349052,"kb_used_data":12380,"kb_used_omap":18,"kb_used_meta":336557,"kb_avail":1124592772,"statfs":{"total":1151940427776,"available":1151582998528,"internally_reserved":0,"allocated":12677120,"data_stored":8105510,"data_compressed":394740,"data_compressed_allocated":540672,"data_compressed_original":2580480,"omap_allocated":19141,"internal_metadata":344634683},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":16,"apply_latency_ms":16,"commit_latency_ns":16000000,"apply_latency_ns":16000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":390,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":82,"num_read_kb":42,"num_write":7,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004391"},"pg_stats":[{"pgid":"3.3f","version":"93'8","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:13:15.297439+0000","last_change":"2024-05-20T16:11:47.187104+0000","last_active":"2024-05-20T16:13:15.297439+0000","last_peered":"2024-05-20T16:13:15.297439+0000","last_clean":"2024-05-20T16:13:15.297439+0000","last_became_active":"2024-05-20T16:11:47.186791+0000","last_became_peered":"2024-05-20T16:11:47.186791+0000","last_unstale":"2024-05-20T16:13:15.297439+0000","last_undegraded":"2024-05-20T16:13:15.297439+0000","last_fullsized":"2024-05-20T16:13:15.297439+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:33:38.914389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"93'4","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:13:15.278999+0000","last_change":"2024-05-20T16:11:47.187846+0000","last_active":"2024-05-20T16:13:15.278999+0000","last_peered":"2024-05-20T16:13:15.278999+0000","last_clean":"2024-05-20T16:13:15.278999+0000","last_became_active":"2024-05-20T16:11:47.187540+0000","last_became_peered":"2024-05-20T16:11:47.187540+0000","last_unstale":"2024-05-20T16:13:15.278999+0000","last_undegraded":"2024-05-20T16:13:15.278999+0000","last_fullsized":"2024-05-20T16:13:15.278999+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:19:59.784437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"93'7","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:13:15.295054+0000","last_change":"2024-05-20T16:11:47.185215+0000","last_active":"2024-05-20T16:13:15.295054+0000","last_peered":"2024-05-20T16:13:15.295054+0000","last_clean":"2024-05-20T16:13:15.295054+0000","last_became_active":"2024-05-20T16:11:47.184880+0000","last_became_peered":"2024-05-20T16:11:47.184880+0000","last_unstale":"2024-05-20T16:13:15.295054+0000","last_undegraded":"2024-05-20T16:13:15.295054+0000","last_fullsized":"2024-05-20T16:13:15.295054+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T18:43:32.125643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"93'8","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:13:15.284035+0000","last_change":"2024-05-20T16:11:47.175849+0000","last_active":"2024-05-20T16:13:15.284035+0000","last_peered":"2024-05-20T16:13:15.284035+0000","last_clean":"2024-05-20T16:13:15.284035+0000","last_became_active":"2024-05-20T16:11:47.175342+0000","last_became_peered":"2024-05-20T16:11:47.175342+0000","last_unstale":"2024-05-20T16:13:15.284035+0000","last_undegraded":"2024-05-20T16:13:15.284035+0000","last_fullsized":"2024-05-20T16:13:15.284035+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:02:43.798388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"93'4","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:13:15.267690+0000","last_change":"2024-05-20T16:11:47.174798+0000","last_active":"2024-05-20T16:13:15.267690+0000","last_peered":"2024-05-20T16:13:15.267690+0000","last_clean":"2024-05-20T16:13:15.267690+0000","last_became_active":"2024-05-20T16:11:47.174474+0000","last_became_peered":"2024-05-20T16:11:47.174474+0000","last_unstale":"2024-05-20T16:13:15.267690+0000","last_undegraded":"2024-05-20T16:13:15.267690+0000","last_fullsized":"2024-05-20T16:13:15.267690+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:41:24.404901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"93'6","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:13:15.278393+0000","last_change":"2024-05-20T16:11:47.185731+0000","last_active":"2024-05-20T16:13:15.278393+0000","last_peered":"2024-05-20T16:13:15.278393+0000","last_clean":"2024-05-20T16:13:15.278393+0000","last_became_active":"2024-05-20T16:11:47.185380+0000","last_became_peered":"2024-05-20T16:11:47.185380+0000","last_unstale":"2024-05-20T16:13:15.278393+0000","last_undegraded":"2024-05-20T16:13:15.278393+0000","last_fullsized":"2024-05-20T16:13:15.278393+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T16:33:28.854205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"93'5","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:13:15.285182+0000","last_change":"2024-05-20T16:11:47.183594+0000","last_active":"2024-05-20T16:13:15.285182+0000","last_peered":"2024-05-20T16:13:15.285182+0000","last_clean":"2024-05-20T16:13:15.285182+0000","last_became_active":"2024-05-20T16:11:47.183260+0000","last_became_peered":"2024-05-20T16:11:47.183260+0000","last_unstale":"2024-05-20T16:13:15.285182+0000","last_undegraded":"2024-05-20T16:13:15.285182+0000","last_fullsized":"2024-05-20T16:13:15.285182+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T17:12:22.781236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"93'9","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:13:15.274042+0000","last_change":"2024-05-20T16:11:47.191784+0000","last_active":"2024-05-20T16:13:15.274042+0000","last_peered":"2024-05-20T16:13:15.274042+0000","last_clean":"2024-05-20T16:13:15.274042+0000","last_became_active":"2024-05-20T16:11:47.190553+0000","last_became_peered":"2024-05-20T16:11:47.190553+0000","last_unstale":"2024-05-20T16:13:15.274042+0000","last_undegraded":"2024-05-20T16:13:15.274042+0000","last_fullsized":"2024-05-20T16:13:15.274042+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T20:17:29.346688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"93'11","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:13:15.297277+0000","last_change":"2024-05-20T16:11:47.174817+0000","last_active":"2024-05-20T16:13:15.297277+0000","last_peered":"2024-05-20T16:13:15.297277+0000","last_clean":"2024-05-20T16:13:15.297277+0000","last_became_active":"2024-05-20T16:11:47.174483+0000","last_became_peered":"2024-05-20T16:11:47.174483+0000","last_unstale":"2024-05-20T16:13:15.297277+0000","last_undegraded":"2024-05-20T16:13:15.297277+0000","last_fullsized":"2024-05-20T16:13:15.297277+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T20:32:30.284912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"93'16","reported_seq":43,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:13:15.279963+0000","last_change":"2024-05-20T16:11:47.186899+0000","last_active":"2024-05-20T16:13:15.279963+0000","last_peered":"2024-05-20T16:13:15.279963+0000","last_clean":"2024-05-20T16:13:15.279963+0000","last_became_active":"2024-05-20T16:11:47.186564+0000","last_became_peered":"2024-05-20T16:11:47.186564+0000","last_unstale":"2024-05-20T16:13:15.279963+0000","last_undegraded":"2024-05-20T16:13:15.279963+0000","last_fullsized":"2024-05-20T16:13:15.279963+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:19:57.110107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"93'6","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:13:15.295373+0000","last_change":"2024-05-20T16:11:47.182696+0000","last_active":"2024-05-20T16:13:15.295373+0000","last_peered":"2024-05-20T16:13:15.295373+0000","last_clean":"2024-05-20T16:13:15.295373+0000","last_became_active":"2024-05-20T16:11:47.182169+0000","last_became_peered":"2024-05-20T16:11:47.182169+0000","last_unstale":"2024-05-20T16:13:15.295373+0000","last_undegraded":"2024-05-20T16:13:15.295373+0000","last_fullsized":"2024-05-20T16:13:15.295373+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:55:16.413493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"93'7","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:13:15.274598+0000","last_change":"2024-05-20T16:11:47.181716+0000","last_active":"2024-05-20T16:13:15.274598+0000","last_peered":"2024-05-20T16:13:15.274598+0000","last_clean":"2024-05-20T16:13:15.274598+0000","last_became_active":"2024-05-20T16:11:47.181403+0000","last_became_peered":"2024-05-20T16:11:47.181403+0000","last_unstale":"2024-05-20T16:13:15.274598+0000","last_undegraded":"2024-05-20T16:13:15.274598+0000","last_fullsized":"2024-05-20T16:13:15.274598+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:38:04.938882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"93'10","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:13:15.279627+0000","last_change":"2024-05-20T16:11:47.183320+0000","last_active":"2024-05-20T16:13:15.279627+0000","last_peered":"2024-05-20T16:13:15.279627+0000","last_clean":"2024-05-20T16:13:15.279627+0000","last_became_active":"2024-05-20T16:11:47.183028+0000","last_became_peered":"2024-05-20T16:11:47.183028+0000","last_unstale":"2024-05-20T16:13:15.279627+0000","last_undegraded":"2024-05-20T16:13:15.279627+0000","last_fullsized":"2024-05-20T16:13:15.279627+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T17:28:33.614173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"93'7","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:13:15.279572+0000","last_change":"2024-05-20T16:11:47.183718+0000","last_active":"2024-05-20T16:13:15.279572+0000","last_peered":"2024-05-20T16:13:15.279572+0000","last_clean":"2024-05-20T16:13:15.279572+0000","last_became_active":"2024-05-20T16:11:47.183378+0000","last_became_peered":"2024-05-20T16:11:47.183378+0000","last_unstale":"2024-05-20T16:13:15.279572+0000","last_undegraded":"2024-05-20T16:13:15.279572+0000","last_fullsized":"2024-05-20T16:13:15.279572+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:08:32.066924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"93'7","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:13:15.277203+0000","last_change":"2024-05-20T16:11:47.174163+0000","last_active":"2024-05-20T16:13:15.277203+0000","last_peered":"2024-05-20T16:13:15.277203+0000","last_clean":"2024-05-20T16:13:15.277203+0000","last_became_active":"2024-05-20T16:11:47.173798+0000","last_became_peered":"2024-05-20T16:11:47.173798+0000","last_unstale":"2024-05-20T16:13:15.277203+0000","last_undegraded":"2024-05-20T16:13:15.277203+0000","last_fullsized":"2024-05-20T16:13:15.277203+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T20:51:20.646273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"93'9","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:13:15.295629+0000","last_change":"2024-05-20T16:11:47.181809+0000","last_active":"2024-05-20T16:13:15.295629+0000","last_peered":"2024-05-20T16:13:15.295629+0000","last_clean":"2024-05-20T16:13:15.295629+0000","last_became_active":"2024-05-20T16:11:47.181473+0000","last_became_peered":"2024-05-20T16:11:47.181473+0000","last_unstale":"2024-05-20T16:13:15.295629+0000","last_undegraded":"2024-05-20T16:13:15.295629+0000","last_fullsized":"2024-05-20T16:13:15.295629+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:20:45.302995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"93'8","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:13:15.297631+0000","last_change":"2024-05-20T16:11:47.187947+0000","last_active":"2024-05-20T16:13:15.297631+0000","last_peered":"2024-05-20T16:13:15.297631+0000","last_clean":"2024-05-20T16:13:15.297631+0000","last_became_active":"2024-05-20T16:11:47.187623+0000","last_became_peered":"2024-05-20T16:11:47.187623+0000","last_unstale":"2024-05-20T16:13:15.297631+0000","last_undegraded":"2024-05-20T16:13:15.297631+0000","last_fullsized":"2024-05-20T16:13:15.297631+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:52:08.674888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"93'10","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:13:15.292659+0000","last_change":"2024-05-20T16:11:47.188599+0000","last_active":"2024-05-20T16:13:15.292659+0000","last_peered":"2024-05-20T16:13:15.292659+0000","last_clean":"2024-05-20T16:13:15.292659+0000","last_became_active":"2024-05-20T16:11:47.188278+0000","last_became_peered":"2024-05-20T16:11:47.188278+0000","last_unstale":"2024-05-20T16:13:15.292659+0000","last_undegraded":"2024-05-20T16:13:15.292659+0000","last_fullsized":"2024-05-20T16:13:15.292659+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T17:45:42.003546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"93'7","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:13:15.266395+0000","last_change":"2024-05-20T16:11:47.176250+0000","last_active":"2024-05-20T16:13:15.266395+0000","last_peered":"2024-05-20T16:13:15.266395+0000","last_clean":"2024-05-20T16:13:15.266395+0000","last_became_active":"2024-05-20T16:11:47.175903+0000","last_became_peered":"2024-05-20T16:11:47.175903+0000","last_unstale":"2024-05-20T16:13:15.266395+0000","last_undegraded":"2024-05-20T16:13:15.266395+0000","last_fullsized":"2024-05-20T16:13:15.266395+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T18:11:26.045745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:12:46.264563+0000","last_change":"2024-05-20T16:11:51.438824+0000","last_active":"2024-05-20T16:12:46.264563+0000","last_peered":"2024-05-20T16:12:46.264563+0000","last_clean":"2024-05-20T16:12:46.264563+0000","last_became_active":"2024-05-20T16:11:51.438279+0000","last_became_peered":"2024-05-20T16:11:51.438279+0000","last_unstale":"2024-05-20T16:12:46.264563+0000","last_undegraded":"2024-05-20T16:12:46.264563+0000","last_fullsized":"2024-05-20T16:12:46.264563+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_clean_scrub_stamp":"2024-05-20T16:11:50.398948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:30:50.680109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"91'9","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:12:46.258066+0000","last_change":"2024-05-20T16:11:46.397871+0000","last_active":"2024-05-20T16:12:46.258066+0000","last_peered":"2024-05-20T16:12:46.258066+0000","last_clean":"2024-05-20T16:12:46.258066+0000","last_became_active":"2024-05-20T16:11:46.397569+0000","last_became_peered":"2024-05-20T16:11:46.397569+0000","last_unstale":"2024-05-20T16:12:46.258066+0000","last_undegraded":"2024-05-20T16:12:46.258066+0000","last_fullsized":"2024-05-20T16:12:46.258066+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_clean_scrub_stamp":"2024-05-20T16:11:45.363714+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:32:24.329823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"93'5","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:13:15.268953+0000","last_change":"2024-05-20T16:11:47.183935+0000","last_active":"2024-05-20T16:13:15.268953+0000","last_peered":"2024-05-20T16:13:15.268953+0000","last_clean":"2024-05-20T16:13:15.268953+0000","last_became_active":"2024-05-20T16:11:47.183604+0000","last_became_peered":"2024-05-20T16:11:47.183604+0000","last_unstale":"2024-05-20T16:13:15.268953+0000","last_undegraded":"2024-05-20T16:13:15.268953+0000","last_fullsized":"2024-05-20T16:13:15.268953+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T16:59:36.924396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:12:46.254966+0000","last_change":"2024-05-20T16:11:51.441603+0000","last_active":"2024-05-20T16:12:46.254966+0000","last_peered":"2024-05-20T16:12:46.254966+0000","last_clean":"2024-05-20T16:12:46.254966+0000","last_became_active":"2024-05-20T16:11:51.441106+0000","last_became_peered":"2024-05-20T16:11:51.441106+0000","last_unstale":"2024-05-20T16:12:46.254966+0000","last_undegraded":"2024-05-20T16:12:46.254966+0000","last_fullsized":"2024-05-20T16:12:46.254966+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_clean_scrub_stamp":"2024-05-20T16:11:50.398948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:07:29.221992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"91'2","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:12:46.260477+0000","last_change":"2024-05-20T16:11:46.401617+0000","last_active":"2024-05-20T16:12:46.260477+0000","last_peered":"2024-05-20T16:12:46.260477+0000","last_clean":"2024-05-20T16:12:46.260477+0000","last_became_active":"2024-05-20T16:11:46.401295+0000","last_became_peered":"2024-05-20T16:11:46.401295+0000","last_unstale":"2024-05-20T16:12:46.260477+0000","last_undegraded":"2024-05-20T16:12:46.260477+0000","last_fullsized":"2024-05-20T16:12:46.260477+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_clean_scrub_stamp":"2024-05-20T16:11:45.363714+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:42:21.039088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"93'6","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:13:15.295162+0000","last_change":"2024-05-20T16:11:47.187157+0000","last_active":"2024-05-20T16:13:15.295162+0000","last_peered":"2024-05-20T16:13:15.295162+0000","last_clean":"2024-05-20T16:13:15.295162+0000","last_became_active":"2024-05-20T16:11:47.186765+0000","last_became_peered":"2024-05-20T16:11:47.186765+0000","last_unstale":"2024-05-20T16:13:15.295162+0000","last_undegraded":"2024-05-20T16:13:15.295162+0000","last_fullsized":"2024-05-20T16:13:15.295162+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T17:09:56.687541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:12:46.263765+0000","last_change":"2024-05-20T16:12:46.263765+0000","last_active":"2024-05-20T16:12:46.263765+0000","last_peered":"2024-05-20T16:12:46.263765+0000","last_clean":"2024-05-20T16:12:46.263765+0000","last_became_active":"2024-05-20T16:12:46.263227+0000","last_became_peered":"2024-05-20T16:12:46.263227+0000","last_unstale":"2024-05-20T16:12:46.263765+0000","last_undegraded":"2024-05-20T16:12:46.263765+0000","last_fullsized":"2024-05-20T16:12:46.263765+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_clean_scrub_stamp":"2024-05-20T16:11:50.398948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:58:15.914997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,8,5],"acting":[4,3,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"91'4","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:12:46.273546+0000","last_change":"2024-05-20T16:12:46.273546+0000","last_active":"2024-05-20T16:12:46.273546+0000","last_peered":"2024-05-20T16:12:46.273546+0000","last_clean":"2024-05-20T16:12:46.273546+0000","last_became_active":"2024-05-20T16:12:46.266687+0000","last_became_peered":"2024-05-20T16:12:46.266687+0000","last_unstale":"2024-05-20T16:12:46.273546+0000","last_undegraded":"2024-05-20T16:12:46.273546+0000","last_fullsized":"2024-05-20T16:12:46.273546+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_clean_scrub_stamp":"2024-05-20T16:11:45.363714+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T20:09:20.348722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"93'9","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:13:15.296581+0000","last_change":"2024-05-20T16:11:47.180014+0000","last_active":"2024-05-20T16:13:15.296581+0000","last_peered":"2024-05-20T16:13:15.296581+0000","last_clean":"2024-05-20T16:13:15.296581+0000","last_became_active":"2024-05-20T16:11:47.179660+0000","last_became_peered":"2024-05-20T16:11:47.179660+0000","last_unstale":"2024-05-20T16:13:15.296581+0000","last_undegraded":"2024-05-20T16:13:15.296581+0000","last_fullsized":"2024-05-20T16:13:15.296581+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:07:26.925346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:12:46.255037+0000","last_change":"2024-05-20T16:11:51.439268+0000","last_active":"2024-05-20T16:12:46.255037+0000","last_peered":"2024-05-20T16:12:46.255037+0000","last_clean":"2024-05-20T16:12:46.255037+0000","last_became_active":"2024-05-20T16:11:51.438767+0000","last_became_peered":"2024-05-20T16:11:51.438767+0000","last_unstale":"2024-05-20T16:12:46.255037+0000","last_undegraded":"2024-05-20T16:12:46.255037+0000","last_fullsized":"2024-05-20T16:12:46.255037+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_clean_scrub_stamp":"2024-05-20T16:11:50.398948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:31:22.394666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:12:46.260172+0000","last_change":"2024-05-20T16:11:46.398922+0000","last_active":"2024-05-20T16:12:46.260172+0000","last_peered":"2024-05-20T16:12:46.260172+0000","last_clean":"2024-05-20T16:12:46.260172+0000","last_became_active":"2024-05-20T16:11:46.398434+0000","last_became_peered":"2024-05-20T16:11:46.398434+0000","last_unstale":"2024-05-20T16:12:46.260172+0000","last_undegraded":"2024-05-20T16:12:46.260172+0000","last_fullsized":"2024-05-20T16:12:46.260172+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_clean_scrub_stamp":"2024-05-20T16:11:45.363714+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T17:38:12.120113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"93'15","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:13:15.292591+0000","last_change":"2024-05-20T16:12:46.332013+0000","last_active":"2024-05-20T16:13:15.292591+0000","last_peered":"2024-05-20T16:13:15.292591+0000","last_clean":"2024-05-20T16:13:15.292591+0000","last_became_active":"2024-05-20T16:12:46.331754+0000","last_became_peered":"2024-05-20T16:12:46.331754+0000","last_unstale":"2024-05-20T16:13:15.292591+0000","last_undegraded":"2024-05-20T16:13:15.292591+0000","last_fullsized":"2024-05-20T16:13:15.292591+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:01:50.678745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:12:46.263663+0000","last_change":"2024-05-20T16:11:51.435981+0000","last_active":"2024-05-20T16:12:46.263663+0000","last_peered":"2024-05-20T16:12:46.263663+0000","last_clean":"2024-05-20T16:12:46.263663+0000","last_became_active":"2024-05-20T16:11:51.435387+0000","last_became_peered":"2024-05-20T16:11:51.435387+0000","last_unstale":"2024-05-20T16:12:46.263663+0000","last_undegraded":"2024-05-20T16:12:46.263663+0000","last_fullsized":"2024-05-20T16:12:46.263663+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_clean_scrub_stamp":"2024-05-20T16:11:50.398948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T16:14:14.877384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"91'5","reported_seq":105,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:13:20.512511+0000","last_change":"2024-05-20T16:11:46.402453+0000","last_active":"2024-05-20T16:13:20.512511+0000","last_peered":"2024-05-20T16:13:20.512511+0000","last_clean":"2024-05-20T16:13:20.512511+0000","last_became_active":"2024-05-20T16:11:46.400824+0000","last_became_peered":"2024-05-20T16:11:46.400824+0000","last_unstale":"2024-05-20T16:13:20.512511+0000","last_undegraded":"2024-05-20T16:13:20.512511+0000","last_fullsized":"2024-05-20T16:13:20.512511+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_clean_scrub_stamp":"2024-05-20T16:11:45.363714+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T17:39:47.431485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":71,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"93'7","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:13:15.270553+0000","last_change":"2024-05-20T16:11:47.180885+0000","last_active":"2024-05-20T16:13:15.270553+0000","last_peered":"2024-05-20T16:13:15.270553+0000","last_clean":"2024-05-20T16:13:15.270553+0000","last_became_active":"2024-05-20T16:11:47.178680+0000","last_became_peered":"2024-05-20T16:11:47.178680+0000","last_unstale":"2024-05-20T16:13:15.270553+0000","last_undegraded":"2024-05-20T16:13:15.270553+0000","last_fullsized":"2024-05-20T16:13:15.270553+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T19:28:00.540561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:12:46.263793+0000","last_change":"2024-05-20T16:11:51.436093+0000","last_active":"2024-05-20T16:12:46.263793+0000","last_peered":"2024-05-20T16:12:46.263793+0000","last_clean":"2024-05-20T16:12:46.263793+0000","last_became_active":"2024-05-20T16:11:51.435448+0000","last_became_peered":"2024-05-20T16:11:51.435448+0000","last_unstale":"2024-05-20T16:12:46.263793+0000","last_undegraded":"2024-05-20T16:12:46.263793+0000","last_fullsized":"2024-05-20T16:12:46.263793+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_clean_scrub_stamp":"2024-05-20T16:11:50.398948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:43:52.365055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"91'3","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:12:46.258555+0000","last_change":"2024-05-20T16:11:46.401695+0000","last_active":"2024-05-20T16:12:46.258555+0000","last_peered":"2024-05-20T16:12:46.258555+0000","last_clean":"2024-05-20T16:12:46.258555+0000","last_became_active":"2024-05-20T16:11:46.401352+0000","last_became_peered":"2024-05-20T16:11:46.401352+0000","last_unstale":"2024-05-20T16:12:46.258555+0000","last_undegraded":"2024-05-20T16:12:46.258555+0000","last_fullsized":"2024-05-20T16:12:46.258555+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_clean_scrub_stamp":"2024-05-20T16:11:45.363714+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:50:58.714842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"93'8","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:13:15.283025+0000","last_change":"2024-05-20T16:11:47.185820+0000","last_active":"2024-05-20T16:13:15.283025+0000","last_peered":"2024-05-20T16:13:15.283025+0000","last_clean":"2024-05-20T16:13:15.283025+0000","last_became_active":"2024-05-20T16:11:47.185491+0000","last_became_peered":"2024-05-20T16:11:47.185491+0000","last_unstale":"2024-05-20T16:13:15.283025+0000","last_undegraded":"2024-05-20T16:13:15.283025+0000","last_fullsized":"2024-05-20T16:13:15.283025+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:23:27.065634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:12:46.263688+0000","last_change":"2024-05-20T16:11:51.432409+0000","last_active":"2024-05-20T16:12:46.263688+0000","last_peered":"2024-05-20T16:12:46.263688+0000","last_clean":"2024-05-20T16:12:46.263688+0000","last_became_active":"2024-05-20T16:11:51.431993+0000","last_became_peered":"2024-05-20T16:11:51.431993+0000","last_unstale":"2024-05-20T16:12:46.263688+0000","last_undegraded":"2024-05-20T16:12:46.263688+0000","last_fullsized":"2024-05-20T16:12:46.263688+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_clean_scrub_stamp":"2024-05-20T16:11:50.398948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:41:08.301120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"91'2","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:12:46.253999+0000","last_change":"2024-05-20T16:11:46.397358+0000","last_active":"2024-05-20T16:12:46.253999+0000","last_peered":"2024-05-20T16:12:46.253999+0000","last_clean":"2024-05-20T16:12:46.253999+0000","last_became_active":"2024-05-20T16:11:46.396989+0000","last_became_peered":"2024-05-20T16:11:46.396989+0000","last_unstale":"2024-05-20T16:12:46.253999+0000","last_undegraded":"2024-05-20T16:12:46.253999+0000","last_fullsized":"2024-05-20T16:12:46.253999+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_clean_scrub_stamp":"2024-05-20T16:11:45.363714+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:31:19.737223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"93'8","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:13:15.269954+0000","last_change":"2024-05-20T16:11:47.175117+0000","last_active":"2024-05-20T16:13:15.269954+0000","last_peered":"2024-05-20T16:13:15.269954+0000","last_clean":"2024-05-20T16:13:15.269954+0000","last_became_active":"2024-05-20T16:11:47.174885+0000","last_became_peered":"2024-05-20T16:11:47.174885+0000","last_unstale":"2024-05-20T16:13:15.269954+0000","last_undegraded":"2024-05-20T16:13:15.269954+0000","last_fullsized":"2024-05-20T16:13:15.269954+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:38:35.765034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:12:46.257454+0000","last_change":"2024-05-20T16:11:51.440650+0000","last_active":"2024-05-20T16:12:46.257454+0000","last_peered":"2024-05-20T16:12:46.257454+0000","last_clean":"2024-05-20T16:12:46.257454+0000","last_became_active":"2024-05-20T16:11:51.440208+0000","last_became_peered":"2024-05-20T16:11:51.440208+0000","last_unstale":"2024-05-20T16:12:46.257454+0000","last_undegraded":"2024-05-20T16:12:46.257454+0000","last_fullsized":"2024-05-20T16:12:46.257454+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_clean_scrub_stamp":"2024-05-20T16:11:50.398948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:02:02.925911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"91'1","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:12:46.254591+0000","last_change":"2024-05-20T16:11:46.402078+0000","last_active":"2024-05-20T16:12:46.254591+0000","last_peered":"2024-05-20T16:12:46.254591+0000","last_clean":"2024-05-20T16:12:46.254591+0000","last_became_active":"2024-05-20T16:11:46.401680+0000","last_became_peered":"2024-05-20T16:11:46.401680+0000","last_unstale":"2024-05-20T16:12:46.254591+0000","last_undegraded":"2024-05-20T16:12:46.254591+0000","last_fullsized":"2024-05-20T16:12:46.254591+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_clean_scrub_stamp":"2024-05-20T16:11:45.363714+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:32:06.118688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"93'11","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:13:15.283528+0000","last_change":"2024-05-20T16:11:47.186933+0000","last_active":"2024-05-20T16:13:15.283528+0000","last_peered":"2024-05-20T16:13:15.283528+0000","last_clean":"2024-05-20T16:13:15.283528+0000","last_became_active":"2024-05-20T16:11:47.186554+0000","last_became_peered":"2024-05-20T16:11:47.186554+0000","last_unstale":"2024-05-20T16:13:15.283528+0000","last_undegraded":"2024-05-20T16:13:15.283528+0000","last_fullsized":"2024-05-20T16:13:15.283528+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T16:24:34.634974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:12:46.260332+0000","last_change":"2024-05-20T16:11:51.442535+0000","last_active":"2024-05-20T16:12:46.260332+0000","last_peered":"2024-05-20T16:12:46.260332+0000","last_clean":"2024-05-20T16:12:46.260332+0000","last_became_active":"2024-05-20T16:11:51.442143+0000","last_became_peered":"2024-05-20T16:11:51.442143+0000","last_unstale":"2024-05-20T16:12:46.260332+0000","last_undegraded":"2024-05-20T16:12:46.260332+0000","last_fullsized":"2024-05-20T16:12:46.260332+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_clean_scrub_stamp":"2024-05-20T16:11:50.398948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T19:31:41.640330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"91'3","reported_seq":101,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:13:20.511734+0000","last_change":"2024-05-20T16:11:46.394319+0000","last_active":"2024-05-20T16:13:20.511734+0000","last_peered":"2024-05-20T16:13:20.511734+0000","last_clean":"2024-05-20T16:13:20.511734+0000","last_became_active":"2024-05-20T16:11:46.393975+0000","last_became_peered":"2024-05-20T16:11:46.393975+0000","last_unstale":"2024-05-20T16:13:20.511734+0000","last_undegraded":"2024-05-20T16:13:20.511734+0000","last_fullsized":"2024-05-20T16:13:20.511734+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_clean_scrub_stamp":"2024-05-20T16:11:45.363714+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:13:18.706944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":69,"num_read_kb":69,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"93'10","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:13:15.297081+0000","last_change":"2024-05-20T16:11:47.177382+0000","last_active":"2024-05-20T16:13:15.297081+0000","last_peered":"2024-05-20T16:13:15.297081+0000","last_clean":"2024-05-20T16:13:15.297081+0000","last_became_active":"2024-05-20T16:11:47.176949+0000","last_became_peered":"2024-05-20T16:11:47.176949+0000","last_unstale":"2024-05-20T16:13:15.297081+0000","last_undegraded":"2024-05-20T16:13:15.297081+0000","last_fullsized":"2024-05-20T16:13:15.297081+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:20:36.689937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:12:46.256517+0000","last_change":"2024-05-20T16:11:51.438622+0000","last_active":"2024-05-20T16:12:46.256517+0000","last_peered":"2024-05-20T16:12:46.256517+0000","last_clean":"2024-05-20T16:12:46.256517+0000","last_became_active":"2024-05-20T16:11:51.438191+0000","last_became_peered":"2024-05-20T16:11:51.438191+0000","last_unstale":"2024-05-20T16:12:46.256517+0000","last_undegraded":"2024-05-20T16:12:46.256517+0000","last_fullsized":"2024-05-20T16:12:46.256517+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_clean_scrub_stamp":"2024-05-20T16:11:50.398948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T16:39:56.327423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"91'2","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:12:46.260527+0000","last_change":"2024-05-20T16:11:46.399436+0000","last_active":"2024-05-20T16:12:46.260527+0000","last_peered":"2024-05-20T16:12:46.260527+0000","last_clean":"2024-05-20T16:12:46.260527+0000","last_became_active":"2024-05-20T16:11:46.398591+0000","last_became_peered":"2024-05-20T16:11:46.398591+0000","last_unstale":"2024-05-20T16:12:46.260527+0000","last_undegraded":"2024-05-20T16:12:46.260527+0000","last_fullsized":"2024-05-20T16:12:46.260527+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_clean_scrub_stamp":"2024-05-20T16:11:45.363714+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:18:26.775585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"93'9","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:13:15.275934+0000","last_change":"2024-05-20T16:12:46.261798+0000","last_active":"2024-05-20T16:13:15.275934+0000","last_peered":"2024-05-20T16:13:15.275934+0000","last_clean":"2024-05-20T16:13:15.275934+0000","last_became_active":"2024-05-20T16:12:46.261372+0000","last_became_peered":"2024-05-20T16:12:46.261372+0000","last_unstale":"2024-05-20T16:13:15.275934+0000","last_undegraded":"2024-05-20T16:13:15.275934+0000","last_fullsized":"2024-05-20T16:13:15.275934+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:42:59.073996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,5],"acting":[0,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:12:46.255212+0000","last_change":"2024-05-20T16:11:51.442169+0000","last_active":"2024-05-20T16:12:46.255212+0000","last_peered":"2024-05-20T16:12:46.255212+0000","last_clean":"2024-05-20T16:12:46.255212+0000","last_became_active":"2024-05-20T16:11:51.441114+0000","last_became_peered":"2024-05-20T16:11:51.441114+0000","last_unstale":"2024-05-20T16:12:46.255212+0000","last_undegraded":"2024-05-20T16:12:46.255212+0000","last_fullsized":"2024-05-20T16:12:46.255212+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_clean_scrub_stamp":"2024-05-20T16:11:50.398948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:01:50.833632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"91'4","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:12:46.254987+0000","last_change":"2024-05-20T16:11:46.403424+0000","last_active":"2024-05-20T16:12:46.254987+0000","last_peered":"2024-05-20T16:12:46.254987+0000","last_clean":"2024-05-20T16:12:46.254987+0000","last_became_active":"2024-05-20T16:11:46.400786+0000","last_became_peered":"2024-05-20T16:11:46.400786+0000","last_unstale":"2024-05-20T16:12:46.254987+0000","last_undegraded":"2024-05-20T16:12:46.254987+0000","last_fullsized":"2024-05-20T16:12:46.254987+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_clean_scrub_stamp":"2024-05-20T16:11:45.363714+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T16:29:33.300267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"93'11","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:13:15.295071+0000","last_change":"2024-05-20T16:11:47.176372+0000","last_active":"2024-05-20T16:13:15.295071+0000","last_peered":"2024-05-20T16:13:15.295071+0000","last_clean":"2024-05-20T16:13:15.295071+0000","last_became_active":"2024-05-20T16:11:47.175535+0000","last_became_peered":"2024-05-20T16:11:47.175535+0000","last_unstale":"2024-05-20T16:13:15.295071+0000","last_undegraded":"2024-05-20T16:13:15.295071+0000","last_fullsized":"2024-05-20T16:13:15.295071+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:40:22.293771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:12:46.256909+0000","last_change":"2024-05-20T16:11:51.442678+0000","last_active":"2024-05-20T16:12:46.256909+0000","last_peered":"2024-05-20T16:12:46.256909+0000","last_clean":"2024-05-20T16:12:46.256909+0000","last_became_active":"2024-05-20T16:11:51.442300+0000","last_became_peered":"2024-05-20T16:11:51.442300+0000","last_unstale":"2024-05-20T16:12:46.256909+0000","last_undegraded":"2024-05-20T16:12:46.256909+0000","last_fullsized":"2024-05-20T16:12:46.256909+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_clean_scrub_stamp":"2024-05-20T16:11:50.398948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:29:14.668640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"93'35","reported_seq":210,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:13:21.513409+0000","last_change":"2024-05-20T16:11:46.398550+0000","last_active":"2024-05-20T16:13:21.513409+0000","last_peered":"2024-05-20T16:13:21.513409+0000","last_clean":"2024-05-20T16:13:21.513409+0000","last_became_active":"2024-05-20T16:11:46.398216+0000","last_became_peered":"2024-05-20T16:11:46.398216+0000","last_unstale":"2024-05-20T16:13:21.513409+0000","last_undegraded":"2024-05-20T16:13:21.513409+0000","last_fullsized":"2024-05-20T16:13:21.513409+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_clean_scrub_stamp":"2024-05-20T16:11:45.363714+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:24:53.292837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62024,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":146,"num_read_kb":134,"num_write":36,"num_write_kb":78,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"93'6","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:13:15.271849+0000","last_change":"2024-05-20T16:11:47.177641+0000","last_active":"2024-05-20T16:13:15.271849+0000","last_peered":"2024-05-20T16:13:15.271849+0000","last_clean":"2024-05-20T16:13:15.271849+0000","last_became_active":"2024-05-20T16:11:47.177333+0000","last_became_peered":"2024-05-20T16:11:47.177333+0000","last_unstale":"2024-05-20T16:13:15.271849+0000","last_undegraded":"2024-05-20T16:13:15.271849+0000","last_fullsized":"2024-05-20T16:13:15.271849+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T17:42:01.674485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:12:46.255496+0000","last_change":"2024-05-20T16:11:51.445804+0000","last_active":"2024-05-20T16:12:46.255496+0000","last_peered":"2024-05-20T16:12:46.255496+0000","last_clean":"2024-05-20T16:12:46.255496+0000","last_became_active":"2024-05-20T16:11:51.445223+0000","last_became_peered":"2024-05-20T16:11:51.445223+0000","last_unstale":"2024-05-20T16:12:46.255496+0000","last_undegraded":"2024-05-20T16:12:46.255496+0000","last_fullsized":"2024-05-20T16:12:46.255496+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_clean_scrub_stamp":"2024-05-20T16:11:50.398948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:45:19.185992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"91'6","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:12:46.263537+0000","last_change":"2024-05-20T16:11:46.398349+0000","last_active":"2024-05-20T16:12:46.263537+0000","last_peered":"2024-05-20T16:12:46.263537+0000","last_clean":"2024-05-20T16:12:46.263537+0000","last_became_active":"2024-05-20T16:11:46.397878+0000","last_became_peered":"2024-05-20T16:11:46.397878+0000","last_unstale":"2024-05-20T16:12:46.263537+0000","last_undegraded":"2024-05-20T16:12:46.263537+0000","last_fullsized":"2024-05-20T16:12:46.263537+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_clean_scrub_stamp":"2024-05-20T16:11:45.363714+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:39:07.374098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"93'9","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:13:15.275775+0000","last_change":"2024-05-20T16:11:47.179322+0000","last_active":"2024-05-20T16:13:15.275775+0000","last_peered":"2024-05-20T16:13:15.275775+0000","last_clean":"2024-05-20T16:13:15.275775+0000","last_became_active":"2024-05-20T16:11:47.178897+0000","last_became_peered":"2024-05-20T16:11:47.178897+0000","last_unstale":"2024-05-20T16:13:15.275775+0000","last_undegraded":"2024-05-20T16:13:15.275775+0000","last_fullsized":"2024-05-20T16:13:15.275775+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T20:01:04.096952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:12:46.264436+0000","last_change":"2024-05-20T16:11:51.439577+0000","last_active":"2024-05-20T16:12:46.264436+0000","last_peered":"2024-05-20T16:12:46.264436+0000","last_clean":"2024-05-20T16:12:46.264436+0000","last_became_active":"2024-05-20T16:11:51.439043+0000","last_became_peered":"2024-05-20T16:11:51.439043+0000","last_unstale":"2024-05-20T16:12:46.264436+0000","last_undegraded":"2024-05-20T16:12:46.264436+0000","last_fullsized":"2024-05-20T16:12:46.264436+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_clean_scrub_stamp":"2024-05-20T16:11:50.398948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:23:18.861715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"91'3","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:12:46.255190+0000","last_change":"2024-05-20T16:11:46.402276+0000","last_active":"2024-05-20T16:12:46.255190+0000","last_peered":"2024-05-20T16:12:46.255190+0000","last_clean":"2024-05-20T16:12:46.255190+0000","last_became_active":"2024-05-20T16:11:46.401923+0000","last_became_peered":"2024-05-20T16:11:46.401923+0000","last_unstale":"2024-05-20T16:12:46.255190+0000","last_undegraded":"2024-05-20T16:12:46.255190+0000","last_fullsized":"2024-05-20T16:12:46.255190+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_clean_scrub_stamp":"2024-05-20T16:11:45.363714+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T17:57:00.320492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"93'15","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:13:15.289691+0000","last_change":"2024-05-20T16:11:47.174448+0000","last_active":"2024-05-20T16:13:15.289691+0000","last_peered":"2024-05-20T16:13:15.289691+0000","last_clean":"2024-05-20T16:13:15.289691+0000","last_became_active":"2024-05-20T16:11:47.174099+0000","last_became_peered":"2024-05-20T16:11:47.174099+0000","last_unstale":"2024-05-20T16:13:15.289691+0000","last_undegraded":"2024-05-20T16:13:15.289691+0000","last_fullsized":"2024-05-20T16:13:15.289691+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T20:53:30.392099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:12:46.257754+0000","last_change":"2024-05-20T16:11:51.446231+0000","last_active":"2024-05-20T16:12:46.257754+0000","last_peered":"2024-05-20T16:12:46.257754+0000","last_clean":"2024-05-20T16:12:46.257754+0000","last_became_active":"2024-05-20T16:11:51.445805+0000","last_became_peered":"2024-05-20T16:11:51.445805+0000","last_unstale":"2024-05-20T16:12:46.257754+0000","last_undegraded":"2024-05-20T16:12:46.257754+0000","last_fullsized":"2024-05-20T16:12:46.257754+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_clean_scrub_stamp":"2024-05-20T16:11:50.398948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:51:29.505139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"93'5","reported_seq":103,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:13:16.506367+0000","last_change":"2024-05-20T16:11:46.392677+0000","last_active":"2024-05-20T16:13:16.506367+0000","last_peered":"2024-05-20T16:13:16.506367+0000","last_clean":"2024-05-20T16:13:16.506367+0000","last_became_active":"2024-05-20T16:11:46.392315+0000","last_became_peered":"2024-05-20T16:11:46.392315+0000","last_unstale":"2024-05-20T16:13:16.506367+0000","last_undegraded":"2024-05-20T16:13:16.506367+0000","last_fullsized":"2024-05-20T16:13:16.506367+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_clean_scrub_stamp":"2024-05-20T16:11:45.363714+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T19:04:14.191754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":70,"num_read_kb":5,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"93'6","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:13:15.284565+0000","last_change":"2024-05-20T16:11:47.187104+0000","last_active":"2024-05-20T16:13:15.284565+0000","last_peered":"2024-05-20T16:13:15.284565+0000","last_clean":"2024-05-20T16:13:15.284565+0000","last_became_active":"2024-05-20T16:11:47.186375+0000","last_became_peered":"2024-05-20T16:11:47.186375+0000","last_unstale":"2024-05-20T16:13:15.284565+0000","last_undegraded":"2024-05-20T16:13:15.284565+0000","last_fullsized":"2024-05-20T16:13:15.284565+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:27:32.478899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:12:46.254578+0000","last_change":"2024-05-20T16:11:51.444872+0000","last_active":"2024-05-20T16:12:46.254578+0000","last_peered":"2024-05-20T16:12:46.254578+0000","last_clean":"2024-05-20T16:12:46.254578+0000","last_became_active":"2024-05-20T16:11:51.444472+0000","last_became_peered":"2024-05-20T16:11:51.444472+0000","last_unstale":"2024-05-20T16:12:46.254578+0000","last_undegraded":"2024-05-20T16:12:46.254578+0000","last_fullsized":"2024-05-20T16:12:46.254578+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_clean_scrub_stamp":"2024-05-20T16:11:50.398948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:37:57.117101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"91'15","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:12:46.259448+0000","last_change":"2024-05-20T16:11:46.407244+0000","last_active":"2024-05-20T16:12:46.259448+0000","last_peered":"2024-05-20T16:12:46.259448+0000","last_clean":"2024-05-20T16:12:46.259448+0000","last_became_active":"2024-05-20T16:11:46.406905+0000","last_became_peered":"2024-05-20T16:11:46.406905+0000","last_unstale":"2024-05-20T16:12:46.259448+0000","last_undegraded":"2024-05-20T16:12:46.259448+0000","last_fullsized":"2024-05-20T16:12:46.259448+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_clean_scrub_stamp":"2024-05-20T16:11:45.363714+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:45:40.497442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"93'6","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:13:15.279102+0000","last_change":"2024-05-20T16:11:47.180326+0000","last_active":"2024-05-20T16:13:15.279102+0000","last_peered":"2024-05-20T16:13:15.279102+0000","last_clean":"2024-05-20T16:13:15.279102+0000","last_became_active":"2024-05-20T16:11:47.179937+0000","last_became_peered":"2024-05-20T16:11:47.179937+0000","last_unstale":"2024-05-20T16:13:15.279102+0000","last_undegraded":"2024-05-20T16:13:15.279102+0000","last_fullsized":"2024-05-20T16:13:15.279102+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:59:59.431365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:12:46.254650+0000","last_change":"2024-05-20T16:11:51.444495+0000","last_active":"2024-05-20T16:12:46.254650+0000","last_peered":"2024-05-20T16:12:46.254650+0000","last_clean":"2024-05-20T16:12:46.254650+0000","last_became_active":"2024-05-20T16:11:51.444136+0000","last_became_peered":"2024-05-20T16:11:51.444136+0000","last_unstale":"2024-05-20T16:12:46.254650+0000","last_undegraded":"2024-05-20T16:12:46.254650+0000","last_fullsized":"2024-05-20T16:12:46.254650+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_clean_scrub_stamp":"2024-05-20T16:11:50.398948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:27:46.260340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"91'2","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:12:46.260524+0000","last_change":"2024-05-20T16:11:46.407252+0000","last_active":"2024-05-20T16:12:46.260524+0000","last_peered":"2024-05-20T16:12:46.260524+0000","last_clean":"2024-05-20T16:12:46.260524+0000","last_became_active":"2024-05-20T16:11:46.406867+0000","last_became_peered":"2024-05-20T16:11:46.406867+0000","last_unstale":"2024-05-20T16:12:46.260524+0000","last_undegraded":"2024-05-20T16:12:46.260524+0000","last_fullsized":"2024-05-20T16:12:46.260524+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_clean_scrub_stamp":"2024-05-20T16:11:45.363714+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T17:53:46.638460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"93'5","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:13:15.287269+0000","last_change":"2024-05-20T16:11:47.179046+0000","last_active":"2024-05-20T16:13:15.287269+0000","last_peered":"2024-05-20T16:13:15.287269+0000","last_clean":"2024-05-20T16:13:15.287269+0000","last_became_active":"2024-05-20T16:11:47.178722+0000","last_became_peered":"2024-05-20T16:11:47.178722+0000","last_unstale":"2024-05-20T16:13:15.287269+0000","last_undegraded":"2024-05-20T16:13:15.287269+0000","last_fullsized":"2024-05-20T16:13:15.287269+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T17:38:15.355878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:12:46.333010+0000","last_change":"2024-05-20T16:12:46.333010+0000","last_active":"2024-05-20T16:12:46.333010+0000","last_peered":"2024-05-20T16:12:46.333010+0000","last_clean":"2024-05-20T16:12:46.333010+0000","last_became_active":"2024-05-20T16:12:46.332454+0000","last_became_peered":"2024-05-20T16:12:46.332454+0000","last_unstale":"2024-05-20T16:12:46.333010+0000","last_undegraded":"2024-05-20T16:12:46.333010+0000","last_fullsized":"2024-05-20T16:12:46.333010+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_clean_scrub_stamp":"2024-05-20T16:11:50.398948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:17:46.394972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,1],"acting":[8,10,11,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:12:46.257445+0000","last_change":"2024-05-20T16:11:46.402585+0000","last_active":"2024-05-20T16:12:46.257445+0000","last_peered":"2024-05-20T16:12:46.257445+0000","last_clean":"2024-05-20T16:12:46.257445+0000","last_became_active":"2024-05-20T16:11:46.402163+0000","last_became_peered":"2024-05-20T16:11:46.402163+0000","last_unstale":"2024-05-20T16:12:46.257445+0000","last_undegraded":"2024-05-20T16:12:46.257445+0000","last_fullsized":"2024-05-20T16:12:46.257445+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_clean_scrub_stamp":"2024-05-20T16:11:45.363714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:35:26.036970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"93'4","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:13:15.279765+0000","last_change":"2024-05-20T16:11:47.183088+0000","last_active":"2024-05-20T16:13:15.279765+0000","last_peered":"2024-05-20T16:13:15.279765+0000","last_clean":"2024-05-20T16:13:15.279765+0000","last_became_active":"2024-05-20T16:11:47.182608+0000","last_became_peered":"2024-05-20T16:11:47.182608+0000","last_unstale":"2024-05-20T16:13:15.279765+0000","last_undegraded":"2024-05-20T16:13:15.279765+0000","last_fullsized":"2024-05-20T16:13:15.279765+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T20:23:37.086778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:12:46.254399+0000","last_change":"2024-05-20T16:11:51.437420+0000","last_active":"2024-05-20T16:12:46.254399+0000","last_peered":"2024-05-20T16:12:46.254399+0000","last_clean":"2024-05-20T16:12:46.254399+0000","last_became_active":"2024-05-20T16:11:51.437018+0000","last_became_peered":"2024-05-20T16:11:51.437018+0000","last_unstale":"2024-05-20T16:12:46.254399+0000","last_undegraded":"2024-05-20T16:12:46.254399+0000","last_fullsized":"2024-05-20T16:12:46.254399+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_clean_scrub_stamp":"2024-05-20T16:11:50.398948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:03:30.650894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"91'1","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:12:46.260053+0000","last_change":"2024-05-20T16:11:46.400818+0000","last_active":"2024-05-20T16:12:46.260053+0000","last_peered":"2024-05-20T16:12:46.260053+0000","last_clean":"2024-05-20T16:12:46.260053+0000","last_became_active":"2024-05-20T16:11:46.400482+0000","last_became_peered":"2024-05-20T16:11:46.400482+0000","last_unstale":"2024-05-20T16:12:46.260053+0000","last_undegraded":"2024-05-20T16:12:46.260053+0000","last_fullsized":"2024-05-20T16:12:46.260053+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_clean_scrub_stamp":"2024-05-20T16:11:45.363714+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:13:13.968284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"93'7","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:13:15.278880+0000","last_change":"2024-05-20T16:12:46.262401+0000","last_active":"2024-05-20T16:13:15.278880+0000","last_peered":"2024-05-20T16:13:15.278880+0000","last_clean":"2024-05-20T16:13:15.278880+0000","last_became_active":"2024-05-20T16:12:46.262027+0000","last_became_peered":"2024-05-20T16:12:46.262027+0000","last_unstale":"2024-05-20T16:13:15.278880+0000","last_undegraded":"2024-05-20T16:13:15.278880+0000","last_fullsized":"2024-05-20T16:13:15.278880+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T16:50:38.920937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:12:46.254575+0000","last_change":"2024-05-20T16:11:51.915176+0000","last_active":"2024-05-20T16:12:46.254575+0000","last_peered":"2024-05-20T16:12:46.254575+0000","last_clean":"2024-05-20T16:12:46.254575+0000","last_became_active":"2024-05-20T16:11:51.914766+0000","last_became_peered":"2024-05-20T16:11:51.914766+0000","last_unstale":"2024-05-20T16:12:46.254575+0000","last_undegraded":"2024-05-20T16:12:46.254575+0000","last_fullsized":"2024-05-20T16:12:46.254575+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_clean_scrub_stamp":"2024-05-20T16:11:50.398948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T16:26:02.276646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"93'8","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:13:15.276028+0000","last_change":"2024-05-20T16:11:47.184502+0000","last_active":"2024-05-20T16:13:15.276028+0000","last_peered":"2024-05-20T16:13:15.276028+0000","last_clean":"2024-05-20T16:13:15.276028+0000","last_became_active":"2024-05-20T16:11:47.184123+0000","last_became_peered":"2024-05-20T16:11:47.184123+0000","last_unstale":"2024-05-20T16:13:15.276028+0000","last_undegraded":"2024-05-20T16:13:15.276028+0000","last_fullsized":"2024-05-20T16:13:15.276028+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:55:23.989206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"93'5","reported_seq":100,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:13:17.507236+0000","last_change":"2024-05-20T16:11:46.399731+0000","last_active":"2024-05-20T16:13:17.507236+0000","last_peered":"2024-05-20T16:13:17.507236+0000","last_clean":"2024-05-20T16:13:17.507236+0000","last_became_active":"2024-05-20T16:11:46.399349+0000","last_became_peered":"2024-05-20T16:11:46.399349+0000","last_unstale":"2024-05-20T16:13:17.507236+0000","last_undegraded":"2024-05-20T16:13:17.507236+0000","last_fullsized":"2024-05-20T16:13:17.507236+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_clean_scrub_stamp":"2024-05-20T16:11:45.363714+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T20:27:39.038116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":66,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"91'1","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:12:46.273802+0000","last_change":"2024-05-20T16:12:46.273802+0000","last_active":"2024-05-20T16:12:46.273802+0000","last_peered":"2024-05-20T16:12:46.273802+0000","last_clean":"2024-05-20T16:12:46.273802+0000","last_became_active":"2024-05-20T16:12:46.266005+0000","last_became_peered":"2024-05-20T16:12:46.266005+0000","last_unstale":"2024-05-20T16:12:46.273802+0000","last_undegraded":"2024-05-20T16:12:46.273802+0000","last_fullsized":"2024-05-20T16:12:46.273802+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_clean_scrub_stamp":"2024-05-20T16:11:50.398948+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:17:16.692728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,10],"acting":[7,5,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"93'8","reported_seq":19,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:13:15.291934+0000","last_change":"2024-05-20T16:12:46.332691+0000","last_active":"2024-05-20T16:13:15.291934+0000","last_peered":"2024-05-20T16:13:15.291934+0000","last_clean":"2024-05-20T16:13:15.291934+0000","last_became_active":"2024-05-20T16:12:46.332456+0000","last_became_peered":"2024-05-20T16:12:46.332456+0000","last_unstale":"2024-05-20T16:13:15.291934+0000","last_undegraded":"2024-05-20T16:13:15.291934+0000","last_fullsized":"2024-05-20T16:13:15.291934+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:15:09.762973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,1],"acting":[9,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.3","version":"91'3","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:12:46.255067+0000","last_change":"2024-05-20T16:11:46.393867+0000","last_active":"2024-05-20T16:12:46.255067+0000","last_peered":"2024-05-20T16:12:46.255067+0000","last_clean":"2024-05-20T16:12:46.255067+0000","last_became_active":"2024-05-20T16:11:46.393523+0000","last_became_peered":"2024-05-20T16:11:46.393523+0000","last_unstale":"2024-05-20T16:12:46.255067+0000","last_undegraded":"2024-05-20T16:12:46.255067+0000","last_fullsized":"2024-05-20T16:12:46.255067+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_clean_scrub_stamp":"2024-05-20T16:11:45.363714+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T18:36:40.067975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":353,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:12:46.259417+0000","last_change":"2024-05-20T16:10:35.566958+0000","last_active":"2024-05-20T16:12:46.259417+0000","last_peered":"2024-05-20T16:12:46.259417+0000","last_clean":"2024-05-20T16:12:46.259417+0000","last_became_active":"2024-05-20T16:10:35.566551+0000","last_became_peered":"2024-05-20T16:10:35.566551+0000","last_unstale":"2024-05-20T16:12:46.259417+0000","last_undegraded":"2024-05-20T16:12:46.259417+0000","last_fullsized":"2024-05-20T16:12:46.259417+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:07:09.543878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:07:09.543878+0000","last_clean_scrub_stamp":"2024-05-20T16:07:09.543878+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:48:26.484769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:12:46.255451+0000","last_change":"2024-05-20T16:11:51.915534+0000","last_active":"2024-05-20T16:12:46.255451+0000","last_peered":"2024-05-20T16:12:46.255451+0000","last_clean":"2024-05-20T16:12:46.255451+0000","last_became_active":"2024-05-20T16:11:51.914903+0000","last_became_peered":"2024-05-20T16:11:51.914903+0000","last_unstale":"2024-05-20T16:12:46.255451+0000","last_undegraded":"2024-05-20T16:12:46.255451+0000","last_fullsized":"2024-05-20T16:12:46.255451+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_clean_scrub_stamp":"2024-05-20T16:11:50.398948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T18:56:47.255129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"93'5","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:13:15.284769+0000","last_change":"2024-05-20T16:11:47.174101+0000","last_active":"2024-05-20T16:13:15.284769+0000","last_peered":"2024-05-20T16:13:15.284769+0000","last_clean":"2024-05-20T16:13:15.284769+0000","last_became_active":"2024-05-20T16:11:47.173772+0000","last_became_peered":"2024-05-20T16:11:47.173772+0000","last_unstale":"2024-05-20T16:13:15.284769+0000","last_undegraded":"2024-05-20T16:13:15.284769+0000","last_fullsized":"2024-05-20T16:13:15.284769+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:33:33.656757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"91'3","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:12:46.259500+0000","last_change":"2024-05-20T16:11:46.401951+0000","last_active":"2024-05-20T16:12:46.259500+0000","last_peered":"2024-05-20T16:12:46.259500+0000","last_clean":"2024-05-20T16:12:46.259500+0000","last_became_active":"2024-05-20T16:11:46.401377+0000","last_became_peered":"2024-05-20T16:11:46.401377+0000","last_unstale":"2024-05-20T16:12:46.259500+0000","last_undegraded":"2024-05-20T16:12:46.259500+0000","last_fullsized":"2024-05-20T16:12:46.259500+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_clean_scrub_stamp":"2024-05-20T16:11:45.363714+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:00:43.538206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:12:46.263417+0000","last_change":"2024-05-20T16:11:51.436284+0000","last_active":"2024-05-20T16:12:46.263417+0000","last_peered":"2024-05-20T16:12:46.263417+0000","last_clean":"2024-05-20T16:12:46.263417+0000","last_became_active":"2024-05-20T16:11:51.435755+0000","last_became_peered":"2024-05-20T16:11:51.435755+0000","last_unstale":"2024-05-20T16:12:46.263417+0000","last_undegraded":"2024-05-20T16:12:46.263417+0000","last_fullsized":"2024-05-20T16:12:46.263417+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_clean_scrub_stamp":"2024-05-20T16:11:50.398948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T20:51:05.434690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"91'1","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:12:46.260188+0000","last_change":"2024-05-20T16:11:46.408055+0000","last_active":"2024-05-20T16:12:46.260188+0000","last_peered":"2024-05-20T16:12:46.260188+0000","last_clean":"2024-05-20T16:12:46.260188+0000","last_became_active":"2024-05-20T16:11:46.407733+0000","last_became_peered":"2024-05-20T16:11:46.407733+0000","last_unstale":"2024-05-20T16:12:46.260188+0000","last_undegraded":"2024-05-20T16:12:46.260188+0000","last_fullsized":"2024-05-20T16:12:46.260188+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_clean_scrub_stamp":"2024-05-20T16:11:45.363714+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T18:32:32.344334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"93'9","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:13:15.265986+0000","last_change":"2024-05-20T16:11:47.179831+0000","last_active":"2024-05-20T16:13:15.265986+0000","last_peered":"2024-05-20T16:13:15.265986+0000","last_clean":"2024-05-20T16:13:15.265986+0000","last_became_active":"2024-05-20T16:11:47.179501+0000","last_became_peered":"2024-05-20T16:11:47.179501+0000","last_unstale":"2024-05-20T16:13:15.265986+0000","last_undegraded":"2024-05-20T16:13:15.265986+0000","last_fullsized":"2024-05-20T16:13:15.265986+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T17:38:13.651180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:12:46.260876+0000","last_change":"2024-05-20T16:11:51.915210+0000","last_active":"2024-05-20T16:12:46.260876+0000","last_peered":"2024-05-20T16:12:46.260876+0000","last_clean":"2024-05-20T16:12:46.260876+0000","last_became_active":"2024-05-20T16:11:51.914751+0000","last_became_peered":"2024-05-20T16:11:51.914751+0000","last_unstale":"2024-05-20T16:12:46.260876+0000","last_undegraded":"2024-05-20T16:12:46.260876+0000","last_fullsized":"2024-05-20T16:12:46.260876+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_clean_scrub_stamp":"2024-05-20T16:11:50.398948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:07:14.817497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"93'9","reported_seq":50,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:12:46.271268+0000","last_change":"2024-05-20T16:12:46.270755+0000","last_active":"2024-05-20T16:12:46.271268+0000","last_peered":"2024-05-20T16:12:46.271268+0000","last_clean":"2024-05-20T16:12:46.271268+0000","last_became_active":"2024-05-20T16:12:46.266551+0000","last_became_peered":"2024-05-20T16:12:46.266551+0000","last_unstale":"2024-05-20T16:12:46.271268+0000","last_undegraded":"2024-05-20T16:12:46.271268+0000","last_fullsized":"2024-05-20T16:12:46.271268+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_clean_scrub_stamp":"2024-05-20T16:11:45.363714+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:44:15.685183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"93'5","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:13:15.279643+0000","last_change":"2024-05-20T16:11:47.184550+0000","last_active":"2024-05-20T16:13:15.279643+0000","last_peered":"2024-05-20T16:13:15.279643+0000","last_clean":"2024-05-20T16:13:15.279643+0000","last_became_active":"2024-05-20T16:11:47.184260+0000","last_became_peered":"2024-05-20T16:11:47.184260+0000","last_unstale":"2024-05-20T16:13:15.279643+0000","last_undegraded":"2024-05-20T16:13:15.279643+0000","last_fullsized":"2024-05-20T16:13:15.279643+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T17:21:43.013563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:12:46.264163+0000","last_change":"2024-05-20T16:11:51.431710+0000","last_active":"2024-05-20T16:12:46.264163+0000","last_peered":"2024-05-20T16:12:46.264163+0000","last_clean":"2024-05-20T16:12:46.264163+0000","last_became_active":"2024-05-20T16:11:51.431286+0000","last_became_peered":"2024-05-20T16:11:51.431286+0000","last_unstale":"2024-05-20T16:12:46.264163+0000","last_undegraded":"2024-05-20T16:12:46.264163+0000","last_fullsized":"2024-05-20T16:12:46.264163+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_clean_scrub_stamp":"2024-05-20T16:11:50.398948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:35:19.016412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"91'4","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:12:46.255791+0000","last_change":"2024-05-20T16:11:46.401772+0000","last_active":"2024-05-20T16:12:46.255791+0000","last_peered":"2024-05-20T16:12:46.255791+0000","last_clean":"2024-05-20T16:12:46.255791+0000","last_became_active":"2024-05-20T16:11:46.401414+0000","last_became_peered":"2024-05-20T16:11:46.401414+0000","last_unstale":"2024-05-20T16:12:46.255791+0000","last_undegraded":"2024-05-20T16:12:46.255791+0000","last_fullsized":"2024-05-20T16:12:46.255791+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_clean_scrub_stamp":"2024-05-20T16:11:45.363714+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:23:59.146573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"93'7","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:13:15.287336+0000","last_change":"2024-05-20T16:11:47.177333+0000","last_active":"2024-05-20T16:13:15.287336+0000","last_peered":"2024-05-20T16:13:15.287336+0000","last_clean":"2024-05-20T16:13:15.287336+0000","last_became_active":"2024-05-20T16:11:47.176161+0000","last_became_peered":"2024-05-20T16:11:47.176161+0000","last_unstale":"2024-05-20T16:13:15.287336+0000","last_undegraded":"2024-05-20T16:13:15.287336+0000","last_fullsized":"2024-05-20T16:13:15.287336+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:48:40.846477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:12:46.262127+0000","last_change":"2024-05-20T16:12:46.262127+0000","last_active":"2024-05-20T16:12:46.262127+0000","last_peered":"2024-05-20T16:12:46.262127+0000","last_clean":"2024-05-20T16:12:46.262127+0000","last_became_active":"2024-05-20T16:12:46.261596+0000","last_became_peered":"2024-05-20T16:12:46.261596+0000","last_unstale":"2024-05-20T16:12:46.262127+0000","last_undegraded":"2024-05-20T16:12:46.262127+0000","last_fullsized":"2024-05-20T16:12:46.262127+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_clean_scrub_stamp":"2024-05-20T16:11:50.398948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T17:27:26.605849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,10,9],"acting":[4,8,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"91'2","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:12:46.331186+0000","last_change":"2024-05-20T16:11:46.407127+0000","last_active":"2024-05-20T16:12:46.331186+0000","last_peered":"2024-05-20T16:12:46.331186+0000","last_clean":"2024-05-20T16:12:46.331186+0000","last_became_active":"2024-05-20T16:11:46.406833+0000","last_became_peered":"2024-05-20T16:11:46.406833+0000","last_unstale":"2024-05-20T16:12:46.331186+0000","last_undegraded":"2024-05-20T16:12:46.331186+0000","last_fullsized":"2024-05-20T16:12:46.331186+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_clean_scrub_stamp":"2024-05-20T16:11:45.363714+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:58:30.464017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"93'2","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:13:15.239238+0000","last_change":"2024-05-20T16:11:47.174434+0000","last_active":"2024-05-20T16:13:15.239238+0000","last_peered":"2024-05-20T16:13:15.239238+0000","last_clean":"2024-05-20T16:13:15.239238+0000","last_became_active":"2024-05-20T16:11:47.174019+0000","last_became_peered":"2024-05-20T16:11:47.174019+0000","last_unstale":"2024-05-20T16:13:15.239238+0000","last_undegraded":"2024-05-20T16:13:15.239238+0000","last_fullsized":"2024-05-20T16:13:15.239238+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T18:32:51.150925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:12:46.257949+0000","last_change":"2024-05-20T16:11:51.446241+0000","last_active":"2024-05-20T16:12:46.257949+0000","last_peered":"2024-05-20T16:12:46.257949+0000","last_clean":"2024-05-20T16:12:46.257949+0000","last_became_active":"2024-05-20T16:11:51.445828+0000","last_became_peered":"2024-05-20T16:11:51.445828+0000","last_unstale":"2024-05-20T16:12:46.257949+0000","last_undegraded":"2024-05-20T16:12:46.257949+0000","last_fullsized":"2024-05-20T16:12:46.257949+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_clean_scrub_stamp":"2024-05-20T16:11:50.398948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:26:00.239338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"91'3","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:12:46.260360+0000","last_change":"2024-05-20T16:11:46.396364+0000","last_active":"2024-05-20T16:12:46.260360+0000","last_peered":"2024-05-20T16:12:46.260360+0000","last_clean":"2024-05-20T16:12:46.260360+0000","last_became_active":"2024-05-20T16:11:46.396014+0000","last_became_peered":"2024-05-20T16:11:46.396014+0000","last_unstale":"2024-05-20T16:12:46.260360+0000","last_undegraded":"2024-05-20T16:12:46.260360+0000","last_fullsized":"2024-05-20T16:12:46.260360+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_clean_scrub_stamp":"2024-05-20T16:11:45.363714+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:02:46.660077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"93'9","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:13:15.277346+0000","last_change":"2024-05-20T16:11:47.184320+0000","last_active":"2024-05-20T16:13:15.277346+0000","last_peered":"2024-05-20T16:13:15.277346+0000","last_clean":"2024-05-20T16:13:15.277346+0000","last_became_active":"2024-05-20T16:11:47.183981+0000","last_became_peered":"2024-05-20T16:11:47.183981+0000","last_unstale":"2024-05-20T16:13:15.277346+0000","last_undegraded":"2024-05-20T16:13:15.277346+0000","last_fullsized":"2024-05-20T16:13:15.277346+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T17:41:47.160563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"91'1","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:12:46.255197+0000","last_change":"2024-05-20T16:11:51.440073+0000","last_active":"2024-05-20T16:12:46.255197+0000","last_peered":"2024-05-20T16:12:46.255197+0000","last_clean":"2024-05-20T16:12:46.255197+0000","last_became_active":"2024-05-20T16:11:51.439560+0000","last_became_peered":"2024-05-20T16:11:51.439560+0000","last_unstale":"2024-05-20T16:12:46.255197+0000","last_undegraded":"2024-05-20T16:12:46.255197+0000","last_fullsized":"2024-05-20T16:12:46.255197+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_clean_scrub_stamp":"2024-05-20T16:11:50.398948+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:50:54.429811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"91'1","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:12:46.264171+0000","last_change":"2024-05-20T16:11:46.404866+0000","last_active":"2024-05-20T16:12:46.264171+0000","last_peered":"2024-05-20T16:12:46.264171+0000","last_clean":"2024-05-20T16:12:46.264171+0000","last_became_active":"2024-05-20T16:11:46.404509+0000","last_became_peered":"2024-05-20T16:11:46.404509+0000","last_unstale":"2024-05-20T16:12:46.264171+0000","last_undegraded":"2024-05-20T16:12:46.264171+0000","last_fullsized":"2024-05-20T16:12:46.264171+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_clean_scrub_stamp":"2024-05-20T16:11:45.363714+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:58:40.303826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"93'8","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:13:15.273412+0000","last_change":"2024-05-20T16:11:47.191505+0000","last_active":"2024-05-20T16:13:15.273412+0000","last_peered":"2024-05-20T16:13:15.273412+0000","last_clean":"2024-05-20T16:13:15.273412+0000","last_became_active":"2024-05-20T16:11:47.191181+0000","last_became_peered":"2024-05-20T16:11:47.191181+0000","last_unstale":"2024-05-20T16:13:15.273412+0000","last_undegraded":"2024-05-20T16:13:15.273412+0000","last_fullsized":"2024-05-20T16:13:15.273412+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T19:30:42.561146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:12:46.255739+0000","last_change":"2024-05-20T16:11:51.915313+0000","last_active":"2024-05-20T16:12:46.255739+0000","last_peered":"2024-05-20T16:12:46.255739+0000","last_clean":"2024-05-20T16:12:46.255739+0000","last_became_active":"2024-05-20T16:11:51.914761+0000","last_became_peered":"2024-05-20T16:11:51.914761+0000","last_unstale":"2024-05-20T16:12:46.255739+0000","last_undegraded":"2024-05-20T16:12:46.255739+0000","last_fullsized":"2024-05-20T16:12:46.255739+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_clean_scrub_stamp":"2024-05-20T16:11:50.398948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T17:15:36.673449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"91'4","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:12:46.259598+0000","last_change":"2024-05-20T16:11:46.404513+0000","last_active":"2024-05-20T16:12:46.259598+0000","last_peered":"2024-05-20T16:12:46.259598+0000","last_clean":"2024-05-20T16:12:46.259598+0000","last_became_active":"2024-05-20T16:11:46.404163+0000","last_became_peered":"2024-05-20T16:11:46.404163+0000","last_unstale":"2024-05-20T16:12:46.259598+0000","last_undegraded":"2024-05-20T16:12:46.259598+0000","last_fullsized":"2024-05-20T16:12:46.259598+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_clean_scrub_stamp":"2024-05-20T16:11:45.363714+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:24:07.155558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"93'5","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:13:15.295027+0000","last_change":"2024-05-20T16:11:47.174527+0000","last_active":"2024-05-20T16:13:15.295027+0000","last_peered":"2024-05-20T16:13:15.295027+0000","last_clean":"2024-05-20T16:13:15.295027+0000","last_became_active":"2024-05-20T16:11:47.174165+0000","last_became_peered":"2024-05-20T16:11:47.174165+0000","last_unstale":"2024-05-20T16:13:15.295027+0000","last_undegraded":"2024-05-20T16:13:15.295027+0000","last_fullsized":"2024-05-20T16:13:15.295027+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T17:44:47.614704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:12:46.331450+0000","last_change":"2024-05-20T16:11:51.439013+0000","last_active":"2024-05-20T16:12:46.331450+0000","last_peered":"2024-05-20T16:12:46.331450+0000","last_clean":"2024-05-20T16:12:46.331450+0000","last_became_active":"2024-05-20T16:11:51.438408+0000","last_became_peered":"2024-05-20T16:11:51.438408+0000","last_unstale":"2024-05-20T16:12:46.331450+0000","last_undegraded":"2024-05-20T16:12:46.331450+0000","last_fullsized":"2024-05-20T16:12:46.331450+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_clean_scrub_stamp":"2024-05-20T16:11:50.398948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T19:27:53.723823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"91'3","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:12:46.263985+0000","last_change":"2024-05-20T16:11:46.407283+0000","last_active":"2024-05-20T16:12:46.263985+0000","last_peered":"2024-05-20T16:12:46.263985+0000","last_clean":"2024-05-20T16:12:46.263985+0000","last_became_active":"2024-05-20T16:11:46.406919+0000","last_became_peered":"2024-05-20T16:11:46.406919+0000","last_unstale":"2024-05-20T16:12:46.263985+0000","last_undegraded":"2024-05-20T16:12:46.263985+0000","last_fullsized":"2024-05-20T16:12:46.263985+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_clean_scrub_stamp":"2024-05-20T16:11:45.363714+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:16:09.166422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"93'7","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:13:15.295515+0000","last_change":"2024-05-20T16:11:47.177973+0000","last_active":"2024-05-20T16:13:15.295515+0000","last_peered":"2024-05-20T16:13:15.295515+0000","last_clean":"2024-05-20T16:13:15.295515+0000","last_became_active":"2024-05-20T16:11:47.177658+0000","last_became_peered":"2024-05-20T16:11:47.177658+0000","last_unstale":"2024-05-20T16:13:15.295515+0000","last_undegraded":"2024-05-20T16:13:15.295515+0000","last_fullsized":"2024-05-20T16:13:15.295515+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:34:04.004730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:12:46.257507+0000","last_change":"2024-05-20T16:11:51.441326+0000","last_active":"2024-05-20T16:12:46.257507+0000","last_peered":"2024-05-20T16:12:46.257507+0000","last_clean":"2024-05-20T16:12:46.257507+0000","last_became_active":"2024-05-20T16:11:51.440903+0000","last_became_peered":"2024-05-20T16:11:51.440903+0000","last_unstale":"2024-05-20T16:12:46.257507+0000","last_undegraded":"2024-05-20T16:12:46.257507+0000","last_fullsized":"2024-05-20T16:12:46.257507+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_clean_scrub_stamp":"2024-05-20T16:11:50.398948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:52:54.988916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"85'1","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:12:46.255364+0000","last_change":"2024-05-20T16:11:46.397548+0000","last_active":"2024-05-20T16:12:46.255364+0000","last_peered":"2024-05-20T16:12:46.255364+0000","last_clean":"2024-05-20T16:12:46.255364+0000","last_became_active":"2024-05-20T16:11:46.397207+0000","last_became_peered":"2024-05-20T16:11:46.397207+0000","last_unstale":"2024-05-20T16:12:46.255364+0000","last_undegraded":"2024-05-20T16:12:46.255364+0000","last_fullsized":"2024-05-20T16:12:46.255364+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_clean_scrub_stamp":"2024-05-20T16:11:45.363714+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:25:50.468298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"93'6","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:13:15.285778+0000","last_change":"2024-05-20T16:11:47.184162+0000","last_active":"2024-05-20T16:13:15.285778+0000","last_peered":"2024-05-20T16:13:15.285778+0000","last_clean":"2024-05-20T16:13:15.285778+0000","last_became_active":"2024-05-20T16:11:47.183845+0000","last_became_peered":"2024-05-20T16:11:47.183845+0000","last_unstale":"2024-05-20T16:13:15.285778+0000","last_undegraded":"2024-05-20T16:13:15.285778+0000","last_fullsized":"2024-05-20T16:13:15.285778+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T20:23:14.728178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:12:46.254464+0000","last_change":"2024-05-20T16:11:51.437974+0000","last_active":"2024-05-20T16:12:46.254464+0000","last_peered":"2024-05-20T16:12:46.254464+0000","last_clean":"2024-05-20T16:12:46.254464+0000","last_became_active":"2024-05-20T16:11:51.437053+0000","last_became_peered":"2024-05-20T16:11:51.437053+0000","last_unstale":"2024-05-20T16:12:46.254464+0000","last_undegraded":"2024-05-20T16:12:46.254464+0000","last_fullsized":"2024-05-20T16:12:46.254464+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_clean_scrub_stamp":"2024-05-20T16:11:50.398948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:09:31.314668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:12:46.254601+0000","last_change":"2024-05-20T16:11:46.398461+0000","last_active":"2024-05-20T16:12:46.254601+0000","last_peered":"2024-05-20T16:12:46.254601+0000","last_clean":"2024-05-20T16:12:46.254601+0000","last_became_active":"2024-05-20T16:11:46.398122+0000","last_became_peered":"2024-05-20T16:11:46.398122+0000","last_unstale":"2024-05-20T16:12:46.254601+0000","last_undegraded":"2024-05-20T16:12:46.254601+0000","last_fullsized":"2024-05-20T16:12:46.254601+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_clean_scrub_stamp":"2024-05-20T16:11:45.363714+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:34:45.592470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"93'9","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:13:15.281805+0000","last_change":"2024-05-20T16:11:47.179160+0000","last_active":"2024-05-20T16:13:15.281805+0000","last_peered":"2024-05-20T16:13:15.281805+0000","last_clean":"2024-05-20T16:13:15.281805+0000","last_became_active":"2024-05-20T16:11:47.178843+0000","last_became_peered":"2024-05-20T16:11:47.178843+0000","last_unstale":"2024-05-20T16:13:15.281805+0000","last_undegraded":"2024-05-20T16:13:15.281805+0000","last_fullsized":"2024-05-20T16:13:15.281805+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T20:05:57.013207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"93'9","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:13:15.277352+0000","last_change":"2024-05-20T16:11:47.185401+0000","last_active":"2024-05-20T16:13:15.277352+0000","last_peered":"2024-05-20T16:13:15.277352+0000","last_clean":"2024-05-20T16:13:15.277352+0000","last_became_active":"2024-05-20T16:11:47.185061+0000","last_became_peered":"2024-05-20T16:11:47.185061+0000","last_unstale":"2024-05-20T16:13:15.277352+0000","last_undegraded":"2024-05-20T16:13:15.277352+0000","last_fullsized":"2024-05-20T16:13:15.277352+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:33:32.528968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"93'12","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:13:15.286641+0000","last_change":"2024-05-20T16:11:47.174418+0000","last_active":"2024-05-20T16:13:15.286641+0000","last_peered":"2024-05-20T16:13:15.286641+0000","last_clean":"2024-05-20T16:13:15.286641+0000","last_became_active":"2024-05-20T16:11:47.174078+0000","last_became_peered":"2024-05-20T16:11:47.174078+0000","last_unstale":"2024-05-20T16:13:15.286641+0000","last_undegraded":"2024-05-20T16:13:15.286641+0000","last_fullsized":"2024-05-20T16:13:15.286641+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T16:20:12.268161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"93'8","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:13:15.265829+0000","last_change":"2024-05-20T16:11:47.177686+0000","last_active":"2024-05-20T16:13:15.265829+0000","last_peered":"2024-05-20T16:13:15.265829+0000","last_clean":"2024-05-20T16:13:15.265829+0000","last_became_active":"2024-05-20T16:11:47.177305+0000","last_became_peered":"2024-05-20T16:11:47.177305+0000","last_unstale":"2024-05-20T16:13:15.265829+0000","last_undegraded":"2024-05-20T16:13:15.265829+0000","last_fullsized":"2024-05-20T16:13:15.265829+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T18:36:35.919491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"93'7","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:13:15.287222+0000","last_change":"2024-05-20T16:11:47.175383+0000","last_active":"2024-05-20T16:13:15.287222+0000","last_peered":"2024-05-20T16:13:15.287222+0000","last_clean":"2024-05-20T16:13:15.287222+0000","last_became_active":"2024-05-20T16:11:47.175079+0000","last_became_peered":"2024-05-20T16:11:47.175079+0000","last_unstale":"2024-05-20T16:13:15.287222+0000","last_undegraded":"2024-05-20T16:13:15.287222+0000","last_fullsized":"2024-05-20T16:13:15.287222+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:33:52.072038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"93'4","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:13:15.286325+0000","last_change":"2024-05-20T16:11:47.182717+0000","last_active":"2024-05-20T16:13:15.286325+0000","last_peered":"2024-05-20T16:13:15.286325+0000","last_clean":"2024-05-20T16:13:15.286325+0000","last_became_active":"2024-05-20T16:11:47.182408+0000","last_became_peered":"2024-05-20T16:11:47.182408+0000","last_unstale":"2024-05-20T16:13:15.286325+0000","last_undegraded":"2024-05-20T16:13:15.286325+0000","last_fullsized":"2024-05-20T16:13:15.286325+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T18:25:13.698066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"93'6","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:13:15.276882+0000","last_change":"2024-05-20T16:11:47.182804+0000","last_active":"2024-05-20T16:13:15.276882+0000","last_peered":"2024-05-20T16:13:15.276882+0000","last_clean":"2024-05-20T16:13:15.276882+0000","last_became_active":"2024-05-20T16:11:47.182486+0000","last_became_peered":"2024-05-20T16:11:47.182486+0000","last_unstale":"2024-05-20T16:13:15.276882+0000","last_undegraded":"2024-05-20T16:13:15.276882+0000","last_fullsized":"2024-05-20T16:13:15.276882+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:24:52.592522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"93'15","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:13:15.292377+0000","last_change":"2024-05-20T16:11:47.175689+0000","last_active":"2024-05-20T16:13:15.292377+0000","last_peered":"2024-05-20T16:13:15.292377+0000","last_clean":"2024-05-20T16:13:15.292377+0000","last_became_active":"2024-05-20T16:11:47.168216+0000","last_became_peered":"2024-05-20T16:11:47.168216+0000","last_unstale":"2024-05-20T16:13:15.292377+0000","last_undegraded":"2024-05-20T16:13:15.292377+0000","last_fullsized":"2024-05-20T16:13:15.292377+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:50:52.707138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"93'2","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:13:15.261590+0000","last_change":"2024-05-20T16:11:47.191398+0000","last_active":"2024-05-20T16:13:15.261590+0000","last_peered":"2024-05-20T16:13:15.261590+0000","last_clean":"2024-05-20T16:13:15.261590+0000","last_became_active":"2024-05-20T16:11:47.191036+0000","last_became_peered":"2024-05-20T16:11:47.191036+0000","last_unstale":"2024-05-20T16:13:15.261590+0000","last_undegraded":"2024-05-20T16:13:15.261590+0000","last_fullsized":"2024-05-20T16:13:15.261590+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:27:49.023581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"93'6","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:13:15.274825+0000","last_change":"2024-05-20T16:11:47.184206+0000","last_active":"2024-05-20T16:13:15.274825+0000","last_peered":"2024-05-20T16:13:15.274825+0000","last_clean":"2024-05-20T16:13:15.274825+0000","last_became_active":"2024-05-20T16:11:47.183860+0000","last_became_peered":"2024-05-20T16:11:47.183860+0000","last_unstale":"2024-05-20T16:13:15.274825+0000","last_undegraded":"2024-05-20T16:13:15.274825+0000","last_fullsized":"2024-05-20T16:13:15.274825+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:52:42.507615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"93'8","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:13:15.270571+0000","last_change":"2024-05-20T16:11:47.180298+0000","last_active":"2024-05-20T16:13:15.270571+0000","last_peered":"2024-05-20T16:13:15.270571+0000","last_clean":"2024-05-20T16:13:15.270571+0000","last_became_active":"2024-05-20T16:11:47.179996+0000","last_became_peered":"2024-05-20T16:11:47.179996+0000","last_unstale":"2024-05-20T16:13:15.270571+0000","last_undegraded":"2024-05-20T16:13:15.270571+0000","last_fullsized":"2024-05-20T16:13:15.270571+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:49:53.432240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"93'10","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:13:15.293376+0000","last_change":"2024-05-20T16:11:47.176123+0000","last_active":"2024-05-20T16:13:15.293376+0000","last_peered":"2024-05-20T16:13:15.293376+0000","last_clean":"2024-05-20T16:13:15.293376+0000","last_became_active":"2024-05-20T16:11:47.175019+0000","last_became_peered":"2024-05-20T16:11:47.175019+0000","last_unstale":"2024-05-20T16:13:15.293376+0000","last_undegraded":"2024-05-20T16:13:15.293376+0000","last_fullsized":"2024-05-20T16:13:15.293376+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:23:32.010352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"93'11","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:13:15.276663+0000","last_change":"2024-05-20T16:11:47.180877+0000","last_active":"2024-05-20T16:13:15.276663+0000","last_peered":"2024-05-20T16:13:15.276663+0000","last_clean":"2024-05-20T16:13:15.276663+0000","last_became_active":"2024-05-20T16:11:47.180567+0000","last_became_peered":"2024-05-20T16:11:47.180567+0000","last_unstale":"2024-05-20T16:13:15.276663+0000","last_undegraded":"2024-05-20T16:13:15.276663+0000","last_fullsized":"2024-05-20T16:13:15.276663+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T17:54:31.165693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"93'8","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:13:15.287571+0000","last_change":"2024-05-20T16:11:47.179607+0000","last_active":"2024-05-20T16:13:15.287571+0000","last_peered":"2024-05-20T16:13:15.287571+0000","last_clean":"2024-05-20T16:13:15.287571+0000","last_became_active":"2024-05-20T16:11:47.179279+0000","last_became_peered":"2024-05-20T16:11:47.179279+0000","last_unstale":"2024-05-20T16:13:15.287571+0000","last_undegraded":"2024-05-20T16:13:15.287571+0000","last_fullsized":"2024-05-20T16:13:15.287571+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T18:38:20.788325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":246552,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":66166,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":435,"num_read_kb":289,"num_write":167,"num_write_kb":119,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":17,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":405504,"data_stored":198498,"data_compressed":4830,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":145,"ondisk_log_size":145,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351007,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1015808,"data_stored":513273,"data_compressed":23570,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612645,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28148,"kb_used_data":1132,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717004,"statfs":{"total":95995035648,"available":95966212096,"internally_reserved":0,"allocated":1159168,"data_stored":1167193,"data_compressed":60870,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1598,"internal_metadata":27654594},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776171,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1032192,"data_stored":513509,"data_compressed":23570,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037807,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32184,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712968,"statfs":{"total":95995035648,"available":95962079232,"internally_reserved":0,"allocated":1032192,"data_stored":513429,"data_compressed":23570,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299445,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32232,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712920,"statfs":{"total":95995035648,"available":95962030080,"internally_reserved":0,"allocated":1081344,"data_stored":1109479,"data_compressed":59260,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462970,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32164,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712988,"statfs":{"total":95995035648,"available":95962099712,"internally_reserved":0,"allocated":1011712,"data_stored":508685,"data_compressed":23570,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574931,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28064,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717088,"statfs":{"total":95995035648,"available":95966298112,"internally_reserved":0,"allocated":1073152,"data_stored":570619,"data_compressed":25180,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738456,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28028,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717124,"statfs":{"total":95995035648,"available":95966334976,"internally_reserved":0,"allocated":1036288,"data_stored":510073,"data_compressed":23570,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411405,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28088,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717064,"statfs":{"total":95995035648,"available":95966273536,"internally_reserved":0,"allocated":1097728,"data_stored":571969,"data_compressed":25180,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149769,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1024000,"data_stored":508879,"data_compressed":23570,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888132,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1024000,"data_stored":513867,"data_compressed":23570,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724607,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28080,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717072,"statfs":{"total":95995035648,"available":95966281728,"internally_reserved":0,"allocated":1089536,"data_stored":1104535,"data_compressed":59260,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34051,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34051,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35690,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35690,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35690,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35690,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35690,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":62024,"data_compressed":1610,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":63374,"data_compressed":1610,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":64134,"data_compressed":1610,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-20T16:13:25.579 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell --fsid 59d1600a-16c2-11ef-bc9a-c7b262605968 -- ceph pg dump --format=json 2024-05-20T16:13:25.635 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:13:25 smithi184 ceph-mon[36209]: from='client.15117 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-20T16:13:25.635 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:13:25 smithi184 ceph-mon[36209]: pgmap v343: 129 pgs: 129 active+clean; 641 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2024-05-20T16:13:25.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:13:25 smithi160 ceph-mon[36347]: from='client.15117 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-20T16:13:25.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:13:25 smithi160 ceph-mon[36347]: pgmap v343: 129 pgs: 129 active+clean; 641 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2024-05-20T16:13:25.818 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T16:13:27.614 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:13:27 smithi160 ceph-mon[36347]: pgmap v344: 129 pgs: 129 active+clean; 641 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2024-05-20T16:13:27.615 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:13:27 smithi082 ceph-mon[32238]: pgmap v344: 129 pgs: 129 active+clean; 641 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2024-05-20T16:13:27.635 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:13:27 smithi184 ceph-mon[36209]: pgmap v344: 129 pgs: 129 active+clean; 641 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2024-05-20T16:13:27.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:27.637+0000 7fc2d9a69700 1 -- 172.21.15.82:0/1260277906 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc2d41086f0 msgr2=0x7fc2d4110050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:27.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:27.637+0000 7fc2d9a69700 1 --2- 172.21.15.82:0/1260277906 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc2d41086f0 0x7fc2d4110050 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fc2cc00b0d0 tx=0x7fc2cc00b3e0 comp rx=0 tx=0).stop 2024-05-20T16:13:27.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:27.637+0000 7fc2d9a69700 1 -- 172.21.15.82:0/1260277906 shutdown_connections 2024-05-20T16:13:27.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:27.637+0000 7fc2d9a69700 1 --2- 172.21.15.82:0/1260277906 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc2d41086f0 0x7fc2d4110050 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:27.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:27.637+0000 7fc2d9a69700 1 --2- 172.21.15.82:0/1260277906 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc2d4103470 0x7fc2d4103880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:27.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:27.637+0000 7fc2d9a69700 1 --2- 172.21.15.82:0/1260277906 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc2d4107da0 0x7fc2d41081b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:27.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:27.637+0000 7fc2d9a69700 1 -- 172.21.15.82:0/1260277906 >> 172.21.15.82:0/1260277906 conn(0x7fc2d40fed40 msgr2=0x7fc2d4101190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:13:27.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:27.637+0000 7fc2d9a69700 1 -- 172.21.15.82:0/1260277906 shutdown_connections 2024-05-20T16:13:27.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:27.637+0000 7fc2d9a69700 1 -- 172.21.15.82:0/1260277906 wait complete. 2024-05-20T16:13:27.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:27.638+0000 7fc2d9a69700 1 Processor -- start 2024-05-20T16:13:27.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:27.638+0000 7fc2d9a69700 1 -- start start 2024-05-20T16:13:27.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:27.638+0000 7fc2d9a69700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc2d4103470 0x7fc2d4075a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:27.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:27.638+0000 7fc2d9a69700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc2d4107da0 0x7fc2d4075f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:27.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:27.638+0000 7fc2d9a69700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc2d41086f0 0x7fc2d40764d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:27.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:27.638+0000 7fc2d9a69700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fc2d4113470 con 0x7fc2d4107da0 2024-05-20T16:13:27.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:27.638+0000 7fc2d9a69700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fc2d41132f0 con 0x7fc2d4103470 2024-05-20T16:13:27.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:27.638+0000 7fc2d9a69700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fc2d41135f0 con 0x7fc2d41086f0 2024-05-20T16:13:27.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:27.638+0000 7fc2cbfff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc2d4107da0 0x7fc2d4075f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:13:27.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:27.638+0000 7fc2cbfff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc2d4107da0 0x7fc2d4075f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:35902/0 (socket says 172.21.15.82:35902) 2024-05-20T16:13:27.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:27.638+0000 7fc2cbfff700 1 -- 172.21.15.82:0/3482880943 learned_addr learned my addr 172.21.15.82:0/3482880943 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:13:27.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:27.639+0000 7fc2d3fff700 1 --2- 172.21.15.82:0/3482880943 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc2d41086f0 0x7fc2d40764d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:13:27.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:27.639+0000 7fc2d37fe700 1 --2- 172.21.15.82:0/3482880943 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc2d4103470 0x7fc2d4075a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:13:27.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:27.639+0000 7fc2cbfff700 1 -- 172.21.15.82:0/3482880943 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc2d41086f0 msgr2=0x7fc2d40764d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:27.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:27.639+0000 7fc2cbfff700 1 --2- 172.21.15.82:0/3482880943 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc2d41086f0 0x7fc2d40764d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:27.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:27.639+0000 7fc2cbfff700 1 -- 172.21.15.82:0/3482880943 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc2d4103470 msgr2=0x7fc2d4075a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:27.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:27.639+0000 7fc2cbfff700 1 --2- 172.21.15.82:0/3482880943 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc2d4103470 0x7fc2d4075a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:27.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:27.639+0000 7fc2cbfff700 1 -- 172.21.15.82:0/3482880943 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2d41a8140 con 0x7fc2d4107da0 2024-05-20T16:13:27.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:27.639+0000 7fc2d3fff700 1 --2- 172.21.15.82:0/3482880943 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc2d41086f0 0x7fc2d40764d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T16:13:27.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:27.639+0000 7fc2d37fe700 1 --2- 172.21.15.82:0/3482880943 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc2d4103470 0x7fc2d4075a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T16:13:27.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:27.639+0000 7fc2cbfff700 1 --2- 172.21.15.82:0/3482880943 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc2d4107da0 0x7fc2d4075f90 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fc2bc00c8f0 tx=0x7fc2bc00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:13:27.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:27.639+0000 7fc2d17fa700 1 -- 172.21.15.82:0/3482880943 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2bc007b00 con 0x7fc2d4107da0 2024-05-20T16:13:27.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:27.639+0000 7fc2d9a69700 1 -- 172.21.15.82:0/3482880943 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2d41a8430 con 0x7fc2d4107da0 2024-05-20T16:13:27.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:27.639+0000 7fc2d17fa700 1 -- 172.21.15.82:0/3482880943 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc2bc007ca0 con 0x7fc2d4107da0 2024-05-20T16:13:27.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:27.639+0000 7fc2d17fa700 1 -- 172.21.15.82:0/3482880943 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2bc018660 con 0x7fc2d4107da0 2024-05-20T16:13:27.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:27.639+0000 7fc2d9a69700 1 -- 172.21.15.82:0/3482880943 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2d41a8970 con 0x7fc2d4107da0 2024-05-20T16:13:27.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:27.640+0000 7fc2d9a69700 1 -- 172.21.15.82:0/3482880943 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2d4050170 con 0x7fc2d4107da0 2024-05-20T16:13:27.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:27.642+0000 7fc2d17fa700 1 -- 172.21.15.82:0/3482880943 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fc2bc0188c0 con 0x7fc2d4107da0 2024-05-20T16:13:27.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:27.643+0000 7fc2d17fa700 1 --2- 172.21.15.82:0/3482880943 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fc2c00a2c20 0x7fc2c00a5130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:27.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:27.643+0000 7fc2d37fe700 1 --2- 172.21.15.82:0/3482880943 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fc2c00a2c20 0x7fc2c00a5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:13:27.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:27.643+0000 7fc2d17fa700 1 -- 172.21.15.82:0/3482880943 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9092+0+0 (secure 0 0 0) 0x7fc2bc0cf330 con 0x7fc2d4107da0 2024-05-20T16:13:27.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:27.643+0000 7fc2d37fe700 1 --2- 172.21.15.82:0/3482880943 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fc2c00a2c20 0x7fc2c00a5130 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fc2c4005fd0 tx=0x7fc2c4005ee0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:13:27.646 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:27.647+0000 7fc2d17fa700 1 -- 172.21.15.82:0/3482880943 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fc2bc014030 con 0x7fc2d4107da0 2024-05-20T16:13:27.836 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:27.835+0000 7fc2d9a69700 1 -- 172.21.15.82:0/3482880943 --> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc2d4062a80 con 0x7fc2c00a2c20 2024-05-20T16:13:27.844 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:27.845+0000 7fc2d17fa700 1 -- 172.21.15.82:0/3482880943 <== mgr.14150 v2:172.21.15.82:6800/379530823 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340766 (secure 0 0 0) 0x7fc2d4062a80 con 0x7fc2c00a2c20 2024-05-20T16:13:27.848 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-05-20T16:13:27.850 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:27.848+0000 7fc2d9a69700 1 -- 172.21.15.82:0/3482880943 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fc2c00a2c20 msgr2=0x7fc2c00a5130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:27.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:27.848+0000 7fc2d9a69700 1 --2- 172.21.15.82:0/3482880943 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fc2c00a2c20 0x7fc2c00a5130 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fc2c4005fd0 tx=0x7fc2c4005ee0 comp rx=0 tx=0).stop 2024-05-20T16:13:27.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:27.848+0000 7fc2d9a69700 1 -- 172.21.15.82:0/3482880943 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc2d4107da0 msgr2=0x7fc2d4075f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:27.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:27.848+0000 7fc2d9a69700 1 --2- 172.21.15.82:0/3482880943 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc2d4107da0 0x7fc2d4075f90 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fc2bc00c8f0 tx=0x7fc2bc00ccb0 comp rx=0 tx=0).stop 2024-05-20T16:13:27.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:27.848+0000 7fc2d9a69700 1 -- 172.21.15.82:0/3482880943 shutdown_connections 2024-05-20T16:13:27.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:27.848+0000 7fc2d9a69700 1 --2- 172.21.15.82:0/3482880943 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fc2c00a2c20 0x7fc2c00a5130 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:27.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:27.848+0000 7fc2d9a69700 1 --2- 172.21.15.82:0/3482880943 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc2d4103470 0x7fc2d4075a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:27.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:27.848+0000 7fc2d9a69700 1 --2- 172.21.15.82:0/3482880943 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc2d41086f0 0x7fc2d40764d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:27.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:27.848+0000 7fc2d9a69700 1 --2- 172.21.15.82:0/3482880943 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc2d4107da0 0x7fc2d4075f90 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:27.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:27.848+0000 7fc2d9a69700 1 -- 172.21.15.82:0/3482880943 >> 172.21.15.82:0/3482880943 conn(0x7fc2d40fed40 msgr2=0x7fc2d4107510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:13:27.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:27.848+0000 7fc2d9a69700 1 -- 172.21.15.82:0/3482880943 shutdown_connections 2024-05-20T16:13:27.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:27.848+0000 7fc2d9a69700 1 -- 172.21.15.82:0/3482880943 wait complete. 2024-05-20T16:13:27.851 INFO:teuthology.orchestra.run.smithi082.stderr:dumped all 2024-05-20T16:13:28.563 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:13:28 smithi082 ceph-mon[32238]: from='client.15123 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-20T16:13:28.590 INFO:teuthology.orchestra.run.smithi082.stdout:{"pg_ready":true,"pg_map":{"version":344,"stamp":"2024-05-20T16:13:27.280399+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":656768,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":571,"num_read_kb":403,"num_write":350,"num_write_kb":3059,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":1180853,"num_keys_recovered":17,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":784,"ondisk_log_size":784,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":349052,"kb_used_data":12380,"kb_used_omap":18,"kb_used_meta":336557,"kb_avail":1124592772,"statfs":{"total":1151940427776,"available":1151582998528,"internally_reserved":0,"allocated":12677120,"data_stored":8105510,"data_compressed":394740,"data_compressed_allocated":540672,"data_compressed_original":2580480,"omap_allocated":19141,"internal_metadata":344634683},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":390,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":62,"num_read_kb":32,"num_write":7,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004581"},"pg_stats":[{"pgid":"3.3f","version":"93'8","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:13:15.297439+0000","last_change":"2024-05-20T16:11:47.187104+0000","last_active":"2024-05-20T16:13:15.297439+0000","last_peered":"2024-05-20T16:13:15.297439+0000","last_clean":"2024-05-20T16:13:15.297439+0000","last_became_active":"2024-05-20T16:11:47.186791+0000","last_became_peered":"2024-05-20T16:11:47.186791+0000","last_unstale":"2024-05-20T16:13:15.297439+0000","last_undegraded":"2024-05-20T16:13:15.297439+0000","last_fullsized":"2024-05-20T16:13:15.297439+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:33:38.914389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"93'4","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:13:15.278999+0000","last_change":"2024-05-20T16:11:47.187846+0000","last_active":"2024-05-20T16:13:15.278999+0000","last_peered":"2024-05-20T16:13:15.278999+0000","last_clean":"2024-05-20T16:13:15.278999+0000","last_became_active":"2024-05-20T16:11:47.187540+0000","last_became_peered":"2024-05-20T16:11:47.187540+0000","last_unstale":"2024-05-20T16:13:15.278999+0000","last_undegraded":"2024-05-20T16:13:15.278999+0000","last_fullsized":"2024-05-20T16:13:15.278999+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:19:59.784437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"93'7","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:13:15.295054+0000","last_change":"2024-05-20T16:11:47.185215+0000","last_active":"2024-05-20T16:13:15.295054+0000","last_peered":"2024-05-20T16:13:15.295054+0000","last_clean":"2024-05-20T16:13:15.295054+0000","last_became_active":"2024-05-20T16:11:47.184880+0000","last_became_peered":"2024-05-20T16:11:47.184880+0000","last_unstale":"2024-05-20T16:13:15.295054+0000","last_undegraded":"2024-05-20T16:13:15.295054+0000","last_fullsized":"2024-05-20T16:13:15.295054+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T18:43:32.125643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"93'8","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:13:15.284035+0000","last_change":"2024-05-20T16:11:47.175849+0000","last_active":"2024-05-20T16:13:15.284035+0000","last_peered":"2024-05-20T16:13:15.284035+0000","last_clean":"2024-05-20T16:13:15.284035+0000","last_became_active":"2024-05-20T16:11:47.175342+0000","last_became_peered":"2024-05-20T16:11:47.175342+0000","last_unstale":"2024-05-20T16:13:15.284035+0000","last_undegraded":"2024-05-20T16:13:15.284035+0000","last_fullsized":"2024-05-20T16:13:15.284035+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:02:43.798388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"93'4","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:13:15.267690+0000","last_change":"2024-05-20T16:11:47.174798+0000","last_active":"2024-05-20T16:13:15.267690+0000","last_peered":"2024-05-20T16:13:15.267690+0000","last_clean":"2024-05-20T16:13:15.267690+0000","last_became_active":"2024-05-20T16:11:47.174474+0000","last_became_peered":"2024-05-20T16:11:47.174474+0000","last_unstale":"2024-05-20T16:13:15.267690+0000","last_undegraded":"2024-05-20T16:13:15.267690+0000","last_fullsized":"2024-05-20T16:13:15.267690+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:41:24.404901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"93'6","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:13:15.278393+0000","last_change":"2024-05-20T16:11:47.185731+0000","last_active":"2024-05-20T16:13:15.278393+0000","last_peered":"2024-05-20T16:13:15.278393+0000","last_clean":"2024-05-20T16:13:15.278393+0000","last_became_active":"2024-05-20T16:11:47.185380+0000","last_became_peered":"2024-05-20T16:11:47.185380+0000","last_unstale":"2024-05-20T16:13:15.278393+0000","last_undegraded":"2024-05-20T16:13:15.278393+0000","last_fullsized":"2024-05-20T16:13:15.278393+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T16:33:28.854205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"93'5","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:13:15.285182+0000","last_change":"2024-05-20T16:11:47.183594+0000","last_active":"2024-05-20T16:13:15.285182+0000","last_peered":"2024-05-20T16:13:15.285182+0000","last_clean":"2024-05-20T16:13:15.285182+0000","last_became_active":"2024-05-20T16:11:47.183260+0000","last_became_peered":"2024-05-20T16:11:47.183260+0000","last_unstale":"2024-05-20T16:13:15.285182+0000","last_undegraded":"2024-05-20T16:13:15.285182+0000","last_fullsized":"2024-05-20T16:13:15.285182+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T17:12:22.781236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"93'9","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:13:15.274042+0000","last_change":"2024-05-20T16:11:47.191784+0000","last_active":"2024-05-20T16:13:15.274042+0000","last_peered":"2024-05-20T16:13:15.274042+0000","last_clean":"2024-05-20T16:13:15.274042+0000","last_became_active":"2024-05-20T16:11:47.190553+0000","last_became_peered":"2024-05-20T16:11:47.190553+0000","last_unstale":"2024-05-20T16:13:15.274042+0000","last_undegraded":"2024-05-20T16:13:15.274042+0000","last_fullsized":"2024-05-20T16:13:15.274042+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T20:17:29.346688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"93'11","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:13:15.297277+0000","last_change":"2024-05-20T16:11:47.174817+0000","last_active":"2024-05-20T16:13:15.297277+0000","last_peered":"2024-05-20T16:13:15.297277+0000","last_clean":"2024-05-20T16:13:15.297277+0000","last_became_active":"2024-05-20T16:11:47.174483+0000","last_became_peered":"2024-05-20T16:11:47.174483+0000","last_unstale":"2024-05-20T16:13:15.297277+0000","last_undegraded":"2024-05-20T16:13:15.297277+0000","last_fullsized":"2024-05-20T16:13:15.297277+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T20:32:30.284912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"93'16","reported_seq":43,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:13:15.279963+0000","last_change":"2024-05-20T16:11:47.186899+0000","last_active":"2024-05-20T16:13:15.279963+0000","last_peered":"2024-05-20T16:13:15.279963+0000","last_clean":"2024-05-20T16:13:15.279963+0000","last_became_active":"2024-05-20T16:11:47.186564+0000","last_became_peered":"2024-05-20T16:11:47.186564+0000","last_unstale":"2024-05-20T16:13:15.279963+0000","last_undegraded":"2024-05-20T16:13:15.279963+0000","last_fullsized":"2024-05-20T16:13:15.279963+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:19:57.110107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"93'6","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:13:15.295373+0000","last_change":"2024-05-20T16:11:47.182696+0000","last_active":"2024-05-20T16:13:15.295373+0000","last_peered":"2024-05-20T16:13:15.295373+0000","last_clean":"2024-05-20T16:13:15.295373+0000","last_became_active":"2024-05-20T16:11:47.182169+0000","last_became_peered":"2024-05-20T16:11:47.182169+0000","last_unstale":"2024-05-20T16:13:15.295373+0000","last_undegraded":"2024-05-20T16:13:15.295373+0000","last_fullsized":"2024-05-20T16:13:15.295373+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:55:16.413493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"93'7","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:13:15.274598+0000","last_change":"2024-05-20T16:11:47.181716+0000","last_active":"2024-05-20T16:13:15.274598+0000","last_peered":"2024-05-20T16:13:15.274598+0000","last_clean":"2024-05-20T16:13:15.274598+0000","last_became_active":"2024-05-20T16:11:47.181403+0000","last_became_peered":"2024-05-20T16:11:47.181403+0000","last_unstale":"2024-05-20T16:13:15.274598+0000","last_undegraded":"2024-05-20T16:13:15.274598+0000","last_fullsized":"2024-05-20T16:13:15.274598+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:38:04.938882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"93'10","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:13:15.279627+0000","last_change":"2024-05-20T16:11:47.183320+0000","last_active":"2024-05-20T16:13:15.279627+0000","last_peered":"2024-05-20T16:13:15.279627+0000","last_clean":"2024-05-20T16:13:15.279627+0000","last_became_active":"2024-05-20T16:11:47.183028+0000","last_became_peered":"2024-05-20T16:11:47.183028+0000","last_unstale":"2024-05-20T16:13:15.279627+0000","last_undegraded":"2024-05-20T16:13:15.279627+0000","last_fullsized":"2024-05-20T16:13:15.279627+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T17:28:33.614173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"93'7","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:13:15.279572+0000","last_change":"2024-05-20T16:11:47.183718+0000","last_active":"2024-05-20T16:13:15.279572+0000","last_peered":"2024-05-20T16:13:15.279572+0000","last_clean":"2024-05-20T16:13:15.279572+0000","last_became_active":"2024-05-20T16:11:47.183378+0000","last_became_peered":"2024-05-20T16:11:47.183378+0000","last_unstale":"2024-05-20T16:13:15.279572+0000","last_undegraded":"2024-05-20T16:13:15.279572+0000","last_fullsized":"2024-05-20T16:13:15.279572+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:08:32.066924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"93'7","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:13:15.277203+0000","last_change":"2024-05-20T16:11:47.174163+0000","last_active":"2024-05-20T16:13:15.277203+0000","last_peered":"2024-05-20T16:13:15.277203+0000","last_clean":"2024-05-20T16:13:15.277203+0000","last_became_active":"2024-05-20T16:11:47.173798+0000","last_became_peered":"2024-05-20T16:11:47.173798+0000","last_unstale":"2024-05-20T16:13:15.277203+0000","last_undegraded":"2024-05-20T16:13:15.277203+0000","last_fullsized":"2024-05-20T16:13:15.277203+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T20:51:20.646273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"93'9","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:13:15.295629+0000","last_change":"2024-05-20T16:11:47.181809+0000","last_active":"2024-05-20T16:13:15.295629+0000","last_peered":"2024-05-20T16:13:15.295629+0000","last_clean":"2024-05-20T16:13:15.295629+0000","last_became_active":"2024-05-20T16:11:47.181473+0000","last_became_peered":"2024-05-20T16:11:47.181473+0000","last_unstale":"2024-05-20T16:13:15.295629+0000","last_undegraded":"2024-05-20T16:13:15.295629+0000","last_fullsized":"2024-05-20T16:13:15.295629+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:20:45.302995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"93'8","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:13:15.297631+0000","last_change":"2024-05-20T16:11:47.187947+0000","last_active":"2024-05-20T16:13:15.297631+0000","last_peered":"2024-05-20T16:13:15.297631+0000","last_clean":"2024-05-20T16:13:15.297631+0000","last_became_active":"2024-05-20T16:11:47.187623+0000","last_became_peered":"2024-05-20T16:11:47.187623+0000","last_unstale":"2024-05-20T16:13:15.297631+0000","last_undegraded":"2024-05-20T16:13:15.297631+0000","last_fullsized":"2024-05-20T16:13:15.297631+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:52:08.674888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"93'10","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:13:15.292659+0000","last_change":"2024-05-20T16:11:47.188599+0000","last_active":"2024-05-20T16:13:15.292659+0000","last_peered":"2024-05-20T16:13:15.292659+0000","last_clean":"2024-05-20T16:13:15.292659+0000","last_became_active":"2024-05-20T16:11:47.188278+0000","last_became_peered":"2024-05-20T16:11:47.188278+0000","last_unstale":"2024-05-20T16:13:15.292659+0000","last_undegraded":"2024-05-20T16:13:15.292659+0000","last_fullsized":"2024-05-20T16:13:15.292659+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T17:45:42.003546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"93'7","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:13:15.266395+0000","last_change":"2024-05-20T16:11:47.176250+0000","last_active":"2024-05-20T16:13:15.266395+0000","last_peered":"2024-05-20T16:13:15.266395+0000","last_clean":"2024-05-20T16:13:15.266395+0000","last_became_active":"2024-05-20T16:11:47.175903+0000","last_became_peered":"2024-05-20T16:11:47.175903+0000","last_unstale":"2024-05-20T16:13:15.266395+0000","last_undegraded":"2024-05-20T16:13:15.266395+0000","last_fullsized":"2024-05-20T16:13:15.266395+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T18:11:26.045745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:12:46.264563+0000","last_change":"2024-05-20T16:11:51.438824+0000","last_active":"2024-05-20T16:12:46.264563+0000","last_peered":"2024-05-20T16:12:46.264563+0000","last_clean":"2024-05-20T16:12:46.264563+0000","last_became_active":"2024-05-20T16:11:51.438279+0000","last_became_peered":"2024-05-20T16:11:51.438279+0000","last_unstale":"2024-05-20T16:12:46.264563+0000","last_undegraded":"2024-05-20T16:12:46.264563+0000","last_fullsized":"2024-05-20T16:12:46.264563+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_clean_scrub_stamp":"2024-05-20T16:11:50.398948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:30:50.680109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"91'9","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:12:46.258066+0000","last_change":"2024-05-20T16:11:46.397871+0000","last_active":"2024-05-20T16:12:46.258066+0000","last_peered":"2024-05-20T16:12:46.258066+0000","last_clean":"2024-05-20T16:12:46.258066+0000","last_became_active":"2024-05-20T16:11:46.397569+0000","last_became_peered":"2024-05-20T16:11:46.397569+0000","last_unstale":"2024-05-20T16:12:46.258066+0000","last_undegraded":"2024-05-20T16:12:46.258066+0000","last_fullsized":"2024-05-20T16:12:46.258066+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_clean_scrub_stamp":"2024-05-20T16:11:45.363714+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:32:24.329823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"93'5","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:13:15.268953+0000","last_change":"2024-05-20T16:11:47.183935+0000","last_active":"2024-05-20T16:13:15.268953+0000","last_peered":"2024-05-20T16:13:15.268953+0000","last_clean":"2024-05-20T16:13:15.268953+0000","last_became_active":"2024-05-20T16:11:47.183604+0000","last_became_peered":"2024-05-20T16:11:47.183604+0000","last_unstale":"2024-05-20T16:13:15.268953+0000","last_undegraded":"2024-05-20T16:13:15.268953+0000","last_fullsized":"2024-05-20T16:13:15.268953+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T16:59:36.924396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:12:46.254966+0000","last_change":"2024-05-20T16:11:51.441603+0000","last_active":"2024-05-20T16:12:46.254966+0000","last_peered":"2024-05-20T16:12:46.254966+0000","last_clean":"2024-05-20T16:12:46.254966+0000","last_became_active":"2024-05-20T16:11:51.441106+0000","last_became_peered":"2024-05-20T16:11:51.441106+0000","last_unstale":"2024-05-20T16:12:46.254966+0000","last_undegraded":"2024-05-20T16:12:46.254966+0000","last_fullsized":"2024-05-20T16:12:46.254966+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_clean_scrub_stamp":"2024-05-20T16:11:50.398948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:07:29.221992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"91'2","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:12:46.260477+0000","last_change":"2024-05-20T16:11:46.401617+0000","last_active":"2024-05-20T16:12:46.260477+0000","last_peered":"2024-05-20T16:12:46.260477+0000","last_clean":"2024-05-20T16:12:46.260477+0000","last_became_active":"2024-05-20T16:11:46.401295+0000","last_became_peered":"2024-05-20T16:11:46.401295+0000","last_unstale":"2024-05-20T16:12:46.260477+0000","last_undegraded":"2024-05-20T16:12:46.260477+0000","last_fullsized":"2024-05-20T16:12:46.260477+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_clean_scrub_stamp":"2024-05-20T16:11:45.363714+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:42:21.039088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"93'6","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:13:15.295162+0000","last_change":"2024-05-20T16:11:47.187157+0000","last_active":"2024-05-20T16:13:15.295162+0000","last_peered":"2024-05-20T16:13:15.295162+0000","last_clean":"2024-05-20T16:13:15.295162+0000","last_became_active":"2024-05-20T16:11:47.186765+0000","last_became_peered":"2024-05-20T16:11:47.186765+0000","last_unstale":"2024-05-20T16:13:15.295162+0000","last_undegraded":"2024-05-20T16:13:15.295162+0000","last_fullsized":"2024-05-20T16:13:15.295162+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T17:09:56.687541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:12:46.263765+0000","last_change":"2024-05-20T16:12:46.263765+0000","last_active":"2024-05-20T16:12:46.263765+0000","last_peered":"2024-05-20T16:12:46.263765+0000","last_clean":"2024-05-20T16:12:46.263765+0000","last_became_active":"2024-05-20T16:12:46.263227+0000","last_became_peered":"2024-05-20T16:12:46.263227+0000","last_unstale":"2024-05-20T16:12:46.263765+0000","last_undegraded":"2024-05-20T16:12:46.263765+0000","last_fullsized":"2024-05-20T16:12:46.263765+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_clean_scrub_stamp":"2024-05-20T16:11:50.398948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:58:15.914997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,8,5],"acting":[4,3,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"91'4","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:12:46.273546+0000","last_change":"2024-05-20T16:12:46.273546+0000","last_active":"2024-05-20T16:12:46.273546+0000","last_peered":"2024-05-20T16:12:46.273546+0000","last_clean":"2024-05-20T16:12:46.273546+0000","last_became_active":"2024-05-20T16:12:46.266687+0000","last_became_peered":"2024-05-20T16:12:46.266687+0000","last_unstale":"2024-05-20T16:12:46.273546+0000","last_undegraded":"2024-05-20T16:12:46.273546+0000","last_fullsized":"2024-05-20T16:12:46.273546+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_clean_scrub_stamp":"2024-05-20T16:11:45.363714+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T20:09:20.348722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"93'9","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:13:15.296581+0000","last_change":"2024-05-20T16:11:47.180014+0000","last_active":"2024-05-20T16:13:15.296581+0000","last_peered":"2024-05-20T16:13:15.296581+0000","last_clean":"2024-05-20T16:13:15.296581+0000","last_became_active":"2024-05-20T16:11:47.179660+0000","last_became_peered":"2024-05-20T16:11:47.179660+0000","last_unstale":"2024-05-20T16:13:15.296581+0000","last_undegraded":"2024-05-20T16:13:15.296581+0000","last_fullsized":"2024-05-20T16:13:15.296581+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:07:26.925346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:12:46.255037+0000","last_change":"2024-05-20T16:11:51.439268+0000","last_active":"2024-05-20T16:12:46.255037+0000","last_peered":"2024-05-20T16:12:46.255037+0000","last_clean":"2024-05-20T16:12:46.255037+0000","last_became_active":"2024-05-20T16:11:51.438767+0000","last_became_peered":"2024-05-20T16:11:51.438767+0000","last_unstale":"2024-05-20T16:12:46.255037+0000","last_undegraded":"2024-05-20T16:12:46.255037+0000","last_fullsized":"2024-05-20T16:12:46.255037+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_clean_scrub_stamp":"2024-05-20T16:11:50.398948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:31:22.394666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:12:46.260172+0000","last_change":"2024-05-20T16:11:46.398922+0000","last_active":"2024-05-20T16:12:46.260172+0000","last_peered":"2024-05-20T16:12:46.260172+0000","last_clean":"2024-05-20T16:12:46.260172+0000","last_became_active":"2024-05-20T16:11:46.398434+0000","last_became_peered":"2024-05-20T16:11:46.398434+0000","last_unstale":"2024-05-20T16:12:46.260172+0000","last_undegraded":"2024-05-20T16:12:46.260172+0000","last_fullsized":"2024-05-20T16:12:46.260172+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_clean_scrub_stamp":"2024-05-20T16:11:45.363714+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T17:38:12.120113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"93'15","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:13:15.292591+0000","last_change":"2024-05-20T16:12:46.332013+0000","last_active":"2024-05-20T16:13:15.292591+0000","last_peered":"2024-05-20T16:13:15.292591+0000","last_clean":"2024-05-20T16:13:15.292591+0000","last_became_active":"2024-05-20T16:12:46.331754+0000","last_became_peered":"2024-05-20T16:12:46.331754+0000","last_unstale":"2024-05-20T16:13:15.292591+0000","last_undegraded":"2024-05-20T16:13:15.292591+0000","last_fullsized":"2024-05-20T16:13:15.292591+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:01:50.678745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:12:46.263663+0000","last_change":"2024-05-20T16:11:51.435981+0000","last_active":"2024-05-20T16:12:46.263663+0000","last_peered":"2024-05-20T16:12:46.263663+0000","last_clean":"2024-05-20T16:12:46.263663+0000","last_became_active":"2024-05-20T16:11:51.435387+0000","last_became_peered":"2024-05-20T16:11:51.435387+0000","last_unstale":"2024-05-20T16:12:46.263663+0000","last_undegraded":"2024-05-20T16:12:46.263663+0000","last_fullsized":"2024-05-20T16:12:46.263663+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_clean_scrub_stamp":"2024-05-20T16:11:50.398948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T16:14:14.877384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"91'5","reported_seq":105,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:13:20.512511+0000","last_change":"2024-05-20T16:11:46.402453+0000","last_active":"2024-05-20T16:13:20.512511+0000","last_peered":"2024-05-20T16:13:20.512511+0000","last_clean":"2024-05-20T16:13:20.512511+0000","last_became_active":"2024-05-20T16:11:46.400824+0000","last_became_peered":"2024-05-20T16:11:46.400824+0000","last_unstale":"2024-05-20T16:13:20.512511+0000","last_undegraded":"2024-05-20T16:13:20.512511+0000","last_fullsized":"2024-05-20T16:13:20.512511+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_clean_scrub_stamp":"2024-05-20T16:11:45.363714+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T17:39:47.431485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":71,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"93'7","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:13:15.270553+0000","last_change":"2024-05-20T16:11:47.180885+0000","last_active":"2024-05-20T16:13:15.270553+0000","last_peered":"2024-05-20T16:13:15.270553+0000","last_clean":"2024-05-20T16:13:15.270553+0000","last_became_active":"2024-05-20T16:11:47.178680+0000","last_became_peered":"2024-05-20T16:11:47.178680+0000","last_unstale":"2024-05-20T16:13:15.270553+0000","last_undegraded":"2024-05-20T16:13:15.270553+0000","last_fullsized":"2024-05-20T16:13:15.270553+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T19:28:00.540561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:12:46.263793+0000","last_change":"2024-05-20T16:11:51.436093+0000","last_active":"2024-05-20T16:12:46.263793+0000","last_peered":"2024-05-20T16:12:46.263793+0000","last_clean":"2024-05-20T16:12:46.263793+0000","last_became_active":"2024-05-20T16:11:51.435448+0000","last_became_peered":"2024-05-20T16:11:51.435448+0000","last_unstale":"2024-05-20T16:12:46.263793+0000","last_undegraded":"2024-05-20T16:12:46.263793+0000","last_fullsized":"2024-05-20T16:12:46.263793+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_clean_scrub_stamp":"2024-05-20T16:11:50.398948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:43:52.365055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"91'3","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:12:46.258555+0000","last_change":"2024-05-20T16:11:46.401695+0000","last_active":"2024-05-20T16:12:46.258555+0000","last_peered":"2024-05-20T16:12:46.258555+0000","last_clean":"2024-05-20T16:12:46.258555+0000","last_became_active":"2024-05-20T16:11:46.401352+0000","last_became_peered":"2024-05-20T16:11:46.401352+0000","last_unstale":"2024-05-20T16:12:46.258555+0000","last_undegraded":"2024-05-20T16:12:46.258555+0000","last_fullsized":"2024-05-20T16:12:46.258555+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_clean_scrub_stamp":"2024-05-20T16:11:45.363714+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:50:58.714842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"93'8","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:13:15.283025+0000","last_change":"2024-05-20T16:11:47.185820+0000","last_active":"2024-05-20T16:13:15.283025+0000","last_peered":"2024-05-20T16:13:15.283025+0000","last_clean":"2024-05-20T16:13:15.283025+0000","last_became_active":"2024-05-20T16:11:47.185491+0000","last_became_peered":"2024-05-20T16:11:47.185491+0000","last_unstale":"2024-05-20T16:13:15.283025+0000","last_undegraded":"2024-05-20T16:13:15.283025+0000","last_fullsized":"2024-05-20T16:13:15.283025+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:23:27.065634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:12:46.263688+0000","last_change":"2024-05-20T16:11:51.432409+0000","last_active":"2024-05-20T16:12:46.263688+0000","last_peered":"2024-05-20T16:12:46.263688+0000","last_clean":"2024-05-20T16:12:46.263688+0000","last_became_active":"2024-05-20T16:11:51.431993+0000","last_became_peered":"2024-05-20T16:11:51.431993+0000","last_unstale":"2024-05-20T16:12:46.263688+0000","last_undegraded":"2024-05-20T16:12:46.263688+0000","last_fullsized":"2024-05-20T16:12:46.263688+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_clean_scrub_stamp":"2024-05-20T16:11:50.398948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:41:08.301120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"91'2","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:12:46.253999+0000","last_change":"2024-05-20T16:11:46.397358+0000","last_active":"2024-05-20T16:12:46.253999+0000","last_peered":"2024-05-20T16:12:46.253999+0000","last_clean":"2024-05-20T16:12:46.253999+0000","last_became_active":"2024-05-20T16:11:46.396989+0000","last_became_peered":"2024-05-20T16:11:46.396989+0000","last_unstale":"2024-05-20T16:12:46.253999+0000","last_undegraded":"2024-05-20T16:12:46.253999+0000","last_fullsized":"2024-05-20T16:12:46.253999+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_clean_scrub_stamp":"2024-05-20T16:11:45.363714+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:31:19.737223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"93'8","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:13:15.269954+0000","last_change":"2024-05-20T16:11:47.175117+0000","last_active":"2024-05-20T16:13:15.269954+0000","last_peered":"2024-05-20T16:13:15.269954+0000","last_clean":"2024-05-20T16:13:15.269954+0000","last_became_active":"2024-05-20T16:11:47.174885+0000","last_became_peered":"2024-05-20T16:11:47.174885+0000","last_unstale":"2024-05-20T16:13:15.269954+0000","last_undegraded":"2024-05-20T16:13:15.269954+0000","last_fullsized":"2024-05-20T16:13:15.269954+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:38:35.765034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:12:46.257454+0000","last_change":"2024-05-20T16:11:51.440650+0000","last_active":"2024-05-20T16:12:46.257454+0000","last_peered":"2024-05-20T16:12:46.257454+0000","last_clean":"2024-05-20T16:12:46.257454+0000","last_became_active":"2024-05-20T16:11:51.440208+0000","last_became_peered":"2024-05-20T16:11:51.440208+0000","last_unstale":"2024-05-20T16:12:46.257454+0000","last_undegraded":"2024-05-20T16:12:46.257454+0000","last_fullsized":"2024-05-20T16:12:46.257454+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_clean_scrub_stamp":"2024-05-20T16:11:50.398948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:02:02.925911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"91'1","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:12:46.254591+0000","last_change":"2024-05-20T16:11:46.402078+0000","last_active":"2024-05-20T16:12:46.254591+0000","last_peered":"2024-05-20T16:12:46.254591+0000","last_clean":"2024-05-20T16:12:46.254591+0000","last_became_active":"2024-05-20T16:11:46.401680+0000","last_became_peered":"2024-05-20T16:11:46.401680+0000","last_unstale":"2024-05-20T16:12:46.254591+0000","last_undegraded":"2024-05-20T16:12:46.254591+0000","last_fullsized":"2024-05-20T16:12:46.254591+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_clean_scrub_stamp":"2024-05-20T16:11:45.363714+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:32:06.118688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"93'11","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:13:15.283528+0000","last_change":"2024-05-20T16:11:47.186933+0000","last_active":"2024-05-20T16:13:15.283528+0000","last_peered":"2024-05-20T16:13:15.283528+0000","last_clean":"2024-05-20T16:13:15.283528+0000","last_became_active":"2024-05-20T16:11:47.186554+0000","last_became_peered":"2024-05-20T16:11:47.186554+0000","last_unstale":"2024-05-20T16:13:15.283528+0000","last_undegraded":"2024-05-20T16:13:15.283528+0000","last_fullsized":"2024-05-20T16:13:15.283528+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T16:24:34.634974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:12:46.260332+0000","last_change":"2024-05-20T16:11:51.442535+0000","last_active":"2024-05-20T16:12:46.260332+0000","last_peered":"2024-05-20T16:12:46.260332+0000","last_clean":"2024-05-20T16:12:46.260332+0000","last_became_active":"2024-05-20T16:11:51.442143+0000","last_became_peered":"2024-05-20T16:11:51.442143+0000","last_unstale":"2024-05-20T16:12:46.260332+0000","last_undegraded":"2024-05-20T16:12:46.260332+0000","last_fullsized":"2024-05-20T16:12:46.260332+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_clean_scrub_stamp":"2024-05-20T16:11:50.398948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T19:31:41.640330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"91'3","reported_seq":101,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:13:20.511734+0000","last_change":"2024-05-20T16:11:46.394319+0000","last_active":"2024-05-20T16:13:20.511734+0000","last_peered":"2024-05-20T16:13:20.511734+0000","last_clean":"2024-05-20T16:13:20.511734+0000","last_became_active":"2024-05-20T16:11:46.393975+0000","last_became_peered":"2024-05-20T16:11:46.393975+0000","last_unstale":"2024-05-20T16:13:20.511734+0000","last_undegraded":"2024-05-20T16:13:20.511734+0000","last_fullsized":"2024-05-20T16:13:20.511734+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_clean_scrub_stamp":"2024-05-20T16:11:45.363714+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:13:18.706944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":69,"num_read_kb":69,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"93'10","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:13:15.297081+0000","last_change":"2024-05-20T16:11:47.177382+0000","last_active":"2024-05-20T16:13:15.297081+0000","last_peered":"2024-05-20T16:13:15.297081+0000","last_clean":"2024-05-20T16:13:15.297081+0000","last_became_active":"2024-05-20T16:11:47.176949+0000","last_became_peered":"2024-05-20T16:11:47.176949+0000","last_unstale":"2024-05-20T16:13:15.297081+0000","last_undegraded":"2024-05-20T16:13:15.297081+0000","last_fullsized":"2024-05-20T16:13:15.297081+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:20:36.689937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:12:46.256517+0000","last_change":"2024-05-20T16:11:51.438622+0000","last_active":"2024-05-20T16:12:46.256517+0000","last_peered":"2024-05-20T16:12:46.256517+0000","last_clean":"2024-05-20T16:12:46.256517+0000","last_became_active":"2024-05-20T16:11:51.438191+0000","last_became_peered":"2024-05-20T16:11:51.438191+0000","last_unstale":"2024-05-20T16:12:46.256517+0000","last_undegraded":"2024-05-20T16:12:46.256517+0000","last_fullsized":"2024-05-20T16:12:46.256517+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_clean_scrub_stamp":"2024-05-20T16:11:50.398948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T16:39:56.327423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"91'2","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:12:46.260527+0000","last_change":"2024-05-20T16:11:46.399436+0000","last_active":"2024-05-20T16:12:46.260527+0000","last_peered":"2024-05-20T16:12:46.260527+0000","last_clean":"2024-05-20T16:12:46.260527+0000","last_became_active":"2024-05-20T16:11:46.398591+0000","last_became_peered":"2024-05-20T16:11:46.398591+0000","last_unstale":"2024-05-20T16:12:46.260527+0000","last_undegraded":"2024-05-20T16:12:46.260527+0000","last_fullsized":"2024-05-20T16:12:46.260527+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_clean_scrub_stamp":"2024-05-20T16:11:45.363714+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:18:26.775585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"93'9","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:13:15.275934+0000","last_change":"2024-05-20T16:12:46.261798+0000","last_active":"2024-05-20T16:13:15.275934+0000","last_peered":"2024-05-20T16:13:15.275934+0000","last_clean":"2024-05-20T16:13:15.275934+0000","last_became_active":"2024-05-20T16:12:46.261372+0000","last_became_peered":"2024-05-20T16:12:46.261372+0000","last_unstale":"2024-05-20T16:13:15.275934+0000","last_undegraded":"2024-05-20T16:13:15.275934+0000","last_fullsized":"2024-05-20T16:13:15.275934+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:42:59.073996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,5],"acting":[0,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:12:46.255212+0000","last_change":"2024-05-20T16:11:51.442169+0000","last_active":"2024-05-20T16:12:46.255212+0000","last_peered":"2024-05-20T16:12:46.255212+0000","last_clean":"2024-05-20T16:12:46.255212+0000","last_became_active":"2024-05-20T16:11:51.441114+0000","last_became_peered":"2024-05-20T16:11:51.441114+0000","last_unstale":"2024-05-20T16:12:46.255212+0000","last_undegraded":"2024-05-20T16:12:46.255212+0000","last_fullsized":"2024-05-20T16:12:46.255212+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_clean_scrub_stamp":"2024-05-20T16:11:50.398948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:01:50.833632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"91'4","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:12:46.254987+0000","last_change":"2024-05-20T16:11:46.403424+0000","last_active":"2024-05-20T16:12:46.254987+0000","last_peered":"2024-05-20T16:12:46.254987+0000","last_clean":"2024-05-20T16:12:46.254987+0000","last_became_active":"2024-05-20T16:11:46.400786+0000","last_became_peered":"2024-05-20T16:11:46.400786+0000","last_unstale":"2024-05-20T16:12:46.254987+0000","last_undegraded":"2024-05-20T16:12:46.254987+0000","last_fullsized":"2024-05-20T16:12:46.254987+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_clean_scrub_stamp":"2024-05-20T16:11:45.363714+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T16:29:33.300267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"93'11","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:13:15.295071+0000","last_change":"2024-05-20T16:11:47.176372+0000","last_active":"2024-05-20T16:13:15.295071+0000","last_peered":"2024-05-20T16:13:15.295071+0000","last_clean":"2024-05-20T16:13:15.295071+0000","last_became_active":"2024-05-20T16:11:47.175535+0000","last_became_peered":"2024-05-20T16:11:47.175535+0000","last_unstale":"2024-05-20T16:13:15.295071+0000","last_undegraded":"2024-05-20T16:13:15.295071+0000","last_fullsized":"2024-05-20T16:13:15.295071+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:40:22.293771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:12:46.256909+0000","last_change":"2024-05-20T16:11:51.442678+0000","last_active":"2024-05-20T16:12:46.256909+0000","last_peered":"2024-05-20T16:12:46.256909+0000","last_clean":"2024-05-20T16:12:46.256909+0000","last_became_active":"2024-05-20T16:11:51.442300+0000","last_became_peered":"2024-05-20T16:11:51.442300+0000","last_unstale":"2024-05-20T16:12:46.256909+0000","last_undegraded":"2024-05-20T16:12:46.256909+0000","last_fullsized":"2024-05-20T16:12:46.256909+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_clean_scrub_stamp":"2024-05-20T16:11:50.398948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:29:14.668640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"93'35","reported_seq":210,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:13:21.513409+0000","last_change":"2024-05-20T16:11:46.398550+0000","last_active":"2024-05-20T16:13:21.513409+0000","last_peered":"2024-05-20T16:13:21.513409+0000","last_clean":"2024-05-20T16:13:21.513409+0000","last_became_active":"2024-05-20T16:11:46.398216+0000","last_became_peered":"2024-05-20T16:11:46.398216+0000","last_unstale":"2024-05-20T16:13:21.513409+0000","last_undegraded":"2024-05-20T16:13:21.513409+0000","last_fullsized":"2024-05-20T16:13:21.513409+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_clean_scrub_stamp":"2024-05-20T16:11:45.363714+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:24:53.292837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62024,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":146,"num_read_kb":134,"num_write":36,"num_write_kb":78,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"93'6","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:13:15.271849+0000","last_change":"2024-05-20T16:11:47.177641+0000","last_active":"2024-05-20T16:13:15.271849+0000","last_peered":"2024-05-20T16:13:15.271849+0000","last_clean":"2024-05-20T16:13:15.271849+0000","last_became_active":"2024-05-20T16:11:47.177333+0000","last_became_peered":"2024-05-20T16:11:47.177333+0000","last_unstale":"2024-05-20T16:13:15.271849+0000","last_undegraded":"2024-05-20T16:13:15.271849+0000","last_fullsized":"2024-05-20T16:13:15.271849+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T17:42:01.674485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:12:46.255496+0000","last_change":"2024-05-20T16:11:51.445804+0000","last_active":"2024-05-20T16:12:46.255496+0000","last_peered":"2024-05-20T16:12:46.255496+0000","last_clean":"2024-05-20T16:12:46.255496+0000","last_became_active":"2024-05-20T16:11:51.445223+0000","last_became_peered":"2024-05-20T16:11:51.445223+0000","last_unstale":"2024-05-20T16:12:46.255496+0000","last_undegraded":"2024-05-20T16:12:46.255496+0000","last_fullsized":"2024-05-20T16:12:46.255496+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_clean_scrub_stamp":"2024-05-20T16:11:50.398948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:45:19.185992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"91'6","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:12:46.263537+0000","last_change":"2024-05-20T16:11:46.398349+0000","last_active":"2024-05-20T16:12:46.263537+0000","last_peered":"2024-05-20T16:12:46.263537+0000","last_clean":"2024-05-20T16:12:46.263537+0000","last_became_active":"2024-05-20T16:11:46.397878+0000","last_became_peered":"2024-05-20T16:11:46.397878+0000","last_unstale":"2024-05-20T16:12:46.263537+0000","last_undegraded":"2024-05-20T16:12:46.263537+0000","last_fullsized":"2024-05-20T16:12:46.263537+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_clean_scrub_stamp":"2024-05-20T16:11:45.363714+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:39:07.374098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"93'9","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:13:15.275775+0000","last_change":"2024-05-20T16:11:47.179322+0000","last_active":"2024-05-20T16:13:15.275775+0000","last_peered":"2024-05-20T16:13:15.275775+0000","last_clean":"2024-05-20T16:13:15.275775+0000","last_became_active":"2024-05-20T16:11:47.178897+0000","last_became_peered":"2024-05-20T16:11:47.178897+0000","last_unstale":"2024-05-20T16:13:15.275775+0000","last_undegraded":"2024-05-20T16:13:15.275775+0000","last_fullsized":"2024-05-20T16:13:15.275775+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T20:01:04.096952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:12:46.264436+0000","last_change":"2024-05-20T16:11:51.439577+0000","last_active":"2024-05-20T16:12:46.264436+0000","last_peered":"2024-05-20T16:12:46.264436+0000","last_clean":"2024-05-20T16:12:46.264436+0000","last_became_active":"2024-05-20T16:11:51.439043+0000","last_became_peered":"2024-05-20T16:11:51.439043+0000","last_unstale":"2024-05-20T16:12:46.264436+0000","last_undegraded":"2024-05-20T16:12:46.264436+0000","last_fullsized":"2024-05-20T16:12:46.264436+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_clean_scrub_stamp":"2024-05-20T16:11:50.398948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:23:18.861715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"91'3","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:12:46.255190+0000","last_change":"2024-05-20T16:11:46.402276+0000","last_active":"2024-05-20T16:12:46.255190+0000","last_peered":"2024-05-20T16:12:46.255190+0000","last_clean":"2024-05-20T16:12:46.255190+0000","last_became_active":"2024-05-20T16:11:46.401923+0000","last_became_peered":"2024-05-20T16:11:46.401923+0000","last_unstale":"2024-05-20T16:12:46.255190+0000","last_undegraded":"2024-05-20T16:12:46.255190+0000","last_fullsized":"2024-05-20T16:12:46.255190+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_clean_scrub_stamp":"2024-05-20T16:11:45.363714+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T17:57:00.320492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"93'15","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:13:15.289691+0000","last_change":"2024-05-20T16:11:47.174448+0000","last_active":"2024-05-20T16:13:15.289691+0000","last_peered":"2024-05-20T16:13:15.289691+0000","last_clean":"2024-05-20T16:13:15.289691+0000","last_became_active":"2024-05-20T16:11:47.174099+0000","last_became_peered":"2024-05-20T16:11:47.174099+0000","last_unstale":"2024-05-20T16:13:15.289691+0000","last_undegraded":"2024-05-20T16:13:15.289691+0000","last_fullsized":"2024-05-20T16:13:15.289691+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T20:53:30.392099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:12:46.257754+0000","last_change":"2024-05-20T16:11:51.446231+0000","last_active":"2024-05-20T16:12:46.257754+0000","last_peered":"2024-05-20T16:12:46.257754+0000","last_clean":"2024-05-20T16:12:46.257754+0000","last_became_active":"2024-05-20T16:11:51.445805+0000","last_became_peered":"2024-05-20T16:11:51.445805+0000","last_unstale":"2024-05-20T16:12:46.257754+0000","last_undegraded":"2024-05-20T16:12:46.257754+0000","last_fullsized":"2024-05-20T16:12:46.257754+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_clean_scrub_stamp":"2024-05-20T16:11:50.398948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:51:29.505139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"93'5","reported_seq":108,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:13:21.513440+0000","last_change":"2024-05-20T16:11:46.392677+0000","last_active":"2024-05-20T16:13:21.513440+0000","last_peered":"2024-05-20T16:13:21.513440+0000","last_clean":"2024-05-20T16:13:21.513440+0000","last_became_active":"2024-05-20T16:11:46.392315+0000","last_became_peered":"2024-05-20T16:11:46.392315+0000","last_unstale":"2024-05-20T16:13:21.513440+0000","last_undegraded":"2024-05-20T16:13:21.513440+0000","last_fullsized":"2024-05-20T16:13:21.513440+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_clean_scrub_stamp":"2024-05-20T16:11:45.363714+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T19:04:14.191754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":75,"num_read_kb":5,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"93'6","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:13:15.284565+0000","last_change":"2024-05-20T16:11:47.187104+0000","last_active":"2024-05-20T16:13:15.284565+0000","last_peered":"2024-05-20T16:13:15.284565+0000","last_clean":"2024-05-20T16:13:15.284565+0000","last_became_active":"2024-05-20T16:11:47.186375+0000","last_became_peered":"2024-05-20T16:11:47.186375+0000","last_unstale":"2024-05-20T16:13:15.284565+0000","last_undegraded":"2024-05-20T16:13:15.284565+0000","last_fullsized":"2024-05-20T16:13:15.284565+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:27:32.478899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:12:46.254578+0000","last_change":"2024-05-20T16:11:51.444872+0000","last_active":"2024-05-20T16:12:46.254578+0000","last_peered":"2024-05-20T16:12:46.254578+0000","last_clean":"2024-05-20T16:12:46.254578+0000","last_became_active":"2024-05-20T16:11:51.444472+0000","last_became_peered":"2024-05-20T16:11:51.444472+0000","last_unstale":"2024-05-20T16:12:46.254578+0000","last_undegraded":"2024-05-20T16:12:46.254578+0000","last_fullsized":"2024-05-20T16:12:46.254578+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_clean_scrub_stamp":"2024-05-20T16:11:50.398948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:37:57.117101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"91'15","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:12:46.259448+0000","last_change":"2024-05-20T16:11:46.407244+0000","last_active":"2024-05-20T16:12:46.259448+0000","last_peered":"2024-05-20T16:12:46.259448+0000","last_clean":"2024-05-20T16:12:46.259448+0000","last_became_active":"2024-05-20T16:11:46.406905+0000","last_became_peered":"2024-05-20T16:11:46.406905+0000","last_unstale":"2024-05-20T16:12:46.259448+0000","last_undegraded":"2024-05-20T16:12:46.259448+0000","last_fullsized":"2024-05-20T16:12:46.259448+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_clean_scrub_stamp":"2024-05-20T16:11:45.363714+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:45:40.497442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"93'6","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:13:15.279102+0000","last_change":"2024-05-20T16:11:47.180326+0000","last_active":"2024-05-20T16:13:15.279102+0000","last_peered":"2024-05-20T16:13:15.279102+0000","last_clean":"2024-05-20T16:13:15.279102+0000","last_became_active":"2024-05-20T16:11:47.179937+0000","last_became_peered":"2024-05-20T16:11:47.179937+0000","last_unstale":"2024-05-20T16:13:15.279102+0000","last_undegraded":"2024-05-20T16:13:15.279102+0000","last_fullsized":"2024-05-20T16:13:15.279102+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:59:59.431365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:12:46.254650+0000","last_change":"2024-05-20T16:11:51.444495+0000","last_active":"2024-05-20T16:12:46.254650+0000","last_peered":"2024-05-20T16:12:46.254650+0000","last_clean":"2024-05-20T16:12:46.254650+0000","last_became_active":"2024-05-20T16:11:51.444136+0000","last_became_peered":"2024-05-20T16:11:51.444136+0000","last_unstale":"2024-05-20T16:12:46.254650+0000","last_undegraded":"2024-05-20T16:12:46.254650+0000","last_fullsized":"2024-05-20T16:12:46.254650+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_clean_scrub_stamp":"2024-05-20T16:11:50.398948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:27:46.260340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"91'2","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:12:46.260524+0000","last_change":"2024-05-20T16:11:46.407252+0000","last_active":"2024-05-20T16:12:46.260524+0000","last_peered":"2024-05-20T16:12:46.260524+0000","last_clean":"2024-05-20T16:12:46.260524+0000","last_became_active":"2024-05-20T16:11:46.406867+0000","last_became_peered":"2024-05-20T16:11:46.406867+0000","last_unstale":"2024-05-20T16:12:46.260524+0000","last_undegraded":"2024-05-20T16:12:46.260524+0000","last_fullsized":"2024-05-20T16:12:46.260524+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_clean_scrub_stamp":"2024-05-20T16:11:45.363714+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T17:53:46.638460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"93'5","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:13:15.287269+0000","last_change":"2024-05-20T16:11:47.179046+0000","last_active":"2024-05-20T16:13:15.287269+0000","last_peered":"2024-05-20T16:13:15.287269+0000","last_clean":"2024-05-20T16:13:15.287269+0000","last_became_active":"2024-05-20T16:11:47.178722+0000","last_became_peered":"2024-05-20T16:11:47.178722+0000","last_unstale":"2024-05-20T16:13:15.287269+0000","last_undegraded":"2024-05-20T16:13:15.287269+0000","last_fullsized":"2024-05-20T16:13:15.287269+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T17:38:15.355878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:12:46.333010+0000","last_change":"2024-05-20T16:12:46.333010+0000","last_active":"2024-05-20T16:12:46.333010+0000","last_peered":"2024-05-20T16:12:46.333010+0000","last_clean":"2024-05-20T16:12:46.333010+0000","last_became_active":"2024-05-20T16:12:46.332454+0000","last_became_peered":"2024-05-20T16:12:46.332454+0000","last_unstale":"2024-05-20T16:12:46.333010+0000","last_undegraded":"2024-05-20T16:12:46.333010+0000","last_fullsized":"2024-05-20T16:12:46.333010+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_clean_scrub_stamp":"2024-05-20T16:11:50.398948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:17:46.394972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,1],"acting":[8,10,11,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:12:46.257445+0000","last_change":"2024-05-20T16:11:46.402585+0000","last_active":"2024-05-20T16:12:46.257445+0000","last_peered":"2024-05-20T16:12:46.257445+0000","last_clean":"2024-05-20T16:12:46.257445+0000","last_became_active":"2024-05-20T16:11:46.402163+0000","last_became_peered":"2024-05-20T16:11:46.402163+0000","last_unstale":"2024-05-20T16:12:46.257445+0000","last_undegraded":"2024-05-20T16:12:46.257445+0000","last_fullsized":"2024-05-20T16:12:46.257445+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_clean_scrub_stamp":"2024-05-20T16:11:45.363714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:35:26.036970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"93'4","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:13:15.279765+0000","last_change":"2024-05-20T16:11:47.183088+0000","last_active":"2024-05-20T16:13:15.279765+0000","last_peered":"2024-05-20T16:13:15.279765+0000","last_clean":"2024-05-20T16:13:15.279765+0000","last_became_active":"2024-05-20T16:11:47.182608+0000","last_became_peered":"2024-05-20T16:11:47.182608+0000","last_unstale":"2024-05-20T16:13:15.279765+0000","last_undegraded":"2024-05-20T16:13:15.279765+0000","last_fullsized":"2024-05-20T16:13:15.279765+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T20:23:37.086778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:12:46.254399+0000","last_change":"2024-05-20T16:11:51.437420+0000","last_active":"2024-05-20T16:12:46.254399+0000","last_peered":"2024-05-20T16:12:46.254399+0000","last_clean":"2024-05-20T16:12:46.254399+0000","last_became_active":"2024-05-20T16:11:51.437018+0000","last_became_peered":"2024-05-20T16:11:51.437018+0000","last_unstale":"2024-05-20T16:12:46.254399+0000","last_undegraded":"2024-05-20T16:12:46.254399+0000","last_fullsized":"2024-05-20T16:12:46.254399+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_clean_scrub_stamp":"2024-05-20T16:11:50.398948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:03:30.650894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"91'1","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:12:46.260053+0000","last_change":"2024-05-20T16:11:46.400818+0000","last_active":"2024-05-20T16:12:46.260053+0000","last_peered":"2024-05-20T16:12:46.260053+0000","last_clean":"2024-05-20T16:12:46.260053+0000","last_became_active":"2024-05-20T16:11:46.400482+0000","last_became_peered":"2024-05-20T16:11:46.400482+0000","last_unstale":"2024-05-20T16:12:46.260053+0000","last_undegraded":"2024-05-20T16:12:46.260053+0000","last_fullsized":"2024-05-20T16:12:46.260053+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_clean_scrub_stamp":"2024-05-20T16:11:45.363714+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:13:13.968284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"93'7","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:13:15.278880+0000","last_change":"2024-05-20T16:12:46.262401+0000","last_active":"2024-05-20T16:13:15.278880+0000","last_peered":"2024-05-20T16:13:15.278880+0000","last_clean":"2024-05-20T16:13:15.278880+0000","last_became_active":"2024-05-20T16:12:46.262027+0000","last_became_peered":"2024-05-20T16:12:46.262027+0000","last_unstale":"2024-05-20T16:13:15.278880+0000","last_undegraded":"2024-05-20T16:13:15.278880+0000","last_fullsized":"2024-05-20T16:13:15.278880+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T16:50:38.920937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:12:46.254575+0000","last_change":"2024-05-20T16:11:51.915176+0000","last_active":"2024-05-20T16:12:46.254575+0000","last_peered":"2024-05-20T16:12:46.254575+0000","last_clean":"2024-05-20T16:12:46.254575+0000","last_became_active":"2024-05-20T16:11:51.914766+0000","last_became_peered":"2024-05-20T16:11:51.914766+0000","last_unstale":"2024-05-20T16:12:46.254575+0000","last_undegraded":"2024-05-20T16:12:46.254575+0000","last_fullsized":"2024-05-20T16:12:46.254575+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_clean_scrub_stamp":"2024-05-20T16:11:50.398948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T16:26:02.276646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"93'8","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:13:15.276028+0000","last_change":"2024-05-20T16:11:47.184502+0000","last_active":"2024-05-20T16:13:15.276028+0000","last_peered":"2024-05-20T16:13:15.276028+0000","last_clean":"2024-05-20T16:13:15.276028+0000","last_became_active":"2024-05-20T16:11:47.184123+0000","last_became_peered":"2024-05-20T16:11:47.184123+0000","last_unstale":"2024-05-20T16:13:15.276028+0000","last_undegraded":"2024-05-20T16:13:15.276028+0000","last_fullsized":"2024-05-20T16:13:15.276028+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:55:23.989206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"93'5","reported_seq":105,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:13:22.514301+0000","last_change":"2024-05-20T16:11:46.399731+0000","last_active":"2024-05-20T16:13:22.514301+0000","last_peered":"2024-05-20T16:13:22.514301+0000","last_clean":"2024-05-20T16:13:22.514301+0000","last_became_active":"2024-05-20T16:11:46.399349+0000","last_became_peered":"2024-05-20T16:11:46.399349+0000","last_unstale":"2024-05-20T16:13:22.514301+0000","last_undegraded":"2024-05-20T16:13:22.514301+0000","last_fullsized":"2024-05-20T16:13:22.514301+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_clean_scrub_stamp":"2024-05-20T16:11:45.363714+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T20:27:39.038116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":71,"num_read_kb":71,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"91'1","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:12:46.273802+0000","last_change":"2024-05-20T16:12:46.273802+0000","last_active":"2024-05-20T16:12:46.273802+0000","last_peered":"2024-05-20T16:12:46.273802+0000","last_clean":"2024-05-20T16:12:46.273802+0000","last_became_active":"2024-05-20T16:12:46.266005+0000","last_became_peered":"2024-05-20T16:12:46.266005+0000","last_unstale":"2024-05-20T16:12:46.273802+0000","last_undegraded":"2024-05-20T16:12:46.273802+0000","last_fullsized":"2024-05-20T16:12:46.273802+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_clean_scrub_stamp":"2024-05-20T16:11:50.398948+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:17:16.692728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,10],"acting":[7,5,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"93'8","reported_seq":19,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:13:15.291934+0000","last_change":"2024-05-20T16:12:46.332691+0000","last_active":"2024-05-20T16:13:15.291934+0000","last_peered":"2024-05-20T16:13:15.291934+0000","last_clean":"2024-05-20T16:13:15.291934+0000","last_became_active":"2024-05-20T16:12:46.332456+0000","last_became_peered":"2024-05-20T16:12:46.332456+0000","last_unstale":"2024-05-20T16:13:15.291934+0000","last_undegraded":"2024-05-20T16:13:15.291934+0000","last_fullsized":"2024-05-20T16:13:15.291934+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:15:09.762973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,1],"acting":[9,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.3","version":"91'3","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:12:46.255067+0000","last_change":"2024-05-20T16:11:46.393867+0000","last_active":"2024-05-20T16:12:46.255067+0000","last_peered":"2024-05-20T16:12:46.255067+0000","last_clean":"2024-05-20T16:12:46.255067+0000","last_became_active":"2024-05-20T16:11:46.393523+0000","last_became_peered":"2024-05-20T16:11:46.393523+0000","last_unstale":"2024-05-20T16:12:46.255067+0000","last_undegraded":"2024-05-20T16:12:46.255067+0000","last_fullsized":"2024-05-20T16:12:46.255067+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_clean_scrub_stamp":"2024-05-20T16:11:45.363714+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T18:36:40.067975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":353,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:12:46.259417+0000","last_change":"2024-05-20T16:10:35.566958+0000","last_active":"2024-05-20T16:12:46.259417+0000","last_peered":"2024-05-20T16:12:46.259417+0000","last_clean":"2024-05-20T16:12:46.259417+0000","last_became_active":"2024-05-20T16:10:35.566551+0000","last_became_peered":"2024-05-20T16:10:35.566551+0000","last_unstale":"2024-05-20T16:12:46.259417+0000","last_undegraded":"2024-05-20T16:12:46.259417+0000","last_fullsized":"2024-05-20T16:12:46.259417+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:07:09.543878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:07:09.543878+0000","last_clean_scrub_stamp":"2024-05-20T16:07:09.543878+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:48:26.484769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:12:46.255451+0000","last_change":"2024-05-20T16:11:51.915534+0000","last_active":"2024-05-20T16:12:46.255451+0000","last_peered":"2024-05-20T16:12:46.255451+0000","last_clean":"2024-05-20T16:12:46.255451+0000","last_became_active":"2024-05-20T16:11:51.914903+0000","last_became_peered":"2024-05-20T16:11:51.914903+0000","last_unstale":"2024-05-20T16:12:46.255451+0000","last_undegraded":"2024-05-20T16:12:46.255451+0000","last_fullsized":"2024-05-20T16:12:46.255451+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_clean_scrub_stamp":"2024-05-20T16:11:50.398948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T18:56:47.255129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"93'5","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:13:15.284769+0000","last_change":"2024-05-20T16:11:47.174101+0000","last_active":"2024-05-20T16:13:15.284769+0000","last_peered":"2024-05-20T16:13:15.284769+0000","last_clean":"2024-05-20T16:13:15.284769+0000","last_became_active":"2024-05-20T16:11:47.173772+0000","last_became_peered":"2024-05-20T16:11:47.173772+0000","last_unstale":"2024-05-20T16:13:15.284769+0000","last_undegraded":"2024-05-20T16:13:15.284769+0000","last_fullsized":"2024-05-20T16:13:15.284769+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:33:33.656757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"91'3","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:12:46.259500+0000","last_change":"2024-05-20T16:11:46.401951+0000","last_active":"2024-05-20T16:12:46.259500+0000","last_peered":"2024-05-20T16:12:46.259500+0000","last_clean":"2024-05-20T16:12:46.259500+0000","last_became_active":"2024-05-20T16:11:46.401377+0000","last_became_peered":"2024-05-20T16:11:46.401377+0000","last_unstale":"2024-05-20T16:12:46.259500+0000","last_undegraded":"2024-05-20T16:12:46.259500+0000","last_fullsized":"2024-05-20T16:12:46.259500+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_clean_scrub_stamp":"2024-05-20T16:11:45.363714+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:00:43.538206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:12:46.263417+0000","last_change":"2024-05-20T16:11:51.436284+0000","last_active":"2024-05-20T16:12:46.263417+0000","last_peered":"2024-05-20T16:12:46.263417+0000","last_clean":"2024-05-20T16:12:46.263417+0000","last_became_active":"2024-05-20T16:11:51.435755+0000","last_became_peered":"2024-05-20T16:11:51.435755+0000","last_unstale":"2024-05-20T16:12:46.263417+0000","last_undegraded":"2024-05-20T16:12:46.263417+0000","last_fullsized":"2024-05-20T16:12:46.263417+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_clean_scrub_stamp":"2024-05-20T16:11:50.398948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T20:51:05.434690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"91'1","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:12:46.260188+0000","last_change":"2024-05-20T16:11:46.408055+0000","last_active":"2024-05-20T16:12:46.260188+0000","last_peered":"2024-05-20T16:12:46.260188+0000","last_clean":"2024-05-20T16:12:46.260188+0000","last_became_active":"2024-05-20T16:11:46.407733+0000","last_became_peered":"2024-05-20T16:11:46.407733+0000","last_unstale":"2024-05-20T16:12:46.260188+0000","last_undegraded":"2024-05-20T16:12:46.260188+0000","last_fullsized":"2024-05-20T16:12:46.260188+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_clean_scrub_stamp":"2024-05-20T16:11:45.363714+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T18:32:32.344334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"93'9","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:13:15.265986+0000","last_change":"2024-05-20T16:11:47.179831+0000","last_active":"2024-05-20T16:13:15.265986+0000","last_peered":"2024-05-20T16:13:15.265986+0000","last_clean":"2024-05-20T16:13:15.265986+0000","last_became_active":"2024-05-20T16:11:47.179501+0000","last_became_peered":"2024-05-20T16:11:47.179501+0000","last_unstale":"2024-05-20T16:13:15.265986+0000","last_undegraded":"2024-05-20T16:13:15.265986+0000","last_fullsized":"2024-05-20T16:13:15.265986+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T17:38:13.651180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:12:46.260876+0000","last_change":"2024-05-20T16:11:51.915210+0000","last_active":"2024-05-20T16:12:46.260876+0000","last_peered":"2024-05-20T16:12:46.260876+0000","last_clean":"2024-05-20T16:12:46.260876+0000","last_became_active":"2024-05-20T16:11:51.914751+0000","last_became_peered":"2024-05-20T16:11:51.914751+0000","last_unstale":"2024-05-20T16:12:46.260876+0000","last_undegraded":"2024-05-20T16:12:46.260876+0000","last_fullsized":"2024-05-20T16:12:46.260876+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_clean_scrub_stamp":"2024-05-20T16:11:50.398948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:07:14.817497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"93'9","reported_seq":50,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:12:46.271268+0000","last_change":"2024-05-20T16:12:46.270755+0000","last_active":"2024-05-20T16:12:46.271268+0000","last_peered":"2024-05-20T16:12:46.271268+0000","last_clean":"2024-05-20T16:12:46.271268+0000","last_became_active":"2024-05-20T16:12:46.266551+0000","last_became_peered":"2024-05-20T16:12:46.266551+0000","last_unstale":"2024-05-20T16:12:46.271268+0000","last_undegraded":"2024-05-20T16:12:46.271268+0000","last_fullsized":"2024-05-20T16:12:46.271268+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_clean_scrub_stamp":"2024-05-20T16:11:45.363714+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:44:15.685183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"93'5","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:13:15.279643+0000","last_change":"2024-05-20T16:11:47.184550+0000","last_active":"2024-05-20T16:13:15.279643+0000","last_peered":"2024-05-20T16:13:15.279643+0000","last_clean":"2024-05-20T16:13:15.279643+0000","last_became_active":"2024-05-20T16:11:47.184260+0000","last_became_peered":"2024-05-20T16:11:47.184260+0000","last_unstale":"2024-05-20T16:13:15.279643+0000","last_undegraded":"2024-05-20T16:13:15.279643+0000","last_fullsized":"2024-05-20T16:13:15.279643+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T17:21:43.013563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:12:46.264163+0000","last_change":"2024-05-20T16:11:51.431710+0000","last_active":"2024-05-20T16:12:46.264163+0000","last_peered":"2024-05-20T16:12:46.264163+0000","last_clean":"2024-05-20T16:12:46.264163+0000","last_became_active":"2024-05-20T16:11:51.431286+0000","last_became_peered":"2024-05-20T16:11:51.431286+0000","last_unstale":"2024-05-20T16:12:46.264163+0000","last_undegraded":"2024-05-20T16:12:46.264163+0000","last_fullsized":"2024-05-20T16:12:46.264163+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_clean_scrub_stamp":"2024-05-20T16:11:50.398948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:35:19.016412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"91'4","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:12:46.255791+0000","last_change":"2024-05-20T16:11:46.401772+0000","last_active":"2024-05-20T16:12:46.255791+0000","last_peered":"2024-05-20T16:12:46.255791+0000","last_clean":"2024-05-20T16:12:46.255791+0000","last_became_active":"2024-05-20T16:11:46.401414+0000","last_became_peered":"2024-05-20T16:11:46.401414+0000","last_unstale":"2024-05-20T16:12:46.255791+0000","last_undegraded":"2024-05-20T16:12:46.255791+0000","last_fullsized":"2024-05-20T16:12:46.255791+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_clean_scrub_stamp":"2024-05-20T16:11:45.363714+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:23:59.146573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"93'7","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:13:15.287336+0000","last_change":"2024-05-20T16:11:47.177333+0000","last_active":"2024-05-20T16:13:15.287336+0000","last_peered":"2024-05-20T16:13:15.287336+0000","last_clean":"2024-05-20T16:13:15.287336+0000","last_became_active":"2024-05-20T16:11:47.176161+0000","last_became_peered":"2024-05-20T16:11:47.176161+0000","last_unstale":"2024-05-20T16:13:15.287336+0000","last_undegraded":"2024-05-20T16:13:15.287336+0000","last_fullsized":"2024-05-20T16:13:15.287336+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:48:40.846477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:12:46.262127+0000","last_change":"2024-05-20T16:12:46.262127+0000","last_active":"2024-05-20T16:12:46.262127+0000","last_peered":"2024-05-20T16:12:46.262127+0000","last_clean":"2024-05-20T16:12:46.262127+0000","last_became_active":"2024-05-20T16:12:46.261596+0000","last_became_peered":"2024-05-20T16:12:46.261596+0000","last_unstale":"2024-05-20T16:12:46.262127+0000","last_undegraded":"2024-05-20T16:12:46.262127+0000","last_fullsized":"2024-05-20T16:12:46.262127+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_clean_scrub_stamp":"2024-05-20T16:11:50.398948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T17:27:26.605849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,10,9],"acting":[4,8,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"91'2","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:12:46.331186+0000","last_change":"2024-05-20T16:11:46.407127+0000","last_active":"2024-05-20T16:12:46.331186+0000","last_peered":"2024-05-20T16:12:46.331186+0000","last_clean":"2024-05-20T16:12:46.331186+0000","last_became_active":"2024-05-20T16:11:46.406833+0000","last_became_peered":"2024-05-20T16:11:46.406833+0000","last_unstale":"2024-05-20T16:12:46.331186+0000","last_undegraded":"2024-05-20T16:12:46.331186+0000","last_fullsized":"2024-05-20T16:12:46.331186+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_clean_scrub_stamp":"2024-05-20T16:11:45.363714+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:58:30.464017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"93'2","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:13:15.239238+0000","last_change":"2024-05-20T16:11:47.174434+0000","last_active":"2024-05-20T16:13:15.239238+0000","last_peered":"2024-05-20T16:13:15.239238+0000","last_clean":"2024-05-20T16:13:15.239238+0000","last_became_active":"2024-05-20T16:11:47.174019+0000","last_became_peered":"2024-05-20T16:11:47.174019+0000","last_unstale":"2024-05-20T16:13:15.239238+0000","last_undegraded":"2024-05-20T16:13:15.239238+0000","last_fullsized":"2024-05-20T16:13:15.239238+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T18:32:51.150925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:12:46.257949+0000","last_change":"2024-05-20T16:11:51.446241+0000","last_active":"2024-05-20T16:12:46.257949+0000","last_peered":"2024-05-20T16:12:46.257949+0000","last_clean":"2024-05-20T16:12:46.257949+0000","last_became_active":"2024-05-20T16:11:51.445828+0000","last_became_peered":"2024-05-20T16:11:51.445828+0000","last_unstale":"2024-05-20T16:12:46.257949+0000","last_undegraded":"2024-05-20T16:12:46.257949+0000","last_fullsized":"2024-05-20T16:12:46.257949+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_clean_scrub_stamp":"2024-05-20T16:11:50.398948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:26:00.239338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"91'3","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:12:46.260360+0000","last_change":"2024-05-20T16:11:46.396364+0000","last_active":"2024-05-20T16:12:46.260360+0000","last_peered":"2024-05-20T16:12:46.260360+0000","last_clean":"2024-05-20T16:12:46.260360+0000","last_became_active":"2024-05-20T16:11:46.396014+0000","last_became_peered":"2024-05-20T16:11:46.396014+0000","last_unstale":"2024-05-20T16:12:46.260360+0000","last_undegraded":"2024-05-20T16:12:46.260360+0000","last_fullsized":"2024-05-20T16:12:46.260360+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_clean_scrub_stamp":"2024-05-20T16:11:45.363714+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:02:46.660077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"93'9","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:13:15.277346+0000","last_change":"2024-05-20T16:11:47.184320+0000","last_active":"2024-05-20T16:13:15.277346+0000","last_peered":"2024-05-20T16:13:15.277346+0000","last_clean":"2024-05-20T16:13:15.277346+0000","last_became_active":"2024-05-20T16:11:47.183981+0000","last_became_peered":"2024-05-20T16:11:47.183981+0000","last_unstale":"2024-05-20T16:13:15.277346+0000","last_undegraded":"2024-05-20T16:13:15.277346+0000","last_fullsized":"2024-05-20T16:13:15.277346+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T17:41:47.160563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"91'1","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:12:46.255197+0000","last_change":"2024-05-20T16:11:51.440073+0000","last_active":"2024-05-20T16:12:46.255197+0000","last_peered":"2024-05-20T16:12:46.255197+0000","last_clean":"2024-05-20T16:12:46.255197+0000","last_became_active":"2024-05-20T16:11:51.439560+0000","last_became_peered":"2024-05-20T16:11:51.439560+0000","last_unstale":"2024-05-20T16:12:46.255197+0000","last_undegraded":"2024-05-20T16:12:46.255197+0000","last_fullsized":"2024-05-20T16:12:46.255197+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_clean_scrub_stamp":"2024-05-20T16:11:50.398948+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:50:54.429811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"91'1","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:12:46.264171+0000","last_change":"2024-05-20T16:11:46.404866+0000","last_active":"2024-05-20T16:12:46.264171+0000","last_peered":"2024-05-20T16:12:46.264171+0000","last_clean":"2024-05-20T16:12:46.264171+0000","last_became_active":"2024-05-20T16:11:46.404509+0000","last_became_peered":"2024-05-20T16:11:46.404509+0000","last_unstale":"2024-05-20T16:12:46.264171+0000","last_undegraded":"2024-05-20T16:12:46.264171+0000","last_fullsized":"2024-05-20T16:12:46.264171+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_clean_scrub_stamp":"2024-05-20T16:11:45.363714+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:58:40.303826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"93'8","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:13:15.273412+0000","last_change":"2024-05-20T16:11:47.191505+0000","last_active":"2024-05-20T16:13:15.273412+0000","last_peered":"2024-05-20T16:13:15.273412+0000","last_clean":"2024-05-20T16:13:15.273412+0000","last_became_active":"2024-05-20T16:11:47.191181+0000","last_became_peered":"2024-05-20T16:11:47.191181+0000","last_unstale":"2024-05-20T16:13:15.273412+0000","last_undegraded":"2024-05-20T16:13:15.273412+0000","last_fullsized":"2024-05-20T16:13:15.273412+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T19:30:42.561146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:12:46.255739+0000","last_change":"2024-05-20T16:11:51.915313+0000","last_active":"2024-05-20T16:12:46.255739+0000","last_peered":"2024-05-20T16:12:46.255739+0000","last_clean":"2024-05-20T16:12:46.255739+0000","last_became_active":"2024-05-20T16:11:51.914761+0000","last_became_peered":"2024-05-20T16:11:51.914761+0000","last_unstale":"2024-05-20T16:12:46.255739+0000","last_undegraded":"2024-05-20T16:12:46.255739+0000","last_fullsized":"2024-05-20T16:12:46.255739+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_clean_scrub_stamp":"2024-05-20T16:11:50.398948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T17:15:36.673449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"91'4","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:12:46.259598+0000","last_change":"2024-05-20T16:11:46.404513+0000","last_active":"2024-05-20T16:12:46.259598+0000","last_peered":"2024-05-20T16:12:46.259598+0000","last_clean":"2024-05-20T16:12:46.259598+0000","last_became_active":"2024-05-20T16:11:46.404163+0000","last_became_peered":"2024-05-20T16:11:46.404163+0000","last_unstale":"2024-05-20T16:12:46.259598+0000","last_undegraded":"2024-05-20T16:12:46.259598+0000","last_fullsized":"2024-05-20T16:12:46.259598+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_clean_scrub_stamp":"2024-05-20T16:11:45.363714+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:24:07.155558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"93'5","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:13:15.295027+0000","last_change":"2024-05-20T16:11:47.174527+0000","last_active":"2024-05-20T16:13:15.295027+0000","last_peered":"2024-05-20T16:13:15.295027+0000","last_clean":"2024-05-20T16:13:15.295027+0000","last_became_active":"2024-05-20T16:11:47.174165+0000","last_became_peered":"2024-05-20T16:11:47.174165+0000","last_unstale":"2024-05-20T16:13:15.295027+0000","last_undegraded":"2024-05-20T16:13:15.295027+0000","last_fullsized":"2024-05-20T16:13:15.295027+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T17:44:47.614704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:12:46.331450+0000","last_change":"2024-05-20T16:11:51.439013+0000","last_active":"2024-05-20T16:12:46.331450+0000","last_peered":"2024-05-20T16:12:46.331450+0000","last_clean":"2024-05-20T16:12:46.331450+0000","last_became_active":"2024-05-20T16:11:51.438408+0000","last_became_peered":"2024-05-20T16:11:51.438408+0000","last_unstale":"2024-05-20T16:12:46.331450+0000","last_undegraded":"2024-05-20T16:12:46.331450+0000","last_fullsized":"2024-05-20T16:12:46.331450+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_clean_scrub_stamp":"2024-05-20T16:11:50.398948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T19:27:53.723823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"91'3","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:12:46.263985+0000","last_change":"2024-05-20T16:11:46.407283+0000","last_active":"2024-05-20T16:12:46.263985+0000","last_peered":"2024-05-20T16:12:46.263985+0000","last_clean":"2024-05-20T16:12:46.263985+0000","last_became_active":"2024-05-20T16:11:46.406919+0000","last_became_peered":"2024-05-20T16:11:46.406919+0000","last_unstale":"2024-05-20T16:12:46.263985+0000","last_undegraded":"2024-05-20T16:12:46.263985+0000","last_fullsized":"2024-05-20T16:12:46.263985+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_clean_scrub_stamp":"2024-05-20T16:11:45.363714+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:16:09.166422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"93'7","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:13:15.295515+0000","last_change":"2024-05-20T16:11:47.177973+0000","last_active":"2024-05-20T16:13:15.295515+0000","last_peered":"2024-05-20T16:13:15.295515+0000","last_clean":"2024-05-20T16:13:15.295515+0000","last_became_active":"2024-05-20T16:11:47.177658+0000","last_became_peered":"2024-05-20T16:11:47.177658+0000","last_unstale":"2024-05-20T16:13:15.295515+0000","last_undegraded":"2024-05-20T16:13:15.295515+0000","last_fullsized":"2024-05-20T16:13:15.295515+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:34:04.004730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:12:46.257507+0000","last_change":"2024-05-20T16:11:51.441326+0000","last_active":"2024-05-20T16:12:46.257507+0000","last_peered":"2024-05-20T16:12:46.257507+0000","last_clean":"2024-05-20T16:12:46.257507+0000","last_became_active":"2024-05-20T16:11:51.440903+0000","last_became_peered":"2024-05-20T16:11:51.440903+0000","last_unstale":"2024-05-20T16:12:46.257507+0000","last_undegraded":"2024-05-20T16:12:46.257507+0000","last_fullsized":"2024-05-20T16:12:46.257507+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_clean_scrub_stamp":"2024-05-20T16:11:50.398948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:52:54.988916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"85'1","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:12:46.255364+0000","last_change":"2024-05-20T16:11:46.397548+0000","last_active":"2024-05-20T16:12:46.255364+0000","last_peered":"2024-05-20T16:12:46.255364+0000","last_clean":"2024-05-20T16:12:46.255364+0000","last_became_active":"2024-05-20T16:11:46.397207+0000","last_became_peered":"2024-05-20T16:11:46.397207+0000","last_unstale":"2024-05-20T16:12:46.255364+0000","last_undegraded":"2024-05-20T16:12:46.255364+0000","last_fullsized":"2024-05-20T16:12:46.255364+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_clean_scrub_stamp":"2024-05-20T16:11:45.363714+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:25:50.468298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"93'6","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:13:15.285778+0000","last_change":"2024-05-20T16:11:47.184162+0000","last_active":"2024-05-20T16:13:15.285778+0000","last_peered":"2024-05-20T16:13:15.285778+0000","last_clean":"2024-05-20T16:13:15.285778+0000","last_became_active":"2024-05-20T16:11:47.183845+0000","last_became_peered":"2024-05-20T16:11:47.183845+0000","last_unstale":"2024-05-20T16:13:15.285778+0000","last_undegraded":"2024-05-20T16:13:15.285778+0000","last_fullsized":"2024-05-20T16:13:15.285778+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T20:23:14.728178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:12:46.254464+0000","last_change":"2024-05-20T16:11:51.437974+0000","last_active":"2024-05-20T16:12:46.254464+0000","last_peered":"2024-05-20T16:12:46.254464+0000","last_clean":"2024-05-20T16:12:46.254464+0000","last_became_active":"2024-05-20T16:11:51.437053+0000","last_became_peered":"2024-05-20T16:11:51.437053+0000","last_unstale":"2024-05-20T16:12:46.254464+0000","last_undegraded":"2024-05-20T16:12:46.254464+0000","last_fullsized":"2024-05-20T16:12:46.254464+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:50.398948+0000","last_clean_scrub_stamp":"2024-05-20T16:11:50.398948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:09:31.314668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:12:46.254601+0000","last_change":"2024-05-20T16:11:46.398461+0000","last_active":"2024-05-20T16:12:46.254601+0000","last_peered":"2024-05-20T16:12:46.254601+0000","last_clean":"2024-05-20T16:12:46.254601+0000","last_became_active":"2024-05-20T16:11:46.398122+0000","last_became_peered":"2024-05-20T16:11:46.398122+0000","last_unstale":"2024-05-20T16:12:46.254601+0000","last_undegraded":"2024-05-20T16:12:46.254601+0000","last_fullsized":"2024-05-20T16:12:46.254601+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:45.363714+0000","last_clean_scrub_stamp":"2024-05-20T16:11:45.363714+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:34:45.592470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"93'9","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:13:15.281805+0000","last_change":"2024-05-20T16:11:47.179160+0000","last_active":"2024-05-20T16:13:15.281805+0000","last_peered":"2024-05-20T16:13:15.281805+0000","last_clean":"2024-05-20T16:13:15.281805+0000","last_became_active":"2024-05-20T16:11:47.178843+0000","last_became_peered":"2024-05-20T16:11:47.178843+0000","last_unstale":"2024-05-20T16:13:15.281805+0000","last_undegraded":"2024-05-20T16:13:15.281805+0000","last_fullsized":"2024-05-20T16:13:15.281805+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T20:05:57.013207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"93'9","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:13:15.277352+0000","last_change":"2024-05-20T16:11:47.185401+0000","last_active":"2024-05-20T16:13:15.277352+0000","last_peered":"2024-05-20T16:13:15.277352+0000","last_clean":"2024-05-20T16:13:15.277352+0000","last_became_active":"2024-05-20T16:11:47.185061+0000","last_became_peered":"2024-05-20T16:11:47.185061+0000","last_unstale":"2024-05-20T16:13:15.277352+0000","last_undegraded":"2024-05-20T16:13:15.277352+0000","last_fullsized":"2024-05-20T16:13:15.277352+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:33:32.528968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"93'12","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:13:15.286641+0000","last_change":"2024-05-20T16:11:47.174418+0000","last_active":"2024-05-20T16:13:15.286641+0000","last_peered":"2024-05-20T16:13:15.286641+0000","last_clean":"2024-05-20T16:13:15.286641+0000","last_became_active":"2024-05-20T16:11:47.174078+0000","last_became_peered":"2024-05-20T16:11:47.174078+0000","last_unstale":"2024-05-20T16:13:15.286641+0000","last_undegraded":"2024-05-20T16:13:15.286641+0000","last_fullsized":"2024-05-20T16:13:15.286641+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T16:20:12.268161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"93'8","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:13:15.265829+0000","last_change":"2024-05-20T16:11:47.177686+0000","last_active":"2024-05-20T16:13:15.265829+0000","last_peered":"2024-05-20T16:13:15.265829+0000","last_clean":"2024-05-20T16:13:15.265829+0000","last_became_active":"2024-05-20T16:11:47.177305+0000","last_became_peered":"2024-05-20T16:11:47.177305+0000","last_unstale":"2024-05-20T16:13:15.265829+0000","last_undegraded":"2024-05-20T16:13:15.265829+0000","last_fullsized":"2024-05-20T16:13:15.265829+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T18:36:35.919491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"93'7","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:13:15.287222+0000","last_change":"2024-05-20T16:11:47.175383+0000","last_active":"2024-05-20T16:13:15.287222+0000","last_peered":"2024-05-20T16:13:15.287222+0000","last_clean":"2024-05-20T16:13:15.287222+0000","last_became_active":"2024-05-20T16:11:47.175079+0000","last_became_peered":"2024-05-20T16:11:47.175079+0000","last_unstale":"2024-05-20T16:13:15.287222+0000","last_undegraded":"2024-05-20T16:13:15.287222+0000","last_fullsized":"2024-05-20T16:13:15.287222+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:33:52.072038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"93'4","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:13:15.286325+0000","last_change":"2024-05-20T16:11:47.182717+0000","last_active":"2024-05-20T16:13:15.286325+0000","last_peered":"2024-05-20T16:13:15.286325+0000","last_clean":"2024-05-20T16:13:15.286325+0000","last_became_active":"2024-05-20T16:11:47.182408+0000","last_became_peered":"2024-05-20T16:11:47.182408+0000","last_unstale":"2024-05-20T16:13:15.286325+0000","last_undegraded":"2024-05-20T16:13:15.286325+0000","last_fullsized":"2024-05-20T16:13:15.286325+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T18:25:13.698066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"93'6","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:13:15.276882+0000","last_change":"2024-05-20T16:11:47.182804+0000","last_active":"2024-05-20T16:13:15.276882+0000","last_peered":"2024-05-20T16:13:15.276882+0000","last_clean":"2024-05-20T16:13:15.276882+0000","last_became_active":"2024-05-20T16:11:47.182486+0000","last_became_peered":"2024-05-20T16:11:47.182486+0000","last_unstale":"2024-05-20T16:13:15.276882+0000","last_undegraded":"2024-05-20T16:13:15.276882+0000","last_fullsized":"2024-05-20T16:13:15.276882+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:24:52.592522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"93'15","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:13:15.292377+0000","last_change":"2024-05-20T16:11:47.175689+0000","last_active":"2024-05-20T16:13:15.292377+0000","last_peered":"2024-05-20T16:13:15.292377+0000","last_clean":"2024-05-20T16:13:15.292377+0000","last_became_active":"2024-05-20T16:11:47.168216+0000","last_became_peered":"2024-05-20T16:11:47.168216+0000","last_unstale":"2024-05-20T16:13:15.292377+0000","last_undegraded":"2024-05-20T16:13:15.292377+0000","last_fullsized":"2024-05-20T16:13:15.292377+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:50:52.707138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"93'2","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:13:15.261590+0000","last_change":"2024-05-20T16:11:47.191398+0000","last_active":"2024-05-20T16:13:15.261590+0000","last_peered":"2024-05-20T16:13:15.261590+0000","last_clean":"2024-05-20T16:13:15.261590+0000","last_became_active":"2024-05-20T16:11:47.191036+0000","last_became_peered":"2024-05-20T16:11:47.191036+0000","last_unstale":"2024-05-20T16:13:15.261590+0000","last_undegraded":"2024-05-20T16:13:15.261590+0000","last_fullsized":"2024-05-20T16:13:15.261590+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:27:49.023581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"93'6","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:13:15.274825+0000","last_change":"2024-05-20T16:11:47.184206+0000","last_active":"2024-05-20T16:13:15.274825+0000","last_peered":"2024-05-20T16:13:15.274825+0000","last_clean":"2024-05-20T16:13:15.274825+0000","last_became_active":"2024-05-20T16:11:47.183860+0000","last_became_peered":"2024-05-20T16:11:47.183860+0000","last_unstale":"2024-05-20T16:13:15.274825+0000","last_undegraded":"2024-05-20T16:13:15.274825+0000","last_fullsized":"2024-05-20T16:13:15.274825+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:52:42.507615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"93'8","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:13:15.270571+0000","last_change":"2024-05-20T16:11:47.180298+0000","last_active":"2024-05-20T16:13:15.270571+0000","last_peered":"2024-05-20T16:13:15.270571+0000","last_clean":"2024-05-20T16:13:15.270571+0000","last_became_active":"2024-05-20T16:11:47.179996+0000","last_became_peered":"2024-05-20T16:11:47.179996+0000","last_unstale":"2024-05-20T16:13:15.270571+0000","last_undegraded":"2024-05-20T16:13:15.270571+0000","last_fullsized":"2024-05-20T16:13:15.270571+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:49:53.432240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"93'10","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:13:15.293376+0000","last_change":"2024-05-20T16:11:47.176123+0000","last_active":"2024-05-20T16:13:15.293376+0000","last_peered":"2024-05-20T16:13:15.293376+0000","last_clean":"2024-05-20T16:13:15.293376+0000","last_became_active":"2024-05-20T16:11:47.175019+0000","last_became_peered":"2024-05-20T16:11:47.175019+0000","last_unstale":"2024-05-20T16:13:15.293376+0000","last_undegraded":"2024-05-20T16:13:15.293376+0000","last_fullsized":"2024-05-20T16:13:15.293376+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:23:32.010352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"93'11","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:13:15.276663+0000","last_change":"2024-05-20T16:11:47.180877+0000","last_active":"2024-05-20T16:13:15.276663+0000","last_peered":"2024-05-20T16:13:15.276663+0000","last_clean":"2024-05-20T16:13:15.276663+0000","last_became_active":"2024-05-20T16:11:47.180567+0000","last_became_peered":"2024-05-20T16:11:47.180567+0000","last_unstale":"2024-05-20T16:13:15.276663+0000","last_undegraded":"2024-05-20T16:13:15.276663+0000","last_fullsized":"2024-05-20T16:13:15.276663+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T17:54:31.165693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"93'8","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-05-20T16:13:15.287571+0000","last_change":"2024-05-20T16:11:47.179607+0000","last_active":"2024-05-20T16:13:15.287571+0000","last_peered":"2024-05-20T16:13:15.287571+0000","last_clean":"2024-05-20T16:13:15.287571+0000","last_became_active":"2024-05-20T16:11:47.179279+0000","last_became_peered":"2024-05-20T16:11:47.179279+0000","last_unstale":"2024-05-20T16:13:15.287571+0000","last_undegraded":"2024-05-20T16:13:15.287571+0000","last_fullsized":"2024-05-20T16:13:15.287571+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T16:11:46.368993+0000","last_clean_scrub_stamp":"2024-05-20T16:11:46.368993+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T18:38:20.788325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":246552,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":66166,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":445,"num_read_kb":294,"num_write":167,"num_write_kb":119,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":17,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":405504,"data_stored":198498,"data_compressed":4830,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":145,"ondisk_log_size":145,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351008,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1015808,"data_stored":513273,"data_compressed":23570,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612645,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28148,"kb_used_data":1132,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717004,"statfs":{"total":95995035648,"available":95966212096,"internally_reserved":0,"allocated":1159168,"data_stored":1167193,"data_compressed":60870,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1598,"internal_metadata":27654594},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776172,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1032192,"data_stored":513509,"data_compressed":23570,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037808,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32184,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712968,"statfs":{"total":95995035648,"available":95962079232,"internally_reserved":0,"allocated":1032192,"data_stored":513429,"data_compressed":23570,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299446,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32232,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712920,"statfs":{"total":95995035648,"available":95962030080,"internally_reserved":0,"allocated":1081344,"data_stored":1109479,"data_compressed":59260,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462971,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32164,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712988,"statfs":{"total":95995035648,"available":95962099712,"internally_reserved":0,"allocated":1011712,"data_stored":508685,"data_compressed":23570,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574932,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28064,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717088,"statfs":{"total":95995035648,"available":95966298112,"internally_reserved":0,"allocated":1073152,"data_stored":570619,"data_compressed":25180,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738457,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28028,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717124,"statfs":{"total":95995035648,"available":95966334976,"internally_reserved":0,"allocated":1036288,"data_stored":510073,"data_compressed":23570,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411406,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28088,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717064,"statfs":{"total":95995035648,"available":95966273536,"internally_reserved":0,"allocated":1097728,"data_stored":571969,"data_compressed":25180,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149770,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1024000,"data_stored":508879,"data_compressed":23570,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888133,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1024000,"data_stored":513867,"data_compressed":23570,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724607,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28080,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717072,"statfs":{"total":95995035648,"available":95966281728,"internally_reserved":0,"allocated":1089536,"data_stored":1104535,"data_compressed":59260,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34051,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34051,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35690,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35690,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35690,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35690,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35690,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":62024,"data_compressed":1610,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":63374,"data_compressed":1610,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":64134,"data_compressed":1610,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-20T16:13:28.595 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-05-20T16:13:28.595 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-05-20T16:13:28.595 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-05-20T16:13:28.596 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell --fsid 59d1600a-16c2-11ef-bc9a-c7b262605968 -- ceph health --format=json 2024-05-20T16:13:28.635 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:13:28 smithi184 ceph-mon[36209]: from='client.15123 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-20T16:13:28.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:13:28 smithi160 ceph-mon[36347]: from='client.15123 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-20T16:13:28.835 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T16:13:29.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:13:29 smithi082 ceph-mon[32238]: pgmap v345: 129 pgs: 129 active+clean; 641 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 426 B/s wr, 8 op/s 2024-05-20T16:13:29.635 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:13:29 smithi184 ceph-mon[36209]: pgmap v345: 129 pgs: 129 active+clean; 641 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 426 B/s wr, 8 op/s 2024-05-20T16:13:29.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:13:29 smithi160 ceph-mon[36347]: pgmap v345: 129 pgs: 129 active+clean; 641 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 426 B/s wr, 8 op/s 2024-05-20T16:13:30.482 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:30.480+0000 7f2a22ea7700 1 -- 172.21.15.82:0/2480829861 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2a1c1023c0 msgr2=0x7f2a1c1027d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:30.482 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:30.480+0000 7f2a22ea7700 1 --2- 172.21.15.82:0/2480829861 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2a1c1023c0 0x7f2a1c1027d0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f2a1000b0d0 tx=0x7f2a1000b3e0 comp rx=0 tx=0).stop 2024-05-20T16:13:30.482 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:30.480+0000 7f2a22ea7700 1 -- 172.21.15.82:0/2480829861 shutdown_connections 2024-05-20T16:13:30.482 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:30.480+0000 7f2a22ea7700 1 --2- 172.21.15.82:0/2480829861 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2a1c1075f0 0x7f2a1c10ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:30.482 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:30.480+0000 7f2a22ea7700 1 --2- 172.21.15.82:0/2480829861 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2a1c1023c0 0x7f2a1c1027d0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:30.482 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:30.480+0000 7f2a22ea7700 1 --2- 172.21.15.82:0/2480829861 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2a1c106ca0 0x7f2a1c1070b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:30.482 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:30.480+0000 7f2a22ea7700 1 -- 172.21.15.82:0/2480829861 >> 172.21.15.82:0/2480829861 conn(0x7f2a1c077cd0 msgr2=0x7f2a1c078110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:13:30.482 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:30.480+0000 7f2a22ea7700 1 -- 172.21.15.82:0/2480829861 shutdown_connections 2024-05-20T16:13:30.483 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:30.480+0000 7f2a22ea7700 1 -- 172.21.15.82:0/2480829861 wait complete. 2024-05-20T16:13:30.483 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:30.481+0000 7f2a22ea7700 1 Processor -- start 2024-05-20T16:13:30.483 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:30.481+0000 7f2a22ea7700 1 -- start start 2024-05-20T16:13:30.483 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:30.482+0000 7f2a22ea7700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2a1c1023c0 0x7f2a1c19ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:30.483 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:30.482+0000 7f2a22ea7700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2a1c106ca0 0x7f2a1c19b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:30.483 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:30.482+0000 7f2a22ea7700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2a1c1075f0 0x7f2a1c19b710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:30.483 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:30.482+0000 7f2a22ea7700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f2a1c1123f0 con 0x7f2a1c106ca0 2024-05-20T16:13:30.483 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:30.482+0000 7f2a22ea7700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f2a1c112270 con 0x7f2a1c1023c0 2024-05-20T16:13:30.483 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:30.482+0000 7f2a22ea7700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f2a1c112570 con 0x7f2a1c1075f0 2024-05-20T16:13:30.483 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:30.482+0000 7f2a1bfff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2a1c106ca0 0x7f2a1c19b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:13:30.483 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:30.482+0000 7f2a1bfff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2a1c106ca0 0x7f2a1c19b1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:33286/0 (socket says 172.21.15.82:33286) 2024-05-20T16:13:30.483 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:30.482+0000 7f2a1bfff700 1 -- 172.21.15.82:0/1977303849 learned_addr learned my addr 172.21.15.82:0/1977303849 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:13:30.483 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:30.482+0000 7f2a20c43700 1 --2- 172.21.15.82:0/1977303849 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2a1c1023c0 0x7f2a1c19ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:13:30.484 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:30.482+0000 7f2a21444700 1 --2- 172.21.15.82:0/1977303849 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2a1c1075f0 0x7f2a1c19b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:13:30.484 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:30.482+0000 7f2a1bfff700 1 -- 172.21.15.82:0/1977303849 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2a1c1075f0 msgr2=0x7f2a1c19b710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:30.484 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:30.482+0000 7f2a1bfff700 1 --2- 172.21.15.82:0/1977303849 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2a1c1075f0 0x7f2a1c19b710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:30.484 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:30.482+0000 7f2a1bfff700 1 -- 172.21.15.82:0/1977303849 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2a1c1023c0 msgr2=0x7f2a1c19ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:30.484 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:30.482+0000 7f2a1bfff700 1 --2- 172.21.15.82:0/1977303849 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2a1c1023c0 0x7f2a1c19ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:30.484 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:30.482+0000 7f2a1bfff700 1 -- 172.21.15.82:0/1977303849 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a1c1a5fb0 con 0x7f2a1c106ca0 2024-05-20T16:13:30.484 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:30.482+0000 7f2a21444700 1 --2- 172.21.15.82:0/1977303849 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2a1c1075f0 0x7f2a1c19b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T16:13:30.484 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:30.482+0000 7f2a1bfff700 1 --2- 172.21.15.82:0/1977303849 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2a1c106ca0 0x7f2a1c19b1d0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f2a0800c8f0 tx=0x7f2a0800cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:13:30.485 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:30.482+0000 7f2a19ffb700 1 -- 172.21.15.82:0/1977303849 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a08007940 con 0x7f2a1c106ca0 2024-05-20T16:13:30.485 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:30.483+0000 7f2a19ffb700 1 -- 172.21.15.82:0/1977303849 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a0800f450 con 0x7f2a1c106ca0 2024-05-20T16:13:30.485 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:30.483+0000 7f2a19ffb700 1 -- 172.21.15.82:0/1977303849 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a08018660 con 0x7f2a1c106ca0 2024-05-20T16:13:30.485 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:30.483+0000 7f2a22ea7700 1 -- 172.21.15.82:0/1977303849 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a1c1a62a0 con 0x7f2a1c106ca0 2024-05-20T16:13:30.485 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:30.483+0000 7f2a22ea7700 1 -- 172.21.15.82:0/1977303849 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a1c1a6830 con 0x7f2a1c106ca0 2024-05-20T16:13:30.485 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:30.484+0000 7f2a22ea7700 1 -- 172.21.15.82:0/1977303849 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a1c050170 con 0x7f2a1c106ca0 2024-05-20T16:13:30.485 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:30.485+0000 7f2a19ffb700 1 -- 172.21.15.82:0/1977303849 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f2a0800f600 con 0x7f2a1c106ca0 2024-05-20T16:13:30.486 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:30.486+0000 7f2a19ffb700 1 --2- 172.21.15.82:0/1977303849 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f2a0c0a2ca0 0x7f2a0c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:30.486 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:30.486+0000 7f2a20c43700 1 --2- 172.21.15.82:0/1977303849 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f2a0c0a2ca0 0x7f2a0c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:13:30.486 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:30.486+0000 7f2a19ffb700 1 -- 172.21.15.82:0/1977303849 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9092+0+0 (secure 0 0 0) 0x7f2a080cf2a0 con 0x7f2a1c106ca0 2024-05-20T16:13:30.486 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:30.486+0000 7f2a20c43700 1 --2- 172.21.15.82:0/1977303849 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f2a0c0a2ca0 0x7f2a0c0a51b0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f2a10009900 tx=0x7f2a1000b040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:13:30.489 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:30.489+0000 7f2a19ffb700 1 -- 172.21.15.82:0/1977303849 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f2a08014030 con 0x7f2a1c106ca0 2024-05-20T16:13:30.729 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:30.729+0000 7f2a22ea7700 1 -- 172.21.15.82:0/1977303849 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f2a1c02dc60 con 0x7f2a1c106ca0 2024-05-20T16:13:30.729 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:30.729+0000 7f2a19ffb700 1 -- 172.21.15.82:0/1977303849 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f2a0809bfb0 con 0x7f2a1c106ca0 2024-05-20T16:13:30.729 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-05-20T16:13:30.730 INFO:teuthology.orchestra.run.smithi082.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-05-20T16:13:30.731 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:30.731+0000 7f2a22ea7700 1 -- 172.21.15.82:0/1977303849 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f2a0c0a2ca0 msgr2=0x7f2a0c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:30.731 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:30.731+0000 7f2a22ea7700 1 --2- 172.21.15.82:0/1977303849 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f2a0c0a2ca0 0x7f2a0c0a51b0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f2a10009900 tx=0x7f2a1000b040 comp rx=0 tx=0).stop 2024-05-20T16:13:30.732 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:30.731+0000 7f2a22ea7700 1 -- 172.21.15.82:0/1977303849 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2a1c106ca0 msgr2=0x7f2a1c19b1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:30.732 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:30.731+0000 7f2a22ea7700 1 --2- 172.21.15.82:0/1977303849 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2a1c106ca0 0x7f2a1c19b1d0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f2a0800c8f0 tx=0x7f2a0800cc00 comp rx=0 tx=0).stop 2024-05-20T16:13:30.732 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:30.731+0000 7f2a22ea7700 1 -- 172.21.15.82:0/1977303849 shutdown_connections 2024-05-20T16:13:30.732 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:30.731+0000 7f2a22ea7700 1 --2- 172.21.15.82:0/1977303849 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f2a0c0a2ca0 0x7f2a0c0a51b0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:30.732 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:30.731+0000 7f2a22ea7700 1 --2- 172.21.15.82:0/1977303849 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2a1c1023c0 0x7f2a1c19ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:30.732 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:30.731+0000 7f2a22ea7700 1 --2- 172.21.15.82:0/1977303849 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2a1c1075f0 0x7f2a1c19b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:30.732 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:30.731+0000 7f2a22ea7700 1 --2- 172.21.15.82:0/1977303849 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2a1c106ca0 0x7f2a1c19b1d0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:30.732 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:30.731+0000 7f2a22ea7700 1 -- 172.21.15.82:0/1977303849 >> 172.21.15.82:0/1977303849 conn(0x7f2a1c077cd0 msgr2=0x7f2a1c0ff720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:13:30.732 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:30.732+0000 7f2a22ea7700 1 -- 172.21.15.82:0/1977303849 shutdown_connections 2024-05-20T16:13:30.732 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:30.732+0000 7f2a22ea7700 1 -- 172.21.15.82:0/1977303849 wait complete. 2024-05-20T16:13:31.054 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:13:30 smithi082 ceph-mon[32238]: from='client.? 172.21.15.82:0/1977303849' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-05-20T16:13:31.062 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:13:30 smithi160 ceph-mon[36347]: from='client.? 172.21.15.82:0/1977303849' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-05-20T16:13:31.135 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:13:30 smithi184 ceph-mon[36209]: from='client.? 172.21.15.82:0/1977303849' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-05-20T16:13:31.485 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-05-20T16:13:31.485 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-20T16:13:31.603 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:31.601+0000 7f9efeee2640 1 -- 172.21.15.82:0/2773246353 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9ef81089f0 msgr2=0x7f9ef810ae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:31.604 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:31.601+0000 7f9efeee2640 1 --2- 172.21.15.82:0/2773246353 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9ef81089f0 0x7f9ef810ae10 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f9ef0009ca0 tx=0x7f9ef002a8c0 comp rx=0 tx=0).stop 2024-05-20T16:13:31.604 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:31.601+0000 7f9efeee2640 1 -- 172.21.15.82:0/2773246353 shutdown_connections 2024-05-20T16:13:31.604 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:31.601+0000 7f9efeee2640 1 --2- 172.21.15.82:0/2773246353 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9ef81089f0 0x7f9ef810ae10 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:31.604 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:31.601+0000 7f9efeee2640 1 --2- 172.21.15.82:0/2773246353 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9ef8103cf0 0x7f9ef8104100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:31.604 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:31.601+0000 7f9efeee2640 1 --2- 172.21.15.82:0/2773246353 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9ef8102ab0 0x7f9ef81037b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:31.604 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:31.601+0000 7f9efeee2640 1 -- 172.21.15.82:0/2773246353 >> 172.21.15.82:0/2773246353 conn(0x7f9ef80fdcc0 msgr2=0x7f9ef8100150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:13:31.604 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:31.601+0000 7f9efeee2640 1 -- 172.21.15.82:0/2773246353 shutdown_connections 2024-05-20T16:13:31.604 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:31.601+0000 7f9efeee2640 1 -- 172.21.15.82:0/2773246353 wait complete. 2024-05-20T16:13:31.604 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:31.602+0000 7f9efeee2640 1 Processor -- start 2024-05-20T16:13:31.604 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:31.602+0000 7f9efeee2640 1 -- start start 2024-05-20T16:13:31.604 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:31.602+0000 7f9efeee2640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9ef8102ab0 0x7f9ef81992f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:31.604 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:31.602+0000 7f9efeee2640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9ef8103cf0 0x7f9ef8199830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:31.604 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:31.602+0000 7f9efeee2640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9ef81089f0 0x7f9ef81a0770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:31.605 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:31.602+0000 7f9efeee2640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f9ef810e780 con 0x7f9ef81089f0 2024-05-20T16:13:31.605 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:31.602+0000 7f9efeee2640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f9ef810e600 con 0x7f9ef8102ab0 2024-05-20T16:13:31.605 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:31.602+0000 7f9efeee2640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f9ef810e900 con 0x7f9ef8103cf0 2024-05-20T16:13:31.605 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:31.602+0000 7f9efd47f640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9ef81089f0 0x7f9ef81a0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:13:31.605 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:31.602+0000 7f9efd47f640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9ef81089f0 0x7f9ef81a0770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:33304/0 (socket says 172.21.15.82:33304) 2024-05-20T16:13:31.605 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:31.602+0000 7f9efd47f640 1 -- 172.21.15.82:0/1087252378 learned_addr learned my addr 172.21.15.82:0/1087252378 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:13:31.605 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:31.603+0000 7f9efcc7e640 1 --2- 172.21.15.82:0/1087252378 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9ef8102ab0 0x7f9ef81992f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:13:31.605 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:31.603+0000 7f9ef7fff640 1 --2- 172.21.15.82:0/1087252378 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9ef8103cf0 0x7f9ef8199830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:13:31.605 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:31.603+0000 7f9efd47f640 1 -- 172.21.15.82:0/1087252378 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9ef8103cf0 msgr2=0x7f9ef8199830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:31.605 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:31.603+0000 7f9efd47f640 1 --2- 172.21.15.82:0/1087252378 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9ef8103cf0 0x7f9ef8199830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:31.605 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:31.603+0000 7f9efd47f640 1 -- 172.21.15.82:0/1087252378 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9ef8102ab0 msgr2=0x7f9ef81992f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:31.605 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:31.603+0000 7f9efd47f640 1 --2- 172.21.15.82:0/1087252378 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9ef8102ab0 0x7f9ef81992f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:31.605 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:31.603+0000 7f9efd47f640 1 -- 172.21.15.82:0/1087252378 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ef81a0de0 con 0x7f9ef81089f0 2024-05-20T16:13:31.605 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:31.603+0000 7f9efcc7e640 1 --2- 172.21.15.82:0/1087252378 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9ef8102ab0 0x7f9ef81992f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T16:13:31.605 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:31.603+0000 7f9efd47f640 1 --2- 172.21.15.82:0/1087252378 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9ef81089f0 0x7f9ef81a0770 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f9ef0003f40 tx=0x7f9ef0003f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:13:31.606 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:31.603+0000 7f9ef5ffb640 1 -- 172.21.15.82:0/1087252378 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ef00316d0 con 0x7f9ef81089f0 2024-05-20T16:13:31.606 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:31.603+0000 7f9efeee2640 1 -- 172.21.15.82:0/1087252378 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ef81a1070 con 0x7f9ef81089f0 2024-05-20T16:13:31.606 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:31.603+0000 7f9ef5ffb640 1 -- 172.21.15.82:0/1087252378 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ef0031870 con 0x7f9ef81089f0 2024-05-20T16:13:31.606 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:31.603+0000 7f9ef5ffb640 1 -- 172.21.15.82:0/1087252378 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ef002fbb0 con 0x7f9ef81089f0 2024-05-20T16:13:31.606 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:31.603+0000 7f9efeee2640 1 -- 172.21.15.82:0/1087252378 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ef81a15c0 con 0x7f9ef81089f0 2024-05-20T16:13:31.607 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:31.604+0000 7f9efeee2640 1 -- 172.21.15.82:0/1087252378 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ef8107800 con 0x7f9ef81089f0 2024-05-20T16:13:31.607 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:31.605+0000 7f9ef5ffb640 1 -- 172.21.15.82:0/1087252378 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f9ef002fd70 con 0x7f9ef81089f0 2024-05-20T16:13:31.607 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:31.606+0000 7f9ef5ffb640 1 --2- 172.21.15.82:0/1087252378 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f9ee80a2b90 0x7f9ee80a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:31.607 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:31.607+0000 7f9efcc7e640 1 --2- 172.21.15.82:0/1087252378 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f9ee80a2b90 0x7f9ee80a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:13:31.607 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:31.607+0000 7f9ef5ffb640 1 -- 172.21.15.82:0/1087252378 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9092+0+0 (secure 0 0 0) 0x7f9ef00f0a50 con 0x7f9ef81089f0 2024-05-20T16:13:31.607 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:31.607+0000 7f9efcc7e640 1 --2- 172.21.15.82:0/1087252378 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f9ee80a2b90 0x7f9ee80a50a0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f9ef81036b0 tx=0x7f9eec005d30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:13:31.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:31.611+0000 7f9ef5ffb640 1 -- 172.21.15.82:0/1087252378 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f9ef00bd650 con 0x7f9ef81089f0 2024-05-20T16:13:31.788 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:31.789+0000 7f9efeee2640 1 -- 172.21.15.82:0/1087252378 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9ef8062750 con 0x7f9ef81089f0 2024-05-20T16:13:31.790 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:31.790+0000 7f9ef5ffb640 1 -- 172.21.15.82:0/1087252378 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) ==== 74+0+24621 (secure 0 0 0) 0x7f9ef00bcf50 con 0x7f9ef81089f0 2024-05-20T16:13:31.790 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-05-20T16:13:31.791 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":93,"fsid":"59d1600a-16c2-11ef-bc9a-c7b262605968","created":"2024-05-20T16:03:46.611556+0000","modified":"2024-05-20T16:12:46.250119+0000","last_up_change":"2024-05-20T16:10:59.303479+0000","last_in_change":"2024-05-20T16:10:39.063561+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-20T16:07:09.493727+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-20T16:11:45.058035+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-20T16:11:45.943862+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-20T16:11:49.637579+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"bae842a9-03ec-490e-b56d-f7957235cdc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":1484330167},{"type":"v1","addr":"172.21.15.82:6803","nonce":1484330167}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":1484330167},{"type":"v1","addr":"172.21.15.82:6805","nonce":1484330167}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1484330167},{"type":"v1","addr":"172.21.15.82:6809","nonce":1484330167}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":1484330167},{"type":"v1","addr":"172.21.15.82:6807","nonce":1484330167}]},"public_addr":"172.21.15.82:6803/1484330167","cluster_addr":"172.21.15.82:6805/1484330167","heartbeat_back_addr":"172.21.15.82:6809/1484330167","heartbeat_front_addr":"172.21.15.82:6807/1484330167","state":["exists","up"]},{"osd":1,"uuid":"59f6a2ba-b5de-4161-81c5-92a417abacdf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":1457003568},{"type":"v1","addr":"172.21.15.160:6801","nonce":1457003568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":1457003568},{"type":"v1","addr":"172.21.15.160:6803","nonce":1457003568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6806","nonce":1457003568},{"type":"v1","addr":"172.21.15.160:6807","nonce":1457003568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":1457003568},{"type":"v1","addr":"172.21.15.160:6805","nonce":1457003568}]},"public_addr":"172.21.15.160:6801/1457003568","cluster_addr":"172.21.15.160:6803/1457003568","heartbeat_back_addr":"172.21.15.160:6807/1457003568","heartbeat_front_addr":"172.21.15.160:6805/1457003568","state":["exists","up"]},{"osd":2,"uuid":"3bbb5587-b449-41f3-bd26-67ec3d6afdfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":4051293934},{"type":"v1","addr":"172.21.15.184:6801","nonce":4051293934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":4051293934},{"type":"v1","addr":"172.21.15.184:6803","nonce":4051293934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":4051293934},{"type":"v1","addr":"172.21.15.184:6807","nonce":4051293934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":4051293934},{"type":"v1","addr":"172.21.15.184:6805","nonce":4051293934}]},"public_addr":"172.21.15.184:6801/4051293934","cluster_addr":"172.21.15.184:6803/4051293934","heartbeat_back_addr":"172.21.15.184:6807/4051293934","heartbeat_front_addr":"172.21.15.184:6805/4051293934","state":["exists","up"]},{"osd":3,"uuid":"ca651f90-09b8-4081-a6f9-40b31268a837","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":3947485599},{"type":"v1","addr":"172.21.15.82:6811","nonce":3947485599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":3947485599},{"type":"v1","addr":"172.21.15.82:6813","nonce":3947485599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":3947485599},{"type":"v1","addr":"172.21.15.82:6817","nonce":3947485599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":3947485599},{"type":"v1","addr":"172.21.15.82:6815","nonce":3947485599}]},"public_addr":"172.21.15.82:6811/3947485599","cluster_addr":"172.21.15.82:6813/3947485599","heartbeat_back_addr":"172.21.15.82:6817/3947485599","heartbeat_front_addr":"172.21.15.82:6815/3947485599","state":["exists","up"]},{"osd":4,"uuid":"cb3dfd56-f677-40cb-9c96-8319ea5ac788","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":4160830014},{"type":"v1","addr":"172.21.15.160:6809","nonce":4160830014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":4160830014},{"type":"v1","addr":"172.21.15.160:6811","nonce":4160830014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":4160830014},{"type":"v1","addr":"172.21.15.160:6815","nonce":4160830014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6812","nonce":4160830014},{"type":"v1","addr":"172.21.15.160:6813","nonce":4160830014}]},"public_addr":"172.21.15.160:6809/4160830014","cluster_addr":"172.21.15.160:6811/4160830014","heartbeat_back_addr":"172.21.15.160:6815/4160830014","heartbeat_front_addr":"172.21.15.160:6813/4160830014","state":["exists","up"]},{"osd":5,"uuid":"c70cfcd1-a4ff-461f-8c73-479b061200fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":1046282672},{"type":"v1","addr":"172.21.15.184:6809","nonce":1046282672}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":1046282672},{"type":"v1","addr":"172.21.15.184:6811","nonce":1046282672}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":1046282672},{"type":"v1","addr":"172.21.15.184:6815","nonce":1046282672}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":1046282672},{"type":"v1","addr":"172.21.15.184:6813","nonce":1046282672}]},"public_addr":"172.21.15.184:6809/1046282672","cluster_addr":"172.21.15.184:6811/1046282672","heartbeat_back_addr":"172.21.15.184:6815/1046282672","heartbeat_front_addr":"172.21.15.184:6813/1046282672","state":["exists","up"]},{"osd":6,"uuid":"d47ce001-9dd1-4830-bcc9-888601187c6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":2789736499},{"type":"v1","addr":"172.21.15.82:6819","nonce":2789736499}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":2789736499},{"type":"v1","addr":"172.21.15.82:6821","nonce":2789736499}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":2789736499},{"type":"v1","addr":"172.21.15.82:6825","nonce":2789736499}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":2789736499},{"type":"v1","addr":"172.21.15.82:6823","nonce":2789736499}]},"public_addr":"172.21.15.82:6819/2789736499","cluster_addr":"172.21.15.82:6821/2789736499","heartbeat_back_addr":"172.21.15.82:6825/2789736499","heartbeat_front_addr":"172.21.15.82:6823/2789736499","state":["exists","up"]},{"osd":7,"uuid":"25c54f43-18cb-41e4-83b4-2732b5a93889","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6816","nonce":2340683378},{"type":"v1","addr":"172.21.15.160:6817","nonce":2340683378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":2340683378},{"type":"v1","addr":"172.21.15.160:6819","nonce":2340683378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":2340683378},{"type":"v1","addr":"172.21.15.160:6823","nonce":2340683378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6820","nonce":2340683378},{"type":"v1","addr":"172.21.15.160:6821","nonce":2340683378}]},"public_addr":"172.21.15.160:6817/2340683378","cluster_addr":"172.21.15.160:6819/2340683378","heartbeat_back_addr":"172.21.15.160:6823/2340683378","heartbeat_front_addr":"172.21.15.160:6821/2340683378","state":["exists","up"]},{"osd":8,"uuid":"1f8b478d-287f-45f5-8526-d1a58c5bc07c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":3831576372},{"type":"v1","addr":"172.21.15.184:6817","nonce":3831576372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":3831576372},{"type":"v1","addr":"172.21.15.184:6819","nonce":3831576372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":3831576372},{"type":"v1","addr":"172.21.15.184:6823","nonce":3831576372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":3831576372},{"type":"v1","addr":"172.21.15.184:6821","nonce":3831576372}]},"public_addr":"172.21.15.184:6817/3831576372","cluster_addr":"172.21.15.184:6819/3831576372","heartbeat_back_addr":"172.21.15.184:6823/3831576372","heartbeat_front_addr":"172.21.15.184:6821/3831576372","state":["exists","up"]},{"osd":9,"uuid":"0d69e319-954f-4b18-95ea-31794f18dc3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":716330782},{"type":"v1","addr":"172.21.15.82:6827","nonce":716330782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":716330782},{"type":"v1","addr":"172.21.15.82:6829","nonce":716330782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":716330782},{"type":"v1","addr":"172.21.15.82:6833","nonce":716330782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":716330782},{"type":"v1","addr":"172.21.15.82:6831","nonce":716330782}]},"public_addr":"172.21.15.82:6827/716330782","cluster_addr":"172.21.15.82:6829/716330782","heartbeat_back_addr":"172.21.15.82:6833/716330782","heartbeat_front_addr":"172.21.15.82:6831/716330782","state":["exists","up"]},{"osd":10,"uuid":"39dc1b78-61cb-42ca-b5b6-1e44ac2b6500","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":2327428014},{"type":"v1","addr":"172.21.15.160:6825","nonce":2327428014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":2327428014},{"type":"v1","addr":"172.21.15.160:6827","nonce":2327428014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":2327428014},{"type":"v1","addr":"172.21.15.160:6831","nonce":2327428014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":2327428014},{"type":"v1","addr":"172.21.15.160:6829","nonce":2327428014}]},"public_addr":"172.21.15.160:6825/2327428014","cluster_addr":"172.21.15.160:6827/2327428014","heartbeat_back_addr":"172.21.15.160:6831/2327428014","heartbeat_front_addr":"172.21.15.160:6829/2327428014","state":["exists","up"]},{"osd":11,"uuid":"4306b3c1-707a-4010-987c-209ea0e948e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":3594364650},{"type":"v1","addr":"172.21.15.184:6825","nonce":3594364650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":3594364650},{"type":"v1","addr":"172.21.15.184:6827","nonce":3594364650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3594364650},{"type":"v1","addr":"172.21.15.184:6831","nonce":3594364650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3594364650},{"type":"v1","addr":"172.21.15.184:6829","nonce":3594364650}]},"public_addr":"172.21.15.184:6825/3594364650","cluster_addr":"172.21.15.184:6827/3594364650","heartbeat_back_addr":"172.21.15.184:6831/3594364650","heartbeat_front_addr":"172.21.15.184:6829/3594364650","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:06:17.855998+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:06:41.473270+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:07:05.304108+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:07:29.887472+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:07:55.387286+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:08:20.579669+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:08:46.333051+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:09:12.289668+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:09:38.180696+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:10:04.471459+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:10:31.041912+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:10:57.477266+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.1d","mappings":[{"from":5,"to":9}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.5","mappings":[{"from":9,"to":10}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.82:0/2005424509":"2024-05-21T16:04:45.161949+0000","172.21.15.82:6801/1763049308":"2024-05-21T16:04:45.161949+0000","172.21.15.82:6801/1959459348":"2024-05-21T16:04:19.208291+0000","172.21.15.82:0/3131904556":"2024-05-21T16:04:19.208291+0000","172.21.15.82:6800/1959459348":"2024-05-21T16:04:19.208291+0000","172.21.15.82:0/2456966342":"2024-05-21T16:04:45.161949+0000","172.21.15.82:0/4179978833":"2024-05-21T16:04:19.208291+0000","172.21.15.82:0/1602862225":"2024-05-21T16:04:45.161949+0000","172.21.15.82:0/106101180":"2024-05-21T16:04:19.208291+0000","172.21.15.82:6800/1763049308":"2024-05-21T16:04:45.161949+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-20T16:13:31.793 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:31.793+0000 7f9efeee2640 1 -- 172.21.15.82:0/1087252378 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f9ee80a2b90 msgr2=0x7f9ee80a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:31.793 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:31.793+0000 7f9efeee2640 1 --2- 172.21.15.82:0/1087252378 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f9ee80a2b90 0x7f9ee80a50a0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f9ef81036b0 tx=0x7f9eec005d30 comp rx=0 tx=0).stop 2024-05-20T16:13:31.793 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:31.793+0000 7f9efeee2640 1 -- 172.21.15.82:0/1087252378 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9ef81089f0 msgr2=0x7f9ef81a0770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:31.793 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:31.793+0000 7f9efeee2640 1 --2- 172.21.15.82:0/1087252378 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9ef81089f0 0x7f9ef81a0770 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f9ef0003f40 tx=0x7f9ef0003f70 comp rx=0 tx=0).stop 2024-05-20T16:13:31.793 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:31.793+0000 7f9efeee2640 1 -- 172.21.15.82:0/1087252378 shutdown_connections 2024-05-20T16:13:31.793 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:31.793+0000 7f9efeee2640 1 --2- 172.21.15.82:0/1087252378 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f9ee80a2b90 0x7f9ee80a50a0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:31.793 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:31.793+0000 7f9efeee2640 1 --2- 172.21.15.82:0/1087252378 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9ef81089f0 0x7f9ef81a0770 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:31.793 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:31.793+0000 7f9efeee2640 1 --2- 172.21.15.82:0/1087252378 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9ef8103cf0 0x7f9ef8199830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:31.793 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:31.793+0000 7f9efeee2640 1 --2- 172.21.15.82:0/1087252378 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9ef8102ab0 0x7f9ef81992f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:31.793 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:31.793+0000 7f9efeee2640 1 -- 172.21.15.82:0/1087252378 >> 172.21.15.82:0/1087252378 conn(0x7f9ef80fdcc0 msgr2=0x7f9ef81049d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:13:31.793 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:31.793+0000 7f9efeee2640 1 -- 172.21.15.82:0/1087252378 shutdown_connections 2024-05-20T16:13:31.793 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:31.793+0000 7f9efeee2640 1 -- 172.21.15.82:0/1087252378 wait complete. 2024-05-20T16:13:31.805 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-20T16:07:09.493727+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-20T16:11:45.058035+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-20T16:11:45.943862+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-20T16:11:49.637579+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '91', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-20T16:13:31.805 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-20T16:13:31.920 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:31.918+0000 7fecc483f640 1 -- 172.21.15.82:0/1781414244 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fecc0103420 msgr2=0x7fecc0109b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:31.920 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:31.918+0000 7fecc483f640 1 --2- 172.21.15.82:0/1781414244 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fecc0103420 0x7fecc0109b60 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fecb8009ca0 tx=0x7fecb802a8c0 comp rx=0 tx=0).stop 2024-05-20T16:13:31.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:31.918+0000 7fecc483f640 1 -- 172.21.15.82:0/1781414244 shutdown_connections 2024-05-20T16:13:31.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:31.918+0000 7fecc483f640 1 --2- 172.21.15.82:0/1781414244 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fecc0103420 0x7fecc0109b60 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:31.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:31.918+0000 7fecc483f640 1 --2- 172.21.15.82:0/1781414244 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fecc01028e0 0x7fecc0102d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:31.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:31.918+0000 7fecc483f640 1 --2- 172.21.15.82:0/1781414244 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fecc0101f00 0x7fecc0102310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:31.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:31.918+0000 7fecc483f640 1 -- 172.21.15.82:0/1781414244 >> 172.21.15.82:0/1781414244 conn(0x7fecc00fc300 msgr2=0x7fecc00fe7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:13:31.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:31.918+0000 7fecc483f640 1 -- 172.21.15.82:0/1781414244 shutdown_connections 2024-05-20T16:13:31.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:31.918+0000 7fecc483f640 1 -- 172.21.15.82:0/1781414244 wait complete. 2024-05-20T16:13:31.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:31.919+0000 7fecc483f640 1 Processor -- start 2024-05-20T16:13:31.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:31.919+0000 7fecc483f640 1 -- start start 2024-05-20T16:13:31.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:31.919+0000 7fecc483f640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fecc0101f00 0x7fecc01992d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:31.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:31.919+0000 7fecc483f640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fecc01028e0 0x7fecc0199810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:31.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:31.919+0000 7fecc483f640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fecc0103420 0x7fecc01a07a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:31.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:31.919+0000 7fecc483f640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fecc010e7f0 con 0x7fecc0101f00 2024-05-20T16:13:31.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:31.919+0000 7fecc483f640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fecc010e670 con 0x7fecc01028e0 2024-05-20T16:13:31.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:31.919+0000 7fecc483f640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fecc010e970 con 0x7fecc0103420 2024-05-20T16:13:31.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:31.919+0000 7fecbe59c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fecc0101f00 0x7fecc01992d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:13:31.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:31.919+0000 7fecbe59c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fecc0101f00 0x7fecc01992d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:33320/0 (socket says 172.21.15.82:33320) 2024-05-20T16:13:31.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:31.919+0000 7fecbe59c640 1 -- 172.21.15.82:0/524819923 learned_addr learned my addr 172.21.15.82:0/524819923 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:13:31.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:31.919+0000 7fecbdd9b640 1 --2- 172.21.15.82:0/524819923 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fecc01028e0 0x7fecc0199810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:13:31.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:31.919+0000 7fecbed9d640 1 --2- 172.21.15.82:0/524819923 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fecc0103420 0x7fecc01a07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:13:31.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:31.920+0000 7fecbe59c640 1 -- 172.21.15.82:0/524819923 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fecc0103420 msgr2=0x7fecc01a07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:31.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:31.920+0000 7fecbe59c640 1 --2- 172.21.15.82:0/524819923 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fecc0103420 0x7fecc01a07a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:31.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:31.920+0000 7fecbe59c640 1 -- 172.21.15.82:0/524819923 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fecc01028e0 msgr2=0x7fecc0199810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:31.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:31.920+0000 7fecbe59c640 1 --2- 172.21.15.82:0/524819923 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fecc01028e0 0x7fecc0199810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:31.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:31.920+0000 7fecbe59c640 1 -- 172.21.15.82:0/524819923 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fecc01a0e10 con 0x7fecc0101f00 2024-05-20T16:13:31.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:31.920+0000 7fecbe59c640 1 --2- 172.21.15.82:0/524819923 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fecc0101f00 0x7fecc01992d0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fecb000c970 tx=0x7fecb000cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:13:31.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:31.920+0000 7fecb77fe640 1 -- 172.21.15.82:0/524819923 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecb0007bb0 con 0x7fecc0101f00 2024-05-20T16:13:31.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:31.920+0000 7fecb77fe640 1 -- 172.21.15.82:0/524819923 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fecb0007d50 con 0x7fecc0101f00 2024-05-20T16:13:31.923 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:31.920+0000 7fecc483f640 1 -- 172.21.15.82:0/524819923 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fecc01a1100 con 0x7fecc0101f00 2024-05-20T16:13:31.923 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:31.920+0000 7fecb77fe640 1 -- 172.21.15.82:0/524819923 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecb000f450 con 0x7fecc0101f00 2024-05-20T16:13:31.923 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:31.920+0000 7fecc483f640 1 -- 172.21.15.82:0/524819923 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fecc01a16d0 con 0x7fecc0101f00 2024-05-20T16:13:31.923 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:31.921+0000 7fecc483f640 1 -- 172.21.15.82:0/524819923 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fecc01088b0 con 0x7fecc0101f00 2024-05-20T16:13:31.923 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:31.922+0000 7fecb77fe640 1 -- 172.21.15.82:0/524819923 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fecb0004eb0 con 0x7fecc0101f00 2024-05-20T16:13:31.924 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:31.923+0000 7fecb77fe640 1 --2- 172.21.15.82:0/524819923 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fecac0a2b90 0x7fecac0a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:31.924 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:31.923+0000 7fecbdd9b640 1 --2- 172.21.15.82:0/524819923 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fecac0a2b90 0x7fecac0a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:13:31.924 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:31.923+0000 7fecb77fe640 1 -- 172.21.15.82:0/524819923 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9092+0+0 (secure 0 0 0) 0x7fecb00cf410 con 0x7fecc0101f00 2024-05-20T16:13:31.924 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:31.924+0000 7fecbdd9b640 1 --2- 172.21.15.82:0/524819923 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fecac0a2b90 0x7fecac0a50a0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fecc019a370 tx=0x7feca8009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:13:31.926 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:31.927+0000 7fecb77fe640 1 -- 172.21.15.82:0/524819923 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fecb0014030 con 0x7fecc0101f00 2024-05-20T16:13:32.062 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:13:31 smithi160 ceph-mon[36347]: pgmap v346: 129 pgs: 129 active+clean; 641 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-20T16:13:32.103 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:32.103+0000 7fecc483f640 1 -- 172.21.15.82:0/524819923 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fecc0062750 con 0x7fecc0101f00 2024-05-20T16:13:32.104 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:32.104+0000 7fecb77fe640 1 -- 172.21.15.82:0/524819923 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v93) ==== 93+0+10 (secure 0 0 0) 0x7fecb009c010 con 0x7fecc0101f00 2024-05-20T16:13:32.104 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 1 2024-05-20T16:13:32.106 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:32.106+0000 7fecc483f640 1 -- 172.21.15.82:0/524819923 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fecac0a2b90 msgr2=0x7fecac0a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:32.106 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:32.106+0000 7fecc483f640 1 --2- 172.21.15.82:0/524819923 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fecac0a2b90 0x7fecac0a50a0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fecc019a370 tx=0x7feca8009110 comp rx=0 tx=0).stop 2024-05-20T16:13:32.106 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:32.106+0000 7fecc483f640 1 -- 172.21.15.82:0/524819923 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fecc0101f00 msgr2=0x7fecc01992d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:32.106 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:32.106+0000 7fecc483f640 1 --2- 172.21.15.82:0/524819923 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fecc0101f00 0x7fecc01992d0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fecb000c970 tx=0x7fecb000cd70 comp rx=0 tx=0).stop 2024-05-20T16:13:32.106 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:32.106+0000 7fecc483f640 1 -- 172.21.15.82:0/524819923 shutdown_connections 2024-05-20T16:13:32.106 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:32.106+0000 7fecc483f640 1 --2- 172.21.15.82:0/524819923 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fecac0a2b90 0x7fecac0a50a0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:32.106 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:32.106+0000 7fecc483f640 1 --2- 172.21.15.82:0/524819923 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fecc0103420 0x7fecc01a07a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:32.106 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:32.106+0000 7fecc483f640 1 --2- 172.21.15.82:0/524819923 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fecc01028e0 0x7fecc0199810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:32.106 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:32.106+0000 7fecc483f640 1 --2- 172.21.15.82:0/524819923 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fecc0101f00 0x7fecc01992d0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:32.106 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:32.106+0000 7fecc483f640 1 -- 172.21.15.82:0/524819923 >> 172.21.15.82:0/524819923 conn(0x7fecc00fc300 msgr2=0x7fecc0105a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:13:32.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:32.106+0000 7fecc483f640 1 -- 172.21.15.82:0/524819923 shutdown_connections 2024-05-20T16:13:32.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:32.106+0000 7fecc483f640 1 -- 172.21.15.82:0/524819923 wait complete. 2024-05-20T16:13:32.108 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:13:31 smithi082 ceph-mon[32238]: pgmap v346: 129 pgs: 129 active+clean; 641 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-20T16:13:32.117 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-20T16:13:32.136 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:13:31 smithi184 ceph-mon[36209]: pgmap v346: 129 pgs: 129 active+clean; 641 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-20T16:13:32.231 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:32.230+0000 7f29b9e08640 1 -- 172.21.15.82:0/2147984934 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f29b4102370 msgr2=0x7f29b4102780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:32.231 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:32.230+0000 7f29b9e08640 1 --2- 172.21.15.82:0/2147984934 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f29b4102370 0x7f29b4102780 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f29a8009ca0 tx=0x7f29a802a8c0 comp rx=0 tx=0).stop 2024-05-20T16:13:32.231 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:32.231+0000 7f29b9e08640 1 -- 172.21.15.82:0/2147984934 shutdown_connections 2024-05-20T16:13:32.231 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:32.231+0000 7f29b9e08640 1 --2- 172.21.15.82:0/2147984934 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f29b4102370 0x7f29b4102780 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:32.231 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:32.231+0000 7f29b9e08640 1 --2- 172.21.15.82:0/2147984934 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f29b4107070 0x7f29b4109490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:32.231 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:32.231+0000 7f29b9e08640 1 --2- 172.21.15.82:0/2147984934 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f29b4101130 0x7f29b4101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:32.231 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:32.231+0000 7f29b9e08640 1 -- 172.21.15.82:0/2147984934 >> 172.21.15.82:0/2147984934 conn(0x7f29b40fc300 msgr2=0x7f29b40fe7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:13:32.231 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:32.231+0000 7f29b9e08640 1 -- 172.21.15.82:0/2147984934 shutdown_connections 2024-05-20T16:13:32.231 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:32.231+0000 7f29b9e08640 1 -- 172.21.15.82:0/2147984934 wait complete. 2024-05-20T16:13:32.231 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:32.231+0000 7f29b9e08640 1 Processor -- start 2024-05-20T16:13:32.231 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:32.231+0000 7f29b9e08640 1 -- start start 2024-05-20T16:13:32.233 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:32.232+0000 7f29b9e08640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f29b4101130 0x7f29b41992f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:32.233 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:32.232+0000 7f29b9e08640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f29b4102370 0x7f29b4199830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:32.233 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:32.232+0000 7f29b9e08640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f29b4107070 0x7f29b41a07c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:32.233 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:32.232+0000 7f29b9e08640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f29b410ce20 con 0x7f29b4102370 2024-05-20T16:13:32.233 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:32.232+0000 7f29b9e08640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f29b410cca0 con 0x7f29b4107070 2024-05-20T16:13:32.233 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:32.232+0000 7f29b9e08640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f29b410cfa0 con 0x7f29b4101130 2024-05-20T16:13:32.233 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:32.232+0000 7f29b2ffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f29b4102370 0x7f29b4199830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:13:32.234 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:32.232+0000 7f29b2ffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f29b4102370 0x7f29b4199830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:33332/0 (socket says 172.21.15.82:33332) 2024-05-20T16:13:32.234 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:32.232+0000 7f29b2ffd640 1 -- 172.21.15.82:0/1698804197 learned_addr learned my addr 172.21.15.82:0/1698804197 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:13:32.234 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:32.232+0000 7f29b3fff640 1 --2- 172.21.15.82:0/1698804197 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f29b4107070 0x7f29b41a07c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:13:32.234 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:32.232+0000 7f29b37fe640 1 --2- 172.21.15.82:0/1698804197 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f29b4101130 0x7f29b41992f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:13:32.234 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:32.232+0000 7f29b2ffd640 1 -- 172.21.15.82:0/1698804197 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f29b4101130 msgr2=0x7f29b41992f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:32.234 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:32.232+0000 7f29b2ffd640 1 --2- 172.21.15.82:0/1698804197 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f29b4101130 0x7f29b41992f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:32.234 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:32.232+0000 7f29b2ffd640 1 -- 172.21.15.82:0/1698804197 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f29b4107070 msgr2=0x7f29b41a07c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:32.234 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:32.232+0000 7f29b2ffd640 1 --2- 172.21.15.82:0/1698804197 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f29b4107070 0x7f29b41a07c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:32.234 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:32.232+0000 7f29b2ffd640 1 -- 172.21.15.82:0/1698804197 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29b41a0e30 con 0x7f29b4102370 2024-05-20T16:13:32.234 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:32.232+0000 7f29b2ffd640 1 --2- 172.21.15.82:0/1698804197 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f29b4102370 0x7f29b4199830 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f29a802ade0 tx=0x7f29a8031ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:13:32.234 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:32.232+0000 7f29b0ff9640 1 -- 172.21.15.82:0/1698804197 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29a802fae0 con 0x7f29b4102370 2024-05-20T16:13:32.234 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:32.232+0000 7f29b9e08640 1 -- 172.21.15.82:0/1698804197 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29b41a10c0 con 0x7f29b4102370 2024-05-20T16:13:32.234 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:32.232+0000 7f29b0ff9640 1 -- 172.21.15.82:0/1698804197 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f29a802fc80 con 0x7f29b4102370 2024-05-20T16:13:32.234 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:32.232+0000 7f29b0ff9640 1 -- 172.21.15.82:0/1698804197 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29a8046750 con 0x7f29b4102370 2024-05-20T16:13:32.235 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:32.233+0000 7f29b9e08640 1 -- 172.21.15.82:0/1698804197 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29b41a1690 con 0x7f29b4102370 2024-05-20T16:13:32.235 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:32.233+0000 7f29b9e08640 1 -- 172.21.15.82:0/1698804197 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29b41a18f0 con 0x7f29b4102370 2024-05-20T16:13:32.235 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:32.235+0000 7f29b0ff9640 1 -- 172.21.15.82:0/1698804197 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f29a80468f0 con 0x7f29b4102370 2024-05-20T16:13:32.235 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:32.235+0000 7f29b0ff9640 1 --2- 172.21.15.82:0/1698804197 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f299c0a2b90 0x7f299c0a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:32.235 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:32.235+0000 7f29b37fe640 1 --2- 172.21.15.82:0/1698804197 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f299c0a2b90 0x7f299c0a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:13:32.235 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:32.236+0000 7f29b0ff9640 1 -- 172.21.15.82:0/1698804197 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9092+0+0 (secure 0 0 0) 0x7f29a80f0c40 con 0x7f29b4102370 2024-05-20T16:13:32.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:32.236+0000 7f29b37fe640 1 --2- 172.21.15.82:0/1698804197 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f299c0a2b90 0x7f299c0a50a0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f29b4101d30 tx=0x7f29a4005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:13:32.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:32.239+0000 7f29b0ff9640 1 -- 172.21.15.82:0/1698804197 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f29a8081850 con 0x7f29b4102370 2024-05-20T16:13:32.418 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:32.418+0000 7f29b9e08640 1 -- 172.21.15.82:0/1698804197 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f29b41a1ac0 con 0x7f29b4102370 2024-05-20T16:13:32.419 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:32.419+0000 7f29b0ff9640 1 -- 172.21.15.82:0/1698804197 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v93) ==== 104+0+11 (secure 0 0 0) 0x7f29a80bd840 con 0x7f29b4102370 2024-05-20T16:13:32.419 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 32 2024-05-20T16:13:32.421 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:32.421+0000 7f29b9e08640 1 -- 172.21.15.82:0/1698804197 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f299c0a2b90 msgr2=0x7f299c0a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:32.421 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:32.421+0000 7f29b9e08640 1 --2- 172.21.15.82:0/1698804197 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f299c0a2b90 0x7f299c0a50a0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f29b4101d30 tx=0x7f29a4005f90 comp rx=0 tx=0).stop 2024-05-20T16:13:32.421 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:32.421+0000 7f29b9e08640 1 -- 172.21.15.82:0/1698804197 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f29b4102370 msgr2=0x7f29b4199830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:32.421 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:32.421+0000 7f29b9e08640 1 --2- 172.21.15.82:0/1698804197 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f29b4102370 0x7f29b4199830 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f29a802ade0 tx=0x7f29a8031ee0 comp rx=0 tx=0).stop 2024-05-20T16:13:32.421 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:32.421+0000 7f29b9e08640 1 -- 172.21.15.82:0/1698804197 shutdown_connections 2024-05-20T16:13:32.421 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:32.421+0000 7f29b9e08640 1 --2- 172.21.15.82:0/1698804197 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f299c0a2b90 0x7f299c0a50a0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:32.422 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:32.421+0000 7f29b9e08640 1 --2- 172.21.15.82:0/1698804197 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f29b4102370 0x7f29b4199830 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:32.422 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:32.421+0000 7f29b9e08640 1 --2- 172.21.15.82:0/1698804197 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f29b4107070 0x7f29b41a07c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:32.422 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:32.421+0000 7f29b9e08640 1 --2- 172.21.15.82:0/1698804197 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f29b4101130 0x7f29b41992f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:32.422 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:32.421+0000 7f29b9e08640 1 -- 172.21.15.82:0/1698804197 >> 172.21.15.82:0/1698804197 conn(0x7f29b40fc300 msgr2=0x7f29b40ff650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:13:32.422 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:32.421+0000 7f29b9e08640 1 -- 172.21.15.82:0/1698804197 shutdown_connections 2024-05-20T16:13:32.422 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:32.421+0000 7f29b9e08640 1 -- 172.21.15.82:0/1698804197 wait complete. 2024-05-20T16:13:32.432 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-20T16:13:32.548 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:32.546+0000 7f0b3d9cc640 1 -- 172.21.15.82:0/2851581718 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0b38102910 msgr2=0x7f0b38102d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:32.549 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:32.546+0000 7f0b3d9cc640 1 --2- 172.21.15.82:0/2851581718 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0b38102910 0x7f0b38102d80 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f0b2c009900 tx=0x7f0b2c02a410 comp rx=0 tx=0).stop 2024-05-20T16:13:32.549 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:32.546+0000 7f0b3d9cc640 1 -- 172.21.15.82:0/2851581718 shutdown_connections 2024-05-20T16:13:32.549 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:32.547+0000 7f0b3d9cc640 1 --2- 172.21.15.82:0/2851581718 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0b38102910 0x7f0b38102d80 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:32.549 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:32.547+0000 7f0b3d9cc640 1 --2- 172.21.15.82:0/2851581718 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0b38103410 0x7f0b38109b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:32.549 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:32.547+0000 7f0b3d9cc640 1 --2- 172.21.15.82:0/2851581718 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0b38101f30 0x7f0b38102340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:32.549 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:32.547+0000 7f0b3d9cc640 1 -- 172.21.15.82:0/2851581718 >> 172.21.15.82:0/2851581718 conn(0x7f0b380fc2f0 msgr2=0x7f0b380fe7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:13:32.549 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:32.547+0000 7f0b3d9cc640 1 -- 172.21.15.82:0/2851581718 shutdown_connections 2024-05-20T16:13:32.549 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:32.547+0000 7f0b3d9cc640 1 -- 172.21.15.82:0/2851581718 wait complete. 2024-05-20T16:13:32.549 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:32.547+0000 7f0b3d9cc640 1 Processor -- start 2024-05-20T16:13:32.549 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:32.547+0000 7f0b3d9cc640 1 -- start start 2024-05-20T16:13:32.549 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:32.548+0000 7f0b3d9cc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0b38101f30 0x7f0b381981e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:32.549 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:32.548+0000 7f0b3d9cc640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0b38102910 0x7f0b38198720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:32.549 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:32.548+0000 7f0b3d9cc640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0b38103410 0x7f0b3819f660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:32.550 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:32.548+0000 7f0b3d9cc640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f0b3810d670 con 0x7f0b38101f30 2024-05-20T16:13:32.550 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:32.548+0000 7f0b3d9cc640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f0b3810d4f0 con 0x7f0b38102910 2024-05-20T16:13:32.550 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:32.548+0000 7f0b3d9cc640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f0b3810d7f0 con 0x7f0b38103410 2024-05-20T16:13:32.550 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:32.548+0000 7f0b36ffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0b38101f30 0x7f0b381981e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:13:32.550 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:32.548+0000 7f0b36ffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0b38101f30 0x7f0b381981e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:33346/0 (socket says 172.21.15.82:33346) 2024-05-20T16:13:32.550 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:32.548+0000 7f0b36ffd640 1 -- 172.21.15.82:0/2708458887 learned_addr learned my addr 172.21.15.82:0/2708458887 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:13:32.550 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:32.548+0000 7f0b377fe640 1 --2- 172.21.15.82:0/2708458887 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0b38103410 0x7f0b3819f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:13:32.550 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:32.548+0000 7f0b36ffd640 1 -- 172.21.15.82:0/2708458887 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0b38103410 msgr2=0x7f0b3819f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:32.550 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:32.548+0000 7f0b36ffd640 1 --2- 172.21.15.82:0/2708458887 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0b38103410 0x7f0b3819f660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:32.550 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:32.548+0000 7f0b36ffd640 1 -- 172.21.15.82:0/2708458887 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0b38102910 msgr2=0x7f0b38198720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:32.550 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:32.548+0000 7f0b367fc640 1 --2- 172.21.15.82:0/2708458887 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0b38102910 0x7f0b38198720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:13:32.550 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:32.548+0000 7f0b36ffd640 1 --2- 172.21.15.82:0/2708458887 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0b38102910 0x7f0b38198720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:32.551 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:32.548+0000 7f0b36ffd640 1 -- 172.21.15.82:0/2708458887 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b3819fcd0 con 0x7f0b38101f30 2024-05-20T16:13:32.551 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:32.548+0000 7f0b36ffd640 1 --2- 172.21.15.82:0/2708458887 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0b38101f30 0x7f0b381981e0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f0b2800e9f0 tx=0x7f0b2800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:13:32.551 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:32.548+0000 7f0b3c9ca640 1 -- 172.21.15.82:0/2708458887 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b2800ccc0 con 0x7f0b38101f30 2024-05-20T16:13:32.551 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:32.548+0000 7f0b3c9ca640 1 -- 172.21.15.82:0/2708458887 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0b280035f0 con 0x7f0b38101f30 2024-05-20T16:13:32.551 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:32.548+0000 7f0b3d9cc640 1 -- 172.21.15.82:0/2708458887 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b3819ffc0 con 0x7f0b38101f30 2024-05-20T16:13:32.551 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:32.548+0000 7f0b3c9ca640 1 -- 172.21.15.82:0/2708458887 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b2800ce60 con 0x7f0b38101f30 2024-05-20T16:13:32.551 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:32.548+0000 7f0b3d9cc640 1 -- 172.21.15.82:0/2708458887 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b381a0420 con 0x7f0b38101f30 2024-05-20T16:13:32.552 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:32.549+0000 7f0b3d9cc640 1 -- 172.21.15.82:0/2708458887 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b38068290 con 0x7f0b38101f30 2024-05-20T16:13:32.552 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:32.551+0000 7f0b3c9ca640 1 -- 172.21.15.82:0/2708458887 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f0b280037a0 con 0x7f0b38101f30 2024-05-20T16:13:32.552 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:32.551+0000 7f0b3c9ca640 1 --2- 172.21.15.82:0/2708458887 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f0b200a2b90 0x7f0b200a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:32.552 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:32.551+0000 7f0b367fc640 1 --2- 172.21.15.82:0/2708458887 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f0b200a2b90 0x7f0b200a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:13:32.552 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:32.551+0000 7f0b3c9ca640 1 -- 172.21.15.82:0/2708458887 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9092+0+0 (secure 0 0 0) 0x7f0b280cf820 con 0x7f0b38101f30 2024-05-20T16:13:32.552 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:32.552+0000 7f0b367fc640 1 --2- 172.21.15.82:0/2708458887 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f0b200a2b90 0x7f0b200a50a0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f0b2c02a930 tx=0x7f0b2c005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:13:32.555 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:32.556+0000 7f0b3c9ca640 1 -- 172.21.15.82:0/2708458887 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f0b2809c420 con 0x7f0b38101f30 2024-05-20T16:13:32.732 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:32.732+0000 7f0b3d9cc640 1 -- 172.21.15.82:0/2708458887 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f0b380627f0 con 0x7f0b38101f30 2024-05-20T16:13:32.732 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:32.732+0000 7f0b3c9ca640 1 -- 172.21.15.82:0/2708458887 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v93) ==== 100+0+11 (secure 0 0 0) 0x7f0b2809bd20 con 0x7f0b38101f30 2024-05-20T16:13:32.732 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 64 2024-05-20T16:13:32.734 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:32.734+0000 7f0b3d9cc640 1 -- 172.21.15.82:0/2708458887 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f0b200a2b90 msgr2=0x7f0b200a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:32.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:32.734+0000 7f0b3d9cc640 1 --2- 172.21.15.82:0/2708458887 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f0b200a2b90 0x7f0b200a50a0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f0b2c02a930 tx=0x7f0b2c005fd0 comp rx=0 tx=0).stop 2024-05-20T16:13:32.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:32.734+0000 7f0b3d9cc640 1 -- 172.21.15.82:0/2708458887 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0b38101f30 msgr2=0x7f0b381981e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:32.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:32.734+0000 7f0b3d9cc640 1 --2- 172.21.15.82:0/2708458887 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0b38101f30 0x7f0b381981e0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f0b2800e9f0 tx=0x7f0b2800eeb0 comp rx=0 tx=0).stop 2024-05-20T16:13:32.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:32.734+0000 7f0b3d9cc640 1 -- 172.21.15.82:0/2708458887 shutdown_connections 2024-05-20T16:13:32.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:32.734+0000 7f0b3d9cc640 1 --2- 172.21.15.82:0/2708458887 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f0b200a2b90 0x7f0b200a50a0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:32.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:32.734+0000 7f0b3d9cc640 1 --2- 172.21.15.82:0/2708458887 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0b38103410 0x7f0b3819f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:32.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:32.734+0000 7f0b3d9cc640 1 --2- 172.21.15.82:0/2708458887 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0b38102910 0x7f0b38198720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:32.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:32.734+0000 7f0b3d9cc640 1 --2- 172.21.15.82:0/2708458887 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0b38101f30 0x7f0b381981e0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:32.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:32.735+0000 7f0b3d9cc640 1 -- 172.21.15.82:0/2708458887 >> 172.21.15.82:0/2708458887 conn(0x7f0b380fc2f0 msgr2=0x7f0b38103aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:13:32.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:32.735+0000 7f0b3d9cc640 1 -- 172.21.15.82:0/2708458887 shutdown_connections 2024-05-20T16:13:32.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:32.735+0000 7f0b3d9cc640 1 -- 172.21.15.82:0/2708458887 wait complete. 2024-05-20T16:13:32.746 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-20T16:13:32.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:32.860+0000 7f6caab4c640 1 -- 172.21.15.82:0/1189352781 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6ca4101f20 msgr2=0x7f6ca4102330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:32.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:32.860+0000 7f6caab4c640 1 --2- 172.21.15.82:0/1189352781 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6ca4101f20 0x7f6ca4102330 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f6c98009900 tx=0x7f6c9802a410 comp rx=0 tx=0).stop 2024-05-20T16:13:32.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:32.860+0000 7f6caab4c640 1 -- 172.21.15.82:0/1189352781 shutdown_connections 2024-05-20T16:13:32.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:32.860+0000 7f6caab4c640 1 --2- 172.21.15.82:0/1189352781 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6ca4103420 0x7f6ca4109b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:32.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:32.860+0000 7f6caab4c640 1 --2- 172.21.15.82:0/1189352781 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6ca4102900 0x7f6ca4102d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:32.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:32.860+0000 7f6caab4c640 1 --2- 172.21.15.82:0/1189352781 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6ca4101f20 0x7f6ca4102330 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:32.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:32.860+0000 7f6caab4c640 1 -- 172.21.15.82:0/1189352781 >> 172.21.15.82:0/1189352781 conn(0x7f6ca40fc300 msgr2=0x7f6ca40fe7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:13:32.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:32.860+0000 7f6caab4c640 1 -- 172.21.15.82:0/1189352781 shutdown_connections 2024-05-20T16:13:32.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:32.860+0000 7f6caab4c640 1 -- 172.21.15.82:0/1189352781 wait complete. 2024-05-20T16:13:32.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:32.861+0000 7f6caab4c640 1 Processor -- start 2024-05-20T16:13:32.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:32.861+0000 7f6caab4c640 1 -- start start 2024-05-20T16:13:32.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:32.861+0000 7f6caab4c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6ca4101f20 0x7f6ca4199290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:32.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:32.861+0000 7f6caab4c640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6ca4102900 0x7f6ca41997d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:32.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:32.861+0000 7f6caab4c640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6ca4103420 0x7f6ca41a0760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:32.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:32.861+0000 7f6caab4c640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f6ca410e740 con 0x7f6ca4101f20 2024-05-20T16:13:32.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:32.861+0000 7f6caab4c640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f6ca410e5c0 con 0x7f6ca4102900 2024-05-20T16:13:32.864 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:32.861+0000 7f6caab4c640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f6ca410e8c0 con 0x7f6ca4103420 2024-05-20T16:13:32.864 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:32.861+0000 7f6ca88e8640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6ca4101f20 0x7f6ca4199290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:13:32.864 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:32.861+0000 7f6ca88e8640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6ca4101f20 0x7f6ca4199290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:33366/0 (socket says 172.21.15.82:33366) 2024-05-20T16:13:32.864 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:32.861+0000 7f6ca88e8640 1 -- 172.21.15.82:0/3249222612 learned_addr learned my addr 172.21.15.82:0/3249222612 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:13:32.864 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:32.861+0000 7f6ca3fff640 1 --2- 172.21.15.82:0/3249222612 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6ca4102900 0x7f6ca41997d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:13:32.864 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:32.862+0000 7f6ca90e9640 1 --2- 172.21.15.82:0/3249222612 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6ca4103420 0x7f6ca41a0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:13:32.864 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:32.862+0000 7f6ca88e8640 1 -- 172.21.15.82:0/3249222612 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6ca4103420 msgr2=0x7f6ca41a0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:32.864 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:32.862+0000 7f6ca88e8640 1 --2- 172.21.15.82:0/3249222612 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6ca4103420 0x7f6ca41a0760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:32.864 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:32.862+0000 7f6ca88e8640 1 -- 172.21.15.82:0/3249222612 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6ca4102900 msgr2=0x7f6ca41997d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:32.864 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:32.862+0000 7f6ca88e8640 1 --2- 172.21.15.82:0/3249222612 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6ca4102900 0x7f6ca41997d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:32.864 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:32.862+0000 7f6ca88e8640 1 -- 172.21.15.82:0/3249222612 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ca41a0dd0 con 0x7f6ca4101f20 2024-05-20T16:13:32.864 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:32.862+0000 7f6ca88e8640 1 --2- 172.21.15.82:0/3249222612 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6ca4101f20 0x7f6ca4199290 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f6c98006010 tx=0x7f6c980043b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:13:32.864 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:32.862+0000 7f6ca1ffb640 1 -- 172.21.15.82:0/3249222612 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c9803f030 con 0x7f6ca4101f20 2024-05-20T16:13:32.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:32.862+0000 7f6caab4c640 1 -- 172.21.15.82:0/3249222612 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ca41a1060 con 0x7f6ca4101f20 2024-05-20T16:13:32.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:32.862+0000 7f6ca1ffb640 1 -- 172.21.15.82:0/3249222612 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c98003ce0 con 0x7f6ca4101f20 2024-05-20T16:13:32.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:32.862+0000 7f6caab4c640 1 -- 172.21.15.82:0/3249222612 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ca41a1550 con 0x7f6ca4101f20 2024-05-20T16:13:32.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:32.862+0000 7f6ca1ffb640 1 -- 172.21.15.82:0/3249222612 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c98049060 con 0x7f6ca4101f20 2024-05-20T16:13:32.866 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:32.863+0000 7f6caab4c640 1 -- 172.21.15.82:0/3249222612 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ca41068c0 con 0x7f6ca4101f20 2024-05-20T16:13:32.866 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:32.864+0000 7f6ca1ffb640 1 -- 172.21.15.82:0/3249222612 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f6c98042030 con 0x7f6ca4101f20 2024-05-20T16:13:32.866 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:32.865+0000 7f6ca1ffb640 1 --2- 172.21.15.82:0/3249222612 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f6c940a2b90 0x7f6c940a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:32.866 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:32.865+0000 7f6ca3fff640 1 --2- 172.21.15.82:0/3249222612 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f6c940a2b90 0x7f6c940a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:13:32.866 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:32.865+0000 7f6ca1ffb640 1 -- 172.21.15.82:0/3249222612 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9092+0+0 (secure 0 0 0) 0x7f6c980efc50 con 0x7f6ca4101f20 2024-05-20T16:13:32.866 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:32.866+0000 7f6ca3fff640 1 --2- 172.21.15.82:0/3249222612 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f6c940a2b90 0x7f6c940a50a0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f6c900058c0 tx=0x7f6c9000a250 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:13:32.869 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:32.870+0000 7f6ca1ffb640 1 -- 172.21.15.82:0/3249222612 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f6c980f5020 con 0x7f6ca4101f20 2024-05-20T16:13:33.044 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:33.044+0000 7f6caab4c640 1 -- 172.21.15.82:0/3249222612 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f6ca4062750 con 0x7f6ca4101f20 2024-05-20T16:13:33.045 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:33.045+0000 7f6ca1ffb640 1 -- 172.21.15.82:0/3249222612 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v93) ==== 103+0+11 (secure 0 0 0) 0x7f6c980bc200 con 0x7f6ca4101f20 2024-05-20T16:13:33.045 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 32 2024-05-20T16:13:33.047 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:33.047+0000 7f6caab4c640 1 -- 172.21.15.82:0/3249222612 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f6c940a2b90 msgr2=0x7f6c940a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:33.047 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:33.047+0000 7f6caab4c640 1 --2- 172.21.15.82:0/3249222612 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f6c940a2b90 0x7f6c940a50a0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f6c900058c0 tx=0x7f6c9000a250 comp rx=0 tx=0).stop 2024-05-20T16:13:33.047 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:33.047+0000 7f6caab4c640 1 -- 172.21.15.82:0/3249222612 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6ca4101f20 msgr2=0x7f6ca4199290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:33.047 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:33.047+0000 7f6caab4c640 1 --2- 172.21.15.82:0/3249222612 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6ca4101f20 0x7f6ca4199290 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f6c98006010 tx=0x7f6c980043b0 comp rx=0 tx=0).stop 2024-05-20T16:13:33.047 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:33.047+0000 7f6caab4c640 1 -- 172.21.15.82:0/3249222612 shutdown_connections 2024-05-20T16:13:33.047 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:33.047+0000 7f6caab4c640 1 --2- 172.21.15.82:0/3249222612 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f6c940a2b90 0x7f6c940a50a0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:33.048 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:33.047+0000 7f6caab4c640 1 --2- 172.21.15.82:0/3249222612 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6ca4103420 0x7f6ca41a0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:33.048 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:33.047+0000 7f6caab4c640 1 --2- 172.21.15.82:0/3249222612 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6ca4102900 0x7f6ca41997d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:33.048 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:33.047+0000 7f6caab4c640 1 --2- 172.21.15.82:0/3249222612 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6ca4101f20 0x7f6ca4199290 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:33.048 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:33.047+0000 7f6caab4c640 1 -- 172.21.15.82:0/3249222612 >> 172.21.15.82:0/3249222612 conn(0x7f6ca40fc300 msgr2=0x7f6ca4103a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:13:33.048 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:33.047+0000 7f6caab4c640 1 -- 172.21.15.82:0/3249222612 shutdown_connections 2024-05-20T16:13:33.048 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:33.047+0000 7f6caab4c640 1 -- 172.21.15.82:0/3249222612 wait complete. 2024-05-20T16:13:33.057 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:13:32 smithi082 ceph-mon[32238]: from='client.? 172.21.15.82:0/1087252378' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-20T16:13:33.057 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:13:32 smithi082 ceph-mon[32238]: from='client.? 172.21.15.82:0/524819923' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-20T16:13:33.057 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:13:32 smithi082 ceph-mon[32238]: from='client.? 172.21.15.82:0/1698804197' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-20T16:13:33.057 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:13:32 smithi082 ceph-mon[32238]: from='client.? 172.21.15.82:0/2708458887' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-20T16:13:33.059 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-20T16:13:33.062 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:13:32 smithi160 ceph-mon[36347]: from='client.? 172.21.15.82:0/1087252378' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-20T16:13:33.062 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:13:32 smithi160 ceph-mon[36347]: from='client.? 172.21.15.82:0/524819923' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-20T16:13:33.062 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:13:32 smithi160 ceph-mon[36347]: from='client.? 172.21.15.82:0/1698804197' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-20T16:13:33.062 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:13:32 smithi160 ceph-mon[36347]: from='client.? 172.21.15.82:0/2708458887' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-20T16:13:33.135 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:13:32 smithi184 ceph-mon[36209]: from='client.? 172.21.15.82:0/1087252378' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-20T16:13:33.136 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:13:32 smithi184 ceph-mon[36209]: from='client.? 172.21.15.82:0/524819923' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-20T16:13:33.136 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:13:32 smithi184 ceph-mon[36209]: from='client.? 172.21.15.82:0/1698804197' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-20T16:13:33.136 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:13:32 smithi184 ceph-mon[36209]: from='client.? 172.21.15.82:0/2708458887' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-20T16:13:33.171 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:33.170+0000 7fc1e07cc640 1 -- 172.21.15.82:0/3468947454 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc1d8105b90 msgr2=0x7fc1d8108070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:33.172 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:33.170+0000 7fc1e07cc640 1 --2- 172.21.15.82:0/3468947454 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc1d8105b90 0x7fc1d8108070 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fc1cc009900 tx=0x7fc1cc02a410 comp rx=0 tx=0).stop 2024-05-20T16:13:33.172 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:33.170+0000 7fc1e07cc640 1 -- 172.21.15.82:0/3468947454 shutdown_connections 2024-05-20T16:13:33.172 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:33.170+0000 7fc1e07cc640 1 --2- 172.21.15.82:0/3468947454 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc1d8105b90 0x7fc1d8108070 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:33.172 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:33.170+0000 7fc1e07cc640 1 --2- 172.21.15.82:0/3468947454 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc1d8108700 0x7fc1d810ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:33.172 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:33.170+0000 7fc1e07cc640 1 --2- 172.21.15.82:0/3468947454 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc1d81031a0 0x7fc1d81055c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:33.172 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:33.170+0000 7fc1e07cc640 1 -- 172.21.15.82:0/3468947454 >> 172.21.15.82:0/3468947454 conn(0x7fc1d80fcb80 msgr2=0x7fc1d80ff010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:13:33.172 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:33.170+0000 7fc1e07cc640 1 -- 172.21.15.82:0/3468947454 shutdown_connections 2024-05-20T16:13:33.172 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:33.170+0000 7fc1e07cc640 1 -- 172.21.15.82:0/3468947454 wait complete. 2024-05-20T16:13:33.172 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:33.170+0000 7fc1e07cc640 1 Processor -- start 2024-05-20T16:13:33.172 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:33.171+0000 7fc1e07cc640 1 -- start start 2024-05-20T16:13:33.173 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:33.171+0000 7fc1e07cc640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc1d81031a0 0x7fc1d81992e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:33.173 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:33.171+0000 7fc1e07cc640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc1d8105b90 0x7fc1d8199820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:33.173 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:33.171+0000 7fc1e07cc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc1d8108700 0x7fc1d81a07b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:33.173 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:33.171+0000 7fc1e07cc640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fc1d810e730 con 0x7fc1d8108700 2024-05-20T16:13:33.173 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:33.171+0000 7fc1e07cc640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fc1d810e5b0 con 0x7fc1d81031a0 2024-05-20T16:13:33.173 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:33.171+0000 7fc1e07cc640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fc1d810e8b0 con 0x7fc1d8105b90 2024-05-20T16:13:33.173 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:33.171+0000 7fc1ded69640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc1d8108700 0x7fc1d81a07b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:13:33.173 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:33.171+0000 7fc1ded69640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc1d8108700 0x7fc1d81a07b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:33384/0 (socket says 172.21.15.82:33384) 2024-05-20T16:13:33.173 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:33.171+0000 7fc1ded69640 1 -- 172.21.15.82:0/2889764950 learned_addr learned my addr 172.21.15.82:0/2889764950 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:13:33.173 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:33.171+0000 7fc1ddd67640 1 --2- 172.21.15.82:0/2889764950 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc1d8105b90 0x7fc1d8199820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:13:33.173 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:33.171+0000 7fc1de568640 1 --2- 172.21.15.82:0/2889764950 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc1d81031a0 0x7fc1d81992e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:13:33.173 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:33.171+0000 7fc1ded69640 1 -- 172.21.15.82:0/2889764950 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc1d8105b90 msgr2=0x7fc1d8199820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:33.173 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:33.171+0000 7fc1ded69640 1 --2- 172.21.15.82:0/2889764950 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc1d8105b90 0x7fc1d8199820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:33.173 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:33.171+0000 7fc1ded69640 1 -- 172.21.15.82:0/2889764950 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc1d81031a0 msgr2=0x7fc1d81992e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:33.174 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:33.171+0000 7fc1ded69640 1 --2- 172.21.15.82:0/2889764950 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc1d81031a0 0x7fc1d81992e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:33.174 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:33.171+0000 7fc1ded69640 1 -- 172.21.15.82:0/2889764950 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1d81a0e20 con 0x7fc1d8108700 2024-05-20T16:13:33.174 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:33.171+0000 7fc1de568640 1 --2- 172.21.15.82:0/2889764950 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc1d81031a0 0x7fc1d81992e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T16:13:33.174 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:33.171+0000 7fc1ded69640 1 --2- 172.21.15.82:0/2889764950 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc1d8108700 0x7fc1d81a07b0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fc1c8009fd0 tx=0x7fc1c800b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:13:33.174 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:33.171+0000 7fc1d77fe640 1 -- 172.21.15.82:0/2889764950 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1c8017070 con 0x7fc1d8108700 2024-05-20T16:13:33.174 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:33.172+0000 7fc1e07cc640 1 -- 172.21.15.82:0/2889764950 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1d81a1110 con 0x7fc1d8108700 2024-05-20T16:13:33.174 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:33.172+0000 7fc1d77fe640 1 -- 172.21.15.82:0/2889764950 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc1c8003e50 con 0x7fc1d8108700 2024-05-20T16:13:33.174 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:33.172+0000 7fc1d77fe640 1 -- 172.21.15.82:0/2889764950 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1c80106c0 con 0x7fc1d8108700 2024-05-20T16:13:33.174 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:33.172+0000 7fc1e07cc640 1 -- 172.21.15.82:0/2889764950 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1d81a1660 con 0x7fc1d8108700 2024-05-20T16:13:33.175 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:33.172+0000 7fc1e07cc640 1 -- 172.21.15.82:0/2889764950 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1d80fe6a0 con 0x7fc1d8108700 2024-05-20T16:13:33.175 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:33.174+0000 7fc1d77fe640 1 -- 172.21.15.82:0/2889764950 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fc1c8003970 con 0x7fc1d8108700 2024-05-20T16:13:33.175 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:33.174+0000 7fc1d77fe640 1 --2- 172.21.15.82:0/2889764950 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fc1c00a2b90 0x7fc1c00a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:33.175 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:33.175+0000 7fc1de568640 1 --2- 172.21.15.82:0/2889764950 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fc1c00a2b90 0x7fc1c00a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:13:33.175 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:33.175+0000 7fc1d77fe640 1 -- 172.21.15.82:0/2889764950 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9092+0+0 (secure 0 0 0) 0x7fc1c80cf1a0 con 0x7fc1d8108700 2024-05-20T16:13:33.176 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:33.175+0000 7fc1de568640 1 --2- 172.21.15.82:0/2889764950 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fc1c00a2b90 0x7fc1c00a50a0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fc1d0006fd0 tx=0x7fc1d0008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:13:33.179 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:33.179+0000 7fc1d77fe640 1 -- 172.21.15.82:0/2889764950 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fc1c809bda0 con 0x7fc1d8108700 2024-05-20T16:13:33.394 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:33.394+0000 7fc1e07cc640 1 -- 172.21.15.82:0/2889764950 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc1d802d2a0 con 0x7fc1d8108700 2024-05-20T16:13:33.395 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:33.395+0000 7fc1d77fe640 1 -- 172.21.15.82:0/2889764950 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11879 (secure 0 0 0) 0x7fc1c809b6a0 con 0x7fc1d8108700 2024-05-20T16:13:33.395 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-05-20T16:13:33.395 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/2044534217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":2044534217},{"type":"v1","addr":"172.21.15.82:6835","nonce":2044534217}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/2635635097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":2635635097},{"type":"v1","addr":"172.21.15.82:6837","nonce":2635635097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14625,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6839/2118233541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6838","nonce":2118233541},{"type":"v1","addr":"172.21.15.82:6839","nonce":2118233541}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14637,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6841/2119517973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6840","nonce":2119517973},{"type":"v1","addr":"172.21.15.82:6841","nonce":2119517973}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6833/4015144007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":4015144007},{"type":"v1","addr":"172.21.15.184:6833","nonce":4015144007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6833/975243372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6832","nonce":975243372},{"type":"v1","addr":"172.21.15.160:6833","nonce":975243372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-20T16:11:46.751632+0000","modified":"2024-05-20T16:12:13.394770+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24548,"mds_1":24511,"mds_2":24506},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24470":{"gid":24470,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.160:6835/664068363","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":664068363},{"type":"v1","addr":"172.21.15.160:6835","nonce":664068363}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6835/4242156090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":4242156090},{"type":"v1","addr":"172.21.15.184:6835","nonce":4242156090}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6837/403262242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":403262242},{"type":"v1","addr":"172.21.15.184:6837","nonce":403262242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.160:6837/2846188680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6836","nonce":2846188680},{"type":"v1","addr":"172.21.15.160:6837","nonce":2846188680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.184:6839/2199955067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":2199955067},{"type":"v1","addr":"172.21.15.184:6839","nonce":2199955067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24548":{"gid":24548,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.160:6839/2765554482","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6838","nonce":2765554482},{"type":"v1","addr":"172.21.15.160:6839","nonce":2765554482}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24548,"qdb_cluster":[24511,24506,24548]},"id":1}]} 2024-05-20T16:13:33.395 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 24 2024-05-20T16:13:33.397 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:33.397+0000 7fc1e07cc640 1 -- 172.21.15.82:0/2889764950 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fc1c00a2b90 msgr2=0x7fc1c00a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:33.397 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:33.397+0000 7fc1e07cc640 1 --2- 172.21.15.82:0/2889764950 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fc1c00a2b90 0x7fc1c00a50a0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fc1d0006fd0 tx=0x7fc1d0008040 comp rx=0 tx=0).stop 2024-05-20T16:13:33.397 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:33.397+0000 7fc1e07cc640 1 -- 172.21.15.82:0/2889764950 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc1d8108700 msgr2=0x7fc1d81a07b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:33.397 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:33.397+0000 7fc1e07cc640 1 --2- 172.21.15.82:0/2889764950 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc1d8108700 0x7fc1d81a07b0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fc1c8009fd0 tx=0x7fc1c800b260 comp rx=0 tx=0).stop 2024-05-20T16:13:33.397 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:33.397+0000 7fc1e07cc640 1 -- 172.21.15.82:0/2889764950 shutdown_connections 2024-05-20T16:13:33.397 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:33.397+0000 7fc1e07cc640 1 --2- 172.21.15.82:0/2889764950 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fc1c00a2b90 0x7fc1c00a50a0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:33.398 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:33.397+0000 7fc1e07cc640 1 --2- 172.21.15.82:0/2889764950 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc1d8108700 0x7fc1d81a07b0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:33.398 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:33.397+0000 7fc1e07cc640 1 --2- 172.21.15.82:0/2889764950 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc1d8105b90 0x7fc1d8199820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:33.398 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:33.397+0000 7fc1e07cc640 1 --2- 172.21.15.82:0/2889764950 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc1d81031a0 0x7fc1d81992e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:33.398 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:33.397+0000 7fc1e07cc640 1 -- 172.21.15.82:0/2889764950 >> 172.21.15.82:0/2889764950 conn(0x7fc1d80fcb80 msgr2=0x7fc1d8103d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:13:33.398 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:33.397+0000 7fc1e07cc640 1 -- 172.21.15.82:0/2889764950 shutdown_connections 2024-05-20T16:13:33.398 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:33.397+0000 7fc1e07cc640 1 -- 172.21.15.82:0/2889764950 wait complete. 2024-05-20T16:13:33.410 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-20T16:13:33.523 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:33.521+0000 7f0617337640 1 -- 172.21.15.82:0/2451386981 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0610105f10 msgr2=0x7f0610108330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:33.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:33.521+0000 7f0617337640 1 --2- 172.21.15.82:0/2451386981 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0610105f10 0x7f0610108330 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f0604009900 tx=0x7f060402a410 comp rx=0 tx=0).stop 2024-05-20T16:13:33.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:33.521+0000 7f0617337640 1 -- 172.21.15.82:0/2451386981 shutdown_connections 2024-05-20T16:13:33.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:33.521+0000 7f0617337640 1 --2- 172.21.15.82:0/2451386981 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0610105f10 0x7f0610108330 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:33.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:33.521+0000 7f0617337640 1 --2- 172.21.15.82:0/2451386981 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0610108870 0x7f061010ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:33.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:33.521+0000 7f0617337640 1 --2- 172.21.15.82:0/2451386981 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f06101035b0 0x7f06101059d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:33.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:33.521+0000 7f0617337640 1 -- 172.21.15.82:0/2451386981 >> 172.21.15.82:0/2451386981 conn(0x7f06100fcbb0 msgr2=0x7f06100ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:13:33.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:33.521+0000 7f0617337640 1 -- 172.21.15.82:0/2451386981 shutdown_connections 2024-05-20T16:13:33.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:33.521+0000 7f0617337640 1 -- 172.21.15.82:0/2451386981 wait complete. 2024-05-20T16:13:33.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:33.521+0000 7f0617337640 1 Processor -- start 2024-05-20T16:13:33.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:33.521+0000 7f0617337640 1 -- start start 2024-05-20T16:13:33.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:33.522+0000 7f0617337640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f06101035b0 0x7f0610102a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:33.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:33.522+0000 7f0617337640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0610105f10 0x7f06101010a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:33.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:33.522+0000 7f0617337640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0610108870 0x7f06101015e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:33.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:33.522+0000 7f0617337640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f061010e740 con 0x7f0610108870 2024-05-20T16:13:33.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:33.522+0000 7f0617337640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f061010e5c0 con 0x7f0610105f10 2024-05-20T16:13:33.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:33.522+0000 7f0617337640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f061010e8c0 con 0x7f06101035b0 2024-05-20T16:13:33.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:33.522+0000 7f06158d4640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0610108870 0x7f06101015e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:13:33.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:33.522+0000 7f06158d4640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0610108870 0x7f06101015e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:33410/0 (socket says 172.21.15.82:33410) 2024-05-20T16:13:33.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:33.522+0000 7f06158d4640 1 -- 172.21.15.82:0/197577417 learned_addr learned my addr 172.21.15.82:0/197577417 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:13:33.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:33.522+0000 7f06148d2640 1 --2- 172.21.15.82:0/197577417 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0610105f10 0x7f06101010a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:13:33.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:33.522+0000 7f06150d3640 1 --2- 172.21.15.82:0/197577417 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f06101035b0 0x7f0610102a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:13:33.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:33.522+0000 7f06158d4640 1 -- 172.21.15.82:0/197577417 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f06101035b0 msgr2=0x7f0610102a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:33.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:33.522+0000 7f06158d4640 1 --2- 172.21.15.82:0/197577417 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f06101035b0 0x7f0610102a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:33.527 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:33.522+0000 7f06158d4640 1 -- 172.21.15.82:0/197577417 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0610105f10 msgr2=0x7f06101010a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:33.527 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:33.522+0000 7f06158d4640 1 --2- 172.21.15.82:0/197577417 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0610105f10 0x7f06101010a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:33.527 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:33.522+0000 7f06158d4640 1 -- 172.21.15.82:0/197577417 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0610101d10 con 0x7f0610108870 2024-05-20T16:13:33.527 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:33.522+0000 7f06150d3640 1 --2- 172.21.15.82:0/197577417 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f06101035b0 0x7f0610102a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T16:13:33.527 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:33.522+0000 7f06148d2640 1 --2- 172.21.15.82:0/197577417 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0610105f10 0x7f06101010a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T16:13:33.527 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:33.522+0000 7f06158d4640 1 --2- 172.21.15.82:0/197577417 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0610108870 0x7f06101015e0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f060800d950 tx=0x7f060800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:13:33.527 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:33.522+0000 7f060e7fc640 1 -- 172.21.15.82:0/197577417 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0608017070 con 0x7f0610108870 2024-05-20T16:13:33.527 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:33.522+0000 7f060e7fc640 1 -- 172.21.15.82:0/197577417 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0608009c30 con 0x7f0610108870 2024-05-20T16:13:33.527 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:33.522+0000 7f0617337640 1 -- 172.21.15.82:0/197577417 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06101a33e0 con 0x7f0610108870 2024-05-20T16:13:33.527 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:33.522+0000 7f060e7fc640 1 -- 172.21.15.82:0/197577417 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f060800f600 con 0x7f0610108870 2024-05-20T16:13:33.527 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:33.522+0000 7f0617337640 1 -- 172.21.15.82:0/197577417 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06101a3850 con 0x7f0610108870 2024-05-20T16:13:33.527 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:33.524+0000 7f0617337640 1 -- 172.21.15.82:0/197577417 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06100fe6d0 con 0x7f0610108870 2024-05-20T16:13:33.528 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:33.525+0000 7f060e7fc640 1 -- 172.21.15.82:0/197577417 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f060801a070 con 0x7f0610108870 2024-05-20T16:13:33.528 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:33.526+0000 7f060e7fc640 1 --2- 172.21.15.82:0/197577417 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f05f80a2b90 0x7f05f80a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:33.528 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:33.526+0000 7f06150d3640 1 --2- 172.21.15.82:0/197577417 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f05f80a2b90 0x7f05f80a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:13:33.528 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:33.526+0000 7f060e7fc640 1 -- 172.21.15.82:0/197577417 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9092+0+0 (secure 0 0 0) 0x7f06080cee60 con 0x7f0610108870 2024-05-20T16:13:33.529 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:33.526+0000 7f06150d3640 1 --2- 172.21.15.82:0/197577417 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f05f80a2b90 0x7f05f80a50a0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f0600005c40 tx=0x7f06000082f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:13:33.529 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:33.530+0000 7f060e7fc640 1 -- 172.21.15.82:0/197577417 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f0608009de0 con 0x7f0610108870 2024-05-20T16:13:33.706 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:33.706+0000 7f0617337640 1 -- 172.21.15.82:0/197577417 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0610062750 con 0x7f0610108870 2024-05-20T16:13:33.708 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:33.708+0000 7f060e7fc640 1 -- 172.21.15.82:0/197577417 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) ==== 74+0+24621 (secure 0 0 0) 0x7f060809ba60 con 0x7f0610108870 2024-05-20T16:13:33.708 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-05-20T16:13:33.708 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":93,"fsid":"59d1600a-16c2-11ef-bc9a-c7b262605968","created":"2024-05-20T16:03:46.611556+0000","modified":"2024-05-20T16:12:46.250119+0000","last_up_change":"2024-05-20T16:10:59.303479+0000","last_in_change":"2024-05-20T16:10:39.063561+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-20T16:07:09.493727+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-20T16:11:45.058035+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-20T16:11:45.943862+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-20T16:11:49.637579+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"bae842a9-03ec-490e-b56d-f7957235cdc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":1484330167},{"type":"v1","addr":"172.21.15.82:6803","nonce":1484330167}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":1484330167},{"type":"v1","addr":"172.21.15.82:6805","nonce":1484330167}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1484330167},{"type":"v1","addr":"172.21.15.82:6809","nonce":1484330167}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":1484330167},{"type":"v1","addr":"172.21.15.82:6807","nonce":1484330167}]},"public_addr":"172.21.15.82:6803/1484330167","cluster_addr":"172.21.15.82:6805/1484330167","heartbeat_back_addr":"172.21.15.82:6809/1484330167","heartbeat_front_addr":"172.21.15.82:6807/1484330167","state":["exists","up"]},{"osd":1,"uuid":"59f6a2ba-b5de-4161-81c5-92a417abacdf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":1457003568},{"type":"v1","addr":"172.21.15.160:6801","nonce":1457003568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":1457003568},{"type":"v1","addr":"172.21.15.160:6803","nonce":1457003568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6806","nonce":1457003568},{"type":"v1","addr":"172.21.15.160:6807","nonce":1457003568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":1457003568},{"type":"v1","addr":"172.21.15.160:6805","nonce":1457003568}]},"public_addr":"172.21.15.160:6801/1457003568","cluster_addr":"172.21.15.160:6803/1457003568","heartbeat_back_addr":"172.21.15.160:6807/1457003568","heartbeat_front_addr":"172.21.15.160:6805/1457003568","state":["exists","up"]},{"osd":2,"uuid":"3bbb5587-b449-41f3-bd26-67ec3d6afdfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":4051293934},{"type":"v1","addr":"172.21.15.184:6801","nonce":4051293934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":4051293934},{"type":"v1","addr":"172.21.15.184:6803","nonce":4051293934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":4051293934},{"type":"v1","addr":"172.21.15.184:6807","nonce":4051293934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":4051293934},{"type":"v1","addr":"172.21.15.184:6805","nonce":4051293934}]},"public_addr":"172.21.15.184:6801/4051293934","cluster_addr":"172.21.15.184:6803/4051293934","heartbeat_back_addr":"172.21.15.184:6807/4051293934","heartbeat_front_addr":"172.21.15.184:6805/4051293934","state":["exists","up"]},{"osd":3,"uuid":"ca651f90-09b8-4081-a6f9-40b31268a837","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":3947485599},{"type":"v1","addr":"172.21.15.82:6811","nonce":3947485599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":3947485599},{"type":"v1","addr":"172.21.15.82:6813","nonce":3947485599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":3947485599},{"type":"v1","addr":"172.21.15.82:6817","nonce":3947485599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":3947485599},{"type":"v1","addr":"172.21.15.82:6815","nonce":3947485599}]},"public_addr":"172.21.15.82:6811/3947485599","cluster_addr":"172.21.15.82:6813/3947485599","heartbeat_back_addr":"172.21.15.82:6817/3947485599","heartbeat_front_addr":"172.21.15.82:6815/3947485599","state":["exists","up"]},{"osd":4,"uuid":"cb3dfd56-f677-40cb-9c96-8319ea5ac788","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":4160830014},{"type":"v1","addr":"172.21.15.160:6809","nonce":4160830014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":4160830014},{"type":"v1","addr":"172.21.15.160:6811","nonce":4160830014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":4160830014},{"type":"v1","addr":"172.21.15.160:6815","nonce":4160830014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6812","nonce":4160830014},{"type":"v1","addr":"172.21.15.160:6813","nonce":4160830014}]},"public_addr":"172.21.15.160:6809/4160830014","cluster_addr":"172.21.15.160:6811/4160830014","heartbeat_back_addr":"172.21.15.160:6815/4160830014","heartbeat_front_addr":"172.21.15.160:6813/4160830014","state":["exists","up"]},{"osd":5,"uuid":"c70cfcd1-a4ff-461f-8c73-479b061200fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":1046282672},{"type":"v1","addr":"172.21.15.184:6809","nonce":1046282672}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":1046282672},{"type":"v1","addr":"172.21.15.184:6811","nonce":1046282672}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":1046282672},{"type":"v1","addr":"172.21.15.184:6815","nonce":1046282672}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":1046282672},{"type":"v1","addr":"172.21.15.184:6813","nonce":1046282672}]},"public_addr":"172.21.15.184:6809/1046282672","cluster_addr":"172.21.15.184:6811/1046282672","heartbeat_back_addr":"172.21.15.184:6815/1046282672","heartbeat_front_addr":"172.21.15.184:6813/1046282672","state":["exists","up"]},{"osd":6,"uuid":"d47ce001-9dd1-4830-bcc9-888601187c6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":2789736499},{"type":"v1","addr":"172.21.15.82:6819","nonce":2789736499}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":2789736499},{"type":"v1","addr":"172.21.15.82:6821","nonce":2789736499}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":2789736499},{"type":"v1","addr":"172.21.15.82:6825","nonce":2789736499}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":2789736499},{"type":"v1","addr":"172.21.15.82:6823","nonce":2789736499}]},"public_addr":"172.21.15.82:6819/2789736499","cluster_addr":"172.21.15.82:6821/2789736499","heartbeat_back_addr":"172.21.15.82:6825/2789736499","heartbeat_front_addr":"172.21.15.82:6823/2789736499","state":["exists","up"]},{"osd":7,"uuid":"25c54f43-18cb-41e4-83b4-2732b5a93889","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6816","nonce":2340683378},{"type":"v1","addr":"172.21.15.160:6817","nonce":2340683378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":2340683378},{"type":"v1","addr":"172.21.15.160:6819","nonce":2340683378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":2340683378},{"type":"v1","addr":"172.21.15.160:6823","nonce":2340683378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6820","nonce":2340683378},{"type":"v1","addr":"172.21.15.160:6821","nonce":2340683378}]},"public_addr":"172.21.15.160:6817/2340683378","cluster_addr":"172.21.15.160:6819/2340683378","heartbeat_back_addr":"172.21.15.160:6823/2340683378","heartbeat_front_addr":"172.21.15.160:6821/2340683378","state":["exists","up"]},{"osd":8,"uuid":"1f8b478d-287f-45f5-8526-d1a58c5bc07c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":3831576372},{"type":"v1","addr":"172.21.15.184:6817","nonce":3831576372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":3831576372},{"type":"v1","addr":"172.21.15.184:6819","nonce":3831576372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":3831576372},{"type":"v1","addr":"172.21.15.184:6823","nonce":3831576372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":3831576372},{"type":"v1","addr":"172.21.15.184:6821","nonce":3831576372}]},"public_addr":"172.21.15.184:6817/3831576372","cluster_addr":"172.21.15.184:6819/3831576372","heartbeat_back_addr":"172.21.15.184:6823/3831576372","heartbeat_front_addr":"172.21.15.184:6821/3831576372","state":["exists","up"]},{"osd":9,"uuid":"0d69e319-954f-4b18-95ea-31794f18dc3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":716330782},{"type":"v1","addr":"172.21.15.82:6827","nonce":716330782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":716330782},{"type":"v1","addr":"172.21.15.82:6829","nonce":716330782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":716330782},{"type":"v1","addr":"172.21.15.82:6833","nonce":716330782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":716330782},{"type":"v1","addr":"172.21.15.82:6831","nonce":716330782}]},"public_addr":"172.21.15.82:6827/716330782","cluster_addr":"172.21.15.82:6829/716330782","heartbeat_back_addr":"172.21.15.82:6833/716330782","heartbeat_front_addr":"172.21.15.82:6831/716330782","state":["exists","up"]},{"osd":10,"uuid":"39dc1b78-61cb-42ca-b5b6-1e44ac2b6500","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":2327428014},{"type":"v1","addr":"172.21.15.160:6825","nonce":2327428014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":2327428014},{"type":"v1","addr":"172.21.15.160:6827","nonce":2327428014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":2327428014},{"type":"v1","addr":"172.21.15.160:6831","nonce":2327428014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":2327428014},{"type":"v1","addr":"172.21.15.160:6829","nonce":2327428014}]},"public_addr":"172.21.15.160:6825/2327428014","cluster_addr":"172.21.15.160:6827/2327428014","heartbeat_back_addr":"172.21.15.160:6831/2327428014","heartbeat_front_addr":"172.21.15.160:6829/2327428014","state":["exists","up"]},{"osd":11,"uuid":"4306b3c1-707a-4010-987c-209ea0e948e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":3594364650},{"type":"v1","addr":"172.21.15.184:6825","nonce":3594364650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":3594364650},{"type":"v1","addr":"172.21.15.184:6827","nonce":3594364650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3594364650},{"type":"v1","addr":"172.21.15.184:6831","nonce":3594364650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3594364650},{"type":"v1","addr":"172.21.15.184:6829","nonce":3594364650}]},"public_addr":"172.21.15.184:6825/3594364650","cluster_addr":"172.21.15.184:6827/3594364650","heartbeat_back_addr":"172.21.15.184:6831/3594364650","heartbeat_front_addr":"172.21.15.184:6829/3594364650","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:06:17.855998+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:06:41.473270+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:07:05.304108+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:07:29.887472+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:07:55.387286+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:08:20.579669+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:08:46.333051+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:09:12.289668+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:09:38.180696+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:10:04.471459+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:10:31.041912+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:10:57.477266+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.1d","mappings":[{"from":5,"to":9}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.5","mappings":[{"from":9,"to":10}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.82:0/2005424509":"2024-05-21T16:04:45.161949+0000","172.21.15.82:6801/1763049308":"2024-05-21T16:04:45.161949+0000","172.21.15.82:6801/1959459348":"2024-05-21T16:04:19.208291+0000","172.21.15.82:0/3131904556":"2024-05-21T16:04:19.208291+0000","172.21.15.82:6800/1959459348":"2024-05-21T16:04:19.208291+0000","172.21.15.82:0/2456966342":"2024-05-21T16:04:45.161949+0000","172.21.15.82:0/4179978833":"2024-05-21T16:04:19.208291+0000","172.21.15.82:0/1602862225":"2024-05-21T16:04:45.161949+0000","172.21.15.82:0/106101180":"2024-05-21T16:04:19.208291+0000","172.21.15.82:6800/1763049308":"2024-05-21T16:04:45.161949+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-20T16:13:33.710 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:33.710+0000 7f0617337640 1 -- 172.21.15.82:0/197577417 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f05f80a2b90 msgr2=0x7f05f80a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:33.711 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:33.710+0000 7f0617337640 1 --2- 172.21.15.82:0/197577417 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f05f80a2b90 0x7f05f80a50a0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f0600005c40 tx=0x7f06000082f0 comp rx=0 tx=0).stop 2024-05-20T16:13:33.711 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:33.710+0000 7f0617337640 1 -- 172.21.15.82:0/197577417 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0610108870 msgr2=0x7f06101015e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:33.711 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:33.710+0000 7f0617337640 1 --2- 172.21.15.82:0/197577417 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0610108870 0x7f06101015e0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f060800d950 tx=0x7f060800de10 comp rx=0 tx=0).stop 2024-05-20T16:13:33.711 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:33.710+0000 7f0617337640 1 -- 172.21.15.82:0/197577417 shutdown_connections 2024-05-20T16:13:33.711 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:33.710+0000 7f0617337640 1 --2- 172.21.15.82:0/197577417 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f05f80a2b90 0x7f05f80a50a0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:33.711 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:33.710+0000 7f0617337640 1 --2- 172.21.15.82:0/197577417 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0610108870 0x7f06101015e0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:33.711 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:33.710+0000 7f0617337640 1 --2- 172.21.15.82:0/197577417 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0610105f10 0x7f06101010a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:33.711 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:33.710+0000 7f0617337640 1 --2- 172.21.15.82:0/197577417 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f06101035b0 0x7f0610102a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:33.711 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:33.710+0000 7f0617337640 1 -- 172.21.15.82:0/197577417 >> 172.21.15.82:0/197577417 conn(0x7f06100fcbb0 msgr2=0x7f0610104180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:13:33.711 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:33.710+0000 7f0617337640 1 -- 172.21.15.82:0/197577417 shutdown_connections 2024-05-20T16:13:33.711 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:33.710+0000 7f0617337640 1 -- 172.21.15.82:0/197577417 wait complete. 2024-05-20T16:13:33.722 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-20T16:07:09.493727+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-20T16:11:45.058035+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-20T16:11:45.943862+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-20T16:11:49.637579+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '91', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-20T16:13:33.723 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-20T16:13:33.838 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:33.838+0000 7f36634bc640 1 -- 172.21.15.82:0/556723562 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f365c1078d0 msgr2=0x7f365c109cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:33.838 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:33.838+0000 7f36634bc640 1 --2- 172.21.15.82:0/556723562 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f365c1078d0 0x7f365c109cf0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f3654009ca0 tx=0x7f365402a8e0 comp rx=0 tx=0).stop 2024-05-20T16:13:33.838 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:33.838+0000 7f36634bc640 1 -- 172.21.15.82:0/556723562 shutdown_connections 2024-05-20T16:13:33.838 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:33.838+0000 7f36634bc640 1 --2- 172.21.15.82:0/556723562 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f365c1078d0 0x7f365c109cf0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:33.838 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:33.838+0000 7f36634bc640 1 --2- 172.21.15.82:0/556723562 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f365c102bd0 0x7f365c102fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:33.838 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:33.838+0000 7f36634bc640 1 --2- 172.21.15.82:0/556723562 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f365c101990 0x7f365c102690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:33.838 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:33.838+0000 7f36634bc640 1 -- 172.21.15.82:0/556723562 >> 172.21.15.82:0/556723562 conn(0x7f365c0fcb80 msgr2=0x7f365c0ff030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:13:33.838 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:33.838+0000 7f36634bc640 1 -- 172.21.15.82:0/556723562 shutdown_connections 2024-05-20T16:13:33.838 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:33.838+0000 7f36634bc640 1 -- 172.21.15.82:0/556723562 wait complete. 2024-05-20T16:13:33.841 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:33.839+0000 7f36634bc640 1 Processor -- start 2024-05-20T16:13:33.841 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:33.839+0000 7f36634bc640 1 -- start start 2024-05-20T16:13:33.841 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:33.839+0000 7f36634bc640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f365c101990 0x7f365c0698d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:33.841 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:33.839+0000 7f36634bc640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f365c102bd0 0x7f365c19e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:33.841 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:33.839+0000 7f36634bc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f365c1078d0 0x7f365c1a08e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:33.841 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:33.839+0000 7f36634bc640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f365c10d680 con 0x7f365c1078d0 2024-05-20T16:13:33.841 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:33.839+0000 7f36634bc640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f365c10d500 con 0x7f365c102bd0 2024-05-20T16:13:33.841 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:33.839+0000 7f36634bc640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f365c10d800 con 0x7f365c101990 2024-05-20T16:13:33.841 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:33.839+0000 7f3661a59640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f365c1078d0 0x7f365c1a08e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:13:33.841 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:33.839+0000 7f3661a59640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f365c1078d0 0x7f365c1a08e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:33418/0 (socket says 172.21.15.82:33418) 2024-05-20T16:13:33.841 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:33.839+0000 7f3661a59640 1 -- 172.21.15.82:0/2516336710 learned_addr learned my addr 172.21.15.82:0/2516336710 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:13:33.841 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:33.839+0000 7f3661258640 1 --2- 172.21.15.82:0/2516336710 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f365c101990 0x7f365c0698d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:13:33.842 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:33.839+0000 7f3660a57640 1 --2- 172.21.15.82:0/2516336710 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f365c102bd0 0x7f365c19e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:13:33.842 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:33.839+0000 7f3661a59640 1 -- 172.21.15.82:0/2516336710 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f365c101990 msgr2=0x7f365c0698d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:33.842 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:33.839+0000 7f3661a59640 1 --2- 172.21.15.82:0/2516336710 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f365c101990 0x7f365c0698d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:33.842 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:33.839+0000 7f3661a59640 1 -- 172.21.15.82:0/2516336710 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f365c102bd0 msgr2=0x7f365c19e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:33.842 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:33.839+0000 7f3661a59640 1 --2- 172.21.15.82:0/2516336710 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f365c102bd0 0x7f365c19e370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:33.842 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:33.839+0000 7f3661a59640 1 -- 172.21.15.82:0/2516336710 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f365c1a1100 con 0x7f365c1078d0 2024-05-20T16:13:33.842 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:33.839+0000 7f3660a57640 1 --2- 172.21.15.82:0/2516336710 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f365c102bd0 0x7f365c19e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T16:13:33.842 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:33.839+0000 7f3661a59640 1 --2- 172.21.15.82:0/2516336710 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f365c1078d0 0x7f365c1a08e0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f3654009dc0 tx=0x7f3654003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:13:33.842 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:33.840+0000 7f36527fc640 1 -- 172.21.15.82:0/2516336710 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3654004240 con 0x7f365c1078d0 2024-05-20T16:13:33.842 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:33.840+0000 7f36527fc640 1 -- 172.21.15.82:0/2516336710 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3654003e40 con 0x7f365c1078d0 2024-05-20T16:13:33.842 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:33.840+0000 7f36527fc640 1 -- 172.21.15.82:0/2516336710 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f365403b400 con 0x7f365c1078d0 2024-05-20T16:13:33.842 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:33.840+0000 7f36634bc640 1 -- 172.21.15.82:0/2516336710 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f365c1a1390 con 0x7f365c1078d0 2024-05-20T16:13:33.842 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:33.840+0000 7f36634bc640 1 -- 172.21.15.82:0/2516336710 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f365c1a1910 con 0x7f365c1078d0 2024-05-20T16:13:33.842 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:33.841+0000 7f36634bc640 1 -- 172.21.15.82:0/2516336710 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f365c1066e0 con 0x7f365c1078d0 2024-05-20T16:13:33.843 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:33.842+0000 7f36527fc640 1 -- 172.21.15.82:0/2516336710 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f36540043e0 con 0x7f365c1078d0 2024-05-20T16:13:33.843 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:33.843+0000 7f36527fc640 1 --2- 172.21.15.82:0/2516336710 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f36480a2c10 0x7f36480a5120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:33.844 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:33.843+0000 7f3661258640 1 --2- 172.21.15.82:0/2516336710 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f36480a2c10 0x7f36480a5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:13:33.844 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:33.844+0000 7f36527fc640 1 -- 172.21.15.82:0/2516336710 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9092+0+0 (secure 0 0 0) 0x7f36540f0b00 con 0x7f365c1078d0 2024-05-20T16:13:33.844 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:33.844+0000 7f3661258640 1 --2- 172.21.15.82:0/2516336710 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f36480a2c10 0x7f36480a5120 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f365c102590 tx=0x7f3658009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:13:33.847 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:33.847+0000 7f36527fc640 1 -- 172.21.15.82:0/2516336710 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f36540bd700 con 0x7f365c1078d0 2024-05-20T16:13:34.023 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:34.023+0000 7f36634bc640 1 -- 172.21.15.82:0/2516336710 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f365c02d480 con 0x7f365c1078d0 2024-05-20T16:13:34.024 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:34.024+0000 7f36527fc640 1 -- 172.21.15.82:0/2516336710 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v93) ==== 93+0+10 (secure 0 0 0) 0x7f36540dd0c0 con 0x7f365c1078d0 2024-05-20T16:13:34.024 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 1 2024-05-20T16:13:34.026 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:34.026+0000 7f36634bc640 1 -- 172.21.15.82:0/2516336710 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f36480a2c10 msgr2=0x7f36480a5120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:34.026 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:34.026+0000 7f36634bc640 1 --2- 172.21.15.82:0/2516336710 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f36480a2c10 0x7f36480a5120 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f365c102590 tx=0x7f3658009110 comp rx=0 tx=0).stop 2024-05-20T16:13:34.026 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:34.026+0000 7f36634bc640 1 -- 172.21.15.82:0/2516336710 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f365c1078d0 msgr2=0x7f365c1a08e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:34.026 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:34.026+0000 7f36634bc640 1 --2- 172.21.15.82:0/2516336710 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f365c1078d0 0x7f365c1a08e0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f3654009dc0 tx=0x7f3654003c20 comp rx=0 tx=0).stop 2024-05-20T16:13:34.026 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:34.026+0000 7f36634bc640 1 -- 172.21.15.82:0/2516336710 shutdown_connections 2024-05-20T16:13:34.026 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:34.026+0000 7f36634bc640 1 --2- 172.21.15.82:0/2516336710 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f36480a2c10 0x7f36480a5120 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:34.026 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:34.026+0000 7f36634bc640 1 --2- 172.21.15.82:0/2516336710 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f365c1078d0 0x7f365c1a08e0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:34.026 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:34.026+0000 7f36634bc640 1 --2- 172.21.15.82:0/2516336710 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f365c102bd0 0x7f365c19e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:34.027 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:34.026+0000 7f36634bc640 1 --2- 172.21.15.82:0/2516336710 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f365c101990 0x7f365c0698d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:34.027 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:34.026+0000 7f36634bc640 1 -- 172.21.15.82:0/2516336710 >> 172.21.15.82:0/2516336710 conn(0x7f365c0fcb80 msgr2=0x7f365c1038b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:13:34.027 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:34.026+0000 7f36634bc640 1 -- 172.21.15.82:0/2516336710 shutdown_connections 2024-05-20T16:13:34.027 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:34.026+0000 7f36634bc640 1 -- 172.21.15.82:0/2516336710 wait complete. 2024-05-20T16:13:34.036 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:13:33 smithi082 ceph-mon[32238]: from='client.? 172.21.15.82:0/3249222612' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-20T16:13:34.036 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:13:33 smithi082 ceph-mon[32238]: pgmap v347: 129 pgs: 129 active+clean; 641 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-20T16:13:34.036 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:13:33 smithi082 ceph-mon[32238]: from='client.? 172.21.15.82:0/2889764950' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T16:13:34.037 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:13:33 smithi082 ceph-mon[32238]: from='client.? 172.21.15.82:0/197577417' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-20T16:13:34.038 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-20T16:13:34.062 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:13:33 smithi160 ceph-mon[36347]: from='client.? 172.21.15.82:0/3249222612' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-20T16:13:34.062 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:13:33 smithi160 ceph-mon[36347]: pgmap v347: 129 pgs: 129 active+clean; 641 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-20T16:13:34.062 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:13:33 smithi160 ceph-mon[36347]: from='client.? 172.21.15.82:0/2889764950' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T16:13:34.062 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:13:33 smithi160 ceph-mon[36347]: from='client.? 172.21.15.82:0/197577417' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-20T16:13:34.136 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:13:33 smithi184 ceph-mon[36209]: from='client.? 172.21.15.82:0/3249222612' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-20T16:13:34.136 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:13:33 smithi184 ceph-mon[36209]: pgmap v347: 129 pgs: 129 active+clean; 641 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-20T16:13:34.136 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:13:33 smithi184 ceph-mon[36209]: from='client.? 172.21.15.82:0/2889764950' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T16:13:34.136 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:13:33 smithi184 ceph-mon[36209]: from='client.? 172.21.15.82:0/197577417' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-20T16:13:34.152 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:34.152+0000 7fc9ada1b640 1 -- 172.21.15.82:0/2068464040 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc9a8103590 msgr2=0x7fc9a81059b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:34.152 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:34.152+0000 7fc9ada1b640 1 --2- 172.21.15.82:0/2068464040 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc9a8103590 0x7fc9a81059b0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fc998009ca0 tx=0x7fc99802a8f0 comp rx=0 tx=0).stop 2024-05-20T16:13:34.152 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:34.152+0000 7fc9ada1b640 1 -- 172.21.15.82:0/2068464040 shutdown_connections 2024-05-20T16:13:34.152 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:34.152+0000 7fc9ada1b640 1 --2- 172.21.15.82:0/2068464040 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc9a8108850 0x7fc9a810ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:34.152 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:34.152+0000 7fc9ada1b640 1 --2- 172.21.15.82:0/2068464040 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc9a8105ef0 0x7fc9a8108310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:34.152 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:34.152+0000 7fc9ada1b640 1 --2- 172.21.15.82:0/2068464040 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc9a8103590 0x7fc9a81059b0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:34.152 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:34.152+0000 7fc9ada1b640 1 -- 172.21.15.82:0/2068464040 >> 172.21.15.82:0/2068464040 conn(0x7fc9a80fcb70 msgr2=0x7fc9a80ff040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:13:34.152 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:34.152+0000 7fc9ada1b640 1 -- 172.21.15.82:0/2068464040 shutdown_connections 2024-05-20T16:13:34.152 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:34.152+0000 7fc9ada1b640 1 -- 172.21.15.82:0/2068464040 wait complete. 2024-05-20T16:13:34.155 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:34.152+0000 7fc9ada1b640 1 Processor -- start 2024-05-20T16:13:34.155 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:34.153+0000 7fc9ada1b640 1 -- start start 2024-05-20T16:13:34.155 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:34.153+0000 7fc9ada1b640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc9a8103590 0x7fc9a8199450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:34.155 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:34.153+0000 7fc9ada1b640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc9a8105ef0 0x7fc9a8199990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:34.155 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:34.153+0000 7fc9ada1b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc9a8108850 0x7fc9a81a0920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:34.155 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:34.153+0000 7fc9ada1b640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fc9a810e810 con 0x7fc9a8108850 2024-05-20T16:13:34.155 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:34.153+0000 7fc9ada1b640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fc9a810e690 con 0x7fc9a8105ef0 2024-05-20T16:13:34.155 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:34.153+0000 7fc9ada1b640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fc9a810e990 con 0x7fc9a8103590 2024-05-20T16:13:34.155 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:34.153+0000 7fc9a77fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc9a8108850 0x7fc9a81a0920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:13:34.155 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:34.153+0000 7fc9a77fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc9a8108850 0x7fc9a81a0920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:33440/0 (socket says 172.21.15.82:33440) 2024-05-20T16:13:34.155 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:34.153+0000 7fc9a77fe640 1 -- 172.21.15.82:0/3605438170 learned_addr learned my addr 172.21.15.82:0/3605438170 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:13:34.155 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:34.153+0000 7fc9a6ffd640 1 --2- 172.21.15.82:0/3605438170 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc9a8103590 0x7fc9a8199450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:13:34.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:34.153+0000 7fc9a67fc640 1 --2- 172.21.15.82:0/3605438170 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc9a8105ef0 0x7fc9a8199990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:13:34.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:34.153+0000 7fc9a77fe640 1 -- 172.21.15.82:0/3605438170 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc9a8103590 msgr2=0x7fc9a8199450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:34.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:34.153+0000 7fc9a77fe640 1 --2- 172.21.15.82:0/3605438170 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc9a8103590 0x7fc9a8199450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:34.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:34.153+0000 7fc9a77fe640 1 -- 172.21.15.82:0/3605438170 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc9a8105ef0 msgr2=0x7fc9a8199990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:34.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:34.153+0000 7fc9a77fe640 1 --2- 172.21.15.82:0/3605438170 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc9a8105ef0 0x7fc9a8199990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:34.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:34.153+0000 7fc9a77fe640 1 -- 172.21.15.82:0/3605438170 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9a81a0f90 con 0x7fc9a8108850 2024-05-20T16:13:34.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:34.154+0000 7fc9a77fe640 1 --2- 172.21.15.82:0/3605438170 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc9a8108850 0x7fc9a81a0920 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fc9a000b3c0 tx=0x7fc9a000b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:13:34.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:34.154+0000 7fc9aca19640 1 -- 172.21.15.82:0/3605438170 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9a0011830 con 0x7fc9a8108850 2024-05-20T16:13:34.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:34.154+0000 7fc9aca19640 1 -- 172.21.15.82:0/3605438170 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc9a0016070 con 0x7fc9a8108850 2024-05-20T16:13:34.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:34.154+0000 7fc9ada1b640 1 -- 172.21.15.82:0/3605438170 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9a81a1280 con 0x7fc9a8108850 2024-05-20T16:13:34.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:34.154+0000 7fc9aca19640 1 -- 172.21.15.82:0/3605438170 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9a000f2e0 con 0x7fc9a8108850 2024-05-20T16:13:34.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:34.154+0000 7fc9ada1b640 1 -- 172.21.15.82:0/3605438170 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9a81a1800 con 0x7fc9a8108850 2024-05-20T16:13:34.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:34.155+0000 7fc9ada1b640 1 -- 172.21.15.82:0/3605438170 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9a80fe690 con 0x7fc9a8108850 2024-05-20T16:13:34.157 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:34.156+0000 7fc9aca19640 1 -- 172.21.15.82:0/3605438170 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fc9a00119d0 con 0x7fc9a8108850 2024-05-20T16:13:34.157 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:34.157+0000 7fc9aca19640 1 --2- 172.21.15.82:0/3605438170 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fc9900a2b90 0x7fc9900a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:34.157 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:34.157+0000 7fc9a6ffd640 1 --2- 172.21.15.82:0/3605438170 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fc9900a2b90 0x7fc9900a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:13:34.157 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:34.157+0000 7fc9aca19640 1 -- 172.21.15.82:0/3605438170 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9092+0+0 (secure 0 0 0) 0x7fc9a00cf010 con 0x7fc9a8108850 2024-05-20T16:13:34.158 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:34.158+0000 7fc9a6ffd640 1 --2- 172.21.15.82:0/3605438170 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fc9900a2b90 0x7fc9900a50a0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fc998000f80 tx=0x7fc998009480 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:13:34.161 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:34.161+0000 7fc9aca19640 1 -- 172.21.15.82:0/3605438170 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fc9a009bc10 con 0x7fc9a8108850 2024-05-20T16:13:34.338 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:34.338+0000 7fc9ada1b640 1 -- 172.21.15.82:0/3605438170 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fc9a8062750 con 0x7fc9a8108850 2024-05-20T16:13:34.338 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:34.338+0000 7fc9aca19640 1 -- 172.21.15.82:0/3605438170 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v93) ==== 104+0+11 (secure 0 0 0) 0x7fc9a009b510 con 0x7fc9a8108850 2024-05-20T16:13:34.338 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 32 2024-05-20T16:13:34.340 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:34.340+0000 7fc9ada1b640 1 -- 172.21.15.82:0/3605438170 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fc9900a2b90 msgr2=0x7fc9900a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:34.340 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:34.340+0000 7fc9ada1b640 1 --2- 172.21.15.82:0/3605438170 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fc9900a2b90 0x7fc9900a50a0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fc998000f80 tx=0x7fc998009480 comp rx=0 tx=0).stop 2024-05-20T16:13:34.340 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:34.340+0000 7fc9ada1b640 1 -- 172.21.15.82:0/3605438170 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc9a8108850 msgr2=0x7fc9a81a0920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:34.340 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:34.340+0000 7fc9ada1b640 1 --2- 172.21.15.82:0/3605438170 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc9a8108850 0x7fc9a81a0920 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fc9a000b3c0 tx=0x7fc9a000b880 comp rx=0 tx=0).stop 2024-05-20T16:13:34.340 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:34.340+0000 7fc9ada1b640 1 -- 172.21.15.82:0/3605438170 shutdown_connections 2024-05-20T16:13:34.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:34.340+0000 7fc9ada1b640 1 --2- 172.21.15.82:0/3605438170 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fc9900a2b90 0x7fc9900a50a0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:34.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:34.340+0000 7fc9ada1b640 1 --2- 172.21.15.82:0/3605438170 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc9a8108850 0x7fc9a81a0920 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:34.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:34.340+0000 7fc9ada1b640 1 --2- 172.21.15.82:0/3605438170 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc9a8105ef0 0x7fc9a8199990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:34.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:34.340+0000 7fc9ada1b640 1 --2- 172.21.15.82:0/3605438170 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc9a8103590 0x7fc9a8199450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:34.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:34.340+0000 7fc9ada1b640 1 -- 172.21.15.82:0/3605438170 >> 172.21.15.82:0/3605438170 conn(0x7fc9a80fcb70 msgr2=0x7fc9a81045d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:13:34.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:34.340+0000 7fc9ada1b640 1 -- 172.21.15.82:0/3605438170 shutdown_connections 2024-05-20T16:13:34.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:34.340+0000 7fc9ada1b640 1 -- 172.21.15.82:0/3605438170 wait complete. 2024-05-20T16:13:34.352 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-20T16:13:34.468 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:34.466+0000 7f28add45640 1 -- 172.21.15.82:0/3713979146 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f28a8102ea0 msgr2=0x7f28a8101780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:34.468 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:34.466+0000 7f28add45640 1 --2- 172.21.15.82:0/3713979146 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f28a8102ea0 0x7f28a8101780 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f2898009ca0 tx=0x7f289802a8c0 comp rx=0 tx=0).stop 2024-05-20T16:13:34.468 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:34.466+0000 7f28add45640 1 -- 172.21.15.82:0/3713979146 shutdown_connections 2024-05-20T16:13:34.468 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:34.466+0000 7f28add45640 1 --2- 172.21.15.82:0/3713979146 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f28a81079d0 0x7f28a8109df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:34.468 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:34.466+0000 7f28add45640 1 --2- 172.21.15.82:0/3713979146 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f28a8103380 0x7f28a8101d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:34.468 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:34.466+0000 7f28add45640 1 --2- 172.21.15.82:0/3713979146 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f28a8102ea0 0x7f28a8101780 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:34.469 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:34.466+0000 7f28add45640 1 -- 172.21.15.82:0/3713979146 >> 172.21.15.82:0/3713979146 conn(0x7f28a80fc300 msgr2=0x7f28a80fe7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:13:34.469 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:34.466+0000 7f28add45640 1 -- 172.21.15.82:0/3713979146 shutdown_connections 2024-05-20T16:13:34.469 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:34.466+0000 7f28add45640 1 -- 172.21.15.82:0/3713979146 wait complete. 2024-05-20T16:13:34.469 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:34.466+0000 7f28add45640 1 Processor -- start 2024-05-20T16:13:34.469 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:34.467+0000 7f28add45640 1 -- start start 2024-05-20T16:13:34.469 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:34.467+0000 7f28add45640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f28a8102ea0 0x7f28a810c280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:34.469 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:34.467+0000 7f28add45640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f28a81079d0 0x7f28a810a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:34.469 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:34.467+0000 7f28add45640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f28a810ae60 0x7f28a810b270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:34.469 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:34.467+0000 7f28add45640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f28a810e730 con 0x7f28a81079d0 2024-05-20T16:13:34.469 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:34.467+0000 7f28add45640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f28a810e5b0 con 0x7f28a810ae60 2024-05-20T16:13:34.469 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:34.467+0000 7f28add45640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f28a810e8b0 con 0x7f28a8102ea0 2024-05-20T16:13:34.469 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:34.467+0000 7f28a6ffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f28a81079d0 0x7f28a810a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:13:34.469 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:34.467+0000 7f28a6ffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f28a81079d0 0x7f28a810a920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:33472/0 (socket says 172.21.15.82:33472) 2024-05-20T16:13:34.469 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:34.467+0000 7f28a6ffd640 1 -- 172.21.15.82:0/2087672772 learned_addr learned my addr 172.21.15.82:0/2087672772 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:13:34.470 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:34.467+0000 7f28a77fe640 1 --2- 172.21.15.82:0/2087672772 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f28a8102ea0 0x7f28a810c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:13:34.470 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:34.467+0000 7f28a7fff640 1 --2- 172.21.15.82:0/2087672772 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f28a810ae60 0x7f28a810b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:13:34.470 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:34.467+0000 7f28a6ffd640 1 -- 172.21.15.82:0/2087672772 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f28a8102ea0 msgr2=0x7f28a810c280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:34.470 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:34.467+0000 7f28a6ffd640 1 --2- 172.21.15.82:0/2087672772 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f28a8102ea0 0x7f28a810c280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:34.470 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:34.467+0000 7f28a6ffd640 1 -- 172.21.15.82:0/2087672772 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f28a810ae60 msgr2=0x7f28a810b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:34.470 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:34.467+0000 7f28a6ffd640 1 --2- 172.21.15.82:0/2087672772 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f28a810ae60 0x7f28a810b270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:34.470 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:34.467+0000 7f28a6ffd640 1 -- 172.21.15.82:0/2087672772 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28a81a2fb0 con 0x7f28a81079d0 2024-05-20T16:13:34.470 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:34.468+0000 7f28a6ffd640 1 --2- 172.21.15.82:0/2087672772 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f28a81079d0 0x7f28a810a920 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f289c00ea20 tx=0x7f289c00ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:13:34.470 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:34.468+0000 7f28a4ff9640 1 -- 172.21.15.82:0/2087672772 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f289c00cd80 con 0x7f28a81079d0 2024-05-20T16:13:34.470 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:34.468+0000 7f28a4ff9640 1 -- 172.21.15.82:0/2087672772 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f289c005500 con 0x7f28a81079d0 2024-05-20T16:13:34.470 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:34.468+0000 7f28add45640 1 -- 172.21.15.82:0/2087672772 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28a81a32a0 con 0x7f28a81079d0 2024-05-20T16:13:34.470 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:34.468+0000 7f28a4ff9640 1 -- 172.21.15.82:0/2087672772 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f289c010690 con 0x7f28a81079d0 2024-05-20T16:13:34.470 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:34.468+0000 7f28add45640 1 -- 172.21.15.82:0/2087672772 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28a81a37a0 con 0x7f28a81079d0 2024-05-20T16:13:34.471 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:34.469+0000 7f28add45640 1 -- 172.21.15.82:0/2087672772 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28a81060d0 con 0x7f28a81079d0 2024-05-20T16:13:34.471 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:34.470+0000 7f28a4ff9640 1 -- 172.21.15.82:0/2087672772 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f289c0046c0 con 0x7f28a81079d0 2024-05-20T16:13:34.472 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:34.471+0000 7f28a4ff9640 1 --2- 172.21.15.82:0/2087672772 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f28900a2b90 0x7f28900a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:34.472 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:34.471+0000 7f28a77fe640 1 --2- 172.21.15.82:0/2087672772 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f28900a2b90 0x7f28900a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:13:34.472 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:34.471+0000 7f28a4ff9640 1 -- 172.21.15.82:0/2087672772 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9092+0+0 (secure 0 0 0) 0x7f289c0cf9b0 con 0x7f28a81079d0 2024-05-20T16:13:34.472 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:34.471+0000 7f28a77fe640 1 --2- 172.21.15.82:0/2087672772 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f28900a2b90 0x7f28900a50a0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f2898031c10 tx=0x7f2898005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:13:34.473 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:34.474+0000 7f28a4ff9640 1 -- 172.21.15.82:0/2087672772 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f289c053850 con 0x7f28a81079d0 2024-05-20T16:13:34.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:34.651+0000 7f28add45640 1 -- 172.21.15.82:0/2087672772 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f28a8062750 con 0x7f28a81079d0 2024-05-20T16:13:34.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:34.652+0000 7f28a4ff9640 1 -- 172.21.15.82:0/2087672772 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v93) ==== 100+0+11 (secure 0 0 0) 0x7f289c09c5b0 con 0x7f28a81079d0 2024-05-20T16:13:34.652 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 64 2024-05-20T16:13:34.654 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:34.654+0000 7f28add45640 1 -- 172.21.15.82:0/2087672772 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f28900a2b90 msgr2=0x7f28900a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:34.654 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:34.654+0000 7f28add45640 1 --2- 172.21.15.82:0/2087672772 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f28900a2b90 0x7f28900a50a0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f2898031c10 tx=0x7f2898005fd0 comp rx=0 tx=0).stop 2024-05-20T16:13:34.654 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:34.654+0000 7f28add45640 1 -- 172.21.15.82:0/2087672772 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f28a81079d0 msgr2=0x7f28a810a920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:34.654 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:34.654+0000 7f28add45640 1 --2- 172.21.15.82:0/2087672772 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f28a81079d0 0x7f28a810a920 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f289c00ea20 tx=0x7f289c00ee20 comp rx=0 tx=0).stop 2024-05-20T16:13:34.654 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:34.654+0000 7f28add45640 1 -- 172.21.15.82:0/2087672772 shutdown_connections 2024-05-20T16:13:34.654 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:34.654+0000 7f28add45640 1 --2- 172.21.15.82:0/2087672772 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f28900a2b90 0x7f28900a50a0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:34.654 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:34.654+0000 7f28add45640 1 --2- 172.21.15.82:0/2087672772 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f28a81079d0 0x7f28a810a920 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:34.654 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:34.654+0000 7f28add45640 1 --2- 172.21.15.82:0/2087672772 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f28a810ae60 0x7f28a810b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:34.654 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:34.654+0000 7f28add45640 1 --2- 172.21.15.82:0/2087672772 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f28a8102ea0 0x7f28a810c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:34.655 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:34.654+0000 7f28add45640 1 -- 172.21.15.82:0/2087672772 >> 172.21.15.82:0/2087672772 conn(0x7f28a80fc300 msgr2=0x7f28a80fe720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:13:34.655 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:34.654+0000 7f28add45640 1 -- 172.21.15.82:0/2087672772 shutdown_connections 2024-05-20T16:13:34.655 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:34.654+0000 7f28add45640 1 -- 172.21.15.82:0/2087672772 wait complete. 2024-05-20T16:13:34.665 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-20T16:13:34.781 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:34.779+0000 7fd0cf102640 1 -- 172.21.15.82:0/490926054 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd0c8102bd0 msgr2=0x7fd0c8102fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:34.781 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:34.779+0000 7fd0cf102640 1 --2- 172.21.15.82:0/490926054 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd0c8102bd0 0x7fd0c8102fe0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fd0b4009900 tx=0x7fd0b402a410 comp rx=0 tx=0).stop 2024-05-20T16:13:34.781 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:34.779+0000 7fd0cf102640 1 -- 172.21.15.82:0/490926054 shutdown_connections 2024-05-20T16:13:34.781 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:34.779+0000 7fd0cf102640 1 --2- 172.21.15.82:0/490926054 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd0c8102bd0 0x7fd0c8102fe0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:34.781 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:34.779+0000 7fd0cf102640 1 --2- 172.21.15.82:0/490926054 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd0c81078d0 0x7fd0c8109cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:34.781 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:34.779+0000 7fd0cf102640 1 --2- 172.21.15.82:0/490926054 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd0c8101990 0x7fd0c8102690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:34.781 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:34.779+0000 7fd0cf102640 1 -- 172.21.15.82:0/490926054 >> 172.21.15.82:0/490926054 conn(0x7fd0c80fcb80 msgr2=0x7fd0c80ff030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:13:34.781 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:34.779+0000 7fd0cf102640 1 -- 172.21.15.82:0/490926054 shutdown_connections 2024-05-20T16:13:34.782 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:34.779+0000 7fd0cf102640 1 -- 172.21.15.82:0/490926054 wait complete. 2024-05-20T16:13:34.782 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:34.780+0000 7fd0cf102640 1 Processor -- start 2024-05-20T16:13:34.782 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:34.780+0000 7fd0cf102640 1 -- start start 2024-05-20T16:13:34.782 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:34.780+0000 7fd0cf102640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd0c8101990 0x7fd0c81984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:34.782 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:34.780+0000 7fd0cf102640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd0c8102bd0 0x7fd0c8198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:34.782 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:34.780+0000 7fd0cf102640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd0c81078d0 0x7fd0c819f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:34.782 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:34.780+0000 7fd0cf102640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fd0c810d600 con 0x7fd0c8102bd0 2024-05-20T16:13:34.782 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:34.780+0000 7fd0cf102640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fd0c810d480 con 0x7fd0c8101990 2024-05-20T16:13:34.782 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:34.780+0000 7fd0cf102640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd0c810d780 con 0x7fd0c81078d0 2024-05-20T16:13:34.782 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:34.780+0000 7fd0c7fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd0c8102bd0 0x7fd0c8198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:13:34.782 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:34.780+0000 7fd0c7fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd0c8102bd0 0x7fd0c8198a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:33490/0 (socket says 172.21.15.82:33490) 2024-05-20T16:13:34.782 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:34.780+0000 7fd0c7fff640 1 -- 172.21.15.82:0/4021219936 learned_addr learned my addr 172.21.15.82:0/4021219936 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:13:34.782 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:34.780+0000 7fd0cd69f640 1 --2- 172.21.15.82:0/4021219936 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd0c81078d0 0x7fd0c819f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:13:34.783 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:34.780+0000 7fd0cce9e640 1 --2- 172.21.15.82:0/4021219936 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd0c8101990 0x7fd0c81984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:13:34.783 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:34.780+0000 7fd0c7fff640 1 -- 172.21.15.82:0/4021219936 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd0c81078d0 msgr2=0x7fd0c819f9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:34.783 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:34.780+0000 7fd0c7fff640 1 --2- 172.21.15.82:0/4021219936 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd0c81078d0 0x7fd0c819f9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:34.783 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:34.780+0000 7fd0c7fff640 1 -- 172.21.15.82:0/4021219936 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd0c8101990 msgr2=0x7fd0c81984d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:34.783 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:34.780+0000 7fd0c7fff640 1 --2- 172.21.15.82:0/4021219936 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd0c8101990 0x7fd0c81984d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:34.783 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:34.780+0000 7fd0c7fff640 1 -- 172.21.15.82:0/4021219936 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0c81a0010 con 0x7fd0c8102bd0 2024-05-20T16:13:34.783 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:34.780+0000 7fd0cce9e640 1 --2- 172.21.15.82:0/4021219936 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd0c8101990 0x7fd0c81984d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T16:13:34.783 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:34.780+0000 7fd0c7fff640 1 --2- 172.21.15.82:0/4021219936 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd0c8102bd0 0x7fd0c8198a10 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fd0b4006010 tx=0x7fd0b4004360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:13:34.783 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:34.781+0000 7fd0c5ffb640 1 -- 172.21.15.82:0/4021219936 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0b403f030 con 0x7fd0c8102bd0 2024-05-20T16:13:34.783 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:34.781+0000 7fd0c5ffb640 1 -- 172.21.15.82:0/4021219936 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd0b4003ce0 con 0x7fd0c8102bd0 2024-05-20T16:13:34.783 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:34.781+0000 7fd0cf102640 1 -- 172.21.15.82:0/4021219936 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0c81a02a0 con 0x7fd0c8102bd0 2024-05-20T16:13:34.783 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:34.781+0000 7fd0c5ffb640 1 -- 172.21.15.82:0/4021219936 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0b4035070 con 0x7fd0c8102bd0 2024-05-20T16:13:34.783 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:34.781+0000 7fd0cf102640 1 -- 172.21.15.82:0/4021219936 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0c81a0790 con 0x7fd0c8102bd0 2024-05-20T16:13:34.784 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:34.782+0000 7fd0cf102640 1 -- 172.21.15.82:0/4021219936 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0c81066e0 con 0x7fd0c8102bd0 2024-05-20T16:13:34.784 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:34.783+0000 7fd0c5ffb640 1 -- 172.21.15.82:0/4021219936 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fd0b4042030 con 0x7fd0c8102bd0 2024-05-20T16:13:34.784 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:34.784+0000 7fd0c5ffb640 1 --2- 172.21.15.82:0/4021219936 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fd0b80a2b90 0x7fd0b80a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:34.785 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:34.784+0000 7fd0cce9e640 1 --2- 172.21.15.82:0/4021219936 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fd0b80a2b90 0x7fd0b80a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:13:34.785 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:34.784+0000 7fd0c5ffb640 1 -- 172.21.15.82:0/4021219936 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9092+0+0 (secure 0 0 0) 0x7fd0b40f0060 con 0x7fd0c8102bd0 2024-05-20T16:13:34.785 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:34.785+0000 7fd0cce9e640 1 --2- 172.21.15.82:0/4021219936 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fd0b80a2b90 0x7fd0b80a50a0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fd0c8102590 tx=0x7fd0bc005e70 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:13:34.787 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:34.787+0000 7fd0c5ffb640 1 -- 172.21.15.82:0/4021219936 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fd0b40bcc60 con 0x7fd0c8102bd0 2024-05-20T16:13:34.966 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:34.966+0000 7fd0cf102640 1 -- 172.21.15.82:0/4021219936 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fd0c802d460 con 0x7fd0c8102bd0 2024-05-20T16:13:34.966 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:34.966+0000 7fd0c5ffb640 1 -- 172.21.15.82:0/4021219936 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v93) ==== 103+0+11 (secure 0 0 0) 0x7fd0b40bc560 con 0x7fd0c8102bd0 2024-05-20T16:13:34.967 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 32 2024-05-20T16:13:34.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:34.968+0000 7fd0cf102640 1 -- 172.21.15.82:0/4021219936 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fd0b80a2b90 msgr2=0x7fd0b80a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:34.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:34.968+0000 7fd0cf102640 1 --2- 172.21.15.82:0/4021219936 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fd0b80a2b90 0x7fd0b80a50a0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fd0c8102590 tx=0x7fd0bc005e70 comp rx=0 tx=0).stop 2024-05-20T16:13:34.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:34.968+0000 7fd0cf102640 1 -- 172.21.15.82:0/4021219936 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd0c8102bd0 msgr2=0x7fd0c8198a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:34.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:34.968+0000 7fd0cf102640 1 --2- 172.21.15.82:0/4021219936 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd0c8102bd0 0x7fd0c8198a10 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fd0b4006010 tx=0x7fd0b4004360 comp rx=0 tx=0).stop 2024-05-20T16:13:34.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:34.968+0000 7fd0cf102640 1 -- 172.21.15.82:0/4021219936 shutdown_connections 2024-05-20T16:13:34.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:34.968+0000 7fd0cf102640 1 --2- 172.21.15.82:0/4021219936 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fd0b80a2b90 0x7fd0b80a50a0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:34.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:34.968+0000 7fd0cf102640 1 --2- 172.21.15.82:0/4021219936 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd0c8102bd0 0x7fd0c8198a10 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:34.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:34.968+0000 7fd0cf102640 1 --2- 172.21.15.82:0/4021219936 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd0c81078d0 0x7fd0c819f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:34.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:34.968+0000 7fd0cf102640 1 --2- 172.21.15.82:0/4021219936 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd0c8101990 0x7fd0c81984d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:34.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:34.968+0000 7fd0cf102640 1 -- 172.21.15.82:0/4021219936 >> 172.21.15.82:0/4021219936 conn(0x7fd0c80fcb80 msgr2=0x7fd0c81038b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:13:34.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:34.968+0000 7fd0cf102640 1 -- 172.21.15.82:0/4021219936 shutdown_connections 2024-05-20T16:13:34.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:34.968+0000 7fd0cf102640 1 -- 172.21.15.82:0/4021219936 wait complete. 2024-05-20T16:13:34.979 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:13:34 smithi082 ceph-mon[32238]: from='client.? 172.21.15.82:0/2516336710' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-20T16:13:34.979 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:13:34 smithi082 ceph-mon[32238]: from='client.? 172.21.15.82:0/3605438170' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-20T16:13:34.979 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:13:34 smithi082 ceph-mon[32238]: from='client.? 172.21.15.82:0/2087672772' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-20T16:13:34.980 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-20T16:13:35.062 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:13:34 smithi160 ceph-mon[36347]: from='client.? 172.21.15.82:0/2516336710' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-20T16:13:35.062 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:13:34 smithi160 ceph-mon[36347]: from='client.? 172.21.15.82:0/3605438170' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-20T16:13:35.063 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:13:34 smithi160 ceph-mon[36347]: from='client.? 172.21.15.82:0/2087672772' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-20T16:13:35.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:35.091+0000 7fca2aa14640 1 -- 172.21.15.82:0/226369391 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fca24102350 msgr2=0x7fca24102760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:35.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:35.091+0000 7fca2aa14640 1 --2- 172.21.15.82:0/226369391 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fca24102350 0x7fca24102760 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fca0c00aa10 tx=0x7fca0c02b720 comp rx=0 tx=0).stop 2024-05-20T16:13:35.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:35.091+0000 7fca2aa14640 1 -- 172.21.15.82:0/226369391 shutdown_connections 2024-05-20T16:13:35.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:35.091+0000 7fca2aa14640 1 --2- 172.21.15.82:0/226369391 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fca24102350 0x7fca24102760 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:35.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:35.091+0000 7fca2aa14640 1 --2- 172.21.15.82:0/226369391 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fca24107050 0x7fca24109470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:35.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:35.091+0000 7fca2aa14640 1 --2- 172.21.15.82:0/226369391 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fca24101110 0x7fca24101e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:35.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:35.091+0000 7fca2aa14640 1 -- 172.21.15.82:0/226369391 >> 172.21.15.82:0/226369391 conn(0x7fca240fc2e0 msgr2=0x7fca240fe7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:13:35.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:35.091+0000 7fca2aa14640 1 -- 172.21.15.82:0/226369391 shutdown_connections 2024-05-20T16:13:35.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:35.091+0000 7fca2aa14640 1 -- 172.21.15.82:0/226369391 wait complete. 2024-05-20T16:13:35.094 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:35.092+0000 7fca2aa14640 1 Processor -- start 2024-05-20T16:13:35.094 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:35.092+0000 7fca2aa14640 1 -- start start 2024-05-20T16:13:35.094 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:35.092+0000 7fca2aa14640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fca24101110 0x7fca241970e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:35.094 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:35.092+0000 7fca2aa14640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fca24102350 0x7fca24197620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:35.094 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:35.092+0000 7fca2aa14640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fca24107050 0x7fca2419e540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:35.094 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:35.092+0000 7fca2aa14640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fca2410c4f0 con 0x7fca24101110 2024-05-20T16:13:35.094 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:35.092+0000 7fca2aa14640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fca2410c370 con 0x7fca24107050 2024-05-20T16:13:35.094 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:35.092+0000 7fca2aa14640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fca2410c670 con 0x7fca24102350 2024-05-20T16:13:35.094 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:35.092+0000 7fca23fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fca24101110 0x7fca241970e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:13:35.094 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:35.092+0000 7fca23fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fca24101110 0x7fca241970e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:33502/0 (socket says 172.21.15.82:33502) 2024-05-20T16:13:35.094 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:35.092+0000 7fca23fff640 1 -- 172.21.15.82:0/92715348 learned_addr learned my addr 172.21.15.82:0/92715348 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:13:35.094 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:35.092+0000 7fca1bfff640 1 --2- 172.21.15.82:0/92715348 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fca24102350 0x7fca24197620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:13:35.094 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:35.092+0000 7fca23fff640 1 -- 172.21.15.82:0/92715348 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fca24102350 msgr2=0x7fca24197620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:35.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:35.092+0000 7fca23fff640 1 --2- 172.21.15.82:0/92715348 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fca24102350 0x7fca24197620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:35.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:35.092+0000 7fca23fff640 1 -- 172.21.15.82:0/92715348 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fca24107050 msgr2=0x7fca2419e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:35.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:35.092+0000 7fca23fff640 1 --2- 172.21.15.82:0/92715348 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fca24107050 0x7fca2419e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:35.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:35.092+0000 7fca23fff640 1 -- 172.21.15.82:0/92715348 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca2419ebb0 con 0x7fca24101110 2024-05-20T16:13:35.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:35.093+0000 7fca23fff640 1 --2- 172.21.15.82:0/92715348 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fca24101110 0x7fca241970e0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fca1400c970 tx=0x7fca1400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:13:35.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:35.093+0000 7fca21ffb640 1 -- 172.21.15.82:0/92715348 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca14004220 con 0x7fca24101110 2024-05-20T16:13:35.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:35.093+0000 7fca21ffb640 1 -- 172.21.15.82:0/92715348 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca140043c0 con 0x7fca24101110 2024-05-20T16:13:35.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:35.093+0000 7fca2aa14640 1 -- 172.21.15.82:0/92715348 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca2419eea0 con 0x7fca24101110 2024-05-20T16:13:35.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:35.093+0000 7fca21ffb640 1 -- 172.21.15.82:0/92715348 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca1400f450 con 0x7fca24101110 2024-05-20T16:13:35.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:35.093+0000 7fca2aa14640 1 -- 172.21.15.82:0/92715348 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca2419f300 con 0x7fca24101110 2024-05-20T16:13:35.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:35.094+0000 7fca2aa14640 1 -- 172.21.15.82:0/92715348 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca240682f0 con 0x7fca24101110 2024-05-20T16:13:35.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:35.095+0000 7fca21ffb640 1 -- 172.21.15.82:0/92715348 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fca140079b0 con 0x7fca24101110 2024-05-20T16:13:35.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:35.096+0000 7fca21ffb640 1 --2- 172.21.15.82:0/92715348 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fca100a2b90 0x7fca100a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:35.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:35.096+0000 7fca1bfff640 1 --2- 172.21.15.82:0/92715348 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fca100a2b90 0x7fca100a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:13:35.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:35.096+0000 7fca21ffb640 1 -- 172.21.15.82:0/92715348 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9092+0+0 (secure 0 0 0) 0x7fca140cf4f0 con 0x7fca24101110 2024-05-20T16:13:35.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:35.097+0000 7fca1bfff640 1 --2- 172.21.15.82:0/92715348 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fca100a2b90 0x7fca100a50a0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fca0c005f70 tx=0x7fca0c000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:13:35.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:35.100+0000 7fca21ffb640 1 -- 172.21.15.82:0/92715348 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fca240682f0 con 0x7fca24101110 2024-05-20T16:13:35.135 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:13:34 smithi184 ceph-mon[36209]: from='client.? 172.21.15.82:0/2516336710' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-20T16:13:35.136 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:13:34 smithi184 ceph-mon[36209]: from='client.? 172.21.15.82:0/3605438170' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-20T16:13:35.136 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:13:34 smithi184 ceph-mon[36209]: from='client.? 172.21.15.82:0/2087672772' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-20T16:13:35.312 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:35.312+0000 7fca2aa14640 1 -- 172.21.15.82:0/92715348 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fca2402d3a0 con 0x7fca24101110 2024-05-20T16:13:35.312 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:35.313+0000 7fca21ffb640 1 -- 172.21.15.82:0/92715348 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11879 (secure 0 0 0) 0x7fca14014030 con 0x7fca24101110 2024-05-20T16:13:35.313 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-05-20T16:13:35.313 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/2044534217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":2044534217},{"type":"v1","addr":"172.21.15.82:6835","nonce":2044534217}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/2635635097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":2635635097},{"type":"v1","addr":"172.21.15.82:6837","nonce":2635635097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14625,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6839/2118233541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6838","nonce":2118233541},{"type":"v1","addr":"172.21.15.82:6839","nonce":2118233541}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14637,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6841/2119517973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6840","nonce":2119517973},{"type":"v1","addr":"172.21.15.82:6841","nonce":2119517973}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6833/4015144007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":4015144007},{"type":"v1","addr":"172.21.15.184:6833","nonce":4015144007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6833/975243372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6832","nonce":975243372},{"type":"v1","addr":"172.21.15.160:6833","nonce":975243372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-20T16:11:46.751632+0000","modified":"2024-05-20T16:12:13.394770+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24548,"mds_1":24511,"mds_2":24506},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24470":{"gid":24470,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.160:6835/664068363","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":664068363},{"type":"v1","addr":"172.21.15.160:6835","nonce":664068363}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6835/4242156090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":4242156090},{"type":"v1","addr":"172.21.15.184:6835","nonce":4242156090}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6837/403262242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":403262242},{"type":"v1","addr":"172.21.15.184:6837","nonce":403262242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.160:6837/2846188680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6836","nonce":2846188680},{"type":"v1","addr":"172.21.15.160:6837","nonce":2846188680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.184:6839/2199955067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":2199955067},{"type":"v1","addr":"172.21.15.184:6839","nonce":2199955067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24548":{"gid":24548,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.160:6839/2765554482","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6838","nonce":2765554482},{"type":"v1","addr":"172.21.15.160:6839","nonce":2765554482}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24548,"qdb_cluster":[24511,24506,24548]},"id":1}]} 2024-05-20T16:13:35.313 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 24 2024-05-20T16:13:35.315 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:35.315+0000 7fca2aa14640 1 -- 172.21.15.82:0/92715348 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fca100a2b90 msgr2=0x7fca100a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:35.315 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:35.315+0000 7fca2aa14640 1 --2- 172.21.15.82:0/92715348 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fca100a2b90 0x7fca100a50a0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fca0c005f70 tx=0x7fca0c000bc0 comp rx=0 tx=0).stop 2024-05-20T16:13:35.315 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:35.315+0000 7fca2aa14640 1 -- 172.21.15.82:0/92715348 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fca24101110 msgr2=0x7fca241970e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:35.315 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:35.315+0000 7fca2aa14640 1 --2- 172.21.15.82:0/92715348 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fca24101110 0x7fca241970e0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fca1400c970 tx=0x7fca1400ce30 comp rx=0 tx=0).stop 2024-05-20T16:13:35.315 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:35.315+0000 7fca2aa14640 1 -- 172.21.15.82:0/92715348 shutdown_connections 2024-05-20T16:13:35.315 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:35.315+0000 7fca2aa14640 1 --2- 172.21.15.82:0/92715348 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fca100a2b90 0x7fca100a50a0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:35.315 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:35.315+0000 7fca2aa14640 1 --2- 172.21.15.82:0/92715348 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fca24107050 0x7fca2419e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:35.315 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:35.315+0000 7fca2aa14640 1 --2- 172.21.15.82:0/92715348 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fca24102350 0x7fca24197620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:35.315 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:35.315+0000 7fca2aa14640 1 --2- 172.21.15.82:0/92715348 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fca24101110 0x7fca241970e0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:35.315 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:35.315+0000 7fca2aa14640 1 -- 172.21.15.82:0/92715348 >> 172.21.15.82:0/92715348 conn(0x7fca240fc2e0 msgr2=0x7fca24103030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:13:35.315 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:35.315+0000 7fca2aa14640 1 -- 172.21.15.82:0/92715348 shutdown_connections 2024-05-20T16:13:35.315 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:35.315+0000 7fca2aa14640 1 -- 172.21.15.82:0/92715348 wait complete. 2024-05-20T16:13:35.327 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-20T16:13:35.439 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:35.439+0000 7f01fec3d640 1 -- 172.21.15.82:0/1527836705 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f01f8102bb0 msgr2=0x7f01f8102fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:35.439 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:35.439+0000 7f01fec3d640 1 --2- 172.21.15.82:0/1527836705 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f01f8102bb0 0x7f01f8102fc0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f01e8009900 tx=0x7f01e802a410 comp rx=0 tx=0).stop 2024-05-20T16:13:35.439 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:35.439+0000 7f01fec3d640 1 -- 172.21.15.82:0/1527836705 shutdown_connections 2024-05-20T16:13:35.439 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:35.439+0000 7f01fec3d640 1 --2- 172.21.15.82:0/1527836705 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f01f8102bb0 0x7f01f8102fc0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:35.439 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:35.439+0000 7f01fec3d640 1 --2- 172.21.15.82:0/1527836705 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f01f81078b0 0x7f01f8109cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:35.439 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:35.439+0000 7f01fec3d640 1 --2- 172.21.15.82:0/1527836705 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f01f8101970 0x7f01f8102670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:35.439 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:35.439+0000 7f01fec3d640 1 -- 172.21.15.82:0/1527836705 >> 172.21.15.82:0/1527836705 conn(0x7f01f80fcb80 msgr2=0x7f01f80ff010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:13:35.439 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:35.439+0000 7f01fec3d640 1 -- 172.21.15.82:0/1527836705 shutdown_connections 2024-05-20T16:13:35.440 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:35.439+0000 7f01fec3d640 1 -- 172.21.15.82:0/1527836705 wait complete. 2024-05-20T16:13:35.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:35.440+0000 7f01fec3d640 1 Processor -- start 2024-05-20T16:13:35.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:35.440+0000 7f01fec3d640 1 -- start start 2024-05-20T16:13:35.442 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:35.440+0000 7f01fec3d640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f01f8101970 0x7f01f819b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:35.442 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:35.440+0000 7f01fec3d640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f01f8102bb0 0x7f01f819b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:35.442 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:35.440+0000 7f01fec3d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f01f81078b0 0x7f01f81a2960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:35.442 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:35.440+0000 7f01fec3d640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f01f810d5f0 con 0x7f01f81078b0 2024-05-20T16:13:35.442 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:35.440+0000 7f01fec3d640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f01f810d470 con 0x7f01f8102bb0 2024-05-20T16:13:35.442 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:35.440+0000 7f01fec3d640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f01f810d770 con 0x7f01f8101970 2024-05-20T16:13:35.442 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:35.440+0000 7f01fd1da640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f01f81078b0 0x7f01f81a2960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:13:35.442 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:35.440+0000 7f01fd1da640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f01f81078b0 0x7f01f81a2960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:33528/0 (socket says 172.21.15.82:33528) 2024-05-20T16:13:35.442 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:35.440+0000 7f01fd1da640 1 -- 172.21.15.82:0/1022431930 learned_addr learned my addr 172.21.15.82:0/1022431930 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:13:35.442 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:35.440+0000 7f01fc9d9640 1 --2- 172.21.15.82:0/1022431930 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f01f8101970 0x7f01f819b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:13:35.442 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:35.440+0000 7f01effff640 1 --2- 172.21.15.82:0/1022431930 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f01f8102bb0 0x7f01f819b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:13:35.442 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:35.440+0000 7f01fd1da640 1 -- 172.21.15.82:0/1022431930 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f01f8101970 msgr2=0x7f01f819b490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:35.442 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:35.440+0000 7f01fd1da640 1 --2- 172.21.15.82:0/1022431930 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f01f8101970 0x7f01f819b490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:35.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:35.440+0000 7f01fd1da640 1 -- 172.21.15.82:0/1022431930 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f01f8102bb0 msgr2=0x7f01f819b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:35.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:35.440+0000 7f01fd1da640 1 --2- 172.21.15.82:0/1022431930 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f01f8102bb0 0x7f01f819b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:35.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:35.441+0000 7f01fd1da640 1 -- 172.21.15.82:0/1022431930 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01f81a2fd0 con 0x7f01f81078b0 2024-05-20T16:13:35.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:35.441+0000 7f01fd1da640 1 --2- 172.21.15.82:0/1022431930 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f01f81078b0 0x7f01f81a2960 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f01f000e970 tx=0x7f01f000ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:13:35.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:35.441+0000 7f01edffb640 1 -- 172.21.15.82:0/1022431930 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01f000ccc0 con 0x7f01f81078b0 2024-05-20T16:13:35.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:35.441+0000 7f01edffb640 1 -- 172.21.15.82:0/1022431930 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f01f000ce60 con 0x7f01f81078b0 2024-05-20T16:13:35.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:35.441+0000 7f01edffb640 1 -- 172.21.15.82:0/1022431930 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01f0010490 con 0x7f01f81078b0 2024-05-20T16:13:35.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:35.441+0000 7f01fec3d640 1 -- 172.21.15.82:0/1022431930 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01f81a32c0 con 0x7f01f81078b0 2024-05-20T16:13:35.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:35.441+0000 7f01fec3d640 1 -- 172.21.15.82:0/1022431930 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01f81a3860 con 0x7f01f81078b0 2024-05-20T16:13:35.446 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:35.442+0000 7f01fec3d640 1 -- 172.21.15.82:0/1022431930 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01f81a3b20 con 0x7f01f81078b0 2024-05-20T16:13:35.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:35.443+0000 7f01edffb640 1 -- 172.21.15.82:0/1022431930 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f01f0005020 con 0x7f01f81078b0 2024-05-20T16:13:35.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:35.444+0000 7f01edffb640 1 --2- 172.21.15.82:0/1022431930 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f01e00a2c10 0x7f01e00a5120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:35.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:35.444+0000 7f01fc9d9640 1 --2- 172.21.15.82:0/1022431930 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f01e00a2c10 0x7f01e00a5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:13:35.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:35.444+0000 7f01edffb640 1 -- 172.21.15.82:0/1022431930 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9092+0+0 (secure 0 0 0) 0x7f01f00cf740 con 0x7f01f81078b0 2024-05-20T16:13:35.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:35.445+0000 7f01fc9d9640 1 --2- 172.21.15.82:0/1022431930 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f01e00a2c10 0x7f01e00a5120 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f01f8102570 tx=0x7f01f4009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:13:35.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:35.449+0000 7f01edffb640 1 -- 172.21.15.82:0/1022431930 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f01f009c340 con 0x7f01f81078b0 2024-05-20T16:13:35.631 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:35.631+0000 7f01fec3d640 1 -- 172.21.15.82:0/1022431930 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f01f81a3cf0 con 0x7f01f81078b0 2024-05-20T16:13:35.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:35.633+0000 7f01edffb640 1 -- 172.21.15.82:0/1022431930 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) ==== 74+0+24621 (secure 0 0 0) 0x7f01f009bc40 con 0x7f01f81078b0 2024-05-20T16:13:35.633 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-05-20T16:13:35.633 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":93,"fsid":"59d1600a-16c2-11ef-bc9a-c7b262605968","created":"2024-05-20T16:03:46.611556+0000","modified":"2024-05-20T16:12:46.250119+0000","last_up_change":"2024-05-20T16:10:59.303479+0000","last_in_change":"2024-05-20T16:10:39.063561+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-20T16:07:09.493727+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-20T16:11:45.058035+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-20T16:11:45.943862+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-20T16:11:49.637579+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"bae842a9-03ec-490e-b56d-f7957235cdc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":1484330167},{"type":"v1","addr":"172.21.15.82:6803","nonce":1484330167}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":1484330167},{"type":"v1","addr":"172.21.15.82:6805","nonce":1484330167}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1484330167},{"type":"v1","addr":"172.21.15.82:6809","nonce":1484330167}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":1484330167},{"type":"v1","addr":"172.21.15.82:6807","nonce":1484330167}]},"public_addr":"172.21.15.82:6803/1484330167","cluster_addr":"172.21.15.82:6805/1484330167","heartbeat_back_addr":"172.21.15.82:6809/1484330167","heartbeat_front_addr":"172.21.15.82:6807/1484330167","state":["exists","up"]},{"osd":1,"uuid":"59f6a2ba-b5de-4161-81c5-92a417abacdf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":1457003568},{"type":"v1","addr":"172.21.15.160:6801","nonce":1457003568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":1457003568},{"type":"v1","addr":"172.21.15.160:6803","nonce":1457003568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6806","nonce":1457003568},{"type":"v1","addr":"172.21.15.160:6807","nonce":1457003568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":1457003568},{"type":"v1","addr":"172.21.15.160:6805","nonce":1457003568}]},"public_addr":"172.21.15.160:6801/1457003568","cluster_addr":"172.21.15.160:6803/1457003568","heartbeat_back_addr":"172.21.15.160:6807/1457003568","heartbeat_front_addr":"172.21.15.160:6805/1457003568","state":["exists","up"]},{"osd":2,"uuid":"3bbb5587-b449-41f3-bd26-67ec3d6afdfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":4051293934},{"type":"v1","addr":"172.21.15.184:6801","nonce":4051293934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":4051293934},{"type":"v1","addr":"172.21.15.184:6803","nonce":4051293934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":4051293934},{"type":"v1","addr":"172.21.15.184:6807","nonce":4051293934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":4051293934},{"type":"v1","addr":"172.21.15.184:6805","nonce":4051293934}]},"public_addr":"172.21.15.184:6801/4051293934","cluster_addr":"172.21.15.184:6803/4051293934","heartbeat_back_addr":"172.21.15.184:6807/4051293934","heartbeat_front_addr":"172.21.15.184:6805/4051293934","state":["exists","up"]},{"osd":3,"uuid":"ca651f90-09b8-4081-a6f9-40b31268a837","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":3947485599},{"type":"v1","addr":"172.21.15.82:6811","nonce":3947485599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":3947485599},{"type":"v1","addr":"172.21.15.82:6813","nonce":3947485599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":3947485599},{"type":"v1","addr":"172.21.15.82:6817","nonce":3947485599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":3947485599},{"type":"v1","addr":"172.21.15.82:6815","nonce":3947485599}]},"public_addr":"172.21.15.82:6811/3947485599","cluster_addr":"172.21.15.82:6813/3947485599","heartbeat_back_addr":"172.21.15.82:6817/3947485599","heartbeat_front_addr":"172.21.15.82:6815/3947485599","state":["exists","up"]},{"osd":4,"uuid":"cb3dfd56-f677-40cb-9c96-8319ea5ac788","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":4160830014},{"type":"v1","addr":"172.21.15.160:6809","nonce":4160830014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":4160830014},{"type":"v1","addr":"172.21.15.160:6811","nonce":4160830014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":4160830014},{"type":"v1","addr":"172.21.15.160:6815","nonce":4160830014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6812","nonce":4160830014},{"type":"v1","addr":"172.21.15.160:6813","nonce":4160830014}]},"public_addr":"172.21.15.160:6809/4160830014","cluster_addr":"172.21.15.160:6811/4160830014","heartbeat_back_addr":"172.21.15.160:6815/4160830014","heartbeat_front_addr":"172.21.15.160:6813/4160830014","state":["exists","up"]},{"osd":5,"uuid":"c70cfcd1-a4ff-461f-8c73-479b061200fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":1046282672},{"type":"v1","addr":"172.21.15.184:6809","nonce":1046282672}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":1046282672},{"type":"v1","addr":"172.21.15.184:6811","nonce":1046282672}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":1046282672},{"type":"v1","addr":"172.21.15.184:6815","nonce":1046282672}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":1046282672},{"type":"v1","addr":"172.21.15.184:6813","nonce":1046282672}]},"public_addr":"172.21.15.184:6809/1046282672","cluster_addr":"172.21.15.184:6811/1046282672","heartbeat_back_addr":"172.21.15.184:6815/1046282672","heartbeat_front_addr":"172.21.15.184:6813/1046282672","state":["exists","up"]},{"osd":6,"uuid":"d47ce001-9dd1-4830-bcc9-888601187c6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":2789736499},{"type":"v1","addr":"172.21.15.82:6819","nonce":2789736499}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":2789736499},{"type":"v1","addr":"172.21.15.82:6821","nonce":2789736499}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":2789736499},{"type":"v1","addr":"172.21.15.82:6825","nonce":2789736499}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":2789736499},{"type":"v1","addr":"172.21.15.82:6823","nonce":2789736499}]},"public_addr":"172.21.15.82:6819/2789736499","cluster_addr":"172.21.15.82:6821/2789736499","heartbeat_back_addr":"172.21.15.82:6825/2789736499","heartbeat_front_addr":"172.21.15.82:6823/2789736499","state":["exists","up"]},{"osd":7,"uuid":"25c54f43-18cb-41e4-83b4-2732b5a93889","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6816","nonce":2340683378},{"type":"v1","addr":"172.21.15.160:6817","nonce":2340683378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":2340683378},{"type":"v1","addr":"172.21.15.160:6819","nonce":2340683378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":2340683378},{"type":"v1","addr":"172.21.15.160:6823","nonce":2340683378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6820","nonce":2340683378},{"type":"v1","addr":"172.21.15.160:6821","nonce":2340683378}]},"public_addr":"172.21.15.160:6817/2340683378","cluster_addr":"172.21.15.160:6819/2340683378","heartbeat_back_addr":"172.21.15.160:6823/2340683378","heartbeat_front_addr":"172.21.15.160:6821/2340683378","state":["exists","up"]},{"osd":8,"uuid":"1f8b478d-287f-45f5-8526-d1a58c5bc07c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":3831576372},{"type":"v1","addr":"172.21.15.184:6817","nonce":3831576372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":3831576372},{"type":"v1","addr":"172.21.15.184:6819","nonce":3831576372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":3831576372},{"type":"v1","addr":"172.21.15.184:6823","nonce":3831576372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":3831576372},{"type":"v1","addr":"172.21.15.184:6821","nonce":3831576372}]},"public_addr":"172.21.15.184:6817/3831576372","cluster_addr":"172.21.15.184:6819/3831576372","heartbeat_back_addr":"172.21.15.184:6823/3831576372","heartbeat_front_addr":"172.21.15.184:6821/3831576372","state":["exists","up"]},{"osd":9,"uuid":"0d69e319-954f-4b18-95ea-31794f18dc3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":716330782},{"type":"v1","addr":"172.21.15.82:6827","nonce":716330782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":716330782},{"type":"v1","addr":"172.21.15.82:6829","nonce":716330782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":716330782},{"type":"v1","addr":"172.21.15.82:6833","nonce":716330782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":716330782},{"type":"v1","addr":"172.21.15.82:6831","nonce":716330782}]},"public_addr":"172.21.15.82:6827/716330782","cluster_addr":"172.21.15.82:6829/716330782","heartbeat_back_addr":"172.21.15.82:6833/716330782","heartbeat_front_addr":"172.21.15.82:6831/716330782","state":["exists","up"]},{"osd":10,"uuid":"39dc1b78-61cb-42ca-b5b6-1e44ac2b6500","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":2327428014},{"type":"v1","addr":"172.21.15.160:6825","nonce":2327428014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":2327428014},{"type":"v1","addr":"172.21.15.160:6827","nonce":2327428014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":2327428014},{"type":"v1","addr":"172.21.15.160:6831","nonce":2327428014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":2327428014},{"type":"v1","addr":"172.21.15.160:6829","nonce":2327428014}]},"public_addr":"172.21.15.160:6825/2327428014","cluster_addr":"172.21.15.160:6827/2327428014","heartbeat_back_addr":"172.21.15.160:6831/2327428014","heartbeat_front_addr":"172.21.15.160:6829/2327428014","state":["exists","up"]},{"osd":11,"uuid":"4306b3c1-707a-4010-987c-209ea0e948e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":3594364650},{"type":"v1","addr":"172.21.15.184:6825","nonce":3594364650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":3594364650},{"type":"v1","addr":"172.21.15.184:6827","nonce":3594364650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3594364650},{"type":"v1","addr":"172.21.15.184:6831","nonce":3594364650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3594364650},{"type":"v1","addr":"172.21.15.184:6829","nonce":3594364650}]},"public_addr":"172.21.15.184:6825/3594364650","cluster_addr":"172.21.15.184:6827/3594364650","heartbeat_back_addr":"172.21.15.184:6831/3594364650","heartbeat_front_addr":"172.21.15.184:6829/3594364650","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:06:17.855998+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:06:41.473270+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:07:05.304108+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:07:29.887472+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:07:55.387286+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:08:20.579669+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:08:46.333051+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:09:12.289668+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:09:38.180696+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:10:04.471459+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:10:31.041912+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:10:57.477266+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.1d","mappings":[{"from":5,"to":9}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.5","mappings":[{"from":9,"to":10}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.82:0/2005424509":"2024-05-21T16:04:45.161949+0000","172.21.15.82:6801/1763049308":"2024-05-21T16:04:45.161949+0000","172.21.15.82:6801/1959459348":"2024-05-21T16:04:19.208291+0000","172.21.15.82:0/3131904556":"2024-05-21T16:04:19.208291+0000","172.21.15.82:6800/1959459348":"2024-05-21T16:04:19.208291+0000","172.21.15.82:0/2456966342":"2024-05-21T16:04:45.161949+0000","172.21.15.82:0/4179978833":"2024-05-21T16:04:19.208291+0000","172.21.15.82:0/1602862225":"2024-05-21T16:04:45.161949+0000","172.21.15.82:0/106101180":"2024-05-21T16:04:19.208291+0000","172.21.15.82:6800/1763049308":"2024-05-21T16:04:45.161949+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-20T16:13:35.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:35.635+0000 7f01fec3d640 1 -- 172.21.15.82:0/1022431930 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f01e00a2c10 msgr2=0x7f01e00a5120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:35.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:35.635+0000 7f01fec3d640 1 --2- 172.21.15.82:0/1022431930 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f01e00a2c10 0x7f01e00a5120 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f01f8102570 tx=0x7f01f4009500 comp rx=0 tx=0).stop 2024-05-20T16:13:35.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:35.635+0000 7f01fec3d640 1 -- 172.21.15.82:0/1022431930 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f01f81078b0 msgr2=0x7f01f81a2960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:35.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:35.635+0000 7f01fec3d640 1 --2- 172.21.15.82:0/1022431930 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f01f81078b0 0x7f01f81a2960 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f01f000e970 tx=0x7f01f000ed70 comp rx=0 tx=0).stop 2024-05-20T16:13:35.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:35.635+0000 7f01fec3d640 1 -- 172.21.15.82:0/1022431930 shutdown_connections 2024-05-20T16:13:35.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:35.635+0000 7f01fec3d640 1 --2- 172.21.15.82:0/1022431930 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f01e00a2c10 0x7f01e00a5120 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:35.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:35.635+0000 7f01fec3d640 1 --2- 172.21.15.82:0/1022431930 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f01f81078b0 0x7f01f81a2960 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:35.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:35.635+0000 7f01fec3d640 1 --2- 172.21.15.82:0/1022431930 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f01f8102bb0 0x7f01f819b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:35.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:35.635+0000 7f01fec3d640 1 --2- 172.21.15.82:0/1022431930 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f01f8101970 0x7f01f819b490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:35.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:35.635+0000 7f01fec3d640 1 -- 172.21.15.82:0/1022431930 >> 172.21.15.82:0/1022431930 conn(0x7f01f80fcb80 msgr2=0x7f01f8068e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:13:35.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:35.635+0000 7f01fec3d640 1 -- 172.21.15.82:0/1022431930 shutdown_connections 2024-05-20T16:13:35.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:35.635+0000 7f01fec3d640 1 -- 172.21.15.82:0/1022431930 wait complete. 2024-05-20T16:13:35.647 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-20T16:13:35.761 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:35.759+0000 7f8b25f5e640 1 -- 172.21.15.82:0/1205520517 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8b20103d00 msgr2=0x7f8b20104110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:35.761 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:35.759+0000 7f8b25f5e640 1 --2- 172.21.15.82:0/1205520517 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8b20103d00 0x7f8b20104110 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f8b08009900 tx=0x7f8b0802a410 comp rx=0 tx=0).stop 2024-05-20T16:13:35.761 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:35.759+0000 7f8b25f5e640 1 -- 172.21.15.82:0/1205520517 shutdown_connections 2024-05-20T16:13:35.761 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:35.759+0000 7f8b25f5e640 1 --2- 172.21.15.82:0/1205520517 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8b20103d00 0x7f8b20104110 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:35.761 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:35.759+0000 7f8b25f5e640 1 --2- 172.21.15.82:0/1205520517 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8b20108a00 0x7f8b2010ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:35.761 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:35.759+0000 7f8b25f5e640 1 --2- 172.21.15.82:0/1205520517 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8b20102ac0 0x7f8b201037c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:35.761 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:35.759+0000 7f8b25f5e640 1 -- 172.21.15.82:0/1205520517 >> 172.21.15.82:0/1205520517 conn(0x7f8b200fdcf0 msgr2=0x7f8b20100160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:13:35.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:35.759+0000 7f8b25f5e640 1 -- 172.21.15.82:0/1205520517 shutdown_connections 2024-05-20T16:13:35.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:35.759+0000 7f8b25f5e640 1 -- 172.21.15.82:0/1205520517 wait complete. 2024-05-20T16:13:35.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:35.760+0000 7f8b25f5e640 1 Processor -- start 2024-05-20T16:13:35.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:35.760+0000 7f8b25f5e640 1 -- start start 2024-05-20T16:13:35.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:35.760+0000 7f8b25f5e640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8b20102ac0 0x7f8b20199320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:35.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:35.760+0000 7f8b25f5e640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8b20103d00 0x7f8b20199860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:35.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:35.760+0000 7f8b25f5e640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8b20108a00 0x7f8b201a0780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:35.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:35.760+0000 7f8b25f5e640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f8b2010e710 con 0x7f8b20102ac0 2024-05-20T16:13:35.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:35.760+0000 7f8b25f5e640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f8b2010e590 con 0x7f8b20108a00 2024-05-20T16:13:35.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:35.760+0000 7f8b25f5e640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f8b2010e890 con 0x7f8b20103d00 2024-05-20T16:13:35.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:35.760+0000 7f8b1f7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8b20102ac0 0x7f8b20199320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:13:35.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:35.760+0000 7f8b1f7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8b20102ac0 0x7f8b20199320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:33546/0 (socket says 172.21.15.82:33546) 2024-05-20T16:13:35.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:35.760+0000 7f8b1f7fe640 1 -- 172.21.15.82:0/2768485576 learned_addr learned my addr 172.21.15.82:0/2768485576 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:13:35.763 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:35.760+0000 7f8b17fff640 1 --2- 172.21.15.82:0/2768485576 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8b20103d00 0x7f8b20199860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:13:35.763 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:35.760+0000 7f8b1ffff640 1 --2- 172.21.15.82:0/2768485576 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8b20108a00 0x7f8b201a0780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:13:35.763 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:35.760+0000 7f8b1f7fe640 1 -- 172.21.15.82:0/2768485576 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8b20103d00 msgr2=0x7f8b20199860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:35.763 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:35.760+0000 7f8b1f7fe640 1 --2- 172.21.15.82:0/2768485576 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8b20103d00 0x7f8b20199860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:35.763 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:35.760+0000 7f8b1f7fe640 1 -- 172.21.15.82:0/2768485576 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8b20108a00 msgr2=0x7f8b201a0780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:35.763 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:35.760+0000 7f8b1f7fe640 1 --2- 172.21.15.82:0/2768485576 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8b20108a00 0x7f8b201a0780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:35.763 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:35.760+0000 7f8b1f7fe640 1 -- 172.21.15.82:0/2768485576 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b201a0df0 con 0x7f8b20102ac0 2024-05-20T16:13:35.763 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:35.760+0000 7f8b17fff640 1 --2- 172.21.15.82:0/2768485576 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8b20103d00 0x7f8b20199860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T16:13:35.763 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:35.760+0000 7f8b1ffff640 1 --2- 172.21.15.82:0/2768485576 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8b20108a00 0x7f8b201a0780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T16:13:35.763 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:35.760+0000 7f8b1f7fe640 1 --2- 172.21.15.82:0/2768485576 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8b20102ac0 0x7f8b20199320 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f8b1000d740 tx=0x7f8b1000dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:13:35.763 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:35.761+0000 7f8b1d7fa640 1 -- 172.21.15.82:0/2768485576 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b10017070 con 0x7f8b20102ac0 2024-05-20T16:13:35.763 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:35.761+0000 7f8b1d7fa640 1 -- 172.21.15.82:0/2768485576 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b1000de50 con 0x7f8b20102ac0 2024-05-20T16:13:35.763 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:35.761+0000 7f8b1d7fa640 1 -- 172.21.15.82:0/2768485576 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b10003e00 con 0x7f8b20102ac0 2024-05-20T16:13:35.763 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:35.761+0000 7f8b25f5e640 1 -- 172.21.15.82:0/2768485576 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b201a10e0 con 0x7f8b20102ac0 2024-05-20T16:13:35.764 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:35.761+0000 7f8b25f5e640 1 -- 172.21.15.82:0/2768485576 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b201a1680 con 0x7f8b20102ac0 2024-05-20T16:13:35.764 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:35.762+0000 7f8b25f5e640 1 -- 172.21.15.82:0/2768485576 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b20107810 con 0x7f8b20102ac0 2024-05-20T16:13:35.765 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:35.763+0000 7f8b1d7fa640 1 -- 172.21.15.82:0/2768485576 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f8b10003fa0 con 0x7f8b20102ac0 2024-05-20T16:13:35.765 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:35.763+0000 7f8b1d7fa640 1 --2- 172.21.15.82:0/2768485576 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f8b0c0a2c10 0x7f8b0c0a5120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:35.765 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:35.764+0000 7f8b17fff640 1 --2- 172.21.15.82:0/2768485576 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f8b0c0a2c10 0x7f8b0c0a5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:13:35.765 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:35.764+0000 7f8b1d7fa640 1 -- 172.21.15.82:0/2768485576 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9092+0+0 (secure 0 0 0) 0x7f8b100cf190 con 0x7f8b20102ac0 2024-05-20T16:13:35.765 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:35.764+0000 7f8b17fff640 1 --2- 172.21.15.82:0/2768485576 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f8b0c0a2c10 0x7f8b0c0a5120 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f8b08000c00 tx=0x7f8b080095f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:13:35.766 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:35.767+0000 7f8b1d7fa640 1 -- 172.21.15.82:0/2768485576 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f8b1009be10 con 0x7f8b20102ac0 2024-05-20T16:13:35.982 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:35.983+0000 7f8b25f5e640 1 -- 172.21.15.82:0/2768485576 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8b2002d2a0 con 0x7f8b20102ac0 2024-05-20T16:13:35.983 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:35.983+0000 7f8b1d7fa640 1 -- 172.21.15.82:0/2768485576 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11879 (secure 0 0 0) 0x7f8b1009b710 con 0x7f8b20102ac0 2024-05-20T16:13:35.983 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-05-20T16:13:35.984 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/2044534217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":2044534217},{"type":"v1","addr":"172.21.15.82:6835","nonce":2044534217}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/2635635097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":2635635097},{"type":"v1","addr":"172.21.15.82:6837","nonce":2635635097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14625,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6839/2118233541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6838","nonce":2118233541},{"type":"v1","addr":"172.21.15.82:6839","nonce":2118233541}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14637,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6841/2119517973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6840","nonce":2119517973},{"type":"v1","addr":"172.21.15.82:6841","nonce":2119517973}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6833/4015144007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":4015144007},{"type":"v1","addr":"172.21.15.184:6833","nonce":4015144007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6833/975243372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6832","nonce":975243372},{"type":"v1","addr":"172.21.15.160:6833","nonce":975243372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-20T16:11:46.751632+0000","modified":"2024-05-20T16:12:13.394770+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24548,"mds_1":24511,"mds_2":24506},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24470":{"gid":24470,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.160:6835/664068363","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":664068363},{"type":"v1","addr":"172.21.15.160:6835","nonce":664068363}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6835/4242156090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":4242156090},{"type":"v1","addr":"172.21.15.184:6835","nonce":4242156090}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6837/403262242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":403262242},{"type":"v1","addr":"172.21.15.184:6837","nonce":403262242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.160:6837/2846188680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6836","nonce":2846188680},{"type":"v1","addr":"172.21.15.160:6837","nonce":2846188680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.184:6839/2199955067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":2199955067},{"type":"v1","addr":"172.21.15.184:6839","nonce":2199955067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24548":{"gid":24548,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.160:6839/2765554482","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6838","nonce":2765554482},{"type":"v1","addr":"172.21.15.160:6839","nonce":2765554482}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24548,"qdb_cluster":[24511,24506,24548]},"id":1}]} 2024-05-20T16:13:35.984 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 24 2024-05-20T16:13:35.986 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:35.985+0000 7f8b25f5e640 1 -- 172.21.15.82:0/2768485576 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f8b0c0a2c10 msgr2=0x7f8b0c0a5120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:35.986 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:35.985+0000 7f8b25f5e640 1 --2- 172.21.15.82:0/2768485576 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f8b0c0a2c10 0x7f8b0c0a5120 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f8b08000c00 tx=0x7f8b080095f0 comp rx=0 tx=0).stop 2024-05-20T16:13:35.986 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:35.986+0000 7f8b25f5e640 1 -- 172.21.15.82:0/2768485576 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8b20102ac0 msgr2=0x7f8b20199320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:35.986 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:35.986+0000 7f8b25f5e640 1 --2- 172.21.15.82:0/2768485576 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8b20102ac0 0x7f8b20199320 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f8b1000d740 tx=0x7f8b1000dc00 comp rx=0 tx=0).stop 2024-05-20T16:13:35.986 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:35.986+0000 7f8b25f5e640 1 -- 172.21.15.82:0/2768485576 shutdown_connections 2024-05-20T16:13:35.986 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:35.986+0000 7f8b25f5e640 1 --2- 172.21.15.82:0/2768485576 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f8b0c0a2c10 0x7f8b0c0a5120 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:35.986 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:35.986+0000 7f8b25f5e640 1 --2- 172.21.15.82:0/2768485576 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8b20108a00 0x7f8b201a0780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:35.986 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:35.986+0000 7f8b25f5e640 1 --2- 172.21.15.82:0/2768485576 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8b20103d00 0x7f8b20199860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:35.986 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:35.986+0000 7f8b25f5e640 1 --2- 172.21.15.82:0/2768485576 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8b20102ac0 0x7f8b20199320 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:35.986 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:35.986+0000 7f8b25f5e640 1 -- 172.21.15.82:0/2768485576 >> 172.21.15.82:0/2768485576 conn(0x7f8b200fdcf0 msgr2=0x7f8b201049e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:13:35.986 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:35.986+0000 7f8b25f5e640 1 -- 172.21.15.82:0/2768485576 shutdown_connections 2024-05-20T16:13:35.986 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:35.986+0000 7f8b25f5e640 1 -- 172.21.15.82:0/2768485576 wait complete. 2024-05-20T16:13:35.996 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:13:35 smithi082 ceph-mon[32238]: from='client.? 172.21.15.82:0/4021219936' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-20T16:13:35.996 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:13:35 smithi082 ceph-mon[32238]: pgmap v348: 129 pgs: 129 active+clean; 641 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-20T16:13:35.996 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:13:35 smithi082 ceph-mon[32238]: from='client.? 172.21.15.82:0/92715348' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T16:13:35.996 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:13:35 smithi082 ceph-mon[32238]: from='client.? 172.21.15.82:0/1022431930' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-20T16:13:35.998 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 24, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-05-20T16:11:46.751632+0000', 'modified': '2024-05-20T16:12:13.394770+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24548, 'mds_1': 24511, 'mds_2': 24506}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24470': {'gid': 24470, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.160:6835/664068363', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.160:6834', 'nonce': 664068363}, {'type': 'v1', 'addr': '172.21.15.160:6835', 'nonce': 664068363}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24478': {'gid': 24478, 'name': 'f', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.184:6835/4242156090', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.184:6834', 'nonce': 4242156090}, {'type': 'v1', 'addr': '172.21.15.184:6835', 'nonce': 4242156090}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24496': {'gid': 24496, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.184:6837/403262242', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.184:6836', 'nonce': 403262242}, {'type': 'v1', 'addr': '172.21.15.184:6837', 'nonce': 403262242}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24506': {'gid': 24506, 'name': 'h', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.160:6837/2846188680', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.160:6836', 'nonce': 2846188680}, {'type': 'v1', 'addr': '172.21.15.160:6837', 'nonce': 2846188680}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24511': {'gid': 24511, 'name': 'i', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.184:6839/2199955067', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.184:6838', 'nonce': 2199955067}, {'type': 'v1', 'addr': '172.21.15.184:6839', 'nonce': 2199955067}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24548': {'gid': 24548, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.160:6839/2765554482', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.160:6838', 'nonce': 2765554482}, {'type': 'v1', 'addr': '172.21.15.160:6839', 'nonce': 2765554482}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24548, 'qdb_cluster': [24511, 24506, 24548]} 2024-05-20T16:13:35.998 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-05-20T16:13:35.998 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2024-05-20T16:13:36.062 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:13:35 smithi160 ceph-mon[36347]: from='client.? 172.21.15.82:0/4021219936' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-20T16:13:36.062 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:13:35 smithi160 ceph-mon[36347]: pgmap v348: 129 pgs: 129 active+clean; 641 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-20T16:13:36.063 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:13:35 smithi160 ceph-mon[36347]: from='client.? 172.21.15.82:0/92715348' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T16:13:36.063 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:13:35 smithi160 ceph-mon[36347]: from='client.? 172.21.15.82:0/1022431930' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-20T16:13:36.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.110+0000 7f22618a0640 1 -- 172.21.15.82:0/3774480694 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f225c1078d0 msgr2=0x7f225c109cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:36.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.110+0000 7f22618a0640 1 --2- 172.21.15.82:0/3774480694 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f225c1078d0 0x7f225c109cf0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f2254009ca0 tx=0x7f225402a8c0 comp rx=0 tx=0).stop 2024-05-20T16:13:36.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.110+0000 7f22618a0640 1 -- 172.21.15.82:0/3774480694 shutdown_connections 2024-05-20T16:13:36.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.110+0000 7f22618a0640 1 --2- 172.21.15.82:0/3774480694 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f225c1078d0 0x7f225c109cf0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:36.113 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.110+0000 7f22618a0640 1 --2- 172.21.15.82:0/3774480694 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f225c102bd0 0x7f225c102fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:36.113 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.110+0000 7f22618a0640 1 --2- 172.21.15.82:0/3774480694 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f225c101990 0x7f225c102690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:36.113 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.110+0000 7f22618a0640 1 -- 172.21.15.82:0/3774480694 >> 172.21.15.82:0/3774480694 conn(0x7f225c0fcb80 msgr2=0x7f225c0ff030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:13:36.113 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.110+0000 7f22618a0640 1 -- 172.21.15.82:0/3774480694 shutdown_connections 2024-05-20T16:13:36.113 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.110+0000 7f22618a0640 1 -- 172.21.15.82:0/3774480694 wait complete. 2024-05-20T16:13:36.113 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.111+0000 7f22618a0640 1 Processor -- start 2024-05-20T16:13:36.113 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.111+0000 7f22618a0640 1 -- start start 2024-05-20T16:13:36.113 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.111+0000 7f22618a0640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f225c101990 0x7f225c198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:36.113 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.111+0000 7f22618a0640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f225c102bd0 0x7f225c198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:36.113 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.111+0000 7f22618a0640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f225c1078d0 0x7f225c19f900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:36.113 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.111+0000 7f22618a0640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f225c10d5f0 con 0x7f225c1078d0 2024-05-20T16:13:36.113 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.111+0000 7f22618a0640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f225c10d470 con 0x7f225c101990 2024-05-20T16:13:36.113 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.111+0000 7f22618a0640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f225c10d770 con 0x7f225c102bd0 2024-05-20T16:13:36.113 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.111+0000 7f225b7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f225c1078d0 0x7f225c19f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:13:36.114 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.111+0000 7f225b7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f225c1078d0 0x7f225c19f900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:33550/0 (socket says 172.21.15.82:33550) 2024-05-20T16:13:36.114 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.111+0000 7f225b7fe640 1 -- 172.21.15.82:0/3954674364 learned_addr learned my addr 172.21.15.82:0/3954674364 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:13:36.114 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.112+0000 7f2253fff640 1 --2- 172.21.15.82:0/3954674364 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f225c102bd0 0x7f225c198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:13:36.114 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.112+0000 7f225affd640 1 --2- 172.21.15.82:0/3954674364 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f225c101990 0x7f225c198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:13:36.114 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.112+0000 7f225b7fe640 1 -- 172.21.15.82:0/3954674364 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f225c102bd0 msgr2=0x7f225c198970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:36.114 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.112+0000 7f225b7fe640 1 --2- 172.21.15.82:0/3954674364 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f225c102bd0 0x7f225c198970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:36.114 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.112+0000 7f225b7fe640 1 -- 172.21.15.82:0/3954674364 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f225c101990 msgr2=0x7f225c198430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:36.114 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.112+0000 7f225b7fe640 1 --2- 172.21.15.82:0/3954674364 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f225c101990 0x7f225c198430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:36.114 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.112+0000 7f225b7fe640 1 -- 172.21.15.82:0/3954674364 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f225c19fec0 con 0x7f225c1078d0 2024-05-20T16:13:36.114 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.112+0000 7f225affd640 1 --2- 172.21.15.82:0/3954674364 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f225c101990 0x7f225c198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T16:13:36.114 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.112+0000 7f2253fff640 1 --2- 172.21.15.82:0/3954674364 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f225c102bd0 0x7f225c198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T16:13:36.114 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.112+0000 7f225b7fe640 1 --2- 172.21.15.82:0/3954674364 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f225c1078d0 0x7f225c19f900 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f2254005cc0 tx=0x7f2254009890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:13:36.115 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.112+0000 7f226089e640 1 -- 172.21.15.82:0/3954674364 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2254003c30 con 0x7f225c1078d0 2024-05-20T16:13:36.115 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.112+0000 7f226089e640 1 -- 172.21.15.82:0/3954674364 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2254003dd0 con 0x7f225c1078d0 2024-05-20T16:13:36.115 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.112+0000 7f22618a0640 1 -- 172.21.15.82:0/3954674364 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f225c1a0150 con 0x7f225c1078d0 2024-05-20T16:13:36.115 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.112+0000 7f226089e640 1 -- 172.21.15.82:0/3954674364 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f225402fc50 con 0x7f225c1078d0 2024-05-20T16:13:36.115 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.112+0000 7f22618a0640 1 -- 172.21.15.82:0/3954674364 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f225c1a06f0 con 0x7f225c1078d0 2024-05-20T16:13:36.121 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.115+0000 7f226089e640 1 -- 172.21.15.82:0/3954674364 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f2254004050 con 0x7f225c1078d0 2024-05-20T16:13:36.121 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.115+0000 7f2268822740 1 Processor -- start 2024-05-20T16:13:36.121 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.115+0000 7f2268822740 1 -- start start 2024-05-20T16:13:36.121 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.115+0000 7f2268822740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a4ff597270 0x55a4ff939680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:36.121 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.115+0000 7f2268822740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a4ff56e3b0 0x55a4ff939c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:36.121 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.115+0000 7f2268822740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a4ff576590 0x55a4ff940b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:36.121 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.115+0000 7f2268822740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55a4ff89b4d0 con 0x55a4ff597270 2024-05-20T16:13:36.121 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.115+0000 7f2268822740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x55a4ff8433e0 con 0x55a4ff576590 2024-05-20T16:13:36.121 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.115+0000 7f2268822740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55a4ff897490 con 0x55a4ff56e3b0 2024-05-20T16:13:36.122 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.115+0000 7f2253fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a4ff597270 0x55a4ff939680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:13:36.122 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.115+0000 7f2253fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a4ff597270 0x55a4ff939680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:33564/0 (socket says 172.21.15.82:33564) 2024-05-20T16:13:36.122 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.115+0000 7f2253fff640 1 -- 172.21.15.82:0/3367073156 learned_addr learned my addr 172.21.15.82:0/3367073156 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:13:36.122 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.115+0000 7f225affd640 1 --2- 172.21.15.82:0/3367073156 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a4ff56e3b0 0x55a4ff939c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:13:36.122 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.115+0000 7f226089e640 1 --2- 172.21.15.82:0/3954674364 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f22480a2c10 0x7f22480a5120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:36.122 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.115+0000 7f2253fff640 1 -- 172.21.15.82:0/3367073156 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a4ff56e3b0 msgr2=0x55a4ff939c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:13:36.122 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.115+0000 7f2253fff640 1 --2- 172.21.15.82:0/3367073156 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a4ff56e3b0 0x55a4ff939c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:36.122 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.115+0000 7f2253fff640 1 -- 172.21.15.82:0/3367073156 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a4ff576590 msgr2=0x55a4ff940b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:13:36.122 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.115+0000 7f2253fff640 1 --2- 172.21.15.82:0/3367073156 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a4ff576590 0x55a4ff940b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:36.122 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.115+0000 7f2253fff640 1 -- 172.21.15.82:0/3367073156 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a4ff5af2d0 con 0x55a4ff597270 2024-05-20T16:13:36.122 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.116+0000 7f225b7fe640 1 --2- 172.21.15.82:0/3954674364 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f22480a2c10 0x7f22480a5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:13:36.122 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.116+0000 7f226089e640 1 -- 172.21.15.82:0/3954674364 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9092+0+0 (secure 0 0 0) 0x7f22540f07a0 con 0x7f225c1078d0 2024-05-20T16:13:36.122 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.116+0000 7f2253fff640 1 --2- 172.21.15.82:0/3367073156 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a4ff597270 0x55a4ff939680 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f2244008cf0 tx=0x7f2244010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f933fd4123a38b31 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:13:36.123 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.116+0000 7f2251ffb640 1 -- 172.21.15.82:0/3367073156 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2244016070 con 0x55a4ff597270 2024-05-20T16:13:36.123 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.116+0000 7f2251ffb640 1 -- 172.21.15.82:0/3367073156 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f22440047a0 con 0x55a4ff597270 2024-05-20T16:13:36.123 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.116+0000 7f2251ffb640 1 -- 172.21.15.82:0/3367073156 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2244012880 con 0x55a4ff597270 2024-05-20T16:13:36.123 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.117+0000 7f2268822740 1 -- 172.21.15.82:0/3367073156 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a4ff597270 msgr2=0x55a4ff939680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:36.123 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.117+0000 7f225b7fe640 1 --2- 172.21.15.82:0/3954674364 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f22480a2c10 0x7f22480a5120 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f22540b9100 tx=0x7f22540095c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:13:36.123 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.117+0000 7f2268822740 1 --2- 172.21.15.82:0/3367073156 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a4ff597270 0x55a4ff939680 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f2244008cf0 tx=0x7f2244010740 comp rx=0 tx=0).stop 2024-05-20T16:13:36.123 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.117+0000 7f2268822740 1 -- 172.21.15.82:0/3367073156 shutdown_connections 2024-05-20T16:13:36.123 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.117+0000 7f2268822740 1 --2- 172.21.15.82:0/3367073156 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a4ff576590 0x55a4ff940b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:36.123 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.117+0000 7f2268822740 1 --2- 172.21.15.82:0/3367073156 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a4ff56e3b0 0x55a4ff939c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:36.123 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.117+0000 7f2268822740 1 --2- 172.21.15.82:0/3367073156 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a4ff597270 0x55a4ff939680 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:36.123 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.117+0000 7f2268822740 1 -- 172.21.15.82:0/3367073156 >> 172.21.15.82:0/3367073156 conn(0x55a4ff58b3c0 msgr2=0x55a4ff932e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:13:36.123 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.117+0000 7f2268822740 1 -- 172.21.15.82:0/3367073156 shutdown_connections 2024-05-20T16:13:36.123 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.117+0000 7f2268822740 1 -- 172.21.15.82:0/3367073156 wait complete. 2024-05-20T16:13:36.123 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.117+0000 7f2268822740 1 -- start start 2024-05-20T16:13:36.123 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.117+0000 7f2268822740 1 Processor -- start 2024-05-20T16:13:36.124 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.118+0000 7f2268822740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a4ff56e3b0 0x55a4ff943500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:36.124 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.118+0000 7f2268822740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a4ff576590 0x55a4ff943a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:36.124 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.118+0000 7f2268822740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a4ff597270 0x55a4ff94a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:36.124 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.118+0000 7f2268822740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55a4ff941550 con 0x55a4ff597270 2024-05-20T16:13:36.124 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.118+0000 7f2268822740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x55a4ff8568e0 con 0x55a4ff56e3b0 2024-05-20T16:13:36.124 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.118+0000 7f2268822740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55a4ff9416d0 con 0x55a4ff576590 2024-05-20T16:13:36.124 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.118+0000 7f2268822740 10 client.0 _ll_register_callbacks cb 0x55a4ff8239c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T16:13:36.124 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.118+0000 7f2253fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a4ff597270 0x55a4ff94a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:13:36.124 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.118+0000 7f2253fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a4ff597270 0x55a4ff94a9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:33572/0 (socket says 172.21.15.82:33572) 2024-05-20T16:13:36.124 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.118+0000 7f2253fff640 1 -- 172.21.15.82:0/676785627 learned_addr learned my addr 172.21.15.82:0/676785627 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:13:36.124 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.118+0000 7f225affd640 1 --2- 172.21.15.82:0/676785627 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a4ff576590 0x55a4ff943a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:13:36.124 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.118+0000 7f2253fff640 1 --2- 172.21.15.82:0/676785627 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a4ff56e3b0 0x55a4ff943500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:13:36.124 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.118+0000 7f2253fff640 1 -- 172.21.15.82:0/676785627 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a4ff576590 msgr2=0x55a4ff943a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:13:36.125 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.118+0000 7f2253fff640 1 --2- 172.21.15.82:0/676785627 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a4ff576590 0x55a4ff943a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:36.125 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.118+0000 7f2253fff640 1 -- 172.21.15.82:0/676785627 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a4ff56e3b0 msgr2=0x55a4ff943500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:13:36.125 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.118+0000 7f2253fff640 1 --2- 172.21.15.82:0/676785627 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a4ff56e3b0 0x55a4ff943500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:36.125 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.118+0000 7f2253fff640 1 -- 172.21.15.82:0/676785627 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a4ff6796b0 con 0x55a4ff597270 2024-05-20T16:13:36.125 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.118+0000 7f2253fff640 1 --2- 172.21.15.82:0/676785627 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a4ff597270 0x55a4ff94a9a0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f2244008cc0 tx=0x7f2244005f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60870d3e82d27ad3 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:13:36.125 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.118+0000 7f223affd640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-05-20T16:13:36.125 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.118+0000 7f223affd640 1 -- 172.21.15.82:0/676785627 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f224401f070 con 0x55a4ff597270 2024-05-20T16:13:36.125 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.118+0000 7f223affd640 1 -- 172.21.15.82:0/676785627 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9092+0+0 (secure 0 0 0) 0x7f2244004fe0 con 0x55a4ff597270 2024-05-20T16:13:36.125 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.118+0000 7f2268822740 1 -- 172.21.15.82:0/676785627 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f22440047a0 con 0x55a4ff597270 2024-05-20T16:13:36.125 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.118+0000 7f223affd640 1 -- 172.21.15.82:0/676785627 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f2244031050 con 0x55a4ff597270 2024-05-20T16:13:36.125 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.119+0000 7f2268822740 10 client.15225 fetch_fsmap learned FSMap version 24 2024-05-20T16:13:36.125 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.119+0000 7f2268822740 1 -- 172.21.15.82:0/676785627 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55a4ff94d940 con 0x55a4ff597270 2024-05-20T16:13:36.125 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.119+0000 7f223affd640 1 -- 172.21.15.82:0/676785627 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f22440047a0 con 0x55a4ff597270 2024-05-20T16:13:36.125 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.119+0000 7f223affd640 1 -- 172.21.15.82:0/676785627 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f224401ab00 con 0x55a4ff597270 2024-05-20T16:13:36.126 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.119+0000 7f2268822740 10 client.15225 fetch_fsmap finished waiting for FSMap version 24 2024-05-20T16:13:36.126 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.119+0000 7f2268822740 10 client.15225 resolve_mds: resolved name 'h' to daemon mds.h 2024-05-20T16:13:36.126 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.119+0000 7f2268822740 20 client.15225 populate_metadata read hostname 'smithi082' 2024-05-20T16:13:36.126 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.119+0000 7f2268822740 1 --2- 172.21.15.82:0/676785627 >> [v2:172.21.15.160:6836/2846188680,v1:172.21.15.160:6837/2846188680] conn(0x55a4ff555e40 0x55a4ff951250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:36.126 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.119+0000 7f2268822740 4 client.15225 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T16:13:36.126 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.119+0000 7f2268822740 1 -- 172.21.15.82:0/676785627 --> [v2:172.21.15.160:6836/2846188680,v1:172.21.15.160:6837/2846188680] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a4ff8a8c00 con 0x55a4ff555e40 2024-05-20T16:13:36.126 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.119+0000 7f225affd640 1 --2- 172.21.15.82:0/676785627 >> [v2:172.21.15.160:6836/2846188680,v1:172.21.15.160:6837/2846188680] conn(0x55a4ff555e40 0x55a4ff951250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:13:36.126 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.120+0000 7f225affd640 1 --2- 172.21.15.82:0/676785627 >> [v2:172.21.15.160:6836/2846188680,v1:172.21.15.160:6837/2846188680] conn(0x55a4ff555e40 0x55a4ff951250 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=fab8c87e88068361 server_cookie=86576e5592bd775 in_seq=0 out_seq=0 2024-05-20T16:13:36.126 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.120+0000 7f223affd640 10 client.15225 ms_handle_connect on v2:172.21.15.160:6836/2846188680 2024-05-20T16:13:36.126 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.121+0000 7f223affd640 1 -- 172.21.15.82:0/676785627 <== mds.2 v2:172.21.15.160:6836/2846188680 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55a4ff8a8c00 con 0x55a4ff555e40 2024-05-20T16:13:36.126 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.121+0000 7f223affd640 10 client.15225 handle_command_reply: tid=0 2024-05-20T16:13:36.126 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.121+0000 7f2268822740 1 client.15225 shutdown 2024-05-20T16:13:36.126 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.121+0000 7f2268822740 1 -- 172.21.15.82:0/676785627 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a4ff597270 msgr2=0x55a4ff94a9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:36.126 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.121+0000 7f2268822740 1 --2- 172.21.15.82:0/676785627 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a4ff597270 0x55a4ff94a9a0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f2244008cc0 tx=0x7f2244005f40 comp rx=0 tx=0).stop 2024-05-20T16:13:36.127 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.121+0000 7f2268822740 1 -- 172.21.15.82:0/676785627 shutdown_connections 2024-05-20T16:13:36.127 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.121+0000 7f2268822740 1 --2- 172.21.15.82:0/676785627 >> [v2:172.21.15.160:6836/2846188680,v1:172.21.15.160:6837/2846188680] conn(0x55a4ff555e40 0x55a4ff951250 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:36.127 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.121+0000 7f2268822740 1 --2- 172.21.15.82:0/676785627 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a4ff597270 0x55a4ff94a9a0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:36.127 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.121+0000 7f2268822740 1 --2- 172.21.15.82:0/676785627 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a4ff576590 0x55a4ff943a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:36.127 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.121+0000 7f223affd640 0 client.15225 ms_handle_reset on v2:172.21.15.160:6836/2846188680 2024-05-20T16:13:36.127 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.121+0000 7f2268822740 1 --2- 172.21.15.82:0/676785627 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a4ff56e3b0 0x55a4ff943500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:36.127 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.121+0000 7f2268822740 1 -- 172.21.15.82:0/676785627 >> 172.21.15.82:0/676785627 conn(0x55a4ff58b3c0 msgr2=0x55a4ff931ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:13:36.127 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.121+0000 7f2268822740 1 -- 172.21.15.82:0/676785627 shutdown_connections 2024-05-20T16:13:36.127 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.121+0000 7f2268822740 1 -- 172.21.15.82:0/676785627 wait complete. 2024-05-20T16:13:36.127 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.121+0000 7f2268822740 20 client.15225 trim_cache size 0 max 16384 2024-05-20T16:13:36.135 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:13:35 smithi184 ceph-mon[36209]: from='client.? 172.21.15.82:0/4021219936' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-20T16:13:36.135 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:13:35 smithi184 ceph-mon[36209]: pgmap v348: 129 pgs: 129 active+clean; 641 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-20T16:13:36.135 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:13:35 smithi184 ceph-mon[36209]: from='client.? 172.21.15.82:0/92715348' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T16:13:36.136 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:13:35 smithi184 ceph-mon[36209]: from='client.? 172.21.15.82:0/1022431930' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-20T16:13:36.141 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.139+0000 7f2268822740 1 Processor -- start 2024-05-20T16:13:36.141 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.139+0000 7f2268822740 1 -- start start 2024-05-20T16:13:36.141 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.139+0000 7f2268822740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a4ff555e40 0x55a4ff942ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:36.141 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.139+0000 7f2268822740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a4ff56e3b0 0x55a4ff93cc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:36.141 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.139+0000 7f2268822740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a4ff576590 0x55a4ff93d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:36.141 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.139+0000 7f2268822740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55a4ff94f590 con 0x55a4ff56e3b0 2024-05-20T16:13:36.141 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.139+0000 7f2268822740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x55a4ff94fc60 con 0x55a4ff576590 2024-05-20T16:13:36.141 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.139+0000 7f2268822740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55a4ff950330 con 0x55a4ff555e40 2024-05-20T16:13:36.141 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.140+0000 7f225affd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a4ff56e3b0 0x55a4ff93cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:13:36.141 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.140+0000 7f225affd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a4ff56e3b0 0x55a4ff93cc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:33588/0 (socket says 172.21.15.82:33588) 2024-05-20T16:13:36.142 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.140+0000 7f225affd640 1 -- 172.21.15.82:0/1029689665 learned_addr learned my addr 172.21.15.82:0/1029689665 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:13:36.142 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.140+0000 7f225affd640 1 -- 172.21.15.82:0/1029689665 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a4ff555e40 msgr2=0x55a4ff942ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:13:36.142 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.140+0000 7f225affd640 1 --2- 172.21.15.82:0/1029689665 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a4ff555e40 0x55a4ff942ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:36.142 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.140+0000 7f225affd640 1 -- 172.21.15.82:0/1029689665 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a4ff576590 msgr2=0x55a4ff93d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:13:36.142 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.140+0000 7f225affd640 1 --2- 172.21.15.82:0/1029689665 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a4ff576590 0x55a4ff93d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:36.142 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.140+0000 7f225affd640 1 -- 172.21.15.82:0/1029689665 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a4ff944570 con 0x55a4ff56e3b0 2024-05-20T16:13:36.142 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.140+0000 7f225affd640 1 --2- 172.21.15.82:0/1029689665 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a4ff56e3b0 0x55a4ff93cc10 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f225c102590 tx=0x7f224c014a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=951ae82c5e9846da server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:13:36.142 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.140+0000 7f22517fa640 1 -- 172.21.15.82:0/1029689665 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f224c005020 con 0x55a4ff56e3b0 2024-05-20T16:13:36.142 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.140+0000 7f22517fa640 1 -- 172.21.15.82:0/1029689665 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f224c0047a0 con 0x55a4ff56e3b0 2024-05-20T16:13:36.142 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.140+0000 7f22517fa640 1 -- 172.21.15.82:0/1029689665 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f224c011830 con 0x55a4ff56e3b0 2024-05-20T16:13:36.142 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.141+0000 7f2268822740 1 -- 172.21.15.82:0/1029689665 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a4ff56e3b0 msgr2=0x55a4ff93cc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:36.142 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.141+0000 7f2268822740 1 --2- 172.21.15.82:0/1029689665 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a4ff56e3b0 0x55a4ff93cc10 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f225c102590 tx=0x7f224c014a00 comp rx=0 tx=0).stop 2024-05-20T16:13:36.142 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.141+0000 7f2268822740 1 -- 172.21.15.82:0/1029689665 shutdown_connections 2024-05-20T16:13:36.143 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.141+0000 7f2268822740 1 --2- 172.21.15.82:0/1029689665 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a4ff56e3b0 0x55a4ff93cc10 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:36.143 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.141+0000 7f2268822740 1 --2- 172.21.15.82:0/1029689665 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a4ff576590 0x55a4ff93d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:36.143 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.141+0000 7f2268822740 1 --2- 172.21.15.82:0/1029689665 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a4ff555e40 0x55a4ff942ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:36.143 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.141+0000 7f2268822740 1 -- 172.21.15.82:0/1029689665 >> 172.21.15.82:0/1029689665 conn(0x55a4ff58b3c0 msgr2=0x55a4ff94edb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:13:36.143 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.141+0000 7f2268822740 1 -- 172.21.15.82:0/1029689665 shutdown_connections 2024-05-20T16:13:36.143 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.141+0000 7f2268822740 1 -- 172.21.15.82:0/1029689665 wait complete. 2024-05-20T16:13:36.143 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.141+0000 7f2268822740 1 -- start start 2024-05-20T16:13:36.146 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.142+0000 7f2268822740 1 Processor -- start 2024-05-20T16:13:36.147 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.142+0000 7f2268822740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a4ff555e40 0x55a4ff944ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:36.147 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.142+0000 7f2268822740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a4ff56e3b0 0x55a4ff949f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:36.147 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.142+0000 7f2268822740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a4ff576590 0x55a4ff945130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:36.147 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.142+0000 7f2268822740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55a4ff9510d0 con 0x55a4ff555e40 2024-05-20T16:13:36.147 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.142+0000 7f2268822740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x55a4ff950a00 con 0x55a4ff56e3b0 2024-05-20T16:13:36.147 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.142+0000 7f2268822740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55a4ff9517a0 con 0x55a4ff576590 2024-05-20T16:13:36.147 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.142+0000 7f2268822740 10 client.0 _ll_register_callbacks cb 0x7f2244008bc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T16:13:36.147 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.142+0000 7f2253fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a4ff555e40 0x55a4ff944ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:13:36.147 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.142+0000 7f2253fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a4ff555e40 0x55a4ff944ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:33594/0 (socket says 172.21.15.82:33594) 2024-05-20T16:13:36.147 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.142+0000 7f2253fff640 1 -- 172.21.15.82:0/1180233172 learned_addr learned my addr 172.21.15.82:0/1180233172 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:13:36.147 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.142+0000 7f2253fff640 1 --2- 172.21.15.82:0/1180233172 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a4ff576590 0x55a4ff945130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:13:36.147 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.142+0000 7f225affd640 1 --2- 172.21.15.82:0/1180233172 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a4ff56e3b0 0x55a4ff949f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:13:36.148 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.142+0000 7f2253fff640 1 -- 172.21.15.82:0/1180233172 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a4ff576590 msgr2=0x55a4ff945130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:13:36.148 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.142+0000 7f2253fff640 1 --2- 172.21.15.82:0/1180233172 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a4ff576590 0x55a4ff945130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:36.148 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.142+0000 7f2253fff640 1 -- 172.21.15.82:0/1180233172 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a4ff56e3b0 msgr2=0x55a4ff949f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:13:36.148 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.142+0000 7f2253fff640 1 --2- 172.21.15.82:0/1180233172 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a4ff56e3b0 0x55a4ff949f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:36.148 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.142+0000 7f2253fff640 1 -- 172.21.15.82:0/1180233172 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a4ff9b80f0 con 0x55a4ff555e40 2024-05-20T16:13:36.148 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.142+0000 7f225affd640 1 --2- 172.21.15.82:0/1180233172 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a4ff56e3b0 0x55a4ff949f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T16:13:36.148 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.142+0000 7f2253fff640 1 --2- 172.21.15.82:0/1180233172 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a4ff555e40 0x55a4ff944ae0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f2244008b00 tx=0x7f224400e450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b746afca267d8f4 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:13:36.148 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.142+0000 7f223affd640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-05-20T16:13:36.148 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.142+0000 7f223affd640 1 -- 172.21.15.82:0/1180233172 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f224401f040 con 0x55a4ff555e40 2024-05-20T16:13:36.148 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.142+0000 7f223affd640 1 -- 172.21.15.82:0/1180233172 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9092+0+0 (secure 0 0 0) 0x7f2244012480 con 0x55a4ff555e40 2024-05-20T16:13:36.148 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.142+0000 7f2268822740 1 -- 172.21.15.82:0/1180233172 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f224c0047a0 con 0x55a4ff555e40 2024-05-20T16:13:36.148 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.143+0000 7f223affd640 1 -- 172.21.15.82:0/1180233172 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f2244016040 con 0x55a4ff555e40 2024-05-20T16:13:36.148 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.143+0000 7f2268822740 10 client.15231 fetch_fsmap learned FSMap version 24 2024-05-20T16:13:36.148 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.143+0000 7f2268822740 1 -- 172.21.15.82:0/1180233172 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55a4ff937080 con 0x55a4ff555e40 2024-05-20T16:13:36.148 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.143+0000 7f223affd640 1 -- 172.21.15.82:0/1180233172 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f224c0047a0 con 0x55a4ff555e40 2024-05-20T16:13:36.149 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.143+0000 7f223affd640 1 -- 172.21.15.82:0/1180233172 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f224400ea20 con 0x55a4ff555e40 2024-05-20T16:13:36.149 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.143+0000 7f2268822740 10 client.15231 fetch_fsmap finished waiting for FSMap version 24 2024-05-20T16:13:36.149 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.143+0000 7f2268822740 10 client.15231 resolve_mds: resolved name 'h' to daemon mds.h 2024-05-20T16:13:36.149 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.143+0000 7f2268822740 20 client.15231 populate_metadata read hostname 'smithi082' 2024-05-20T16:13:36.149 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.143+0000 7f2268822740 1 --2- 172.21.15.82:0/1180233172 >> [v2:172.21.15.160:6836/2846188680,v1:172.21.15.160:6837/2846188680] conn(0x55a4ff597270 0x55a4ff938130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:36.149 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.143+0000 7f2268822740 4 client.15231 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-20T16:13:36.149 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.143+0000 7f2268822740 1 -- 172.21.15.82:0/1180233172 --> [v2:172.21.15.160:6836/2846188680,v1:172.21.15.160:6837/2846188680] -- command(tid 0: {"prefix": "status"}) -- 0x55a4ff8a2f50 con 0x55a4ff597270 2024-05-20T16:13:36.149 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.143+0000 7f225affd640 1 --2- 172.21.15.82:0/1180233172 >> [v2:172.21.15.160:6836/2846188680,v1:172.21.15.160:6837/2846188680] conn(0x55a4ff597270 0x55a4ff938130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:13:36.149 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.144+0000 7f223affd640 10 client.15231 ms_handle_connect on v2:172.21.15.160:6836/2846188680 2024-05-20T16:13:36.149 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.144+0000 7f225affd640 1 --2- 172.21.15.82:0/1180233172 >> [v2:172.21.15.160:6836/2846188680,v1:172.21.15.160:6837/2846188680] conn(0x55a4ff597270 0x55a4ff938130 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a2c934a0841a0c10 server_cookie=9e4aa0177180bb34 in_seq=0 out_seq=0 2024-05-20T16:13:36.149 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.144+0000 7f223affd640 1 -- 172.21.15.82:0/1180233172 <== mds.2 v2:172.21.15.160:6836/2846188680 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55a4ff8a2f50 con 0x55a4ff597270 2024-05-20T16:13:36.149 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.144+0000 7f223affd640 10 client.15231 handle_command_reply: tid=0 2024-05-20T16:13:36.149 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.144+0000 7f2268822740 1 client.15231 shutdown 2024-05-20T16:13:36.149 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.144+0000 7f2268822740 1 -- 172.21.15.82:0/1180233172 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a4ff555e40 msgr2=0x55a4ff944ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:36.150 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.144+0000 7f2268822740 1 --2- 172.21.15.82:0/1180233172 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a4ff555e40 0x55a4ff944ae0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f2244008b00 tx=0x7f224400e450 comp rx=0 tx=0).stop 2024-05-20T16:13:36.150 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.144+0000 7f2268822740 1 -- 172.21.15.82:0/1180233172 shutdown_connections 2024-05-20T16:13:36.150 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.144+0000 7f2268822740 1 --2- 172.21.15.82:0/1180233172 >> [v2:172.21.15.160:6836/2846188680,v1:172.21.15.160:6837/2846188680] conn(0x55a4ff597270 0x55a4ff938130 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:36.150 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.144+0000 7f2268822740 1 --2- 172.21.15.82:0/1180233172 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a4ff576590 0x55a4ff945130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:36.150 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.144+0000 7f223affd640 0 client.15231 ms_handle_reset on v2:172.21.15.160:6836/2846188680 2024-05-20T16:13:36.150 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.144+0000 7f2268822740 1 --2- 172.21.15.82:0/1180233172 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a4ff56e3b0 0x55a4ff949f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:36.150 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.144+0000 7f2268822740 1 --2- 172.21.15.82:0/1180233172 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a4ff555e40 0x55a4ff944ae0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:36.150 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.144+0000 7f2268822740 1 -- 172.21.15.82:0/1180233172 >> 172.21.15.82:0/1180233172 conn(0x55a4ff58b3c0 msgr2=0x55a4ff93cb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:13:36.150 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.144+0000 7f2268822740 1 -- 172.21.15.82:0/1180233172 shutdown_connections 2024-05-20T16:13:36.150 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.144+0000 7f2268822740 1 -- 172.21.15.82:0/1180233172 wait complete. 2024-05-20T16:13:36.150 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.144+0000 7f2268822740 20 client.15231 trim_cache size 0 max 16384 2024-05-20T16:13:36.151 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-05-20T16:13:36.151 INFO:teuthology.orchestra.run.smithi082.stdout: "cluster_fsid": "59d1600a-16c2-11ef-bc9a-c7b262605968", 2024-05-20T16:13:36.151 INFO:teuthology.orchestra.run.smithi082.stdout: "whoami": 2, 2024-05-20T16:13:36.151 INFO:teuthology.orchestra.run.smithi082.stdout: "id": 24506, 2024-05-20T16:13:36.151 INFO:teuthology.orchestra.run.smithi082.stdout: "want_state": "up:active", 2024-05-20T16:13:36.151 INFO:teuthology.orchestra.run.smithi082.stdout: "state": "up:active", 2024-05-20T16:13:36.151 INFO:teuthology.orchestra.run.smithi082.stdout: "fs_name": "cephfs", 2024-05-20T16:13:36.151 INFO:teuthology.orchestra.run.smithi082.stdout: "rank_uptime": 83.757758456999994, 2024-05-20T16:13:36.151 INFO:teuthology.orchestra.run.smithi082.stdout: "mdsmap_epoch": 24, 2024-05-20T16:13:36.151 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_epoch": 93, 2024-05-20T16:13:36.151 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_epoch_barrier": 91, 2024-05-20T16:13:36.151 INFO:teuthology.orchestra.run.smithi082.stdout: "uptime": 124.11712645199999 2024-05-20T16:13:36.151 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-05-20T16:13:36.152 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.145+0000 7f22618a0640 1 -- 172.21.15.82:0/3954674364 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f22480a2c10 msgr2=0x7f22480a5120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:36.152 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.145+0000 7f22618a0640 1 --2- 172.21.15.82:0/3954674364 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f22480a2c10 0x7f22480a5120 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f22540b9100 tx=0x7f22540095c0 comp rx=0 tx=0).stop 2024-05-20T16:13:36.152 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.145+0000 7f22618a0640 1 -- 172.21.15.82:0/3954674364 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f225c1078d0 msgr2=0x7f225c19f900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:36.152 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.145+0000 7f22618a0640 1 --2- 172.21.15.82:0/3954674364 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f225c1078d0 0x7f225c19f900 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f2254005cc0 tx=0x7f2254009890 comp rx=0 tx=0).stop 2024-05-20T16:13:36.153 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.145+0000 7f22618a0640 1 -- 172.21.15.82:0/3954674364 shutdown_connections 2024-05-20T16:13:36.153 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.145+0000 7f22618a0640 1 --2- 172.21.15.82:0/3954674364 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f22480a2c10 0x7f22480a5120 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:36.153 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.145+0000 7f22618a0640 1 --2- 172.21.15.82:0/3954674364 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f225c1078d0 0x7f225c19f900 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:36.153 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.145+0000 7f22618a0640 1 --2- 172.21.15.82:0/3954674364 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f225c102bd0 0x7f225c198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:36.153 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.145+0000 7f22618a0640 1 --2- 172.21.15.82:0/3954674364 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f225c101990 0x7f225c198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:36.153 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.145+0000 7f22618a0640 1 -- 172.21.15.82:0/3954674364 >> 172.21.15.82:0/3954674364 conn(0x7f225c0fcb80 msgr2=0x7f225c1038b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:13:36.153 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.145+0000 7f22618a0640 1 -- 172.21.15.82:0/3954674364 shutdown_connections 2024-05-20T16:13:36.153 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.145+0000 7f22618a0640 1 -- 172.21.15.82:0/3954674364 wait complete. 2024-05-20T16:13:36.156 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-05-20T16:13:36.269 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.269+0000 7f8321e61640 1 -- 172.21.15.82:0/1690073006 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f831c1019b0 msgr2=0x7f831c1026b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:36.269 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.269+0000 7f8321e61640 1 --2- 172.21.15.82:0/1690073006 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f831c1019b0 0x7f831c1026b0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f830c009900 tx=0x7f830c02a410 comp rx=0 tx=0).stop 2024-05-20T16:13:36.269 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.269+0000 7f8321e61640 1 -- 172.21.15.82:0/1690073006 shutdown_connections 2024-05-20T16:13:36.269 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.269+0000 7f8321e61640 1 --2- 172.21.15.82:0/1690073006 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f831c1078f0 0x7f831c109d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:36.269 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.269+0000 7f8321e61640 1 --2- 172.21.15.82:0/1690073006 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f831c102bf0 0x7f831c103000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:36.269 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.269+0000 7f8321e61640 1 --2- 172.21.15.82:0/1690073006 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f831c1019b0 0x7f831c1026b0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:36.269 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.269+0000 7f8321e61640 1 -- 172.21.15.82:0/1690073006 >> 172.21.15.82:0/1690073006 conn(0x7f831c0fcba0 msgr2=0x7f831c0ff050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:13:36.269 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.269+0000 7f8321e61640 1 -- 172.21.15.82:0/1690073006 shutdown_connections 2024-05-20T16:13:36.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.269+0000 7f8321e61640 1 -- 172.21.15.82:0/1690073006 wait complete. 2024-05-20T16:13:36.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.269+0000 7f8321e61640 1 Processor -- start 2024-05-20T16:13:36.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.270+0000 7f8321e61640 1 -- start start 2024-05-20T16:13:36.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.270+0000 7f8321e61640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f831c1019b0 0x7f831c1992d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:36.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.270+0000 7f8321e61640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f831c102bf0 0x7f831c199810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:36.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.270+0000 7f8321e61640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f831c1078f0 0x7f831c1a07a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:36.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.270+0000 7f8321e61640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f831c10e800 con 0x7f831c102bf0 2024-05-20T16:13:36.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.270+0000 7f8321e61640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f831c074c10 con 0x7f831c1019b0 2024-05-20T16:13:36.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.270+0000 7f8321e61640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f831c10e980 con 0x7f831c1078f0 2024-05-20T16:13:36.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.270+0000 7f831affd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f831c102bf0 0x7f831c199810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:13:36.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.270+0000 7f831affd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f831c102bf0 0x7f831c199810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:33602/0 (socket says 172.21.15.82:33602) 2024-05-20T16:13:36.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.270+0000 7f831affd640 1 -- 172.21.15.82:0/1547364361 learned_addr learned my addr 172.21.15.82:0/1547364361 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:13:36.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.270+0000 7f831bfff640 1 --2- 172.21.15.82:0/1547364361 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f831c1078f0 0x7f831c1a07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:13:36.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.270+0000 7f831b7fe640 1 --2- 172.21.15.82:0/1547364361 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f831c1019b0 0x7f831c1992d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:13:36.273 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.270+0000 7f831affd640 1 -- 172.21.15.82:0/1547364361 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f831c1078f0 msgr2=0x7f831c1a07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:36.273 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.270+0000 7f831affd640 1 --2- 172.21.15.82:0/1547364361 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f831c1078f0 0x7f831c1a07a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:36.273 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.270+0000 7f831affd640 1 -- 172.21.15.82:0/1547364361 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f831c1019b0 msgr2=0x7f831c1992d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:36.273 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.270+0000 7f831affd640 1 --2- 172.21.15.82:0/1547364361 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f831c1019b0 0x7f831c1992d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:36.273 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.270+0000 7f831affd640 1 -- 172.21.15.82:0/1547364361 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f831c1a0e10 con 0x7f831c102bf0 2024-05-20T16:13:36.273 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.270+0000 7f831b7fe640 1 --2- 172.21.15.82:0/1547364361 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f831c1019b0 0x7f831c1992d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T16:13:36.273 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.271+0000 7f831affd640 1 --2- 172.21.15.82:0/1547364361 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f831c102bf0 0x7f831c199810 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f831000d770 tx=0x7f831000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:13:36.273 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.271+0000 7f8318ff9640 1 -- 172.21.15.82:0/1547364361 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8310017070 con 0x7f831c102bf0 2024-05-20T16:13:36.273 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.271+0000 7f8318ff9640 1 -- 172.21.15.82:0/1547364361 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8310005500 con 0x7f831c102bf0 2024-05-20T16:13:36.273 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.271+0000 7f8318ff9640 1 -- 172.21.15.82:0/1547364361 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8310004e00 con 0x7f831c102bf0 2024-05-20T16:13:36.273 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.271+0000 7f8321e61640 1 -- 172.21.15.82:0/1547364361 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f831c1a1100 con 0x7f831c102bf0 2024-05-20T16:13:36.274 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.271+0000 7f8321e61640 1 -- 172.21.15.82:0/1547364361 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f831c1a1650 con 0x7f831c102bf0 2024-05-20T16:13:36.280 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.273+0000 7f8318ff9640 1 -- 172.21.15.82:0/1547364361 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f8310004fa0 con 0x7f831c102bf0 2024-05-20T16:13:36.280 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.273+0000 7f8328de8740 1 Processor -- start 2024-05-20T16:13:36.280 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.273+0000 7f8328de8740 1 -- start start 2024-05-20T16:13:36.280 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.274+0000 7f8328de8740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5653ec4c7270 0x5653ec867470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:36.280 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.274+0000 7f8328de8740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5653ec49e3b0 0x5653ec8679f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:36.280 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.274+0000 7f8328de8740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5653ec4a6590 0x5653ec86e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:36.280 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.274+0000 7f8328de8740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x5653ec7cb4d0 con 0x5653ec4a6590 2024-05-20T16:13:36.280 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.274+0000 7f8328de8740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x5653ec7733e0 con 0x5653ec49e3b0 2024-05-20T16:13:36.281 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.274+0000 7f8328de8740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5653ec7c7490 con 0x5653ec4c7270 2024-05-20T16:13:36.281 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.274+0000 7f831affd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5653ec4a6590 0x5653ec86e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:13:36.281 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.274+0000 7f831affd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5653ec4a6590 0x5653ec86e8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:33604/0 (socket says 172.21.15.82:33604) 2024-05-20T16:13:36.281 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.274+0000 7f831affd640 1 -- 172.21.15.82:0/1494053717 learned_addr learned my addr 172.21.15.82:0/1494053717 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:13:36.281 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.274+0000 7f831bfff640 1 --2- 172.21.15.82:0/1494053717 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5653ec4c7270 0x5653ec867470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:13:36.281 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.274+0000 7f8318ff9640 1 --2- 172.21.15.82:0/1547364361 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f83040a2c10 0x7f83040a5120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:36.281 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.274+0000 7f831b7fe640 1 --2- 172.21.15.82:0/1494053717 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5653ec49e3b0 0x5653ec8679f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:13:36.281 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.274+0000 7f831affd640 1 -- 172.21.15.82:0/1494053717 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5653ec4c7270 msgr2=0x5653ec867470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:13:36.281 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.274+0000 7f831affd640 1 --2- 172.21.15.82:0/1494053717 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5653ec4c7270 0x5653ec867470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:36.281 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.274+0000 7f831affd640 1 -- 172.21.15.82:0/1494053717 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5653ec49e3b0 msgr2=0x5653ec8679f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:13:36.281 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.274+0000 7f831affd640 1 --2- 172.21.15.82:0/1494053717 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5653ec49e3b0 0x5653ec8679f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:36.281 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.274+0000 7f831affd640 1 -- 172.21.15.82:0/1494053717 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5653ec4df2d0 con 0x5653ec4a6590 2024-05-20T16:13:36.281 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.274+0000 7f831b7fe640 1 --2- 172.21.15.82:0/1494053717 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5653ec49e3b0 0x5653ec8679f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T16:13:36.281 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.274+0000 7f831bfff640 1 --2- 172.21.15.82:0/1547364361 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f83040a2c10 0x7f83040a5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:13:36.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.274+0000 7f8318ff9640 1 -- 172.21.15.82:0/1547364361 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9092+0+0 (secure 0 0 0) 0x7f83100ced70 con 0x7f831c102bf0 2024-05-20T16:13:36.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.275+0000 7f831affd640 1 --2- 172.21.15.82:0/1494053717 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5653ec4a6590 0x5653ec86e8f0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f8310088080 tx=0x7f8310065ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fed31f67b21d2396 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:13:36.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.275+0000 7f8301ffb640 1 -- 172.21.15.82:0/1494053717 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f831009adb0 con 0x5653ec4a6590 2024-05-20T16:13:36.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.275+0000 7f8301ffb640 1 -- 172.21.15.82:0/1494053717 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8310088830 con 0x5653ec4a6590 2024-05-20T16:13:36.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.275+0000 7f831bfff640 1 --2- 172.21.15.82:0/1547364361 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f83040a2c10 0x7f83040a5120 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x5653ec868640 tx=0x7f8314004a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:13:36.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.275+0000 7f8301ffb640 1 -- 172.21.15.82:0/1494053717 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8310088630 con 0x5653ec4a6590 2024-05-20T16:13:36.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.275+0000 7f8328de8740 1 -- 172.21.15.82:0/1494053717 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5653ec4a6590 msgr2=0x5653ec86e8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:36.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.275+0000 7f8328de8740 1 --2- 172.21.15.82:0/1494053717 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5653ec4a6590 0x5653ec86e8f0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f8310088080 tx=0x7f8310065ef0 comp rx=0 tx=0).stop 2024-05-20T16:13:36.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.276+0000 7f8328de8740 1 -- 172.21.15.82:0/1494053717 shutdown_connections 2024-05-20T16:13:36.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.276+0000 7f8328de8740 1 --2- 172.21.15.82:0/1494053717 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5653ec4a6590 0x5653ec86e8f0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:36.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.276+0000 7f8328de8740 1 --2- 172.21.15.82:0/1494053717 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5653ec49e3b0 0x5653ec8679f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:36.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.276+0000 7f8328de8740 1 --2- 172.21.15.82:0/1494053717 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5653ec4c7270 0x5653ec867470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:36.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.276+0000 7f8328de8740 1 -- 172.21.15.82:0/1494053717 >> 172.21.15.82:0/1494053717 conn(0x5653ec4bb3c0 msgr2=0x5653ec862e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:13:36.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.276+0000 7f8328de8740 1 -- 172.21.15.82:0/1494053717 shutdown_connections 2024-05-20T16:13:36.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.276+0000 7f8328de8740 1 -- 172.21.15.82:0/1494053717 wait complete. 2024-05-20T16:13:36.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.276+0000 7f8328de8740 1 -- start start 2024-05-20T16:13:36.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.276+0000 7f8328de8740 1 Processor -- start 2024-05-20T16:13:36.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.276+0000 7f8328de8740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5653ec49e3b0 0x5653ec8711e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:36.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.276+0000 7f8328de8740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5653ec4a6590 0x5653ec871720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:36.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.276+0000 7f8328de8740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5653ec4c7270 0x5653ec878680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:36.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.276+0000 7f8328de8740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x5653ec86f2e0 con 0x5653ec4c7270 2024-05-20T16:13:36.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.276+0000 7f8328de8740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x5653ec7868e0 con 0x5653ec49e3b0 2024-05-20T16:13:36.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.276+0000 7f8328de8740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5653ec86f460 con 0x5653ec4a6590 2024-05-20T16:13:36.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.277+0000 7f8328de8740 10 client.0 _ll_register_callbacks cb 0x5653ec7539c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T16:13:36.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.277+0000 7f831bfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5653ec4c7270 0x5653ec878680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:13:36.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.277+0000 7f831bfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5653ec4c7270 0x5653ec878680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:33614/0 (socket says 172.21.15.82:33614) 2024-05-20T16:13:36.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.277+0000 7f831bfff640 1 -- 172.21.15.82:0/2026485897 learned_addr learned my addr 172.21.15.82:0/2026485897 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:13:36.284 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.277+0000 7f831affd640 1 --2- 172.21.15.82:0/2026485897 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5653ec4a6590 0x5653ec871720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:13:36.284 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.277+0000 7f831b7fe640 1 --2- 172.21.15.82:0/2026485897 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5653ec49e3b0 0x5653ec8711e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:13:36.284 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.277+0000 7f831bfff640 1 -- 172.21.15.82:0/2026485897 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5653ec4a6590 msgr2=0x5653ec871720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:13:36.284 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.277+0000 7f831bfff640 1 --2- 172.21.15.82:0/2026485897 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5653ec4a6590 0x5653ec871720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:36.284 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.277+0000 7f831bfff640 1 -- 172.21.15.82:0/2026485897 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5653ec49e3b0 msgr2=0x5653ec8711e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:13:36.284 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.277+0000 7f831bfff640 1 --2- 172.21.15.82:0/2026485897 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5653ec49e3b0 0x5653ec8711e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:36.284 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.277+0000 7f831bfff640 1 -- 172.21.15.82:0/2026485897 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5653ec5a96b0 con 0x5653ec4c7270 2024-05-20T16:13:36.284 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.277+0000 7f831affd640 1 --2- 172.21.15.82:0/2026485897 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5653ec4a6590 0x5653ec871720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T16:13:36.284 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.277+0000 7f831bfff640 1 --2- 172.21.15.82:0/2026485897 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5653ec4c7270 0x5653ec878680 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f83140126f0 tx=0x7f8314004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=861210917fb593a0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:13:36.284 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.277+0000 7f82faffd640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-05-20T16:13:36.284 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.277+0000 7f82faffd640 1 -- 172.21.15.82:0/2026485897 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8314010910 con 0x5653ec4c7270 2024-05-20T16:13:36.284 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.277+0000 7f82faffd640 1 -- 172.21.15.82:0/2026485897 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9092+0+0 (secure 0 0 0) 0x7f8314010ab0 con 0x5653ec4c7270 2024-05-20T16:13:36.284 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.277+0000 7f8328de8740 1 -- 172.21.15.82:0/2026485897 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5653ec7d8c00 con 0x5653ec4c7270 2024-05-20T16:13:36.285 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.277+0000 7f82faffd640 1 -- 172.21.15.82:0/2026485897 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f8314030050 con 0x5653ec4c7270 2024-05-20T16:13:36.285 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.277+0000 7f8328de8740 10 client.15243 fetch_fsmap learned FSMap version 24 2024-05-20T16:13:36.285 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.277+0000 7f8328de8740 1 -- 172.21.15.82:0/2026485897 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5653ec87b5f0 con 0x5653ec4c7270 2024-05-20T16:13:36.285 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.278+0000 7f82faffd640 1 -- 172.21.15.82:0/2026485897 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5653ec7d8c00 con 0x5653ec4c7270 2024-05-20T16:13:36.285 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.278+0000 7f82faffd640 1 -- 172.21.15.82:0/2026485897 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f831400f040 con 0x5653ec4c7270 2024-05-20T16:13:36.285 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.278+0000 7f8328de8740 10 client.15243 fetch_fsmap finished waiting for FSMap version 24 2024-05-20T16:13:36.285 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.278+0000 7f8328de8740 10 client.15243 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-20T16:13:36.285 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.278+0000 7f8328de8740 20 client.15243 populate_metadata read hostname 'smithi082' 2024-05-20T16:13:36.285 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.278+0000 7f8328de8740 1 --2- 172.21.15.82:0/2026485897 >> [v2:172.21.15.184:6838/2199955067,v1:172.21.15.184:6839/2199955067] conn(0x5653ec485e40 0x5653ec87f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:36.285 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.278+0000 7f8328de8740 4 client.15243 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T16:13:36.285 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.278+0000 7f8328de8740 1 -- 172.21.15.82:0/2026485897 --> [v2:172.21.15.184:6838/2199955067,v1:172.21.15.184:6839/2199955067] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5653ec7d2f50 con 0x5653ec485e40 2024-05-20T16:13:36.285 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.278+0000 7f831b7fe640 1 --2- 172.21.15.82:0/2026485897 >> [v2:172.21.15.184:6838/2199955067,v1:172.21.15.184:6839/2199955067] conn(0x5653ec485e40 0x5653ec87f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:13:36.285 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.279+0000 7f82faffd640 10 client.15243 ms_handle_connect on v2:172.21.15.184:6838/2199955067 2024-05-20T16:13:36.285 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.279+0000 7f831b7fe640 1 --2- 172.21.15.82:0/2026485897 >> [v2:172.21.15.184:6838/2199955067,v1:172.21.15.184:6839/2199955067] conn(0x5653ec485e40 0x5653ec87f0e0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e09040053a78c8ed server_cookie=f6da1f45f911ab94 in_seq=0 out_seq=0 2024-05-20T16:13:36.285 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.279+0000 7f82faffd640 1 -- 172.21.15.82:0/2026485897 <== mds.1 v2:172.21.15.184:6838/2199955067 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5653ec7d2f50 con 0x5653ec485e40 2024-05-20T16:13:36.286 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.279+0000 7f82faffd640 10 client.15243 handle_command_reply: tid=0 2024-05-20T16:13:36.286 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.280+0000 7f8328de8740 1 client.15243 shutdown 2024-05-20T16:13:36.286 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.280+0000 7f8328de8740 1 -- 172.21.15.82:0/2026485897 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5653ec4c7270 msgr2=0x5653ec878680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:36.286 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.280+0000 7f8328de8740 1 --2- 172.21.15.82:0/2026485897 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5653ec4c7270 0x5653ec878680 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f83140126f0 tx=0x7f8314004ab0 comp rx=0 tx=0).stop 2024-05-20T16:13:36.286 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.280+0000 7f8328de8740 1 -- 172.21.15.82:0/2026485897 shutdown_connections 2024-05-20T16:13:36.286 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.280+0000 7f8328de8740 1 --2- 172.21.15.82:0/2026485897 >> [v2:172.21.15.184:6838/2199955067,v1:172.21.15.184:6839/2199955067] conn(0x5653ec485e40 0x5653ec87f0e0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:36.286 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.280+0000 7f8328de8740 1 --2- 172.21.15.82:0/2026485897 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5653ec4c7270 0x5653ec878680 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:36.286 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.280+0000 7f8328de8740 1 --2- 172.21.15.82:0/2026485897 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5653ec4a6590 0x5653ec871720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:36.286 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.280+0000 7f82faffd640 0 client.15243 ms_handle_reset on v2:172.21.15.184:6838/2199955067 2024-05-20T16:13:36.286 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.280+0000 7f8328de8740 1 --2- 172.21.15.82:0/2026485897 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5653ec49e3b0 0x5653ec8711e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:36.286 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.280+0000 7f8328de8740 1 -- 172.21.15.82:0/2026485897 >> 172.21.15.82:0/2026485897 conn(0x5653ec4bb3c0 msgr2=0x5653ec861ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:13:36.286 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.280+0000 7f8328de8740 1 -- 172.21.15.82:0/2026485897 shutdown_connections 2024-05-20T16:13:36.286 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.280+0000 7f8328de8740 1 -- 172.21.15.82:0/2026485897 wait complete. 2024-05-20T16:13:36.286 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.280+0000 7f8328de8740 20 client.15243 trim_cache size 0 max 16384 2024-05-20T16:13:36.300 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.298+0000 7f8328de8740 1 Processor -- start 2024-05-20T16:13:36.300 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.298+0000 7f8328de8740 1 -- start start 2024-05-20T16:13:36.300 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.298+0000 7f8328de8740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5653ec485e40 0x5653ec870db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:36.300 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.298+0000 7f8328de8740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5653ec49e3b0 0x5653ec8785a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:36.300 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.298+0000 7f8328de8740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5653ec4a6590 0x5653ec8712f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:36.300 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.298+0000 7f8328de8740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x5653ec87d420 con 0x5653ec485e40 2024-05-20T16:13:36.301 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.298+0000 7f8328de8740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x5653ec87daf0 con 0x5653ec4a6590 2024-05-20T16:13:36.301 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.298+0000 7f8328de8740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5653ec87e1c0 con 0x5653ec49e3b0 2024-05-20T16:13:36.301 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.299+0000 7f831b7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5653ec485e40 0x5653ec870db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:13:36.301 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.299+0000 7f831b7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5653ec485e40 0x5653ec870db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:33620/0 (socket says 172.21.15.82:33620) 2024-05-20T16:13:36.301 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.299+0000 7f831b7fe640 1 -- 172.21.15.82:0/3324092975 learned_addr learned my addr 172.21.15.82:0/3324092975 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:13:36.301 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.299+0000 7f831bfff640 1 --2- 172.21.15.82:0/3324092975 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5653ec4a6590 0x5653ec8712f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:13:36.301 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.299+0000 7f831affd640 1 --2- 172.21.15.82:0/3324092975 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5653ec49e3b0 0x5653ec8785a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:13:36.301 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.299+0000 7f831b7fe640 1 -- 172.21.15.82:0/3324092975 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5653ec49e3b0 msgr2=0x5653ec8785a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:13:36.301 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.299+0000 7f831b7fe640 1 --2- 172.21.15.82:0/3324092975 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5653ec49e3b0 0x5653ec8785a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:36.301 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.299+0000 7f831b7fe640 1 -- 172.21.15.82:0/3324092975 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5653ec4a6590 msgr2=0x5653ec8712f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:13:36.301 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.299+0000 7f831b7fe640 1 --2- 172.21.15.82:0/3324092975 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5653ec4a6590 0x5653ec8712f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:36.301 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.299+0000 7f831b7fe640 1 -- 172.21.15.82:0/3324092975 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5653ec871de0 con 0x5653ec485e40 2024-05-20T16:13:36.301 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.299+0000 7f831affd640 1 --2- 172.21.15.82:0/3324092975 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5653ec49e3b0 0x5653ec8785a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T16:13:36.302 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.299+0000 7f831b7fe640 1 --2- 172.21.15.82:0/3324092975 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5653ec485e40 0x5653ec870db0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f830c009900 tx=0x7f830c0326c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dea58dc7136b870a server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:13:36.302 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.299+0000 7f83017fa640 1 -- 172.21.15.82:0/3324092975 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f830c04a070 con 0x5653ec485e40 2024-05-20T16:13:36.302 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.299+0000 7f83017fa640 1 -- 172.21.15.82:0/3324092975 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f830c004040 con 0x5653ec485e40 2024-05-20T16:13:36.302 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.299+0000 7f83017fa640 1 -- 172.21.15.82:0/3324092975 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f830c035040 con 0x5653ec485e40 2024-05-20T16:13:36.302 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.300+0000 7f8328de8740 1 -- 172.21.15.82:0/3324092975 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5653ec485e40 msgr2=0x5653ec870db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:36.302 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.300+0000 7f8328de8740 1 --2- 172.21.15.82:0/3324092975 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5653ec485e40 0x5653ec870db0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f830c009900 tx=0x7f830c0326c0 comp rx=0 tx=0).stop 2024-05-20T16:13:36.302 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.300+0000 7f8328de8740 1 -- 172.21.15.82:0/3324092975 shutdown_connections 2024-05-20T16:13:36.302 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.300+0000 7f8328de8740 1 --2- 172.21.15.82:0/3324092975 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5653ec4a6590 0x5653ec8712f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:36.302 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.300+0000 7f8328de8740 1 --2- 172.21.15.82:0/3324092975 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5653ec49e3b0 0x5653ec8785a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:36.302 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.300+0000 7f8328de8740 1 --2- 172.21.15.82:0/3324092975 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5653ec485e40 0x5653ec870db0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:36.302 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.300+0000 7f8328de8740 1 -- 172.21.15.82:0/3324092975 >> 172.21.15.82:0/3324092975 conn(0x5653ec4bb3c0 msgr2=0x5653ec86b6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:13:36.302 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.300+0000 7f8328de8740 1 -- 172.21.15.82:0/3324092975 shutdown_connections 2024-05-20T16:13:36.302 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.300+0000 7f8328de8740 1 -- 172.21.15.82:0/3324092975 wait complete. 2024-05-20T16:13:36.302 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.300+0000 7f8328de8740 1 -- start start 2024-05-20T16:13:36.303 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.301+0000 7f8328de8740 1 Processor -- start 2024-05-20T16:13:36.306 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.301+0000 7f8328de8740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5653ec485e40 0x5653ec873bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:36.306 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.301+0000 7f8328de8740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5653ec49e3b0 0x5653ec8740f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:36.306 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.301+0000 7f8328de8740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5653ec4a6590 0x5653ec877030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:36.306 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.301+0000 7f8328de8740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x5653ec87ef60 con 0x5653ec49e3b0 2024-05-20T16:13:36.306 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.301+0000 7f8328de8740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x5653ec87e890 con 0x5653ec485e40 2024-05-20T16:13:36.306 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.301+0000 7f8328de8740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5653ec87f630 con 0x5653ec4a6590 2024-05-20T16:13:36.306 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.301+0000 7f8328de8740 10 client.0 _ll_register_callbacks cb 0x7f8314009e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T16:13:36.306 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.301+0000 7f831affd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5653ec49e3b0 0x5653ec8740f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:13:36.307 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.301+0000 7f831affd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5653ec49e3b0 0x5653ec8740f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:33624/0 (socket says 172.21.15.82:33624) 2024-05-20T16:13:36.307 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.301+0000 7f831affd640 1 -- 172.21.15.82:0/101021290 learned_addr learned my addr 172.21.15.82:0/101021290 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:13:36.307 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.301+0000 7f831b7fe640 1 --2- 172.21.15.82:0/101021290 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5653ec485e40 0x5653ec873bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:13:36.307 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.301+0000 7f831bfff640 1 --2- 172.21.15.82:0/101021290 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5653ec4a6590 0x5653ec877030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:13:36.307 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.301+0000 7f831affd640 1 -- 172.21.15.82:0/101021290 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5653ec4a6590 msgr2=0x5653ec877030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:13:36.307 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.301+0000 7f831affd640 1 --2- 172.21.15.82:0/101021290 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5653ec4a6590 0x5653ec877030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:36.307 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.301+0000 7f831affd640 1 -- 172.21.15.82:0/101021290 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5653ec485e40 msgr2=0x5653ec873bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:13:36.307 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.301+0000 7f831affd640 1 --2- 172.21.15.82:0/101021290 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5653ec485e40 0x5653ec873bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:36.307 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.301+0000 7f831affd640 1 -- 172.21.15.82:0/101021290 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5653ec8775f0 con 0x5653ec49e3b0 2024-05-20T16:13:36.307 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.301+0000 7f831bfff640 1 --2- 172.21.15.82:0/101021290 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5653ec4a6590 0x5653ec877030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T16:13:36.307 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.301+0000 7f831affd640 1 --2- 172.21.15.82:0/101021290 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5653ec49e3b0 0x5653ec8740f0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f8310009610 tx=0x7f831000de80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54855adc2c0c34e3 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:13:36.307 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.301+0000 7f82faffd640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-05-20T16:13:36.307 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.302+0000 7f82faffd640 1 -- 172.21.15.82:0/101021290 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8310020040 con 0x5653ec49e3b0 2024-05-20T16:13:36.307 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.302+0000 7f82faffd640 1 -- 172.21.15.82:0/101021290 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9092+0+0 (secure 0 0 0) 0x7f8310005290 con 0x5653ec49e3b0 2024-05-20T16:13:36.307 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.302+0000 7f8328de8740 1 -- 172.21.15.82:0/101021290 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f830c004040 con 0x5653ec49e3b0 2024-05-20T16:13:36.308 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.302+0000 7f82faffd640 1 -- 172.21.15.82:0/101021290 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f831006a260 con 0x5653ec49e3b0 2024-05-20T16:13:36.308 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.302+0000 7f8328de8740 10 client.15249 fetch_fsmap learned FSMap version 24 2024-05-20T16:13:36.308 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.302+0000 7f8328de8740 1 -- 172.21.15.82:0/101021290 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5653ec8787e0 con 0x5653ec49e3b0 2024-05-20T16:13:36.308 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.302+0000 7f82faffd640 1 -- 172.21.15.82:0/101021290 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f830c004040 con 0x5653ec49e3b0 2024-05-20T16:13:36.308 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.302+0000 7f82faffd640 1 -- 172.21.15.82:0/101021290 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f83100978e0 con 0x5653ec49e3b0 2024-05-20T16:13:36.308 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.302+0000 7f8328de8740 10 client.15249 fetch_fsmap finished waiting for FSMap version 24 2024-05-20T16:13:36.308 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.302+0000 7f8328de8740 10 client.15249 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-20T16:13:36.308 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.302+0000 7f8328de8740 20 client.15249 populate_metadata read hostname 'smithi082' 2024-05-20T16:13:36.308 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.302+0000 7f8328de8740 1 --2- 172.21.15.82:0/101021290 >> [v2:172.21.15.184:6838/2199955067,v1:172.21.15.184:6839/2199955067] conn(0x5653ec4c7270 0x5653ec8e8330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:36.308 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.302+0000 7f8328de8740 4 client.15249 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-20T16:13:36.308 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.302+0000 7f8328de8740 1 -- 172.21.15.82:0/101021290 --> [v2:172.21.15.184:6838/2199955067,v1:172.21.15.184:6839/2199955067] -- command(tid 0: {"prefix": "status"}) -- 0x5653ec750b80 con 0x5653ec4c7270 2024-05-20T16:13:36.308 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.302+0000 7f831b7fe640 1 --2- 172.21.15.82:0/101021290 >> [v2:172.21.15.184:6838/2199955067,v1:172.21.15.184:6839/2199955067] conn(0x5653ec4c7270 0x5653ec8e8330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:13:36.308 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.303+0000 7f82faffd640 10 client.15249 ms_handle_connect on v2:172.21.15.184:6838/2199955067 2024-05-20T16:13:36.308 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.303+0000 7f831b7fe640 1 --2- 172.21.15.82:0/101021290 >> [v2:172.21.15.184:6838/2199955067,v1:172.21.15.184:6839/2199955067] conn(0x5653ec4c7270 0x5653ec8e8330 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2b05c00a60ef91f7 server_cookie=756991a6760e0e4f in_seq=0 out_seq=0 2024-05-20T16:13:36.309 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.303+0000 7f82faffd640 1 -- 172.21.15.82:0/101021290 <== mds.1 v2:172.21.15.184:6838/2199955067 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5653ec750b80 con 0x5653ec4c7270 2024-05-20T16:13:36.309 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.303+0000 7f82faffd640 10 client.15249 handle_command_reply: tid=0 2024-05-20T16:13:36.309 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.303+0000 7f8328de8740 1 client.15249 shutdown 2024-05-20T16:13:36.309 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.303+0000 7f8328de8740 1 -- 172.21.15.82:0/101021290 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5653ec49e3b0 msgr2=0x5653ec8740f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:36.309 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.303+0000 7f8328de8740 1 --2- 172.21.15.82:0/101021290 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5653ec49e3b0 0x5653ec8740f0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f8310009610 tx=0x7f831000de80 comp rx=0 tx=0).stop 2024-05-20T16:13:36.309 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.303+0000 7f8328de8740 1 -- 172.21.15.82:0/101021290 shutdown_connections 2024-05-20T16:13:36.309 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.303+0000 7f8328de8740 1 --2- 172.21.15.82:0/101021290 >> [v2:172.21.15.184:6838/2199955067,v1:172.21.15.184:6839/2199955067] conn(0x5653ec4c7270 0x5653ec8e8330 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:36.309 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.303+0000 7f8328de8740 1 --2- 172.21.15.82:0/101021290 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5653ec49e3b0 0x5653ec8740f0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:36.309 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.303+0000 7f82faffd640 0 client.15249 ms_handle_reset on v2:172.21.15.184:6838/2199955067 2024-05-20T16:13:36.309 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.303+0000 7f8328de8740 1 --2- 172.21.15.82:0/101021290 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5653ec4a6590 0x5653ec877030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:36.309 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.303+0000 7f8328de8740 1 --2- 172.21.15.82:0/101021290 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5653ec485e40 0x5653ec873bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:36.309 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.303+0000 7f8328de8740 1 -- 172.21.15.82:0/101021290 >> 172.21.15.82:0/101021290 conn(0x5653ec4bb3c0 msgr2=0x5653ec861470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:13:36.309 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.303+0000 7f8328de8740 1 -- 172.21.15.82:0/101021290 shutdown_connections 2024-05-20T16:13:36.309 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.303+0000 7f8328de8740 1 -- 172.21.15.82:0/101021290 wait complete. 2024-05-20T16:13:36.309 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.303+0000 7f8328de8740 20 client.15249 trim_cache size 0 max 16384 2024-05-20T16:13:36.310 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-05-20T16:13:36.310 INFO:teuthology.orchestra.run.smithi082.stdout: "cluster_fsid": "59d1600a-16c2-11ef-bc9a-c7b262605968", 2024-05-20T16:13:36.310 INFO:teuthology.orchestra.run.smithi082.stdout: "whoami": 1, 2024-05-20T16:13:36.310 INFO:teuthology.orchestra.run.smithi082.stdout: "id": 24511, 2024-05-20T16:13:36.310 INFO:teuthology.orchestra.run.smithi082.stdout: "want_state": "up:active", 2024-05-20T16:13:36.310 INFO:teuthology.orchestra.run.smithi082.stdout: "state": "up:active", 2024-05-20T16:13:36.310 INFO:teuthology.orchestra.run.smithi082.stdout: "fs_name": "cephfs", 2024-05-20T16:13:36.310 INFO:teuthology.orchestra.run.smithi082.stdout: "rank_uptime": 84.927790986000005, 2024-05-20T16:13:36.310 INFO:teuthology.orchestra.run.smithi082.stdout: "mdsmap_epoch": 24, 2024-05-20T16:13:36.310 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_epoch": 93, 2024-05-20T16:13:36.311 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_epoch_barrier": 91, 2024-05-20T16:13:36.311 INFO:teuthology.orchestra.run.smithi082.stdout: "uptime": 121.526860961 2024-05-20T16:13:36.311 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-05-20T16:13:36.311 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.304+0000 7f8321e61640 1 -- 172.21.15.82:0/1547364361 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f83040a2c10 msgr2=0x7f83040a5120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:36.311 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.304+0000 7f8321e61640 1 --2- 172.21.15.82:0/1547364361 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f83040a2c10 0x7f83040a5120 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x5653ec868640 tx=0x7f8314004a40 comp rx=0 tx=0).stop 2024-05-20T16:13:36.312 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.304+0000 7f8321e61640 1 -- 172.21.15.82:0/1547364361 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f831c102bf0 msgr2=0x7f831c199810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:36.312 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.304+0000 7f8321e61640 1 --2- 172.21.15.82:0/1547364361 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f831c102bf0 0x7f831c199810 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f831000d770 tx=0x7f831000dc30 comp rx=0 tx=0).stop 2024-05-20T16:13:36.312 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.304+0000 7f8321e61640 1 -- 172.21.15.82:0/1547364361 shutdown_connections 2024-05-20T16:13:36.312 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.304+0000 7f8321e61640 1 --2- 172.21.15.82:0/1547364361 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f83040a2c10 0x7f83040a5120 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:36.312 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.304+0000 7f8321e61640 1 --2- 172.21.15.82:0/1547364361 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f831c102bf0 0x7f831c199810 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:36.312 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.304+0000 7f8321e61640 1 --2- 172.21.15.82:0/1547364361 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f831c1078f0 0x7f831c1a07a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:36.312 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.304+0000 7f8321e61640 1 --2- 172.21.15.82:0/1547364361 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f831c1019b0 0x7f831c1992d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:36.312 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.304+0000 7f8321e61640 1 -- 172.21.15.82:0/1547364361 >> 172.21.15.82:0/1547364361 conn(0x7f831c0fcba0 msgr2=0x7f831c1038d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:13:36.312 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.304+0000 7f8321e61640 1 -- 172.21.15.82:0/1547364361 shutdown_connections 2024-05-20T16:13:36.312 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.304+0000 7f8321e61640 1 -- 172.21.15.82:0/1547364361 wait complete. 2024-05-20T16:13:36.316 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-05-20T16:13:36.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.428+0000 7f09dbaf0640 1 -- 172.21.15.82:0/4149905577 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f09d4102be0 msgr2=0x7f09d4102ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:36.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.428+0000 7f09dbaf0640 1 --2- 172.21.15.82:0/4149905577 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f09d4102be0 0x7f09d4102ff0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f09c8009ca0 tx=0x7f09c802a8c0 comp rx=0 tx=0).stop 2024-05-20T16:13:36.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.428+0000 7f09dbaf0640 1 -- 172.21.15.82:0/4149905577 shutdown_connections 2024-05-20T16:13:36.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.428+0000 7f09dbaf0640 1 --2- 172.21.15.82:0/4149905577 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f09d4102be0 0x7f09d4102ff0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:36.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.428+0000 7f09dbaf0640 1 --2- 172.21.15.82:0/4149905577 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f09d41078e0 0x7f09d4109d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:36.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.428+0000 7f09dbaf0640 1 --2- 172.21.15.82:0/4149905577 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f09d41019a0 0x7f09d41026a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:36.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.428+0000 7f09dbaf0640 1 -- 172.21.15.82:0/4149905577 >> 172.21.15.82:0/4149905577 conn(0x7f09d40fcb70 msgr2=0x7f09d40ff040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:13:36.429 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.428+0000 7f09dbaf0640 1 -- 172.21.15.82:0/4149905577 shutdown_connections 2024-05-20T16:13:36.429 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.428+0000 7f09dbaf0640 1 -- 172.21.15.82:0/4149905577 wait complete. 2024-05-20T16:13:36.431 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.429+0000 7f09dbaf0640 1 Processor -- start 2024-05-20T16:13:36.431 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.429+0000 7f09dbaf0640 1 -- start start 2024-05-20T16:13:36.431 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.429+0000 7f09dbaf0640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f09d41019a0 0x7f09d4198190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:36.431 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.429+0000 7f09dbaf0640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f09d4102be0 0x7f09d41986d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:36.431 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.429+0000 7f09dbaf0640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f09d41078e0 0x7f09d419f660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:36.431 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.429+0000 7f09dbaf0640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f09d410d610 con 0x7f09d41019a0 2024-05-20T16:13:36.431 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.429+0000 7f09dbaf0640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f09d410d490 con 0x7f09d41078e0 2024-05-20T16:13:36.431 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.429+0000 7f09dbaf0640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f09d410d790 con 0x7f09d4102be0 2024-05-20T16:13:36.431 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.429+0000 7f09d988c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f09d41019a0 0x7f09d4198190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:13:36.431 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.429+0000 7f09d988c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f09d41019a0 0x7f09d4198190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:33638/0 (socket says 172.21.15.82:33638) 2024-05-20T16:13:36.431 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.429+0000 7f09d988c640 1 -- 172.21.15.82:0/689845425 learned_addr learned my addr 172.21.15.82:0/689845425 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:13:36.432 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.429+0000 7f09d908b640 1 --2- 172.21.15.82:0/689845425 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f09d4102be0 0x7f09d41986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:13:36.432 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.429+0000 7f09da08d640 1 --2- 172.21.15.82:0/689845425 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f09d41078e0 0x7f09d419f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:13:36.432 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.429+0000 7f09d988c640 1 -- 172.21.15.82:0/689845425 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f09d4102be0 msgr2=0x7f09d41986d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:36.432 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.429+0000 7f09d988c640 1 --2- 172.21.15.82:0/689845425 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f09d4102be0 0x7f09d41986d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:36.432 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.429+0000 7f09d988c640 1 -- 172.21.15.82:0/689845425 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f09d41078e0 msgr2=0x7f09d419f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:36.432 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.429+0000 7f09d988c640 1 --2- 172.21.15.82:0/689845425 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f09d41078e0 0x7f09d419f660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:36.432 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.429+0000 7f09d988c640 1 -- 172.21.15.82:0/689845425 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09d419fcd0 con 0x7f09d41019a0 2024-05-20T16:13:36.432 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.429+0000 7f09d908b640 1 --2- 172.21.15.82:0/689845425 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f09d4102be0 0x7f09d41986d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T16:13:36.432 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.429+0000 7f09d988c640 1 --2- 172.21.15.82:0/689845425 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f09d41019a0 0x7f09d4198190 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f09c4005d00 tx=0x7f09c400dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:13:36.432 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.430+0000 7f09d2ffd640 1 -- 172.21.15.82:0/689845425 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09c4017070 con 0x7f09d41019a0 2024-05-20T16:13:36.432 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.430+0000 7f09d2ffd640 1 -- 172.21.15.82:0/689845425 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f09c4009c30 con 0x7f09d41019a0 2024-05-20T16:13:36.432 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.430+0000 7f09dbaf0640 1 -- 172.21.15.82:0/689845425 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09d419ffc0 con 0x7f09d41019a0 2024-05-20T16:13:36.432 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.430+0000 7f09d2ffd640 1 -- 172.21.15.82:0/689845425 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09c400f620 con 0x7f09d41019a0 2024-05-20T16:13:36.432 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.430+0000 7f09dbaf0640 1 -- 172.21.15.82:0/689845425 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09d41a03d0 con 0x7f09d41019a0 2024-05-20T16:13:36.439 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.432+0000 7f09d2ffd640 1 -- 172.21.15.82:0/689845425 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f09c401a070 con 0x7f09d41019a0 2024-05-20T16:13:36.439 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.432+0000 7f09e2997740 1 Processor -- start 2024-05-20T16:13:36.439 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.433+0000 7f09e2997740 1 -- start start 2024-05-20T16:13:36.439 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.433+0000 7f09e2997740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56250956b270 0x56250990b470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:36.439 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.433+0000 7f09e2997740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5625095423b0 0x56250990b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:36.439 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.433+0000 7f09e2997740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56250954a590 0x5625099128f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:36.439 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.433+0000 7f09e2997740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x56250986f4d0 con 0x56250956b270 2024-05-20T16:13:36.439 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.433+0000 7f09e2997740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x5625098173e0 con 0x5625095423b0 2024-05-20T16:13:36.439 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.433+0000 7f09e2997740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x56250986b490 con 0x56250954a590 2024-05-20T16:13:36.439 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.433+0000 7f09da08d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56250956b270 0x56250990b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:13:36.440 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.433+0000 7f09da08d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56250956b270 0x56250990b470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:33652/0 (socket says 172.21.15.82:33652) 2024-05-20T16:13:36.440 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.433+0000 7f09da08d640 1 -- 172.21.15.82:0/369934735 learned_addr learned my addr 172.21.15.82:0/369934735 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:13:36.440 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.433+0000 7f09d988c640 1 --2- 172.21.15.82:0/369934735 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5625095423b0 0x56250990b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:13:36.440 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.433+0000 7f09d908b640 1 --2- 172.21.15.82:0/369934735 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56250954a590 0x5625099128f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:13:36.440 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.433+0000 7f09d2ffd640 1 --2- 172.21.15.82:0/689845425 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f09bc0a2b90 0x7f09bc0a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:36.440 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.433+0000 7f09da08d640 1 -- 172.21.15.82:0/369934735 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56250954a590 msgr2=0x5625099128f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:13:36.440 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.433+0000 7f09da08d640 1 --2- 172.21.15.82:0/369934735 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56250954a590 0x5625099128f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:36.440 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.433+0000 7f09da08d640 1 -- 172.21.15.82:0/369934735 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5625095423b0 msgr2=0x56250990b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:13:36.440 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.433+0000 7f09da08d640 1 --2- 172.21.15.82:0/369934735 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5625095423b0 0x56250990b9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:36.440 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.433+0000 7f09da08d640 1 -- 172.21.15.82:0/369934735 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5625095832d0 con 0x56250956b270 2024-05-20T16:13:36.440 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.433+0000 7f09da08d640 1 --2- 172.21.15.82:0/689845425 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f09bc0a2b90 0x7f09bc0a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:13:36.440 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.433+0000 7f09d2ffd640 1 -- 172.21.15.82:0/689845425 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9092+0+0 (secure 0 0 0) 0x7f09c40ceeb0 con 0x7f09d41019a0 2024-05-20T16:13:36.440 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.434+0000 7f09da08d640 1 --2- 172.21.15.82:0/369934735 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56250956b270 0x56250990b470 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f09cc00bdb0 tx=0x7f09cc00f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cba94716fe52b30b server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:13:36.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.434+0000 7f09bbfff640 1 -- 172.21.15.82:0/369934735 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09cc01c580 con 0x56250956b270 2024-05-20T16:13:36.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.434+0000 7f09bbfff640 1 -- 172.21.15.82:0/369934735 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f09cc01cc00 con 0x56250956b270 2024-05-20T16:13:36.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.434+0000 7f09bbfff640 1 -- 172.21.15.82:0/369934735 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09cc01bb40 con 0x56250956b270 2024-05-20T16:13:36.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.434+0000 7f09da08d640 1 --2- 172.21.15.82:0/689845425 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f09bc0a2b90 0x7f09bc0a50a0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f09cc020000 tx=0x7f09cc01d000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:13:36.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.435+0000 7f09e2997740 1 -- 172.21.15.82:0/369934735 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56250956b270 msgr2=0x56250990b470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:36.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.435+0000 7f09e2997740 1 --2- 172.21.15.82:0/369934735 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56250956b270 0x56250990b470 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f09cc00bdb0 tx=0x7f09cc00f730 comp rx=0 tx=0).stop 2024-05-20T16:13:36.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.435+0000 7f09e2997740 1 -- 172.21.15.82:0/369934735 shutdown_connections 2024-05-20T16:13:36.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.435+0000 7f09e2997740 1 --2- 172.21.15.82:0/369934735 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56250954a590 0x5625099128f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:36.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.435+0000 7f09e2997740 1 --2- 172.21.15.82:0/369934735 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5625095423b0 0x56250990b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:36.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.435+0000 7f09e2997740 1 --2- 172.21.15.82:0/369934735 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56250956b270 0x56250990b470 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:36.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.435+0000 7f09e2997740 1 -- 172.21.15.82:0/369934735 >> 172.21.15.82:0/369934735 conn(0x56250955f3c0 msgr2=0x562509906e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:13:36.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.435+0000 7f09e2997740 1 -- 172.21.15.82:0/369934735 shutdown_connections 2024-05-20T16:13:36.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.435+0000 7f09e2997740 1 -- 172.21.15.82:0/369934735 wait complete. 2024-05-20T16:13:36.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.435+0000 7f09e2997740 1 -- start start 2024-05-20T16:13:36.442 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.435+0000 7f09e2997740 1 Processor -- start 2024-05-20T16:13:36.442 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.436+0000 7f09e2997740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5625095423b0 0x562509915260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:36.442 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.436+0000 7f09e2997740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56250954a590 0x5625099157a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:36.442 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.436+0000 7f09e2997740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56250956b270 0x56250991c700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:36.442 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.436+0000 7f09e2997740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x562509913340 con 0x5625095423b0 2024-05-20T16:13:36.442 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.436+0000 7f09e2997740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x56250982a8e0 con 0x56250954a590 2024-05-20T16:13:36.442 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.436+0000 7f09e2997740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5625099134c0 con 0x56250956b270 2024-05-20T16:13:36.442 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.436+0000 7f09e2997740 10 client.0 _ll_register_callbacks cb 0x5625097f79c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T16:13:36.442 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.436+0000 7f09d988c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5625095423b0 0x562509915260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:13:36.442 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.436+0000 7f09d988c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5625095423b0 0x562509915260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:33654/0 (socket says 172.21.15.82:33654) 2024-05-20T16:13:36.442 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.436+0000 7f09d988c640 1 -- 172.21.15.82:0/1783187395 learned_addr learned my addr 172.21.15.82:0/1783187395 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:13:36.442 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.436+0000 7f09da08d640 1 --2- 172.21.15.82:0/1783187395 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56250956b270 0x56250991c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:13:36.442 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.436+0000 7f09d908b640 1 --2- 172.21.15.82:0/1783187395 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56250954a590 0x5625099157a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:13:36.442 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.436+0000 7f09d988c640 1 -- 172.21.15.82:0/1783187395 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56250956b270 msgr2=0x56250991c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:13:36.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.436+0000 7f09d988c640 1 --2- 172.21.15.82:0/1783187395 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56250956b270 0x56250991c700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:36.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.436+0000 7f09d988c640 1 -- 172.21.15.82:0/1783187395 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56250954a590 msgr2=0x5625099157a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:13:36.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.436+0000 7f09d988c640 1 --2- 172.21.15.82:0/1783187395 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56250954a590 0x5625099157a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:36.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.436+0000 7f09d988c640 1 -- 172.21.15.82:0/1783187395 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56250964d6b0 con 0x5625095423b0 2024-05-20T16:13:36.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.436+0000 7f09d908b640 1 --2- 172.21.15.82:0/1783187395 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56250954a590 0x5625099157a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T16:13:36.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.436+0000 7f09da08d640 1 --2- 172.21.15.82:0/1783187395 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56250956b270 0x56250991c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T16:13:36.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.436+0000 7f09d988c640 1 --2- 172.21.15.82:0/1783187395 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5625095423b0 0x562509915260 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f09c40cc650 tx=0x7f09c4095d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf2efc1cce9a88ae server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:13:36.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.436+0000 7f09b97fa640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-05-20T16:13:36.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.437+0000 7f09b97fa640 1 -- 172.21.15.82:0/1783187395 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09c409ad10 con 0x5625095423b0 2024-05-20T16:13:36.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.437+0000 7f09b97fa640 1 -- 172.21.15.82:0/1783187395 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9092+0+0 (secure 0 0 0) 0x7f09c409a800 con 0x5625095423b0 2024-05-20T16:13:36.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.437+0000 7f09e2997740 1 -- 172.21.15.82:0/1783187395 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f09cc01cc00 con 0x5625095423b0 2024-05-20T16:13:36.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.437+0000 7f09b97fa640 1 -- 172.21.15.82:0/1783187395 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f09c409bab0 con 0x5625095423b0 2024-05-20T16:13:36.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.437+0000 7f09e2997740 10 client.15261 fetch_fsmap learned FSMap version 24 2024-05-20T16:13:36.444 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.437+0000 7f09e2997740 1 -- 172.21.15.82:0/1783187395 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x56250991f6a0 con 0x5625095423b0 2024-05-20T16:13:36.444 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.437+0000 7f09b97fa640 1 -- 172.21.15.82:0/1783187395 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f09cc01cc00 con 0x5625095423b0 2024-05-20T16:13:36.444 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.437+0000 7f09b97fa640 1 -- 172.21.15.82:0/1783187395 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f09c4097100 con 0x5625095423b0 2024-05-20T16:13:36.444 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.437+0000 7f09e2997740 10 client.15261 fetch_fsmap finished waiting for FSMap version 24 2024-05-20T16:13:36.444 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.437+0000 7f09e2997740 10 client.15261 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-20T16:13:36.444 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.437+0000 7f09e2997740 20 client.15261 populate_metadata read hostname 'smithi082' 2024-05-20T16:13:36.444 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.437+0000 7f09e2997740 1 --2- 172.21.15.82:0/1783187395 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x562509529e40 0x562509923130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:36.444 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.437+0000 7f09e2997740 4 client.15261 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T16:13:36.444 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.437+0000 7f09e2997740 1 -- 172.21.15.82:0/1783187395 --> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56250987cc00 con 0x562509529e40 2024-05-20T16:13:36.444 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.437+0000 7f09d908b640 1 --2- 172.21.15.82:0/1783187395 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x562509529e40 0x562509923130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:13:36.444 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.438+0000 7f09b97fa640 10 client.15261 ms_handle_connect on v2:172.21.15.160:6838/2765554482 2024-05-20T16:13:36.444 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.438+0000 7f09d908b640 1 --2- 172.21.15.82:0/1783187395 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x562509529e40 0x562509923130 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9a59990a77390667 server_cookie=eb887301684ec6ab in_seq=0 out_seq=0 2024-05-20T16:13:36.444 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.439+0000 7f09b97fa640 1 -- 172.21.15.82:0/1783187395 <== mds.0 v2:172.21.15.160:6838/2765554482 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56250987cc00 con 0x562509529e40 2024-05-20T16:13:36.444 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.439+0000 7f09b97fa640 10 client.15261 handle_command_reply: tid=0 2024-05-20T16:13:36.444 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.439+0000 7f09e2997740 1 client.15261 shutdown 2024-05-20T16:13:36.445 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.439+0000 7f09e2997740 1 -- 172.21.15.82:0/1783187395 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5625095423b0 msgr2=0x562509915260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:36.445 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.439+0000 7f09e2997740 1 --2- 172.21.15.82:0/1783187395 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5625095423b0 0x562509915260 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f09c40cc650 tx=0x7f09c4095d90 comp rx=0 tx=0).stop 2024-05-20T16:13:36.445 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.439+0000 7f09e2997740 1 -- 172.21.15.82:0/1783187395 shutdown_connections 2024-05-20T16:13:36.445 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.439+0000 7f09e2997740 1 --2- 172.21.15.82:0/1783187395 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x562509529e40 0x562509923130 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:36.445 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.439+0000 7f09e2997740 1 --2- 172.21.15.82:0/1783187395 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56250956b270 0x56250991c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:36.445 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.439+0000 7f09e2997740 1 --2- 172.21.15.82:0/1783187395 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56250954a590 0x5625099157a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:36.445 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.439+0000 7f09e2997740 1 --2- 172.21.15.82:0/1783187395 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5625095423b0 0x562509915260 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:36.445 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.439+0000 7f09b97fa640 0 client.15261 ms_handle_reset on v2:172.21.15.160:6838/2765554482 2024-05-20T16:13:36.445 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.439+0000 7f09e2997740 1 -- 172.21.15.82:0/1783187395 >> 172.21.15.82:0/1783187395 conn(0x56250955f3c0 msgr2=0x562509905ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:13:36.445 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.439+0000 7f09e2997740 1 -- 172.21.15.82:0/1783187395 shutdown_connections 2024-05-20T16:13:36.445 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.439+0000 7f09e2997740 1 -- 172.21.15.82:0/1783187395 wait complete. 2024-05-20T16:13:36.445 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.439+0000 7f09e2997740 20 client.15261 trim_cache size 0 max 16384 2024-05-20T16:13:36.462 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.457+0000 7f09e2997740 1 Processor -- start 2024-05-20T16:13:36.462 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.457+0000 7f09e2997740 1 -- start start 2024-05-20T16:13:36.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.457+0000 7f09e2997740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5625095423b0 0x562509913220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:36.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.458+0000 7f09e2997740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56250954a590 0x56250991cc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:36.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.458+0000 7f09e2997740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56250956b270 0x5625099067f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:36.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.458+0000 7f09e2997740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x562509921470 con 0x56250956b270 2024-05-20T16:13:36.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.458+0000 7f09e2997740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x562509921b40 con 0x5625095423b0 2024-05-20T16:13:36.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.458+0000 7f09e2997740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x562509922210 con 0x56250954a590 2024-05-20T16:13:36.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.458+0000 7f09da08d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56250956b270 0x5625099067f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:13:36.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.458+0000 7f09da08d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56250956b270 0x5625099067f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:33660/0 (socket says 172.21.15.82:33660) 2024-05-20T16:13:36.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.458+0000 7f09da08d640 1 -- 172.21.15.82:0/2202779534 learned_addr learned my addr 172.21.15.82:0/2202779534 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:13:36.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.458+0000 7f09d908b640 1 --2- 172.21.15.82:0/2202779534 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56250954a590 0x56250991cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:13:36.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.458+0000 7f09d988c640 1 --2- 172.21.15.82:0/2202779534 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5625095423b0 0x562509913220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:13:36.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.458+0000 7f09da08d640 1 -- 172.21.15.82:0/2202779534 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56250954a590 msgr2=0x56250991cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:13:36.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.458+0000 7f09da08d640 1 --2- 172.21.15.82:0/2202779534 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56250954a590 0x56250991cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:36.464 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.458+0000 7f09da08d640 1 -- 172.21.15.82:0/2202779534 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5625095423b0 msgr2=0x562509913220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:13:36.464 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.458+0000 7f09da08d640 1 --2- 172.21.15.82:0/2202779534 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5625095423b0 0x562509913220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:36.464 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.458+0000 7f09da08d640 1 -- 172.21.15.82:0/2202779534 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56250990b020 con 0x56250956b270 2024-05-20T16:13:36.464 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.458+0000 7f09d908b640 1 --2- 172.21.15.82:0/2202779534 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56250954a590 0x56250991cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T16:13:36.464 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.458+0000 7f09d988c640 1 --2- 172.21.15.82:0/2202779534 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5625095423b0 0x562509913220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T16:13:36.464 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.458+0000 7f09da08d640 1 --2- 172.21.15.82:0/2202779534 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56250956b270 0x5625099067f0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f09cc00fc30 tx=0x7f09cc00bcb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea0d5570fa9747e1 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:13:36.464 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.458+0000 7f09bb7fe640 1 -- 172.21.15.82:0/2202779534 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09cc035070 con 0x56250956b270 2024-05-20T16:13:36.464 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.458+0000 7f09bb7fe640 1 -- 172.21.15.82:0/2202779534 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f09cc01be20 con 0x56250956b270 2024-05-20T16:13:36.464 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.458+0000 7f09bb7fe640 1 -- 172.21.15.82:0/2202779534 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09cc01cbe0 con 0x56250956b270 2024-05-20T16:13:36.464 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.459+0000 7f09e2997740 1 -- 172.21.15.82:0/2202779534 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56250956b270 msgr2=0x5625099067f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:36.464 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.459+0000 7f09e2997740 1 --2- 172.21.15.82:0/2202779534 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56250956b270 0x5625099067f0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f09cc00fc30 tx=0x7f09cc00bcb0 comp rx=0 tx=0).stop 2024-05-20T16:13:36.464 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.459+0000 7f09e2997740 1 -- 172.21.15.82:0/2202779534 shutdown_connections 2024-05-20T16:13:36.464 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.459+0000 7f09e2997740 1 --2- 172.21.15.82:0/2202779534 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56250956b270 0x5625099067f0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:36.464 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.459+0000 7f09e2997740 1 --2- 172.21.15.82:0/2202779534 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56250954a590 0x56250991cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:36.465 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.459+0000 7f09e2997740 1 --2- 172.21.15.82:0/2202779534 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5625095423b0 0x562509913220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:36.465 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.459+0000 7f09e2997740 1 -- 172.21.15.82:0/2202779534 >> 172.21.15.82:0/2202779534 conn(0x56250955f3c0 msgr2=0x562509915200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:13:36.465 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.459+0000 7f09e2997740 1 -- 172.21.15.82:0/2202779534 shutdown_connections 2024-05-20T16:13:36.465 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.459+0000 7f09e2997740 1 -- 172.21.15.82:0/2202779534 wait complete. 2024-05-20T16:13:36.465 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.459+0000 7f09e2997740 1 -- start start 2024-05-20T16:13:36.465 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.459+0000 7f09e2997740 1 Processor -- start 2024-05-20T16:13:36.465 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.460+0000 7f09e2997740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5625095423b0 0x5625099179b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:36.465 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.460+0000 7f09e2997740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56250954a590 0x562509917ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:36.465 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.460+0000 7f09e2997740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56250956b270 0x56250998e520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:36.465 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.460+0000 7f09e2997740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x562509922fb0 con 0x5625095423b0 2024-05-20T16:13:36.465 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.460+0000 7f09e2997740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x5625099228e0 con 0x56250956b270 2024-05-20T16:13:36.465 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.460+0000 7f09e2997740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x562509923680 con 0x56250954a590 2024-05-20T16:13:36.465 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.460+0000 7f09e2997740 10 client.0 _ll_register_callbacks cb 0x7f09c40d2680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T16:13:36.465 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.460+0000 7f09d988c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5625095423b0 0x5625099179b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:13:36.466 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.460+0000 7f09d988c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5625095423b0 0x5625099179b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:33672/0 (socket says 172.21.15.82:33672) 2024-05-20T16:13:36.466 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.460+0000 7f09d988c640 1 -- 172.21.15.82:0/3499481610 learned_addr learned my addr 172.21.15.82:0/3499481610 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:13:36.466 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.460+0000 7f09d908b640 1 --2- 172.21.15.82:0/3499481610 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56250954a590 0x562509917ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:13:36.466 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.460+0000 7f09da08d640 1 --2- 172.21.15.82:0/3499481610 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56250956b270 0x56250998e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:13:36.466 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.460+0000 7f09d988c640 1 -- 172.21.15.82:0/3499481610 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56250954a590 msgr2=0x562509917ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:13:36.466 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.460+0000 7f09d988c640 1 --2- 172.21.15.82:0/3499481610 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56250954a590 0x562509917ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:36.466 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.460+0000 7f09d988c640 1 -- 172.21.15.82:0/3499481610 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56250956b270 msgr2=0x56250998e520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:13:36.466 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.460+0000 7f09d988c640 1 --2- 172.21.15.82:0/3499481610 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56250956b270 0x56250998e520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:36.466 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.460+0000 7f09d988c640 1 -- 172.21.15.82:0/3499481610 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56250998eae0 con 0x5625095423b0 2024-05-20T16:13:36.466 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.460+0000 7f09d988c640 1 --2- 172.21.15.82:0/3499481610 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5625095423b0 0x5625099179b0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f09c4099660 tx=0x7f09c409aa30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e0b97e77e16b612 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:13:36.466 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.460+0000 7f09b97fa640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-05-20T16:13:36.466 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.461+0000 7f09b97fa640 1 -- 172.21.15.82:0/3499481610 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09c409b180 con 0x5625095423b0 2024-05-20T16:13:36.467 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.461+0000 7f09b97fa640 1 -- 172.21.15.82:0/3499481610 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9092+0+0 (secure 0 0 0) 0x7f09c40d20d0 con 0x5625095423b0 2024-05-20T16:13:36.467 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.461+0000 7f09e2997740 1 -- 172.21.15.82:0/3499481610 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f09cc01be20 con 0x5625095423b0 2024-05-20T16:13:36.467 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.461+0000 7f09b97fa640 1 -- 172.21.15.82:0/3499481610 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f09c40d22d0 con 0x5625095423b0 2024-05-20T16:13:36.467 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.461+0000 7f09e2997740 10 client.15267 fetch_fsmap learned FSMap version 24 2024-05-20T16:13:36.467 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.461+0000 7f09e2997740 1 -- 172.21.15.82:0/3499481610 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x56250998fc40 con 0x5625095423b0 2024-05-20T16:13:36.467 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.461+0000 7f09b97fa640 1 -- 172.21.15.82:0/3499481610 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f09cc01be20 con 0x5625095423b0 2024-05-20T16:13:36.467 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.461+0000 7f09b97fa640 1 -- 172.21.15.82:0/3499481610 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f09c40b41b0 con 0x5625095423b0 2024-05-20T16:13:36.467 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.461+0000 7f09e2997740 10 client.15267 fetch_fsmap finished waiting for FSMap version 24 2024-05-20T16:13:36.467 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.461+0000 7f09e2997740 10 client.15267 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-20T16:13:36.467 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.461+0000 7f09e2997740 20 client.15267 populate_metadata read hostname 'smithi082' 2024-05-20T16:13:36.467 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.461+0000 7f09e2997740 1 --2- 172.21.15.82:0/3499481610 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x5625094ff280 0x5625099912c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:36.467 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.461+0000 7f09e2997740 4 client.15267 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-20T16:13:36.467 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.461+0000 7f09e2997740 1 -- 172.21.15.82:0/3499481610 --> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] -- command(tid 0: {"prefix": "status"}) -- 0x562509876f50 con 0x5625094ff280 2024-05-20T16:13:36.467 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.461+0000 7f09d908b640 1 --2- 172.21.15.82:0/3499481610 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x5625094ff280 0x5625099912c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:13:36.468 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.462+0000 7f09b97fa640 10 client.15267 ms_handle_connect on v2:172.21.15.160:6838/2765554482 2024-05-20T16:13:36.468 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.462+0000 7f09d908b640 1 --2- 172.21.15.82:0/3499481610 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x5625094ff280 0x5625099912c0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=22755c6f043a2bbb server_cookie=9feb8653bfd43fdd in_seq=0 out_seq=0 2024-05-20T16:13:36.468 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.462+0000 7f09b97fa640 1 -- 172.21.15.82:0/3499481610 <== mds.0 v2:172.21.15.160:6838/2765554482 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x562509876f50 con 0x5625094ff280 2024-05-20T16:13:36.468 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.462+0000 7f09b97fa640 10 client.15267 handle_command_reply: tid=0 2024-05-20T16:13:36.468 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.462+0000 7f09e2997740 1 client.15267 shutdown 2024-05-20T16:13:36.468 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.462+0000 7f09e2997740 1 -- 172.21.15.82:0/3499481610 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5625095423b0 msgr2=0x5625099179b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:36.468 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.462+0000 7f09e2997740 1 --2- 172.21.15.82:0/3499481610 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5625095423b0 0x5625099179b0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f09c4099660 tx=0x7f09c409aa30 comp rx=0 tx=0).stop 2024-05-20T16:13:36.468 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.462+0000 7f09e2997740 1 -- 172.21.15.82:0/3499481610 shutdown_connections 2024-05-20T16:13:36.468 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.462+0000 7f09e2997740 1 --2- 172.21.15.82:0/3499481610 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x5625094ff280 0x5625099912c0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:36.468 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.462+0000 7f09e2997740 1 --2- 172.21.15.82:0/3499481610 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56250956b270 0x56250998e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:36.468 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.462+0000 7f09e2997740 1 --2- 172.21.15.82:0/3499481610 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56250954a590 0x562509917ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:36.468 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.462+0000 7f09b97fa640 0 client.15267 ms_handle_reset on v2:172.21.15.160:6838/2765554482 2024-05-20T16:13:36.468 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.462+0000 7f09e2997740 1 --2- 172.21.15.82:0/3499481610 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5625095423b0 0x5625099179b0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:36.468 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.462+0000 7f09e2997740 1 -- 172.21.15.82:0/3499481610 >> 172.21.15.82:0/3499481610 conn(0x56250955f3c0 msgr2=0x562509915200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:13:36.469 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.462+0000 7f09e2997740 1 -- 172.21.15.82:0/3499481610 shutdown_connections 2024-05-20T16:13:36.469 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.462+0000 7f09e2997740 1 -- 172.21.15.82:0/3499481610 wait complete. 2024-05-20T16:13:36.469 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.463+0000 7f09e2997740 20 client.15267 trim_cache size 0 max 16384 2024-05-20T16:13:36.469 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-05-20T16:13:36.469 INFO:teuthology.orchestra.run.smithi082.stdout: "cluster_fsid": "59d1600a-16c2-11ef-bc9a-c7b262605968", 2024-05-20T16:13:36.469 INFO:teuthology.orchestra.run.smithi082.stdout: "whoami": 0, 2024-05-20T16:13:36.469 INFO:teuthology.orchestra.run.smithi082.stdout: "id": 24548, 2024-05-20T16:13:36.469 INFO:teuthology.orchestra.run.smithi082.stdout: "want_state": "up:active", 2024-05-20T16:13:36.469 INFO:teuthology.orchestra.run.smithi082.stdout: "state": "up:active", 2024-05-20T16:13:36.469 INFO:teuthology.orchestra.run.smithi082.stdout: "fs_name": "cephfs", 2024-05-20T16:13:36.469 INFO:teuthology.orchestra.run.smithi082.stdout: "rank_uptime": 109.081410455, 2024-05-20T16:13:36.470 INFO:teuthology.orchestra.run.smithi082.stdout: "mdsmap_epoch": 24, 2024-05-20T16:13:36.470 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_epoch": 93, 2024-05-20T16:13:36.470 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_epoch_barrier": 91, 2024-05-20T16:13:36.470 INFO:teuthology.orchestra.run.smithi082.stdout: "uptime": 113.21382859800001 2024-05-20T16:13:36.470 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-05-20T16:13:36.471 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.463+0000 7f09dbaf0640 1 -- 172.21.15.82:0/689845425 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f09bc0a2b90 msgr2=0x7f09bc0a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:36.471 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.463+0000 7f09dbaf0640 1 --2- 172.21.15.82:0/689845425 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f09bc0a2b90 0x7f09bc0a50a0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f09cc020000 tx=0x7f09cc01d000 comp rx=0 tx=0).stop 2024-05-20T16:13:36.471 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.463+0000 7f09dbaf0640 1 -- 172.21.15.82:0/689845425 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f09d41019a0 msgr2=0x7f09d4198190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:36.471 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.463+0000 7f09dbaf0640 1 --2- 172.21.15.82:0/689845425 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f09d41019a0 0x7f09d4198190 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f09c4005d00 tx=0x7f09c400dde0 comp rx=0 tx=0).stop 2024-05-20T16:13:36.471 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.463+0000 7f09dbaf0640 1 -- 172.21.15.82:0/689845425 shutdown_connections 2024-05-20T16:13:36.471 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.463+0000 7f09dbaf0640 1 --2- 172.21.15.82:0/689845425 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f09bc0a2b90 0x7f09bc0a50a0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:36.471 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.463+0000 7f09dbaf0640 1 --2- 172.21.15.82:0/689845425 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f09d41078e0 0x7f09d419f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:36.471 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.463+0000 7f09dbaf0640 1 --2- 172.21.15.82:0/689845425 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f09d4102be0 0x7f09d41986d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:36.471 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.463+0000 7f09dbaf0640 1 --2- 172.21.15.82:0/689845425 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f09d41019a0 0x7f09d4198190 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:36.471 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.463+0000 7f09dbaf0640 1 -- 172.21.15.82:0/689845425 >> 172.21.15.82:0/689845425 conn(0x7f09d40fcb70 msgr2=0x7f09d41038c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:13:36.471 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.463+0000 7f09dbaf0640 1 -- 172.21.15.82:0/689845425 shutdown_connections 2024-05-20T16:13:36.471 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:36.463+0000 7f09dbaf0640 1 -- 172.21.15.82:0/689845425 wait complete. 2024-05-20T16:13:36.474 INFO:tasks.cephadm:Setup complete, yielding 2024-05-20T16:13:36.475 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-05-20T16:13:36.485 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi082.front.sepia.ceph.com 2024-05-20T16:13:36.485 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 59d1600a-16c2-11ef-bc9a-c7b262605968 -- bash -c 'ceph orch status' 2024-05-20T16:13:36.723 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T16:13:37.062 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:13:36 smithi160 ceph-mon[36347]: from='client.? 172.21.15.82:0/2768485576' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T16:13:37.101 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:13:36 smithi082 ceph-mon[32238]: from='client.? 172.21.15.82:0/2768485576' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T16:13:37.135 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:13:36 smithi184 ceph-mon[36209]: from='client.? 172.21.15.82:0/2768485576' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T16:13:38.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:13:37 smithi082 ceph-mon[32238]: pgmap v349: 129 pgs: 129 active+clean; 641 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-20T16:13:38.135 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:13:37 smithi184 ceph-mon[36209]: pgmap v349: 129 pgs: 129 active+clean; 641 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-20T16:13:38.312 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:13:37 smithi160 ceph-mon[36347]: pgmap v349: 129 pgs: 129 active+clean; 641 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-20T16:13:38.576 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:38.574+0000 7f3f2aec9700 1 -- 172.21.15.82:0/172942575 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3f24101860 msgr2=0x7f3f24101c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:38.576 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:38.574+0000 7f3f2aec9700 1 --2- 172.21.15.82:0/172942575 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3f24101860 0x7f3f24101c70 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f3f1800aaa0 tx=0x7f3f1800adb0 comp rx=0 tx=0).stop 2024-05-20T16:13:38.576 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:38.574+0000 7f3f2aec9700 1 -- 172.21.15.82:0/172942575 shutdown_connections 2024-05-20T16:13:38.576 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:38.574+0000 7f3f2aec9700 1 --2- 172.21.15.82:0/172942575 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3f24103330 0x7f3f24109be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:38.576 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:38.574+0000 7f3f2aec9700 1 --2- 172.21.15.82:0/172942575 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3f24101860 0x7f3f24101c70 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:38.576 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:38.574+0000 7f3f2aec9700 1 --2- 172.21.15.82:0/172942575 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3f24102990 0x7f3f24102da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:38.576 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:38.574+0000 7f3f2aec9700 1 -- 172.21.15.82:0/172942575 >> 172.21.15.82:0/172942575 conn(0x7f3f240fcea0 msgr2=0x7f3f240ff350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:13:38.576 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:38.574+0000 7f3f2aec9700 1 -- 172.21.15.82:0/172942575 shutdown_connections 2024-05-20T16:13:38.577 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:38.574+0000 7f3f2aec9700 1 -- 172.21.15.82:0/172942575 wait complete. 2024-05-20T16:13:38.577 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:38.575+0000 7f3f2aec9700 1 Processor -- start 2024-05-20T16:13:38.577 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:38.575+0000 7f3f2aec9700 1 -- start start 2024-05-20T16:13:38.577 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:38.575+0000 7f3f2aec9700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3f24101860 0x7f3f24199940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:38.577 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:38.575+0000 7f3f2aec9700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3f24102990 0x7f3f24199e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:38.577 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:38.575+0000 7f3f2aec9700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3f24103330 0x7f3f241a0e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:38.577 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:38.575+0000 7f3f2aec9700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f3f2410cbf0 con 0x7f3f24103330 2024-05-20T16:13:38.577 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:38.575+0000 7f3f2aec9700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f3f2410ca70 con 0x7f3f24101860 2024-05-20T16:13:38.577 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:38.575+0000 7f3f2aec9700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f3f2410cd70 con 0x7f3f24102990 2024-05-20T16:13:38.577 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:38.575+0000 7f3f29466700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3f24103330 0x7f3f241a0e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:13:38.577 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:38.575+0000 7f3f28c65700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3f24101860 0x7f3f24199940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:13:38.577 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:38.575+0000 7f3f29466700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3f24103330 0x7f3f241a0e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:33696/0 (socket says 172.21.15.82:33696) 2024-05-20T16:13:38.577 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:38.575+0000 7f3f29466700 1 -- 172.21.15.82:0/603967992 learned_addr learned my addr 172.21.15.82:0/603967992 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:13:38.577 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:38.575+0000 7f3f23fff700 1 --2- 172.21.15.82:0/603967992 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3f24102990 0x7f3f24199e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:13:38.578 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:38.575+0000 7f3f29466700 1 -- 172.21.15.82:0/603967992 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3f24102990 msgr2=0x7f3f24199e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:38.578 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:38.575+0000 7f3f29466700 1 --2- 172.21.15.82:0/603967992 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3f24102990 0x7f3f24199e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:38.578 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:38.575+0000 7f3f29466700 1 -- 172.21.15.82:0/603967992 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3f24101860 msgr2=0x7f3f24199940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:38.578 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:38.575+0000 7f3f29466700 1 --2- 172.21.15.82:0/603967992 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3f24101860 0x7f3f24199940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:38.578 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:38.575+0000 7f3f29466700 1 -- 172.21.15.82:0/603967992 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f241a1410 con 0x7f3f24103330 2024-05-20T16:13:38.578 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:38.576+0000 7f3f29466700 1 --2- 172.21.15.82:0/603967992 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3f24103330 0x7f3f241a0e20 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f3f1c007d70 tx=0x7f3f1c00a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:13:38.578 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:38.576+0000 7f3f21ffb700 1 -- 172.21.15.82:0/603967992 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f1c016020 con 0x7f3f24103330 2024-05-20T16:13:38.578 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:38.576+0000 7f3f21ffb700 1 -- 172.21.15.82:0/603967992 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3f1c00ac90 con 0x7f3f24103330 2024-05-20T16:13:38.578 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:38.576+0000 7f3f21ffb700 1 -- 172.21.15.82:0/603967992 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f1c0115c0 con 0x7f3f24103330 2024-05-20T16:13:38.578 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:38.576+0000 7f3f2aec9700 1 -- 172.21.15.82:0/603967992 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f241a1700 con 0x7f3f24103330 2024-05-20T16:13:38.578 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:38.576+0000 7f3f2aec9700 1 -- 172.21.15.82:0/603967992 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f241a1cc0 con 0x7f3f24103330 2024-05-20T16:13:38.579 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:38.577+0000 7f3f2aec9700 1 -- 172.21.15.82:0/603967992 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f24050170 con 0x7f3f24103330 2024-05-20T16:13:38.579 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:38.578+0000 7f3f21ffb700 1 -- 172.21.15.82:0/603967992 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f3f1c019020 con 0x7f3f24103330 2024-05-20T16:13:38.579 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:38.579+0000 7f3f21ffb700 1 --2- 172.21.15.82:0/603967992 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f3f140a2ca0 0x7f3f140a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:38.580 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:38.579+0000 7f3f28c65700 1 --2- 172.21.15.82:0/603967992 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f3f140a2ca0 0x7f3f140a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:13:38.580 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:38.579+0000 7f3f21ffb700 1 -- 172.21.15.82:0/603967992 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9092+0+0 (secure 0 0 0) 0x7f3f1c0cf280 con 0x7f3f24103330 2024-05-20T16:13:38.580 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:38.579+0000 7f3f28c65700 1 --2- 172.21.15.82:0/603967992 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f3f140a2ca0 0x7f3f140a51b0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f3f18003d60 tx=0x7f3f18003cf0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:13:38.583 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:38.584+0000 7f3f21ffb700 1 -- 172.21.15.82:0/603967992 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f3f1c020020 con 0x7f3f24103330 2024-05-20T16:13:38.794 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:38.795+0000 7f3f2aec9700 1 -- 172.21.15.82:0/603967992 --> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7f3f241a1f80 con 0x7f3f140a2ca0 2024-05-20T16:13:38.796 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:38.796+0000 7f3f21ffb700 1 -- 172.21.15.82:0/603967992 <== mgr.14150 v2:172.21.15.82:6800/379530823 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7f3f241a1f80 con 0x7f3f140a2ca0 2024-05-20T16:13:38.796 INFO:teuthology.orchestra.run.smithi082.stdout:Backend: cephadm 2024-05-20T16:13:38.796 INFO:teuthology.orchestra.run.smithi082.stdout:Available: Yes 2024-05-20T16:13:38.796 INFO:teuthology.orchestra.run.smithi082.stdout:Paused: No 2024-05-20T16:13:38.798 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:38.798+0000 7f3f2aec9700 1 -- 172.21.15.82:0/603967992 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f3f140a2ca0 msgr2=0x7f3f140a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:38.799 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:38.798+0000 7f3f2aec9700 1 --2- 172.21.15.82:0/603967992 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f3f140a2ca0 0x7f3f140a51b0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f3f18003d60 tx=0x7f3f18003cf0 comp rx=0 tx=0).stop 2024-05-20T16:13:38.799 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:38.798+0000 7f3f2aec9700 1 -- 172.21.15.82:0/603967992 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3f24103330 msgr2=0x7f3f241a0e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:38.799 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:38.798+0000 7f3f2aec9700 1 --2- 172.21.15.82:0/603967992 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3f24103330 0x7f3f241a0e20 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f3f1c007d70 tx=0x7f3f1c00a340 comp rx=0 tx=0).stop 2024-05-20T16:13:38.799 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:38.799+0000 7f3f2aec9700 1 -- 172.21.15.82:0/603967992 shutdown_connections 2024-05-20T16:13:38.799 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:38.799+0000 7f3f2aec9700 1 --2- 172.21.15.82:0/603967992 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f3f140a2ca0 0x7f3f140a51b0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:38.799 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:38.799+0000 7f3f2aec9700 1 --2- 172.21.15.82:0/603967992 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3f24101860 0x7f3f24199940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:38.799 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:38.799+0000 7f3f2aec9700 1 --2- 172.21.15.82:0/603967992 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3f24103330 0x7f3f241a0e20 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:38.799 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:38.799+0000 7f3f2aec9700 1 --2- 172.21.15.82:0/603967992 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3f24102990 0x7f3f24199e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:38.799 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:38.799+0000 7f3f2aec9700 1 -- 172.21.15.82:0/603967992 >> 172.21.15.82:0/603967992 conn(0x7f3f240fcea0 msgr2=0x7f3f24107dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:13:38.799 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:38.799+0000 7f3f2aec9700 1 -- 172.21.15.82:0/603967992 shutdown_connections 2024-05-20T16:13:38.799 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:38.799+0000 7f3f2aec9700 1 -- 172.21.15.82:0/603967992 wait complete. 2024-05-20T16:13:39.440 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:13:39 smithi082 ceph-mon[32238]: from='client.15273 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-20T16:13:39.440 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:13:39 smithi082 ceph-mon[32238]: pgmap v350: 129 pgs: 129 active+clean; 641 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-20T16:13:39.442 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 59d1600a-16c2-11ef-bc9a-c7b262605968 -- bash -c 'ceph orch ps' 2024-05-20T16:13:39.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:13:39 smithi184 ceph-mon[36209]: from='client.15273 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-20T16:13:39.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:13:39 smithi184 ceph-mon[36209]: pgmap v350: 129 pgs: 129 active+clean; 641 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-20T16:13:39.683 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T16:13:39.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:13:39 smithi160 ceph-mon[36347]: from='client.15273 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-20T16:13:39.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:13:39 smithi160 ceph-mon[36347]: pgmap v350: 129 pgs: 129 active+clean; 641 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-20T16:13:41.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:41.391+0000 7f63d5043700 1 -- 172.21.15.82:0/2154609369 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f63d0103b70 msgr2=0x7f63d010a640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:41.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:41.391+0000 7f63d5043700 1 --2- 172.21.15.82:0/2154609369 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f63d0103b70 0x7f63d010a640 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f63c800b0a0 tx=0x7f63c800b3b0 comp rx=0 tx=0).stop 2024-05-20T16:13:41.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:41.391+0000 7f63d5043700 1 -- 172.21.15.82:0/2154609369 shutdown_connections 2024-05-20T16:13:41.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:41.391+0000 7f63d5043700 1 --2- 172.21.15.82:0/2154609369 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f63d0102670 0x7f63d0102a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:41.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:41.391+0000 7f63d5043700 1 --2- 172.21.15.82:0/2154609369 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f63d0103b70 0x7f63d010a640 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:41.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:41.391+0000 7f63d5043700 1 --2- 172.21.15.82:0/2154609369 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f63d0103050 0x7f63d01034e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:41.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:41.391+0000 7f63d5043700 1 -- 172.21.15.82:0/2154609369 >> 172.21.15.82:0/2154609369 conn(0x7f63d00fcea0 msgr2=0x7f63d00ff350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:13:41.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:41.391+0000 7f63d5043700 1 -- 172.21.15.82:0/2154609369 shutdown_connections 2024-05-20T16:13:41.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:41.391+0000 7f63d5043700 1 -- 172.21.15.82:0/2154609369 wait complete. 2024-05-20T16:13:41.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:41.392+0000 7f63d5043700 1 Processor -- start 2024-05-20T16:13:41.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:41.392+0000 7f63d5043700 1 -- start start 2024-05-20T16:13:41.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:41.392+0000 7f63d5043700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f63d0102670 0x7f63d0199840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:41.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:41.392+0000 7f63d5043700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f63d0103050 0x7f63d0199d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:41.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:41.392+0000 7f63d5043700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f63d0103b70 0x7f63d01a0d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:41.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:41.392+0000 7f63d5043700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f63d010ee30 con 0x7f63d0102670 2024-05-20T16:13:41.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:41.392+0000 7f63d5043700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f63d010ecb0 con 0x7f63d0103b70 2024-05-20T16:13:41.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:41.392+0000 7f63d5043700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f63d010efb0 con 0x7f63d0103050 2024-05-20T16:13:41.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:41.392+0000 7f63ced9d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f63d0102670 0x7f63d0199840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:13:41.394 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:41.392+0000 7f63ced9d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f63d0102670 0x7f63d0199840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:49318/0 (socket says 172.21.15.82:49318) 2024-05-20T16:13:41.394 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:41.392+0000 7f63ced9d700 1 -- 172.21.15.82:0/383207682 learned_addr learned my addr 172.21.15.82:0/383207682 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:13:41.394 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:41.393+0000 7f63cf59e700 1 --2- 172.21.15.82:0/383207682 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f63d0103b70 0x7f63d01a0d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:13:41.394 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:41.393+0000 7f63ce59c700 1 --2- 172.21.15.82:0/383207682 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f63d0103050 0x7f63d0199d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:13:41.394 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:41.393+0000 7f63ced9d700 1 -- 172.21.15.82:0/383207682 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f63d0103050 msgr2=0x7f63d0199d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:41.394 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:41.393+0000 7f63ced9d700 1 --2- 172.21.15.82:0/383207682 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f63d0103050 0x7f63d0199d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:41.394 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:41.393+0000 7f63ced9d700 1 -- 172.21.15.82:0/383207682 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f63d0103b70 msgr2=0x7f63d01a0d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:41.394 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:41.393+0000 7f63ced9d700 1 --2- 172.21.15.82:0/383207682 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f63d0103b70 0x7f63d01a0d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:41.394 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:41.393+0000 7f63ced9d700 1 -- 172.21.15.82:0/383207682 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63d01a13a0 con 0x7f63d0102670 2024-05-20T16:13:41.394 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:41.393+0000 7f63ce59c700 1 --2- 172.21.15.82:0/383207682 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f63d0103050 0x7f63d0199d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T16:13:41.394 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:41.393+0000 7f63ced9d700 1 --2- 172.21.15.82:0/383207682 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f63d0102670 0x7f63d0199840 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f63c000c8c0 tx=0x7f63c000cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:13:41.394 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:41.393+0000 7f63b7fff700 1 -- 172.21.15.82:0/383207682 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63c0004230 con 0x7f63d0102670 2024-05-20T16:13:41.394 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:41.393+0000 7f63d5043700 1 -- 172.21.15.82:0/383207682 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63d01a1690 con 0x7f63d0102670 2024-05-20T16:13:41.394 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:41.393+0000 7f63b7fff700 1 -- 172.21.15.82:0/383207682 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f63c0004d10 con 0x7f63d0102670 2024-05-20T16:13:41.395 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:41.393+0000 7f63b7fff700 1 -- 172.21.15.82:0/383207682 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63c0007710 con 0x7f63d0102670 2024-05-20T16:13:41.395 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:41.393+0000 7f63d5043700 1 -- 172.21.15.82:0/383207682 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63d01a1bd0 con 0x7f63d0102670 2024-05-20T16:13:41.395 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:41.394+0000 7f63d5043700 1 -- 172.21.15.82:0/383207682 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f63d0050170 con 0x7f63d0102670 2024-05-20T16:13:41.395 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:41.396+0000 7f63b7fff700 1 -- 172.21.15.82:0/383207682 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f63c0004750 con 0x7f63d0102670 2024-05-20T16:13:41.396 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:41.396+0000 7f63b7fff700 1 --2- 172.21.15.82:0/383207682 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f63b80a2c20 0x7f63b80a5130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:41.396 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:41.396+0000 7f63ce59c700 1 --2- 172.21.15.82:0/383207682 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f63b80a2c20 0x7f63b80a5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:13:41.396 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:41.396+0000 7f63b7fff700 1 -- 172.21.15.82:0/383207682 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9092+0+0 (secure 0 0 0) 0x7f63c00cf390 con 0x7f63d0102670 2024-05-20T16:13:41.396 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:41.397+0000 7f63ce59c700 1 --2- 172.21.15.82:0/383207682 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f63b80a2c20 0x7f63b80a5130 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f63d019a8f0 tx=0x7f63c4008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:13:41.399 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:41.400+0000 7f63b7fff700 1 -- 172.21.15.82:0/383207682 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f63c0014070 con 0x7f63d0102670 2024-05-20T16:13:41.620 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:41.620+0000 7f63d5043700 1 -- 172.21.15.82:0/383207682 --> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f63d0062a80 con 0x7f63b80a2c20 2024-05-20T16:13:41.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:41.634+0000 7f63b7fff700 1 -- 172.21.15.82:0/383207682 <== mgr.14150 v2:172.21.15.82:6800/379530823 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4185 (secure 0 0 0) 0x7f63d0062a80 con 0x7f63b80a2c20 2024-05-20T16:13:41.634 INFO:teuthology.orchestra.run.smithi082.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-20T16:13:41.634 INFO:teuthology.orchestra.run.smithi082.stdout:mds.a smithi082 running (2m) 18s ago 2m 81.1M - 19.0.0-2744-gd2b507c7 cfcc61b2d476 d27456f6239e 2024-05-20T16:13:41.634 INFO:teuthology.orchestra.run.smithi082.stdout:mds.b smithi160 running (118s) 52s ago 118s 92.7M - 19.0.0-2744-gd2b507c7 cfcc61b2d476 adc717c8bfc1 2024-05-20T16:13:41.634 INFO:teuthology.orchestra.run.smithi082.stdout:mds.c smithi184 running (2m) 52s ago 2m 83.6M - 19.0.0-2744-gd2b507c7 cfcc61b2d476 aa07be6b7215 2024-05-20T16:13:41.634 INFO:teuthology.orchestra.run.smithi082.stdout:mds.d smithi082 running (2m) 18s ago 2m 83.7M - 19.0.0-2744-gd2b507c7 cfcc61b2d476 98a34b38a262 2024-05-20T16:13:41.634 INFO:teuthology.orchestra.run.smithi082.stdout:mds.e smithi160 running (2m) 52s ago 2m 85.6M - 19.0.0-2744-gd2b507c7 cfcc61b2d476 cc8764d81953 2024-05-20T16:13:41.634 INFO:teuthology.orchestra.run.smithi082.stdout:mds.f smithi184 running (2m) 52s ago 2m 85.0M - 19.0.0-2744-gd2b507c7 cfcc61b2d476 70d70110f88e 2024-05-20T16:13:41.634 INFO:teuthology.orchestra.run.smithi082.stdout:mds.g smithi082 running (2m) 18s ago 2m 83.9M - 19.0.0-2744-gd2b507c7 cfcc61b2d476 e65fc67c67e4 2024-05-20T16:13:41.634 INFO:teuthology.orchestra.run.smithi082.stdout:mds.h smithi160 running (2m) 52s ago 2m 84.6M - 19.0.0-2744-gd2b507c7 cfcc61b2d476 0e2acd3f66e0 2024-05-20T16:13:41.634 INFO:teuthology.orchestra.run.smithi082.stdout:mds.i smithi184 running (2m) 52s ago 2m 84.3M - 19.0.0-2744-gd2b507c7 cfcc61b2d476 a27a46359d78 2024-05-20T16:13:41.635 INFO:teuthology.orchestra.run.smithi082.stdout:mds.j smithi082 running (2m) 18s ago 2m 81.7M - 19.0.0-2744-gd2b507c7 cfcc61b2d476 94754e0ff0ac 2024-05-20T16:13:41.635 INFO:teuthology.orchestra.run.smithi082.stdout:mds.k smithi160 running (2m) 52s ago 2m 81.4M - 19.0.0-2744-gd2b507c7 cfcc61b2d476 d601d0207359 2024-05-20T16:13:41.635 INFO:teuthology.orchestra.run.smithi082.stdout:mds.l smithi184 running (2m) 52s ago 2m 83.9M - 19.0.0-2744-gd2b507c7 cfcc61b2d476 e6e63d62472b 2024-05-20T16:13:41.635 INFO:teuthology.orchestra.run.smithi082.stdout:mgr.x smithi082 *:9283,8765 running (9m) 18s ago 9m 583M - 19.0.0-2744-gd2b507c7 cfcc61b2d476 c80a162fbf8b 2024-05-20T16:13:41.635 INFO:teuthology.orchestra.run.smithi082.stdout:mgr.y smithi160 *:8443,8765 running (7m) 52s ago 7m 522M - 19.0.0-2744-gd2b507c7 cfcc61b2d476 83c00ac57e95 2024-05-20T16:13:41.635 INFO:teuthology.orchestra.run.smithi082.stdout:mgr.z smithi184 *:8443,8765 running (7m) 52s ago 7m 520M - 19.0.0-2744-gd2b507c7 cfcc61b2d476 f9abb8907d23 2024-05-20T16:13:41.635 INFO:teuthology.orchestra.run.smithi082.stdout:mon.a smithi082 running (9m) 18s ago 9m 117M 2048M 19.0.0-2744-gd2b507c7 cfcc61b2d476 1c5b1c87352a 2024-05-20T16:13:41.635 INFO:teuthology.orchestra.run.smithi082.stdout:mon.b smithi160 running (8m) 52s ago 8m 108M 2048M 19.0.0-2744-gd2b507c7 cfcc61b2d476 8de30f4d82e8 2024-05-20T16:13:41.635 INFO:teuthology.orchestra.run.smithi082.stdout:mon.c smithi184 running (8m) 52s ago 8m 106M 2048M 19.0.0-2744-gd2b507c7 cfcc61b2d476 6f3a6f7f153e 2024-05-20T16:13:41.635 INFO:teuthology.orchestra.run.smithi082.stdout:osd.0 smithi082 running (7m) 18s ago 7m 116M 4272M 19.0.0-2744-gd2b507c7 cfcc61b2d476 b6842cf5e490 2024-05-20T16:13:41.635 INFO:teuthology.orchestra.run.smithi082.stdout:osd.1 smithi160 running (7m) 52s ago 7m 114M 4272M 19.0.0-2744-gd2b507c7 cfcc61b2d476 fb3a05234447 2024-05-20T16:13:41.635 INFO:teuthology.orchestra.run.smithi082.stdout:osd.2 smithi184 running (6m) 52s ago 6m 115M 4272M 19.0.0-2744-gd2b507c7 cfcc61b2d476 d38d48c4a484 2024-05-20T16:13:41.635 INFO:teuthology.orchestra.run.smithi082.stdout:osd.3 smithi082 running (6m) 18s ago 6m 114M 4272M 19.0.0-2744-gd2b507c7 cfcc61b2d476 4c47bfe5902b 2024-05-20T16:13:41.635 INFO:teuthology.orchestra.run.smithi082.stdout:osd.4 smithi160 running (5m) 52s ago 5m 111M 4272M 19.0.0-2744-gd2b507c7 cfcc61b2d476 2ff7d831d05d 2024-05-20T16:13:41.635 INFO:teuthology.orchestra.run.smithi082.stdout:osd.5 smithi184 running (5m) 52s ago 5m 111M 4272M 19.0.0-2744-gd2b507c7 cfcc61b2d476 70cb5e21cdb7 2024-05-20T16:13:41.636 INFO:teuthology.orchestra.run.smithi082.stdout:osd.6 smithi082 running (5m) 18s ago 5m 122M 4272M 19.0.0-2744-gd2b507c7 cfcc61b2d476 c86c6eea4a11 2024-05-20T16:13:41.636 INFO:teuthology.orchestra.run.smithi082.stdout:osd.7 smithi160 running (4m) 52s ago 4m 122M 4272M 19.0.0-2744-gd2b507c7 cfcc61b2d476 d91279f7998d 2024-05-20T16:13:41.636 INFO:teuthology.orchestra.run.smithi082.stdout:osd.8 smithi184 running (4m) 52s ago 4m 121M 4272M 19.0.0-2744-gd2b507c7 cfcc61b2d476 4b0db37e2c62 2024-05-20T16:13:41.636 INFO:teuthology.orchestra.run.smithi082.stdout:osd.9 smithi082 running (3m) 18s ago 3m 109M 4272M 19.0.0-2744-gd2b507c7 cfcc61b2d476 1e12de28d19e 2024-05-20T16:13:41.636 INFO:teuthology.orchestra.run.smithi082.stdout:osd.10 smithi160 running (3m) 52s ago 3m 114M 4272M 19.0.0-2744-gd2b507c7 cfcc61b2d476 14aa686b3d1e 2024-05-20T16:13:41.636 INFO:teuthology.orchestra.run.smithi082.stdout:osd.11 smithi184 running (2m) 52s ago 2m 109M 4272M 19.0.0-2744-gd2b507c7 cfcc61b2d476 69f1b6e3fbc5 2024-05-20T16:13:41.636 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:13:41 smithi082 ceph-mon[32238]: pgmap v351: 129 pgs: 129 active+clean; 641 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-20T16:13:41.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:41.636+0000 7f63d5043700 1 -- 172.21.15.82:0/383207682 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f63b80a2c20 msgr2=0x7f63b80a5130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:41.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:41.636+0000 7f63d5043700 1 --2- 172.21.15.82:0/383207682 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f63b80a2c20 0x7f63b80a5130 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f63d019a8f0 tx=0x7f63c4008040 comp rx=0 tx=0).stop 2024-05-20T16:13:41.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:41.636+0000 7f63d5043700 1 -- 172.21.15.82:0/383207682 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f63d0102670 msgr2=0x7f63d0199840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:41.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:41.636+0000 7f63d5043700 1 --2- 172.21.15.82:0/383207682 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f63d0102670 0x7f63d0199840 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f63c000c8c0 tx=0x7f63c000cbd0 comp rx=0 tx=0).stop 2024-05-20T16:13:41.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:41.636+0000 7f63d5043700 1 -- 172.21.15.82:0/383207682 shutdown_connections 2024-05-20T16:13:41.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:41.636+0000 7f63d5043700 1 --2- 172.21.15.82:0/383207682 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f63b80a2c20 0x7f63b80a5130 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:41.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:41.636+0000 7f63d5043700 1 --2- 172.21.15.82:0/383207682 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f63d0103b70 0x7f63d01a0d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:41.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:41.636+0000 7f63d5043700 1 --2- 172.21.15.82:0/383207682 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f63d0102670 0x7f63d0199840 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:41.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:41.636+0000 7f63d5043700 1 --2- 172.21.15.82:0/383207682 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f63d0103050 0x7f63d0199d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:41.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:41.636+0000 7f63d5043700 1 -- 172.21.15.82:0/383207682 >> 172.21.15.82:0/383207682 conn(0x7f63d00fcea0 msgr2=0x7f63d0106620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:13:41.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:41.636+0000 7f63d5043700 1 -- 172.21.15.82:0/383207682 shutdown_connections 2024-05-20T16:13:41.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:41.636+0000 7f63d5043700 1 -- 172.21.15.82:0/383207682 wait complete. 2024-05-20T16:13:41.638 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:13:41 smithi184 ceph-mon[36209]: pgmap v351: 129 pgs: 129 active+clean; 641 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-20T16:13:41.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:13:41 smithi160 ceph-mon[36347]: pgmap v351: 129 pgs: 129 active+clean; 641 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-20T16:13:42.402 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 59d1600a-16c2-11ef-bc9a-c7b262605968 -- bash -c 'ceph orch ls' 2024-05-20T16:13:42.647 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T16:13:42.769 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:13:42 smithi082 ceph-mon[32238]: from='client.15279 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-20T16:13:42.885 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:13:42 smithi184 ceph-mon[36209]: from='client.15279 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-20T16:13:43.062 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:13:42 smithi160 ceph-mon[36347]: from='client.15279 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-20T16:13:43.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:13:43 smithi082 ceph-mon[32238]: pgmap v352: 129 pgs: 129 active+clean; 641 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-20T16:13:43.885 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:13:43 smithi184 ceph-mon[36209]: pgmap v352: 129 pgs: 129 active+clean; 641 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-20T16:13:44.062 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:13:43 smithi160 ceph-mon[36347]: pgmap v352: 129 pgs: 129 active+clean; 641 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-20T16:13:44.385 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:44.385+0000 7ff227b43700 1 -- 172.21.15.82:0/109693985 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff2201020c0 msgr2=0x7ff2201024d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:44.386 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:44.385+0000 7ff227b43700 1 --2- 172.21.15.82:0/109693985 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff2201020c0 0x7ff2201024d0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7ff21800b0a0 tx=0x7ff21800b3b0 comp rx=0 tx=0).stop 2024-05-20T16:13:44.386 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:44.385+0000 7ff227b43700 1 -- 172.21.15.82:0/109693985 shutdown_connections 2024-05-20T16:13:44.386 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:44.385+0000 7ff227b43700 1 --2- 172.21.15.82:0/109693985 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff220103b90 0x7ff22010a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:44.386 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:44.385+0000 7ff227b43700 1 --2- 172.21.15.82:0/109693985 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff2201020c0 0x7ff2201024d0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:44.386 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:44.385+0000 7ff227b43700 1 --2- 172.21.15.82:0/109693985 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff2201031f0 0x7ff220103600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:44.386 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:44.385+0000 7ff227b43700 1 -- 172.21.15.82:0/109693985 >> 172.21.15.82:0/109693985 conn(0x7ff2200fd700 msgr2=0x7ff2200ffbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:13:44.386 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:44.386+0000 7ff227b43700 1 -- 172.21.15.82:0/109693985 shutdown_connections 2024-05-20T16:13:44.386 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:44.386+0000 7ff227b43700 1 -- 172.21.15.82:0/109693985 wait complete. 2024-05-20T16:13:44.388 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:44.386+0000 7ff227b43700 1 Processor -- start 2024-05-20T16:13:44.388 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:44.386+0000 7ff227b43700 1 -- start start 2024-05-20T16:13:44.388 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:44.386+0000 7ff227b43700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff2201020c0 0x7ff22006aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:44.388 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:44.386+0000 7ff227b43700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff2201031f0 0x7ff22019e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:44.388 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:44.386+0000 7ff227b43700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff220103b90 0x7ff2201a0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:44.388 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:44.386+0000 7ff227b43700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7ff22010dc80 con 0x7ff2201031f0 2024-05-20T16:13:44.388 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:44.386+0000 7ff227b43700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7ff22010db00 con 0x7ff220103b90 2024-05-20T16:13:44.388 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:44.386+0000 7ff227b43700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ff22010de00 con 0x7ff2201020c0 2024-05-20T16:13:44.389 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:44.387+0000 7ff2250de700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff2201031f0 0x7ff22019e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:13:44.389 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:44.387+0000 7ff2250de700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff2201031f0 0x7ff22019e650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:49336/0 (socket says 172.21.15.82:49336) 2024-05-20T16:13:44.389 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:44.387+0000 7ff2250de700 1 -- 172.21.15.82:0/2792389377 learned_addr learned my addr 172.21.15.82:0/2792389377 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:13:44.389 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:44.387+0000 7ff2258df700 1 --2- 172.21.15.82:0/2792389377 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff2201020c0 0x7ff22006aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:13:44.389 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:44.387+0000 7ff2260e0700 1 --2- 172.21.15.82:0/2792389377 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff220103b90 0x7ff2201a0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:13:44.389 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:44.387+0000 7ff2250de700 1 -- 172.21.15.82:0/2792389377 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff2201020c0 msgr2=0x7ff22006aef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:44.389 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:44.387+0000 7ff2250de700 1 --2- 172.21.15.82:0/2792389377 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff2201020c0 0x7ff22006aef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:44.389 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:44.387+0000 7ff2250de700 1 -- 172.21.15.82:0/2792389377 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff220103b90 msgr2=0x7ff2201a0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:44.389 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:44.387+0000 7ff2250de700 1 --2- 172.21.15.82:0/2792389377 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff220103b90 0x7ff2201a0bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:44.389 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:44.387+0000 7ff2250de700 1 -- 172.21.15.82:0/2792389377 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2201a13f0 con 0x7ff2201031f0 2024-05-20T16:13:44.389 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:44.387+0000 7ff2260e0700 1 --2- 172.21.15.82:0/2792389377 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff220103b90 0x7ff2201a0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T16:13:44.389 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:44.387+0000 7ff2250de700 1 --2- 172.21.15.82:0/2792389377 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff2201031f0 0x7ff22019e650 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7ff21400d950 tx=0x7ff21400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:13:44.389 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:44.387+0000 7ff21effd700 1 -- 172.21.15.82:0/2792389377 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff214016070 con 0x7ff2201031f0 2024-05-20T16:13:44.390 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:44.387+0000 7ff21effd700 1 -- 172.21.15.82:0/2792389377 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff214009e20 con 0x7ff2201031f0 2024-05-20T16:13:44.390 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:44.387+0000 7ff227b43700 1 -- 172.21.15.82:0/2792389377 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2201a16e0 con 0x7ff2201031f0 2024-05-20T16:13:44.390 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:44.387+0000 7ff21effd700 1 -- 172.21.15.82:0/2792389377 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff21400f620 con 0x7ff2201031f0 2024-05-20T16:13:44.390 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:44.387+0000 7ff227b43700 1 -- 172.21.15.82:0/2792389377 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2201a1c20 con 0x7ff2201031f0 2024-05-20T16:13:44.390 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:44.388+0000 7ff227b43700 1 -- 172.21.15.82:0/2792389377 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff220050170 con 0x7ff2201031f0 2024-05-20T16:13:44.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:44.390+0000 7ff21effd700 1 -- 172.21.15.82:0/2792389377 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7ff214019070 con 0x7ff2201031f0 2024-05-20T16:13:44.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:44.390+0000 7ff21effd700 1 --2- 172.21.15.82:0/2792389377 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7ff2080a2c20 0x7ff2080a5130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:44.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:44.391+0000 7ff2258df700 1 --2- 172.21.15.82:0/2792389377 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7ff2080a2c20 0x7ff2080a5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:13:44.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:44.391+0000 7ff21effd700 1 -- 172.21.15.82:0/2792389377 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9092+0+0 (secure 0 0 0) 0x7ff2140cefa0 con 0x7ff2201031f0 2024-05-20T16:13:44.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:44.391+0000 7ff2258df700 1 --2- 172.21.15.82:0/2792389377 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7ff2080a2c20 0x7ff2080a5130 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7ff218007fa0 tx=0x7ff21800b7b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:13:44.394 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:44.394+0000 7ff21effd700 1 -- 172.21.15.82:0/2792389377 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7ff21409bba0 con 0x7ff2201031f0 2024-05-20T16:13:44.598 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:44.598+0000 7ff227b43700 1 -- 172.21.15.82:0/2792389377 --> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7ff220062a80 con 0x7ff2080a2c20 2024-05-20T16:13:44.604 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:44.605+0000 7ff21effd700 1 -- 172.21.15.82:0/2792389377 <== mgr.14150 v2:172.21.15.82:6800/379530823 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7ff220062a80 con 0x7ff2080a2c20 2024-05-20T16:13:44.604 INFO:teuthology.orchestra.run.smithi082.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-05-20T16:13:44.605 INFO:teuthology.orchestra.run.smithi082.stdout:mds.all 12/12 55s ago 2m smithi082=a;smithi082=d;smithi082=g;smithi082=j;smithi160=b;smithi160=e;smithi160=h;smithi160=k;smithi184=c;smithi184=f;smithi184=i;smithi184=l;count:12 2024-05-20T16:13:44.605 INFO:teuthology.orchestra.run.smithi082.stdout:mgr 3/3 55s ago 7m smithi082=x;smithi160=y;smithi184=z;count:3 2024-05-20T16:13:44.605 INFO:teuthology.orchestra.run.smithi082.stdout:mon 3/3 55s ago 8m smithi082:172.21.15.82=a;smithi160:172.21.15.160=b;smithi184:172.21.15.184=c;count:3 2024-05-20T16:13:44.605 INFO:teuthology.orchestra.run.smithi082.stdout:osd 12 55s ago - 2024-05-20T16:13:44.607 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:44.607+0000 7ff227b43700 1 -- 172.21.15.82:0/2792389377 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7ff2080a2c20 msgr2=0x7ff2080a5130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:44.607 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:44.607+0000 7ff227b43700 1 --2- 172.21.15.82:0/2792389377 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7ff2080a2c20 0x7ff2080a5130 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7ff218007fa0 tx=0x7ff21800b7b0 comp rx=0 tx=0).stop 2024-05-20T16:13:44.607 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:44.607+0000 7ff227b43700 1 -- 172.21.15.82:0/2792389377 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff2201031f0 msgr2=0x7ff22019e650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:44.607 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:44.607+0000 7ff227b43700 1 --2- 172.21.15.82:0/2792389377 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff2201031f0 0x7ff22019e650 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7ff21400d950 tx=0x7ff21400dc60 comp rx=0 tx=0).stop 2024-05-20T16:13:44.607 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:44.607+0000 7ff227b43700 1 -- 172.21.15.82:0/2792389377 shutdown_connections 2024-05-20T16:13:44.607 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:44.607+0000 7ff227b43700 1 --2- 172.21.15.82:0/2792389377 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7ff2080a2c20 0x7ff2080a5130 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:44.607 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:44.607+0000 7ff227b43700 1 --2- 172.21.15.82:0/2792389377 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff220103b90 0x7ff2201a0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:44.607 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:44.607+0000 7ff227b43700 1 --2- 172.21.15.82:0/2792389377 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff2201020c0 0x7ff22006aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:44.607 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:44.607+0000 7ff227b43700 1 --2- 172.21.15.82:0/2792389377 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff2201031f0 0x7ff22019e650 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:44.608 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:44.607+0000 7ff227b43700 1 -- 172.21.15.82:0/2792389377 >> 172.21.15.82:0/2792389377 conn(0x7ff2200fd700 msgr2=0x7ff220106420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:13:44.608 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:44.607+0000 7ff227b43700 1 -- 172.21.15.82:0/2792389377 shutdown_connections 2024-05-20T16:13:44.608 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:44.607+0000 7ff227b43700 1 -- 172.21.15.82:0/2792389377 wait complete. 2024-05-20T16:13:45.423 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 59d1600a-16c2-11ef-bc9a-c7b262605968 -- bash -c 'ceph orch host ls' 2024-05-20T16:13:45.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:13:45 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 1]}]: dispatch 2024-05-20T16:13:45.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:13:45 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]: dispatch 2024-05-20T16:13:45.635 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:13:45 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 1]}]: dispatch 2024-05-20T16:13:45.635 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:13:45 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]: dispatch 2024-05-20T16:13:45.667 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T16:13:45.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:13:45 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 1]}]: dispatch 2024-05-20T16:13:45.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:13:45 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]: dispatch 2024-05-20T16:13:46.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:13:46 smithi082 ceph-mon[32238]: from='client.15285 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-20T16:13:46.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:13:46 smithi082 ceph-mon[32238]: pgmap v353: 129 pgs: 129 active+clean; 641 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-20T16:13:46.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:13:46 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 1]}]': finished 2024-05-20T16:13:46.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:13:46 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]': finished 2024-05-20T16:13:46.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:13:46 smithi082 ceph-mon[32238]: osdmap e94: 12 total, 12 up, 12 in 2024-05-20T16:13:46.635 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:13:46 smithi184 ceph-mon[36209]: from='client.15285 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-20T16:13:46.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:13:46 smithi184 ceph-mon[36209]: pgmap v353: 129 pgs: 129 active+clean; 641 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-20T16:13:46.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:13:46 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 1]}]': finished 2024-05-20T16:13:46.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:13:46 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]': finished 2024-05-20T16:13:46.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:13:46 smithi184 ceph-mon[36209]: osdmap e94: 12 total, 12 up, 12 in 2024-05-20T16:13:46.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:13:46 smithi160 ceph-mon[36347]: from='client.15285 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-20T16:13:46.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:13:46 smithi160 ceph-mon[36347]: pgmap v353: 129 pgs: 129 active+clean; 641 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-20T16:13:46.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:13:46 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 1]}]': finished 2024-05-20T16:13:46.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:13:46 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]': finished 2024-05-20T16:13:46.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:13:46 smithi160 ceph-mon[36347]: osdmap e94: 12 total, 12 up, 12 in 2024-05-20T16:13:47.423 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:47.423+0000 7f5069fbf700 1 -- 172.21.15.82:0/1004014131 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5064104cc0 msgr2=0x7f506410b570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:47.423 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:47.423+0000 7f5069fbf700 1 --2- 172.21.15.82:0/1004014131 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5064104cc0 0x7f506410b570 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f505c00b0a0 tx=0x7f505c00b3b0 comp rx=0 tx=0).stop 2024-05-20T16:13:47.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:47.423+0000 7f5069fbf700 1 -- 172.21.15.82:0/1004014131 shutdown_connections 2024-05-20T16:13:47.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:47.423+0000 7f5069fbf700 1 --2- 172.21.15.82:0/1004014131 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5064104cc0 0x7f506410b570 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:47.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:47.423+0000 7f5069fbf700 1 --2- 172.21.15.82:0/1004014131 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f50641031f0 0x7f5064103600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:47.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:47.423+0000 7f5069fbf700 1 --2- 172.21.15.82:0/1004014131 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5064104320 0x7f5064104730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:47.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:47.423+0000 7f5069fbf700 1 -- 172.21.15.82:0/1004014131 >> 172.21.15.82:0/1004014131 conn(0x7f50640fe850 msgr2=0x7f5064100ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:13:47.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:47.424+0000 7f5069fbf700 1 -- 172.21.15.82:0/1004014131 shutdown_connections 2024-05-20T16:13:47.424 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:13:47 smithi082 ceph-mon[32238]: osdmap e95: 12 total, 12 up, 12 in 2024-05-20T16:13:47.424 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:13:47 smithi082 ceph-mon[32238]: pgmap v356: 129 pgs: 129 active+clean; 641 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 3 op/s 2024-05-20T16:13:47.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:47.424+0000 7f5069fbf700 1 -- 172.21.15.82:0/1004014131 wait complete. 2024-05-20T16:13:47.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:47.424+0000 7f5069fbf700 1 Processor -- start 2024-05-20T16:13:47.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:47.424+0000 7f5069fbf700 1 -- start start 2024-05-20T16:13:47.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:47.425+0000 7f5069fbf700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f50641031f0 0x7f5064199820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:47.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:47.425+0000 7f5069fbf700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5064104320 0x7f5064199d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:47.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:47.425+0000 7f5069fbf700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5064104cc0 0x7f50641a0d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:47.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:47.425+0000 7f5069fbf700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f506410eda0 con 0x7f50641031f0 2024-05-20T16:13:47.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:47.425+0000 7f5069fbf700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f506410ec20 con 0x7f5064104320 2024-05-20T16:13:47.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:47.425+0000 7f5069fbf700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f506410ef20 con 0x7f5064104cc0 2024-05-20T16:13:47.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:47.425+0000 7f50637fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f50641031f0 0x7f5064199820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:13:47.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:47.425+0000 7f50637fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f50641031f0 0x7f5064199820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:49358/0 (socket says 172.21.15.82:49358) 2024-05-20T16:13:47.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:47.425+0000 7f50637fe700 1 -- 172.21.15.82:0/878297982 learned_addr learned my addr 172.21.15.82:0/878297982 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:13:47.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:47.425+0000 7f505bfff700 1 --2- 172.21.15.82:0/878297982 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5064104320 0x7f5064199d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:13:47.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:47.425+0000 7f5063fff700 1 --2- 172.21.15.82:0/878297982 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5064104cc0 0x7f50641a0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:13:47.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:47.425+0000 7f50637fe700 1 -- 172.21.15.82:0/878297982 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5064104cc0 msgr2=0x7f50641a0d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:47.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:47.425+0000 7f50637fe700 1 --2- 172.21.15.82:0/878297982 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5064104cc0 0x7f50641a0d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:47.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:47.425+0000 7f50637fe700 1 -- 172.21.15.82:0/878297982 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5064104320 msgr2=0x7f5064199d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:47.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:47.425+0000 7f50637fe700 1 --2- 172.21.15.82:0/878297982 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5064104320 0x7f5064199d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:47.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:47.425+0000 7f50637fe700 1 -- 172.21.15.82:0/878297982 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50641a1380 con 0x7f50641031f0 2024-05-20T16:13:47.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:47.425+0000 7f505bfff700 1 --2- 172.21.15.82:0/878297982 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5064104320 0x7f5064199d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T16:13:47.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:47.425+0000 7f50637fe700 1 --2- 172.21.15.82:0/878297982 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f50641031f0 0x7f5064199820 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f505400d950 tx=0x7f505400dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:13:47.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:47.425+0000 7f50617fa700 1 -- 172.21.15.82:0/878297982 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5054017070 con 0x7f50641031f0 2024-05-20T16:13:47.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:47.426+0000 7f5069fbf700 1 -- 172.21.15.82:0/878297982 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50641a1670 con 0x7f50641031f0 2024-05-20T16:13:47.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:47.426+0000 7f50617fa700 1 -- 172.21.15.82:0/878297982 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f505400de30 con 0x7f50641031f0 2024-05-20T16:13:47.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:47.426+0000 7f50617fa700 1 -- 172.21.15.82:0/878297982 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5054003c70 con 0x7f50641031f0 2024-05-20T16:13:47.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:47.426+0000 7f5069fbf700 1 -- 172.21.15.82:0/878297982 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50641a1bb0 con 0x7f50641031f0 2024-05-20T16:13:47.429 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:47.427+0000 7f5069fbf700 1 -- 172.21.15.82:0/878297982 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5064050170 con 0x7f50641031f0 2024-05-20T16:13:47.429 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:47.428+0000 7f50617fa700 1 -- 172.21.15.82:0/878297982 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f5054009940 con 0x7f50641031f0 2024-05-20T16:13:47.429 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:47.429+0000 7f50617fa700 1 --2- 172.21.15.82:0/878297982 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f50500a2c20 0x7f50500a5130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:47.429 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:47.429+0000 7f50617fa700 1 -- 172.21.15.82:0/878297982 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9150+0+0 (secure 0 0 0) 0x7f50540cf720 con 0x7f50641031f0 2024-05-20T16:13:47.429 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:47.429+0000 7f505bfff700 1 --2- 172.21.15.82:0/878297982 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f50500a2c20 0x7f50500a5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:13:47.429 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:47.429+0000 7f505bfff700 1 --2- 172.21.15.82:0/878297982 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f50500a2c20 0x7f50500a5130 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f504c005950 tx=0x7f504c0058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:13:47.432 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:47.433+0000 7f50617fa700 1 -- 172.21.15.82:0/878297982 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f5054020030 con 0x7f50641031f0 2024-05-20T16:13:47.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:47.634+0000 7f5069fbf700 1 -- 172.21.15.82:0/878297982 --> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7f5064062a80 con 0x7f50500a2c20 2024-05-20T16:13:47.635 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:13:47 smithi184 ceph-mon[36209]: osdmap e95: 12 total, 12 up, 12 in 2024-05-20T16:13:47.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:13:47 smithi184 ceph-mon[36209]: pgmap v356: 129 pgs: 129 active+clean; 641 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 3 op/s 2024-05-20T16:13:47.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:47.636+0000 7f50617fa700 1 -- 172.21.15.82:0/878297982 <== mgr.14150 v2:172.21.15.82:6800/379530823 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7f5064062a80 con 0x7f50500a2c20 2024-05-20T16:13:47.636 INFO:teuthology.orchestra.run.smithi082.stdout:HOST ADDR LABELS STATUS 2024-05-20T16:13:47.636 INFO:teuthology.orchestra.run.smithi082.stdout:smithi082 172.21.15.82 2024-05-20T16:13:47.636 INFO:teuthology.orchestra.run.smithi082.stdout:smithi160 172.21.15.160 2024-05-20T16:13:47.636 INFO:teuthology.orchestra.run.smithi082.stdout:smithi184 172.21.15.184 2024-05-20T16:13:47.637 INFO:teuthology.orchestra.run.smithi082.stdout:3 hosts in cluster 2024-05-20T16:13:47.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:47.638+0000 7f5069fbf700 1 -- 172.21.15.82:0/878297982 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f50500a2c20 msgr2=0x7f50500a5130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:47.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:47.638+0000 7f5069fbf700 1 --2- 172.21.15.82:0/878297982 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f50500a2c20 0x7f50500a5130 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f504c005950 tx=0x7f504c0058e0 comp rx=0 tx=0).stop 2024-05-20T16:13:47.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:47.638+0000 7f5069fbf700 1 -- 172.21.15.82:0/878297982 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f50641031f0 msgr2=0x7f5064199820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:47.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:47.638+0000 7f5069fbf700 1 --2- 172.21.15.82:0/878297982 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f50641031f0 0x7f5064199820 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f505400d950 tx=0x7f505400dd10 comp rx=0 tx=0).stop 2024-05-20T16:13:47.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:47.638+0000 7f5069fbf700 1 -- 172.21.15.82:0/878297982 shutdown_connections 2024-05-20T16:13:47.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:47.638+0000 7f5069fbf700 1 --2- 172.21.15.82:0/878297982 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f50500a2c20 0x7f50500a5130 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:47.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:47.638+0000 7f5069fbf700 1 --2- 172.21.15.82:0/878297982 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5064104cc0 0x7f50641a0d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:47.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:47.638+0000 7f5069fbf700 1 --2- 172.21.15.82:0/878297982 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f50641031f0 0x7f5064199820 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:47.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:47.638+0000 7f5069fbf700 1 --2- 172.21.15.82:0/878297982 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5064104320 0x7f5064199d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:47.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:47.638+0000 7f5069fbf700 1 -- 172.21.15.82:0/878297982 >> 172.21.15.82:0/878297982 conn(0x7f50640fe850 msgr2=0x7f5064107550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:13:47.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:47.639+0000 7f5069fbf700 1 -- 172.21.15.82:0/878297982 shutdown_connections 2024-05-20T16:13:47.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:47.639+0000 7f5069fbf700 1 -- 172.21.15.82:0/878297982 wait complete. 2024-05-20T16:13:47.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:13:47 smithi160 ceph-mon[36347]: osdmap e95: 12 total, 12 up, 12 in 2024-05-20T16:13:47.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:13:47 smithi160 ceph-mon[36347]: pgmap v356: 129 pgs: 129 active+clean; 641 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 3 op/s 2024-05-20T16:13:48.425 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 59d1600a-16c2-11ef-bc9a-c7b262605968 -- bash -c 'ceph orch device ls' 2024-05-20T16:13:48.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:13:48 smithi082 ceph-mon[32238]: from='client.15291 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-20T16:13:48.635 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:13:48 smithi184 ceph-mon[36209]: from='client.15291 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-20T16:13:48.669 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T16:13:48.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:13:48 smithi160 ceph-mon[36347]: from='client.15291 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-20T16:13:49.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:13:49 smithi082 ceph-mon[32238]: pgmap v357: 129 pgs: 129 active+clean; 641 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s; 0 B/s, 0 objects/s recovering 2024-05-20T16:13:49.635 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:13:49 smithi184 ceph-mon[36209]: pgmap v357: 129 pgs: 129 active+clean; 641 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s; 0 B/s, 0 objects/s recovering 2024-05-20T16:13:49.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:13:49 smithi160 ceph-mon[36347]: pgmap v357: 129 pgs: 129 active+clean; 641 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s; 0 B/s, 0 objects/s recovering 2024-05-20T16:13:50.350 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:50.347+0000 7fe451bb1700 1 -- 172.21.15.82:0/1104847683 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe44c104780 msgr2=0x7fe44c104b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:50.351 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:50.347+0000 7fe451bb1700 1 --2- 172.21.15.82:0/1104847683 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe44c104780 0x7fe44c104b90 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fe434009990 tx=0x7fe434009ca0 comp rx=0 tx=0).stop 2024-05-20T16:13:50.351 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:50.347+0000 7fe451bb1700 1 -- 172.21.15.82:0/1104847683 shutdown_connections 2024-05-20T16:13:50.351 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:50.347+0000 7fe451bb1700 1 --2- 172.21.15.82:0/1104847683 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe44c101b20 0x7fe44c10b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:50.351 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:50.347+0000 7fe451bb1700 1 --2- 172.21.15.82:0/1104847683 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe44c104780 0x7fe44c104b90 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:50.351 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:50.347+0000 7fe451bb1700 1 --2- 172.21.15.82:0/1104847683 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe44c1011d0 0x7fe44c1015e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:50.351 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:50.347+0000 7fe451bb1700 1 -- 172.21.15.82:0/1104847683 >> 172.21.15.82:0/1104847683 conn(0x7fe44c0fcea0 msgr2=0x7fe44c0ff350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:13:50.351 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:50.348+0000 7fe451bb1700 1 -- 172.21.15.82:0/1104847683 shutdown_connections 2024-05-20T16:13:50.351 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:50.348+0000 7fe451bb1700 1 -- 172.21.15.82:0/1104847683 wait complete. 2024-05-20T16:13:50.351 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:50.348+0000 7fe451bb1700 1 Processor -- start 2024-05-20T16:13:50.351 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:50.348+0000 7fe451bb1700 1 -- start start 2024-05-20T16:13:50.351 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:50.349+0000 7fe451bb1700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe44c1011d0 0x7fe44c199860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:50.351 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:50.349+0000 7fe451bb1700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe44c101b20 0x7fe44c199da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:50.351 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:50.349+0000 7fe451bb1700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe44c104780 0x7fe44c1a0d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:50.352 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:50.349+0000 7fe451bb1700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fe44c10ed80 con 0x7fe44c1011d0 2024-05-20T16:13:50.352 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:50.349+0000 7fe451bb1700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fe44c10ec00 con 0x7fe44c104780 2024-05-20T16:13:50.352 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:50.349+0000 7fe451bb1700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fe44c10ef00 con 0x7fe44c101b20 2024-05-20T16:13:50.352 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:50.349+0000 7fe44b7fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe44c1011d0 0x7fe44c199860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:13:50.352 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:50.349+0000 7fe44b7fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe44c1011d0 0x7fe44c199860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:46422/0 (socket says 172.21.15.82:46422) 2024-05-20T16:13:50.352 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:50.349+0000 7fe44b7fe700 1 -- 172.21.15.82:0/2481570060 learned_addr learned my addr 172.21.15.82:0/2481570060 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:13:50.352 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:50.349+0000 7fe44bfff700 1 --2- 172.21.15.82:0/2481570060 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe44c104780 0x7fe44c1a0d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:13:50.352 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:50.349+0000 7fe44affd700 1 --2- 172.21.15.82:0/2481570060 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe44c101b20 0x7fe44c199da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:13:50.352 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:50.349+0000 7fe44b7fe700 1 -- 172.21.15.82:0/2481570060 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe44c101b20 msgr2=0x7fe44c199da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:50.352 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:50.349+0000 7fe44b7fe700 1 --2- 172.21.15.82:0/2481570060 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe44c101b20 0x7fe44c199da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:50.352 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:50.349+0000 7fe44b7fe700 1 -- 172.21.15.82:0/2481570060 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe44c104780 msgr2=0x7fe44c1a0d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:50.352 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:50.349+0000 7fe44b7fe700 1 --2- 172.21.15.82:0/2481570060 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe44c104780 0x7fe44c1a0d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:50.352 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:50.349+0000 7fe44b7fe700 1 -- 172.21.15.82:0/2481570060 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe44c1a1330 con 0x7fe44c1011d0 2024-05-20T16:13:50.353 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:50.349+0000 7fe44bfff700 1 --2- 172.21.15.82:0/2481570060 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe44c104780 0x7fe44c1a0d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T16:13:50.353 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:50.349+0000 7fe44b7fe700 1 --2- 172.21.15.82:0/2481570060 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe44c1011d0 0x7fe44c199860 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fe434006010 tx=0x7fe43400bc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:13:50.353 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:50.350+0000 7fe448ff9700 1 -- 172.21.15.82:0/2481570060 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe434026070 con 0x7fe44c1011d0 2024-05-20T16:13:50.353 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:50.350+0000 7fe451bb1700 1 -- 172.21.15.82:0/2481570060 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe44c1a15c0 con 0x7fe44c1011d0 2024-05-20T16:13:50.353 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:50.350+0000 7fe448ff9700 1 -- 172.21.15.82:0/2481570060 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe43400be40 con 0x7fe44c1011d0 2024-05-20T16:13:50.353 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:50.350+0000 7fe448ff9700 1 -- 172.21.15.82:0/2481570060 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe434021670 con 0x7fe44c1011d0 2024-05-20T16:13:50.353 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:50.350+0000 7fe451bb1700 1 -- 172.21.15.82:0/2481570060 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe44c1a1a10 con 0x7fe44c1011d0 2024-05-20T16:13:50.354 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:50.351+0000 7fe451bb1700 1 -- 172.21.15.82:0/2481570060 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe44c050170 con 0x7fe44c1011d0 2024-05-20T16:13:50.354 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:50.352+0000 7fe448ff9700 1 -- 172.21.15.82:0/2481570060 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fe434029070 con 0x7fe44c1011d0 2024-05-20T16:13:50.354 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:50.353+0000 7fe448ff9700 1 --2- 172.21.15.82:0/2481570060 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fe4380a2c20 0x7fe4380a5130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:50.354 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:50.353+0000 7fe44affd700 1 --2- 172.21.15.82:0/2481570060 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fe4380a2c20 0x7fe4380a5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:13:50.354 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:50.353+0000 7fe448ff9700 1 -- 172.21.15.82:0/2481570060 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9150+0+0 (secure 0 0 0) 0x7fe4340d07e0 con 0x7fe44c1011d0 2024-05-20T16:13:50.354 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:50.353+0000 7fe44affd700 1 --2- 172.21.15.82:0/2481570060 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fe4380a2c20 0x7fe4380a5130 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fe44c19a910 tx=0x7fe43c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:13:50.357 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:50.356+0000 7fe448ff9700 1 -- 172.21.15.82:0/2481570060 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fe434030080 con 0x7fe44c1011d0 2024-05-20T16:13:50.559 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:50.558+0000 7fe451bb1700 1 -- 172.21.15.82:0/2481570060 --> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7fe44c062a80 con 0x7fe4380a2c20 2024-05-20T16:13:50.561 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:50.560+0000 7fe448ff9700 1 -- 172.21.15.82:0/2481570060 <== mgr.14150 v2:172.21.15.82:6800/379530823 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7fe44c062a80 con 0x7fe4380a2c20 2024-05-20T16:13:50.561 INFO:teuthology.orchestra.run.smithi082.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-05-20T16:13:50.561 INFO:teuthology.orchestra.run.smithi082.stdout:smithi082 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400SD400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-05-20T16:13:50.561 INFO:teuthology.orchestra.run.smithi082.stdout:smithi160 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT623300JY400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-05-20T16:13:50.561 INFO:teuthology.orchestra.run.smithi082.stdout:smithi184 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT626500CE400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-05-20T16:13:50.563 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:50.563+0000 7fe451bb1700 1 -- 172.21.15.82:0/2481570060 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fe4380a2c20 msgr2=0x7fe4380a5130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:50.564 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:50.563+0000 7fe451bb1700 1 --2- 172.21.15.82:0/2481570060 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fe4380a2c20 0x7fe4380a5130 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fe44c19a910 tx=0x7fe43c009260 comp rx=0 tx=0).stop 2024-05-20T16:13:50.564 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:50.563+0000 7fe451bb1700 1 -- 172.21.15.82:0/2481570060 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe44c1011d0 msgr2=0x7fe44c199860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:50.564 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:50.563+0000 7fe451bb1700 1 --2- 172.21.15.82:0/2481570060 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe44c1011d0 0x7fe44c199860 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fe434006010 tx=0x7fe43400bc60 comp rx=0 tx=0).stop 2024-05-20T16:13:50.564 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:50.563+0000 7fe451bb1700 1 -- 172.21.15.82:0/2481570060 shutdown_connections 2024-05-20T16:13:50.564 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:50.563+0000 7fe451bb1700 1 --2- 172.21.15.82:0/2481570060 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fe4380a2c20 0x7fe4380a5130 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:50.564 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:50.563+0000 7fe451bb1700 1 --2- 172.21.15.82:0/2481570060 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe44c104780 0x7fe44c1a0d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:50.564 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:50.563+0000 7fe451bb1700 1 --2- 172.21.15.82:0/2481570060 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe44c1011d0 0x7fe44c199860 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:50.564 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:50.563+0000 7fe451bb1700 1 --2- 172.21.15.82:0/2481570060 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe44c101b20 0x7fe44c199da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:50.564 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:50.563+0000 7fe451bb1700 1 -- 172.21.15.82:0/2481570060 >> 172.21.15.82:0/2481570060 conn(0x7fe44c0fcea0 msgr2=0x7fe44c107720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:13:50.564 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:50.563+0000 7fe451bb1700 1 -- 172.21.15.82:0/2481570060 shutdown_connections 2024-05-20T16:13:50.564 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:50.563+0000 7fe451bb1700 1 -- 172.21.15.82:0/2481570060 wait complete. 2024-05-20T16:13:51.225 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-05-20T16:13:51.233 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi082.front.sepia.ceph.com 2024-05-20T16:13:51.233 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 59d1600a-16c2-11ef-bc9a-c7b262605968 -- bash -c 'ceph fs dump' 2024-05-20T16:13:51.479 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T16:13:51.601 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:13:51 smithi082 ceph-mon[32238]: from='client.15297 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-20T16:13:51.601 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:13:51 smithi082 ceph-mon[32238]: pgmap v358: 129 pgs: 129 active+clean; 641 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s; 13 B/s, 1 objects/s recovering 2024-05-20T16:13:51.635 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:13:51 smithi184 ceph-mon[36209]: from='client.15297 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-20T16:13:51.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:13:51 smithi184 ceph-mon[36209]: pgmap v358: 129 pgs: 129 active+clean; 641 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s; 13 B/s, 1 objects/s recovering 2024-05-20T16:13:51.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:13:51 smithi160 ceph-mon[36347]: from='client.15297 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-20T16:13:51.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:13:51 smithi160 ceph-mon[36347]: pgmap v358: 129 pgs: 129 active+clean; 641 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s; 13 B/s, 1 objects/s recovering 2024-05-20T16:13:53.124 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:53.124+0000 7f18fd694700 1 -- 172.21.15.82:0/4137465307 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f18f8103330 msgr2=0x7f18f8109be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:53.124 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:53.124+0000 7f18fd694700 1 --2- 172.21.15.82:0/4137465307 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f18f8103330 0x7f18f8109be0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f18f000b0a0 tx=0x7f18f000b3b0 comp rx=0 tx=0).stop 2024-05-20T16:13:53.124 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:53.124+0000 7f18fd694700 1 -- 172.21.15.82:0/4137465307 shutdown_connections 2024-05-20T16:13:53.124 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:53.124+0000 7f18fd694700 1 --2- 172.21.15.82:0/4137465307 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f18f8101860 0x7f18f8101c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:53.124 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:53.124+0000 7f18fd694700 1 --2- 172.21.15.82:0/4137465307 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f18f8103330 0x7f18f8109be0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:53.124 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:53.124+0000 7f18fd694700 1 --2- 172.21.15.82:0/4137465307 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f18f8102990 0x7f18f8102da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:53.124 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:53.124+0000 7f18fd694700 1 -- 172.21.15.82:0/4137465307 >> 172.21.15.82:0/4137465307 conn(0x7f18f80fcea0 msgr2=0x7f18f80ff350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:13:53.124 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:53.124+0000 7f18fd694700 1 -- 172.21.15.82:0/4137465307 shutdown_connections 2024-05-20T16:13:53.124 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:53.124+0000 7f18fd694700 1 -- 172.21.15.82:0/4137465307 wait complete. 2024-05-20T16:13:53.126 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:53.125+0000 7f18fd694700 1 Processor -- start 2024-05-20T16:13:53.126 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:53.125+0000 7f18fd694700 1 -- start start 2024-05-20T16:13:53.126 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:53.125+0000 7f18fd694700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f18f8101860 0x7f18f80fecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:53.126 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:53.125+0000 7f18fd694700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f18f8102990 0x7f18f819e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:53.126 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:53.125+0000 7f18fd694700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f18f8103330 0x7f18f81a0ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:53.126 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:53.125+0000 7f18fd694700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f18f810d510 con 0x7f18f8103330 2024-05-20T16:13:53.126 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:53.125+0000 7f18fd694700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f18f810d390 con 0x7f18f8102990 2024-05-20T16:13:53.127 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:53.125+0000 7f18fd694700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f18f810d690 con 0x7f18f8101860 2024-05-20T16:13:53.127 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:53.125+0000 7f18f77fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f18f8103330 0x7f18f81a0ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:13:53.127 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:53.125+0000 7f18f77fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f18f8103330 0x7f18f81a0ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:46442/0 (socket says 172.21.15.82:46442) 2024-05-20T16:13:53.127 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:53.125+0000 7f18f77fe700 1 -- 172.21.15.82:0/2265269965 learned_addr learned my addr 172.21.15.82:0/2265269965 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:13:53.127 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:53.125+0000 7f18f6ffd700 1 --2- 172.21.15.82:0/2265269965 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f18f8101860 0x7f18f80fecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:13:53.127 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:53.125+0000 7f18f67fc700 1 --2- 172.21.15.82:0/2265269965 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f18f8102990 0x7f18f819e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:13:53.127 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:53.125+0000 7f18f77fe700 1 -- 172.21.15.82:0/2265269965 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f18f8101860 msgr2=0x7f18f80fecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:53.127 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:53.125+0000 7f18f77fe700 1 --2- 172.21.15.82:0/2265269965 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f18f8101860 0x7f18f80fecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:53.127 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:53.125+0000 7f18f77fe700 1 -- 172.21.15.82:0/2265269965 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f18f8102990 msgr2=0x7f18f819e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:53.127 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:53.125+0000 7f18f77fe700 1 --2- 172.21.15.82:0/2265269965 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f18f8102990 0x7f18f819e710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:53.127 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:53.126+0000 7f18f77fe700 1 -- 172.21.15.82:0/2265269965 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18f81a14d0 con 0x7f18f8103330 2024-05-20T16:13:53.127 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:53.126+0000 7f18f67fc700 1 --2- 172.21.15.82:0/2265269965 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f18f8102990 0x7f18f819e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T16:13:53.127 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:53.126+0000 7f18f77fe700 1 --2- 172.21.15.82:0/2265269965 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f18f8103330 0x7f18f81a0ca0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f18f0009bb0 tx=0x7f18f0004e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:13:53.127 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:53.126+0000 7f18dffff700 1 -- 172.21.15.82:0/2265269965 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18f0021070 con 0x7f18f8103330 2024-05-20T16:13:53.128 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:53.126+0000 7f18dffff700 1 -- 172.21.15.82:0/2265269965 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f18f0007d00 con 0x7f18f8103330 2024-05-20T16:13:53.128 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:53.126+0000 7f18fd694700 1 -- 172.21.15.82:0/2265269965 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18f81a1760 con 0x7f18f8103330 2024-05-20T16:13:53.128 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:53.126+0000 7f18dffff700 1 -- 172.21.15.82:0/2265269965 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18f001d910 con 0x7f18f8103330 2024-05-20T16:13:53.128 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:53.126+0000 7f18fd694700 1 -- 172.21.15.82:0/2265269965 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18f81a1ce0 con 0x7f18f8103330 2024-05-20T16:13:53.128 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:53.127+0000 7f18fd694700 1 -- 172.21.15.82:0/2265269965 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18f8050170 con 0x7f18f8103330 2024-05-20T16:13:53.128 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:53.129+0000 7f18dffff700 1 -- 172.21.15.82:0/2265269965 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f18f0024070 con 0x7f18f8103330 2024-05-20T16:13:53.129 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:53.129+0000 7f18dffff700 1 --2- 172.21.15.82:0/2265269965 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f18e00a2ca0 0x7f18e00a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:53.129 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:53.130+0000 7f18f6ffd700 1 --2- 172.21.15.82:0/2265269965 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f18e00a2ca0 0x7f18e00a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:13:53.129 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:53.130+0000 7f18dffff700 1 -- 172.21.15.82:0/2265269965 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9150+0+0 (secure 0 0 0) 0x7f18f000f020 con 0x7f18f8103330 2024-05-20T16:13:53.129 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:53.130+0000 7f18f6ffd700 1 --2- 172.21.15.82:0/2265269965 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f18e00a2ca0 0x7f18e00a51b0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f18f8102890 tx=0x7f18e8009190 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:13:53.133 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:53.133+0000 7f18dffff700 1 -- 172.21.15.82:0/2265269965 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f18f002b020 con 0x7f18f8103330 2024-05-20T16:13:53.373 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:53.373+0000 7f18fd694700 1 -- 172.21.15.82:0/2265269965 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f18f802dc60 con 0x7f18f8103330 2024-05-20T16:13:53.373 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:53.373+0000 7f18dffff700 1 -- 172.21.15.82:0/2265269965 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 24 v24) ==== 76+0+3056 (secure 0 0 0) 0x7f18f009dbd0 con 0x7f18f8103330 2024-05-20T16:13:53.373 INFO:teuthology.orchestra.run.smithi082.stdout:e24 2024-05-20T16:13:53.374 INFO:teuthology.orchestra.run.smithi082.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-20T16:13:53.374 INFO:teuthology.orchestra.run.smithi082.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-05-20T16:13:53.374 INFO:teuthology.orchestra.run.smithi082.stdout:legacy client fscid: 1 2024-05-20T16:13:53.374 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-05-20T16:13:53.374 INFO:teuthology.orchestra.run.smithi082.stdout:Filesystem 'cephfs' (1) 2024-05-20T16:13:53.374 INFO:teuthology.orchestra.run.smithi082.stdout:fs_name cephfs 2024-05-20T16:13:53.374 INFO:teuthology.orchestra.run.smithi082.stdout:epoch 24 2024-05-20T16:13:53.374 INFO:teuthology.orchestra.run.smithi082.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-05-20T16:13:53.374 INFO:teuthology.orchestra.run.smithi082.stdout:created 2024-05-20T16:11:46.751632+0000 2024-05-20T16:13:53.374 INFO:teuthology.orchestra.run.smithi082.stdout:modified 2024-05-20T16:12:13.394770+0000 2024-05-20T16:13:53.374 INFO:teuthology.orchestra.run.smithi082.stdout:tableserver 0 2024-05-20T16:13:53.375 INFO:teuthology.orchestra.run.smithi082.stdout:root 0 2024-05-20T16:13:53.375 INFO:teuthology.orchestra.run.smithi082.stdout:session_timeout 300 2024-05-20T16:13:53.375 INFO:teuthology.orchestra.run.smithi082.stdout:session_autoclose 300 2024-05-20T16:13:53.375 INFO:teuthology.orchestra.run.smithi082.stdout:max_file_size 1099511627776 2024-05-20T16:13:53.375 INFO:teuthology.orchestra.run.smithi082.stdout:max_xattr_size 65536 2024-05-20T16:13:53.375 INFO:teuthology.orchestra.run.smithi082.stdout:required_client_features {} 2024-05-20T16:13:53.375 INFO:teuthology.orchestra.run.smithi082.stdout:last_failure 0 2024-05-20T16:13:53.375 INFO:teuthology.orchestra.run.smithi082.stdout:last_failure_osd_epoch 0 2024-05-20T16:13:53.375 INFO:teuthology.orchestra.run.smithi082.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-05-20T16:13:53.375 INFO:teuthology.orchestra.run.smithi082.stdout:max_mds 3 2024-05-20T16:13:53.375 INFO:teuthology.orchestra.run.smithi082.stdout:in 0,1,2 2024-05-20T16:13:53.375 INFO:teuthology.orchestra.run.smithi082.stdout:up {0=24548,1=24511,2=24506} 2024-05-20T16:13:53.375 INFO:teuthology.orchestra.run.smithi082.stdout:failed 2024-05-20T16:13:53.376 INFO:teuthology.orchestra.run.smithi082.stdout:damaged 2024-05-20T16:13:53.376 INFO:teuthology.orchestra.run.smithi082.stdout:stopped 2024-05-20T16:13:53.376 INFO:teuthology.orchestra.run.smithi082.stdout:data_pools [3,4] 2024-05-20T16:13:53.376 INFO:teuthology.orchestra.run.smithi082.stdout:metadata_pool 2 2024-05-20T16:13:53.376 INFO:teuthology.orchestra.run.smithi082.stdout:inline_data disabled 2024-05-20T16:13:53.376 INFO:teuthology.orchestra.run.smithi082.stdout:balancer 2024-05-20T16:13:53.376 INFO:teuthology.orchestra.run.smithi082.stdout:bal_rank_mask -1 2024-05-20T16:13:53.376 INFO:teuthology.orchestra.run.smithi082.stdout:standby_count_wanted 0 2024-05-20T16:13:53.376 INFO:teuthology.orchestra.run.smithi082.stdout:qdb_cluster leader: 24548 members: 24511,24506,24548 2024-05-20T16:13:53.376 INFO:teuthology.orchestra.run.smithi082.stdout:[mds.b{0:24548} state up:active seq 3 addr [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] compat {c=[1],r=[1],i=[1fff]}] 2024-05-20T16:13:53.376 INFO:teuthology.orchestra.run.smithi082.stdout:[mds.c{0:24496} state up:standby-replay seq 1 addr [v2:172.21.15.184:6836/403262242,v1:172.21.15.184:6837/403262242] compat {c=[1],r=[1],i=[1fff]}] 2024-05-20T16:13:53.376 INFO:teuthology.orchestra.run.smithi082.stdout:[mds.i{1:24511} state up:active seq 11 addr [v2:172.21.15.184:6838/2199955067,v1:172.21.15.184:6839/2199955067] compat {c=[1],r=[1],i=[1fff]}] 2024-05-20T16:13:53.376 INFO:teuthology.orchestra.run.smithi082.stdout:[mds.f{1:24478} state up:standby-replay seq 1 addr [v2:172.21.15.184:6834/4242156090,v1:172.21.15.184:6835/4242156090] compat {c=[1],r=[1],i=[1fff]}] 2024-05-20T16:13:53.377 INFO:teuthology.orchestra.run.smithi082.stdout:[mds.h{2:24506} state up:active seq 12 addr [v2:172.21.15.160:6836/2846188680,v1:172.21.15.160:6837/2846188680] compat {c=[1],r=[1],i=[1fff]}] 2024-05-20T16:13:53.377 INFO:teuthology.orchestra.run.smithi082.stdout:[mds.e{2:24470} state up:standby-replay seq 1 addr [v2:172.21.15.160:6834/664068363,v1:172.21.15.160:6835/664068363] compat {c=[1],r=[1],i=[1fff]}] 2024-05-20T16:13:53.377 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-05-20T16:13:53.377 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-05-20T16:13:53.377 INFO:teuthology.orchestra.run.smithi082.stdout:Standby daemons: 2024-05-20T16:13:53.377 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-05-20T16:13:53.377 INFO:teuthology.orchestra.run.smithi082.stdout:[mds.j{-1:14538} state up:standby seq 1 addr [v2:172.21.15.82:6834/2044534217,v1:172.21.15.82:6835/2044534217] compat {c=[1],r=[1],i=[1fff]}] 2024-05-20T16:13:53.377 INFO:teuthology.orchestra.run.smithi082.stdout:[mds.d{-1:14550} state up:standby seq 1 addr [v2:172.21.15.82:6836/2635635097,v1:172.21.15.82:6837/2635635097] compat {c=[1],r=[1],i=[1fff]}] 2024-05-20T16:13:53.377 INFO:teuthology.orchestra.run.smithi082.stdout:[mds.g{-1:14625} state up:standby seq 1 addr [v2:172.21.15.82:6838/2118233541,v1:172.21.15.82:6839/2118233541] compat {c=[1],r=[1],i=[1fff]}] 2024-05-20T16:13:53.377 INFO:teuthology.orchestra.run.smithi082.stdout:[mds.a{-1:14637} state up:standby seq 1 addr [v2:172.21.15.82:6840/2119517973,v1:172.21.15.82:6841/2119517973] compat {c=[1],r=[1],i=[1fff]}] 2024-05-20T16:13:53.377 INFO:teuthology.orchestra.run.smithi082.stdout:[mds.l{-1:24430} state up:standby seq 1 addr [v2:172.21.15.184:6832/4015144007,v1:172.21.15.184:6833/4015144007] compat {c=[1],r=[1],i=[1fff]}] 2024-05-20T16:13:53.377 INFO:teuthology.orchestra.run.smithi082.stdout:[mds.k{-1:24434} state up:standby seq 1 addr [v2:172.21.15.160:6832/975243372,v1:172.21.15.160:6833/975243372] compat {c=[1],r=[1],i=[1fff]}] 2024-05-20T16:13:53.378 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:53.376+0000 7f18fd694700 1 -- 172.21.15.82:0/2265269965 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f18e00a2ca0 msgr2=0x7f18e00a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:53.378 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:53.376+0000 7f18fd694700 1 --2- 172.21.15.82:0/2265269965 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f18e00a2ca0 0x7f18e00a51b0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f18f8102890 tx=0x7f18e8009190 comp rx=0 tx=0).stop 2024-05-20T16:13:53.378 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:53.376+0000 7f18fd694700 1 -- 172.21.15.82:0/2265269965 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f18f8103330 msgr2=0x7f18f81a0ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:53.378 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:53.376+0000 7f18fd694700 1 --2- 172.21.15.82:0/2265269965 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f18f8103330 0x7f18f81a0ca0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f18f0009bb0 tx=0x7f18f0004e20 comp rx=0 tx=0).stop 2024-05-20T16:13:53.378 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:53.376+0000 7f18fd694700 1 -- 172.21.15.82:0/2265269965 shutdown_connections 2024-05-20T16:13:53.378 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:53.376+0000 7f18fd694700 1 --2- 172.21.15.82:0/2265269965 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f18e00a2ca0 0x7f18e00a51b0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:53.378 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:53.376+0000 7f18fd694700 1 --2- 172.21.15.82:0/2265269965 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f18f8103330 0x7f18f81a0ca0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:53.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:53.376+0000 7f18fd694700 1 --2- 172.21.15.82:0/2265269965 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f18f8101860 0x7f18f80fecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:53.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:53.376+0000 7f18fd694700 1 --2- 172.21.15.82:0/2265269965 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f18f8102990 0x7f18f819e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:53.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:53.376+0000 7f18fd694700 1 -- 172.21.15.82:0/2265269965 >> 172.21.15.82:0/2265269965 conn(0x7f18f80fcea0 msgr2=0x7f18f81065f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:13:53.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:53.376+0000 7f18fd694700 1 -- 172.21.15.82:0/2265269965 shutdown_connections 2024-05-20T16:13:53.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:53.376+0000 7f18fd694700 1 -- 172.21.15.82:0/2265269965 wait complete. 2024-05-20T16:13:53.379 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 24 2024-05-20T16:13:53.635 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:13:53 smithi184 ceph-mon[36209]: pgmap v359: 129 pgs: 129 active+clean; 641 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s; 13 B/s, 1 objects/s recovering 2024-05-20T16:13:53.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:13:53 smithi184 ceph-mon[36209]: from='client.? 172.21.15.82:0/2265269965' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-20T16:13:53.663 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:13:53 smithi082 ceph-mon[32238]: pgmap v359: 129 pgs: 129 active+clean; 641 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s; 13 B/s, 1 objects/s recovering 2024-05-20T16:13:53.663 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:13:53 smithi082 ceph-mon[32238]: from='client.? 172.21.15.82:0/2265269965' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-20T16:13:53.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:13:53 smithi160 ceph-mon[36347]: pgmap v359: 129 pgs: 129 active+clean; 641 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s; 13 B/s, 1 objects/s recovering 2024-05-20T16:13:53.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:13:53 smithi160 ceph-mon[36347]: from='client.? 172.21.15.82:0/2265269965' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-20T16:13:54.024 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 59d1600a-16c2-11ef-bc9a-c7b262605968 -- bash -c 'ceph osd dump' 2024-05-20T16:13:54.265 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T16:13:55.635 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:13:55 smithi184 ceph-mon[36209]: pgmap v360: 129 pgs: 129 active+clean; 641 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s; 11 B/s, 0 objects/s recovering 2024-05-20T16:13:55.720 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:13:55 smithi082 ceph-mon[32238]: pgmap v360: 129 pgs: 129 active+clean; 641 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s; 11 B/s, 0 objects/s recovering 2024-05-20T16:13:55.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:13:55 smithi160 ceph-mon[36347]: pgmap v360: 129 pgs: 129 active+clean; 641 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s; 11 B/s, 0 objects/s recovering 2024-05-20T16:13:56.057 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:56.057+0000 7f3d38c0d700 1 -- 172.21.15.82:0/2223428619 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3d34104cc0 msgr2=0x7f3d3410b570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:56.057 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:56.057+0000 7f3d38c0d700 1 --2- 172.21.15.82:0/2223428619 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3d34104cc0 0x7f3d3410b570 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f3d2c00b0a0 tx=0x7f3d2c00b3b0 comp rx=0 tx=0).stop 2024-05-20T16:13:56.057 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:56.057+0000 7f3d38c0d700 1 -- 172.21.15.82:0/2223428619 shutdown_connections 2024-05-20T16:13:56.057 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:56.057+0000 7f3d38c0d700 1 --2- 172.21.15.82:0/2223428619 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3d341031f0 0x7f3d34103600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:56.057 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:56.057+0000 7f3d38c0d700 1 --2- 172.21.15.82:0/2223428619 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3d34104cc0 0x7f3d3410b570 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:56.057 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:56.057+0000 7f3d38c0d700 1 --2- 172.21.15.82:0/2223428619 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3d34104320 0x7f3d34104730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:56.058 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:56.057+0000 7f3d38c0d700 1 -- 172.21.15.82:0/2223428619 >> 172.21.15.82:0/2223428619 conn(0x7f3d340fe850 msgr2=0x7f3d34100ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:13:56.058 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:56.057+0000 7f3d38c0d700 1 -- 172.21.15.82:0/2223428619 shutdown_connections 2024-05-20T16:13:56.058 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:56.057+0000 7f3d38c0d700 1 -- 172.21.15.82:0/2223428619 wait complete. 2024-05-20T16:13:56.060 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:56.058+0000 7f3d38c0d700 1 Processor -- start 2024-05-20T16:13:56.060 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:56.058+0000 7f3d38c0d700 1 -- start start 2024-05-20T16:13:56.060 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:56.058+0000 7f3d38c0d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3d341031f0 0x7f3d34199880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:56.060 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:56.058+0000 7f3d38c0d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3d34104320 0x7f3d34199dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:56.060 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:56.058+0000 7f3d38c0d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3d34104cc0 0x7f3d341a0d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:56.060 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:56.058+0000 7f3d38c0d700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f3d3410ee20 con 0x7f3d34104320 2024-05-20T16:13:56.060 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:56.058+0000 7f3d38c0d700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f3d3410eca0 con 0x7f3d34104cc0 2024-05-20T16:13:56.060 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:56.058+0000 7f3d38c0d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f3d3410efa0 con 0x7f3d341031f0 2024-05-20T16:13:56.060 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:56.058+0000 7f3d2bfff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3d34104320 0x7f3d34199dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:13:56.060 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:56.058+0000 7f3d2bfff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3d34104320 0x7f3d34199dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:46446/0 (socket says 172.21.15.82:46446) 2024-05-20T16:13:56.060 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:56.058+0000 7f3d2bfff700 1 -- 172.21.15.82:0/3801769361 learned_addr learned my addr 172.21.15.82:0/3801769361 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:13:56.061 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:56.058+0000 7f3d3259c700 1 --2- 172.21.15.82:0/3801769361 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3d341031f0 0x7f3d34199880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:13:56.061 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:56.058+0000 7f3d32d9d700 1 --2- 172.21.15.82:0/3801769361 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3d34104cc0 0x7f3d341a0d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:13:56.061 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:56.059+0000 7f3d2bfff700 1 -- 172.21.15.82:0/3801769361 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3d341031f0 msgr2=0x7f3d34199880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:56.061 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:56.059+0000 7f3d2bfff700 1 --2- 172.21.15.82:0/3801769361 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3d341031f0 0x7f3d34199880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:56.061 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:56.059+0000 7f3d2bfff700 1 -- 172.21.15.82:0/3801769361 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3d34104cc0 msgr2=0x7f3d341a0d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:56.061 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:56.059+0000 7f3d2bfff700 1 --2- 172.21.15.82:0/3801769361 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3d34104cc0 0x7f3d341a0d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:56.061 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:56.059+0000 7f3d2bfff700 1 -- 172.21.15.82:0/3801769361 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d341a13e0 con 0x7f3d34104320 2024-05-20T16:13:56.061 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:56.059+0000 7f3d2bfff700 1 --2- 172.21.15.82:0/3801769361 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3d34104320 0x7f3d34199dc0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f3d1c00d950 tx=0x7f3d1c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:13:56.061 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:56.059+0000 7f3d2b7fe700 1 -- 172.21.15.82:0/3801769361 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d1c016070 con 0x7f3d34104320 2024-05-20T16:13:56.061 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:56.059+0000 7f3d38c0d700 1 -- 172.21.15.82:0/3801769361 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d341a16d0 con 0x7f3d34104320 2024-05-20T16:13:56.061 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:56.059+0000 7f3d2b7fe700 1 -- 172.21.15.82:0/3801769361 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3d1c00fd70 con 0x7f3d34104320 2024-05-20T16:13:56.061 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:56.059+0000 7f3d2b7fe700 1 -- 172.21.15.82:0/3801769361 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d1c00e610 con 0x7f3d34104320 2024-05-20T16:13:56.061 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:56.059+0000 7f3d38c0d700 1 -- 172.21.15.82:0/3801769361 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d341a1ce0 con 0x7f3d34104320 2024-05-20T16:13:56.061 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:56.060+0000 7f3d38c0d700 1 -- 172.21.15.82:0/3801769361 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d34050170 con 0x7f3d34104320 2024-05-20T16:13:56.062 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:56.061+0000 7f3d2b7fe700 1 -- 172.21.15.82:0/3801769361 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f3d1c019070 con 0x7f3d34104320 2024-05-20T16:13:56.062 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:56.062+0000 7f3d2b7fe700 1 --2- 172.21.15.82:0/3801769361 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f3d200a2c20 0x7f3d200a5130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:56.062 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:56.062+0000 7f3d3259c700 1 --2- 172.21.15.82:0/3801769361 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f3d200a2c20 0x7f3d200a5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:13:56.062 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:56.062+0000 7f3d2b7fe700 1 -- 172.21.15.82:0/3801769361 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9150+0+0 (secure 0 0 0) 0x7f3d1c0cee60 con 0x7f3d34104320 2024-05-20T16:13:56.063 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:56.062+0000 7f3d3259c700 1 --2- 172.21.15.82:0/3801769361 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f3d200a2c20 0x7f3d200a5130 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f3d34104220 tx=0x7f3d24009190 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:13:56.065 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:56.066+0000 7f3d2b7fe700 1 -- 172.21.15.82:0/3801769361 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f3d1c09ba30 con 0x7f3d34104320 2024-05-20T16:13:56.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:56.265+0000 7f3d38c0d700 1 -- 172.21.15.82:0/3801769361 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7f3d34063250 con 0x7f3d34104320 2024-05-20T16:13:56.265 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:56.266+0000 7f3d2b7fe700 1 -- 172.21.15.82:0/3801769361 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v95) ==== 56+0+5660 (secure 0 0 0) 0x7f3d1c09b330 con 0x7f3d34104320 2024-05-20T16:13:56.265 INFO:teuthology.orchestra.run.smithi082.stdout:epoch 95 2024-05-20T16:13:56.266 INFO:teuthology.orchestra.run.smithi082.stdout:fsid 59d1600a-16c2-11ef-bc9a-c7b262605968 2024-05-20T16:13:56.266 INFO:teuthology.orchestra.run.smithi082.stdout:created 2024-05-20T16:03:46.611556+0000 2024-05-20T16:13:56.266 INFO:teuthology.orchestra.run.smithi082.stdout:modified 2024-05-20T16:13:46.328150+0000 2024-05-20T16:13:56.266 INFO:teuthology.orchestra.run.smithi082.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-05-20T16:13:56.266 INFO:teuthology.orchestra.run.smithi082.stdout:crush_version 27 2024-05-20T16:13:56.266 INFO:teuthology.orchestra.run.smithi082.stdout:full_ratio 0.95 2024-05-20T16:13:56.266 INFO:teuthology.orchestra.run.smithi082.stdout:backfillfull_ratio 0.9 2024-05-20T16:13:56.266 INFO:teuthology.orchestra.run.smithi082.stdout:nearfull_ratio 0.85 2024-05-20T16:13:56.266 INFO:teuthology.orchestra.run.smithi082.stdout:require_min_compat_client luminous 2024-05-20T16:13:56.266 INFO:teuthology.orchestra.run.smithi082.stdout:min_compat_client luminous 2024-05-20T16:13:56.266 INFO:teuthology.orchestra.run.smithi082.stdout:require_osd_release squid 2024-05-20T16:13:56.267 INFO:teuthology.orchestra.run.smithi082.stdout:stretch_mode_enabled false 2024-05-20T16:13:56.267 INFO:teuthology.orchestra.run.smithi082.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 22 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-05-20T16:13:56.267 INFO:teuthology.orchestra.run.smithi082.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 85 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-05-20T16:13:56.267 INFO:teuthology.orchestra.run.smithi082.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 86 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.05 application cephfs read_balance_score 1.50 2024-05-20T16:13:56.267 INFO:teuthology.orchestra.run.smithi082.stdout:pool 4 'cephfs_data_ec' erasure profile cephfs_data_ec size 4 min_size 3 crush_rule 1 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 91 flags hashpspool,ec_overwrites stripe_width 8192 pg_num_min 64 target_size_ratio 0.9 application cephfs 2024-05-20T16:13:56.267 INFO:teuthology.orchestra.run.smithi082.stdout:max_osd 12 2024-05-20T16:13:56.267 INFO:teuthology.orchestra.run.smithi082.stdout:osd.0 up in weight 1 up_from 8 up_thru 92 down_at 0 last_clean_interval [0,0) [v2:172.21.15.82:6802/1484330167,v1:172.21.15.82:6803/1484330167] [v2:172.21.15.82:6804/1484330167,v1:172.21.15.82:6805/1484330167] exists,up bae842a9-03ec-490e-b56d-f7957235cdc8 2024-05-20T16:13:56.267 INFO:teuthology.orchestra.run.smithi082.stdout:osd.1 up in weight 1 up_from 13 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.160:6800/1457003568,v1:172.21.15.160:6801/1457003568] [v2:172.21.15.160:6802/1457003568,v1:172.21.15.160:6803/1457003568] exists,up 59f6a2ba-b5de-4161-81c5-92a417abacdf 2024-05-20T16:13:56.267 INFO:teuthology.orchestra.run.smithi082.stdout:osd.2 up in weight 1 up_from 18 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.184:6800/4051293934,v1:172.21.15.184:6801/4051293934] [v2:172.21.15.184:6802/4051293934,v1:172.21.15.184:6803/4051293934] exists,up 3bbb5587-b449-41f3-bd26-67ec3d6afdfc 2024-05-20T16:13:56.267 INFO:teuthology.orchestra.run.smithi082.stdout:osd.3 up in weight 1 up_from 26 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.82:6810/3947485599,v1:172.21.15.82:6811/3947485599] [v2:172.21.15.82:6812/3947485599,v1:172.21.15.82:6813/3947485599] exists,up ca651f90-09b8-4081-a6f9-40b31268a837 2024-05-20T16:13:56.267 INFO:teuthology.orchestra.run.smithi082.stdout:osd.4 up in weight 1 up_from 34 up_thru 92 down_at 0 last_clean_interval [0,0) [v2:172.21.15.160:6808/4160830014,v1:172.21.15.160:6809/4160830014] [v2:172.21.15.160:6810/4160830014,v1:172.21.15.160:6811/4160830014] exists,up cb3dfd56-f677-40cb-9c96-8319ea5ac788 2024-05-20T16:13:56.267 INFO:teuthology.orchestra.run.smithi082.stdout:osd.5 up in weight 1 up_from 39 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.184:6808/1046282672,v1:172.21.15.184:6809/1046282672] [v2:172.21.15.184:6810/1046282672,v1:172.21.15.184:6811/1046282672] exists,up c70cfcd1-a4ff-461f-8c73-479b061200fb 2024-05-20T16:13:56.268 INFO:teuthology.orchestra.run.smithi082.stdout:osd.6 up in weight 1 up_from 47 up_thru 94 down_at 0 last_clean_interval [0,0) [v2:172.21.15.82:6818/2789736499,v1:172.21.15.82:6819/2789736499] [v2:172.21.15.82:6820/2789736499,v1:172.21.15.82:6821/2789736499] exists,up d47ce001-9dd1-4830-bcc9-888601187c6d 2024-05-20T16:13:56.268 INFO:teuthology.orchestra.run.smithi082.stdout:osd.7 up in weight 1 up_from 52 up_thru 92 down_at 0 last_clean_interval [0,0) [v2:172.21.15.160:6816/2340683378,v1:172.21.15.160:6817/2340683378] [v2:172.21.15.160:6818/2340683378,v1:172.21.15.160:6819/2340683378] exists,up 25c54f43-18cb-41e4-83b4-2732b5a93889 2024-05-20T16:13:56.268 INFO:teuthology.orchestra.run.smithi082.stdout:osd.8 up in weight 1 up_from 60 up_thru 92 down_at 0 last_clean_interval [0,0) [v2:172.21.15.184:6816/3831576372,v1:172.21.15.184:6817/3831576372] [v2:172.21.15.184:6818/3831576372,v1:172.21.15.184:6819/3831576372] exists,up 1f8b478d-287f-45f5-8526-d1a58c5bc07c 2024-05-20T16:13:56.268 INFO:teuthology.orchestra.run.smithi082.stdout:osd.9 up in weight 1 up_from 68 up_thru 92 down_at 0 last_clean_interval [0,0) [v2:172.21.15.82:6826/716330782,v1:172.21.15.82:6827/716330782] [v2:172.21.15.82:6828/716330782,v1:172.21.15.82:6829/716330782] exists,up 0d69e319-954f-4b18-95ea-31794f18dc3a 2024-05-20T16:13:56.268 INFO:teuthology.orchestra.run.smithi082.stdout:osd.10 up in weight 1 up_from 73 up_thru 92 down_at 0 last_clean_interval [0,0) [v2:172.21.15.160:6824/2327428014,v1:172.21.15.160:6825/2327428014] [v2:172.21.15.160:6826/2327428014,v1:172.21.15.160:6827/2327428014] exists,up 39dc1b78-61cb-42ca-b5b6-1e44ac2b6500 2024-05-20T16:13:56.268 INFO:teuthology.orchestra.run.smithi082.stdout:osd.11 up in weight 1 up_from 81 up_thru 94 down_at 0 last_clean_interval [0,0) [v2:172.21.15.184:6824/3594364650,v1:172.21.15.184:6825/3594364650] [v2:172.21.15.184:6826/3594364650,v1:172.21.15.184:6827/3594364650] exists,up 4306b3c1-707a-4010-987c-209ea0e948e3 2024-05-20T16:13:56.268 INFO:teuthology.orchestra.run.smithi082.stdout:pg_upmap_items 2.6 [8,11] 2024-05-20T16:13:56.268 INFO:teuthology.orchestra.run.smithi082.stdout:pg_upmap_items 2.e [2,1] 2024-05-20T16:13:56.268 INFO:teuthology.orchestra.run.smithi082.stdout:pg_upmap_items 2.10 [8,11] 2024-05-20T16:13:56.268 INFO:teuthology.orchestra.run.smithi082.stdout:pg_upmap_items 2.1d [2,4] 2024-05-20T16:13:56.268 INFO:teuthology.orchestra.run.smithi082.stdout:pg_upmap_items 3.2 [3,9] 2024-05-20T16:13:56.268 INFO:teuthology.orchestra.run.smithi082.stdout:pg_upmap_items 3.3 [4,9] 2024-05-20T16:13:56.268 INFO:teuthology.orchestra.run.smithi082.stdout:pg_upmap_items 3.17 [4,8] 2024-05-20T16:13:56.268 INFO:teuthology.orchestra.run.smithi082.stdout:pg_upmap_items 3.1d [5,9] 2024-05-20T16:13:56.268 INFO:teuthology.orchestra.run.smithi082.stdout:pg_upmap_items 4.2 [9,1] 2024-05-20T16:13:56.269 INFO:teuthology.orchestra.run.smithi082.stdout:pg_upmap_items 4.5 [9,10] 2024-05-20T16:13:56.269 INFO:teuthology.orchestra.run.smithi082.stdout:pg_upmap_items 4.e [5,8] 2024-05-20T16:13:56.269 INFO:teuthology.orchestra.run.smithi082.stdout:pg_upmap_items 4.1b [9,8] 2024-05-20T16:13:56.269 INFO:teuthology.orchestra.run.smithi082.stdout:blocklist 172.21.15.82:0/2005424509 expires 2024-05-21T16:04:45.161949+0000 2024-05-20T16:13:56.269 INFO:teuthology.orchestra.run.smithi082.stdout:blocklist 172.21.15.82:6801/1763049308 expires 2024-05-21T16:04:45.161949+0000 2024-05-20T16:13:56.269 INFO:teuthology.orchestra.run.smithi082.stdout:blocklist 172.21.15.82:6801/1959459348 expires 2024-05-21T16:04:19.208291+0000 2024-05-20T16:13:56.269 INFO:teuthology.orchestra.run.smithi082.stdout:blocklist 172.21.15.82:0/3131904556 expires 2024-05-21T16:04:19.208291+0000 2024-05-20T16:13:56.269 INFO:teuthology.orchestra.run.smithi082.stdout:blocklist 172.21.15.82:6800/1959459348 expires 2024-05-21T16:04:19.208291+0000 2024-05-20T16:13:56.269 INFO:teuthology.orchestra.run.smithi082.stdout:blocklist 172.21.15.82:0/2456966342 expires 2024-05-21T16:04:45.161949+0000 2024-05-20T16:13:56.269 INFO:teuthology.orchestra.run.smithi082.stdout:blocklist 172.21.15.82:0/4179978833 expires 2024-05-21T16:04:19.208291+0000 2024-05-20T16:13:56.269 INFO:teuthology.orchestra.run.smithi082.stdout:blocklist 172.21.15.82:0/1602862225 expires 2024-05-21T16:04:45.161949+0000 2024-05-20T16:13:56.269 INFO:teuthology.orchestra.run.smithi082.stdout:blocklist 172.21.15.82:0/106101180 expires 2024-05-21T16:04:19.208291+0000 2024-05-20T16:13:56.269 INFO:teuthology.orchestra.run.smithi082.stdout:blocklist 172.21.15.82:6800/1763049308 expires 2024-05-21T16:04:45.161949+0000 2024-05-20T16:13:56.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:56.268+0000 7f3d38c0d700 1 -- 172.21.15.82:0/3801769361 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f3d200a2c20 msgr2=0x7f3d200a5130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:56.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:56.268+0000 7f3d38c0d700 1 --2- 172.21.15.82:0/3801769361 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f3d200a2c20 0x7f3d200a5130 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f3d34104220 tx=0x7f3d24009190 comp rx=0 tx=0).stop 2024-05-20T16:13:56.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:56.268+0000 7f3d38c0d700 1 -- 172.21.15.82:0/3801769361 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3d34104320 msgr2=0x7f3d34199dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:56.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:56.268+0000 7f3d38c0d700 1 --2- 172.21.15.82:0/3801769361 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3d34104320 0x7f3d34199dc0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f3d1c00d950 tx=0x7f3d1c00dc60 comp rx=0 tx=0).stop 2024-05-20T16:13:56.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:56.268+0000 7f3d38c0d700 1 -- 172.21.15.82:0/3801769361 shutdown_connections 2024-05-20T16:13:56.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:56.268+0000 7f3d38c0d700 1 --2- 172.21.15.82:0/3801769361 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f3d200a2c20 0x7f3d200a5130 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:56.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:56.268+0000 7f3d38c0d700 1 --2- 172.21.15.82:0/3801769361 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3d34104cc0 0x7f3d341a0d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:56.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:56.268+0000 7f3d38c0d700 1 --2- 172.21.15.82:0/3801769361 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3d341031f0 0x7f3d34199880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:56.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:56.268+0000 7f3d38c0d700 1 --2- 172.21.15.82:0/3801769361 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3d34104320 0x7f3d34199dc0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:56.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:56.268+0000 7f3d38c0d700 1 -- 172.21.15.82:0/3801769361 >> 172.21.15.82:0/3801769361 conn(0x7f3d340fe850 msgr2=0x7f3d34107550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:13:56.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:56.268+0000 7f3d38c0d700 1 -- 172.21.15.82:0/3801769361 shutdown_connections 2024-05-20T16:13:56.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:56.268+0000 7f3d38c0d700 1 -- 172.21.15.82:0/3801769361 wait complete. 2024-05-20T16:13:56.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:13:56 smithi082 ceph-mon[32238]: from='client.? 172.21.15.82:0/3801769361' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-05-20T16:13:56.635 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:13:56 smithi184 ceph-mon[36209]: from='client.? 172.21.15.82:0/3801769361' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-05-20T16:13:56.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:13:56 smithi160 ceph-mon[36347]: from='client.? 172.21.15.82:0/3801769361' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-05-20T16:13:56.986 INFO:teuthology.run_tasks:Running task fs.ready... 2024-05-20T16:13:56.997 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-20T16:13:57.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:57.112+0000 7fc4a76ab640 1 -- 172.21.15.82:0/2058206498 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc4a0101370 msgr2=0x7fc4a0101780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:57.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:57.112+0000 7fc4a76ab640 1 --2- 172.21.15.82:0/2058206498 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc4a0101370 0x7fc4a0101780 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fc49c009cd0 tx=0x7fc49c02a8e0 comp rx=0 tx=0).stop 2024-05-20T16:13:57.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:57.112+0000 7fc4a76ab640 1 -- 172.21.15.82:0/2058206498 shutdown_connections 2024-05-20T16:13:57.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:57.112+0000 7fc4a76ab640 1 --2- 172.21.15.82:0/2058206498 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc4a0102610 0x7fc4a0102a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:57.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:57.112+0000 7fc4a76ab640 1 --2- 172.21.15.82:0/2058206498 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc4a0101cc0 0x7fc4a01020d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:57.113 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:57.112+0000 7fc4a76ab640 1 --2- 172.21.15.82:0/2058206498 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc4a0101370 0x7fc4a0101780 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:57.113 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:57.112+0000 7fc4a76ab640 1 -- 172.21.15.82:0/2058206498 >> 172.21.15.82:0/2058206498 conn(0x7fc4a00fcb80 msgr2=0x7fc4a00ff030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:13:57.113 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:57.112+0000 7fc4a76ab640 1 -- 172.21.15.82:0/2058206498 shutdown_connections 2024-05-20T16:13:57.113 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:57.112+0000 7fc4a76ab640 1 -- 172.21.15.82:0/2058206498 wait complete. 2024-05-20T16:13:57.115 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:57.113+0000 7fc4a76ab640 1 Processor -- start 2024-05-20T16:13:57.115 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:57.113+0000 7fc4a76ab640 1 -- start start 2024-05-20T16:13:57.115 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:57.113+0000 7fc4a76ab640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc4a0101370 0x7fc4a019b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:57.115 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:57.113+0000 7fc4a76ab640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc4a0101cc0 0x7fc4a019bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:57.115 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:57.113+0000 7fc4a76ab640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc4a0102610 0x7fc4a01a2a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:57.115 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:57.113+0000 7fc4a76ab640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fc4a01108f0 con 0x7fc4a0101cc0 2024-05-20T16:13:57.115 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:57.113+0000 7fc4a76ab640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fc4a0110770 con 0x7fc4a0101370 2024-05-20T16:13:57.116 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:57.113+0000 7fc4a76ab640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fc4a0110a70 con 0x7fc4a0102610 2024-05-20T16:13:57.116 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:57.113+0000 7fc4a4c46640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc4a0101cc0 0x7fc4a019bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:13:57.116 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:57.113+0000 7fc4a4c46640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc4a0101cc0 0x7fc4a019bac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:46470/0 (socket says 172.21.15.82:46470) 2024-05-20T16:13:57.116 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:57.113+0000 7fc4a4c46640 1 -- 172.21.15.82:0/3813853135 learned_addr learned my addr 172.21.15.82:0/3813853135 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:13:57.116 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:57.113+0000 7fc4a5447640 1 --2- 172.21.15.82:0/3813853135 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc4a0101370 0x7fc4a019b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:13:57.116 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:57.113+0000 7fc4a5c48640 1 --2- 172.21.15.82:0/3813853135 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc4a0102610 0x7fc4a01a2a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:13:57.116 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:57.114+0000 7fc4a4c46640 1 -- 172.21.15.82:0/3813853135 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc4a0102610 msgr2=0x7fc4a01a2a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:57.116 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:57.114+0000 7fc4a4c46640 1 --2- 172.21.15.82:0/3813853135 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc4a0102610 0x7fc4a01a2a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:57.116 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:57.114+0000 7fc4a4c46640 1 -- 172.21.15.82:0/3813853135 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc4a0101370 msgr2=0x7fc4a019b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:57.116 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:57.114+0000 7fc4a4c46640 1 --2- 172.21.15.82:0/3813853135 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc4a0101370 0x7fc4a019b580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:57.116 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:57.114+0000 7fc4a4c46640 1 -- 172.21.15.82:0/3813853135 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4a01a3010 con 0x7fc4a0101cc0 2024-05-20T16:13:57.116 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:57.114+0000 7fc4a5447640 1 --2- 172.21.15.82:0/3813853135 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc4a0101370 0x7fc4a019b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T16:13:57.116 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:57.114+0000 7fc4a5c48640 1 --2- 172.21.15.82:0/3813853135 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc4a0102610 0x7fc4a01a2a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T16:13:57.117 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:57.114+0000 7fc4a4c46640 1 --2- 172.21.15.82:0/3813853135 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc4a0101cc0 0x7fc4a019bac0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fc49000c970 tx=0x7fc49000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:13:57.117 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:57.114+0000 7fc4967fc640 1 -- 172.21.15.82:0/3813853135 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc490007bb0 con 0x7fc4a0101cc0 2024-05-20T16:13:57.117 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:57.114+0000 7fc4967fc640 1 -- 172.21.15.82:0/3813853135 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc490007d50 con 0x7fc4a0101cc0 2024-05-20T16:13:57.117 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:57.114+0000 7fc4a76ab640 1 -- 172.21.15.82:0/3813853135 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4a01a3300 con 0x7fc4a0101cc0 2024-05-20T16:13:57.117 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:57.114+0000 7fc4967fc640 1 -- 172.21.15.82:0/3813853135 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc49000f450 con 0x7fc4a0101cc0 2024-05-20T16:13:57.117 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:57.114+0000 7fc4a76ab640 1 -- 172.21.15.82:0/3813853135 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4a01a3800 con 0x7fc4a0101cc0 2024-05-20T16:13:57.117 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:57.115+0000 7fc4a76ab640 1 -- 172.21.15.82:0/3813853135 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4a010c3b0 con 0x7fc4a0101cc0 2024-05-20T16:13:57.118 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:57.117+0000 7fc4967fc640 1 -- 172.21.15.82:0/3813853135 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fc490005020 con 0x7fc4a0101cc0 2024-05-20T16:13:57.118 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:57.117+0000 7fc4967fc640 1 --2- 172.21.15.82:0/3813853135 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fc48c0a2b90 0x7fc48c0a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:57.118 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:57.118+0000 7fc4a5447640 1 --2- 172.21.15.82:0/3813853135 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fc48c0a2b90 0x7fc48c0a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:13:57.118 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:57.118+0000 7fc4967fc640 1 -- 172.21.15.82:0/3813853135 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9150+0+0 (secure 0 0 0) 0x7fc4900cf260 con 0x7fc4a0101cc0 2024-05-20T16:13:57.118 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:57.118+0000 7fc4a5447640 1 --2- 172.21.15.82:0/3813853135 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fc48c0a2b90 0x7fc48c0a50a0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fc49c031920 tx=0x7fc49c009800 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:13:57.122 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:57.122+0000 7fc4967fc640 1 -- 172.21.15.82:0/3813853135 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fc490014030 con 0x7fc4a0101cc0 2024-05-20T16:13:57.300 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:57.300+0000 7fc4a76ab640 1 -- 172.21.15.82:0/3813853135 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc4a0062750 con 0x7fc4a0101cc0 2024-05-20T16:13:57.301 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:57.302+0000 7fc4967fc640 1 -- 172.21.15.82:0/3813853135 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) ==== 74+0+24771 (secure 0 0 0) 0x7fc49009be30 con 0x7fc4a0101cc0 2024-05-20T16:13:57.302 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-05-20T16:13:57.302 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":95,"fsid":"59d1600a-16c2-11ef-bc9a-c7b262605968","created":"2024-05-20T16:03:46.611556+0000","modified":"2024-05-20T16:13:46.328150+0000","last_up_change":"2024-05-20T16:10:59.303479+0000","last_in_change":"2024-05-20T16:10:39.063561+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-20T16:07:09.493727+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-20T16:11:45.058035+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-20T16:11:45.943862+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-20T16:11:49.637579+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"bae842a9-03ec-490e-b56d-f7957235cdc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":1484330167},{"type":"v1","addr":"172.21.15.82:6803","nonce":1484330167}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":1484330167},{"type":"v1","addr":"172.21.15.82:6805","nonce":1484330167}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1484330167},{"type":"v1","addr":"172.21.15.82:6809","nonce":1484330167}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":1484330167},{"type":"v1","addr":"172.21.15.82:6807","nonce":1484330167}]},"public_addr":"172.21.15.82:6803/1484330167","cluster_addr":"172.21.15.82:6805/1484330167","heartbeat_back_addr":"172.21.15.82:6809/1484330167","heartbeat_front_addr":"172.21.15.82:6807/1484330167","state":["exists","up"]},{"osd":1,"uuid":"59f6a2ba-b5de-4161-81c5-92a417abacdf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":1457003568},{"type":"v1","addr":"172.21.15.160:6801","nonce":1457003568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":1457003568},{"type":"v1","addr":"172.21.15.160:6803","nonce":1457003568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6806","nonce":1457003568},{"type":"v1","addr":"172.21.15.160:6807","nonce":1457003568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":1457003568},{"type":"v1","addr":"172.21.15.160:6805","nonce":1457003568}]},"public_addr":"172.21.15.160:6801/1457003568","cluster_addr":"172.21.15.160:6803/1457003568","heartbeat_back_addr":"172.21.15.160:6807/1457003568","heartbeat_front_addr":"172.21.15.160:6805/1457003568","state":["exists","up"]},{"osd":2,"uuid":"3bbb5587-b449-41f3-bd26-67ec3d6afdfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":4051293934},{"type":"v1","addr":"172.21.15.184:6801","nonce":4051293934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":4051293934},{"type":"v1","addr":"172.21.15.184:6803","nonce":4051293934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":4051293934},{"type":"v1","addr":"172.21.15.184:6807","nonce":4051293934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":4051293934},{"type":"v1","addr":"172.21.15.184:6805","nonce":4051293934}]},"public_addr":"172.21.15.184:6801/4051293934","cluster_addr":"172.21.15.184:6803/4051293934","heartbeat_back_addr":"172.21.15.184:6807/4051293934","heartbeat_front_addr":"172.21.15.184:6805/4051293934","state":["exists","up"]},{"osd":3,"uuid":"ca651f90-09b8-4081-a6f9-40b31268a837","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":3947485599},{"type":"v1","addr":"172.21.15.82:6811","nonce":3947485599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":3947485599},{"type":"v1","addr":"172.21.15.82:6813","nonce":3947485599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":3947485599},{"type":"v1","addr":"172.21.15.82:6817","nonce":3947485599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":3947485599},{"type":"v1","addr":"172.21.15.82:6815","nonce":3947485599}]},"public_addr":"172.21.15.82:6811/3947485599","cluster_addr":"172.21.15.82:6813/3947485599","heartbeat_back_addr":"172.21.15.82:6817/3947485599","heartbeat_front_addr":"172.21.15.82:6815/3947485599","state":["exists","up"]},{"osd":4,"uuid":"cb3dfd56-f677-40cb-9c96-8319ea5ac788","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":4160830014},{"type":"v1","addr":"172.21.15.160:6809","nonce":4160830014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":4160830014},{"type":"v1","addr":"172.21.15.160:6811","nonce":4160830014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":4160830014},{"type":"v1","addr":"172.21.15.160:6815","nonce":4160830014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6812","nonce":4160830014},{"type":"v1","addr":"172.21.15.160:6813","nonce":4160830014}]},"public_addr":"172.21.15.160:6809/4160830014","cluster_addr":"172.21.15.160:6811/4160830014","heartbeat_back_addr":"172.21.15.160:6815/4160830014","heartbeat_front_addr":"172.21.15.160:6813/4160830014","state":["exists","up"]},{"osd":5,"uuid":"c70cfcd1-a4ff-461f-8c73-479b061200fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":1046282672},{"type":"v1","addr":"172.21.15.184:6809","nonce":1046282672}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":1046282672},{"type":"v1","addr":"172.21.15.184:6811","nonce":1046282672}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":1046282672},{"type":"v1","addr":"172.21.15.184:6815","nonce":1046282672}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":1046282672},{"type":"v1","addr":"172.21.15.184:6813","nonce":1046282672}]},"public_addr":"172.21.15.184:6809/1046282672","cluster_addr":"172.21.15.184:6811/1046282672","heartbeat_back_addr":"172.21.15.184:6815/1046282672","heartbeat_front_addr":"172.21.15.184:6813/1046282672","state":["exists","up"]},{"osd":6,"uuid":"d47ce001-9dd1-4830-bcc9-888601187c6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":2789736499},{"type":"v1","addr":"172.21.15.82:6819","nonce":2789736499}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":2789736499},{"type":"v1","addr":"172.21.15.82:6821","nonce":2789736499}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":2789736499},{"type":"v1","addr":"172.21.15.82:6825","nonce":2789736499}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":2789736499},{"type":"v1","addr":"172.21.15.82:6823","nonce":2789736499}]},"public_addr":"172.21.15.82:6819/2789736499","cluster_addr":"172.21.15.82:6821/2789736499","heartbeat_back_addr":"172.21.15.82:6825/2789736499","heartbeat_front_addr":"172.21.15.82:6823/2789736499","state":["exists","up"]},{"osd":7,"uuid":"25c54f43-18cb-41e4-83b4-2732b5a93889","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6816","nonce":2340683378},{"type":"v1","addr":"172.21.15.160:6817","nonce":2340683378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":2340683378},{"type":"v1","addr":"172.21.15.160:6819","nonce":2340683378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":2340683378},{"type":"v1","addr":"172.21.15.160:6823","nonce":2340683378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6820","nonce":2340683378},{"type":"v1","addr":"172.21.15.160:6821","nonce":2340683378}]},"public_addr":"172.21.15.160:6817/2340683378","cluster_addr":"172.21.15.160:6819/2340683378","heartbeat_back_addr":"172.21.15.160:6823/2340683378","heartbeat_front_addr":"172.21.15.160:6821/2340683378","state":["exists","up"]},{"osd":8,"uuid":"1f8b478d-287f-45f5-8526-d1a58c5bc07c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":3831576372},{"type":"v1","addr":"172.21.15.184:6817","nonce":3831576372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":3831576372},{"type":"v1","addr":"172.21.15.184:6819","nonce":3831576372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":3831576372},{"type":"v1","addr":"172.21.15.184:6823","nonce":3831576372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":3831576372},{"type":"v1","addr":"172.21.15.184:6821","nonce":3831576372}]},"public_addr":"172.21.15.184:6817/3831576372","cluster_addr":"172.21.15.184:6819/3831576372","heartbeat_back_addr":"172.21.15.184:6823/3831576372","heartbeat_front_addr":"172.21.15.184:6821/3831576372","state":["exists","up"]},{"osd":9,"uuid":"0d69e319-954f-4b18-95ea-31794f18dc3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":716330782},{"type":"v1","addr":"172.21.15.82:6827","nonce":716330782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":716330782},{"type":"v1","addr":"172.21.15.82:6829","nonce":716330782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":716330782},{"type":"v1","addr":"172.21.15.82:6833","nonce":716330782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":716330782},{"type":"v1","addr":"172.21.15.82:6831","nonce":716330782}]},"public_addr":"172.21.15.82:6827/716330782","cluster_addr":"172.21.15.82:6829/716330782","heartbeat_back_addr":"172.21.15.82:6833/716330782","heartbeat_front_addr":"172.21.15.82:6831/716330782","state":["exists","up"]},{"osd":10,"uuid":"39dc1b78-61cb-42ca-b5b6-1e44ac2b6500","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":2327428014},{"type":"v1","addr":"172.21.15.160:6825","nonce":2327428014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":2327428014},{"type":"v1","addr":"172.21.15.160:6827","nonce":2327428014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":2327428014},{"type":"v1","addr":"172.21.15.160:6831","nonce":2327428014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":2327428014},{"type":"v1","addr":"172.21.15.160:6829","nonce":2327428014}]},"public_addr":"172.21.15.160:6825/2327428014","cluster_addr":"172.21.15.160:6827/2327428014","heartbeat_back_addr":"172.21.15.160:6831/2327428014","heartbeat_front_addr":"172.21.15.160:6829/2327428014","state":["exists","up"]},{"osd":11,"uuid":"4306b3c1-707a-4010-987c-209ea0e948e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":3594364650},{"type":"v1","addr":"172.21.15.184:6825","nonce":3594364650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":3594364650},{"type":"v1","addr":"172.21.15.184:6827","nonce":3594364650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3594364650},{"type":"v1","addr":"172.21.15.184:6831","nonce":3594364650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3594364650},{"type":"v1","addr":"172.21.15.184:6829","nonce":3594364650}]},"public_addr":"172.21.15.184:6825/3594364650","cluster_addr":"172.21.15.184:6827/3594364650","heartbeat_back_addr":"172.21.15.184:6831/3594364650","heartbeat_front_addr":"172.21.15.184:6829/3594364650","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:06:17.855998+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:06:41.473270+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:07:05.304108+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:07:29.887472+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:07:55.387286+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:08:20.579669+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:08:46.333051+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:09:12.289668+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:09:38.180696+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:10:04.471459+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:10:31.041912+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:10:57.477266+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.1d","mappings":[{"from":5,"to":9}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.5","mappings":[{"from":9,"to":10}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.82:0/2005424509":"2024-05-21T16:04:45.161949+0000","172.21.15.82:6801/1763049308":"2024-05-21T16:04:45.161949+0000","172.21.15.82:6801/1959459348":"2024-05-21T16:04:19.208291+0000","172.21.15.82:0/3131904556":"2024-05-21T16:04:19.208291+0000","172.21.15.82:6800/1959459348":"2024-05-21T16:04:19.208291+0000","172.21.15.82:0/2456966342":"2024-05-21T16:04:45.161949+0000","172.21.15.82:0/4179978833":"2024-05-21T16:04:19.208291+0000","172.21.15.82:0/1602862225":"2024-05-21T16:04:45.161949+0000","172.21.15.82:0/106101180":"2024-05-21T16:04:19.208291+0000","172.21.15.82:6800/1763049308":"2024-05-21T16:04:45.161949+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-20T16:13:57.304 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:57.304+0000 7fc4a76ab640 1 -- 172.21.15.82:0/3813853135 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fc48c0a2b90 msgr2=0x7fc48c0a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:57.304 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:57.304+0000 7fc4a76ab640 1 --2- 172.21.15.82:0/3813853135 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fc48c0a2b90 0x7fc48c0a50a0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fc49c031920 tx=0x7fc49c009800 comp rx=0 tx=0).stop 2024-05-20T16:13:57.304 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:57.304+0000 7fc4a76ab640 1 -- 172.21.15.82:0/3813853135 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc4a0101cc0 msgr2=0x7fc4a019bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:57.304 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:57.304+0000 7fc4a76ab640 1 --2- 172.21.15.82:0/3813853135 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc4a0101cc0 0x7fc4a019bac0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fc49000c970 tx=0x7fc49000ce30 comp rx=0 tx=0).stop 2024-05-20T16:13:57.304 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:57.304+0000 7fc4a76ab640 1 -- 172.21.15.82:0/3813853135 shutdown_connections 2024-05-20T16:13:57.305 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:57.304+0000 7fc4a76ab640 1 --2- 172.21.15.82:0/3813853135 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fc48c0a2b90 0x7fc48c0a50a0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:57.305 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:57.304+0000 7fc4a76ab640 1 --2- 172.21.15.82:0/3813853135 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc4a0101cc0 0x7fc4a019bac0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:57.305 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:57.304+0000 7fc4a76ab640 1 --2- 172.21.15.82:0/3813853135 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc4a0102610 0x7fc4a01a2a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:57.305 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:57.304+0000 7fc4a76ab640 1 --2- 172.21.15.82:0/3813853135 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc4a0101370 0x7fc4a019b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:57.305 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:57.304+0000 7fc4a76ab640 1 -- 172.21.15.82:0/3813853135 >> 172.21.15.82:0/3813853135 conn(0x7fc4a00fcb80 msgr2=0x7fc4a0109580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:13:57.305 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:57.304+0000 7fc4a76ab640 1 -- 172.21.15.82:0/3813853135 shutdown_connections 2024-05-20T16:13:57.305 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:57.304+0000 7fc4a76ab640 1 -- 172.21.15.82:0/3813853135 wait complete. 2024-05-20T16:13:57.316 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-20T16:07:09.493727+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-20T16:11:45.058035+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-20T16:11:45.943862+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-20T16:11:49.637579+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '91', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-20T16:13:57.317 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-20T16:13:57.434 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:57.431+0000 7fa5363aa640 1 -- 172.21.15.82:0/1566943850 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa530101990 msgr2=0x7fa530102690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:57.434 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:57.431+0000 7fa5363aa640 1 --2- 172.21.15.82:0/1566943850 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa530101990 0x7fa530102690 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fa520009900 tx=0x7fa52002a410 comp rx=0 tx=0).stop 2024-05-20T16:13:57.434 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:57.432+0000 7fa5363aa640 1 -- 172.21.15.82:0/1566943850 shutdown_connections 2024-05-20T16:13:57.434 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:57.432+0000 7fa5363aa640 1 --2- 172.21.15.82:0/1566943850 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa5301078d0 0x7fa530109cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:57.434 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:57.432+0000 7fa5363aa640 1 --2- 172.21.15.82:0/1566943850 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa530102bd0 0x7fa530102fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:57.435 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:57.432+0000 7fa5363aa640 1 --2- 172.21.15.82:0/1566943850 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa530101990 0x7fa530102690 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:57.435 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:57.432+0000 7fa5363aa640 1 -- 172.21.15.82:0/1566943850 >> 172.21.15.82:0/1566943850 conn(0x7fa5300fcb80 msgr2=0x7fa5300ff030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:13:57.435 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:57.432+0000 7fa5363aa640 1 -- 172.21.15.82:0/1566943850 shutdown_connections 2024-05-20T16:13:57.435 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:57.432+0000 7fa5363aa640 1 -- 172.21.15.82:0/1566943850 wait complete. 2024-05-20T16:13:57.435 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:57.432+0000 7fa5363aa640 1 Processor -- start 2024-05-20T16:13:57.435 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:57.432+0000 7fa5363aa640 1 -- start start 2024-05-20T16:13:57.435 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:57.433+0000 7fa5363aa640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa530101990 0x7fa530199380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:57.435 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:57.433+0000 7fa5363aa640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa530102bd0 0x7fa5301998c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:57.435 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:57.433+0000 7fa5363aa640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa5301078d0 0x7fa5301a0850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:57.435 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:57.433+0000 7fa5363aa640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fa53010e750 con 0x7fa530102bd0 2024-05-20T16:13:57.435 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:57.433+0000 7fa5363aa640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fa53010e5d0 con 0x7fa5301078d0 2024-05-20T16:13:57.435 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:57.433+0000 7fa5363aa640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa53010e8d0 con 0x7fa530101990 2024-05-20T16:13:57.435 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:57.433+0000 7fa52f7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa530102bd0 0x7fa5301998c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:13:57.436 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:57.433+0000 7fa52f7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa530102bd0 0x7fa5301998c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:46500/0 (socket says 172.21.15.82:46500) 2024-05-20T16:13:57.436 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:57.433+0000 7fa52f7fe640 1 -- 172.21.15.82:0/2329424043 learned_addr learned my addr 172.21.15.82:0/2329424043 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:13:57.436 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:57.433+0000 7fa52ffff640 1 --2- 172.21.15.82:0/2329424043 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa530101990 0x7fa530199380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:13:57.436 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:57.433+0000 7fa534947640 1 --2- 172.21.15.82:0/2329424043 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa5301078d0 0x7fa5301a0850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:13:57.436 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:57.433+0000 7fa52f7fe640 1 -- 172.21.15.82:0/2329424043 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa530101990 msgr2=0x7fa530199380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:57.436 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:57.433+0000 7fa52f7fe640 1 --2- 172.21.15.82:0/2329424043 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa530101990 0x7fa530199380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:57.436 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:57.433+0000 7fa52f7fe640 1 -- 172.21.15.82:0/2329424043 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa5301078d0 msgr2=0x7fa5301a0850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:57.436 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:57.433+0000 7fa52f7fe640 1 --2- 172.21.15.82:0/2329424043 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa5301078d0 0x7fa5301a0850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:57.436 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:57.433+0000 7fa52f7fe640 1 -- 172.21.15.82:0/2329424043 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5301a0ec0 con 0x7fa530102bd0 2024-05-20T16:13:57.436 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:57.433+0000 7fa534947640 1 --2- 172.21.15.82:0/2329424043 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa5301078d0 0x7fa5301a0850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T16:13:57.436 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:57.433+0000 7fa52f7fe640 1 --2- 172.21.15.82:0/2329424043 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa530102bd0 0x7fa5301998c0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fa52400c890 tx=0x7fa52400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:13:57.436 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:57.433+0000 7fa52d7fa640 1 -- 172.21.15.82:0/2329424043 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa524004220 con 0x7fa530102bd0 2024-05-20T16:13:57.436 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:57.433+0000 7fa5363aa640 1 -- 172.21.15.82:0/2329424043 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5301a11b0 con 0x7fa530102bd0 2024-05-20T16:13:57.436 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:57.433+0000 7fa52d7fa640 1 -- 172.21.15.82:0/2329424043 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa5240043c0 con 0x7fa530102bd0 2024-05-20T16:13:57.437 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:57.433+0000 7fa52d7fa640 1 -- 172.21.15.82:0/2329424043 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa524004a00 con 0x7fa530102bd0 2024-05-20T16:13:57.437 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:57.433+0000 7fa5363aa640 1 -- 172.21.15.82:0/2329424043 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5301a17d0 con 0x7fa530102bd0 2024-05-20T16:13:57.437 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:57.434+0000 7fa5363aa640 1 -- 172.21.15.82:0/2329424043 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5301066e0 con 0x7fa530102bd0 2024-05-20T16:13:57.437 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:57.436+0000 7fa52d7fa640 1 -- 172.21.15.82:0/2329424043 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fa5240079b0 con 0x7fa530102bd0 2024-05-20T16:13:57.437 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:57.436+0000 7fa52d7fa640 1 --2- 172.21.15.82:0/2329424043 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fa5180a2b90 0x7fa5180a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:57.438 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:57.436+0000 7fa52d7fa640 1 -- 172.21.15.82:0/2329424043 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9150+0+0 (secure 0 0 0) 0x7fa5240cf580 con 0x7fa530102bd0 2024-05-20T16:13:57.438 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:57.436+0000 7fa52ffff640 1 --2- 172.21.15.82:0/2329424043 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fa5180a2b90 0x7fa5180a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:13:57.438 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:57.437+0000 7fa52ffff640 1 --2- 172.21.15.82:0/2329424043 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fa5180a2b90 0x7fa5180a50a0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fa520031610 tx=0x7fa520000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:13:57.440 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:57.440+0000 7fa52d7fa640 1 -- 172.21.15.82:0/2329424043 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fa524014030 con 0x7fa530102bd0 2024-05-20T16:13:57.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:13:57 smithi082 ceph-mon[32238]: pgmap v361: 129 pgs: 129 active+clean; 641 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s; 10 B/s, 0 objects/s recovering 2024-05-20T16:13:57.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:13:57 smithi082 ceph-mon[32238]: from='client.? 172.21.15.82:0/3813853135' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-20T16:13:57.617 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:57.618+0000 7fa5363aa640 1 -- 172.21.15.82:0/2329424043 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fa530062750 con 0x7fa530102bd0 2024-05-20T16:13:57.618 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:57.618+0000 7fa52d7fa640 1 -- 172.21.15.82:0/2329424043 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v95) ==== 93+0+10 (secure 0 0 0) 0x7fa5240ce0a0 con 0x7fa530102bd0 2024-05-20T16:13:57.618 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 1 2024-05-20T16:13:57.620 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:57.620+0000 7fa5363aa640 1 -- 172.21.15.82:0/2329424043 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fa5180a2b90 msgr2=0x7fa5180a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:57.620 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:57.620+0000 7fa5363aa640 1 --2- 172.21.15.82:0/2329424043 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fa5180a2b90 0x7fa5180a50a0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fa520031610 tx=0x7fa520000bc0 comp rx=0 tx=0).stop 2024-05-20T16:13:57.620 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:57.620+0000 7fa5363aa640 1 -- 172.21.15.82:0/2329424043 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa530102bd0 msgr2=0x7fa5301998c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:57.620 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:57.620+0000 7fa5363aa640 1 --2- 172.21.15.82:0/2329424043 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa530102bd0 0x7fa5301998c0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fa52400c890 tx=0x7fa52400cd50 comp rx=0 tx=0).stop 2024-05-20T16:13:57.620 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:57.620+0000 7fa5363aa640 1 -- 172.21.15.82:0/2329424043 shutdown_connections 2024-05-20T16:13:57.620 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:57.620+0000 7fa5363aa640 1 --2- 172.21.15.82:0/2329424043 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fa5180a2b90 0x7fa5180a50a0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:57.620 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:57.620+0000 7fa5363aa640 1 --2- 172.21.15.82:0/2329424043 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa530102bd0 0x7fa5301998c0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:57.621 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:57.620+0000 7fa5363aa640 1 --2- 172.21.15.82:0/2329424043 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa5301078d0 0x7fa5301a0850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:57.621 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:57.620+0000 7fa5363aa640 1 --2- 172.21.15.82:0/2329424043 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa530101990 0x7fa530199380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:57.621 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:57.620+0000 7fa5363aa640 1 -- 172.21.15.82:0/2329424043 >> 172.21.15.82:0/2329424043 conn(0x7fa5300fcb80 msgr2=0x7fa5301038b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:13:57.621 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:57.620+0000 7fa5363aa640 1 -- 172.21.15.82:0/2329424043 shutdown_connections 2024-05-20T16:13:57.621 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:57.620+0000 7fa5363aa640 1 -- 172.21.15.82:0/2329424043 wait complete. 2024-05-20T16:13:57.632 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-20T16:13:57.713 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:13:57 smithi160 ceph-mon[36347]: pgmap v361: 129 pgs: 129 active+clean; 641 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s; 10 B/s, 0 objects/s recovering 2024-05-20T16:13:57.713 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:13:57 smithi160 ceph-mon[36347]: from='client.? 172.21.15.82:0/3813853135' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-20T16:13:57.746 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:57.746+0000 7f72c82c3640 1 -- 172.21.15.82:0/740138981 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f72c0102bd0 msgr2=0x7f72c0102fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:57.746 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:57.746+0000 7f72c82c3640 1 --2- 172.21.15.82:0/740138981 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f72c0102bd0 0x7f72c0102fe0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f72b0009900 tx=0x7f72b002a410 comp rx=0 tx=0).stop 2024-05-20T16:13:57.746 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:57.746+0000 7f72c82c3640 1 -- 172.21.15.82:0/740138981 shutdown_connections 2024-05-20T16:13:57.746 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:57.746+0000 7f72c82c3640 1 --2- 172.21.15.82:0/740138981 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f72c0102bd0 0x7f72c0102fe0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:57.746 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:57.746+0000 7f72c82c3640 1 --2- 172.21.15.82:0/740138981 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f72c01078d0 0x7f72c0109cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:57.746 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:57.746+0000 7f72c82c3640 1 --2- 172.21.15.82:0/740138981 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f72c0101990 0x7f72c0102690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:57.747 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:57.746+0000 7f72c82c3640 1 -- 172.21.15.82:0/740138981 >> 172.21.15.82:0/740138981 conn(0x7f72c00fcb80 msgr2=0x7f72c00ff030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:13:57.747 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:57.746+0000 7f72c82c3640 1 -- 172.21.15.82:0/740138981 shutdown_connections 2024-05-20T16:13:57.747 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:57.746+0000 7f72c82c3640 1 -- 172.21.15.82:0/740138981 wait complete. 2024-05-20T16:13:57.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:57.747+0000 7f72c82c3640 1 Processor -- start 2024-05-20T16:13:57.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:57.747+0000 7f72c82c3640 1 -- start start 2024-05-20T16:13:57.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:57.747+0000 7f72c82c3640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f72c0101990 0x7f72c01984f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:57.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:57.747+0000 7f72c82c3640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f72c0102bd0 0x7f72c0198a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:57.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:57.747+0000 7f72c82c3640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f72c01078d0 0x7f72c019f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:57.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:57.747+0000 7f72c82c3640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f72c010d650 con 0x7f72c01078d0 2024-05-20T16:13:57.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:57.747+0000 7f72c82c3640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f72c010d4d0 con 0x7f72c0101990 2024-05-20T16:13:57.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:57.747+0000 7f72c82c3640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f72c010d7d0 con 0x7f72c0102bd0 2024-05-20T16:13:57.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:57.747+0000 7f72c6860640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f72c01078d0 0x7f72c019f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:13:57.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:57.747+0000 7f72c6860640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f72c01078d0 0x7f72c019f9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:46520/0 (socket says 172.21.15.82:46520) 2024-05-20T16:13:57.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:57.747+0000 7f72c6860640 1 -- 172.21.15.82:0/606565183 learned_addr learned my addr 172.21.15.82:0/606565183 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:13:57.750 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:57.747+0000 7f72c605f640 1 --2- 172.21.15.82:0/606565183 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f72c0101990 0x7f72c01984f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:13:57.750 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:57.748+0000 7f72c6860640 1 -- 172.21.15.82:0/606565183 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f72c0102bd0 msgr2=0x7f72c0198a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:57.750 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:57.748+0000 7f72c6860640 1 --2- 172.21.15.82:0/606565183 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f72c0102bd0 0x7f72c0198a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:57.750 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:57.748+0000 7f72c6860640 1 -- 172.21.15.82:0/606565183 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f72c0101990 msgr2=0x7f72c01984f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:57.750 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:57.748+0000 7f72c6860640 1 --2- 172.21.15.82:0/606565183 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f72c0101990 0x7f72c01984f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:57.750 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:57.748+0000 7f72c6860640 1 -- 172.21.15.82:0/606565183 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72c019ff80 con 0x7f72c01078d0 2024-05-20T16:13:57.750 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:57.748+0000 7f72c6860640 1 --2- 172.21.15.82:0/606565183 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f72c01078d0 0x7f72c019f9c0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f72b800cce0 tx=0x7f72b800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:13:57.750 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:57.748+0000 7f72b77fe640 1 -- 172.21.15.82:0/606565183 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72b800f450 con 0x7f72c01078d0 2024-05-20T16:13:57.750 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:57.748+0000 7f72b77fe640 1 -- 172.21.15.82:0/606565183 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f72b800fad0 con 0x7f72c01078d0 2024-05-20T16:13:57.750 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:57.748+0000 7f72b77fe640 1 -- 172.21.15.82:0/606565183 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72b8018c10 con 0x7f72c01078d0 2024-05-20T16:13:57.750 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:57.748+0000 7f72c82c3640 1 -- 172.21.15.82:0/606565183 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72c01a0270 con 0x7f72c01078d0 2024-05-20T16:13:57.750 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:57.748+0000 7f72c82c3640 1 -- 172.21.15.82:0/606565183 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72c01a0770 con 0x7f72c01078d0 2024-05-20T16:13:57.750 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:57.749+0000 7f72c82c3640 1 -- 172.21.15.82:0/606565183 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f72c01066e0 con 0x7f72c01078d0 2024-05-20T16:13:57.751 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:57.750+0000 7f72b77fe640 1 -- 172.21.15.82:0/606565183 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f72b800f5f0 con 0x7f72c01078d0 2024-05-20T16:13:57.751 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:57.751+0000 7f72b77fe640 1 --2- 172.21.15.82:0/606565183 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f72ac0a2b90 0x7f72ac0a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:57.751 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:57.751+0000 7f72b77fe640 1 -- 172.21.15.82:0/606565183 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9150+0+0 (secure 0 0 0) 0x7f72b80cfbd0 con 0x7f72c01078d0 2024-05-20T16:13:57.751 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:57.751+0000 7f72c605f640 1 --2- 172.21.15.82:0/606565183 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f72ac0a2b90 0x7f72ac0a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:13:57.751 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:57.751+0000 7f72c605f640 1 --2- 172.21.15.82:0/606565183 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f72ac0a2b90 0x7f72ac0a50a0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f72bc0058c0 tx=0x7f72bc00a4c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:13:57.754 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:57.754+0000 7f72b77fe640 1 -- 172.21.15.82:0/606565183 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f72b8014030 con 0x7f72c01078d0 2024-05-20T16:13:57.885 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:13:57 smithi184 ceph-mon[36209]: pgmap v361: 129 pgs: 129 active+clean; 641 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s; 10 B/s, 0 objects/s recovering 2024-05-20T16:13:57.885 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:13:57 smithi184 ceph-mon[36209]: from='client.? 172.21.15.82:0/3813853135' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-20T16:13:57.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:57.934+0000 7f72c82c3640 1 -- 172.21.15.82:0/606565183 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f72c002d430 con 0x7f72c01078d0 2024-05-20T16:13:57.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:57.935+0000 7f72b77fe640 1 -- 172.21.15.82:0/606565183 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v95) ==== 104+0+11 (secure 0 0 0) 0x7f72b809c830 con 0x7f72c01078d0 2024-05-20T16:13:57.934 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 32 2024-05-20T16:13:57.937 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:57.937+0000 7f72c82c3640 1 -- 172.21.15.82:0/606565183 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f72ac0a2b90 msgr2=0x7f72ac0a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:57.937 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:57.937+0000 7f72c82c3640 1 --2- 172.21.15.82:0/606565183 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f72ac0a2b90 0x7f72ac0a50a0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f72bc0058c0 tx=0x7f72bc00a4c0 comp rx=0 tx=0).stop 2024-05-20T16:13:57.937 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:57.937+0000 7f72c82c3640 1 -- 172.21.15.82:0/606565183 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f72c01078d0 msgr2=0x7f72c019f9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:57.937 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:57.937+0000 7f72c82c3640 1 --2- 172.21.15.82:0/606565183 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f72c01078d0 0x7f72c019f9c0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f72b800cce0 tx=0x7f72b800a620 comp rx=0 tx=0).stop 2024-05-20T16:13:57.937 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:57.937+0000 7f72c82c3640 1 -- 172.21.15.82:0/606565183 shutdown_connections 2024-05-20T16:13:57.937 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:57.937+0000 7f72c82c3640 1 --2- 172.21.15.82:0/606565183 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f72ac0a2b90 0x7f72ac0a50a0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:57.937 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:57.937+0000 7f72c82c3640 1 --2- 172.21.15.82:0/606565183 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f72c01078d0 0x7f72c019f9c0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:57.937 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:57.937+0000 7f72c82c3640 1 --2- 172.21.15.82:0/606565183 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f72c0102bd0 0x7f72c0198a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:57.937 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:57.937+0000 7f72c82c3640 1 --2- 172.21.15.82:0/606565183 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f72c0101990 0x7f72c01984f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:57.937 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:57.937+0000 7f72c82c3640 1 -- 172.21.15.82:0/606565183 >> 172.21.15.82:0/606565183 conn(0x7f72c00fcb80 msgr2=0x7f72c01038b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:13:57.937 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:57.937+0000 7f72c82c3640 1 -- 172.21.15.82:0/606565183 shutdown_connections 2024-05-20T16:13:57.938 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:57.937+0000 7f72c82c3640 1 -- 172.21.15.82:0/606565183 wait complete. 2024-05-20T16:13:57.948 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-20T16:13:58.065 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:58.063+0000 7f40d24e9640 1 -- 172.21.15.82:0/908428144 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f40cc1019b0 msgr2=0x7f40cc1026b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:58.065 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:58.063+0000 7f40d24e9640 1 --2- 172.21.15.82:0/908428144 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f40cc1019b0 0x7f40cc1026b0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f40bc009900 tx=0x7f40bc02a410 comp rx=0 tx=0).stop 2024-05-20T16:13:58.065 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:58.063+0000 7f40d24e9640 1 -- 172.21.15.82:0/908428144 shutdown_connections 2024-05-20T16:13:58.066 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:58.063+0000 7f40d24e9640 1 --2- 172.21.15.82:0/908428144 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f40cc1078f0 0x7f40cc109d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:58.066 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:58.063+0000 7f40d24e9640 1 --2- 172.21.15.82:0/908428144 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f40cc102bf0 0x7f40cc103000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:58.066 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:58.063+0000 7f40d24e9640 1 --2- 172.21.15.82:0/908428144 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f40cc1019b0 0x7f40cc1026b0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:58.066 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:58.063+0000 7f40d24e9640 1 -- 172.21.15.82:0/908428144 >> 172.21.15.82:0/908428144 conn(0x7f40cc0fcba0 msgr2=0x7f40cc0ff050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:13:58.066 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:58.063+0000 7f40d24e9640 1 -- 172.21.15.82:0/908428144 shutdown_connections 2024-05-20T16:13:58.066 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:58.063+0000 7f40d24e9640 1 -- 172.21.15.82:0/908428144 wait complete. 2024-05-20T16:13:58.066 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:58.064+0000 7f40d24e9640 1 Processor -- start 2024-05-20T16:13:58.066 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:58.064+0000 7f40d24e9640 1 -- start start 2024-05-20T16:13:58.066 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:58.064+0000 7f40d24e9640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f40cc1019b0 0x7f40cc1981b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:58.066 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:58.064+0000 7f40d24e9640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f40cc102bf0 0x7f40cc1986f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:58.066 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:58.064+0000 7f40d24e9640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f40cc1078f0 0x7f40cc19f680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:58.066 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:58.064+0000 7f40d24e9640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f40cc10d5e0 con 0x7f40cc1078f0 2024-05-20T16:13:58.066 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:58.064+0000 7f40d24e9640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f40cc10d460 con 0x7f40cc1019b0 2024-05-20T16:13:58.067 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:58.064+0000 7f40d24e9640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f40cc10d760 con 0x7f40cc102bf0 2024-05-20T16:13:58.067 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:58.064+0000 7f40d0a86640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f40cc1078f0 0x7f40cc19f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:13:58.067 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:58.064+0000 7f40d0a86640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f40cc1078f0 0x7f40cc19f680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:46528/0 (socket says 172.21.15.82:46528) 2024-05-20T16:13:58.067 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:58.064+0000 7f40d0a86640 1 -- 172.21.15.82:0/3516951462 learned_addr learned my addr 172.21.15.82:0/3516951462 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:13:58.067 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:58.064+0000 7f40cbfff640 1 --2- 172.21.15.82:0/3516951462 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f40cc1019b0 0x7f40cc1981b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:13:58.067 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:58.065+0000 7f40c3fff640 1 --2- 172.21.15.82:0/3516951462 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f40cc102bf0 0x7f40cc1986f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:13:58.067 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:58.065+0000 7f40d0a86640 1 -- 172.21.15.82:0/3516951462 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f40cc102bf0 msgr2=0x7f40cc1986f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:58.067 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:58.065+0000 7f40d0a86640 1 --2- 172.21.15.82:0/3516951462 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f40cc102bf0 0x7f40cc1986f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:58.067 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:58.065+0000 7f40d0a86640 1 -- 172.21.15.82:0/3516951462 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f40cc1019b0 msgr2=0x7f40cc1981b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:58.067 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:58.065+0000 7f40d0a86640 1 --2- 172.21.15.82:0/3516951462 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f40cc1019b0 0x7f40cc1981b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:58.067 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:58.065+0000 7f40d0a86640 1 -- 172.21.15.82:0/3516951462 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40cc19fcf0 con 0x7f40cc1078f0 2024-05-20T16:13:58.067 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:58.065+0000 7f40d0a86640 1 --2- 172.21.15.82:0/3516951462 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f40cc1078f0 0x7f40cc19f680 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f40c400eb10 tx=0x7f40c400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:13:58.067 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:58.065+0000 7f40c9ffb640 1 -- 172.21.15.82:0/3516951462 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40c4004b20 con 0x7f40cc1078f0 2024-05-20T16:13:58.067 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:58.065+0000 7f40c9ffb640 1 -- 172.21.15.82:0/3516951462 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f40c4004cc0 con 0x7f40cc1078f0 2024-05-20T16:13:58.067 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:58.065+0000 7f40d24e9640 1 -- 172.21.15.82:0/3516951462 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40cc19ffe0 con 0x7f40cc1078f0 2024-05-20T16:13:58.068 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:58.065+0000 7f40c9ffb640 1 -- 172.21.15.82:0/3516951462 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40c4010760 con 0x7f40cc1078f0 2024-05-20T16:13:58.068 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:58.065+0000 7f40d24e9640 1 -- 172.21.15.82:0/3516951462 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40cc1a03f0 con 0x7f40cc1078f0 2024-05-20T16:13:58.068 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:58.066+0000 7f40d24e9640 1 -- 172.21.15.82:0/3516951462 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40cc106700 con 0x7f40cc1078f0 2024-05-20T16:13:58.069 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:58.067+0000 7f40c9ffb640 1 -- 172.21.15.82:0/3516951462 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f40c4005020 con 0x7f40cc1078f0 2024-05-20T16:13:58.069 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:58.068+0000 7f40c9ffb640 1 --2- 172.21.15.82:0/3516951462 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f40b80a2b90 0x7f40b80a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:58.069 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:58.068+0000 7f40cbfff640 1 --2- 172.21.15.82:0/3516951462 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f40b80a2b90 0x7f40b80a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:13:58.069 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:58.068+0000 7f40c9ffb640 1 -- 172.21.15.82:0/3516951462 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9150+0+0 (secure 0 0 0) 0x7f40c40cf960 con 0x7f40cc1078f0 2024-05-20T16:13:58.069 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:58.069+0000 7f40cbfff640 1 --2- 172.21.15.82:0/3516951462 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f40b80a2b90 0x7f40b80a50a0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f40bc005ab0 tx=0x7f40bc031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:13:58.072 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:58.073+0000 7f40c9ffb640 1 -- 172.21.15.82:0/3516951462 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f40c409c530 con 0x7f40cc1078f0 2024-05-20T16:13:58.249 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:58.250+0000 7f40d24e9640 1 -- 172.21.15.82:0/3516951462 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f40cc062820 con 0x7f40cc1078f0 2024-05-20T16:13:58.250 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:58.250+0000 7f40c9ffb640 1 -- 172.21.15.82:0/3516951462 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v95) ==== 100+0+11 (secure 0 0 0) 0x7f40c409be30 con 0x7f40cc1078f0 2024-05-20T16:13:58.250 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 64 2024-05-20T16:13:58.252 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:58.252+0000 7f40d24e9640 1 -- 172.21.15.82:0/3516951462 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f40b80a2b90 msgr2=0x7f40b80a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:58.252 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:58.252+0000 7f40d24e9640 1 --2- 172.21.15.82:0/3516951462 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f40b80a2b90 0x7f40b80a50a0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f40bc005ab0 tx=0x7f40bc031400 comp rx=0 tx=0).stop 2024-05-20T16:13:58.252 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:58.252+0000 7f40d24e9640 1 -- 172.21.15.82:0/3516951462 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f40cc1078f0 msgr2=0x7f40cc19f680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:58.252 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:58.252+0000 7f40d24e9640 1 --2- 172.21.15.82:0/3516951462 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f40cc1078f0 0x7f40cc19f680 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f40c400eb10 tx=0x7f40c400c2d0 comp rx=0 tx=0).stop 2024-05-20T16:13:58.252 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:58.252+0000 7f40d24e9640 1 -- 172.21.15.82:0/3516951462 shutdown_connections 2024-05-20T16:13:58.253 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:58.252+0000 7f40d24e9640 1 --2- 172.21.15.82:0/3516951462 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f40b80a2b90 0x7f40b80a50a0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:58.253 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:58.252+0000 7f40d24e9640 1 --2- 172.21.15.82:0/3516951462 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f40cc1078f0 0x7f40cc19f680 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:58.253 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:58.252+0000 7f40d24e9640 1 --2- 172.21.15.82:0/3516951462 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f40cc102bf0 0x7f40cc1986f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:58.253 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:58.252+0000 7f40d24e9640 1 --2- 172.21.15.82:0/3516951462 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f40cc1019b0 0x7f40cc1981b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:58.253 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:58.252+0000 7f40d24e9640 1 -- 172.21.15.82:0/3516951462 >> 172.21.15.82:0/3516951462 conn(0x7f40cc0fcba0 msgr2=0x7f40cc1038d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:13:58.253 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:58.252+0000 7f40d24e9640 1 -- 172.21.15.82:0/3516951462 shutdown_connections 2024-05-20T16:13:58.253 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:58.252+0000 7f40d24e9640 1 -- 172.21.15.82:0/3516951462 wait complete. 2024-05-20T16:13:58.264 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-20T16:13:58.380 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:58.378+0000 7fb0ae7e0640 1 -- 172.21.15.82:0/3278459536 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb0a81089e0 msgr2=0x7fb0a810ae00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:58.380 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:58.378+0000 7fb0ae7e0640 1 --2- 172.21.15.82:0/3278459536 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb0a81089e0 0x7fb0a810ae00 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fb0a0009ca0 tx=0x7fb0a002a8c0 comp rx=0 tx=0).stop 2024-05-20T16:13:58.380 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:58.378+0000 7fb0ae7e0640 1 -- 172.21.15.82:0/3278459536 shutdown_connections 2024-05-20T16:13:58.380 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:58.378+0000 7fb0ae7e0640 1 --2- 172.21.15.82:0/3278459536 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb0a81089e0 0x7fb0a810ae00 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:58.380 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:58.378+0000 7fb0ae7e0640 1 --2- 172.21.15.82:0/3278459536 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb0a8103ce0 0x7fb0a81040f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:58.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:58.378+0000 7fb0ae7e0640 1 --2- 172.21.15.82:0/3278459536 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb0a8102aa0 0x7fb0a81037a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:58.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:58.378+0000 7fb0ae7e0640 1 -- 172.21.15.82:0/3278459536 >> 172.21.15.82:0/3278459536 conn(0x7fb0a80fdc90 msgr2=0x7fb0a8100140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:13:58.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:58.378+0000 7fb0ae7e0640 1 -- 172.21.15.82:0/3278459536 shutdown_connections 2024-05-20T16:13:58.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:58.378+0000 7fb0ae7e0640 1 -- 172.21.15.82:0/3278459536 wait complete. 2024-05-20T16:13:58.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:58.378+0000 7fb0ae7e0640 1 Processor -- start 2024-05-20T16:13:58.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:58.378+0000 7fb0ae7e0640 1 -- start start 2024-05-20T16:13:58.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:58.379+0000 7fb0ae7e0640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb0a8102aa0 0x7fb0a8199230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:58.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:58.379+0000 7fb0ae7e0640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb0a8103ce0 0x7fb0a8199770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:58.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:58.379+0000 7fb0ae7e0640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb0a81089e0 0x7fb0a81a0700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:58.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:58.379+0000 7fb0ae7e0640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fb0a810e720 con 0x7fb0a8102aa0 2024-05-20T16:13:58.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:58.379+0000 7fb0ae7e0640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fb0a810e5a0 con 0x7fb0a8103ce0 2024-05-20T16:13:58.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:58.379+0000 7fb0ae7e0640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fb0a810e8a0 con 0x7fb0a81089e0 2024-05-20T16:13:58.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:58.379+0000 7fb0a7fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb0a8102aa0 0x7fb0a8199230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:13:58.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:58.379+0000 7fb0a7fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb0a8102aa0 0x7fb0a8199230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:46558/0 (socket says 172.21.15.82:46558) 2024-05-20T16:13:58.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:58.379+0000 7fb0a7fff640 1 -- 172.21.15.82:0/3202861248 learned_addr learned my addr 172.21.15.82:0/3202861248 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:13:58.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:58.379+0000 7fb0a7fff640 1 -- 172.21.15.82:0/3202861248 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb0a81089e0 msgr2=0x7fb0a81a0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:58.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:58.379+0000 7fb0a7fff640 1 --2- 172.21.15.82:0/3202861248 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb0a81089e0 0x7fb0a81a0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:58.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:58.379+0000 7fb0a7fff640 1 -- 172.21.15.82:0/3202861248 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb0a8103ce0 msgr2=0x7fb0a8199770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:58.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:58.379+0000 7fb09ffff640 1 --2- 172.21.15.82:0/3202861248 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb0a8103ce0 0x7fb0a8199770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:13:58.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:58.379+0000 7fb0a7fff640 1 --2- 172.21.15.82:0/3202861248 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb0a8103ce0 0x7fb0a8199770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:58.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:58.379+0000 7fb0a7fff640 1 -- 172.21.15.82:0/3202861248 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0a81a0d70 con 0x7fb0a8102aa0 2024-05-20T16:13:58.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:58.379+0000 7fb09ffff640 1 --2- 172.21.15.82:0/3202861248 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb0a8103ce0 0x7fb0a8199770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T16:13:58.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:58.379+0000 7fb0a7fff640 1 --2- 172.21.15.82:0/3202861248 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb0a8102aa0 0x7fb0a8199230 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fb09800e9f0 tx=0x7fb09800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:13:58.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:58.379+0000 7fb0a5ffb640 1 -- 172.21.15.82:0/3202861248 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb09800ccc0 con 0x7fb0a8102aa0 2024-05-20T16:13:58.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:58.379+0000 7fb0a5ffb640 1 -- 172.21.15.82:0/3202861248 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb0980035f0 con 0x7fb0a8102aa0 2024-05-20T16:13:58.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:58.379+0000 7fb0ae7e0640 1 -- 172.21.15.82:0/3202861248 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0a81a1060 con 0x7fb0a8102aa0 2024-05-20T16:13:58.383 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:58.379+0000 7fb0a5ffb640 1 -- 172.21.15.82:0/3202861248 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb09800ce60 con 0x7fb0a8102aa0 2024-05-20T16:13:58.383 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:58.379+0000 7fb0ae7e0640 1 -- 172.21.15.82:0/3202861248 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0a81a1600 con 0x7fb0a8102aa0 2024-05-20T16:13:58.383 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:58.380+0000 7fb0ae7e0640 1 -- 172.21.15.82:0/3202861248 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0a81077f0 con 0x7fb0a8102aa0 2024-05-20T16:13:58.383 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:58.382+0000 7fb0a5ffb640 1 -- 172.21.15.82:0/3202861248 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fb0980037a0 con 0x7fb0a8102aa0 2024-05-20T16:13:58.384 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:58.382+0000 7fb0a5ffb640 1 --2- 172.21.15.82:0/3202861248 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fb0940a2c10 0x7fb0940a5120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:58.384 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:58.383+0000 7fb09ffff640 1 --2- 172.21.15.82:0/3202861248 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fb0940a2c10 0x7fb0940a5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:13:58.384 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:58.383+0000 7fb0a5ffb640 1 -- 172.21.15.82:0/3202861248 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9150+0+0 (secure 0 0 0) 0x7fb0980cf880 con 0x7fb0a8102aa0 2024-05-20T16:13:58.384 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:58.383+0000 7fb09ffff640 1 --2- 172.21.15.82:0/3202861248 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fb0940a2c10 0x7fb0940a5120 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fb0a819a2d0 tx=0x7fb090005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:13:58.385 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:58.386+0000 7fb0a5ffb640 1 -- 172.21.15.82:0/3202861248 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fb09809c450 con 0x7fb0a8102aa0 2024-05-20T16:13:58.561 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:58.561+0000 7fb0ae7e0640 1 -- 172.21.15.82:0/3202861248 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fb0a802d550 con 0x7fb0a8102aa0 2024-05-20T16:13:58.562 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:58.562+0000 7fb0a5ffb640 1 -- 172.21.15.82:0/3202861248 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v95) ==== 103+0+11 (secure 0 0 0) 0x7fb09809bd50 con 0x7fb0a8102aa0 2024-05-20T16:13:58.562 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 32 2024-05-20T16:13:58.564 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:58.564+0000 7fb0ae7e0640 1 -- 172.21.15.82:0/3202861248 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fb0940a2c10 msgr2=0x7fb0940a5120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:58.564 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:58.564+0000 7fb0ae7e0640 1 --2- 172.21.15.82:0/3202861248 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fb0940a2c10 0x7fb0940a5120 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fb0a819a2d0 tx=0x7fb090005f90 comp rx=0 tx=0).stop 2024-05-20T16:13:58.564 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:58.564+0000 7fb0ae7e0640 1 -- 172.21.15.82:0/3202861248 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb0a8102aa0 msgr2=0x7fb0a8199230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:58.564 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:58.564+0000 7fb0ae7e0640 1 --2- 172.21.15.82:0/3202861248 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb0a8102aa0 0x7fb0a8199230 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fb09800e9f0 tx=0x7fb09800eeb0 comp rx=0 tx=0).stop 2024-05-20T16:13:58.564 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:58.564+0000 7fb0ae7e0640 1 -- 172.21.15.82:0/3202861248 shutdown_connections 2024-05-20T16:13:58.564 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:58.564+0000 7fb0ae7e0640 1 --2- 172.21.15.82:0/3202861248 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fb0940a2c10 0x7fb0940a5120 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:58.564 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:58.564+0000 7fb0ae7e0640 1 --2- 172.21.15.82:0/3202861248 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb0a81089e0 0x7fb0a81a0700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:58.564 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:58.564+0000 7fb0ae7e0640 1 --2- 172.21.15.82:0/3202861248 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb0a8103ce0 0x7fb0a8199770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:58.565 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:58.564+0000 7fb0ae7e0640 1 --2- 172.21.15.82:0/3202861248 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb0a8102aa0 0x7fb0a8199230 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:58.565 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:58.564+0000 7fb0ae7e0640 1 -- 172.21.15.82:0/3202861248 >> 172.21.15.82:0/3202861248 conn(0x7fb0a80fdc90 msgr2=0x7fb0a81049c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:13:58.565 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:58.564+0000 7fb0ae7e0640 1 -- 172.21.15.82:0/3202861248 shutdown_connections 2024-05-20T16:13:58.565 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:58.564+0000 7fb0ae7e0640 1 -- 172.21.15.82:0/3202861248 wait complete. 2024-05-20T16:13:58.574 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:13:58 smithi082 ceph-mon[32238]: from='client.? 172.21.15.82:0/2329424043' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-20T16:13:58.574 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:13:58 smithi082 ceph-mon[32238]: from='client.? 172.21.15.82:0/606565183' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-20T16:13:58.574 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:13:58 smithi082 ceph-mon[32238]: from='client.? 172.21.15.82:0/3516951462' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-20T16:13:58.575 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-20T16:13:58.687 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:58.687+0000 7ff69659d640 1 -- 172.21.15.82:0/751547419 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff690101970 msgr2=0x7ff690102670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:58.687 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:58.687+0000 7ff69659d640 1 --2- 172.21.15.82:0/751547419 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff690101970 0x7ff690102670 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7ff680009900 tx=0x7ff68002a410 comp rx=0 tx=0).stop 2024-05-20T16:13:58.687 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:58.687+0000 7ff69659d640 1 -- 172.21.15.82:0/751547419 shutdown_connections 2024-05-20T16:13:58.687 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:58.687+0000 7ff69659d640 1 --2- 172.21.15.82:0/751547419 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff6901078b0 0x7ff690109cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:58.687 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:58.687+0000 7ff69659d640 1 --2- 172.21.15.82:0/751547419 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff690102bb0 0x7ff690102fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:58.687 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:58.687+0000 7ff69659d640 1 --2- 172.21.15.82:0/751547419 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff690101970 0x7ff690102670 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:58.687 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:58.687+0000 7ff69659d640 1 -- 172.21.15.82:0/751547419 >> 172.21.15.82:0/751547419 conn(0x7ff6900fcb80 msgr2=0x7ff6900ff010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:13:58.688 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:58.687+0000 7ff69659d640 1 -- 172.21.15.82:0/751547419 shutdown_connections 2024-05-20T16:13:58.688 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:58.687+0000 7ff69659d640 1 -- 172.21.15.82:0/751547419 wait complete. 2024-05-20T16:13:58.688 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:58.687+0000 7ff69659d640 1 Processor -- start 2024-05-20T16:13:58.688 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:58.687+0000 7ff69659d640 1 -- start start 2024-05-20T16:13:58.690 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:58.688+0000 7ff69659d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff690101970 0x7ff690199330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:58.690 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:58.688+0000 7ff69659d640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff690102bb0 0x7ff690199870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:58.690 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:58.688+0000 7ff69659d640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff6901078b0 0x7ff6901a0800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:58.690 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:58.688+0000 7ff69659d640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7ff69010d670 con 0x7ff690101970 2024-05-20T16:13:58.690 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:58.688+0000 7ff69659d640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7ff69010d4f0 con 0x7ff6901078b0 2024-05-20T16:13:58.690 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:58.688+0000 7ff69659d640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ff69010d7f0 con 0x7ff690102bb0 2024-05-20T16:13:58.690 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:58.688+0000 7ff68ffff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff690101970 0x7ff690199330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:13:58.690 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:58.688+0000 7ff68ffff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff690101970 0x7ff690199330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:46590/0 (socket says 172.21.15.82:46590) 2024-05-20T16:13:58.690 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:58.688+0000 7ff68ffff640 1 -- 172.21.15.82:0/299233696 learned_addr learned my addr 172.21.15.82:0/299233696 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:13:58.690 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:58.688+0000 7ff687fff640 1 --2- 172.21.15.82:0/299233696 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff690102bb0 0x7ff690199870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:13:58.691 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:58.688+0000 7ff694b3a640 1 --2- 172.21.15.82:0/299233696 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff6901078b0 0x7ff6901a0800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:13:58.691 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:58.688+0000 7ff68ffff640 1 -- 172.21.15.82:0/299233696 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff690102bb0 msgr2=0x7ff690199870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:58.691 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:58.688+0000 7ff68ffff640 1 --2- 172.21.15.82:0/299233696 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff690102bb0 0x7ff690199870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:58.691 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:58.688+0000 7ff68ffff640 1 -- 172.21.15.82:0/299233696 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff6901078b0 msgr2=0x7ff6901a0800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:58.691 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:58.688+0000 7ff68ffff640 1 --2- 172.21.15.82:0/299233696 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff6901078b0 0x7ff6901a0800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:58.691 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:58.688+0000 7ff68ffff640 1 -- 172.21.15.82:0/299233696 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6901a0e70 con 0x7ff690101970 2024-05-20T16:13:58.691 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:58.688+0000 7ff68ffff640 1 --2- 172.21.15.82:0/299233696 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff690101970 0x7ff690199330 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7ff680031440 tx=0x7ff680032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:13:58.691 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:58.688+0000 7ff68dffb640 1 -- 172.21.15.82:0/299233696 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff68003f030 con 0x7ff690101970 2024-05-20T16:13:58.691 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:58.688+0000 7ff68dffb640 1 -- 172.21.15.82:0/299233696 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff680003ce0 con 0x7ff690101970 2024-05-20T16:13:58.691 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:58.688+0000 7ff69659d640 1 -- 172.21.15.82:0/299233696 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6901a1100 con 0x7ff690101970 2024-05-20T16:13:58.691 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:58.688+0000 7ff68dffb640 1 -- 172.21.15.82:0/299233696 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff680035070 con 0x7ff690101970 2024-05-20T16:13:58.691 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:58.689+0000 7ff69659d640 1 -- 172.21.15.82:0/299233696 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6901a1670 con 0x7ff690101970 2024-05-20T16:13:58.692 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:58.689+0000 7ff69659d640 1 -- 172.21.15.82:0/299233696 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff6901a18d0 con 0x7ff690101970 2024-05-20T16:13:58.692 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:58.691+0000 7ff68dffb640 1 -- 172.21.15.82:0/299233696 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7ff680042030 con 0x7ff690101970 2024-05-20T16:13:58.693 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:58.692+0000 7ff68dffb640 1 --2- 172.21.15.82:0/299233696 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7ff67c0a2b90 0x7ff67c0a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:58.693 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:58.692+0000 7ff687fff640 1 --2- 172.21.15.82:0/299233696 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7ff67c0a2b90 0x7ff67c0a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:13:58.693 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:58.692+0000 7ff68dffb640 1 -- 172.21.15.82:0/299233696 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9150+0+0 (secure 0 0 0) 0x7ff6800efe00 con 0x7ff690101970 2024-05-20T16:13:58.693 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:58.693+0000 7ff687fff640 1 --2- 172.21.15.82:0/299233696 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7ff67c0a2b90 0x7ff67c0a50a0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7ff678005fd0 tx=0x7ff678005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:13:58.695 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:58.696+0000 7ff68dffb640 1 -- 172.21.15.82:0/299233696 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7ff6800bc9d0 con 0x7ff690101970 2024-05-20T16:13:58.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:13:58 smithi160 ceph-mon[36347]: from='client.? 172.21.15.82:0/2329424043' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-20T16:13:58.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:13:58 smithi160 ceph-mon[36347]: from='client.? 172.21.15.82:0/606565183' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-20T16:13:58.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:13:58 smithi160 ceph-mon[36347]: from='client.? 172.21.15.82:0/3516951462' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-20T16:13:58.885 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:13:58 smithi184 ceph-mon[36209]: from='client.? 172.21.15.82:0/2329424043' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-20T16:13:58.885 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:13:58 smithi184 ceph-mon[36209]: from='client.? 172.21.15.82:0/606565183' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-20T16:13:58.886 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:13:58 smithi184 ceph-mon[36209]: from='client.? 172.21.15.82:0/3516951462' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-20T16:13:58.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:58.909+0000 7ff69659d640 1 -- 172.21.15.82:0/299233696 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff6901a1aa0 con 0x7ff690101970 2024-05-20T16:13:58.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:58.910+0000 7ff68dffb640 1 -- 172.21.15.82:0/299233696 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11879 (secure 0 0 0) 0x7ff6800bc2d0 con 0x7ff690101970 2024-05-20T16:13:58.910 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-05-20T16:13:58.910 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/2044534217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":2044534217},{"type":"v1","addr":"172.21.15.82:6835","nonce":2044534217}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/2635635097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":2635635097},{"type":"v1","addr":"172.21.15.82:6837","nonce":2635635097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14625,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6839/2118233541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6838","nonce":2118233541},{"type":"v1","addr":"172.21.15.82:6839","nonce":2118233541}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14637,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6841/2119517973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6840","nonce":2119517973},{"type":"v1","addr":"172.21.15.82:6841","nonce":2119517973}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6833/4015144007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":4015144007},{"type":"v1","addr":"172.21.15.184:6833","nonce":4015144007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6833/975243372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6832","nonce":975243372},{"type":"v1","addr":"172.21.15.160:6833","nonce":975243372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-20T16:11:46.751632+0000","modified":"2024-05-20T16:12:13.394770+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24548,"mds_1":24511,"mds_2":24506},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24470":{"gid":24470,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.160:6835/664068363","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":664068363},{"type":"v1","addr":"172.21.15.160:6835","nonce":664068363}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6835/4242156090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":4242156090},{"type":"v1","addr":"172.21.15.184:6835","nonce":4242156090}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6837/403262242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":403262242},{"type":"v1","addr":"172.21.15.184:6837","nonce":403262242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.160:6837/2846188680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6836","nonce":2846188680},{"type":"v1","addr":"172.21.15.160:6837","nonce":2846188680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.184:6839/2199955067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":2199955067},{"type":"v1","addr":"172.21.15.184:6839","nonce":2199955067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24548":{"gid":24548,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.160:6839/2765554482","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6838","nonce":2765554482},{"type":"v1","addr":"172.21.15.160:6839","nonce":2765554482}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24548,"qdb_cluster":[24511,24506,24548]},"id":1}]} 2024-05-20T16:13:58.910 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 24 2024-05-20T16:13:58.912 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:58.912+0000 7ff69659d640 1 -- 172.21.15.82:0/299233696 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7ff67c0a2b90 msgr2=0x7ff67c0a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:58.912 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:58.912+0000 7ff69659d640 1 --2- 172.21.15.82:0/299233696 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7ff67c0a2b90 0x7ff67c0a50a0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7ff678005fd0 tx=0x7ff678005850 comp rx=0 tx=0).stop 2024-05-20T16:13:58.912 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:58.912+0000 7ff69659d640 1 -- 172.21.15.82:0/299233696 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff690101970 msgr2=0x7ff690199330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:58.913 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:58.912+0000 7ff69659d640 1 --2- 172.21.15.82:0/299233696 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff690101970 0x7ff690199330 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7ff680031440 tx=0x7ff680032ce0 comp rx=0 tx=0).stop 2024-05-20T16:13:58.913 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:58.912+0000 7ff69659d640 1 -- 172.21.15.82:0/299233696 shutdown_connections 2024-05-20T16:13:58.913 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:58.912+0000 7ff69659d640 1 --2- 172.21.15.82:0/299233696 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7ff67c0a2b90 0x7ff67c0a50a0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:58.913 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:58.912+0000 7ff69659d640 1 --2- 172.21.15.82:0/299233696 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff6901078b0 0x7ff6901a0800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:58.913 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:58.912+0000 7ff69659d640 1 --2- 172.21.15.82:0/299233696 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff690102bb0 0x7ff690199870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:58.913 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:58.912+0000 7ff69659d640 1 --2- 172.21.15.82:0/299233696 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff690101970 0x7ff690199330 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:58.913 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:58.912+0000 7ff69659d640 1 -- 172.21.15.82:0/299233696 >> 172.21.15.82:0/299233696 conn(0x7ff6900fcb80 msgr2=0x7ff690108b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:13:58.913 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:58.912+0000 7ff69659d640 1 -- 172.21.15.82:0/299233696 shutdown_connections 2024-05-20T16:13:58.913 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:58.912+0000 7ff69659d640 1 -- 172.21.15.82:0/299233696 wait complete. 2024-05-20T16:13:58.924 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-20T16:13:59.036 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:59.034+0000 7f3da01b5640 1 -- 172.21.15.82:0/1878312037 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3d981078d0 msgr2=0x7f3d98109cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:59.036 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:59.034+0000 7f3da01b5640 1 --2- 172.21.15.82:0/1878312037 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3d981078d0 0x7f3d98109cf0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f3d90009ca0 tx=0x7f3d9002a8c0 comp rx=0 tx=0).stop 2024-05-20T16:13:59.036 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:59.034+0000 7f3da01b5640 1 -- 172.21.15.82:0/1878312037 shutdown_connections 2024-05-20T16:13:59.037 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:59.034+0000 7f3da01b5640 1 --2- 172.21.15.82:0/1878312037 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3d981078d0 0x7f3d98109cf0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:59.037 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:59.034+0000 7f3da01b5640 1 --2- 172.21.15.82:0/1878312037 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3d98102bd0 0x7f3d98102fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:59.037 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:59.034+0000 7f3da01b5640 1 --2- 172.21.15.82:0/1878312037 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3d98101990 0x7f3d98102690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:59.037 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:59.034+0000 7f3da01b5640 1 -- 172.21.15.82:0/1878312037 >> 172.21.15.82:0/1878312037 conn(0x7f3d980fcb80 msgr2=0x7f3d980ff030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:13:59.037 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:59.034+0000 7f3da01b5640 1 -- 172.21.15.82:0/1878312037 shutdown_connections 2024-05-20T16:13:59.037 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:59.034+0000 7f3da01b5640 1 -- 172.21.15.82:0/1878312037 wait complete. 2024-05-20T16:13:59.037 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:59.035+0000 7f3da01b5640 1 Processor -- start 2024-05-20T16:13:59.037 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:59.035+0000 7f3da01b5640 1 -- start start 2024-05-20T16:13:59.037 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:59.035+0000 7f3da01b5640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3d98101990 0x7f3d98074780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:59.037 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:59.035+0000 7f3da01b5640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3d98102bd0 0x7f3d9819e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:59.037 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:59.035+0000 7f3da01b5640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3d981078d0 0x7f3d981a0600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:59.037 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:59.035+0000 7f3da01b5640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f3d9810d5f0 con 0x7f3d981078d0 2024-05-20T16:13:59.038 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:59.035+0000 7f3da01b5640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f3d9810d470 con 0x7f3d98101990 2024-05-20T16:13:59.038 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:59.035+0000 7f3da01b5640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f3d9810d770 con 0x7f3d98102bd0 2024-05-20T16:13:59.038 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:59.035+0000 7f3d9e752640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3d981078d0 0x7f3d981a0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:13:59.038 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:59.035+0000 7f3d9e752640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3d981078d0 0x7f3d981a0600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:46610/0 (socket says 172.21.15.82:46610) 2024-05-20T16:13:59.038 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:59.035+0000 7f3d9e752640 1 -- 172.21.15.82:0/3507652997 learned_addr learned my addr 172.21.15.82:0/3507652997 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:13:59.038 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:59.036+0000 7f3d9df51640 1 --2- 172.21.15.82:0/3507652997 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3d98101990 0x7f3d98074780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:13:59.038 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:59.036+0000 7f3d9d750640 1 --2- 172.21.15.82:0/3507652997 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3d98102bd0 0x7f3d9819e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:13:59.038 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:59.036+0000 7f3d9e752640 1 -- 172.21.15.82:0/3507652997 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3d98102bd0 msgr2=0x7f3d9819e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:59.038 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:59.036+0000 7f3d9e752640 1 --2- 172.21.15.82:0/3507652997 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3d98102bd0 0x7f3d9819e0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:59.038 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:59.036+0000 7f3d9e752640 1 -- 172.21.15.82:0/3507652997 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3d98101990 msgr2=0x7f3d98074780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:59.038 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:59.036+0000 7f3d9e752640 1 --2- 172.21.15.82:0/3507652997 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3d98101990 0x7f3d98074780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:59.038 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:59.036+0000 7f3d9e752640 1 -- 172.21.15.82:0/3507652997 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d981a0dd0 con 0x7f3d981078d0 2024-05-20T16:13:59.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:59.036+0000 7f3d9d750640 1 --2- 172.21.15.82:0/3507652997 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3d98102bd0 0x7f3d9819e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T16:13:59.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:59.036+0000 7f3d9e752640 1 --2- 172.21.15.82:0/3507652997 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3d981078d0 0x7f3d981a0600 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f3d90031ee0 tx=0x7f3d90005df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:13:59.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:59.036+0000 7f3d8effd640 1 -- 172.21.15.82:0/3507652997 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d90003b70 con 0x7f3d981078d0 2024-05-20T16:13:59.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:59.036+0000 7f3da01b5640 1 -- 172.21.15.82:0/3507652997 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d981a1060 con 0x7f3d981078d0 2024-05-20T16:13:59.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:59.036+0000 7f3d8effd640 1 -- 172.21.15.82:0/3507652997 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3d90003d10 con 0x7f3d981078d0 2024-05-20T16:13:59.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:59.036+0000 7f3d8effd640 1 -- 172.21.15.82:0/3507652997 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d9002fb40 con 0x7f3d981078d0 2024-05-20T16:13:59.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:59.036+0000 7f3da01b5640 1 -- 172.21.15.82:0/3507652997 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d981a15b0 con 0x7f3d981078d0 2024-05-20T16:13:59.040 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:59.037+0000 7f3da01b5640 1 -- 172.21.15.82:0/3507652997 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d981066e0 con 0x7f3d981078d0 2024-05-20T16:13:59.040 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:59.038+0000 7f3d8effd640 1 -- 172.21.15.82:0/3507652997 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f3d90004630 con 0x7f3d981078d0 2024-05-20T16:13:59.040 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:59.039+0000 7f3d8effd640 1 --2- 172.21.15.82:0/3507652997 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f3d840a2b90 0x7f3d840a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:59.040 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:59.039+0000 7f3d8effd640 1 -- 172.21.15.82:0/3507652997 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9150+0+0 (secure 0 0 0) 0x7f3d900f0650 con 0x7f3d981078d0 2024-05-20T16:13:59.040 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:59.039+0000 7f3d9df51640 1 --2- 172.21.15.82:0/3507652997 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f3d840a2b90 0x7f3d840a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:13:59.040 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:59.040+0000 7f3d9df51640 1 --2- 172.21.15.82:0/3507652997 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f3d840a2b90 0x7f3d840a50a0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f3d98102590 tx=0x7f3d94005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:13:59.042 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:59.043+0000 7f3d8effd640 1 -- 172.21.15.82:0/3507652997 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f3d900bd220 con 0x7f3d981078d0 2024-05-20T16:13:59.219 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:59.219+0000 7f3da01b5640 1 -- 172.21.15.82:0/3507652997 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3d98062750 con 0x7f3d981078d0 2024-05-20T16:13:59.221 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:59.221+0000 7f3d8effd640 1 -- 172.21.15.82:0/3507652997 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) ==== 74+0+24771 (secure 0 0 0) 0x7f3d900bcb20 con 0x7f3d981078d0 2024-05-20T16:13:59.221 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-05-20T16:13:59.221 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":95,"fsid":"59d1600a-16c2-11ef-bc9a-c7b262605968","created":"2024-05-20T16:03:46.611556+0000","modified":"2024-05-20T16:13:46.328150+0000","last_up_change":"2024-05-20T16:10:59.303479+0000","last_in_change":"2024-05-20T16:10:39.063561+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-20T16:07:09.493727+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-20T16:11:45.058035+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-20T16:11:45.943862+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-20T16:11:49.637579+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"bae842a9-03ec-490e-b56d-f7957235cdc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":1484330167},{"type":"v1","addr":"172.21.15.82:6803","nonce":1484330167}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":1484330167},{"type":"v1","addr":"172.21.15.82:6805","nonce":1484330167}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1484330167},{"type":"v1","addr":"172.21.15.82:6809","nonce":1484330167}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":1484330167},{"type":"v1","addr":"172.21.15.82:6807","nonce":1484330167}]},"public_addr":"172.21.15.82:6803/1484330167","cluster_addr":"172.21.15.82:6805/1484330167","heartbeat_back_addr":"172.21.15.82:6809/1484330167","heartbeat_front_addr":"172.21.15.82:6807/1484330167","state":["exists","up"]},{"osd":1,"uuid":"59f6a2ba-b5de-4161-81c5-92a417abacdf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":1457003568},{"type":"v1","addr":"172.21.15.160:6801","nonce":1457003568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":1457003568},{"type":"v1","addr":"172.21.15.160:6803","nonce":1457003568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6806","nonce":1457003568},{"type":"v1","addr":"172.21.15.160:6807","nonce":1457003568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":1457003568},{"type":"v1","addr":"172.21.15.160:6805","nonce":1457003568}]},"public_addr":"172.21.15.160:6801/1457003568","cluster_addr":"172.21.15.160:6803/1457003568","heartbeat_back_addr":"172.21.15.160:6807/1457003568","heartbeat_front_addr":"172.21.15.160:6805/1457003568","state":["exists","up"]},{"osd":2,"uuid":"3bbb5587-b449-41f3-bd26-67ec3d6afdfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":4051293934},{"type":"v1","addr":"172.21.15.184:6801","nonce":4051293934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":4051293934},{"type":"v1","addr":"172.21.15.184:6803","nonce":4051293934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":4051293934},{"type":"v1","addr":"172.21.15.184:6807","nonce":4051293934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":4051293934},{"type":"v1","addr":"172.21.15.184:6805","nonce":4051293934}]},"public_addr":"172.21.15.184:6801/4051293934","cluster_addr":"172.21.15.184:6803/4051293934","heartbeat_back_addr":"172.21.15.184:6807/4051293934","heartbeat_front_addr":"172.21.15.184:6805/4051293934","state":["exists","up"]},{"osd":3,"uuid":"ca651f90-09b8-4081-a6f9-40b31268a837","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":3947485599},{"type":"v1","addr":"172.21.15.82:6811","nonce":3947485599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":3947485599},{"type":"v1","addr":"172.21.15.82:6813","nonce":3947485599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":3947485599},{"type":"v1","addr":"172.21.15.82:6817","nonce":3947485599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":3947485599},{"type":"v1","addr":"172.21.15.82:6815","nonce":3947485599}]},"public_addr":"172.21.15.82:6811/3947485599","cluster_addr":"172.21.15.82:6813/3947485599","heartbeat_back_addr":"172.21.15.82:6817/3947485599","heartbeat_front_addr":"172.21.15.82:6815/3947485599","state":["exists","up"]},{"osd":4,"uuid":"cb3dfd56-f677-40cb-9c96-8319ea5ac788","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":4160830014},{"type":"v1","addr":"172.21.15.160:6809","nonce":4160830014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":4160830014},{"type":"v1","addr":"172.21.15.160:6811","nonce":4160830014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":4160830014},{"type":"v1","addr":"172.21.15.160:6815","nonce":4160830014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6812","nonce":4160830014},{"type":"v1","addr":"172.21.15.160:6813","nonce":4160830014}]},"public_addr":"172.21.15.160:6809/4160830014","cluster_addr":"172.21.15.160:6811/4160830014","heartbeat_back_addr":"172.21.15.160:6815/4160830014","heartbeat_front_addr":"172.21.15.160:6813/4160830014","state":["exists","up"]},{"osd":5,"uuid":"c70cfcd1-a4ff-461f-8c73-479b061200fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":1046282672},{"type":"v1","addr":"172.21.15.184:6809","nonce":1046282672}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":1046282672},{"type":"v1","addr":"172.21.15.184:6811","nonce":1046282672}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":1046282672},{"type":"v1","addr":"172.21.15.184:6815","nonce":1046282672}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":1046282672},{"type":"v1","addr":"172.21.15.184:6813","nonce":1046282672}]},"public_addr":"172.21.15.184:6809/1046282672","cluster_addr":"172.21.15.184:6811/1046282672","heartbeat_back_addr":"172.21.15.184:6815/1046282672","heartbeat_front_addr":"172.21.15.184:6813/1046282672","state":["exists","up"]},{"osd":6,"uuid":"d47ce001-9dd1-4830-bcc9-888601187c6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":2789736499},{"type":"v1","addr":"172.21.15.82:6819","nonce":2789736499}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":2789736499},{"type":"v1","addr":"172.21.15.82:6821","nonce":2789736499}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":2789736499},{"type":"v1","addr":"172.21.15.82:6825","nonce":2789736499}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":2789736499},{"type":"v1","addr":"172.21.15.82:6823","nonce":2789736499}]},"public_addr":"172.21.15.82:6819/2789736499","cluster_addr":"172.21.15.82:6821/2789736499","heartbeat_back_addr":"172.21.15.82:6825/2789736499","heartbeat_front_addr":"172.21.15.82:6823/2789736499","state":["exists","up"]},{"osd":7,"uuid":"25c54f43-18cb-41e4-83b4-2732b5a93889","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6816","nonce":2340683378},{"type":"v1","addr":"172.21.15.160:6817","nonce":2340683378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":2340683378},{"type":"v1","addr":"172.21.15.160:6819","nonce":2340683378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":2340683378},{"type":"v1","addr":"172.21.15.160:6823","nonce":2340683378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6820","nonce":2340683378},{"type":"v1","addr":"172.21.15.160:6821","nonce":2340683378}]},"public_addr":"172.21.15.160:6817/2340683378","cluster_addr":"172.21.15.160:6819/2340683378","heartbeat_back_addr":"172.21.15.160:6823/2340683378","heartbeat_front_addr":"172.21.15.160:6821/2340683378","state":["exists","up"]},{"osd":8,"uuid":"1f8b478d-287f-45f5-8526-d1a58c5bc07c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":3831576372},{"type":"v1","addr":"172.21.15.184:6817","nonce":3831576372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":3831576372},{"type":"v1","addr":"172.21.15.184:6819","nonce":3831576372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":3831576372},{"type":"v1","addr":"172.21.15.184:6823","nonce":3831576372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":3831576372},{"type":"v1","addr":"172.21.15.184:6821","nonce":3831576372}]},"public_addr":"172.21.15.184:6817/3831576372","cluster_addr":"172.21.15.184:6819/3831576372","heartbeat_back_addr":"172.21.15.184:6823/3831576372","heartbeat_front_addr":"172.21.15.184:6821/3831576372","state":["exists","up"]},{"osd":9,"uuid":"0d69e319-954f-4b18-95ea-31794f18dc3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":716330782},{"type":"v1","addr":"172.21.15.82:6827","nonce":716330782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":716330782},{"type":"v1","addr":"172.21.15.82:6829","nonce":716330782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":716330782},{"type":"v1","addr":"172.21.15.82:6833","nonce":716330782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":716330782},{"type":"v1","addr":"172.21.15.82:6831","nonce":716330782}]},"public_addr":"172.21.15.82:6827/716330782","cluster_addr":"172.21.15.82:6829/716330782","heartbeat_back_addr":"172.21.15.82:6833/716330782","heartbeat_front_addr":"172.21.15.82:6831/716330782","state":["exists","up"]},{"osd":10,"uuid":"39dc1b78-61cb-42ca-b5b6-1e44ac2b6500","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":2327428014},{"type":"v1","addr":"172.21.15.160:6825","nonce":2327428014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":2327428014},{"type":"v1","addr":"172.21.15.160:6827","nonce":2327428014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":2327428014},{"type":"v1","addr":"172.21.15.160:6831","nonce":2327428014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":2327428014},{"type":"v1","addr":"172.21.15.160:6829","nonce":2327428014}]},"public_addr":"172.21.15.160:6825/2327428014","cluster_addr":"172.21.15.160:6827/2327428014","heartbeat_back_addr":"172.21.15.160:6831/2327428014","heartbeat_front_addr":"172.21.15.160:6829/2327428014","state":["exists","up"]},{"osd":11,"uuid":"4306b3c1-707a-4010-987c-209ea0e948e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":3594364650},{"type":"v1","addr":"172.21.15.184:6825","nonce":3594364650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":3594364650},{"type":"v1","addr":"172.21.15.184:6827","nonce":3594364650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3594364650},{"type":"v1","addr":"172.21.15.184:6831","nonce":3594364650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3594364650},{"type":"v1","addr":"172.21.15.184:6829","nonce":3594364650}]},"public_addr":"172.21.15.184:6825/3594364650","cluster_addr":"172.21.15.184:6827/3594364650","heartbeat_back_addr":"172.21.15.184:6831/3594364650","heartbeat_front_addr":"172.21.15.184:6829/3594364650","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:06:17.855998+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:06:41.473270+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:07:05.304108+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:07:29.887472+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:07:55.387286+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:08:20.579669+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:08:46.333051+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:09:12.289668+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:09:38.180696+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:10:04.471459+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:10:31.041912+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:10:57.477266+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.1d","mappings":[{"from":5,"to":9}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.5","mappings":[{"from":9,"to":10}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.82:0/2005424509":"2024-05-21T16:04:45.161949+0000","172.21.15.82:6801/1763049308":"2024-05-21T16:04:45.161949+0000","172.21.15.82:6801/1959459348":"2024-05-21T16:04:19.208291+0000","172.21.15.82:0/3131904556":"2024-05-21T16:04:19.208291+0000","172.21.15.82:6800/1959459348":"2024-05-21T16:04:19.208291+0000","172.21.15.82:0/2456966342":"2024-05-21T16:04:45.161949+0000","172.21.15.82:0/4179978833":"2024-05-21T16:04:19.208291+0000","172.21.15.82:0/1602862225":"2024-05-21T16:04:45.161949+0000","172.21.15.82:0/106101180":"2024-05-21T16:04:19.208291+0000","172.21.15.82:6800/1763049308":"2024-05-21T16:04:45.161949+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-20T16:13:59.223 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:59.223+0000 7f3da01b5640 1 -- 172.21.15.82:0/3507652997 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f3d840a2b90 msgr2=0x7f3d840a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:59.223 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:59.223+0000 7f3da01b5640 1 --2- 172.21.15.82:0/3507652997 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f3d840a2b90 0x7f3d840a50a0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f3d98102590 tx=0x7f3d94005f90 comp rx=0 tx=0).stop 2024-05-20T16:13:59.223 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:59.223+0000 7f3da01b5640 1 -- 172.21.15.82:0/3507652997 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3d981078d0 msgr2=0x7f3d981a0600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:59.223 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:59.223+0000 7f3da01b5640 1 --2- 172.21.15.82:0/3507652997 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3d981078d0 0x7f3d981a0600 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f3d90031ee0 tx=0x7f3d90005df0 comp rx=0 tx=0).stop 2024-05-20T16:13:59.223 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:59.223+0000 7f3da01b5640 1 -- 172.21.15.82:0/3507652997 shutdown_connections 2024-05-20T16:13:59.224 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:59.223+0000 7f3da01b5640 1 --2- 172.21.15.82:0/3507652997 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f3d840a2b90 0x7f3d840a50a0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:59.224 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:59.223+0000 7f3da01b5640 1 --2- 172.21.15.82:0/3507652997 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3d981078d0 0x7f3d981a0600 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:59.224 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:59.223+0000 7f3da01b5640 1 --2- 172.21.15.82:0/3507652997 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3d98102bd0 0x7f3d9819e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:59.224 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:59.223+0000 7f3da01b5640 1 --2- 172.21.15.82:0/3507652997 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3d98101990 0x7f3d98074780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:59.224 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:59.223+0000 7f3da01b5640 1 -- 172.21.15.82:0/3507652997 >> 172.21.15.82:0/3507652997 conn(0x7f3d980fcb80 msgr2=0x7f3d981038b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:13:59.224 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:59.223+0000 7f3da01b5640 1 -- 172.21.15.82:0/3507652997 shutdown_connections 2024-05-20T16:13:59.224 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:59.223+0000 7f3da01b5640 1 -- 172.21.15.82:0/3507652997 wait complete. 2024-05-20T16:13:59.235 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-20T16:07:09.493727+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-20T16:11:45.058035+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-20T16:11:45.943862+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-20T16:11:49.637579+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '91', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-20T16:13:59.235 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-20T16:13:59.350 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:59.350+0000 7fb150c3f640 1 -- 172.21.15.82:0/1787224179 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb14c1078d0 msgr2=0x7fb14c109cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:59.350 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:59.350+0000 7fb150c3f640 1 --2- 172.21.15.82:0/1787224179 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb14c1078d0 0x7fb14c109cf0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fb144009ca0 tx=0x7fb14402a8c0 comp rx=0 tx=0).stop 2024-05-20T16:13:59.350 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:59.350+0000 7fb150c3f640 1 -- 172.21.15.82:0/1787224179 shutdown_connections 2024-05-20T16:13:59.351 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:59.350+0000 7fb150c3f640 1 --2- 172.21.15.82:0/1787224179 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb14c1078d0 0x7fb14c109cf0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:59.351 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:59.350+0000 7fb150c3f640 1 --2- 172.21.15.82:0/1787224179 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb14c102bd0 0x7fb14c102fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:59.351 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:59.350+0000 7fb150c3f640 1 --2- 172.21.15.82:0/1787224179 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb14c101990 0x7fb14c102690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:59.351 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:59.350+0000 7fb150c3f640 1 -- 172.21.15.82:0/1787224179 >> 172.21.15.82:0/1787224179 conn(0x7fb14c0fcb80 msgr2=0x7fb14c0ff030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:13:59.351 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:59.350+0000 7fb150c3f640 1 -- 172.21.15.82:0/1787224179 shutdown_connections 2024-05-20T16:13:59.351 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:59.350+0000 7fb150c3f640 1 -- 172.21.15.82:0/1787224179 wait complete. 2024-05-20T16:13:59.353 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:59.351+0000 7fb150c3f640 1 Processor -- start 2024-05-20T16:13:59.353 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:59.351+0000 7fb150c3f640 1 -- start start 2024-05-20T16:13:59.353 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:59.351+0000 7fb150c3f640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb14c101990 0x7fb14c0747f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:59.353 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:59.351+0000 7fb150c3f640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb14c102bd0 0x7fb14c19dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:59.353 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:59.351+0000 7fb150c3f640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb14c1078d0 0x7fb14c1a0530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:59.353 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:59.351+0000 7fb150c3f640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fb14c10d5f0 con 0x7fb14c101990 2024-05-20T16:13:59.353 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:59.351+0000 7fb150c3f640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fb14c10d470 con 0x7fb14c102bd0 2024-05-20T16:13:59.354 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:59.351+0000 7fb150c3f640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fb14c10d770 con 0x7fb14c1078d0 2024-05-20T16:13:59.354 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:59.351+0000 7fb14a59c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb14c101990 0x7fb14c0747f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:13:59.354 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:59.351+0000 7fb14a59c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb14c101990 0x7fb14c0747f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:46620/0 (socket says 172.21.15.82:46620) 2024-05-20T16:13:59.354 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:59.351+0000 7fb14a59c640 1 -- 172.21.15.82:0/2735827395 learned_addr learned my addr 172.21.15.82:0/2735827395 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:13:59.354 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:59.352+0000 7fb14a59c640 1 -- 172.21.15.82:0/2735827395 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb14c1078d0 msgr2=0x7fb14c1a0530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:59.354 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:59.352+0000 7fb14a59c640 1 --2- 172.21.15.82:0/2735827395 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb14c1078d0 0x7fb14c1a0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:59.354 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:59.352+0000 7fb14a59c640 1 -- 172.21.15.82:0/2735827395 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb14c102bd0 msgr2=0x7fb14c19dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:59.354 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:59.352+0000 7fb14a59c640 1 --2- 172.21.15.82:0/2735827395 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb14c102bd0 0x7fb14c19dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:59.354 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:59.352+0000 7fb14a59c640 1 -- 172.21.15.82:0/2735827395 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb14c1a0cc0 con 0x7fb14c101990 2024-05-20T16:13:59.354 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:59.352+0000 7fb14a59c640 1 --2- 172.21.15.82:0/2735827395 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb14c101990 0x7fb14c0747f0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fb13c00c970 tx=0x7fb13c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:13:59.354 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:59.352+0000 7fb1437fe640 1 -- 172.21.15.82:0/2735827395 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb13c007c70 con 0x7fb14c101990 2024-05-20T16:13:59.354 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:59.352+0000 7fb150c3f640 1 -- 172.21.15.82:0/2735827395 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb14c1a0fb0 con 0x7fb14c101990 2024-05-20T16:13:59.354 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:59.352+0000 7fb1437fe640 1 -- 172.21.15.82:0/2735827395 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb13c007e10 con 0x7fb14c101990 2024-05-20T16:13:59.355 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:59.352+0000 7fb1437fe640 1 -- 172.21.15.82:0/2735827395 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb13c0188c0 con 0x7fb14c101990 2024-05-20T16:13:59.355 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:59.352+0000 7fb150c3f640 1 -- 172.21.15.82:0/2735827395 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb14c1a1470 con 0x7fb14c101990 2024-05-20T16:13:59.355 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:59.353+0000 7fb150c3f640 1 -- 172.21.15.82:0/2735827395 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb14c1066e0 con 0x7fb14c101990 2024-05-20T16:13:59.355 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:59.354+0000 7fb1437fe640 1 -- 172.21.15.82:0/2735827395 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fb13c018a60 con 0x7fb14c101990 2024-05-20T16:13:59.356 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:59.355+0000 7fb1437fe640 1 --2- 172.21.15.82:0/2735827395 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fb1380a2b90 0x7fb1380a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:59.356 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:59.355+0000 7fb143fff640 1 --2- 172.21.15.82:0/2735827395 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fb1380a2b90 0x7fb1380a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:13:59.356 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:59.355+0000 7fb1437fe640 1 -- 172.21.15.82:0/2735827395 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9150+0+0 (secure 0 0 0) 0x7fb13c0cf5d0 con 0x7fb14c101990 2024-05-20T16:13:59.356 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:59.355+0000 7fb143fff640 1 --2- 172.21.15.82:0/2735827395 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fb1380a2b90 0x7fb1380a50a0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fb14c074190 tx=0x7fb134005c70 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:13:59.358 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:59.358+0000 7fb1437fe640 1 -- 172.21.15.82:0/2735827395 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fb13c014030 con 0x7fb14c101990 2024-05-20T16:13:59.542 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:59.542+0000 7fb150c3f640 1 -- 172.21.15.82:0/2735827395 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fb14c02d450 con 0x7fb14c101990 2024-05-20T16:13:59.542 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:59.542+0000 7fb1437fe640 1 -- 172.21.15.82:0/2735827395 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v95) ==== 93+0+10 (secure 0 0 0) 0x7fb13c09bbe0 con 0x7fb14c101990 2024-05-20T16:13:59.542 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 1 2024-05-20T16:13:59.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:59.544+0000 7fb150c3f640 1 -- 172.21.15.82:0/2735827395 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fb1380a2b90 msgr2=0x7fb1380a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:59.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:59.544+0000 7fb150c3f640 1 --2- 172.21.15.82:0/2735827395 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fb1380a2b90 0x7fb1380a50a0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fb14c074190 tx=0x7fb134005c70 comp rx=0 tx=0).stop 2024-05-20T16:13:59.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:59.544+0000 7fb150c3f640 1 -- 172.21.15.82:0/2735827395 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb14c101990 msgr2=0x7fb14c0747f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:59.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:59.544+0000 7fb150c3f640 1 --2- 172.21.15.82:0/2735827395 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb14c101990 0x7fb14c0747f0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fb13c00c970 tx=0x7fb13c00ce30 comp rx=0 tx=0).stop 2024-05-20T16:13:59.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:59.544+0000 7fb150c3f640 1 -- 172.21.15.82:0/2735827395 shutdown_connections 2024-05-20T16:13:59.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:59.544+0000 7fb150c3f640 1 --2- 172.21.15.82:0/2735827395 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fb1380a2b90 0x7fb1380a50a0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:59.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:59.544+0000 7fb150c3f640 1 --2- 172.21.15.82:0/2735827395 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb14c1078d0 0x7fb14c1a0530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:59.545 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:59.544+0000 7fb150c3f640 1 --2- 172.21.15.82:0/2735827395 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb14c102bd0 0x7fb14c19dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:59.545 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:59.544+0000 7fb150c3f640 1 --2- 172.21.15.82:0/2735827395 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb14c101990 0x7fb14c0747f0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:59.545 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:59.544+0000 7fb150c3f640 1 -- 172.21.15.82:0/2735827395 >> 172.21.15.82:0/2735827395 conn(0x7fb14c0fcb80 msgr2=0x7fb14c1038b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:13:59.545 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:59.544+0000 7fb150c3f640 1 -- 172.21.15.82:0/2735827395 shutdown_connections 2024-05-20T16:13:59.545 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:59.544+0000 7fb150c3f640 1 -- 172.21.15.82:0/2735827395 wait complete. 2024-05-20T16:13:59.555 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:13:59 smithi082 ceph-mon[32238]: from='client.? 172.21.15.82:0/3202861248' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-20T16:13:59.555 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:13:59 smithi082 ceph-mon[32238]: from='client.? 172.21.15.82:0/299233696' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T16:13:59.555 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:13:59 smithi082 ceph-mon[32238]: from='client.? 172.21.15.82:0/3507652997' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-20T16:13:59.555 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:13:59 smithi082 ceph-mon[32238]: pgmap v362: 129 pgs: 129 active+clean; 641 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s; 9 B/s, 0 objects/s recovering 2024-05-20T16:13:59.556 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-20T16:13:59.673 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:59.670+0000 7fc89f138640 1 -- 172.21.15.82:0/1086041231 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc8981078e0 msgr2=0x7fc898109d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:59.673 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:59.670+0000 7fc89f138640 1 --2- 172.21.15.82:0/1086041231 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc8981078e0 0x7fc898109d00 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fc890009900 tx=0x7fc89002a410 comp rx=0 tx=0).stop 2024-05-20T16:13:59.673 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:59.670+0000 7fc89f138640 1 -- 172.21.15.82:0/1086041231 shutdown_connections 2024-05-20T16:13:59.673 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:59.670+0000 7fc89f138640 1 --2- 172.21.15.82:0/1086041231 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc8981078e0 0x7fc898109d00 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:59.673 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:59.670+0000 7fc89f138640 1 --2- 172.21.15.82:0/1086041231 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc898102be0 0x7fc898102ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:59.674 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:59.670+0000 7fc89f138640 1 --2- 172.21.15.82:0/1086041231 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc8981019a0 0x7fc8981026a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:59.674 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:59.670+0000 7fc89f138640 1 -- 172.21.15.82:0/1086041231 >> 172.21.15.82:0/1086041231 conn(0x7fc8980fcb70 msgr2=0x7fc8980ff040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:13:59.674 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:59.670+0000 7fc89f138640 1 -- 172.21.15.82:0/1086041231 shutdown_connections 2024-05-20T16:13:59.674 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:59.671+0000 7fc89f138640 1 -- 172.21.15.82:0/1086041231 wait complete. 2024-05-20T16:13:59.674 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:59.671+0000 7fc89f138640 1 Processor -- start 2024-05-20T16:13:59.674 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:59.671+0000 7fc89f138640 1 -- start start 2024-05-20T16:13:59.674 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:59.671+0000 7fc89f138640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc8981019a0 0x7fc898199220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:59.674 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:59.671+0000 7fc89f138640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc898102be0 0x7fc898199760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:59.674 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:59.671+0000 7fc89f138640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc8981078e0 0x7fc8981a06f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:59.674 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:59.671+0000 7fc89f138640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fc89810e780 con 0x7fc8981078e0 2024-05-20T16:13:59.674 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:59.671+0000 7fc89f138640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fc89810e600 con 0x7fc898102be0 2024-05-20T16:13:59.675 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:59.671+0000 7fc89f138640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fc89810e900 con 0x7fc8981019a0 2024-05-20T16:13:59.675 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:59.672+0000 7fc89d6d5640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc8981078e0 0x7fc8981a06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:13:59.675 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:59.672+0000 7fc89d6d5640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc8981078e0 0x7fc8981a06f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58440/0 (socket says 172.21.15.82:58440) 2024-05-20T16:13:59.675 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:59.672+0000 7fc89d6d5640 1 -- 172.21.15.82:0/4246881390 learned_addr learned my addr 172.21.15.82:0/4246881390 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:13:59.675 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:59.672+0000 7fc89ced4640 1 --2- 172.21.15.82:0/4246881390 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc8981019a0 0x7fc898199220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:13:59.675 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:59.672+0000 7fc88ffff640 1 --2- 172.21.15.82:0/4246881390 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc898102be0 0x7fc898199760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:13:59.675 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:59.672+0000 7fc89d6d5640 1 -- 172.21.15.82:0/4246881390 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc8981019a0 msgr2=0x7fc898199220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:59.675 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:59.672+0000 7fc89d6d5640 1 --2- 172.21.15.82:0/4246881390 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc8981019a0 0x7fc898199220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:59.675 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:59.672+0000 7fc89d6d5640 1 -- 172.21.15.82:0/4246881390 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc898102be0 msgr2=0x7fc898199760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:59.675 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:59.672+0000 7fc89d6d5640 1 --2- 172.21.15.82:0/4246881390 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc898102be0 0x7fc898199760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:59.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:59.672+0000 7fc89d6d5640 1 -- 172.21.15.82:0/4246881390 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8981a0d60 con 0x7fc8981078e0 2024-05-20T16:13:59.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:59.672+0000 7fc88ffff640 1 --2- 172.21.15.82:0/4246881390 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc898102be0 0x7fc898199760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T16:13:59.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:59.672+0000 7fc89d6d5640 1 --2- 172.21.15.82:0/4246881390 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc8981078e0 0x7fc8981a06f0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fc89002a930 tx=0x7fc890032c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:13:59.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:59.672+0000 7fc88dffb640 1 -- 172.21.15.82:0/4246881390 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc89003f030 con 0x7fc8981078e0 2024-05-20T16:13:59.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:59.672+0000 7fc88dffb640 1 -- 172.21.15.82:0/4246881390 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc890032e50 con 0x7fc8981078e0 2024-05-20T16:13:59.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:59.672+0000 7fc89f138640 1 -- 172.21.15.82:0/4246881390 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8981a0ff0 con 0x7fc8981078e0 2024-05-20T16:13:59.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:59.672+0000 7fc88dffb640 1 -- 172.21.15.82:0/4246881390 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc890035070 con 0x7fc8981078e0 2024-05-20T16:13:59.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:59.672+0000 7fc89f138640 1 -- 172.21.15.82:0/4246881390 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8981a1450 con 0x7fc8981078e0 2024-05-20T16:13:59.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:59.673+0000 7fc89f138640 1 -- 172.21.15.82:0/4246881390 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8981066f0 con 0x7fc8981078e0 2024-05-20T16:13:59.677 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:59.675+0000 7fc88dffb640 1 -- 172.21.15.82:0/4246881390 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fc890042030 con 0x7fc8981078e0 2024-05-20T16:13:59.677 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:59.675+0000 7fc88dffb640 1 --2- 172.21.15.82:0/4246881390 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fc8800a2c10 0x7fc8800a5120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:59.677 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:59.676+0000 7fc89ced4640 1 --2- 172.21.15.82:0/4246881390 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fc8800a2c10 0x7fc8800a5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:13:59.677 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:59.676+0000 7fc88dffb640 1 -- 172.21.15.82:0/4246881390 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9150+0+0 (secure 0 0 0) 0x7fc8900efc60 con 0x7fc8981078e0 2024-05-20T16:13:59.677 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:59.676+0000 7fc89ced4640 1 --2- 172.21.15.82:0/4246881390 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fc8800a2c10 0x7fc8800a5120 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fc8981025a0 tx=0x7fc894009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:13:59.678 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:59.679+0000 7fc88dffb640 1 -- 172.21.15.82:0/4246881390 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fc8900bc830 con 0x7fc8981078e0 2024-05-20T16:13:59.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:13:59 smithi160 ceph-mon[36347]: from='client.? 172.21.15.82:0/3202861248' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-20T16:13:59.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:13:59 smithi160 ceph-mon[36347]: from='client.? 172.21.15.82:0/299233696' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T16:13:59.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:13:59 smithi160 ceph-mon[36347]: from='client.? 172.21.15.82:0/3507652997' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-20T16:13:59.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:13:59 smithi160 ceph-mon[36347]: pgmap v362: 129 pgs: 129 active+clean; 641 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s; 9 B/s, 0 objects/s recovering 2024-05-20T16:13:59.854 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:59.854+0000 7fc89f138640 1 -- 172.21.15.82:0/4246881390 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fc89802d4f0 con 0x7fc8981078e0 2024-05-20T16:13:59.854 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:59.855+0000 7fc88dffb640 1 -- 172.21.15.82:0/4246881390 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v95) ==== 104+0+11 (secure 0 0 0) 0x7fc8900bc130 con 0x7fc8981078e0 2024-05-20T16:13:59.855 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 32 2024-05-20T16:13:59.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:59.857+0000 7fc89f138640 1 -- 172.21.15.82:0/4246881390 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fc8800a2c10 msgr2=0x7fc8800a5120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:59.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:59.857+0000 7fc89f138640 1 --2- 172.21.15.82:0/4246881390 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fc8800a2c10 0x7fc8800a5120 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fc8981025a0 tx=0x7fc894009500 comp rx=0 tx=0).stop 2024-05-20T16:13:59.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:59.857+0000 7fc89f138640 1 -- 172.21.15.82:0/4246881390 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc8981078e0 msgr2=0x7fc8981a06f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:59.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:59.857+0000 7fc89f138640 1 --2- 172.21.15.82:0/4246881390 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc8981078e0 0x7fc8981a06f0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fc89002a930 tx=0x7fc890032c60 comp rx=0 tx=0).stop 2024-05-20T16:13:59.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:59.857+0000 7fc89f138640 1 -- 172.21.15.82:0/4246881390 shutdown_connections 2024-05-20T16:13:59.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:59.857+0000 7fc89f138640 1 --2- 172.21.15.82:0/4246881390 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fc8800a2c10 0x7fc8800a5120 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:59.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:59.857+0000 7fc89f138640 1 --2- 172.21.15.82:0/4246881390 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc8981078e0 0x7fc8981a06f0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:59.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:59.857+0000 7fc89f138640 1 --2- 172.21.15.82:0/4246881390 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc898102be0 0x7fc898199760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:59.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:59.857+0000 7fc89f138640 1 --2- 172.21.15.82:0/4246881390 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc8981019a0 0x7fc898199220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:59.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:59.857+0000 7fc89f138640 1 -- 172.21.15.82:0/4246881390 >> 172.21.15.82:0/4246881390 conn(0x7fc8980fcb70 msgr2=0x7fc8981038c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:13:59.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:59.857+0000 7fc89f138640 1 -- 172.21.15.82:0/4246881390 shutdown_connections 2024-05-20T16:13:59.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:59.857+0000 7fc89f138640 1 -- 172.21.15.82:0/4246881390 wait complete. 2024-05-20T16:13:59.869 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-20T16:13:59.885 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:13:59 smithi184 ceph-mon[36209]: from='client.? 172.21.15.82:0/3202861248' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-20T16:13:59.885 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:13:59 smithi184 ceph-mon[36209]: from='client.? 172.21.15.82:0/299233696' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T16:13:59.885 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:13:59 smithi184 ceph-mon[36209]: from='client.? 172.21.15.82:0/3507652997' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-20T16:13:59.886 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:13:59 smithi184 ceph-mon[36209]: pgmap v362: 129 pgs: 129 active+clean; 641 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s; 9 B/s, 0 objects/s recovering 2024-05-20T16:13:59.983 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:59.983+0000 7f6ba1510640 1 -- 172.21.15.82:0/541812097 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6b9c1078e0 msgr2=0x7f6b9c109d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:59.984 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:59.983+0000 7f6ba1510640 1 --2- 172.21.15.82:0/541812097 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6b9c1078e0 0x7f6b9c109d00 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f6b94009ca0 tx=0x7f6b9402a8c0 comp rx=0 tx=0).stop 2024-05-20T16:13:59.984 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:59.983+0000 7f6ba1510640 1 -- 172.21.15.82:0/541812097 shutdown_connections 2024-05-20T16:13:59.984 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:59.983+0000 7f6ba1510640 1 --2- 172.21.15.82:0/541812097 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6b9c1078e0 0x7f6b9c109d00 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:59.984 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:59.983+0000 7f6ba1510640 1 --2- 172.21.15.82:0/541812097 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6b9c102be0 0x7f6b9c102ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:59.984 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:59.983+0000 7f6ba1510640 1 --2- 172.21.15.82:0/541812097 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6b9c1019a0 0x7f6b9c1026a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:59.984 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:59.983+0000 7f6ba1510640 1 -- 172.21.15.82:0/541812097 >> 172.21.15.82:0/541812097 conn(0x7f6b9c0fcb70 msgr2=0x7f6b9c0ff040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:13:59.984 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:59.983+0000 7f6ba1510640 1 -- 172.21.15.82:0/541812097 shutdown_connections 2024-05-20T16:13:59.984 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:59.983+0000 7f6ba1510640 1 -- 172.21.15.82:0/541812097 wait complete. 2024-05-20T16:13:59.984 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:59.984+0000 7f6ba1510640 1 Processor -- start 2024-05-20T16:13:59.986 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:59.984+0000 7f6ba1510640 1 -- start start 2024-05-20T16:13:59.986 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:59.984+0000 7f6ba1510640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6b9c1019a0 0x7f6b9c198140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:59.986 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:59.984+0000 7f6ba1510640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6b9c102be0 0x7f6b9c198680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:59.986 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:59.984+0000 7f6ba1510640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6b9c1078e0 0x7f6b9c19f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:59.986 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:59.984+0000 7f6ba1510640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f6b9c10d620 con 0x7f6b9c1019a0 2024-05-20T16:13:59.986 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:59.984+0000 7f6ba1510640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f6b9c10d4a0 con 0x7f6b9c1078e0 2024-05-20T16:13:59.986 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:59.984+0000 7f6ba1510640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f6b9c10d7a0 con 0x7f6b9c102be0 2024-05-20T16:13:59.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:59.985+0000 7f6b9affd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6b9c1019a0 0x7f6b9c198140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:13:59.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:59.985+0000 7f6b9affd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6b9c1019a0 0x7f6b9c198140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58472/0 (socket says 172.21.15.82:58472) 2024-05-20T16:13:59.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:59.985+0000 7f6b9affd640 1 -- 172.21.15.82:0/3316524550 learned_addr learned my addr 172.21.15.82:0/3316524550 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:13:59.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:59.985+0000 7f6b9b7fe640 1 --2- 172.21.15.82:0/3316524550 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6b9c1078e0 0x7f6b9c19f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:13:59.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:59.985+0000 7f6b9affd640 1 -- 172.21.15.82:0/3316524550 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6b9c102be0 msgr2=0x7f6b9c198680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:59.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:59.985+0000 7f6b9affd640 1 --2- 172.21.15.82:0/3316524550 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6b9c102be0 0x7f6b9c198680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:59.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:59.985+0000 7f6b9affd640 1 -- 172.21.15.82:0/3316524550 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6b9c1078e0 msgr2=0x7f6b9c19f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:13:59.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:59.985+0000 7f6b9affd640 1 --2- 172.21.15.82:0/3316524550 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6b9c1078e0 0x7f6b9c19f610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:13:59.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:59.985+0000 7f6b9affd640 1 -- 172.21.15.82:0/3316524550 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b9c19fc80 con 0x7f6b9c1019a0 2024-05-20T16:13:59.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:59.985+0000 7f6b9b7fe640 1 --2- 172.21.15.82:0/3316524550 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6b9c1078e0 0x7f6b9c19f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T16:13:59.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:59.985+0000 7f6b9affd640 1 --2- 172.21.15.82:0/3316524550 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6b9c1019a0 0x7f6b9c198140 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f6b8c00c970 tx=0x7f6b8c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:13:59.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:59.985+0000 7f6b98ff9640 1 -- 172.21.15.82:0/3316524550 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b8c004220 con 0x7f6b9c1019a0 2024-05-20T16:13:59.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:59.985+0000 7f6b98ff9640 1 -- 172.21.15.82:0/3316524550 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b8c0043c0 con 0x7f6b9c1019a0 2024-05-20T16:13:59.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:59.985+0000 7f6ba1510640 1 -- 172.21.15.82:0/3316524550 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b9c19ff70 con 0x7f6b9c1019a0 2024-05-20T16:13:59.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:59.985+0000 7f6b98ff9640 1 -- 172.21.15.82:0/3316524550 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b8c004a00 con 0x7f6b9c1019a0 2024-05-20T16:13:59.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:59.985+0000 7f6ba1510640 1 -- 172.21.15.82:0/3316524550 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b9c1a03e0 con 0x7f6b9c1019a0 2024-05-20T16:13:59.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:59.986+0000 7f6ba1510640 1 -- 172.21.15.82:0/3316524550 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b9c1066f0 con 0x7f6b9c1019a0 2024-05-20T16:13:59.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:59.988+0000 7f6b98ff9640 1 -- 172.21.15.82:0/3316524550 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f6b8c0079b0 con 0x7f6b9c1019a0 2024-05-20T16:13:59.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:59.988+0000 7f6b98ff9640 1 --2- 172.21.15.82:0/3316524550 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f6b880a2b90 0x7f6b880a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:13:59.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:59.989+0000 7f6b927fc640 1 --2- 172.21.15.82:0/3316524550 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f6b880a2b90 0x7f6b880a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:13:59.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:59.989+0000 7f6b98ff9640 1 -- 172.21.15.82:0/3316524550 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9150+0+0 (secure 0 0 0) 0x7f6b8c0cf650 con 0x7f6b9c1019a0 2024-05-20T16:13:59.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:59.989+0000 7f6b927fc640 1 --2- 172.21.15.82:0/3316524550 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f6b880a2b90 0x7f6b880a50a0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f6b84005fd0 tx=0x7f6b84005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:13:59.991 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:13:59.992+0000 7f6b98ff9640 1 -- 172.21.15.82:0/3316524550 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f6b8c014030 con 0x7f6b9c1019a0 2024-05-20T16:14:00.168 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:00.168+0000 7f6ba1510640 1 -- 172.21.15.82:0/3316524550 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f6b9c02d430 con 0x7f6b9c1019a0 2024-05-20T16:14:00.169 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:00.169+0000 7f6b98ff9640 1 -- 172.21.15.82:0/3316524550 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v95) ==== 100+0+11 (secure 0 0 0) 0x7f6b8c09c220 con 0x7f6b9c1019a0 2024-05-20T16:14:00.169 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 64 2024-05-20T16:14:00.171 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:00.171+0000 7f6ba1510640 1 -- 172.21.15.82:0/3316524550 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f6b880a2b90 msgr2=0x7f6b880a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:14:00.171 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:00.171+0000 7f6ba1510640 1 --2- 172.21.15.82:0/3316524550 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f6b880a2b90 0x7f6b880a50a0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f6b84005fd0 tx=0x7f6b84005850 comp rx=0 tx=0).stop 2024-05-20T16:14:00.171 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:00.171+0000 7f6ba1510640 1 -- 172.21.15.82:0/3316524550 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6b9c1019a0 msgr2=0x7f6b9c198140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:14:00.171 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:00.171+0000 7f6ba1510640 1 --2- 172.21.15.82:0/3316524550 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6b9c1019a0 0x7f6b9c198140 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f6b8c00c970 tx=0x7f6b8c00ce30 comp rx=0 tx=0).stop 2024-05-20T16:14:00.171 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:00.171+0000 7f6ba1510640 1 -- 172.21.15.82:0/3316524550 shutdown_connections 2024-05-20T16:14:00.171 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:00.171+0000 7f6ba1510640 1 --2- 172.21.15.82:0/3316524550 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f6b880a2b90 0x7f6b880a50a0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:00.171 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:00.171+0000 7f6ba1510640 1 --2- 172.21.15.82:0/3316524550 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6b9c1078e0 0x7f6b9c19f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:00.171 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:00.171+0000 7f6ba1510640 1 --2- 172.21.15.82:0/3316524550 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6b9c102be0 0x7f6b9c198680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:00.172 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:00.171+0000 7f6ba1510640 1 --2- 172.21.15.82:0/3316524550 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6b9c1019a0 0x7f6b9c198140 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:00.172 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:00.171+0000 7f6ba1510640 1 -- 172.21.15.82:0/3316524550 >> 172.21.15.82:0/3316524550 conn(0x7f6b9c0fcb70 msgr2=0x7f6b9c1038c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:14:00.172 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:00.171+0000 7f6ba1510640 1 -- 172.21.15.82:0/3316524550 shutdown_connections 2024-05-20T16:14:00.172 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:00.171+0000 7f6ba1510640 1 -- 172.21.15.82:0/3316524550 wait complete. 2024-05-20T16:14:00.182 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-20T16:14:00.298 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:00.296+0000 7ffb0475e640 1 -- 172.21.15.82:0/720684651 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffafc1031c0 msgr2=0x7ffafc1055e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:14:00.298 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:00.296+0000 7ffb0475e640 1 --2- 172.21.15.82:0/720684651 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffafc1031c0 0x7ffafc1055e0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7ffaf8009900 tx=0x7ffaf802a410 comp rx=0 tx=0).stop 2024-05-20T16:14:00.299 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:00.297+0000 7ffb0475e640 1 -- 172.21.15.82:0/720684651 shutdown_connections 2024-05-20T16:14:00.299 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:00.297+0000 7ffb0475e640 1 --2- 172.21.15.82:0/720684651 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffafc108720 0x7ffafc10ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:00.299 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:00.297+0000 7ffb0475e640 1 --2- 172.21.15.82:0/720684651 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ffafc105bb0 0x7ffafc108090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:00.299 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:00.297+0000 7ffb0475e640 1 --2- 172.21.15.82:0/720684651 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffafc1031c0 0x7ffafc1055e0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:00.299 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:00.297+0000 7ffb0475e640 1 -- 172.21.15.82:0/720684651 >> 172.21.15.82:0/720684651 conn(0x7ffafc0fcb80 msgr2=0x7ffafc0ff030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:14:00.299 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:00.297+0000 7ffb0475e640 1 -- 172.21.15.82:0/720684651 shutdown_connections 2024-05-20T16:14:00.299 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:00.297+0000 7ffb0475e640 1 -- 172.21.15.82:0/720684651 wait complete. 2024-05-20T16:14:00.299 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:00.297+0000 7ffb0475e640 1 Processor -- start 2024-05-20T16:14:00.299 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:00.297+0000 7ffb0475e640 1 -- start start 2024-05-20T16:14:00.299 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:00.297+0000 7ffb0475e640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ffafc1031c0 0x7ffafc102a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:14:00.299 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:00.297+0000 7ffb0475e640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffafc105bb0 0x7ffafc1010d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:14:00.299 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:00.297+0000 7ffb0475e640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffafc108720 0x7ffafc101610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:14:00.299 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:00.297+0000 7ffb0475e640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7ffafc10e6d0 con 0x7ffafc108720 2024-05-20T16:14:00.300 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:00.297+0000 7ffb0475e640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7ffafc10e550 con 0x7ffafc1031c0 2024-05-20T16:14:00.300 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:00.297+0000 7ffb0475e640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ffafc10e850 con 0x7ffafc105bb0 2024-05-20T16:14:00.300 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:00.298+0000 7ffb02cfb640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffafc108720 0x7ffafc101610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:14:00.300 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:00.298+0000 7ffb02cfb640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffafc108720 0x7ffafc101610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58480/0 (socket says 172.21.15.82:58480) 2024-05-20T16:14:00.300 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:00.298+0000 7ffb02cfb640 1 -- 172.21.15.82:0/829440101 learned_addr learned my addr 172.21.15.82:0/829440101 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:14:00.300 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:00.298+0000 7ffb024fa640 1 --2- 172.21.15.82:0/829440101 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ffafc1031c0 0x7ffafc102a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:14:00.300 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:00.298+0000 7ffb02cfb640 1 -- 172.21.15.82:0/829440101 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffafc105bb0 msgr2=0x7ffafc1010d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:14:00.300 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:00.298+0000 7ffb01cf9640 1 --2- 172.21.15.82:0/829440101 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffafc105bb0 0x7ffafc1010d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:14:00.300 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:00.298+0000 7ffb02cfb640 1 --2- 172.21.15.82:0/829440101 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffafc105bb0 0x7ffafc1010d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:00.300 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:00.298+0000 7ffb02cfb640 1 -- 172.21.15.82:0/829440101 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ffafc1031c0 msgr2=0x7ffafc102a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:14:00.300 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:00.298+0000 7ffb02cfb640 1 --2- 172.21.15.82:0/829440101 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ffafc1031c0 0x7ffafc102a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:00.300 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:00.298+0000 7ffb02cfb640 1 -- 172.21.15.82:0/829440101 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffafc101d40 con 0x7ffafc108720 2024-05-20T16:14:00.300 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:00.298+0000 7ffb02cfb640 1 --2- 172.21.15.82:0/829440101 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffafc108720 0x7ffafc101610 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7ffaf400eaa0 tx=0x7ffaf400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:14:00.300 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:00.298+0000 7ffaf37fe640 1 -- 172.21.15.82:0/829440101 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaf400cdb0 con 0x7ffafc108720 2024-05-20T16:14:00.301 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:00.298+0000 7ffaf37fe640 1 -- 172.21.15.82:0/829440101 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffaf4005020 con 0x7ffafc108720 2024-05-20T16:14:00.301 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:00.298+0000 7ffb0475e640 1 -- 172.21.15.82:0/829440101 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffafc1a3680 con 0x7ffafc108720 2024-05-20T16:14:00.301 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:00.298+0000 7ffaf37fe640 1 -- 172.21.15.82:0/829440101 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaf40116b0 con 0x7ffafc108720 2024-05-20T16:14:00.301 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:00.298+0000 7ffb0475e640 1 -- 172.21.15.82:0/829440101 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffafc1a3b40 con 0x7ffafc108720 2024-05-20T16:14:00.302 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:00.299+0000 7ffb0475e640 1 -- 172.21.15.82:0/829440101 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffafc06b250 con 0x7ffafc108720 2024-05-20T16:14:00.302 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:00.301+0000 7ffaf37fe640 1 -- 172.21.15.82:0/829440101 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7ffaf4005560 con 0x7ffafc108720 2024-05-20T16:14:00.302 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:00.301+0000 7ffaf37fe640 1 --2- 172.21.15.82:0/829440101 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7ffae80a2b90 0x7ffae80a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:14:00.302 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:00.301+0000 7ffb024fa640 1 --2- 172.21.15.82:0/829440101 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7ffae80a2b90 0x7ffae80a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:14:00.302 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:00.301+0000 7ffaf37fe640 1 -- 172.21.15.82:0/829440101 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9150+0+0 (secure 0 0 0) 0x7ffaf40cf640 con 0x7ffafc108720 2024-05-20T16:14:00.302 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:00.302+0000 7ffb024fa640 1 --2- 172.21.15.82:0/829440101 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7ffae80a2b90 0x7ffae80a50a0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7ffaf8009230 tx=0x7ffaf8031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:14:00.305 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:00.306+0000 7ffaf37fe640 1 -- 172.21.15.82:0/829440101 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7ffaf4016030 con 0x7ffafc108720 2024-05-20T16:14:00.483 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:00.483+0000 7ffb0475e640 1 -- 172.21.15.82:0/829440101 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7ffafc1a41c0 con 0x7ffafc108720 2024-05-20T16:14:00.484 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:00.484+0000 7ffaf37fe640 1 -- 172.21.15.82:0/829440101 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v95) ==== 103+0+11 (secure 0 0 0) 0x7ffaf409c210 con 0x7ffafc108720 2024-05-20T16:14:00.484 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 32 2024-05-20T16:14:00.486 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:00.486+0000 7ffb0475e640 1 -- 172.21.15.82:0/829440101 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7ffae80a2b90 msgr2=0x7ffae80a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:14:00.486 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:00.486+0000 7ffb0475e640 1 --2- 172.21.15.82:0/829440101 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7ffae80a2b90 0x7ffae80a50a0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7ffaf8009230 tx=0x7ffaf8031400 comp rx=0 tx=0).stop 2024-05-20T16:14:00.486 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:00.486+0000 7ffb0475e640 1 -- 172.21.15.82:0/829440101 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffafc108720 msgr2=0x7ffafc101610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:14:00.486 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:00.486+0000 7ffb0475e640 1 --2- 172.21.15.82:0/829440101 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffafc108720 0x7ffafc101610 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7ffaf400eaa0 tx=0x7ffaf400ef60 comp rx=0 tx=0).stop 2024-05-20T16:14:00.487 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:00.486+0000 7ffb0475e640 1 -- 172.21.15.82:0/829440101 shutdown_connections 2024-05-20T16:14:00.487 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:00.486+0000 7ffb0475e640 1 --2- 172.21.15.82:0/829440101 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7ffae80a2b90 0x7ffae80a50a0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:00.487 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:00.486+0000 7ffb0475e640 1 --2- 172.21.15.82:0/829440101 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffafc108720 0x7ffafc101610 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:00.487 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:00.486+0000 7ffb0475e640 1 --2- 172.21.15.82:0/829440101 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffafc105bb0 0x7ffafc1010d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:00.487 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:00.486+0000 7ffb0475e640 1 --2- 172.21.15.82:0/829440101 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ffafc1031c0 0x7ffafc102a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:00.487 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:00.486+0000 7ffb0475e640 1 -- 172.21.15.82:0/829440101 >> 172.21.15.82:0/829440101 conn(0x7ffafc0fcb80 msgr2=0x7ffafc104ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:14:00.487 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:00.486+0000 7ffb0475e640 1 -- 172.21.15.82:0/829440101 shutdown_connections 2024-05-20T16:14:00.487 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:00.486+0000 7ffb0475e640 1 -- 172.21.15.82:0/829440101 wait complete. 2024-05-20T16:14:00.497 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:14:00 smithi082 ceph-mon[32238]: from='client.? 172.21.15.82:0/2735827395' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-20T16:14:00.497 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:14:00 smithi082 ceph-mon[32238]: from='client.? 172.21.15.82:0/4246881390' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-20T16:14:00.497 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:14:00 smithi082 ceph-mon[32238]: from='client.? 172.21.15.82:0/3316524550' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-20T16:14:00.498 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-20T16:14:00.610 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:00.610+0000 7fc8920e9640 1 -- 172.21.15.82:0/1393894838 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc88c1078b0 msgr2=0x7fc88c109cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:14:00.610 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:00.610+0000 7fc8920e9640 1 --2- 172.21.15.82:0/1393894838 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc88c1078b0 0x7fc88c109cd0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fc884009ca0 tx=0x7fc88402a8c0 comp rx=0 tx=0).stop 2024-05-20T16:14:00.610 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:00.610+0000 7fc8920e9640 1 -- 172.21.15.82:0/1393894838 shutdown_connections 2024-05-20T16:14:00.610 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:00.610+0000 7fc8920e9640 1 --2- 172.21.15.82:0/1393894838 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc88c1078b0 0x7fc88c109cd0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:00.610 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:00.610+0000 7fc8920e9640 1 --2- 172.21.15.82:0/1393894838 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc88c102bb0 0x7fc88c102fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:00.610 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:00.610+0000 7fc8920e9640 1 --2- 172.21.15.82:0/1393894838 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc88c101970 0x7fc88c102670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:00.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:00.610+0000 7fc8920e9640 1 -- 172.21.15.82:0/1393894838 >> 172.21.15.82:0/1393894838 conn(0x7fc88c0fcb80 msgr2=0x7fc88c0ff010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:14:00.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:00.610+0000 7fc8920e9640 1 -- 172.21.15.82:0/1393894838 shutdown_connections 2024-05-20T16:14:00.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:00.610+0000 7fc8920e9640 1 -- 172.21.15.82:0/1393894838 wait complete. 2024-05-20T16:14:00.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:00.610+0000 7fc8920e9640 1 Processor -- start 2024-05-20T16:14:00.614 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:00.611+0000 7fc8920e9640 1 -- start start 2024-05-20T16:14:00.614 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:00.611+0000 7fc8920e9640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc88c101970 0x7fc88c199290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:14:00.614 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:00.611+0000 7fc8920e9640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc88c102bb0 0x7fc88c1997d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:14:00.614 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:00.611+0000 7fc8920e9640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc88c1078b0 0x7fc88c1a0760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:14:00.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:00.611+0000 7fc8920e9640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fc88c10e750 con 0x7fc88c101970 2024-05-20T16:14:00.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:00.611+0000 7fc8920e9640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fc88c10e5d0 con 0x7fc88c1078b0 2024-05-20T16:14:00.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:00.611+0000 7fc8920e9640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fc88c10e8d0 con 0x7fc88c102bb0 2024-05-20T16:14:00.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:00.611+0000 7fc88b7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc88c101970 0x7fc88c199290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:14:00.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:00.611+0000 7fc88b7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc88c101970 0x7fc88c199290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58510/0 (socket says 172.21.15.82:58510) 2024-05-20T16:14:00.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:00.611+0000 7fc88b7fe640 1 -- 172.21.15.82:0/1801532231 learned_addr learned my addr 172.21.15.82:0/1801532231 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:14:00.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:00.611+0000 7fc88affd640 1 --2- 172.21.15.82:0/1801532231 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc88c102bb0 0x7fc88c1997d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:14:00.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:00.611+0000 7fc88bfff640 1 --2- 172.21.15.82:0/1801532231 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc88c1078b0 0x7fc88c1a0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:14:00.616 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:00.611+0000 7fc88b7fe640 1 -- 172.21.15.82:0/1801532231 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc88c102bb0 msgr2=0x7fc88c1997d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:14:00.616 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:00.611+0000 7fc88b7fe640 1 --2- 172.21.15.82:0/1801532231 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc88c102bb0 0x7fc88c1997d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:00.616 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:00.611+0000 7fc88b7fe640 1 -- 172.21.15.82:0/1801532231 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc88c1078b0 msgr2=0x7fc88c1a0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:14:00.616 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:00.611+0000 7fc88b7fe640 1 --2- 172.21.15.82:0/1801532231 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc88c1078b0 0x7fc88c1a0760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:00.616 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:00.611+0000 7fc88b7fe640 1 -- 172.21.15.82:0/1801532231 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc88c1a0dd0 con 0x7fc88c101970 2024-05-20T16:14:00.616 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:00.612+0000 7fc88b7fe640 1 --2- 172.21.15.82:0/1801532231 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc88c101970 0x7fc88c199290 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fc87c00c970 tx=0x7fc87c00cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:14:00.616 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:00.612+0000 7fc888ff9640 1 -- 172.21.15.82:0/1801532231 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc87c004220 con 0x7fc88c101970 2024-05-20T16:14:00.616 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:00.612+0000 7fc8920e9640 1 -- 172.21.15.82:0/1801532231 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc88c1a10c0 con 0x7fc88c101970 2024-05-20T16:14:00.616 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:00.612+0000 7fc888ff9640 1 -- 172.21.15.82:0/1801532231 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc87c0043c0 con 0x7fc88c101970 2024-05-20T16:14:00.617 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:00.612+0000 7fc888ff9640 1 -- 172.21.15.82:0/1801532231 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc87c00f450 con 0x7fc88c101970 2024-05-20T16:14:00.617 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:00.612+0000 7fc8920e9640 1 -- 172.21.15.82:0/1801532231 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc88c1a1690 con 0x7fc88c101970 2024-05-20T16:14:00.617 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:00.613+0000 7fc8920e9640 1 -- 172.21.15.82:0/1801532231 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc88c1066c0 con 0x7fc88c101970 2024-05-20T16:14:00.618 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:00.615+0000 7fc888ff9640 1 -- 172.21.15.82:0/1801532231 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fc87c0079b0 con 0x7fc88c101970 2024-05-20T16:14:00.618 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:00.616+0000 7fc888ff9640 1 --2- 172.21.15.82:0/1801532231 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fc8740a2b90 0x7fc8740a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:14:00.618 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:00.616+0000 7fc88affd640 1 --2- 172.21.15.82:0/1801532231 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fc8740a2b90 0x7fc8740a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:14:00.618 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:00.616+0000 7fc888ff9640 1 -- 172.21.15.82:0/1801532231 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9150+0+0 (secure 0 0 0) 0x7fc87c0cf450 con 0x7fc88c101970 2024-05-20T16:14:00.618 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:00.616+0000 7fc88affd640 1 --2- 172.21.15.82:0/1801532231 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fc8740a2b90 0x7fc8740a50a0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fc88c19a330 tx=0x7fc880009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:14:00.619 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:00.619+0000 7fc888ff9640 1 -- 172.21.15.82:0/1801532231 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fc87c014030 con 0x7fc88c101970 2024-05-20T16:14:00.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:14:00 smithi160 ceph-mon[36347]: from='client.? 172.21.15.82:0/2735827395' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-20T16:14:00.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:14:00 smithi160 ceph-mon[36347]: from='client.? 172.21.15.82:0/4246881390' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-20T16:14:00.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:14:00 smithi160 ceph-mon[36347]: from='client.? 172.21.15.82:0/3316524550' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-20T16:14:00.832 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:00.832+0000 7fc8920e9640 1 -- 172.21.15.82:0/1801532231 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc88c02d2a0 con 0x7fc88c101970 2024-05-20T16:14:00.833 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:00.833+0000 7fc888ff9640 1 -- 172.21.15.82:0/1801532231 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11879 (secure 0 0 0) 0x7fc87c09c020 con 0x7fc88c101970 2024-05-20T16:14:00.833 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-05-20T16:14:00.833 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/2044534217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":2044534217},{"type":"v1","addr":"172.21.15.82:6835","nonce":2044534217}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/2635635097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":2635635097},{"type":"v1","addr":"172.21.15.82:6837","nonce":2635635097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14625,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6839/2118233541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6838","nonce":2118233541},{"type":"v1","addr":"172.21.15.82:6839","nonce":2118233541}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14637,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6841/2119517973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6840","nonce":2119517973},{"type":"v1","addr":"172.21.15.82:6841","nonce":2119517973}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6833/4015144007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":4015144007},{"type":"v1","addr":"172.21.15.184:6833","nonce":4015144007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6833/975243372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6832","nonce":975243372},{"type":"v1","addr":"172.21.15.160:6833","nonce":975243372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-20T16:11:46.751632+0000","modified":"2024-05-20T16:12:13.394770+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24548,"mds_1":24511,"mds_2":24506},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24470":{"gid":24470,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.160:6835/664068363","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":664068363},{"type":"v1","addr":"172.21.15.160:6835","nonce":664068363}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6835/4242156090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":4242156090},{"type":"v1","addr":"172.21.15.184:6835","nonce":4242156090}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6837/403262242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":403262242},{"type":"v1","addr":"172.21.15.184:6837","nonce":403262242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.160:6837/2846188680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6836","nonce":2846188680},{"type":"v1","addr":"172.21.15.160:6837","nonce":2846188680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.184:6839/2199955067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":2199955067},{"type":"v1","addr":"172.21.15.184:6839","nonce":2199955067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24548":{"gid":24548,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.160:6839/2765554482","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6838","nonce":2765554482},{"type":"v1","addr":"172.21.15.160:6839","nonce":2765554482}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24548,"qdb_cluster":[24511,24506,24548]},"id":1}]} 2024-05-20T16:14:00.834 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 24 2024-05-20T16:14:00.835 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:00.835+0000 7fc8920e9640 1 -- 172.21.15.82:0/1801532231 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fc8740a2b90 msgr2=0x7fc8740a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:14:00.835 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:00.835+0000 7fc8920e9640 1 --2- 172.21.15.82:0/1801532231 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fc8740a2b90 0x7fc8740a50a0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fc88c19a330 tx=0x7fc880009110 comp rx=0 tx=0).stop 2024-05-20T16:14:00.835 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:00.835+0000 7fc8920e9640 1 -- 172.21.15.82:0/1801532231 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc88c101970 msgr2=0x7fc88c199290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:14:00.836 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:00.835+0000 7fc8920e9640 1 --2- 172.21.15.82:0/1801532231 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc88c101970 0x7fc88c199290 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fc87c00c970 tx=0x7fc87c00cd70 comp rx=0 tx=0).stop 2024-05-20T16:14:00.836 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:00.835+0000 7fc8920e9640 1 -- 172.21.15.82:0/1801532231 shutdown_connections 2024-05-20T16:14:00.836 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:00.835+0000 7fc8920e9640 1 --2- 172.21.15.82:0/1801532231 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fc8740a2b90 0x7fc8740a50a0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:00.836 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:00.835+0000 7fc8920e9640 1 --2- 172.21.15.82:0/1801532231 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc88c1078b0 0x7fc88c1a0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:00.836 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:00.835+0000 7fc8920e9640 1 --2- 172.21.15.82:0/1801532231 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc88c102bb0 0x7fc88c1997d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:00.836 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:00.835+0000 7fc8920e9640 1 --2- 172.21.15.82:0/1801532231 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc88c101970 0x7fc88c199290 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:00.836 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:00.835+0000 7fc8920e9640 1 -- 172.21.15.82:0/1801532231 >> 172.21.15.82:0/1801532231 conn(0x7fc88c0fcb80 msgr2=0x7fc88c103890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:14:00.836 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:00.835+0000 7fc8920e9640 1 -- 172.21.15.82:0/1801532231 shutdown_connections 2024-05-20T16:14:00.836 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:00.835+0000 7fc8920e9640 1 -- 172.21.15.82:0/1801532231 wait complete. 2024-05-20T16:14:00.848 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-20T16:14:00.886 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:14:00 smithi184 ceph-mon[36209]: from='client.? 172.21.15.82:0/2735827395' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-20T16:14:00.886 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:14:00 smithi184 ceph-mon[36209]: from='client.? 172.21.15.82:0/4246881390' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-20T16:14:00.886 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:14:00 smithi184 ceph-mon[36209]: from='client.? 172.21.15.82:0/3316524550' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-20T16:14:00.960 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:00.960+0000 7efe4c661640 1 -- 172.21.15.82:0/503152631 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efe44101970 msgr2=0x7efe44102670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:14:00.960 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:00.960+0000 7efe4c661640 1 --2- 172.21.15.82:0/503152631 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efe44101970 0x7efe44102670 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7efe34009ca0 tx=0x7efe3402a8c0 comp rx=0 tx=0).stop 2024-05-20T16:14:00.960 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:00.960+0000 7efe4c661640 1 -- 172.21.15.82:0/503152631 shutdown_connections 2024-05-20T16:14:00.961 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:00.960+0000 7efe4c661640 1 --2- 172.21.15.82:0/503152631 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efe441078b0 0x7efe44109cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:00.961 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:00.960+0000 7efe4c661640 1 --2- 172.21.15.82:0/503152631 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efe44102bb0 0x7efe44102fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:00.961 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:00.960+0000 7efe4c661640 1 --2- 172.21.15.82:0/503152631 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efe44101970 0x7efe44102670 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:00.961 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:00.960+0000 7efe4c661640 1 -- 172.21.15.82:0/503152631 >> 172.21.15.82:0/503152631 conn(0x7efe440fcb80 msgr2=0x7efe440ff010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:14:00.961 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:00.960+0000 7efe4c661640 1 -- 172.21.15.82:0/503152631 shutdown_connections 2024-05-20T16:14:00.961 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:00.960+0000 7efe4c661640 1 -- 172.21.15.82:0/503152631 wait complete. 2024-05-20T16:14:00.961 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:00.961+0000 7efe4c661640 1 Processor -- start 2024-05-20T16:14:00.961 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:00.961+0000 7efe4c661640 1 -- start start 2024-05-20T16:14:00.963 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:00.961+0000 7efe4c661640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efe44101970 0x7efe441992c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:14:00.963 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:00.961+0000 7efe4c661640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efe44102bb0 0x7efe44199800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:14:00.963 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:00.961+0000 7efe4c661640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efe441078b0 0x7efe441a0790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:14:00.963 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:00.961+0000 7efe4c661640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7efe4410d5c0 con 0x7efe44101970 2024-05-20T16:14:00.963 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:00.961+0000 7efe4c661640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7efe4410d440 con 0x7efe441078b0 2024-05-20T16:14:00.963 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:00.961+0000 7efe4c661640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7efe4410d740 con 0x7efe44102bb0 2024-05-20T16:14:00.964 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:00.961+0000 7efe4a3fd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efe44101970 0x7efe441992c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:14:00.964 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:00.961+0000 7efe4a3fd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efe44101970 0x7efe441992c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58532/0 (socket says 172.21.15.82:58532) 2024-05-20T16:14:00.964 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:00.961+0000 7efe4a3fd640 1 -- 172.21.15.82:0/224258168 learned_addr learned my addr 172.21.15.82:0/224258168 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:14:00.964 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:00.961+0000 7efe4abfe640 1 --2- 172.21.15.82:0/224258168 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efe441078b0 0x7efe441a0790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:14:00.964 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:00.961+0000 7efe49bfc640 1 --2- 172.21.15.82:0/224258168 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efe44102bb0 0x7efe44199800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:14:00.964 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:00.961+0000 7efe4a3fd640 1 -- 172.21.15.82:0/224258168 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efe44102bb0 msgr2=0x7efe44199800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:14:00.964 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:00.961+0000 7efe4a3fd640 1 --2- 172.21.15.82:0/224258168 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efe44102bb0 0x7efe44199800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:00.964 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:00.961+0000 7efe4a3fd640 1 -- 172.21.15.82:0/224258168 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efe441078b0 msgr2=0x7efe441a0790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:14:00.964 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:00.961+0000 7efe4a3fd640 1 --2- 172.21.15.82:0/224258168 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efe441078b0 0x7efe441a0790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:00.964 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:00.961+0000 7efe4a3fd640 1 -- 172.21.15.82:0/224258168 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe441a0e00 con 0x7efe44101970 2024-05-20T16:14:00.964 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:00.961+0000 7efe4abfe640 1 --2- 172.21.15.82:0/224258168 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efe441078b0 0x7efe441a0790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T16:14:00.964 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:00.962+0000 7efe4a3fd640 1 --2- 172.21.15.82:0/224258168 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efe44101970 0x7efe441992c0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7efe34004af0 tx=0x7efe34031ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:14:00.964 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:00.962+0000 7efe437fe640 1 -- 172.21.15.82:0/224258168 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe3402faa0 con 0x7efe44101970 2024-05-20T16:14:00.965 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:00.962+0000 7efe437fe640 1 -- 172.21.15.82:0/224258168 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe3402fc40 con 0x7efe44101970 2024-05-20T16:14:00.965 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:00.962+0000 7efe437fe640 1 -- 172.21.15.82:0/224258168 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe3403a7f0 con 0x7efe44101970 2024-05-20T16:14:00.965 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:00.962+0000 7efe4c661640 1 -- 172.21.15.82:0/224258168 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe441a1090 con 0x7efe44101970 2024-05-20T16:14:00.965 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:00.962+0000 7efe4c661640 1 -- 172.21.15.82:0/224258168 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe441a15e0 con 0x7efe44101970 2024-05-20T16:14:00.965 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:00.963+0000 7efe4c661640 1 -- 172.21.15.82:0/224258168 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe441a1840 con 0x7efe44101970 2024-05-20T16:14:00.965 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:00.964+0000 7efe437fe640 1 -- 172.21.15.82:0/224258168 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7efe34030bc0 con 0x7efe44101970 2024-05-20T16:14:00.966 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:00.965+0000 7efe437fe640 1 --2- 172.21.15.82:0/224258168 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7efe2c0a2b90 0x7efe2c0a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:14:00.966 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:00.965+0000 7efe49bfc640 1 --2- 172.21.15.82:0/224258168 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7efe2c0a2b90 0x7efe2c0a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:14:00.966 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:00.965+0000 7efe437fe640 1 -- 172.21.15.82:0/224258168 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9150+0+0 (secure 0 0 0) 0x7efe340f0d90 con 0x7efe44101970 2024-05-20T16:14:00.966 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:00.966+0000 7efe49bfc640 1 --2- 172.21.15.82:0/224258168 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7efe2c0a2b90 0x7efe2c0a50a0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7efe38005d00 tx=0x7efe38005c70 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:14:00.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:00.969+0000 7efe437fe640 1 -- 172.21.15.82:0/224258168 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7efe340bd960 con 0x7efe44101970 2024-05-20T16:14:01.146 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.146+0000 7efe4c661640 1 -- 172.21.15.82:0/224258168 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7efe441a1a10 con 0x7efe44101970 2024-05-20T16:14:01.147 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.147+0000 7efe437fe640 1 -- 172.21.15.82:0/224258168 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) ==== 74+0+24771 (secure 0 0 0) 0x7efe340bd260 con 0x7efe44101970 2024-05-20T16:14:01.148 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-05-20T16:14:01.148 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":95,"fsid":"59d1600a-16c2-11ef-bc9a-c7b262605968","created":"2024-05-20T16:03:46.611556+0000","modified":"2024-05-20T16:13:46.328150+0000","last_up_change":"2024-05-20T16:10:59.303479+0000","last_in_change":"2024-05-20T16:10:39.063561+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-20T16:07:09.493727+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-20T16:11:45.058035+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-20T16:11:45.943862+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-20T16:11:49.637579+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"bae842a9-03ec-490e-b56d-f7957235cdc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":1484330167},{"type":"v1","addr":"172.21.15.82:6803","nonce":1484330167}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":1484330167},{"type":"v1","addr":"172.21.15.82:6805","nonce":1484330167}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1484330167},{"type":"v1","addr":"172.21.15.82:6809","nonce":1484330167}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":1484330167},{"type":"v1","addr":"172.21.15.82:6807","nonce":1484330167}]},"public_addr":"172.21.15.82:6803/1484330167","cluster_addr":"172.21.15.82:6805/1484330167","heartbeat_back_addr":"172.21.15.82:6809/1484330167","heartbeat_front_addr":"172.21.15.82:6807/1484330167","state":["exists","up"]},{"osd":1,"uuid":"59f6a2ba-b5de-4161-81c5-92a417abacdf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":1457003568},{"type":"v1","addr":"172.21.15.160:6801","nonce":1457003568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":1457003568},{"type":"v1","addr":"172.21.15.160:6803","nonce":1457003568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6806","nonce":1457003568},{"type":"v1","addr":"172.21.15.160:6807","nonce":1457003568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":1457003568},{"type":"v1","addr":"172.21.15.160:6805","nonce":1457003568}]},"public_addr":"172.21.15.160:6801/1457003568","cluster_addr":"172.21.15.160:6803/1457003568","heartbeat_back_addr":"172.21.15.160:6807/1457003568","heartbeat_front_addr":"172.21.15.160:6805/1457003568","state":["exists","up"]},{"osd":2,"uuid":"3bbb5587-b449-41f3-bd26-67ec3d6afdfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":4051293934},{"type":"v1","addr":"172.21.15.184:6801","nonce":4051293934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":4051293934},{"type":"v1","addr":"172.21.15.184:6803","nonce":4051293934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":4051293934},{"type":"v1","addr":"172.21.15.184:6807","nonce":4051293934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":4051293934},{"type":"v1","addr":"172.21.15.184:6805","nonce":4051293934}]},"public_addr":"172.21.15.184:6801/4051293934","cluster_addr":"172.21.15.184:6803/4051293934","heartbeat_back_addr":"172.21.15.184:6807/4051293934","heartbeat_front_addr":"172.21.15.184:6805/4051293934","state":["exists","up"]},{"osd":3,"uuid":"ca651f90-09b8-4081-a6f9-40b31268a837","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":3947485599},{"type":"v1","addr":"172.21.15.82:6811","nonce":3947485599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":3947485599},{"type":"v1","addr":"172.21.15.82:6813","nonce":3947485599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":3947485599},{"type":"v1","addr":"172.21.15.82:6817","nonce":3947485599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":3947485599},{"type":"v1","addr":"172.21.15.82:6815","nonce":3947485599}]},"public_addr":"172.21.15.82:6811/3947485599","cluster_addr":"172.21.15.82:6813/3947485599","heartbeat_back_addr":"172.21.15.82:6817/3947485599","heartbeat_front_addr":"172.21.15.82:6815/3947485599","state":["exists","up"]},{"osd":4,"uuid":"cb3dfd56-f677-40cb-9c96-8319ea5ac788","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":4160830014},{"type":"v1","addr":"172.21.15.160:6809","nonce":4160830014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":4160830014},{"type":"v1","addr":"172.21.15.160:6811","nonce":4160830014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":4160830014},{"type":"v1","addr":"172.21.15.160:6815","nonce":4160830014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6812","nonce":4160830014},{"type":"v1","addr":"172.21.15.160:6813","nonce":4160830014}]},"public_addr":"172.21.15.160:6809/4160830014","cluster_addr":"172.21.15.160:6811/4160830014","heartbeat_back_addr":"172.21.15.160:6815/4160830014","heartbeat_front_addr":"172.21.15.160:6813/4160830014","state":["exists","up"]},{"osd":5,"uuid":"c70cfcd1-a4ff-461f-8c73-479b061200fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":1046282672},{"type":"v1","addr":"172.21.15.184:6809","nonce":1046282672}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":1046282672},{"type":"v1","addr":"172.21.15.184:6811","nonce":1046282672}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":1046282672},{"type":"v1","addr":"172.21.15.184:6815","nonce":1046282672}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":1046282672},{"type":"v1","addr":"172.21.15.184:6813","nonce":1046282672}]},"public_addr":"172.21.15.184:6809/1046282672","cluster_addr":"172.21.15.184:6811/1046282672","heartbeat_back_addr":"172.21.15.184:6815/1046282672","heartbeat_front_addr":"172.21.15.184:6813/1046282672","state":["exists","up"]},{"osd":6,"uuid":"d47ce001-9dd1-4830-bcc9-888601187c6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":2789736499},{"type":"v1","addr":"172.21.15.82:6819","nonce":2789736499}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":2789736499},{"type":"v1","addr":"172.21.15.82:6821","nonce":2789736499}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":2789736499},{"type":"v1","addr":"172.21.15.82:6825","nonce":2789736499}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":2789736499},{"type":"v1","addr":"172.21.15.82:6823","nonce":2789736499}]},"public_addr":"172.21.15.82:6819/2789736499","cluster_addr":"172.21.15.82:6821/2789736499","heartbeat_back_addr":"172.21.15.82:6825/2789736499","heartbeat_front_addr":"172.21.15.82:6823/2789736499","state":["exists","up"]},{"osd":7,"uuid":"25c54f43-18cb-41e4-83b4-2732b5a93889","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6816","nonce":2340683378},{"type":"v1","addr":"172.21.15.160:6817","nonce":2340683378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":2340683378},{"type":"v1","addr":"172.21.15.160:6819","nonce":2340683378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":2340683378},{"type":"v1","addr":"172.21.15.160:6823","nonce":2340683378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6820","nonce":2340683378},{"type":"v1","addr":"172.21.15.160:6821","nonce":2340683378}]},"public_addr":"172.21.15.160:6817/2340683378","cluster_addr":"172.21.15.160:6819/2340683378","heartbeat_back_addr":"172.21.15.160:6823/2340683378","heartbeat_front_addr":"172.21.15.160:6821/2340683378","state":["exists","up"]},{"osd":8,"uuid":"1f8b478d-287f-45f5-8526-d1a58c5bc07c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":3831576372},{"type":"v1","addr":"172.21.15.184:6817","nonce":3831576372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":3831576372},{"type":"v1","addr":"172.21.15.184:6819","nonce":3831576372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":3831576372},{"type":"v1","addr":"172.21.15.184:6823","nonce":3831576372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":3831576372},{"type":"v1","addr":"172.21.15.184:6821","nonce":3831576372}]},"public_addr":"172.21.15.184:6817/3831576372","cluster_addr":"172.21.15.184:6819/3831576372","heartbeat_back_addr":"172.21.15.184:6823/3831576372","heartbeat_front_addr":"172.21.15.184:6821/3831576372","state":["exists","up"]},{"osd":9,"uuid":"0d69e319-954f-4b18-95ea-31794f18dc3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":716330782},{"type":"v1","addr":"172.21.15.82:6827","nonce":716330782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":716330782},{"type":"v1","addr":"172.21.15.82:6829","nonce":716330782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":716330782},{"type":"v1","addr":"172.21.15.82:6833","nonce":716330782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":716330782},{"type":"v1","addr":"172.21.15.82:6831","nonce":716330782}]},"public_addr":"172.21.15.82:6827/716330782","cluster_addr":"172.21.15.82:6829/716330782","heartbeat_back_addr":"172.21.15.82:6833/716330782","heartbeat_front_addr":"172.21.15.82:6831/716330782","state":["exists","up"]},{"osd":10,"uuid":"39dc1b78-61cb-42ca-b5b6-1e44ac2b6500","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":2327428014},{"type":"v1","addr":"172.21.15.160:6825","nonce":2327428014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":2327428014},{"type":"v1","addr":"172.21.15.160:6827","nonce":2327428014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":2327428014},{"type":"v1","addr":"172.21.15.160:6831","nonce":2327428014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":2327428014},{"type":"v1","addr":"172.21.15.160:6829","nonce":2327428014}]},"public_addr":"172.21.15.160:6825/2327428014","cluster_addr":"172.21.15.160:6827/2327428014","heartbeat_back_addr":"172.21.15.160:6831/2327428014","heartbeat_front_addr":"172.21.15.160:6829/2327428014","state":["exists","up"]},{"osd":11,"uuid":"4306b3c1-707a-4010-987c-209ea0e948e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":3594364650},{"type":"v1","addr":"172.21.15.184:6825","nonce":3594364650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":3594364650},{"type":"v1","addr":"172.21.15.184:6827","nonce":3594364650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3594364650},{"type":"v1","addr":"172.21.15.184:6831","nonce":3594364650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3594364650},{"type":"v1","addr":"172.21.15.184:6829","nonce":3594364650}]},"public_addr":"172.21.15.184:6825/3594364650","cluster_addr":"172.21.15.184:6827/3594364650","heartbeat_back_addr":"172.21.15.184:6831/3594364650","heartbeat_front_addr":"172.21.15.184:6829/3594364650","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:06:17.855998+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:06:41.473270+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:07:05.304108+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:07:29.887472+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:07:55.387286+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:08:20.579669+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:08:46.333051+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:09:12.289668+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:09:38.180696+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:10:04.471459+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:10:31.041912+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:10:57.477266+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.1d","mappings":[{"from":5,"to":9}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.5","mappings":[{"from":9,"to":10}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.82:0/2005424509":"2024-05-21T16:04:45.161949+0000","172.21.15.82:6801/1763049308":"2024-05-21T16:04:45.161949+0000","172.21.15.82:6801/1959459348":"2024-05-21T16:04:19.208291+0000","172.21.15.82:0/3131904556":"2024-05-21T16:04:19.208291+0000","172.21.15.82:6800/1959459348":"2024-05-21T16:04:19.208291+0000","172.21.15.82:0/2456966342":"2024-05-21T16:04:45.161949+0000","172.21.15.82:0/4179978833":"2024-05-21T16:04:19.208291+0000","172.21.15.82:0/1602862225":"2024-05-21T16:04:45.161949+0000","172.21.15.82:0/106101180":"2024-05-21T16:04:19.208291+0000","172.21.15.82:6800/1763049308":"2024-05-21T16:04:45.161949+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-20T16:14:01.150 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.150+0000 7efe4c661640 1 -- 172.21.15.82:0/224258168 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7efe2c0a2b90 msgr2=0x7efe2c0a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:14:01.150 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.150+0000 7efe4c661640 1 --2- 172.21.15.82:0/224258168 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7efe2c0a2b90 0x7efe2c0a50a0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7efe38005d00 tx=0x7efe38005c70 comp rx=0 tx=0).stop 2024-05-20T16:14:01.150 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.150+0000 7efe4c661640 1 -- 172.21.15.82:0/224258168 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efe44101970 msgr2=0x7efe441992c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:14:01.150 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.150+0000 7efe4c661640 1 --2- 172.21.15.82:0/224258168 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efe44101970 0x7efe441992c0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7efe34004af0 tx=0x7efe34031ec0 comp rx=0 tx=0).stop 2024-05-20T16:14:01.150 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.150+0000 7efe4c661640 1 -- 172.21.15.82:0/224258168 shutdown_connections 2024-05-20T16:14:01.150 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.150+0000 7efe4c661640 1 --2- 172.21.15.82:0/224258168 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7efe2c0a2b90 0x7efe2c0a50a0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:01.150 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.150+0000 7efe4c661640 1 --2- 172.21.15.82:0/224258168 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efe441078b0 0x7efe441a0790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:01.150 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.150+0000 7efe4c661640 1 --2- 172.21.15.82:0/224258168 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efe44102bb0 0x7efe44199800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:01.150 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.150+0000 7efe4c661640 1 --2- 172.21.15.82:0/224258168 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efe44101970 0x7efe441992c0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:01.151 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.150+0000 7efe4c661640 1 -- 172.21.15.82:0/224258168 >> 172.21.15.82:0/224258168 conn(0x7efe440fcb80 msgr2=0x7efe44068e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:14:01.151 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.150+0000 7efe4c661640 1 -- 172.21.15.82:0/224258168 shutdown_connections 2024-05-20T16:14:01.151 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.150+0000 7efe4c661640 1 -- 172.21.15.82:0/224258168 wait complete. 2024-05-20T16:14:01.163 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 24, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-05-20T16:11:46.751632+0000', 'modified': '2024-05-20T16:12:13.394770+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24548, 'mds_1': 24511, 'mds_2': 24506}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24470': {'gid': 24470, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.160:6835/664068363', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.160:6834', 'nonce': 664068363}, {'type': 'v1', 'addr': '172.21.15.160:6835', 'nonce': 664068363}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24478': {'gid': 24478, 'name': 'f', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.184:6835/4242156090', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.184:6834', 'nonce': 4242156090}, {'type': 'v1', 'addr': '172.21.15.184:6835', 'nonce': 4242156090}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24496': {'gid': 24496, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.184:6837/403262242', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.184:6836', 'nonce': 403262242}, {'type': 'v1', 'addr': '172.21.15.184:6837', 'nonce': 403262242}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24506': {'gid': 24506, 'name': 'h', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.160:6837/2846188680', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.160:6836', 'nonce': 2846188680}, {'type': 'v1', 'addr': '172.21.15.160:6837', 'nonce': 2846188680}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24511': {'gid': 24511, 'name': 'i', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.184:6839/2199955067', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.184:6838', 'nonce': 2199955067}, {'type': 'v1', 'addr': '172.21.15.184:6839', 'nonce': 2199955067}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24548': {'gid': 24548, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.160:6839/2765554482', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.160:6838', 'nonce': 2765554482}, {'type': 'v1', 'addr': '172.21.15.160:6839', 'nonce': 2765554482}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24548, 'qdb_cluster': [24511, 24506, 24548]} 2024-05-20T16:14:01.163 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-05-20T16:14:01.163 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2024-05-20T16:14:01.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.281+0000 7fea9b299640 1 -- 172.21.15.82:0/2483800623 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fea941006f0 msgr2=0x7fea94100b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:14:01.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.281+0000 7fea9b299640 1 --2- 172.21.15.82:0/2483800623 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fea941006f0 0x7fea94100b60 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fea90009900 tx=0x7fea9002a440 comp rx=0 tx=0).stop 2024-05-20T16:14:01.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.281+0000 7fea9b299640 1 -- 172.21.15.82:0/2483800623 shutdown_connections 2024-05-20T16:14:01.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.281+0000 7fea9b299640 1 --2- 172.21.15.82:0/2483800623 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fea941006f0 0x7fea94100b60 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:01.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.281+0000 7fea9b299640 1 --2- 172.21.15.82:0/2483800623 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fea941011f0 0x7fea9410ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:01.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.281+0000 7fea9b299640 1 --2- 172.21.15.82:0/2483800623 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fea94104040 0x7fea94104450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:01.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.281+0000 7fea9b299640 1 -- 172.21.15.82:0/2483800623 >> 172.21.15.82:0/2483800623 conn(0x7fea940fc2f0 msgr2=0x7fea940fe7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:14:01.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.281+0000 7fea9b299640 1 -- 172.21.15.82:0/2483800623 shutdown_connections 2024-05-20T16:14:01.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.281+0000 7fea9b299640 1 -- 172.21.15.82:0/2483800623 wait complete. 2024-05-20T16:14:01.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.282+0000 7fea9b299640 1 Processor -- start 2024-05-20T16:14:01.284 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.282+0000 7fea9b299640 1 -- start start 2024-05-20T16:14:01.284 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.282+0000 7fea9b299640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fea941006f0 0x7fea94199440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:14:01.284 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.282+0000 7fea9b299640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fea941011f0 0x7fea94199980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:14:01.284 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.282+0000 7fea9b299640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fea94104040 0x7fea941a0910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:14:01.284 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.282+0000 7fea9b299640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fea9410e810 con 0x7fea941006f0 2024-05-20T16:14:01.284 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.282+0000 7fea9b299640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fea9410e690 con 0x7fea941011f0 2024-05-20T16:14:01.284 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.282+0000 7fea9b299640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fea9410e990 con 0x7fea94104040 2024-05-20T16:14:01.284 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.282+0000 7fea99035640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fea941006f0 0x7fea94199440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:14:01.284 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.282+0000 7fea99035640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fea941006f0 0x7fea94199440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58558/0 (socket says 172.21.15.82:58558) 2024-05-20T16:14:01.284 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.282+0000 7fea99035640 1 -- 172.21.15.82:0/3647459570 learned_addr learned my addr 172.21.15.82:0/3647459570 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:14:01.284 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.283+0000 7fea99035640 1 -- 172.21.15.82:0/3647459570 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fea94104040 msgr2=0x7fea941a0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:14:01.284 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.283+0000 7fea99035640 1 --2- 172.21.15.82:0/3647459570 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fea94104040 0x7fea941a0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:01.284 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.283+0000 7fea99035640 1 -- 172.21.15.82:0/3647459570 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fea941011f0 msgr2=0x7fea94199980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:14:01.284 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.283+0000 7fea99035640 1 --2- 172.21.15.82:0/3647459570 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fea941011f0 0x7fea94199980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:01.285 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.283+0000 7fea99035640 1 -- 172.21.15.82:0/3647459570 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea941a0f80 con 0x7fea941006f0 2024-05-20T16:14:01.285 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.283+0000 7fea99035640 1 --2- 172.21.15.82:0/3647459570 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fea941006f0 0x7fea94199440 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fea8400b450 tx=0x7fea8400b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:14:01.285 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.283+0000 7fea8a7fc640 1 -- 172.21.15.82:0/3647459570 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea84015020 con 0x7fea941006f0 2024-05-20T16:14:01.285 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.283+0000 7fea8a7fc640 1 -- 172.21.15.82:0/3647459570 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea84011d70 con 0x7fea941006f0 2024-05-20T16:14:01.285 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.283+0000 7fea9b299640 1 -- 172.21.15.82:0/3647459570 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea941a1270 con 0x7fea941006f0 2024-05-20T16:14:01.285 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.283+0000 7fea8a7fc640 1 -- 172.21.15.82:0/3647459570 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea8400f340 con 0x7fea941006f0 2024-05-20T16:14:01.285 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.283+0000 7fea9b299640 1 -- 172.21.15.82:0/3647459570 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea941a17f0 con 0x7fea941006f0 2024-05-20T16:14:01.287 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.286+0000 7feaa221d740 1 Processor -- start 2024-05-20T16:14:01.288 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.286+0000 7feaa221d740 1 -- start start 2024-05-20T16:14:01.288 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.286+0000 7fea8a7fc640 1 -- 172.21.15.82:0/3647459570 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fea84018020 con 0x7fea941006f0 2024-05-20T16:14:01.288 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.287+0000 7feaa221d740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5594dc92d270 0x5594dcccd470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:14:01.288 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.287+0000 7feaa221d740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5594dc9043b0 0x5594dcccd9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:14:01.288 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.287+0000 7feaa221d740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5594dc90c590 0x5594dccd48f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:14:01.288 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.287+0000 7feaa221d740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x5594dcc314d0 con 0x5594dc92d270 2024-05-20T16:14:01.288 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.287+0000 7feaa221d740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x5594dcbd93e0 con 0x5594dc9043b0 2024-05-20T16:14:01.288 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.287+0000 7feaa221d740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5594dcc2d490 con 0x5594dc90c590 2024-05-20T16:14:01.288 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.287+0000 7fea99836640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5594dc92d270 0x5594dcccd470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:14:01.288 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.287+0000 7fea99836640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5594dc92d270 0x5594dcccd470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58568/0 (socket says 172.21.15.82:58568) 2024-05-20T16:14:01.288 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.287+0000 7fea99836640 1 -- 172.21.15.82:0/4084480987 learned_addr learned my addr 172.21.15.82:0/4084480987 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:14:01.288 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.287+0000 7fea99035640 1 --2- 172.21.15.82:0/4084480987 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5594dc9043b0 0x5594dcccd9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:14:01.289 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.287+0000 7fea99836640 1 -- 172.21.15.82:0/4084480987 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5594dc90c590 msgr2=0x5594dccd48f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:14:01.289 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.287+0000 7fea99836640 1 --2- 172.21.15.82:0/4084480987 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5594dc90c590 0x5594dccd48f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:01.289 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.287+0000 7fea99836640 1 -- 172.21.15.82:0/4084480987 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5594dc9043b0 msgr2=0x5594dcccd9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:14:01.289 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.287+0000 7fea99836640 1 --2- 172.21.15.82:0/4084480987 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5594dc9043b0 0x5594dcccd9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:01.289 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.287+0000 7fea99836640 1 -- 172.21.15.82:0/4084480987 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5594dc9452d0 con 0x5594dc92d270 2024-05-20T16:14:01.289 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.287+0000 7fea8a7fc640 1 --2- 172.21.15.82:0/3647459570 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fea800a2b90 0x7fea800a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:14:01.289 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.288+0000 7fea99836640 1 --2- 172.21.15.82:0/4084480987 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5594dc92d270 0x5594dcccd470 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fea8c009d20 tx=0x7fea8c00b420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37c346658b790656 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:14:01.289 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.288+0000 7fea99035640 1 --2- 172.21.15.82:0/3647459570 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fea800a2b90 0x7fea800a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:14:01.289 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.288+0000 7fea8a7fc640 1 -- 172.21.15.82:0/3647459570 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9150+0+0 (secure 0 0 0) 0x7fea840cef80 con 0x7fea941006f0 2024-05-20T16:14:01.289 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.288+0000 7fea7f7fe640 1 -- 172.21.15.82:0/4084480987 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea8c00bd40 con 0x5594dc92d270 2024-05-20T16:14:01.289 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.288+0000 7fea7f7fe640 1 -- 172.21.15.82:0/4084480987 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea8c017070 con 0x5594dc92d270 2024-05-20T16:14:01.289 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.288+0000 7fea7f7fe640 1 -- 172.21.15.82:0/4084480987 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea8c011790 con 0x5594dc92d270 2024-05-20T16:14:01.293 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.289+0000 7feaa221d740 1 -- 172.21.15.82:0/4084480987 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5594dc92d270 msgr2=0x5594dcccd470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:14:01.293 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.289+0000 7fea99035640 1 --2- 172.21.15.82:0/3647459570 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fea800a2b90 0x7fea800a50a0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x5594dccce850 tx=0x7fea840d2b40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:14:01.293 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.289+0000 7feaa221d740 1 --2- 172.21.15.82:0/4084480987 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5594dc92d270 0x5594dcccd470 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fea8c009d20 tx=0x7fea8c00b420 comp rx=0 tx=0).stop 2024-05-20T16:14:01.293 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.289+0000 7feaa221d740 1 -- 172.21.15.82:0/4084480987 shutdown_connections 2024-05-20T16:14:01.293 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.289+0000 7feaa221d740 1 --2- 172.21.15.82:0/4084480987 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5594dc90c590 0x5594dccd48f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:01.293 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.289+0000 7feaa221d740 1 --2- 172.21.15.82:0/4084480987 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5594dc9043b0 0x5594dcccd9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:01.293 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.289+0000 7feaa221d740 1 --2- 172.21.15.82:0/4084480987 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5594dc92d270 0x5594dcccd470 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:01.293 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.289+0000 7feaa221d740 1 -- 172.21.15.82:0/4084480987 >> 172.21.15.82:0/4084480987 conn(0x5594dc9213c0 msgr2=0x5594dccc8e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:14:01.293 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.289+0000 7feaa221d740 1 -- 172.21.15.82:0/4084480987 shutdown_connections 2024-05-20T16:14:01.293 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.289+0000 7feaa221d740 1 -- 172.21.15.82:0/4084480987 wait complete. 2024-05-20T16:14:01.293 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.289+0000 7feaa221d740 1 -- start start 2024-05-20T16:14:01.293 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.289+0000 7feaa221d740 1 Processor -- start 2024-05-20T16:14:01.294 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.290+0000 7feaa221d740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5594dc9043b0 0x5594dccd72d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:14:01.294 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.290+0000 7feaa221d740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5594dc90c590 0x5594dccd7810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:14:01.294 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.290+0000 7feaa221d740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5594dc92d270 0x5594dccde770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:14:01.294 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.290+0000 7feaa221d740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x5594dccd5340 con 0x5594dc90c590 2024-05-20T16:14:01.294 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.290+0000 7feaa221d740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x5594dcbec8e0 con 0x5594dc92d270 2024-05-20T16:14:01.294 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.290+0000 7feaa221d740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5594dccd54c0 con 0x5594dc9043b0 2024-05-20T16:14:01.294 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.290+0000 7feaa221d740 10 client.0 _ll_register_callbacks cb 0x5594dcbb99c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T16:14:01.294 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.290+0000 7fea98834640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5594dc90c590 0x5594dccd7810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:14:01.294 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.290+0000 7fea98834640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5594dc90c590 0x5594dccd7810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58578/0 (socket says 172.21.15.82:58578) 2024-05-20T16:14:01.294 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.290+0000 7fea98834640 1 -- 172.21.15.82:0/2393066576 learned_addr learned my addr 172.21.15.82:0/2393066576 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:14:01.294 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.290+0000 7fea99836640 1 --2- 172.21.15.82:0/2393066576 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5594dc9043b0 0x5594dccd72d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:14:01.294 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.290+0000 7fea98834640 1 -- 172.21.15.82:0/2393066576 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5594dc9043b0 msgr2=0x5594dccd72d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:14:01.294 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.290+0000 7fea98834640 1 --2- 172.21.15.82:0/2393066576 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5594dc9043b0 0x5594dccd72d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:01.294 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.290+0000 7fea98834640 1 -- 172.21.15.82:0/2393066576 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5594dc92d270 msgr2=0x5594dccde770 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-20T16:14:01.295 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.290+0000 7fea98834640 1 --2- 172.21.15.82:0/2393066576 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5594dc92d270 0x5594dccde770 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:01.295 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.291+0000 7fea98834640 1 -- 172.21.15.82:0/2393066576 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5594dca0f6b0 con 0x5594dc90c590 2024-05-20T16:14:01.295 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.291+0000 7fea98834640 1 --2- 172.21.15.82:0/2393066576 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5594dc90c590 0x5594dccd7810 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fea90005ab0 tx=0x7fea900316b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9432efc4e7943a7 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:14:01.295 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.291+0000 7fea7cff9640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-05-20T16:14:01.295 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.291+0000 7fea7cff9640 1 -- 172.21.15.82:0/2393066576 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea9003e030 con 0x5594dc90c590 2024-05-20T16:14:01.295 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.291+0000 7fea7cff9640 1 -- 172.21.15.82:0/2393066576 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9150+0+0 (secure 0 0 0) 0x7fea90004050 con 0x5594dc90c590 2024-05-20T16:14:01.295 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.291+0000 7feaa221d740 1 -- 172.21.15.82:0/2393066576 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea8c017070 con 0x5594dc90c590 2024-05-20T16:14:01.295 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.291+0000 7fea7cff9640 1 -- 172.21.15.82:0/2393066576 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fea9004b050 con 0x5594dc90c590 2024-05-20T16:14:01.295 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.291+0000 7feaa221d740 10 client.15399 fetch_fsmap learned FSMap version 24 2024-05-20T16:14:01.295 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.291+0000 7feaa221d740 1 -- 172.21.15.82:0/2393066576 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5594dcce1710 con 0x5594dc90c590 2024-05-20T16:14:01.295 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.291+0000 7fea7cff9640 1 -- 172.21.15.82:0/2393066576 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea8c017070 con 0x5594dc90c590 2024-05-20T16:14:01.295 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.291+0000 7fea7cff9640 1 -- 172.21.15.82:0/2393066576 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7fea9003a3f0 con 0x5594dc90c590 2024-05-20T16:14:01.295 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.292+0000 7feaa221d740 10 client.15399 fetch_fsmap finished waiting for FSMap version 24 2024-05-20T16:14:01.295 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.292+0000 7feaa221d740 10 client.15399 resolve_mds: resolved name 'h' to daemon mds.h 2024-05-20T16:14:01.296 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.292+0000 7feaa221d740 20 client.15399 populate_metadata read hostname 'smithi082' 2024-05-20T16:14:01.296 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.292+0000 7feaa221d740 1 --2- 172.21.15.82:0/2393066576 >> [v2:172.21.15.160:6836/2846188680,v1:172.21.15.160:6837/2846188680] conn(0x5594dc8ebe40 0x5594dcce51b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:14:01.296 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.292+0000 7feaa221d740 4 client.15399 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T16:14:01.296 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.292+0000 7feaa221d740 1 -- 172.21.15.82:0/2393066576 --> [v2:172.21.15.160:6836/2846188680,v1:172.21.15.160:6837/2846188680] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5594dcc3ec00 con 0x5594dc8ebe40 2024-05-20T16:14:01.296 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.294+0000 7fea99836640 1 --2- 172.21.15.82:0/2393066576 >> [v2:172.21.15.160:6836/2846188680,v1:172.21.15.160:6837/2846188680] conn(0x5594dc8ebe40 0x5594dcce51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:14:01.297 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.298+0000 7fea7cff9640 10 client.15399 ms_handle_connect on v2:172.21.15.160:6836/2846188680 2024-05-20T16:14:01.297 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.298+0000 7fea99836640 1 --2- 172.21.15.82:0/2393066576 >> [v2:172.21.15.160:6836/2846188680,v1:172.21.15.160:6837/2846188680] conn(0x5594dc8ebe40 0x5594dcce51b0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ffc4ef2eb5560914 server_cookie=1bbbf768b3f09d74 in_seq=0 out_seq=0 2024-05-20T16:14:01.299 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.299+0000 7fea7cff9640 1 -- 172.21.15.82:0/2393066576 <== mds.2 v2:172.21.15.160:6836/2846188680 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5594dcc3ec00 con 0x5594dc8ebe40 2024-05-20T16:14:01.299 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.299+0000 7fea7cff9640 10 client.15399 handle_command_reply: tid=0 2024-05-20T16:14:01.299 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.299+0000 7feaa221d740 1 client.15399 shutdown 2024-05-20T16:14:01.299 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.299+0000 7feaa221d740 1 -- 172.21.15.82:0/2393066576 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5594dc90c590 msgr2=0x5594dccd7810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:14:01.299 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.299+0000 7feaa221d740 1 --2- 172.21.15.82:0/2393066576 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5594dc90c590 0x5594dccd7810 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fea90005ab0 tx=0x7fea900316b0 comp rx=0 tx=0).stop 2024-05-20T16:14:01.300 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.299+0000 7feaa221d740 1 -- 172.21.15.82:0/2393066576 shutdown_connections 2024-05-20T16:14:01.300 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.299+0000 7feaa221d740 1 --2- 172.21.15.82:0/2393066576 >> [v2:172.21.15.160:6836/2846188680,v1:172.21.15.160:6837/2846188680] conn(0x5594dc8ebe40 0x5594dcce51b0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:01.300 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.299+0000 7feaa221d740 1 --2- 172.21.15.82:0/2393066576 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5594dc90c590 0x5594dccd7810 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:01.300 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.299+0000 7fea7cff9640 0 client.15399 ms_handle_reset on v2:172.21.15.160:6836/2846188680 2024-05-20T16:14:01.300 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.299+0000 7feaa221d740 1 --2- 172.21.15.82:0/2393066576 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5594dc92d270 0x5594dccde770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:01.300 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.299+0000 7feaa221d740 1 --2- 172.21.15.82:0/2393066576 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5594dc9043b0 0x5594dccd72d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:01.300 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.299+0000 7feaa221d740 1 -- 172.21.15.82:0/2393066576 >> 172.21.15.82:0/2393066576 conn(0x5594dc9213c0 msgr2=0x5594dccc7ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:14:01.300 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.299+0000 7feaa221d740 1 -- 172.21.15.82:0/2393066576 shutdown_connections 2024-05-20T16:14:01.300 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.299+0000 7feaa221d740 1 -- 172.21.15.82:0/2393066576 wait complete. 2024-05-20T16:14:01.300 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.299+0000 7feaa221d740 20 client.15399 trim_cache size 0 max 16384 2024-05-20T16:14:01.321 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.320+0000 7feaa221d740 1 Processor -- start 2024-05-20T16:14:01.321 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.320+0000 7feaa221d740 1 -- start start 2024-05-20T16:14:01.321 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.320+0000 7feaa221d740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5594dc9043b0 0x5594dccc87f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:14:01.321 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.320+0000 7feaa221d740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5594dc90c590 0x5594dccde690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:14:01.322 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.320+0000 7feaa221d740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5594dc92d270 0x5594dccc8d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:14:01.322 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.320+0000 7feaa221d740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x5594dcce34f0 con 0x5594dc90c590 2024-05-20T16:14:01.322 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.320+0000 7feaa221d740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x5594dcce3bc0 con 0x5594dc9043b0 2024-05-20T16:14:01.322 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.320+0000 7feaa221d740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5594dcce4290 con 0x5594dc92d270 2024-05-20T16:14:01.322 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.320+0000 7fea98834640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5594dc90c590 0x5594dccde690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:14:01.322 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.320+0000 7fea98834640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5594dc90c590 0x5594dccde690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58594/0 (socket says 172.21.15.82:58594) 2024-05-20T16:14:01.322 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.320+0000 7fea98834640 1 -- 172.21.15.82:0/1843754593 learned_addr learned my addr 172.21.15.82:0/1843754593 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:14:01.322 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.320+0000 7fea99836640 1 --2- 172.21.15.82:0/1843754593 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5594dc92d270 0x5594dccc8d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:14:01.322 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.321+0000 7fea98834640 1 -- 172.21.15.82:0/1843754593 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5594dc92d270 msgr2=0x5594dccc8d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:14:01.322 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.321+0000 7fea98834640 1 --2- 172.21.15.82:0/1843754593 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5594dc92d270 0x5594dccc8d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:01.322 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.321+0000 7fea99836640 1 --2- 172.21.15.82:0/1843754593 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5594dc9043b0 0x5594dccc87f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:14:01.322 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.321+0000 7fea98834640 1 -- 172.21.15.82:0/1843754593 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5594dc9043b0 msgr2=0x5594dccc87f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:14:01.322 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.321+0000 7fea98834640 1 --2- 172.21.15.82:0/1843754593 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5594dc9043b0 0x5594dccc87f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:01.323 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.321+0000 7fea98834640 1 -- 172.21.15.82:0/1843754593 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5594dccd6350 con 0x5594dc90c590 2024-05-20T16:14:01.323 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.321+0000 7fea98834640 1 --2- 172.21.15.82:0/1843754593 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5594dc90c590 0x5594dccde690 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fea9002a960 tx=0x7fea90009680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71927ab9c9266482 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:14:01.323 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.321+0000 7fea7effd640 1 -- 172.21.15.82:0/1843754593 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea9003e070 con 0x5594dc90c590 2024-05-20T16:14:01.323 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.321+0000 7fea7effd640 1 -- 172.21.15.82:0/1843754593 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea90005850 con 0x5594dc90c590 2024-05-20T16:14:01.323 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.321+0000 7fea7effd640 1 -- 172.21.15.82:0/1843754593 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea90035040 con 0x5594dc90c590 2024-05-20T16:14:01.327 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.322+0000 7feaa221d740 1 -- 172.21.15.82:0/1843754593 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5594dc90c590 msgr2=0x5594dccde690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:14:01.327 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.322+0000 7feaa221d740 1 --2- 172.21.15.82:0/1843754593 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5594dc90c590 0x5594dccde690 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fea9002a960 tx=0x7fea90009680 comp rx=0 tx=0).stop 2024-05-20T16:14:01.327 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.322+0000 7feaa221d740 1 -- 172.21.15.82:0/1843754593 shutdown_connections 2024-05-20T16:14:01.327 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.322+0000 7feaa221d740 1 --2- 172.21.15.82:0/1843754593 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5594dc90c590 0x5594dccde690 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:01.327 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.322+0000 7feaa221d740 1 --2- 172.21.15.82:0/1843754593 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5594dc92d270 0x5594dccc8d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:01.328 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.322+0000 7feaa221d740 1 --2- 172.21.15.82:0/1843754593 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5594dc9043b0 0x5594dccc87f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:01.328 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.322+0000 7feaa221d740 1 -- 172.21.15.82:0/1843754593 >> 172.21.15.82:0/1843754593 conn(0x5594dc9213c0 msgr2=0x5594dccd1730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:14:01.328 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.322+0000 7feaa221d740 1 -- 172.21.15.82:0/1843754593 shutdown_connections 2024-05-20T16:14:01.328 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.322+0000 7feaa221d740 1 -- 172.21.15.82:0/1843754593 wait complete. 2024-05-20T16:14:01.328 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.322+0000 7feaa221d740 1 -- start start 2024-05-20T16:14:01.328 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.322+0000 7feaa221d740 1 Processor -- start 2024-05-20T16:14:01.328 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.323+0000 7feaa221d740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5594dc9043b0 0x5594dccd9860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:14:01.328 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.323+0000 7feaa221d740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5594dc90c590 0x5594dccd9da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:14:01.328 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.323+0000 7feaa221d740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5594dc92d270 0x5594dccdcce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:14:01.328 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.323+0000 7feaa221d740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x5594dcce5030 con 0x5594dc9043b0 2024-05-20T16:14:01.328 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.323+0000 7feaa221d740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x5594dcce4960 con 0x5594dc92d270 2024-05-20T16:14:01.328 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.323+0000 7feaa221d740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5594dcce5700 con 0x5594dc90c590 2024-05-20T16:14:01.328 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.323+0000 7feaa221d740 10 client.0 _ll_register_callbacks cb 0x7fea90009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T16:14:01.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.323+0000 7fea99836640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5594dc9043b0 0x5594dccd9860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:14:01.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.323+0000 7fea99836640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5594dc9043b0 0x5594dccd9860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58604/0 (socket says 172.21.15.82:58604) 2024-05-20T16:14:01.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.323+0000 7fea99836640 1 -- 172.21.15.82:0/54806765 learned_addr learned my addr 172.21.15.82:0/54806765 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:14:01.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.323+0000 7fea98834640 1 --2- 172.21.15.82:0/54806765 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5594dc90c590 0x5594dccd9da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:14:01.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.323+0000 7fea99836640 1 --2- 172.21.15.82:0/54806765 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5594dc92d270 0x5594dccdcce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:14:01.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.323+0000 7fea99836640 1 -- 172.21.15.82:0/54806765 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5594dc90c590 msgr2=0x5594dccd9da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:14:01.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.323+0000 7fea99836640 1 --2- 172.21.15.82:0/54806765 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5594dc90c590 0x5594dccd9da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:01.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.323+0000 7fea99836640 1 -- 172.21.15.82:0/54806765 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5594dc92d270 msgr2=0x5594dccdcce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:14:01.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.323+0000 7fea99836640 1 --2- 172.21.15.82:0/54806765 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5594dc92d270 0x5594dccdcce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:01.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.323+0000 7fea99836640 1 -- 172.21.15.82:0/54806765 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5594dccc9270 con 0x5594dc9043b0 2024-05-20T16:14:01.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.323+0000 7fea99836640 1 --2- 172.21.15.82:0/54806765 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5594dc9043b0 0x5594dccd9860 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fea8c009d20 tx=0x7fea90005a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=480ebd7f5f3c1438 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:14:01.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.323+0000 7fea7cff9640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-05-20T16:14:01.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.324+0000 7fea7cff9640 1 -- 172.21.15.82:0/54806765 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea8c02c030 con 0x5594dc9043b0 2024-05-20T16:14:01.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.324+0000 7fea7cff9640 1 -- 172.21.15.82:0/54806765 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9150+0+0 (secure 0 0 0) 0x7fea8c00f040 con 0x5594dc9043b0 2024-05-20T16:14:01.330 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.324+0000 7feaa221d740 1 -- 172.21.15.82:0/54806765 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea90005850 con 0x5594dc9043b0 2024-05-20T16:14:01.330 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.324+0000 7fea7cff9640 1 -- 172.21.15.82:0/54806765 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fea8c038050 con 0x5594dc9043b0 2024-05-20T16:14:01.330 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.324+0000 7feaa221d740 10 client.15405 fetch_fsmap learned FSMap version 24 2024-05-20T16:14:01.330 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.324+0000 7feaa221d740 1 -- 172.21.15.82:0/54806765 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5594dccde3f0 con 0x5594dc9043b0 2024-05-20T16:14:01.330 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.324+0000 7fea7cff9640 1 -- 172.21.15.82:0/54806765 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea90005850 con 0x5594dc9043b0 2024-05-20T16:14:01.330 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.324+0000 7fea7cff9640 1 -- 172.21.15.82:0/54806765 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7fea8c011070 con 0x5594dc9043b0 2024-05-20T16:14:01.330 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.324+0000 7feaa221d740 10 client.15405 fetch_fsmap finished waiting for FSMap version 24 2024-05-20T16:14:01.330 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.324+0000 7feaa221d740 10 client.15405 resolve_mds: resolved name 'h' to daemon mds.h 2024-05-20T16:14:01.330 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.324+0000 7feaa221d740 20 client.15405 populate_metadata read hostname 'smithi082' 2024-05-20T16:14:01.330 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.324+0000 7feaa221d740 1 --2- 172.21.15.82:0/54806765 >> [v2:172.21.15.160:6836/2846188680,v1:172.21.15.160:6837/2846188680] conn(0x5594dc8c1280 0x5594dcd50e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:14:01.330 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.324+0000 7feaa221d740 4 client.15405 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-20T16:14:01.330 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.324+0000 7feaa221d740 1 -- 172.21.15.82:0/54806765 --> [v2:172.21.15.160:6836/2846188680,v1:172.21.15.160:6837/2846188680] -- command(tid 0: {"prefix": "status"}) -- 0x5594dcc38f50 con 0x5594dc8c1280 2024-05-20T16:14:01.330 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.325+0000 7fea98834640 1 --2- 172.21.15.82:0/54806765 >> [v2:172.21.15.160:6836/2846188680,v1:172.21.15.160:6837/2846188680] conn(0x5594dc8c1280 0x5594dcd50e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:14:01.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.325+0000 7fea7cff9640 10 client.15405 ms_handle_connect on v2:172.21.15.160:6836/2846188680 2024-05-20T16:14:01.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.325+0000 7fea98834640 1 --2- 172.21.15.82:0/54806765 >> [v2:172.21.15.160:6836/2846188680,v1:172.21.15.160:6837/2846188680] conn(0x5594dc8c1280 0x5594dcd50e30 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ce3d88139a6eb42d server_cookie=8deee027c3c35f0c in_seq=0 out_seq=0 2024-05-20T16:14:01.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.325+0000 7fea7cff9640 1 -- 172.21.15.82:0/54806765 <== mds.2 v2:172.21.15.160:6836/2846188680 1 ==== command_reply(tid 0: 0 ) ==== 8+0+326 (crc 0 0 0) 0x5594dcc38f50 con 0x5594dc8c1280 2024-05-20T16:14:01.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.325+0000 7fea7cff9640 10 client.15405 handle_command_reply: tid=0 2024-05-20T16:14:01.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.325+0000 7feaa221d740 1 client.15405 shutdown 2024-05-20T16:14:01.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.325+0000 7feaa221d740 1 -- 172.21.15.82:0/54806765 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5594dc9043b0 msgr2=0x5594dccd9860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:14:01.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.325+0000 7feaa221d740 1 --2- 172.21.15.82:0/54806765 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5594dc9043b0 0x5594dccd9860 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fea8c009d20 tx=0x7fea90005a20 comp rx=0 tx=0).stop 2024-05-20T16:14:01.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.325+0000 7feaa221d740 1 -- 172.21.15.82:0/54806765 shutdown_connections 2024-05-20T16:14:01.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.325+0000 7feaa221d740 1 --2- 172.21.15.82:0/54806765 >> [v2:172.21.15.160:6836/2846188680,v1:172.21.15.160:6837/2846188680] conn(0x5594dc8c1280 0x5594dcd50e30 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:01.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.325+0000 7feaa221d740 1 --2- 172.21.15.82:0/54806765 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5594dc92d270 0x5594dccdcce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:01.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.325+0000 7feaa221d740 1 --2- 172.21.15.82:0/54806765 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5594dc90c590 0x5594dccd9da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:01.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.325+0000 7feaa221d740 1 --2- 172.21.15.82:0/54806765 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5594dc9043b0 0x5594dccd9860 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:01.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.325+0000 7fea7cff9640 0 client.15405 ms_handle_reset on v2:172.21.15.160:6836/2846188680 2024-05-20T16:14:01.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.325+0000 7feaa221d740 1 -- 172.21.15.82:0/54806765 >> 172.21.15.82:0/54806765 conn(0x5594dc9213c0 msgr2=0x5594dccd1730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:14:01.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.326+0000 7feaa221d740 1 -- 172.21.15.82:0/54806765 shutdown_connections 2024-05-20T16:14:01.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.326+0000 7feaa221d740 1 -- 172.21.15.82:0/54806765 wait complete. 2024-05-20T16:14:01.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.326+0000 7feaa221d740 20 client.15405 trim_cache size 0 max 16384 2024-05-20T16:14:01.332 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-05-20T16:14:01.332 INFO:teuthology.orchestra.run.smithi082.stdout: "cluster_fsid": "59d1600a-16c2-11ef-bc9a-c7b262605968", 2024-05-20T16:14:01.332 INFO:teuthology.orchestra.run.smithi082.stdout: "whoami": 2, 2024-05-20T16:14:01.332 INFO:teuthology.orchestra.run.smithi082.stdout: "id": 24506, 2024-05-20T16:14:01.332 INFO:teuthology.orchestra.run.smithi082.stdout: "want_state": "up:active", 2024-05-20T16:14:01.332 INFO:teuthology.orchestra.run.smithi082.stdout: "state": "up:active", 2024-05-20T16:14:01.332 INFO:teuthology.orchestra.run.smithi082.stdout: "fs_name": "cephfs", 2024-05-20T16:14:01.332 INFO:teuthology.orchestra.run.smithi082.stdout: "rank_uptime": 108.93899021, 2024-05-20T16:14:01.332 INFO:teuthology.orchestra.run.smithi082.stdout: "mdsmap_epoch": 24, 2024-05-20T16:14:01.333 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_epoch": 95, 2024-05-20T16:14:01.333 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_epoch_barrier": 91, 2024-05-20T16:14:01.333 INFO:teuthology.orchestra.run.smithi082.stdout: "uptime": 149.29835811000001 2024-05-20T16:14:01.333 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-05-20T16:14:01.333 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.326+0000 7fea9b299640 1 -- 172.21.15.82:0/3647459570 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fea800a2b90 msgr2=0x7fea800a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:14:01.333 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.326+0000 7fea9b299640 1 --2- 172.21.15.82:0/3647459570 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fea800a2b90 0x7fea800a50a0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x5594dccce850 tx=0x7fea840d2b40 comp rx=0 tx=0).stop 2024-05-20T16:14:01.333 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.326+0000 7fea9b299640 1 -- 172.21.15.82:0/3647459570 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fea941006f0 msgr2=0x7fea94199440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:14:01.334 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.326+0000 7fea9b299640 1 --2- 172.21.15.82:0/3647459570 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fea941006f0 0x7fea94199440 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fea8400b450 tx=0x7fea8400b910 comp rx=0 tx=0).stop 2024-05-20T16:14:01.334 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.326+0000 7fea9b299640 1 -- 172.21.15.82:0/3647459570 shutdown_connections 2024-05-20T16:14:01.334 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.326+0000 7fea9b299640 1 --2- 172.21.15.82:0/3647459570 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fea800a2b90 0x7fea800a50a0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:01.334 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.326+0000 7fea9b299640 1 --2- 172.21.15.82:0/3647459570 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fea94104040 0x7fea941a0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:01.334 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.326+0000 7fea9b299640 1 --2- 172.21.15.82:0/3647459570 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fea941011f0 0x7fea94199980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:01.334 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.326+0000 7fea9b299640 1 --2- 172.21.15.82:0/3647459570 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fea941006f0 0x7fea94199440 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:01.334 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.326+0000 7fea9b299640 1 -- 172.21.15.82:0/3647459570 >> 172.21.15.82:0/3647459570 conn(0x7fea940fc2f0 msgr2=0x7fea94104ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:14:01.334 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.326+0000 7fea9b299640 1 -- 172.21.15.82:0/3647459570 shutdown_connections 2024-05-20T16:14:01.334 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.326+0000 7fea9b299640 1 -- 172.21.15.82:0/3647459570 wait complete. 2024-05-20T16:14:01.338 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-05-20T16:14:01.410 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:14:01 smithi082 ceph-mon[32238]: from='client.? 172.21.15.82:0/829440101' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-20T16:14:01.410 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:14:01 smithi082 ceph-mon[32238]: from='client.? 172.21.15.82:0/1801532231' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T16:14:01.411 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:14:01 smithi082 ceph-mon[32238]: from='client.? 172.21.15.82:0/224258168' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-20T16:14:01.411 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:14:01 smithi082 ceph-mon[32238]: pgmap v363: 129 pgs: 129 active+clean; 641 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s; 9 B/s, 0 objects/s recovering 2024-05-20T16:14:01.454 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.452+0000 7f0a19e61640 1 -- 172.21.15.82:0/4125729145 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0a14103cf0 msgr2=0x7f0a14104100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:14:01.455 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.452+0000 7f0a19e61640 1 --2- 172.21.15.82:0/4125729145 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0a14103cf0 0x7f0a14104100 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f0a08009900 tx=0x7f0a0802a410 comp rx=0 tx=0).stop 2024-05-20T16:14:01.455 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.452+0000 7f0a19e61640 1 -- 172.21.15.82:0/4125729145 shutdown_connections 2024-05-20T16:14:01.455 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.452+0000 7f0a19e61640 1 --2- 172.21.15.82:0/4125729145 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0a14103cf0 0x7f0a14104100 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:01.455 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.452+0000 7f0a19e61640 1 --2- 172.21.15.82:0/4125729145 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0a141089f0 0x7f0a1410ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:01.455 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.452+0000 7f0a19e61640 1 --2- 172.21.15.82:0/4125729145 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0a14102ab0 0x7f0a141037b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:01.455 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.452+0000 7f0a19e61640 1 -- 172.21.15.82:0/4125729145 >> 172.21.15.82:0/4125729145 conn(0x7f0a140fdca0 msgr2=0x7f0a14100150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:14:01.455 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.452+0000 7f0a19e61640 1 -- 172.21.15.82:0/4125729145 shutdown_connections 2024-05-20T16:14:01.455 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.452+0000 7f0a19e61640 1 -- 172.21.15.82:0/4125729145 wait complete. 2024-05-20T16:14:01.455 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.453+0000 7f0a19e61640 1 Processor -- start 2024-05-20T16:14:01.455 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.453+0000 7f0a19e61640 1 -- start start 2024-05-20T16:14:01.456 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.453+0000 7f0a19e61640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0a14102ab0 0x7f0a14199270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:14:01.456 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.453+0000 7f0a19e61640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0a14103cf0 0x7f0a141997b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:14:01.456 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.453+0000 7f0a19e61640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0a141089f0 0x7f0a141a0740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:14:01.456 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.453+0000 7f0a19e61640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f0a1410e750 con 0x7f0a14103cf0 2024-05-20T16:14:01.456 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.453+0000 7f0a19e61640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f0a1410e5d0 con 0x7f0a141089f0 2024-05-20T16:14:01.456 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.453+0000 7f0a19e61640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f0a1410e8d0 con 0x7f0a14102ab0 2024-05-20T16:14:01.456 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.453+0000 7f0a12ffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0a14103cf0 0x7f0a141997b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:14:01.456 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.453+0000 7f0a12ffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0a14103cf0 0x7f0a141997b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58628/0 (socket says 172.21.15.82:58628) 2024-05-20T16:14:01.456 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.453+0000 7f0a12ffd640 1 -- 172.21.15.82:0/1830346232 learned_addr learned my addr 172.21.15.82:0/1830346232 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:14:01.456 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.454+0000 7f0a13fff640 1 --2- 172.21.15.82:0/1830346232 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0a141089f0 0x7f0a141a0740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:14:01.456 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.454+0000 7f0a137fe640 1 --2- 172.21.15.82:0/1830346232 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0a14102ab0 0x7f0a14199270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:14:01.456 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.454+0000 7f0a12ffd640 1 -- 172.21.15.82:0/1830346232 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0a14102ab0 msgr2=0x7f0a14199270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:14:01.457 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.454+0000 7f0a12ffd640 1 --2- 172.21.15.82:0/1830346232 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0a14102ab0 0x7f0a14199270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:01.457 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.454+0000 7f0a12ffd640 1 -- 172.21.15.82:0/1830346232 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0a141089f0 msgr2=0x7f0a141a0740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:14:01.457 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.454+0000 7f0a12ffd640 1 --2- 172.21.15.82:0/1830346232 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0a141089f0 0x7f0a141a0740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:01.457 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.454+0000 7f0a12ffd640 1 -- 172.21.15.82:0/1830346232 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a141a0db0 con 0x7f0a14103cf0 2024-05-20T16:14:01.457 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.454+0000 7f0a137fe640 1 --2- 172.21.15.82:0/1830346232 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0a14102ab0 0x7f0a14199270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T16:14:01.457 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.454+0000 7f0a13fff640 1 --2- 172.21.15.82:0/1830346232 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0a141089f0 0x7f0a141a0740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T16:14:01.457 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.454+0000 7f0a12ffd640 1 --2- 172.21.15.82:0/1830346232 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0a14103cf0 0x7f0a141997b0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f0a08031440 tx=0x7f0a08032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:14:01.457 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.454+0000 7f0a10ff9640 1 -- 172.21.15.82:0/1830346232 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a0803f030 con 0x7f0a14103cf0 2024-05-20T16:14:01.457 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.454+0000 7f0a10ff9640 1 -- 172.21.15.82:0/1830346232 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0a08003ce0 con 0x7f0a14103cf0 2024-05-20T16:14:01.457 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.454+0000 7f0a19e61640 1 -- 172.21.15.82:0/1830346232 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a141a1040 con 0x7f0a14103cf0 2024-05-20T16:14:01.457 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.454+0000 7f0a10ff9640 1 -- 172.21.15.82:0/1830346232 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a08035070 con 0x7f0a14103cf0 2024-05-20T16:14:01.457 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.454+0000 7f0a19e61640 1 -- 172.21.15.82:0/1830346232 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a141a1530 con 0x7f0a14103cf0 2024-05-20T16:14:01.464 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.457+0000 7f0a10ff9640 1 -- 172.21.15.82:0/1830346232 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f0a08042030 con 0x7f0a14103cf0 2024-05-20T16:14:01.464 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.457+0000 7f0a20de1740 1 Processor -- start 2024-05-20T16:14:01.464 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.457+0000 7f0a20de1740 1 -- start start 2024-05-20T16:14:01.464 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.457+0000 7f0a20de1740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563e94d8c270 0x563e9512c470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:14:01.464 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.457+0000 7f0a20de1740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563e94d633b0 0x563e9512c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:14:01.464 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.457+0000 7f0a20de1740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563e94d6b590 0x563e951338f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:14:01.464 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.457+0000 7f0a20de1740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x563e950904d0 con 0x563e94d8c270 2024-05-20T16:14:01.464 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.457+0000 7f0a20de1740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x563e950383e0 con 0x563e94d6b590 2024-05-20T16:14:01.464 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.457+0000 7f0a20de1740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x563e9508c490 con 0x563e94d633b0 2024-05-20T16:14:01.464 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.457+0000 7f0a13fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563e94d8c270 0x563e9512c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:14:01.464 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.457+0000 7f0a13fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563e94d8c270 0x563e9512c470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58644/0 (socket says 172.21.15.82:58644) 2024-05-20T16:14:01.465 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.457+0000 7f0a13fff640 1 -- 172.21.15.82:0/1352308700 learned_addr learned my addr 172.21.15.82:0/1352308700 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:14:01.465 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.457+0000 7f0a137fe640 1 --2- 172.21.15.82:0/1352308700 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563e94d633b0 0x563e9512c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:14:01.465 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.457+0000 7f0a12ffd640 1 --2- 172.21.15.82:0/1352308700 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563e94d6b590 0x563e951338f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:14:01.465 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.458+0000 7f0a10ff9640 1 --2- 172.21.15.82:0/1830346232 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f09fc0a2b90 0x7f09fc0a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:14:01.465 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.458+0000 7f0a13fff640 1 -- 172.21.15.82:0/1352308700 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563e94d633b0 msgr2=0x563e9512c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:14:01.465 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.458+0000 7f0a13fff640 1 --2- 172.21.15.82:0/1352308700 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563e94d633b0 0x563e9512c9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:01.465 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.458+0000 7f0a13fff640 1 -- 172.21.15.82:0/1352308700 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563e94d6b590 msgr2=0x563e951338f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:14:01.465 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.458+0000 7f0a13fff640 1 --2- 172.21.15.82:0/1352308700 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563e94d6b590 0x563e951338f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:01.465 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.458+0000 7f0a13fff640 1 -- 172.21.15.82:0/1352308700 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563e94da42d0 con 0x563e94d8c270 2024-05-20T16:14:01.465 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.458+0000 7f0a10ff9640 1 -- 172.21.15.82:0/1830346232 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9150+0+0 (secure 0 0 0) 0x7f0a080efa40 con 0x7f0a14103cf0 2024-05-20T16:14:01.465 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.458+0000 7f0a13fff640 1 --2- 172.21.15.82:0/1352308700 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563e94d8c270 0x563e9512c470 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f0a0c009fb0 tx=0x7f0a0c010800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69d22bd0a32e108f server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:14:01.465 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.458+0000 7f0a13fff640 1 --2- 172.21.15.82:0/1830346232 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f09fc0a2b90 0x7f09fc0a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:14:01.465 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.458+0000 7f09f9ffb640 1 -- 172.21.15.82:0/1352308700 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a0c022070 con 0x563e94d8c270 2024-05-20T16:14:01.466 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.458+0000 7f09f9ffb640 1 -- 172.21.15.82:0/1352308700 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0a0c0113e0 con 0x563e94d8c270 2024-05-20T16:14:01.466 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.458+0000 7f09f9ffb640 1 -- 172.21.15.82:0/1352308700 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a0c016070 con 0x563e94d8c270 2024-05-20T16:14:01.466 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.459+0000 7f0a20de1740 1 -- 172.21.15.82:0/1352308700 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563e94d8c270 msgr2=0x563e9512c470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:14:01.466 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.459+0000 7f0a13fff640 1 --2- 172.21.15.82:0/1830346232 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f09fc0a2b90 0x7f09fc0a50a0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f0a0c01f000 tx=0x7f0a0c022000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:14:01.466 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.459+0000 7f0a20de1740 1 --2- 172.21.15.82:0/1352308700 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563e94d8c270 0x563e9512c470 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f0a0c009fb0 tx=0x7f0a0c010800 comp rx=0 tx=0).stop 2024-05-20T16:14:01.466 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.459+0000 7f0a20de1740 1 -- 172.21.15.82:0/1352308700 shutdown_connections 2024-05-20T16:14:01.466 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.459+0000 7f0a20de1740 1 --2- 172.21.15.82:0/1352308700 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563e94d6b590 0x563e951338f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:01.466 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.459+0000 7f0a20de1740 1 --2- 172.21.15.82:0/1352308700 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563e94d633b0 0x563e9512c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:01.466 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.459+0000 7f0a20de1740 1 --2- 172.21.15.82:0/1352308700 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563e94d8c270 0x563e9512c470 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:01.466 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.459+0000 7f0a20de1740 1 -- 172.21.15.82:0/1352308700 >> 172.21.15.82:0/1352308700 conn(0x563e94d803c0 msgr2=0x563e95127e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:14:01.466 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.459+0000 7f0a20de1740 1 -- 172.21.15.82:0/1352308700 shutdown_connections 2024-05-20T16:14:01.466 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.459+0000 7f0a20de1740 1 -- 172.21.15.82:0/1352308700 wait complete. 2024-05-20T16:14:01.466 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.459+0000 7f0a20de1740 1 -- start start 2024-05-20T16:14:01.466 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.460+0000 7f0a20de1740 1 Processor -- start 2024-05-20T16:14:01.467 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.460+0000 7f0a20de1740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563e94d633b0 0x563e951362c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:14:01.467 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.460+0000 7f0a20de1740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563e94d6b590 0x563e95136800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:14:01.467 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.460+0000 7f0a20de1740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563e94d8c270 0x563e9513d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:14:01.467 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.460+0000 7f0a20de1740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x563e95134340 con 0x563e94d633b0 2024-05-20T16:14:01.467 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.460+0000 7f0a20de1740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x563e9504b8e0 con 0x563e94d6b590 2024-05-20T16:14:01.467 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.460+0000 7f0a20de1740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x563e951344c0 con 0x563e94d8c270 2024-05-20T16:14:01.467 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.460+0000 7f0a20de1740 10 client.0 _ll_register_callbacks cb 0x563e950189c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T16:14:01.467 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.460+0000 7f0a137fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563e94d633b0 0x563e951362c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:14:01.467 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.460+0000 7f0a137fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563e94d633b0 0x563e951362c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58654/0 (socket says 172.21.15.82:58654) 2024-05-20T16:14:01.467 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.460+0000 7f0a137fe640 1 -- 172.21.15.82:0/1415165924 learned_addr learned my addr 172.21.15.82:0/1415165924 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:14:01.467 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.460+0000 7f0a13fff640 1 --2- 172.21.15.82:0/1415165924 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563e94d8c270 0x563e9513d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:14:01.467 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.460+0000 7f0a137fe640 1 -- 172.21.15.82:0/1415165924 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563e94d8c270 msgr2=0x563e9513d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:14:01.467 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.460+0000 7f0a137fe640 1 --2- 172.21.15.82:0/1415165924 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563e94d8c270 0x563e9513d760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:01.468 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.460+0000 7f0a137fe640 1 -- 172.21.15.82:0/1415165924 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563e94d6b590 msgr2=0x563e95136800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:14:01.468 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.460+0000 7f0a137fe640 1 --2- 172.21.15.82:0/1415165924 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563e94d6b590 0x563e95136800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:01.468 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.460+0000 7f0a137fe640 1 -- 172.21.15.82:0/1415165924 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563e94e6e6b0 con 0x563e94d633b0 2024-05-20T16:14:01.468 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.461+0000 7f0a137fe640 1 --2- 172.21.15.82:0/1415165924 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563e94d633b0 0x563e951362c0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f0a04010750 tx=0x7f0a04010c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c76fbd01d42e7cf server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:14:01.468 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.461+0000 7f09f2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-05-20T16:14:01.468 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.461+0000 7f09f2ffd640 1 -- 172.21.15.82:0/1415165924 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a0401d060 con 0x563e94d633b0 2024-05-20T16:14:01.468 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.461+0000 7f09f2ffd640 1 -- 172.21.15.82:0/1415165924 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9150+0+0 (secure 0 0 0) 0x7f0a0400ed40 con 0x563e94d633b0 2024-05-20T16:14:01.468 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.461+0000 7f0a20de1740 1 -- 172.21.15.82:0/1415165924 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0a0c0113e0 con 0x563e94d633b0 2024-05-20T16:14:01.468 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.461+0000 7f09f2ffd640 1 -- 172.21.15.82:0/1415165924 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f0a04024050 con 0x563e94d633b0 2024-05-20T16:14:01.468 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.461+0000 7f0a20de1740 10 client.15417 fetch_fsmap learned FSMap version 24 2024-05-20T16:14:01.468 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.461+0000 7f0a20de1740 1 -- 172.21.15.82:0/1415165924 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x563e95140760 con 0x563e94d633b0 2024-05-20T16:14:01.468 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.461+0000 7f09f2ffd640 1 -- 172.21.15.82:0/1415165924 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0a0c0113e0 con 0x563e94d633b0 2024-05-20T16:14:01.469 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.461+0000 7f09f2ffd640 1 -- 172.21.15.82:0/1415165924 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f0a04016470 con 0x563e94d633b0 2024-05-20T16:14:01.469 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.461+0000 7f0a20de1740 10 client.15417 fetch_fsmap finished waiting for FSMap version 24 2024-05-20T16:14:01.469 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.461+0000 7f0a20de1740 10 client.15417 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-20T16:14:01.469 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.461+0000 7f0a20de1740 20 client.15417 populate_metadata read hostname 'smithi082' 2024-05-20T16:14:01.469 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.461+0000 7f0a20de1740 1 --2- 172.21.15.82:0/1415165924 >> [v2:172.21.15.184:6838/2199955067,v1:172.21.15.184:6839/2199955067] conn(0x563e94d4ae40 0x563e951442b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:14:01.469 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.461+0000 7f0a20de1740 4 client.15417 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T16:14:01.469 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.461+0000 7f0a20de1740 1 -- 172.21.15.82:0/1415165924 --> [v2:172.21.15.184:6838/2199955067,v1:172.21.15.184:6839/2199955067] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563e9509dc00 con 0x563e94d4ae40 2024-05-20T16:14:01.469 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.462+0000 7f0a12ffd640 1 --2- 172.21.15.82:0/1415165924 >> [v2:172.21.15.184:6838/2199955067,v1:172.21.15.184:6839/2199955067] conn(0x563e94d4ae40 0x563e951442b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:14:01.469 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.462+0000 7f09f2ffd640 10 client.15417 ms_handle_connect on v2:172.21.15.184:6838/2199955067 2024-05-20T16:14:01.469 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.462+0000 7f0a12ffd640 1 --2- 172.21.15.82:0/1415165924 >> [v2:172.21.15.184:6838/2199955067,v1:172.21.15.184:6839/2199955067] conn(0x563e94d4ae40 0x563e951442b0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=75eac9aa3b124bfc server_cookie=4ce0acf93176c06e in_seq=0 out_seq=0 2024-05-20T16:14:01.469 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.463+0000 7f09f2ffd640 1 -- 172.21.15.82:0/1415165924 <== mds.1 v2:172.21.15.184:6838/2199955067 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x563e9509dc00 con 0x563e94d4ae40 2024-05-20T16:14:01.469 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.463+0000 7f09f2ffd640 10 client.15417 handle_command_reply: tid=0 2024-05-20T16:14:01.469 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.463+0000 7f0a20de1740 1 client.15417 shutdown 2024-05-20T16:14:01.469 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.463+0000 7f0a20de1740 1 -- 172.21.15.82:0/1415165924 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563e94d633b0 msgr2=0x563e951362c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:14:01.470 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.463+0000 7f0a20de1740 1 --2- 172.21.15.82:0/1415165924 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563e94d633b0 0x563e951362c0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f0a04010750 tx=0x7f0a04010c10 comp rx=0 tx=0).stop 2024-05-20T16:14:01.470 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.463+0000 7f0a20de1740 1 -- 172.21.15.82:0/1415165924 shutdown_connections 2024-05-20T16:14:01.470 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.463+0000 7f0a20de1740 1 --2- 172.21.15.82:0/1415165924 >> [v2:172.21.15.184:6838/2199955067,v1:172.21.15.184:6839/2199955067] conn(0x563e94d4ae40 0x563e951442b0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:01.470 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.463+0000 7f0a20de1740 1 --2- 172.21.15.82:0/1415165924 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563e94d8c270 0x563e9513d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:01.470 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.463+0000 7f0a20de1740 1 --2- 172.21.15.82:0/1415165924 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563e94d6b590 0x563e95136800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:01.470 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.463+0000 7f09f2ffd640 0 client.15417 ms_handle_reset on v2:172.21.15.184:6838/2199955067 2024-05-20T16:14:01.470 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.463+0000 7f0a20de1740 1 --2- 172.21.15.82:0/1415165924 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563e94d633b0 0x563e951362c0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:01.470 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.463+0000 7f0a20de1740 1 -- 172.21.15.82:0/1415165924 >> 172.21.15.82:0/1415165924 conn(0x563e94d803c0 msgr2=0x563e95126ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:14:01.470 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.463+0000 7f0a20de1740 1 -- 172.21.15.82:0/1415165924 shutdown_connections 2024-05-20T16:14:01.470 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.463+0000 7f0a20de1740 1 -- 172.21.15.82:0/1415165924 wait complete. 2024-05-20T16:14:01.470 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.463+0000 7f0a20de1740 20 client.15417 trim_cache size 0 max 16384 2024-05-20T16:14:01.487 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.482+0000 7f0a20de1740 1 Processor -- start 2024-05-20T16:14:01.487 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.482+0000 7f0a20de1740 1 -- start start 2024-05-20T16:14:01.487 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.482+0000 7f0a20de1740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563e94d4ae40 0x563e95134860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:14:01.488 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.482+0000 7f0a20de1740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563e94d633b0 0x563e95134da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:14:01.488 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.482+0000 7f0a20de1740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563e94d6b590 0x563e951277f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:14:01.488 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.482+0000 7f0a20de1740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x563e951425f0 con 0x563e94d6b590 2024-05-20T16:14:01.488 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.482+0000 7f0a20de1740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x563e95142cc0 con 0x563e94d633b0 2024-05-20T16:14:01.488 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.482+0000 7f0a20de1740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x563e95143390 con 0x563e94d4ae40 2024-05-20T16:14:01.488 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.482+0000 7f0a13fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563e94d6b590 0x563e951277f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:14:01.488 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.482+0000 7f0a13fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563e94d6b590 0x563e951277f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58660/0 (socket says 172.21.15.82:58660) 2024-05-20T16:14:01.488 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.482+0000 7f0a13fff640 1 -- 172.21.15.82:0/446347147 learned_addr learned my addr 172.21.15.82:0/446347147 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:14:01.488 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.482+0000 7f0a137fe640 1 --2- 172.21.15.82:0/446347147 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563e94d4ae40 0x563e95134860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:14:01.488 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.482+0000 7f0a12ffd640 1 --2- 172.21.15.82:0/446347147 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563e94d633b0 0x563e95134da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:14:01.488 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.482+0000 7f0a13fff640 1 -- 172.21.15.82:0/446347147 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563e94d4ae40 msgr2=0x563e95134860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:14:01.488 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.482+0000 7f0a13fff640 1 --2- 172.21.15.82:0/446347147 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563e94d4ae40 0x563e95134860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:01.488 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.482+0000 7f0a13fff640 1 -- 172.21.15.82:0/446347147 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563e94d633b0 msgr2=0x563e95134da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:14:01.489 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.482+0000 7f0a13fff640 1 --2- 172.21.15.82:0/446347147 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563e94d633b0 0x563e95134da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:01.489 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.482+0000 7f0a13fff640 1 -- 172.21.15.82:0/446347147 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563e951373b0 con 0x563e94d6b590 2024-05-20T16:14:01.489 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.482+0000 7f0a12ffd640 1 --2- 172.21.15.82:0/446347147 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563e94d633b0 0x563e95134da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T16:14:01.489 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.482+0000 7f0a137fe640 1 --2- 172.21.15.82:0/446347147 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563e94d4ae40 0x563e95134860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T16:14:01.489 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.483+0000 7f0a13fff640 1 --2- 172.21.15.82:0/446347147 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563e94d6b590 0x563e951277f0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x563e9512d640 tx=0x7f0a0c027000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee86704ebe5868fe server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:14:01.489 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.483+0000 7f09f97fa640 1 -- 172.21.15.82:0/446347147 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a0c036070 con 0x563e94d6b590 2024-05-20T16:14:01.489 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.483+0000 7f09f97fa640 1 -- 172.21.15.82:0/446347147 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0a0c012a10 con 0x563e94d6b590 2024-05-20T16:14:01.489 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.483+0000 7f09f97fa640 1 -- 172.21.15.82:0/446347147 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a0c016050 con 0x563e94d6b590 2024-05-20T16:14:01.489 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.484+0000 7f0a20de1740 1 -- 172.21.15.82:0/446347147 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563e94d6b590 msgr2=0x563e951277f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:14:01.489 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.484+0000 7f0a20de1740 1 --2- 172.21.15.82:0/446347147 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563e94d6b590 0x563e951277f0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x563e9512d640 tx=0x7f0a0c027000 comp rx=0 tx=0).stop 2024-05-20T16:14:01.489 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.484+0000 7f0a20de1740 1 -- 172.21.15.82:0/446347147 shutdown_connections 2024-05-20T16:14:01.489 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.484+0000 7f0a20de1740 1 --2- 172.21.15.82:0/446347147 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563e94d6b590 0x563e951277f0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:01.489 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.484+0000 7f0a20de1740 1 --2- 172.21.15.82:0/446347147 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563e94d633b0 0x563e95134da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:01.490 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.484+0000 7f0a20de1740 1 --2- 172.21.15.82:0/446347147 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563e94d4ae40 0x563e95134860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:01.490 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.484+0000 7f0a20de1740 1 -- 172.21.15.82:0/446347147 >> 172.21.15.82:0/446347147 conn(0x563e94d803c0 msgr2=0x563e95141c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:14:01.490 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.484+0000 7f0a20de1740 1 -- 172.21.15.82:0/446347147 shutdown_connections 2024-05-20T16:14:01.490 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.484+0000 7f0a20de1740 1 -- 172.21.15.82:0/446347147 wait complete. 2024-05-20T16:14:01.490 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.484+0000 7f0a20de1740 1 -- start start 2024-05-20T16:14:01.490 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.484+0000 7f0a20de1740 1 Processor -- start 2024-05-20T16:14:01.490 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.484+0000 7f0a20de1740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563e94d4ae40 0x563e95138040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:14:01.490 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.484+0000 7f0a20de1740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563e94d633b0 0x563e95138580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:14:01.490 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.484+0000 7f0a20de1740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563e94d6b590 0x563e9513b400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:14:01.490 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.484+0000 7f0a20de1740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x563e95144130 con 0x563e94d4ae40 2024-05-20T16:14:01.490 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.484+0000 7f0a20de1740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x563e95143a60 con 0x563e94d6b590 2024-05-20T16:14:01.490 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.484+0000 7f0a20de1740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x563e95144800 con 0x563e94d633b0 2024-05-20T16:14:01.490 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.484+0000 7f0a20de1740 10 client.0 _ll_register_callbacks cb 0x7f0a0400b630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T16:14:01.490 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.485+0000 7f0a137fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563e94d4ae40 0x563e95138040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:14:01.491 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.485+0000 7f0a137fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563e94d4ae40 0x563e95138040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58674/0 (socket says 172.21.15.82:58674) 2024-05-20T16:14:01.491 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.485+0000 7f0a137fe640 1 -- 172.21.15.82:0/2188975097 learned_addr learned my addr 172.21.15.82:0/2188975097 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:14:01.491 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.485+0000 7f0a137fe640 1 -- 172.21.15.82:0/2188975097 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563e94d633b0 msgr2=0x563e95138580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:14:01.491 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.485+0000 7f0a12ffd640 1 --2- 172.21.15.82:0/2188975097 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563e94d633b0 0x563e95138580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:14:01.491 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.485+0000 7f0a13fff640 1 --2- 172.21.15.82:0/2188975097 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563e94d6b590 0x563e9513b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:14:01.491 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.485+0000 7f0a137fe640 1 --2- 172.21.15.82:0/2188975097 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563e94d633b0 0x563e95138580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:01.491 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.485+0000 7f0a137fe640 1 -- 172.21.15.82:0/2188975097 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563e94d6b590 msgr2=0x563e9513b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:14:01.491 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.485+0000 7f0a137fe640 1 --2- 172.21.15.82:0/2188975097 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563e94d6b590 0x563e9513b400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:01.491 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.485+0000 7f0a137fe640 1 -- 172.21.15.82:0/2188975097 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563e9513b9c0 con 0x563e94d4ae40 2024-05-20T16:14:01.491 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.485+0000 7f0a137fe640 1 --2- 172.21.15.82:0/2188975097 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563e94d4ae40 0x563e95138040 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f0a04016840 tx=0x7f0a0400bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48a1adb73aa22cff server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:14:01.491 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.485+0000 7f09f2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-05-20T16:14:01.491 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.485+0000 7f09f2ffd640 1 -- 172.21.15.82:0/2188975097 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a04010e60 con 0x563e94d4ae40 2024-05-20T16:14:01.492 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.485+0000 7f09f2ffd640 1 -- 172.21.15.82:0/2188975097 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9150+0+0 (secure 0 0 0) 0x7f0a04016330 con 0x563e94d4ae40 2024-05-20T16:14:01.492 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.485+0000 7f0a20de1740 1 -- 172.21.15.82:0/2188975097 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0a0c012a10 con 0x563e94d4ae40 2024-05-20T16:14:01.492 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.485+0000 7f09f2ffd640 1 -- 172.21.15.82:0/2188975097 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f0a0401d030 con 0x563e94d4ae40 2024-05-20T16:14:01.492 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.485+0000 7f0a20de1740 10 client.15423 fetch_fsmap learned FSMap version 24 2024-05-20T16:14:01.492 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.485+0000 7f0a20de1740 1 -- 172.21.15.82:0/2188975097 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x563e9513cc80 con 0x563e94d4ae40 2024-05-20T16:14:01.492 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.485+0000 7f09f2ffd640 1 -- 172.21.15.82:0/2188975097 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0a0c012a10 con 0x563e94d4ae40 2024-05-20T16:14:01.492 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.486+0000 7f09f2ffd640 1 -- 172.21.15.82:0/2188975097 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f0a0402e020 con 0x563e94d4ae40 2024-05-20T16:14:01.492 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.486+0000 7f0a20de1740 10 client.15423 fetch_fsmap finished waiting for FSMap version 24 2024-05-20T16:14:01.492 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.486+0000 7f0a20de1740 10 client.15423 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-20T16:14:01.492 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.486+0000 7f0a20de1740 20 client.15423 populate_metadata read hostname 'smithi082' 2024-05-20T16:14:01.492 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.486+0000 7f0a20de1740 1 --2- 172.21.15.82:0/2188975097 >> [v2:172.21.15.184:6838/2199955067,v1:172.21.15.184:6839/2199955067] conn(0x563e94d8c270 0x563e951b0410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:14:01.492 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.486+0000 7f0a20de1740 4 client.15423 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-20T16:14:01.492 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.486+0000 7f0a20de1740 1 -- 172.21.15.82:0/2188975097 --> [v2:172.21.15.184:6838/2199955067,v1:172.21.15.184:6839/2199955067] -- command(tid 0: {"prefix": "status"}) -- 0x563e95097f50 con 0x563e94d8c270 2024-05-20T16:14:01.492 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.486+0000 7f0a12ffd640 1 --2- 172.21.15.82:0/2188975097 >> [v2:172.21.15.184:6838/2199955067,v1:172.21.15.184:6839/2199955067] conn(0x563e94d8c270 0x563e951b0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:14:01.493 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.486+0000 7f09f2ffd640 10 client.15423 ms_handle_connect on v2:172.21.15.184:6838/2199955067 2024-05-20T16:14:01.493 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.486+0000 7f0a12ffd640 1 --2- 172.21.15.82:0/2188975097 >> [v2:172.21.15.184:6838/2199955067,v1:172.21.15.184:6839/2199955067] conn(0x563e94d8c270 0x563e951b0410 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d7497bc85598cec7 server_cookie=a02f178266a70313 in_seq=0 out_seq=0 2024-05-20T16:14:01.493 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.487+0000 7f09f2ffd640 1 -- 172.21.15.82:0/2188975097 <== mds.1 v2:172.21.15.184:6838/2199955067 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x563e95097f50 con 0x563e94d8c270 2024-05-20T16:14:01.493 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.487+0000 7f09f2ffd640 10 client.15423 handle_command_reply: tid=0 2024-05-20T16:14:01.493 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.487+0000 7f0a20de1740 1 client.15423 shutdown 2024-05-20T16:14:01.493 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.487+0000 7f0a20de1740 1 -- 172.21.15.82:0/2188975097 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563e94d4ae40 msgr2=0x563e95138040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:14:01.493 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.487+0000 7f0a20de1740 1 --2- 172.21.15.82:0/2188975097 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563e94d4ae40 0x563e95138040 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f0a04016840 tx=0x7f0a0400bc30 comp rx=0 tx=0).stop 2024-05-20T16:14:01.493 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.487+0000 7f0a20de1740 1 -- 172.21.15.82:0/2188975097 shutdown_connections 2024-05-20T16:14:01.493 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.487+0000 7f0a20de1740 1 --2- 172.21.15.82:0/2188975097 >> [v2:172.21.15.184:6838/2199955067,v1:172.21.15.184:6839/2199955067] conn(0x563e94d8c270 0x563e951b0410 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:01.493 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.487+0000 7f0a20de1740 1 --2- 172.21.15.82:0/2188975097 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563e94d6b590 0x563e9513b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:01.493 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.487+0000 7f0a20de1740 1 --2- 172.21.15.82:0/2188975097 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563e94d633b0 0x563e95138580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:01.493 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.487+0000 7f09f2ffd640 0 client.15423 ms_handle_reset on v2:172.21.15.184:6838/2199955067 2024-05-20T16:14:01.493 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.487+0000 7f0a20de1740 1 --2- 172.21.15.82:0/2188975097 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563e94d4ae40 0x563e95138040 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:01.494 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.487+0000 7f0a20de1740 1 -- 172.21.15.82:0/2188975097 >> 172.21.15.82:0/2188975097 conn(0x563e94d803c0 msgr2=0x563e95134860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:14:01.494 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.487+0000 7f0a20de1740 1 -- 172.21.15.82:0/2188975097 shutdown_connections 2024-05-20T16:14:01.494 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.487+0000 7f0a20de1740 1 -- 172.21.15.82:0/2188975097 wait complete. 2024-05-20T16:14:01.494 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.487+0000 7f0a20de1740 20 client.15423 trim_cache size 0 max 16384 2024-05-20T16:14:01.494 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-05-20T16:14:01.494 INFO:teuthology.orchestra.run.smithi082.stdout: "cluster_fsid": "59d1600a-16c2-11ef-bc9a-c7b262605968", 2024-05-20T16:14:01.494 INFO:teuthology.orchestra.run.smithi082.stdout: "whoami": 1, 2024-05-20T16:14:01.494 INFO:teuthology.orchestra.run.smithi082.stdout: "id": 24511, 2024-05-20T16:14:01.494 INFO:teuthology.orchestra.run.smithi082.stdout: "want_state": "up:active", 2024-05-20T16:14:01.494 INFO:teuthology.orchestra.run.smithi082.stdout: "state": "up:active", 2024-05-20T16:14:01.495 INFO:teuthology.orchestra.run.smithi082.stdout: "fs_name": "cephfs", 2024-05-20T16:14:01.495 INFO:teuthology.orchestra.run.smithi082.stdout: "rank_uptime": 110.111290817, 2024-05-20T16:14:01.495 INFO:teuthology.orchestra.run.smithi082.stdout: "mdsmap_epoch": 24, 2024-05-20T16:14:01.495 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_epoch": 95, 2024-05-20T16:14:01.495 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_epoch_barrier": 91, 2024-05-20T16:14:01.495 INFO:teuthology.orchestra.run.smithi082.stdout: "uptime": 146.71035966400001 2024-05-20T16:14:01.495 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-05-20T16:14:01.496 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.487+0000 7f0a19e61640 1 -- 172.21.15.82:0/1830346232 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f09fc0a2b90 msgr2=0x7f09fc0a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:14:01.496 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.487+0000 7f0a19e61640 1 --2- 172.21.15.82:0/1830346232 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f09fc0a2b90 0x7f09fc0a50a0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f0a0c01f000 tx=0x7f0a0c022000 comp rx=0 tx=0).stop 2024-05-20T16:14:01.496 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.487+0000 7f0a19e61640 1 -- 172.21.15.82:0/1830346232 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0a14103cf0 msgr2=0x7f0a141997b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:14:01.496 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.487+0000 7f0a19e61640 1 --2- 172.21.15.82:0/1830346232 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0a14103cf0 0x7f0a141997b0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f0a08031440 tx=0x7f0a08032ce0 comp rx=0 tx=0).stop 2024-05-20T16:14:01.496 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.487+0000 7f0a19e61640 1 -- 172.21.15.82:0/1830346232 shutdown_connections 2024-05-20T16:14:01.496 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.487+0000 7f0a19e61640 1 --2- 172.21.15.82:0/1830346232 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f09fc0a2b90 0x7f09fc0a50a0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:01.496 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.487+0000 7f0a19e61640 1 --2- 172.21.15.82:0/1830346232 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0a14103cf0 0x7f0a141997b0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:01.496 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.487+0000 7f0a19e61640 1 --2- 172.21.15.82:0/1830346232 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0a141089f0 0x7f0a141a0740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:01.496 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.487+0000 7f0a19e61640 1 --2- 172.21.15.82:0/1830346232 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0a14102ab0 0x7f0a14199270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:01.496 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.487+0000 7f0a19e61640 1 -- 172.21.15.82:0/1830346232 >> 172.21.15.82:0/1830346232 conn(0x7f0a140fdca0 msgr2=0x7f0a141049d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:14:01.496 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.488+0000 7f0a19e61640 1 -- 172.21.15.82:0/1830346232 shutdown_connections 2024-05-20T16:14:01.497 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.488+0000 7f0a19e61640 1 -- 172.21.15.82:0/1830346232 wait complete. 2024-05-20T16:14:01.499 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-05-20T16:14:01.612 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.612+0000 7f5bfab72640 1 -- 172.21.15.82:0/274475585 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5bf4105bc0 msgr2=0x7f5bf41080a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:14:01.612 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.612+0000 7f5bfab72640 1 --2- 172.21.15.82:0/274475585 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5bf4105bc0 0x7f5bf41080a0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f5be8009900 tx=0x7f5be802a410 comp rx=0 tx=0).stop 2024-05-20T16:14:01.612 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.612+0000 7f5bfab72640 1 -- 172.21.15.82:0/274475585 shutdown_connections 2024-05-20T16:14:01.612 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.612+0000 7f5bfab72640 1 --2- 172.21.15.82:0/274475585 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5bf4105bc0 0x7f5bf41080a0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:01.612 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.612+0000 7f5bfab72640 1 --2- 172.21.15.82:0/274475585 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5bf4108730 0x7f5bf410ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:01.612 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.612+0000 7f5bfab72640 1 --2- 172.21.15.82:0/274475585 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5bf4100de0 0x7f5bf41011f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:01.612 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.612+0000 7f5bfab72640 1 -- 172.21.15.82:0/274475585 >> 172.21.15.82:0/274475585 conn(0x7f5bf40fc2f0 msgr2=0x7f5bf40fe7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:14:01.612 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.612+0000 7f5bfab72640 1 -- 172.21.15.82:0/274475585 shutdown_connections 2024-05-20T16:14:01.612 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.612+0000 7f5bfab72640 1 -- 172.21.15.82:0/274475585 wait complete. 2024-05-20T16:14:01.614 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.612+0000 7f5bfab72640 1 Processor -- start 2024-05-20T16:14:01.614 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.612+0000 7f5bfab72640 1 -- start start 2024-05-20T16:14:01.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.613+0000 7f5bfab72640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5bf4100de0 0x7f5bf4199280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:14:01.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.613+0000 7f5bfab72640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5bf4105bc0 0x7f5bf41997c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:14:01.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.613+0000 7f5bfab72640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5bf4108730 0x7f5bf41a0750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:14:01.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.613+0000 7f5bfab72640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f5bf410e710 con 0x7f5bf4105bc0 2024-05-20T16:14:01.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.613+0000 7f5bfab72640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f5bf410e590 con 0x7f5bf4108730 2024-05-20T16:14:01.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.613+0000 7f5bfab72640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f5bf410e890 con 0x7f5bf4100de0 2024-05-20T16:14:01.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.613+0000 7f5bf3fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5bf4105bc0 0x7f5bf41997c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:14:01.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.613+0000 7f5bf3fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5bf4105bc0 0x7f5bf41997c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58696/0 (socket says 172.21.15.82:58696) 2024-05-20T16:14:01.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.613+0000 7f5bf3fff640 1 -- 172.21.15.82:0/178219469 learned_addr learned my addr 172.21.15.82:0/178219469 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:14:01.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.613+0000 7f5bf890e640 1 --2- 172.21.15.82:0/178219469 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5bf4100de0 0x7f5bf4199280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:14:01.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.613+0000 7f5bf910f640 1 --2- 172.21.15.82:0/178219469 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5bf4108730 0x7f5bf41a0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:14:01.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.613+0000 7f5bf3fff640 1 -- 172.21.15.82:0/178219469 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5bf4100de0 msgr2=0x7f5bf4199280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:14:01.616 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.613+0000 7f5bf3fff640 1 --2- 172.21.15.82:0/178219469 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5bf4100de0 0x7f5bf4199280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:01.616 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.613+0000 7f5bf3fff640 1 -- 172.21.15.82:0/178219469 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5bf4108730 msgr2=0x7f5bf41a0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:14:01.616 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.613+0000 7f5bf3fff640 1 --2- 172.21.15.82:0/178219469 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5bf4108730 0x7f5bf41a0750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:01.616 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.613+0000 7f5bf3fff640 1 -- 172.21.15.82:0/178219469 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bf41a0dc0 con 0x7f5bf4105bc0 2024-05-20T16:14:01.616 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.613+0000 7f5bf3fff640 1 --2- 172.21.15.82:0/178219469 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5bf4105bc0 0x7f5bf41997c0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f5be8006010 tx=0x7f5be8004360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:14:01.616 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.613+0000 7f5bf1ffb640 1 -- 172.21.15.82:0/178219469 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5be803f030 con 0x7f5bf4105bc0 2024-05-20T16:14:01.616 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.613+0000 7f5bf1ffb640 1 -- 172.21.15.82:0/178219469 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5be8003ce0 con 0x7f5bf4105bc0 2024-05-20T16:14:01.616 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.613+0000 7f5bfab72640 1 -- 172.21.15.82:0/178219469 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5bf41a1050 con 0x7f5bf4105bc0 2024-05-20T16:14:01.616 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.614+0000 7f5bf1ffb640 1 -- 172.21.15.82:0/178219469 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5be8035070 con 0x7f5bf4105bc0 2024-05-20T16:14:01.616 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.614+0000 7f5bfab72640 1 -- 172.21.15.82:0/178219469 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5bf41a1540 con 0x7f5bf4105bc0 2024-05-20T16:14:01.619 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.617+0000 7f5c01a16740 1 Processor -- start 2024-05-20T16:14:01.619 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.617+0000 7f5c01a16740 1 -- start start 2024-05-20T16:14:01.619 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.617+0000 7f5bf1ffb640 1 -- 172.21.15.82:0/178219469 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f5be8042030 con 0x7f5bf4105bc0 2024-05-20T16:14:01.619 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.617+0000 7f5c01a16740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560948ad9270 0x560948e84390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:14:01.620 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.617+0000 7f5c01a16740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560948ab03b0 0x560948e868e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:14:01.620 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.617+0000 7f5c01a16740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560948ab8590 0x560948e89020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:14:01.620 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.617+0000 7f5c01a16740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x560948ddd4d0 con 0x560948ad9270 2024-05-20T16:14:01.620 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.617+0000 7f5c01a16740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x560948d853e0 con 0x560948ab03b0 2024-05-20T16:14:01.620 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.617+0000 7f5c01a16740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x560948dd9490 con 0x560948ab8590 2024-05-20T16:14:01.620 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.617+0000 7f5bf910f640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560948ad9270 0x560948e84390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:14:01.620 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.617+0000 7f5bf910f640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560948ad9270 0x560948e84390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58710/0 (socket says 172.21.15.82:58710) 2024-05-20T16:14:01.620 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.617+0000 7f5bf910f640 1 -- 172.21.15.82:0/4071926812 learned_addr learned my addr 172.21.15.82:0/4071926812 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:14:01.620 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.618+0000 7f5bf3fff640 1 --2- 172.21.15.82:0/4071926812 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560948ab8590 0x560948e89020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:14:01.620 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.618+0000 7f5bf890e640 1 --2- 172.21.15.82:0/4071926812 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560948ab03b0 0x560948e868e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:14:01.620 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.618+0000 7f5bf910f640 1 -- 172.21.15.82:0/4071926812 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560948ab8590 msgr2=0x560948e89020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:14:01.620 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.618+0000 7f5bf910f640 1 --2- 172.21.15.82:0/4071926812 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560948ab8590 0x560948e89020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:01.620 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.618+0000 7f5bf910f640 1 -- 172.21.15.82:0/4071926812 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560948ab03b0 msgr2=0x560948e868e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:14:01.620 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.618+0000 7f5bf910f640 1 --2- 172.21.15.82:0/4071926812 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560948ab03b0 0x560948e868e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:01.621 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.618+0000 7f5bf910f640 1 -- 172.21.15.82:0/4071926812 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560948af12d0 con 0x560948ad9270 2024-05-20T16:14:01.621 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.618+0000 7f5bf1ffb640 1 --2- 172.21.15.82:0/178219469 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f5bdc0a2b90 0x7f5bdc0a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:14:01.621 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.618+0000 7f5bf910f640 1 --2- 172.21.15.82:0/4071926812 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560948ad9270 0x560948e84390 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f5be400ab80 tx=0x7f5be400f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ad00b77760236ab server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:14:01.621 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.618+0000 7f5bf890e640 1 --2- 172.21.15.82:0/178219469 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f5bdc0a2b90 0x7f5bdc0a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:14:01.621 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.618+0000 7f5bf1ffb640 1 -- 172.21.15.82:0/178219469 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9150+0+0 (secure 0 0 0) 0x7f5be80efd20 con 0x7f5bf4105bc0 2024-05-20T16:14:01.621 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.618+0000 7f5bdaffd640 1 -- 172.21.15.82:0/4071926812 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5be401d070 con 0x560948ad9270 2024-05-20T16:14:01.621 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.618+0000 7f5bdaffd640 1 -- 172.21.15.82:0/4071926812 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5be4018b20 con 0x560948ad9270 2024-05-20T16:14:01.621 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.618+0000 7f5bdaffd640 1 -- 172.21.15.82:0/4071926812 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5be4010ad0 con 0x560948ad9270 2024-05-20T16:14:01.621 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.619+0000 7f5c01a16740 1 -- 172.21.15.82:0/4071926812 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560948ad9270 msgr2=0x560948e84390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:14:01.621 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.619+0000 7f5bf890e640 1 --2- 172.21.15.82:0/178219469 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f5bdc0a2b90 0x7f5bdc0a50a0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x560948e787a0 tx=0x7f5bec009680 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:14:01.621 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.619+0000 7f5c01a16740 1 --2- 172.21.15.82:0/4071926812 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560948ad9270 0x560948e84390 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f5be400ab80 tx=0x7f5be400f790 comp rx=0 tx=0).stop 2024-05-20T16:14:01.621 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.619+0000 7f5c01a16740 1 -- 172.21.15.82:0/4071926812 shutdown_connections 2024-05-20T16:14:01.621 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.619+0000 7f5c01a16740 1 --2- 172.21.15.82:0/4071926812 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560948ab8590 0x560948e89020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:01.622 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.619+0000 7f5c01a16740 1 --2- 172.21.15.82:0/4071926812 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560948ab03b0 0x560948e868e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:01.622 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.619+0000 7f5c01a16740 1 --2- 172.21.15.82:0/4071926812 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560948ad9270 0x560948e84390 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:01.622 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.619+0000 7f5c01a16740 1 -- 172.21.15.82:0/4071926812 >> 172.21.15.82:0/4071926812 conn(0x560948acd3c0 msgr2=0x560948e74d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:14:01.622 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.619+0000 7f5c01a16740 1 -- 172.21.15.82:0/4071926812 shutdown_connections 2024-05-20T16:14:01.622 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.619+0000 7f5c01a16740 1 -- 172.21.15.82:0/4071926812 wait complete. 2024-05-20T16:14:01.622 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.620+0000 7f5c01a16740 1 -- start start 2024-05-20T16:14:01.622 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.620+0000 7f5c01a16740 1 Processor -- start 2024-05-20T16:14:01.626 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.620+0000 7f5c01a16740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560948ab03b0 0x560948e8bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:14:01.626 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.620+0000 7f5c01a16740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560948ab8590 0x560948e8c110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:14:01.626 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.620+0000 7f5c01a16740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560948ad9270 0x560948e93070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:14:01.626 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.620+0000 7f5c01a16740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x560948e89b40 con 0x560948ab8590 2024-05-20T16:14:01.626 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.620+0000 7f5c01a16740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x560948d988e0 con 0x560948ab03b0 2024-05-20T16:14:01.626 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.620+0000 7f5c01a16740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x560948e89cc0 con 0x560948ad9270 2024-05-20T16:14:01.626 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.620+0000 7f5c01a16740 10 client.0 _ll_register_callbacks cb 0x560948d659c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T16:14:01.626 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.620+0000 7f5bf3fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560948ab8590 0x560948e8c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:14:01.626 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.620+0000 7f5bf3fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560948ab8590 0x560948e8c110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58714/0 (socket says 172.21.15.82:58714) 2024-05-20T16:14:01.626 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.620+0000 7f5bf3fff640 1 -- 172.21.15.82:0/3058437521 learned_addr learned my addr 172.21.15.82:0/3058437521 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:14:01.627 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.620+0000 7f5bf910f640 1 --2- 172.21.15.82:0/3058437521 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560948ad9270 0x560948e93070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:14:01.627 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.620+0000 7f5bf890e640 1 --2- 172.21.15.82:0/3058437521 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560948ab03b0 0x560948e8bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:14:01.627 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.620+0000 7f5bf3fff640 1 -- 172.21.15.82:0/3058437521 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560948ad9270 msgr2=0x560948e93070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:14:01.627 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.620+0000 7f5bf3fff640 1 --2- 172.21.15.82:0/3058437521 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560948ad9270 0x560948e93070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:01.627 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.620+0000 7f5bf3fff640 1 -- 172.21.15.82:0/3058437521 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560948ab03b0 msgr2=0x560948e8bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:14:01.627 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.620+0000 7f5bf3fff640 1 --2- 172.21.15.82:0/3058437521 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560948ab03b0 0x560948e8bbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:01.627 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.620+0000 7f5bf3fff640 1 -- 172.21.15.82:0/3058437521 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560948bbb6b0 con 0x560948ab8590 2024-05-20T16:14:01.627 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.621+0000 7f5bf890e640 1 --2- 172.21.15.82:0/3058437521 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560948ab03b0 0x560948e8bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T16:14:01.627 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.621+0000 7f5bf3fff640 1 --2- 172.21.15.82:0/3058437521 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560948ab8590 0x560948e8c110 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f5be80bc230 tx=0x7f5be8004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=430eae02e54adf9a server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:14:01.627 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.621+0000 7f5bd3fff640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-05-20T16:14:01.627 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.621+0000 7f5bd3fff640 1 -- 172.21.15.82:0/3058437521 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5be8049070 con 0x560948ab8590 2024-05-20T16:14:01.627 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.621+0000 7f5c01a16740 1 -- 172.21.15.82:0/3058437521 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5be4018b20 con 0x560948ab8590 2024-05-20T16:14:01.627 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.621+0000 7f5bd3fff640 1 -- 172.21.15.82:0/3058437521 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9150+0+0 (secure 0 0 0) 0x7f5be80ba5a0 con 0x560948ab8590 2024-05-20T16:14:01.627 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.621+0000 7f5bd3fff640 1 -- 172.21.15.82:0/3058437521 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f5be80b8540 con 0x560948ab8590 2024-05-20T16:14:01.628 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.621+0000 7f5c01a16740 10 client.15435 fetch_fsmap learned FSMap version 24 2024-05-20T16:14:01.628 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.621+0000 7f5c01a16740 1 -- 172.21.15.82:0/3058437521 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x560948e95fa0 con 0x560948ab8590 2024-05-20T16:14:01.628 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.621+0000 7f5bd3fff640 1 -- 172.21.15.82:0/3058437521 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5be4018b20 con 0x560948ab8590 2024-05-20T16:14:01.628 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.622+0000 7f5bd3fff640 1 -- 172.21.15.82:0/3058437521 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f5be803b460 con 0x560948ab8590 2024-05-20T16:14:01.628 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.622+0000 7f5c01a16740 10 client.15435 fetch_fsmap finished waiting for FSMap version 24 2024-05-20T16:14:01.628 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.622+0000 7f5c01a16740 10 client.15435 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-20T16:14:01.628 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.622+0000 7f5c01a16740 20 client.15435 populate_metadata read hostname 'smithi082' 2024-05-20T16:14:01.628 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.622+0000 7f5c01a16740 1 --2- 172.21.15.82:0/3058437521 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x560948a97e40 0x560948e99b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:14:01.628 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.622+0000 7f5c01a16740 4 client.15435 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T16:14:01.628 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.622+0000 7f5c01a16740 1 -- 172.21.15.82:0/3058437521 --> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560948deac00 con 0x560948a97e40 2024-05-20T16:14:01.628 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.622+0000 7f5bf890e640 1 --2- 172.21.15.82:0/3058437521 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x560948a97e40 0x560948e99b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:14:01.628 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.623+0000 7f5bd3fff640 10 client.15435 ms_handle_connect on v2:172.21.15.160:6838/2765554482 2024-05-20T16:14:01.628 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.623+0000 7f5bf890e640 1 --2- 172.21.15.82:0/3058437521 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x560948a97e40 0x560948e99b20 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7714f6a22647bed server_cookie=1e4a499ac523f887 in_seq=0 out_seq=0 2024-05-20T16:14:01.628 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.624+0000 7f5bd3fff640 1 -- 172.21.15.82:0/3058437521 <== mds.0 v2:172.21.15.160:6838/2765554482 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x560948deac00 con 0x560948a97e40 2024-05-20T16:14:01.629 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.624+0000 7f5bd3fff640 10 client.15435 handle_command_reply: tid=0 2024-05-20T16:14:01.629 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.624+0000 7f5c01a16740 1 client.15435 shutdown 2024-05-20T16:14:01.629 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.624+0000 7f5c01a16740 1 -- 172.21.15.82:0/3058437521 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560948ab8590 msgr2=0x560948e8c110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:14:01.629 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.624+0000 7f5c01a16740 1 --2- 172.21.15.82:0/3058437521 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560948ab8590 0x560948e8c110 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f5be80bc230 tx=0x7f5be8004610 comp rx=0 tx=0).stop 2024-05-20T16:14:01.629 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.624+0000 7f5c01a16740 1 -- 172.21.15.82:0/3058437521 shutdown_connections 2024-05-20T16:14:01.629 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.624+0000 7f5c01a16740 1 --2- 172.21.15.82:0/3058437521 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x560948a97e40 0x560948e99b20 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:01.629 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.624+0000 7f5c01a16740 1 --2- 172.21.15.82:0/3058437521 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560948ab8590 0x560948e8c110 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:01.629 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.624+0000 7f5bd3fff640 0 client.15435 ms_handle_reset on v2:172.21.15.160:6838/2765554482 2024-05-20T16:14:01.629 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.624+0000 7f5c01a16740 1 --2- 172.21.15.82:0/3058437521 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560948ad9270 0x560948e93070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:01.629 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.624+0000 7f5c01a16740 1 --2- 172.21.15.82:0/3058437521 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560948ab03b0 0x560948e8bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:01.629 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.624+0000 7f5c01a16740 1 -- 172.21.15.82:0/3058437521 >> 172.21.15.82:0/3058437521 conn(0x560948acd3c0 msgr2=0x560948e73ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:14:01.629 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.624+0000 7f5c01a16740 1 -- 172.21.15.82:0/3058437521 shutdown_connections 2024-05-20T16:14:01.629 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.624+0000 7f5c01a16740 1 -- 172.21.15.82:0/3058437521 wait complete. 2024-05-20T16:14:01.629 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.624+0000 7f5c01a16740 20 client.15435 trim_cache size 0 max 16384 2024-05-20T16:14:01.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.642+0000 7f5c01a16740 1 Processor -- start 2024-05-20T16:14:01.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.642+0000 7f5c01a16740 1 -- start start 2024-05-20T16:14:01.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.643+0000 7f5c01a16740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560948a97e40 0x560948e8c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:14:01.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.643+0000 7f5c01a16740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560948ab03b0 0x560948ea1e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:14:01.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.643+0000 7f5c01a16740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560948ab8590 0x560948ea2340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:14:01.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.643+0000 7f5c01a16740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x560948e97e60 con 0x560948ab03b0 2024-05-20T16:14:01.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.643+0000 7f5c01a16740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x560948e98530 con 0x560948ab8590 2024-05-20T16:14:01.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.643+0000 7f5c01a16740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x560948e98c00 con 0x560948a97e40 2024-05-20T16:14:01.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.643+0000 7f5bf3fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560948ab03b0 0x560948ea1e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:14:01.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.643+0000 7f5bf3fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560948ab03b0 0x560948ea1e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58730/0 (socket says 172.21.15.82:58730) 2024-05-20T16:14:01.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.643+0000 7f5bf3fff640 1 -- 172.21.15.82:0/288844864 learned_addr learned my addr 172.21.15.82:0/288844864 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:14:01.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.643+0000 7f5bf3fff640 1 -- 172.21.15.82:0/288844864 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560948a97e40 msgr2=0x560948e8c1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:14:01.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.643+0000 7f5bf3fff640 1 --2- 172.21.15.82:0/288844864 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560948a97e40 0x560948e8c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:01.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.643+0000 7f5bf3fff640 1 -- 172.21.15.82:0/288844864 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560948ab8590 msgr2=0x560948ea2340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:14:01.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.643+0000 7f5bf3fff640 1 --2- 172.21.15.82:0/288844864 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560948ab8590 0x560948ea2340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:01.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.643+0000 7f5bf3fff640 1 -- 172.21.15.82:0/288844864 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560948e785c0 con 0x560948ab03b0 2024-05-20T16:14:01.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.643+0000 7f5bf3fff640 1 --2- 172.21.15.82:0/288844864 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560948ab03b0 0x560948ea1e00 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f5be8005940 tx=0x7f5be80f00b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b731fc2d8880f6c1 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:14:01.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.643+0000 7f5bda7fc640 1 -- 172.21.15.82:0/288844864 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5be8049070 con 0x560948ab03b0 2024-05-20T16:14:01.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.643+0000 7f5bda7fc640 1 -- 172.21.15.82:0/288844864 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5be80c4e60 con 0x560948ab03b0 2024-05-20T16:14:01.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.643+0000 7f5bda7fc640 1 -- 172.21.15.82:0/288844864 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5be80bbc00 con 0x560948ab03b0 2024-05-20T16:14:01.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.644+0000 7f5c01a16740 1 -- 172.21.15.82:0/288844864 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560948ab03b0 msgr2=0x560948ea1e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:14:01.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.644+0000 7f5c01a16740 1 --2- 172.21.15.82:0/288844864 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560948ab03b0 0x560948ea1e00 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f5be8005940 tx=0x7f5be80f00b0 comp rx=0 tx=0).stop 2024-05-20T16:14:01.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.644+0000 7f5c01a16740 1 -- 172.21.15.82:0/288844864 shutdown_connections 2024-05-20T16:14:01.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.644+0000 7f5c01a16740 1 --2- 172.21.15.82:0/288844864 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560948ab03b0 0x560948ea1e00 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:01.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.644+0000 7f5c01a16740 1 --2- 172.21.15.82:0/288844864 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560948ab8590 0x560948ea2340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:01.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.644+0000 7f5c01a16740 1 --2- 172.21.15.82:0/288844864 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560948a97e40 0x560948e8c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:01.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.644+0000 7f5c01a16740 1 -- 172.21.15.82:0/288844864 >> 172.21.15.82:0/288844864 conn(0x560948acd3c0 msgr2=0x560948e81370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:14:01.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.644+0000 7f5c01a16740 1 -- 172.21.15.82:0/288844864 shutdown_connections 2024-05-20T16:14:01.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.644+0000 7f5c01a16740 1 -- 172.21.15.82:0/288844864 wait complete. 2024-05-20T16:14:01.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.644+0000 7f5c01a16740 1 -- start start 2024-05-20T16:14:01.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.644+0000 7f5c01a16740 1 Processor -- start 2024-05-20T16:14:01.650 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.645+0000 7f5c01a16740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560948a97e40 0x560948e91690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:14:01.650 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.645+0000 7f5c01a16740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560948ab03b0 0x560948e91bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:14:01.650 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.645+0000 7f5c01a16740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560948ab8590 0x560948e92ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:14:01.650 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.645+0000 7f5c01a16740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x560948e999a0 con 0x560948ab03b0 2024-05-20T16:14:01.650 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.645+0000 7f5c01a16740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x560948e992d0 con 0x560948ab8590 2024-05-20T16:14:01.650 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.645+0000 7f5c01a16740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x560948e9a070 con 0x560948a97e40 2024-05-20T16:14:01.650 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.645+0000 7f5c01a16740 10 client.0 _ll_register_callbacks cb 0x7f5be80f34f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T16:14:01.650 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.645+0000 7f5bf3fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560948ab03b0 0x560948e91bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:14:01.650 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.645+0000 7f5bf3fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560948ab03b0 0x560948e91bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58740/0 (socket says 172.21.15.82:58740) 2024-05-20T16:14:01.650 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.645+0000 7f5bf3fff640 1 -- 172.21.15.82:0/1608624054 learned_addr learned my addr 172.21.15.82:0/1608624054 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:14:01.650 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.645+0000 7f5bf890e640 1 --2- 172.21.15.82:0/1608624054 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560948a97e40 0x560948e91690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:14:01.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.645+0000 7f5bf3fff640 1 -- 172.21.15.82:0/1608624054 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560948a97e40 msgr2=0x560948e91690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:14:01.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.645+0000 7f5bf3fff640 1 --2- 172.21.15.82:0/1608624054 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560948a97e40 0x560948e91690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:01.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.645+0000 7f5bf910f640 1 --2- 172.21.15.82:0/1608624054 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560948ab8590 0x560948e92ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:14:01.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.645+0000 7f5bf3fff640 1 -- 172.21.15.82:0/1608624054 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560948ab8590 msgr2=0x560948e92ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:14:01.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.645+0000 7f5bf3fff640 1 --2- 172.21.15.82:0/1608624054 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560948ab8590 0x560948e92ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:01.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.645+0000 7f5bf3fff640 1 -- 172.21.15.82:0/1608624054 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560948e93070 con 0x560948ab03b0 2024-05-20T16:14:01.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.645+0000 7f5bf890e640 1 --2- 172.21.15.82:0/1608624054 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560948a97e40 0x560948e91690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T16:14:01.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.645+0000 7f5bf910f640 1 --2- 172.21.15.82:0/1608624054 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560948ab8590 0x560948e92ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T16:14:01.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.645+0000 7f5bf3fff640 1 --2- 172.21.15.82:0/1608624054 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560948ab03b0 0x560948e91bd0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f5be8000c00 tx=0x7f5be802a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b14ec4b28f864c3 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:14:01.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.645+0000 7f5bd3fff640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-05-20T16:14:01.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.646+0000 7f5bd3fff640 1 -- 172.21.15.82:0/1608624054 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5be8049040 con 0x560948ab03b0 2024-05-20T16:14:01.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.646+0000 7f5bd3fff640 1 -- 172.21.15.82:0/1608624054 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9150+0+0 (secure 0 0 0) 0x7f5be80bc500 con 0x560948ab03b0 2024-05-20T16:14:01.652 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.646+0000 7f5c01a16740 1 -- 172.21.15.82:0/1608624054 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5be80c4e60 con 0x560948ab03b0 2024-05-20T16:14:01.652 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.646+0000 7f5bd3fff640 1 -- 172.21.15.82:0/1608624054 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f5be80a5050 con 0x560948ab03b0 2024-05-20T16:14:01.652 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.646+0000 7f5c01a16740 10 client.15441 fetch_fsmap learned FSMap version 24 2024-05-20T16:14:01.652 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.646+0000 7f5c01a16740 1 -- 172.21.15.82:0/1608624054 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x560948e93070 con 0x560948ab03b0 2024-05-20T16:14:01.652 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.646+0000 7f5bd3fff640 1 -- 172.21.15.82:0/1608624054 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5be80c4e60 con 0x560948ab03b0 2024-05-20T16:14:01.652 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.646+0000 7f5bd3fff640 1 -- 172.21.15.82:0/1608624054 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f5be80c00f0 con 0x560948ab03b0 2024-05-20T16:14:01.652 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.646+0000 7f5c01a16740 10 client.15441 fetch_fsmap finished waiting for FSMap version 24 2024-05-20T16:14:01.652 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.646+0000 7f5c01a16740 10 client.15441 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-20T16:14:01.652 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.646+0000 7f5c01a16740 20 client.15441 populate_metadata read hostname 'smithi082' 2024-05-20T16:14:01.652 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.646+0000 7f5c01a16740 1 --2- 172.21.15.82:0/1608624054 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x560948ad9270 0x560948f06210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:14:01.652 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.646+0000 7f5c01a16740 4 client.15441 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-20T16:14:01.653 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.646+0000 7f5c01a16740 1 -- 172.21.15.82:0/1608624054 --> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] -- command(tid 0: {"prefix": "status"}) -- 0x560948de4f50 con 0x560948ad9270 2024-05-20T16:14:01.653 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.647+0000 7f5bf890e640 1 --2- 172.21.15.82:0/1608624054 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x560948ad9270 0x560948f06210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:14:01.653 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.648+0000 7f5bd3fff640 10 client.15441 ms_handle_connect on v2:172.21.15.160:6838/2765554482 2024-05-20T16:14:01.653 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.648+0000 7f5bf890e640 1 --2- 172.21.15.82:0/1608624054 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x560948ad9270 0x560948f06210 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=48a90bcf01e9a7f7 server_cookie=9b788d1ba110891d in_seq=0 out_seq=0 2024-05-20T16:14:01.653 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.648+0000 7f5bd3fff640 1 -- 172.21.15.82:0/1608624054 <== mds.0 v2:172.21.15.160:6838/2765554482 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x560948de4f50 con 0x560948ad9270 2024-05-20T16:14:01.653 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.648+0000 7f5bd3fff640 10 client.15441 handle_command_reply: tid=0 2024-05-20T16:14:01.653 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.648+0000 7f5c01a16740 1 client.15441 shutdown 2024-05-20T16:14:01.653 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.648+0000 7f5c01a16740 1 -- 172.21.15.82:0/1608624054 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560948ab03b0 msgr2=0x560948e91bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:14:01.653 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.648+0000 7f5c01a16740 1 --2- 172.21.15.82:0/1608624054 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560948ab03b0 0x560948e91bd0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f5be8000c00 tx=0x7f5be802a410 comp rx=0 tx=0).stop 2024-05-20T16:14:01.653 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.648+0000 7f5c01a16740 1 -- 172.21.15.82:0/1608624054 shutdown_connections 2024-05-20T16:14:01.653 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.648+0000 7f5c01a16740 1 --2- 172.21.15.82:0/1608624054 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x560948ad9270 0x560948f06210 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:01.653 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.648+0000 7f5c01a16740 1 --2- 172.21.15.82:0/1608624054 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560948ab03b0 0x560948e91bd0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:01.653 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.648+0000 7f5c01a16740 1 --2- 172.21.15.82:0/1608624054 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560948ab8590 0x560948e92ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:01.653 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.648+0000 7f5bd3fff640 0 client.15441 ms_handle_reset on v2:172.21.15.160:6838/2765554482 2024-05-20T16:14:01.653 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.648+0000 7f5c01a16740 1 --2- 172.21.15.82:0/1608624054 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560948a97e40 0x560948e91690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:01.654 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.648+0000 7f5c01a16740 1 -- 172.21.15.82:0/1608624054 >> 172.21.15.82:0/1608624054 conn(0x560948acd3c0 msgr2=0x560948e8bf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:14:01.654 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.648+0000 7f5c01a16740 1 -- 172.21.15.82:0/1608624054 shutdown_connections 2024-05-20T16:14:01.654 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.648+0000 7f5c01a16740 1 -- 172.21.15.82:0/1608624054 wait complete. 2024-05-20T16:14:01.654 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.649+0000 7f5c01a16740 20 client.15441 trim_cache size 0 max 16384 2024-05-20T16:14:01.654 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-05-20T16:14:01.654 INFO:teuthology.orchestra.run.smithi082.stdout: "cluster_fsid": "59d1600a-16c2-11ef-bc9a-c7b262605968", 2024-05-20T16:14:01.654 INFO:teuthology.orchestra.run.smithi082.stdout: "whoami": 0, 2024-05-20T16:14:01.654 INFO:teuthology.orchestra.run.smithi082.stdout: "id": 24548, 2024-05-20T16:14:01.654 INFO:teuthology.orchestra.run.smithi082.stdout: "want_state": "up:active", 2024-05-20T16:14:01.654 INFO:teuthology.orchestra.run.smithi082.stdout: "state": "up:active", 2024-05-20T16:14:01.655 INFO:teuthology.orchestra.run.smithi082.stdout: "fs_name": "cephfs", 2024-05-20T16:14:01.655 INFO:teuthology.orchestra.run.smithi082.stdout: "rank_uptime": 134.26721726900001, 2024-05-20T16:14:01.655 INFO:teuthology.orchestra.run.smithi082.stdout: "mdsmap_epoch": 24, 2024-05-20T16:14:01.655 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_epoch": 95, 2024-05-20T16:14:01.655 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_epoch_barrier": 91, 2024-05-20T16:14:01.655 INFO:teuthology.orchestra.run.smithi082.stdout: "uptime": 138.39963588399999 2024-05-20T16:14:01.655 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-05-20T16:14:01.656 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.649+0000 7f5bfab72640 1 -- 172.21.15.82:0/178219469 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f5bdc0a2b90 msgr2=0x7f5bdc0a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:14:01.656 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.649+0000 7f5bfab72640 1 --2- 172.21.15.82:0/178219469 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f5bdc0a2b90 0x7f5bdc0a50a0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x560948e787a0 tx=0x7f5bec009680 comp rx=0 tx=0).stop 2024-05-20T16:14:01.656 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.649+0000 7f5bfab72640 1 -- 172.21.15.82:0/178219469 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5bf4105bc0 msgr2=0x7f5bf41997c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:14:01.656 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.649+0000 7f5bfab72640 1 --2- 172.21.15.82:0/178219469 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5bf4105bc0 0x7f5bf41997c0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f5be8006010 tx=0x7f5be8004360 comp rx=0 tx=0).stop 2024-05-20T16:14:01.656 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.649+0000 7f5bfab72640 1 -- 172.21.15.82:0/178219469 shutdown_connections 2024-05-20T16:14:01.656 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.649+0000 7f5bfab72640 1 --2- 172.21.15.82:0/178219469 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f5bdc0a2b90 0x7f5bdc0a50a0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:01.656 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.649+0000 7f5bfab72640 1 --2- 172.21.15.82:0/178219469 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5bf4105bc0 0x7f5bf41997c0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:01.656 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.649+0000 7f5bfab72640 1 --2- 172.21.15.82:0/178219469 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5bf4108730 0x7f5bf41a0750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:01.656 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.649+0000 7f5bfab72640 1 --2- 172.21.15.82:0/178219469 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5bf4100de0 0x7f5bf4199280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:01.656 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.649+0000 7f5bfab72640 1 -- 172.21.15.82:0/178219469 >> 172.21.15.82:0/178219469 conn(0x7f5bf40fc2f0 msgr2=0x7f5bf41092e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:14:01.656 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.649+0000 7f5bfab72640 1 -- 172.21.15.82:0/178219469 shutdown_connections 2024-05-20T16:14:01.656 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:01.649+0000 7f5bfab72640 1 -- 172.21.15.82:0/178219469 wait complete. 2024-05-20T16:14:01.660 INFO:teuthology.run_tasks:Running task sequential... 2024-05-20T16:14:01.672 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-05-20T16:14:01.672 INFO:teuthology.task.sequential:In sequential, running task print... 2024-05-20T16:14:01.672 INFO:teuthology.task.print:Enabling mgr modules 2024-05-20T16:14:01.672 INFO:teuthology.run_tasks:Running task kclient... 2024-05-20T16:14:01.685 INFO:tasks.kclient:Mounting kernel clients... 2024-05-20T16:14:01.685 INFO:tasks.kclient:config is {'client.0': {}} 2024-05-20T16:14:01.686 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T16:14:01.686 DEBUG:teuthology.orchestra.run.smithi082:> ip netns list 2024-05-20T16:14:01.702 INFO:teuthology.orchestra.run.smithi082.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-05-20T16:14:01.702 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T16:14:01.702 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-20T16:14:01.768 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T16:14:01.768 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip link delete ceph-brx 2024-05-20T16:14:01.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:14:01 smithi160 ceph-mon[36347]: from='client.? 172.21.15.82:0/829440101' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-20T16:14:01.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:14:01 smithi160 ceph-mon[36347]: from='client.? 172.21.15.82:0/1801532231' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T16:14:01.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:14:01 smithi160 ceph-mon[36347]: from='client.? 172.21.15.82:0/224258168' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-20T16:14:01.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:14:01 smithi160 ceph-mon[36347]: pgmap v363: 129 pgs: 129 active+clean; 641 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s; 9 B/s, 0 objects/s recovering 2024-05-20T16:14:01.859 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=crc', 'wsync'], 'syntax': 'v2', 'mount_subvol_num': 1} 2024-05-20T16:14:01.859 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-20T16:14:01.859 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-20T16:14:01.859 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell --fsid 59d1600a-16c2-11ef-bc9a-c7b262605968 -- ceph fs ls 2024-05-20T16:14:01.885 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:14:01 smithi184 ceph-mon[36209]: from='client.? 172.21.15.82:0/829440101' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-20T16:14:01.885 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:14:01 smithi184 ceph-mon[36209]: from='client.? 172.21.15.82:0/1801532231' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T16:14:01.886 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:14:01 smithi184 ceph-mon[36209]: from='client.? 172.21.15.82:0/224258168' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-20T16:14:01.886 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:14:01 smithi184 ceph-mon[36209]: pgmap v363: 129 pgs: 129 active+clean; 641 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s; 9 B/s, 0 objects/s recovering 2024-05-20T16:14:02.101 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T16:14:03.521 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:14:03 smithi082 ceph-mon[32238]: pgmap v364: 129 pgs: 129 active+clean; 641 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-20T16:14:03.635 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:14:03 smithi184 ceph-mon[36209]: pgmap v364: 129 pgs: 129 active+clean; 641 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-20T16:14:03.786 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:03.786+0000 7f107a3e7700 1 -- 172.21.15.82:0/2617928064 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f10741075c0 msgr2=0x7f107410ef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:14:03.786 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:03.786+0000 7f107a3e7700 1 --2- 172.21.15.82:0/2617928064 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f10741075c0 0x7f107410ef20 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f106c00b0d0 tx=0x7f106c00b3e0 comp rx=0 tx=0).stop 2024-05-20T16:14:03.786 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:03.786+0000 7f107a3e7700 1 -- 172.21.15.82:0/2617928064 shutdown_connections 2024-05-20T16:14:03.786 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:03.786+0000 7f107a3e7700 1 --2- 172.21.15.82:0/2617928064 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f10741075c0 0x7f107410ef20 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:03.786 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:03.786+0000 7f107a3e7700 1 --2- 172.21.15.82:0/2617928064 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1074102340 0x7f1074102750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:03.786 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:03.786+0000 7f107a3e7700 1 --2- 172.21.15.82:0/2617928064 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1074106c70 0x7f1074107080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:03.786 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:03.786+0000 7f107a3e7700 1 -- 172.21.15.82:0/2617928064 >> 172.21.15.82:0/2617928064 conn(0x7f10740fdc70 msgr2=0x7f1074100090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:14:03.786 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:03.786+0000 7f107a3e7700 1 -- 172.21.15.82:0/2617928064 shutdown_connections 2024-05-20T16:14:03.787 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:03.786+0000 7f107a3e7700 1 -- 172.21.15.82:0/2617928064 wait complete. 2024-05-20T16:14:03.788 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:03.787+0000 7f107a3e7700 1 Processor -- start 2024-05-20T16:14:03.788 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:03.787+0000 7f107a3e7700 1 -- start start 2024-05-20T16:14:03.788 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:03.787+0000 7f107a3e7700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1074102340 0x7f1074198da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:14:03.788 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:03.787+0000 7f107a3e7700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1074106c70 0x7f1074199340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:14:03.788 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:03.787+0000 7f107a3e7700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f10741075c0 0x7f107419d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:14:03.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:03.787+0000 7f107a3e7700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f1074112430 con 0x7f1074102340 2024-05-20T16:14:03.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:03.787+0000 7f107a3e7700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f10741122b0 con 0x7f1074106c70 2024-05-20T16:14:03.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:03.787+0000 7f107a3e7700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f10741125b0 con 0x7f10741075c0 2024-05-20T16:14:03.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:03.787+0000 7f1073fff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1074102340 0x7f1074198da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:14:03.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:03.787+0000 7f1073fff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1074102340 0x7f1074198da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58766/0 (socket says 172.21.15.82:58766) 2024-05-20T16:14:03.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:03.787+0000 7f1073fff700 1 -- 172.21.15.82:0/3754891558 learned_addr learned my addr 172.21.15.82:0/3754891558 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:14:03.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:03.788+0000 7f1078984700 1 --2- 172.21.15.82:0/3754891558 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f10741075c0 0x7f107419d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:14:03.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:03.788+0000 7f1073fff700 1 -- 172.21.15.82:0/3754891558 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f10741075c0 msgr2=0x7f107419d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:14:03.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:03.788+0000 7f1073fff700 1 --2- 172.21.15.82:0/3754891558 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f10741075c0 0x7f107419d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:03.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:03.788+0000 7f1073fff700 1 -- 172.21.15.82:0/3754891558 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1074106c70 msgr2=0x7f1074199340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:14:03.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:03.788+0000 7f1073fff700 1 --2- 172.21.15.82:0/3754891558 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1074106c70 0x7f1074199340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:03.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:03.788+0000 7f1073fff700 1 -- 172.21.15.82:0/3754891558 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f107419dc70 con 0x7f1074102340 2024-05-20T16:14:03.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:03.788+0000 7f1073fff700 1 --2- 172.21.15.82:0/3754891558 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1074102340 0x7f1074198da0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f106400d9e0 tx=0x7f106400dda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:14:03.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:03.788+0000 7f10717fa700 1 -- 172.21.15.82:0/3754891558 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f106401b070 con 0x7f1074102340 2024-05-20T16:14:03.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:03.788+0000 7f10717fa700 1 -- 172.21.15.82:0/3754891558 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f106400ecd0 con 0x7f1074102340 2024-05-20T16:14:03.790 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:03.788+0000 7f107a3e7700 1 -- 172.21.15.82:0/3754891558 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f107419df60 con 0x7f1074102340 2024-05-20T16:14:03.790 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:03.788+0000 7f10717fa700 1 -- 172.21.15.82:0/3754891558 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1064014760 con 0x7f1074102340 2024-05-20T16:14:03.790 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:03.788+0000 7f107a3e7700 1 -- 172.21.15.82:0/3754891558 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1074074870 con 0x7f1074102340 2024-05-20T16:14:03.790 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:03.789+0000 7f107a3e7700 1 -- 172.21.15.82:0/3754891558 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10741037f0 con 0x7f1074102340 2024-05-20T16:14:03.791 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:03.791+0000 7f10717fa700 1 -- 172.21.15.82:0/3754891558 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f106401e070 con 0x7f1074102340 2024-05-20T16:14:03.792 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:03.792+0000 7f10717fa700 1 --2- 172.21.15.82:0/3754891558 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f105c0a2c20 0x7f105c0a5130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:14:03.792 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:03.792+0000 7f10737fe700 1 --2- 172.21.15.82:0/3754891558 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f105c0a2c20 0x7f105c0a5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:14:03.792 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:03.792+0000 7f10717fa700 1 -- 172.21.15.82:0/3754891558 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9150+0+0 (secure 0 0 0) 0x7f10640d0ea0 con 0x7f1074102340 2024-05-20T16:14:03.792 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:03.793+0000 7f10737fe700 1 --2- 172.21.15.82:0/3754891558 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f105c0a2c20 0x7f105c0a5130 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f1074101730 tx=0x7f1068006d20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:14:03.795 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:03.796+0000 7f10717fa700 1 -- 172.21.15.82:0/3754891558 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f106409db00 con 0x7f1074102340 2024-05-20T16:14:03.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:14:03 smithi160 ceph-mon[36347]: pgmap v364: 129 pgs: 129 active+clean; 641 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-20T16:14:04.037 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:04.037+0000 7f107a3e7700 1 -- 172.21.15.82:0/3754891558 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f1074050170 con 0x7f1074102340 2024-05-20T16:14:04.037 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:04.037+0000 7f10717fa700 1 -- 172.21.15.82:0/3754891558 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v24) ==== 53+0+88 (secure 0 0 0) 0x7f1074050170 con 0x7f1074102340 2024-05-20T16:14:04.037 INFO:teuthology.orchestra.run.smithi082.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-05-20T16:14:04.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:04.039+0000 7f107a3e7700 1 -- 172.21.15.82:0/3754891558 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f105c0a2c20 msgr2=0x7f105c0a5130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:14:04.040 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:04.040+0000 7f107a3e7700 1 --2- 172.21.15.82:0/3754891558 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f105c0a2c20 0x7f105c0a5130 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f1074101730 tx=0x7f1068006d20 comp rx=0 tx=0).stop 2024-05-20T16:14:04.040 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:04.040+0000 7f107a3e7700 1 -- 172.21.15.82:0/3754891558 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1074102340 msgr2=0x7f1074198da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:14:04.040 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:04.040+0000 7f107a3e7700 1 --2- 172.21.15.82:0/3754891558 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1074102340 0x7f1074198da0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f106400d9e0 tx=0x7f106400dda0 comp rx=0 tx=0).stop 2024-05-20T16:14:04.040 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:04.040+0000 7f107a3e7700 1 -- 172.21.15.82:0/3754891558 shutdown_connections 2024-05-20T16:14:04.040 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:04.040+0000 7f107a3e7700 1 --2- 172.21.15.82:0/3754891558 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f105c0a2c20 0x7f105c0a5130 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:04.040 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:04.040+0000 7f107a3e7700 1 --2- 172.21.15.82:0/3754891558 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f10741075c0 0x7f107419d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:04.040 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:04.040+0000 7f107a3e7700 1 --2- 172.21.15.82:0/3754891558 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1074102340 0x7f1074198da0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:04.040 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:04.040+0000 7f107a3e7700 1 --2- 172.21.15.82:0/3754891558 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1074106c70 0x7f1074199340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:04.040 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:04.040+0000 7f107a3e7700 1 -- 172.21.15.82:0/3754891558 >> 172.21.15.82:0/3754891558 conn(0x7f10740fdc70 msgr2=0x7f10741063e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:14:04.040 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:04.040+0000 7f107a3e7700 1 -- 172.21.15.82:0/3754891558 shutdown_connections 2024-05-20T16:14:04.040 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:04.040+0000 7f107a3e7700 1 -- 172.21.15.82:0/3754891558 wait complete. 2024-05-20T16:14:04.643 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:14:04 smithi082 ceph-mon[32238]: from='client.? 172.21.15.82:0/3754891558' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-20T16:14:04.672 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-20T16:14:04.672 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-20T16:14:04.672 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi082.front.sepia.ceph.com 2024-05-20T16:14:04.672 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-20T16:14:04.672 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-20T16:14:04.672 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-20T16:14:04.672 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-20T16:14:04.672 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-20T16:14:04.672 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-20T16:14:04.673 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T16:14:04.673 DEBUG:teuthology.orchestra.run.smithi082:> ip addr 2024-05-20T16:14:04.690 INFO:teuthology.orchestra.run.smithi082.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-20T16:14:04.690 INFO:teuthology.orchestra.run.smithi082.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-20T16:14:04.690 INFO:teuthology.orchestra.run.smithi082.stdout: inet 127.0.0.1/8 scope host lo 2024-05-20T16:14:04.690 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-05-20T16:14:04.690 INFO:teuthology.orchestra.run.smithi082.stdout: inet6 ::1/128 scope host 2024-05-20T16:14:04.690 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-05-20T16:14:04.691 INFO:teuthology.orchestra.run.smithi082.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-20T16:14:04.691 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 0c:c4:7a:d8:da:c6 brd ff:ff:ff:ff:ff:ff 2024-05-20T16:14:04.691 INFO:teuthology.orchestra.run.smithi082.stdout: altname enp7s0f0 2024-05-20T16:14:04.691 INFO:teuthology.orchestra.run.smithi082.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-20T16:14:04.691 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 0c:c4:7a:d8:da:c7 brd ff:ff:ff:ff:ff:ff 2024-05-20T16:14:04.691 INFO:teuthology.orchestra.run.smithi082.stdout: altname enp7s0f1 2024-05-20T16:14:04.691 INFO:teuthology.orchestra.run.smithi082.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-20T16:14:04.691 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 0c:c4:7a:8f:d3:fe brd ff:ff:ff:ff:ff:ff 2024-05-20T16:14:04.691 INFO:teuthology.orchestra.run.smithi082.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-20T16:14:04.691 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 0c:c4:7a:8f:d3:ff brd ff:ff:ff:ff:ff:ff 2024-05-20T16:14:04.691 INFO:teuthology.orchestra.run.smithi082.stdout: inet 172.21.15.82/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-05-20T16:14:04.691 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft 42078sec preferred_lft 42078sec 2024-05-20T16:14:04.691 INFO:teuthology.orchestra.run.smithi082.stdout: inet6 fe80::ec4:7aff:fe8f:d3ff/64 scope link 2024-05-20T16:14:04.692 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-05-20T16:14:04.692 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-05-20T16:14:04.692 DEBUG:teuthology.orchestra.run.smithi082:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-20T16:14:04.692 DEBUG:teuthology.orchestra.run.smithi082:> set -e 2024-05-20T16:14:04.692 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip link add name ceph-brx type bridge 2024-05-20T16:14:04.692 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip addr flush dev ceph-brx 2024-05-20T16:14:04.692 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip link set ceph-brx up 2024-05-20T16:14:04.692 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-05-20T16:14:04.692 DEBUG:teuthology.orchestra.run.smithi082:> ') 2024-05-20T16:14:04.781 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:04 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-20T16:14:04.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:14:04 smithi160 ceph-mon[36347]: from='client.? 172.21.15.82:0/3754891558' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-20T16:14:04.827 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-20T16:14:04.835 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T16:14:04.835 DEBUG:teuthology.orchestra.run.smithi082:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-20T16:14:04.859 INFO:teuthology.orchestra.run.smithi082.stdout:1 2024-05-20T16:14:04.861 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T16:14:04.861 DEBUG:teuthology.orchestra.run.smithi082:> route 2024-05-20T16:14:04.886 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:14:04 smithi184 ceph-mon[36209]: from='client.? 172.21.15.82:0/3754891558' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-20T16:14:04.916 INFO:teuthology.orchestra.run.smithi082.stdout:Kernel IP routing table 2024-05-20T16:14:04.916 INFO:teuthology.orchestra.run.smithi082.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-20T16:14:04.916 INFO:teuthology.orchestra.run.smithi082.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-20T16:14:04.916 INFO:teuthology.orchestra.run.smithi082.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-20T16:14:04.917 INFO:teuthology.orchestra.run.smithi082.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-20T16:14:04.917 DEBUG:teuthology.orchestra.run.smithi082:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-20T16:14:04.917 DEBUG:teuthology.orchestra.run.smithi082:> set -e 2024-05-20T16:14:04.917 DEBUG:teuthology.orchestra.run.smithi082:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-20T16:14:04.917 DEBUG:teuthology.orchestra.run.smithi082:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-20T16:14:04.917 DEBUG:teuthology.orchestra.run.smithi082:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-20T16:14:04.917 DEBUG:teuthology.orchestra.run.smithi082:> ') 2024-05-20T16:14:05.005 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:05 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-20T16:14:05.037 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-20T16:14:05.043 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T16:14:05.044 DEBUG:teuthology.orchestra.run.smithi082:> ip netns list 2024-05-20T16:14:05.059 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T16:14:05.059 DEBUG:teuthology.orchestra.run.smithi082:> ip netns list-id 2024-05-20T16:14:05.115 DEBUG:teuthology.orchestra.run.smithi082:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-20T16:14:05.115 DEBUG:teuthology.orchestra.run.smithi082:> set -e 2024-05-20T16:14:05.115 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-20T16:14:05.115 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-05-20T16:14:05.115 DEBUG:teuthology.orchestra.run.smithi082:> ') 2024-05-20T16:14:05.204 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:05 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-20T16:14:05.227 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-20T16:14:05.233 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-05-20T16:14:05.233 DEBUG:teuthology.orchestra.run.smithi082:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-20T16:14:05.233 DEBUG:teuthology.orchestra.run.smithi082:> set -e 2024-05-20T16:14:05.233 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-05-20T16:14:05.233 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-05-20T16:14:05.234 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-05-20T16:14:05.234 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-05-20T16:14:05.234 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-05-20T16:14:05.234 DEBUG:teuthology.orchestra.run.smithi082:> ') 2024-05-20T16:14:05.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:05 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-20T16:14:05.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-20T16:14:05.348 DEBUG:teuthology.orchestra.run.smithi082:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-20T16:14:05.348 DEBUG:teuthology.orchestra.run.smithi082:> set -e 2024-05-20T16:14:05.348 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip link set brx.0 up 2024-05-20T16:14:05.348 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip link set dev brx.0 master ceph-brx 2024-05-20T16:14:05.348 DEBUG:teuthology.orchestra.run.smithi082:> ') 2024-05-20T16:14:05.398 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:05 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-20T16:14:05.420 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-20T16:14:05.427 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-20T16:14:05.427 DEBUG:teuthology.orchestra.run.smithi082:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-20T16:14:05.442 INFO:teuthology.orchestra.run.smithi082.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-20T16:14:05.442 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-20T16:14:05.442 DEBUG:teuthology.orchestra.run.smithi082:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-20T16:14:05.498 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-05-20T16:14:05.498 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-05-20T16:14:05.498 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/volumes/qa/sv_1/02a0ab78-ccb9-4d90-ba91-3f79600eded7 2024-05-20T16:14:05.498 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T16:14:05.498 DEBUG:teuthology.orchestra.run.smithi082:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/volumes/qa/sv_1/02a0ab78-ccb9-4d90-ba91-3f79600eded7 /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=crc,wsync,nofallback 2024-05-20T16:14:05.559 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:14:05 smithi082 ceph-mon[32238]: pgmap v365: 129 pgs: 129 active+clean; 641 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-20T16:14:05.591 INFO:teuthology.orchestra.run.smithi082.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=crc,wsync,nofallback 2024-05-20T16:14:05.591 INFO:teuthology.orchestra.run.smithi082.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=crc,wsync". 2024-05-20T16:14:05.706 INFO:teuthology.orchestra.run.smithi082.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=crc,wsync,nofallback 2024-05-20T16:14:05.706 INFO:teuthology.orchestra.run.smithi082.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=crc,wsync". 2024-05-20T16:14:05.706 INFO:teuthology.orchestra.run.smithi082.stdout:mount.ceph: resolved to: "172.21.15.82:3300,172.21.15.160:3300,172.21.15.184:3300" 2024-05-20T16:14:05.707 INFO:teuthology.orchestra.run.smithi082.stdout:mount.ceph: trying mount with new device syntax: 0@59d1600a-16c2-11ef-bc9a-c7b262605968.cephfs=/volumes/qa/sv_1/02a0ab78-ccb9-4d90-ba91-3f79600eded7 2024-05-20T16:14:05.707 INFO:teuthology.orchestra.run.smithi082.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=crc,wsync,name=0,key=0,mon_addr=172.21.15.82:3300/172.21.15.160:3300/172.21.15.184:3300" will pass to kernel 2024-05-20T16:14:05.710 INFO:teuthology.orchestra.run.smithi082.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-05-20T16:14:05.710 INFO:teuthology.orchestra.run.smithi082.stdout: You just mounted a file system that supports labels which does not 2024-05-20T16:14:05.710 INFO:teuthology.orchestra.run.smithi082.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-05-20T16:14:05.710 INFO:teuthology.orchestra.run.smithi082.stdout: applications will generate AVC messages and not be allowed access to 2024-05-20T16:14:05.710 INFO:teuthology.orchestra.run.smithi082.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-05-20T16:14:05.710 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-20T16:14:05.711 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T16:14:05.711 DEBUG:teuthology.orchestra.run.smithi082:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-20T16:14:05.781 DEBUG:teuthology.orchestra.run.smithi082:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-20T16:14:05.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:14:05 smithi160 ceph-mon[36347]: pgmap v365: 129 pgs: 129 active+clean; 641 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-20T16:14:05.871 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:05 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-20T16:14:05.873 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-20T16:14:05.880 INFO:teuthology.orchestra.run.smithi082.stdout:client25013 2024-05-20T16:14:05.880 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-05-20T16:14:05.880 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-20T16:14:05.885 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:14:05 smithi184 ceph-mon[36209]: pgmap v365: 129 pgs: 129 active+clean; 641 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-20T16:14:05.896 DEBUG:teuthology.orchestra.run.smithi082:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-20T16:14:05.985 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:05 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-20T16:14:05.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-20T16:14:05.993 INFO:teuthology.orchestra.run.smithi082.stdout:client25013 2024-05-20T16:14:05.993 DEBUG:teuthology.orchestra.run.smithi082:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-20T16:14:06.042 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:06 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-20T16:14:06.045 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-20T16:14:06.050 INFO:teuthology.orchestra.run.smithi082.stdout:client25013 2024-05-20T16:14:06.050 DEBUG:teuthology.orchestra.run.smithi082:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968.client25013/status') 2024-05-20T16:14:06.099 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:06 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-20T16:14:06.109 INFO:teuthology.orchestra.run.smithi082.stdout:instance: client.25013 (3)172.21.15.82:0/1778484833 2024-05-20T16:14:06.109 INFO:teuthology.orchestra.run.smithi082.stdout:blocklisted: false 2024-05-20T16:14:06.110 INFO:teuthology.orchestra.run.smithi082.stderr:0+1 records in 2024-05-20T16:14:06.110 INFO:teuthology.orchestra.run.smithi082.stderr:0+1 records out 2024-05-20T16:14:06.110 INFO:teuthology.orchestra.run.smithi082.stderr:71 bytes copied, 2.1978e-05 s, 3.2 MB/s 2024-05-20T16:14:06.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-20T16:14:06.117 INFO:teuthology.run_tasks:Running task exec... 2024-05-20T16:14:06.127 INFO:teuthology.task.exec:Executing custom commands... 2024-05-20T16:14:06.127 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi082.front.sepia.ceph.com 2024-05-20T16:14:06.127 DEBUG:teuthology.orchestra.run.smithi082:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate false' 2024-05-20T16:14:06.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:06.240+0000 7f508154c640 1 -- 172.21.15.82:0/782561602 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f507c102c10 msgr2=0x7f507c103020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:14:06.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:06.240+0000 7f508154c640 1 --2- 172.21.15.82:0/782561602 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f507c102c10 0x7f507c103020 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f5064009900 tx=0x7f506402a410 comp rx=0 tx=0).stop 2024-05-20T16:14:06.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:06.240+0000 7f508154c640 1 -- 172.21.15.82:0/782561602 shutdown_connections 2024-05-20T16:14:06.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:06.240+0000 7f508154c640 1 --2- 172.21.15.82:0/782561602 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f507c102c10 0x7f507c103020 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:06.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:06.240+0000 7f508154c640 1 --2- 172.21.15.82:0/782561602 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f507c107910 0x7f507c109d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:06.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:06.240+0000 7f508154c640 1 --2- 172.21.15.82:0/782561602 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f507c1019d0 0x7f507c1026d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:06.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:06.240+0000 7f508154c640 1 -- 172.21.15.82:0/782561602 >> 172.21.15.82:0/782561602 conn(0x7f507c0fcbc0 msgr2=0x7f507c0ff070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:14:06.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:06.240+0000 7f508154c640 1 -- 172.21.15.82:0/782561602 shutdown_connections 2024-05-20T16:14:06.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:06.240+0000 7f508154c640 1 -- 172.21.15.82:0/782561602 wait complete. 2024-05-20T16:14:06.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:06.240+0000 7f508154c640 1 Processor -- start 2024-05-20T16:14:06.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:06.240+0000 7f508154c640 1 -- start start 2024-05-20T16:14:06.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:06.240+0000 7f508154c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f507c1019d0 0x7f507c069de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:14:06.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:06.241+0000 7f508154c640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f507c102c10 0x7f507c19e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:14:06.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:06.241+0000 7f508154c640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f507c107910 0x7f507c1a0630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:14:06.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:06.241+0000 7f508154c640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f507c10d6c0 con 0x7f507c1019d0 2024-05-20T16:14:06.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:06.241+0000 7f508154c640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f507c10d540 con 0x7f507c107910 2024-05-20T16:14:06.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:06.241+0000 7f508154c640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f507c10d840 con 0x7f507c102c10 2024-05-20T16:14:06.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:06.241+0000 7f507affd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f507c1019d0 0x7f507c069de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:14:06.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:06.241+0000 7f507affd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f507c1019d0 0x7f507c069de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58778/0 (socket says 172.21.15.82:58778) 2024-05-20T16:14:06.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:06.241+0000 7f507affd640 1 -- 172.21.15.82:0/1944462079 learned_addr learned my addr 172.21.15.82:0/1944462079 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:14:06.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:06.241+0000 7f507affd640 1 -- 172.21.15.82:0/1944462079 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f507c102c10 msgr2=0x7f507c19e0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:14:06.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:06.241+0000 7f507affd640 1 --2- 172.21.15.82:0/1944462079 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f507c102c10 0x7f507c19e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:06.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:06.241+0000 7f507affd640 1 -- 172.21.15.82:0/1944462079 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f507c107910 msgr2=0x7f507c1a0630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:14:06.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:06.241+0000 7f507affd640 1 --2- 172.21.15.82:0/1944462079 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f507c107910 0x7f507c1a0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:06.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:06.241+0000 7f507affd640 1 -- 172.21.15.82:0/1944462079 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f507c1a0e50 con 0x7f507c1019d0 2024-05-20T16:14:06.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:06.241+0000 7f507affd640 1 --2- 172.21.15.82:0/1944462079 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f507c1019d0 0x7f507c069de0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f506c006fd0 tx=0x7f506c00b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:14:06.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:06.241+0000 7f5078ff9640 1 -- 172.21.15.82:0/1944462079 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f506c011830 con 0x7f507c1019d0 2024-05-20T16:14:06.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:06.241+0000 7f5078ff9640 1 -- 172.21.15.82:0/1944462079 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f506c016070 con 0x7f507c1019d0 2024-05-20T16:14:06.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:06.241+0000 7f508154c640 1 -- 172.21.15.82:0/1944462079 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f507c1a1140 con 0x7f507c1019d0 2024-05-20T16:14:06.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:06.241+0000 7f5078ff9640 1 -- 172.21.15.82:0/1944462079 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f506c00f2e0 con 0x7f507c1019d0 2024-05-20T16:14:06.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:06.241+0000 7f508154c640 1 -- 172.21.15.82:0/1944462079 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f507c1a1760 con 0x7f507c1019d0 2024-05-20T16:14:06.245 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:06.242+0000 7f508154c640 1 -- 172.21.15.82:0/1944462079 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f507c106720 con 0x7f507c1019d0 2024-05-20T16:14:06.245 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:06.244+0000 7f5078ff9640 1 -- 172.21.15.82:0/1944462079 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f506c01b460 con 0x7f507c1019d0 2024-05-20T16:14:06.245 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:06.244+0000 7f5078ff9640 1 --2- 172.21.15.82:0/1944462079 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f50680a2c00 0x7f50680a5110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:14:06.245 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:06.244+0000 7f5073fff640 1 --2- 172.21.15.82:0/1944462079 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f50680a2c00 0x7f50680a5110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:14:06.245 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:06.244+0000 7f5078ff9640 1 -- 172.21.15.82:0/1944462079 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9150+0+0 (secure 0 0 0) 0x7f506c0cea70 con 0x7f507c1019d0 2024-05-20T16:14:06.245 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:06.245+0000 7f5073fff640 1 --2- 172.21.15.82:0/1944462079 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f50680a2c00 0x7f50680a5110 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f50640096c0 tx=0x7f5064031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:14:06.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:06.249+0000 7f5078ff9640 1 -- 172.21.15.82:0/1944462079 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f506c09b640 con 0x7f507c1019d0 2024-05-20T16:14:06.468 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:06.469+0000 7f508154c640 1 -- 172.21.15.82:0/1944462079 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"} v 0) -- 0x7f507c062750 con 0x7f507c1019d0 2024-05-20T16:14:06.521 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:06.521+0000 7f5078ff9640 1 -- 172.21.15.82:0/1944462079 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]=0 v25) ==== 118+0+0 (secure 0 0 0) 0x7f506c09af40 con 0x7f507c1019d0 2024-05-20T16:14:06.523 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:06.523+0000 7f508154c640 1 -- 172.21.15.82:0/1944462079 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f50680a2c00 msgr2=0x7f50680a5110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:14:06.523 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:06.523+0000 7f508154c640 1 --2- 172.21.15.82:0/1944462079 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f50680a2c00 0x7f50680a5110 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f50640096c0 tx=0x7f5064031400 comp rx=0 tx=0).stop 2024-05-20T16:14:06.523 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:06.523+0000 7f508154c640 1 -- 172.21.15.82:0/1944462079 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f507c1019d0 msgr2=0x7f507c069de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:14:06.523 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:06.523+0000 7f508154c640 1 --2- 172.21.15.82:0/1944462079 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f507c1019d0 0x7f507c069de0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f506c006fd0 tx=0x7f506c00b8e0 comp rx=0 tx=0).stop 2024-05-20T16:14:06.523 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:06.523+0000 7f508154c640 1 -- 172.21.15.82:0/1944462079 shutdown_connections 2024-05-20T16:14:06.523 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:06.523+0000 7f508154c640 1 --2- 172.21.15.82:0/1944462079 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f50680a2c00 0x7f50680a5110 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:06.523 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:06.523+0000 7f508154c640 1 --2- 172.21.15.82:0/1944462079 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f507c107910 0x7f507c1a0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:06.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:06.523+0000 7f508154c640 1 --2- 172.21.15.82:0/1944462079 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f507c102c10 0x7f507c19e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:06.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:06.523+0000 7f508154c640 1 --2- 172.21.15.82:0/1944462079 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f507c1019d0 0x7f507c069de0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:06.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:06.523+0000 7f508154c640 1 -- 172.21.15.82:0/1944462079 >> 172.21.15.82:0/1944462079 conn(0x7f507c0fcbc0 msgr2=0x7f507c1038f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:14:06.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:06.523+0000 7f508154c640 1 -- 172.21.15.82:0/1944462079 shutdown_connections 2024-05-20T16:14:06.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:06.523+0000 7f508154c640 1 -- 172.21.15.82:0/1944462079 wait complete. 2024-05-20T16:14:06.535 DEBUG:teuthology.orchestra.run.smithi082:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs subvolumegroup pin cephfs qa random 0.10' 2024-05-20T16:14:06.652 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:06.650+0000 7f7e9d461640 1 -- 172.21.15.82:0/188889042 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7e98102ae0 msgr2=0x7f7e981037e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:14:06.652 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:06.650+0000 7f7e9d461640 1 --2- 172.21.15.82:0/188889042 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7e98102ae0 0x7f7e981037e0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f7e88009900 tx=0x7f7e8802a410 comp rx=0 tx=0).stop 2024-05-20T16:14:06.652 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:06.650+0000 7f7e9d461640 1 -- 172.21.15.82:0/188889042 shutdown_connections 2024-05-20T16:14:06.652 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:06.650+0000 7f7e9d461640 1 --2- 172.21.15.82:0/188889042 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7e98108a20 0x7f7e9810ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:06.652 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:06.650+0000 7f7e9d461640 1 --2- 172.21.15.82:0/188889042 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7e98103d20 0x7f7e98104130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:06.652 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:06.650+0000 7f7e9d461640 1 --2- 172.21.15.82:0/188889042 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7e98102ae0 0x7f7e981037e0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:06.652 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:06.650+0000 7f7e9d461640 1 -- 172.21.15.82:0/188889042 >> 172.21.15.82:0/188889042 conn(0x7f7e980fdcb0 msgr2=0x7f7e98100180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:14:06.652 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:06.650+0000 7f7e9d461640 1 -- 172.21.15.82:0/188889042 shutdown_connections 2024-05-20T16:14:06.653 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:06.650+0000 7f7e9d461640 1 -- 172.21.15.82:0/188889042 wait complete. 2024-05-20T16:14:06.653 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:06.650+0000 7f7e9d461640 1 Processor -- start 2024-05-20T16:14:06.653 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:06.651+0000 7f7e9d461640 1 -- start start 2024-05-20T16:14:06.653 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:06.651+0000 7f7e9d461640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7e98102ae0 0x7f7e98199370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:14:06.653 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:06.651+0000 7f7e9d461640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7e98103d20 0x7f7e981998b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:14:06.653 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:06.651+0000 7f7e9d461640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7e98108a20 0x7f7e981a0840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:14:06.653 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:06.651+0000 7f7e9d461640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f7e9810e790 con 0x7f7e98102ae0 2024-05-20T16:14:06.653 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:06.651+0000 7f7e9d461640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f7e9810e610 con 0x7f7e98108a20 2024-05-20T16:14:06.653 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:06.651+0000 7f7e9d461640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f7e9810e910 con 0x7f7e98103d20 2024-05-20T16:14:06.653 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:06.651+0000 7f7e96ffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7e98102ae0 0x7f7e98199370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:14:06.653 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:06.651+0000 7f7e96ffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7e98102ae0 0x7f7e98199370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58796/0 (socket says 172.21.15.82:58796) 2024-05-20T16:14:06.653 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:06.651+0000 7f7e96ffd640 1 -- 172.21.15.82:0/1897232963 learned_addr learned my addr 172.21.15.82:0/1897232963 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:14:06.653 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:06.651+0000 7f7e977fe640 1 --2- 172.21.15.82:0/1897232963 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7e98108a20 0x7f7e981a0840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:14:06.653 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:06.651+0000 7f7e96ffd640 1 -- 172.21.15.82:0/1897232963 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7e98103d20 msgr2=0x7f7e981998b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:14:06.654 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:06.651+0000 7f7e967fc640 1 --2- 172.21.15.82:0/1897232963 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7e98103d20 0x7f7e981998b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:14:06.654 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:06.651+0000 7f7e96ffd640 1 --2- 172.21.15.82:0/1897232963 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7e98103d20 0x7f7e981998b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:06.654 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:06.651+0000 7f7e96ffd640 1 -- 172.21.15.82:0/1897232963 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7e98108a20 msgr2=0x7f7e981a0840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:14:06.654 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:06.651+0000 7f7e96ffd640 1 --2- 172.21.15.82:0/1897232963 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7e98108a20 0x7f7e981a0840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:06.654 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:06.651+0000 7f7e96ffd640 1 -- 172.21.15.82:0/1897232963 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e981a0eb0 con 0x7f7e98102ae0 2024-05-20T16:14:06.654 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:06.651+0000 7f7e977fe640 1 --2- 172.21.15.82:0/1897232963 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7e98108a20 0x7f7e981a0840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T16:14:06.654 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:06.651+0000 7f7e967fc640 1 --2- 172.21.15.82:0/1897232963 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7e98103d20 0x7f7e981998b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T16:14:06.654 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:06.651+0000 7f7e96ffd640 1 --2- 172.21.15.82:0/1897232963 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7e98102ae0 0x7f7e98199370 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f7e88006010 tx=0x7f7e880043b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:14:06.654 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:06.652+0000 7f7e7ffff640 1 -- 172.21.15.82:0/1897232963 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e8803f030 con 0x7f7e98102ae0 2024-05-20T16:14:06.654 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:06.652+0000 7f7e9d461640 1 -- 172.21.15.82:0/1897232963 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e981a1140 con 0x7f7e98102ae0 2024-05-20T16:14:06.654 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:06.652+0000 7f7e7ffff640 1 -- 172.21.15.82:0/1897232963 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7e88003ce0 con 0x7f7e98102ae0 2024-05-20T16:14:06.654 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:06.652+0000 7f7e7ffff640 1 -- 172.21.15.82:0/1897232963 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e88035070 con 0x7f7e98102ae0 2024-05-20T16:14:06.655 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:06.652+0000 7f7e9d461640 1 -- 172.21.15.82:0/1897232963 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e981a16b0 con 0x7f7e98102ae0 2024-05-20T16:14:06.655 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:06.653+0000 7f7e9d461640 1 -- 172.21.15.82:0/1897232963 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e98107830 con 0x7f7e98102ae0 2024-05-20T16:14:06.655 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:06.654+0000 7f7e7ffff640 1 -- 172.21.15.82:0/1897232963 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f7e88042030 con 0x7f7e98102ae0 2024-05-20T16:14:06.656 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:06.655+0000 7f7e7ffff640 1 --2- 172.21.15.82:0/1897232963 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f7e800a2b80 0x7f7e800a5090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:14:06.656 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:06.655+0000 7f7e967fc640 1 --2- 172.21.15.82:0/1897232963 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f7e800a2b80 0x7f7e800a5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:14:06.656 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:06.655+0000 7f7e7ffff640 1 -- 172.21.15.82:0/1897232963 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9150+0+0 (secure 0 0 0) 0x7f7e880efcc0 con 0x7f7e98102ae0 2024-05-20T16:14:06.656 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:06.655+0000 7f7e967fc640 1 --2- 172.21.15.82:0/1897232963 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f7e800a2b80 0x7f7e800a5090 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f7e8c006fd0 tx=0x7f7e8c008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:14:06.658 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:06.658+0000 7f7e7ffff640 1 -- 172.21.15.82:0/1897232963 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f7e880bc890 con 0x7f7e98102ae0 2024-05-20T16:14:06.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:14:06 smithi160 ceph-mon[36347]: from='client.? 172.21.15.82:0/1944462079' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-05-20T16:14:06.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:14:06 smithi082 ceph-mon[32238]: from='client.? 172.21.15.82:0/1944462079' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-05-20T16:14:06.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:06.860+0000 7f7e9d461640 1 -- 172.21.15.82:0/1897232963 --> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}) -- 0x7f7e98062030 con 0x7f7e800a2b80 2024-05-20T16:14:06.880 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:06.881+0000 7f7e7ffff640 1 -- 172.21.15.82:0/1897232963 <== mgr.14150 v2:172.21.15.82:6800/379530823 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3 (secure 0 0 0) 0x7f7e98062030 con 0x7f7e800a2b80 2024-05-20T16:14:06.880 INFO:teuthology.orchestra.run.smithi082.stdout:{} 2024-05-20T16:14:06.883 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:06.883+0000 7f7e9d461640 1 -- 172.21.15.82:0/1897232963 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f7e800a2b80 msgr2=0x7f7e800a5090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:14:06.883 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:06.883+0000 7f7e9d461640 1 --2- 172.21.15.82:0/1897232963 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f7e800a2b80 0x7f7e800a5090 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f7e8c006fd0 tx=0x7f7e8c008040 comp rx=0 tx=0).stop 2024-05-20T16:14:06.883 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:06.883+0000 7f7e9d461640 1 -- 172.21.15.82:0/1897232963 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7e98102ae0 msgr2=0x7f7e98199370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:14:06.883 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:06.883+0000 7f7e9d461640 1 --2- 172.21.15.82:0/1897232963 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7e98102ae0 0x7f7e98199370 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f7e88006010 tx=0x7f7e880043b0 comp rx=0 tx=0).stop 2024-05-20T16:14:06.883 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:06.883+0000 7f7e9d461640 1 -- 172.21.15.82:0/1897232963 shutdown_connections 2024-05-20T16:14:06.883 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:06.883+0000 7f7e9d461640 1 --2- 172.21.15.82:0/1897232963 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f7e800a2b80 0x7f7e800a5090 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:06.883 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:06.883+0000 7f7e9d461640 1 --2- 172.21.15.82:0/1897232963 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7e98108a20 0x7f7e981a0840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:06.883 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:06.883+0000 7f7e9d461640 1 --2- 172.21.15.82:0/1897232963 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7e98103d20 0x7f7e981998b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:06.884 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:06.883+0000 7f7e9d461640 1 --2- 172.21.15.82:0/1897232963 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7e98102ae0 0x7f7e98199370 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:06.884 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:06.883+0000 7f7e9d461640 1 -- 172.21.15.82:0/1897232963 >> 172.21.15.82:0/1897232963 conn(0x7f7e980fdcb0 msgr2=0x7f7e98104a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:14:06.884 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:06.883+0000 7f7e9d461640 1 -- 172.21.15.82:0/1897232963 shutdown_connections 2024-05-20T16:14:06.884 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:14:06.883+0000 7f7e9d461640 1 -- 172.21.15.82:0/1897232963 wait complete. 2024-05-20T16:14:06.885 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:14:06 smithi184 ceph-mon[36209]: from='client.? 172.21.15.82:0/1944462079' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-05-20T16:14:06.894 INFO:teuthology.run_tasks:Running task check-counter... 2024-05-20T16:14:06.905 INFO:teuthology.run_tasks:Running task background_exec... 2024-05-20T16:14:06.914 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi082.front.sepia.ceph.com 2024-05-20T16:14:06.914 DEBUG:teuthology.orchestra.run.smithi082:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2024-05-20T16:14:06.917 INFO:teuthology.run_tasks:Running task workunit... 2024-05-20T16:14:06.930 INFO:tasks.workunit:Pulling workunits from ref d2b507c724ef6c8295ee0a35d185cdab167fd61d 2024-05-20T16:14:06.930 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-05-20T16:14:06.930 INFO:tasks.workunit:timeout=3h 2024-05-20T16:14:06.930 INFO:tasks.workunit:cleanup=True 2024-05-20T16:14:06.930 DEBUG:teuthology.orchestra.run.smithi082:> stat -- /home/ubuntu/cephtest/mnt.0 2024-05-20T16:14:06.945 INFO:teuthology.orchestra.run.smithi082.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-05-20T16:14:06.945 INFO:teuthology.orchestra.run.smithi082.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-05-20T16:14:06.945 INFO:teuthology.orchestra.run.smithi082.stdout:Device: 77h/119d Inode: 1099511628283 Links: 2 2024-05-20T16:14:06.946 INFO:teuthology.orchestra.run.smithi082.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-20T16:14:06.946 INFO:teuthology.orchestra.run.smithi082.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-05-20T16:14:06.946 INFO:teuthology.orchestra.run.smithi082.stdout:Access: 2024-05-20 16:12:14.853702670 +0000 2024-05-20T16:14:06.946 INFO:teuthology.orchestra.run.smithi082.stdout:Modify: 2024-05-20 16:12:14.853702670 +0000 2024-05-20T16:14:06.946 INFO:teuthology.orchestra.run.smithi082.stdout:Change: 2024-05-20 16:14:05.774675582 +0000 2024-05-20T16:14:06.946 INFO:teuthology.orchestra.run.smithi082.stdout: Birth: 2024-05-20 16:12:14.853702670 +0000 2024-05-20T16:14:06.946 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-05-20T16:14:06.946 DEBUG:teuthology.orchestra.run.smithi082:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-05-20T16:14:07.017 DEBUG:teuthology.orchestra.run.smithi082:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout d2b507c724ef6c8295ee0a35d185cdab167fd61d 2024-05-20T16:14:07.073 INFO:tasks.workunit.client.0.smithi082.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-05-20T16:14:07.359 INFO:journalctl@ceph.mgr.x.smithi082.stdout:May 20 16:14:06 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mgr-x[32467]: 2024-05-20T16:14:06.865+0000 7f1a2d4ae700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-20T16:14:07.359 INFO:journalctl@ceph.mgr.x.smithi082.stdout:May 20 16:14:06 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mgr-x[32467]: 2024-05-20T16:14:06.865+0000 7f1a2d4ae700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-20T16:14:07.359 INFO:journalctl@ceph.mgr.x.smithi082.stdout:May 20 16:14:06 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mgr-x[32467]: 2024-05-20T16:14:06.865+0000 7f1a2d4ae700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-20T16:14:07.359 INFO:journalctl@ceph.mgr.x.smithi082.stdout:May 20 16:14:06 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mgr-x[32467]: 2024-05-20T16:14:06.865+0000 7f1a2d4ae700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-20T16:14:07.359 INFO:journalctl@ceph.mgr.x.smithi082.stdout:May 20 16:14:06 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mgr-x[32467]: 2024-05-20T16:14:06.865+0000 7f1a2d4ae700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-20T16:14:07.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:14:07 smithi160 ceph-mon[36347]: from='client.? 172.21.15.82:0/1944462079' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-05-20T16:14:07.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:14:07 smithi160 ceph-mon[36347]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-05-20T16:14:07.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:14:07 smithi160 ceph-mon[36347]: from='client.15459 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-05-20T16:14:07.813 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:14:07 smithi160 ceph-mon[36347]: pgmap v366: 129 pgs: 129 active+clean; 641 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2024-05-20T16:14:07.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:14:07 smithi082 ceph-mon[32238]: from='client.? 172.21.15.82:0/1944462079' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-05-20T16:14:07.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:14:07 smithi082 ceph-mon[32238]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-05-20T16:14:07.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:14:07 smithi082 ceph-mon[32238]: from='client.15459 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-05-20T16:14:07.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:14:07 smithi082 ceph-mon[32238]: pgmap v366: 129 pgs: 129 active+clean; 641 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2024-05-20T16:14:07.885 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:14:07 smithi184 ceph-mon[36209]: from='client.? 172.21.15.82:0/1944462079' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-05-20T16:14:07.886 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:14:07 smithi184 ceph-mon[36209]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-05-20T16:14:07.886 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:14:07 smithi184 ceph-mon[36209]: from='client.15459 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-05-20T16:14:07.886 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:14:07 smithi184 ceph-mon[36209]: pgmap v366: 129 pgs: 129 active+clean; 641 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2024-05-20T16:14:08.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:14:08 smithi160 ceph-mon[36347]: mgrmap e19: x(active, since 9m), standbys: z, y 2024-05-20T16:14:08.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:14:08 smithi082 ceph-mon[32238]: mgrmap e19: x(active, since 9m), standbys: z, y 2024-05-20T16:14:08.885 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:14:08 smithi184 ceph-mon[36209]: mgrmap e19: x(active, since 9m), standbys: z, y 2024-05-20T16:14:09.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:14:09 smithi160 ceph-mon[36347]: pgmap v367: 129 pgs: 129 active+clean; 653 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-05-20T16:14:09.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:14:09 smithi082 ceph-mon[32238]: pgmap v367: 129 pgs: 129 active+clean; 653 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-05-20T16:14:09.885 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:14:09 smithi184 ceph-mon[36209]: pgmap v367: 129 pgs: 129 active+clean; 653 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-05-20T16:14:11.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:14:11 smithi160 ceph-mon[36347]: pgmap v368: 129 pgs: 129 active+clean; 653 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.8 KiB/s wr, 6 op/s 2024-05-20T16:14:11.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:14:11 smithi082 ceph-mon[32238]: pgmap v368: 129 pgs: 129 active+clean; 653 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.8 KiB/s wr, 6 op/s 2024-05-20T16:14:11.885 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:14:11 smithi184 ceph-mon[36209]: pgmap v368: 129 pgs: 129 active+clean; 653 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.8 KiB/s wr, 6 op/s 2024-05-20T16:14:13.661 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:14:13 smithi082 ceph-mon[32238]: pgmap v369: 129 pgs: 129 active+clean; 653 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.9 KiB/s wr, 8 op/s 2024-05-20T16:14:13.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:14:13 smithi160 ceph-mon[36347]: pgmap v369: 129 pgs: 129 active+clean; 653 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.9 KiB/s wr, 8 op/s 2024-05-20T16:14:13.885 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:14:13 smithi184 ceph-mon[36209]: pgmap v369: 129 pgs: 129 active+clean; 653 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.9 KiB/s wr, 8 op/s 2024-05-20T16:14:15.635 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:14:15 smithi184 ceph-mon[36209]: pgmap v370: 129 pgs: 129 active+clean; 653 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.9 KiB/s wr, 7 op/s 2024-05-20T16:14:15.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:14:15 smithi160 ceph-mon[36347]: pgmap v370: 129 pgs: 129 active+clean; 653 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.9 KiB/s wr, 7 op/s 2024-05-20T16:14:15.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:14:15 smithi082 ceph-mon[32238]: pgmap v370: 129 pgs: 129 active+clean; 653 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.9 KiB/s wr, 7 op/s 2024-05-20T16:14:17.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:14:17 smithi160 ceph-mon[36347]: pgmap v371: 129 pgs: 129 active+clean; 653 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.9 KiB/s wr, 6 op/s 2024-05-20T16:14:17.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:14:17 smithi082 ceph-mon[32238]: pgmap v371: 129 pgs: 129 active+clean; 653 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.9 KiB/s wr, 6 op/s 2024-05-20T16:14:17.885 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:14:17 smithi184 ceph-mon[36209]: pgmap v371: 129 pgs: 129 active+clean; 653 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.9 KiB/s wr, 6 op/s 2024-05-20T16:14:19.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:14:19 smithi160 ceph-mon[36347]: pgmap v372: 129 pgs: 129 active+clean; 655 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-05-20T16:14:19.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:14:19 smithi082 ceph-mon[32238]: pgmap v372: 129 pgs: 129 active+clean; 655 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-05-20T16:14:19.886 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:14:19 smithi184 ceph-mon[36209]: pgmap v372: 129 pgs: 129 active+clean; 655 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-05-20T16:14:20.067 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:20.067+0000 7f031caa2640 1 -- 172.21.15.82:0/2014292265 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0318103d10 msgr2=0x7f0318104120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:14:20.067 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:20.067+0000 7f031caa2640 1 --2- 172.21.15.82:0/2014292265 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0318103d10 0x7f0318104120 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f0300009900 tx=0x7f030002a410 comp rx=0 tx=0).stop 2024-05-20T16:14:20.067 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:20.067+0000 7f031caa2640 1 -- 172.21.15.82:0/2014292265 shutdown_connections 2024-05-20T16:14:20.067 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:20.067+0000 7f031caa2640 1 --2- 172.21.15.82:0/2014292265 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0318103d10 0x7f0318104120 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:20.067 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:20.067+0000 7f031caa2640 1 --2- 172.21.15.82:0/2014292265 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0318108a10 0x7f031810ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:20.068 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:20.067+0000 7f031caa2640 1 --2- 172.21.15.82:0/2014292265 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0318102ad0 0x7f03181037d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:20.068 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:20.067+0000 7f031caa2640 1 -- 172.21.15.82:0/2014292265 >> 172.21.15.82:0/2014292265 conn(0x7f03180fdcc0 msgr2=0x7f0318100170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:14:20.068 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:20.067+0000 7f031caa2640 1 -- 172.21.15.82:0/2014292265 shutdown_connections 2024-05-20T16:14:20.068 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:20.067+0000 7f031caa2640 1 -- 172.21.15.82:0/2014292265 wait complete. 2024-05-20T16:14:20.070 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:20.068+0000 7f031caa2640 1 Processor -- start 2024-05-20T16:14:20.070 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:20.068+0000 7f031caa2640 1 -- start start 2024-05-20T16:14:20.070 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:20.068+0000 7f031caa2640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0318102ad0 0x7f03181992b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:14:20.070 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:20.068+0000 7f031caa2640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0318103d10 0x7f03181997f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:14:20.070 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:20.068+0000 7f031caa2640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0318108a10 0x7f03181a0780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:14:20.070 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:20.068+0000 7f031caa2640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f031810e7a0 con 0x7f0318108a10 2024-05-20T16:14:20.070 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:20.068+0000 7f031caa2640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f031810e620 con 0x7f0318102ad0 2024-05-20T16:14:20.070 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:20.068+0000 7f031caa2640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f031810e920 con 0x7f0318103d10 2024-05-20T16:14:20.071 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:20.068+0000 7f0316d9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0318108a10 0x7f03181a0780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:14:20.071 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:20.068+0000 7f0316d9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0318108a10 0x7f03181a0780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:46556/0 (socket says 172.21.15.82:46556) 2024-05-20T16:14:20.071 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:20.068+0000 7f0316d9d640 1 -- 172.21.15.82:0/1290747199 learned_addr learned my addr 172.21.15.82:0/1290747199 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:14:20.071 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:20.068+0000 7f031659c640 1 --2- 172.21.15.82:0/1290747199 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0318102ad0 0x7f03181992b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:14:20.071 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:20.068+0000 7f0315d9b640 1 --2- 172.21.15.82:0/1290747199 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0318103d10 0x7f03181997f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:14:20.071 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:20.068+0000 7f0316d9d640 1 -- 172.21.15.82:0/1290747199 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0318103d10 msgr2=0x7f03181997f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:14:20.071 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:20.068+0000 7f0316d9d640 1 --2- 172.21.15.82:0/1290747199 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0318103d10 0x7f03181997f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:20.071 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:20.069+0000 7f0316d9d640 1 -- 172.21.15.82:0/1290747199 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0318102ad0 msgr2=0x7f03181992b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:14:20.071 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:20.069+0000 7f0316d9d640 1 --2- 172.21.15.82:0/1290747199 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0318102ad0 0x7f03181992b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:20.071 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:20.069+0000 7f0316d9d640 1 -- 172.21.15.82:0/1290747199 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03181a0df0 con 0x7f0318108a10 2024-05-20T16:14:20.071 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:20.069+0000 7f0316d9d640 1 --2- 172.21.15.82:0/1290747199 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0318108a10 0x7f03181a0780 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f031000c970 tx=0x7f031000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:14:20.071 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:20.069+0000 7f030f7fe640 1 -- 172.21.15.82:0/1290747199 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0310007bb0 con 0x7f0318108a10 2024-05-20T16:14:20.071 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:20.069+0000 7f030f7fe640 1 -- 172.21.15.82:0/1290747199 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0310007d50 con 0x7f0318108a10 2024-05-20T16:14:20.071 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:20.069+0000 7f030f7fe640 1 -- 172.21.15.82:0/1290747199 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f031000f450 con 0x7f0318108a10 2024-05-20T16:14:20.072 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:20.069+0000 7f031caa2640 1 -- 172.21.15.82:0/1290747199 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03181a10e0 con 0x7f0318108a10 2024-05-20T16:14:20.072 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:20.069+0000 7f031caa2640 1 -- 172.21.15.82:0/1290747199 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03181a15b0 con 0x7f0318108a10 2024-05-20T16:14:20.075 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:20.072+0000 7f0323a23740 1 Processor -- start 2024-05-20T16:14:20.075 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:20.072+0000 7f030f7fe640 1 -- 172.21.15.82:0/1290747199 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 19) ==== 137884+0+0 (secure 0 0 0) 0x7f0310004f20 con 0x7f0318108a10 2024-05-20T16:14:20.075 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:20.072+0000 7f0323a23740 1 -- start start 2024-05-20T16:14:20.075 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:20.073+0000 7f0323a23740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5580d9e2b1b0 0x5580da1cae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:14:20.075 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:20.073+0000 7f0323a23740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5580d9e022f0 0x5580da1cb3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:14:20.075 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:20.073+0000 7f0323a23740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5580d9e0a4d0 0x5580da1d22e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:14:20.075 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:20.073+0000 7f0323a23740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x5580da12f340 con 0x5580d9e0a4d0 2024-05-20T16:14:20.075 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:20.073+0000 7f0323a23740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x5580da0d7290 con 0x5580d9e022f0 2024-05-20T16:14:20.075 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:20.073+0000 7f0323a23740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5580da12b300 con 0x5580d9e2b1b0 2024-05-20T16:14:20.075 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:20.073+0000 7f0315d9b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5580d9e0a4d0 0x5580da1d22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:14:20.075 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:20.073+0000 7f0315d9b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5580d9e0a4d0 0x5580da1d22e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:46570/0 (socket says 172.21.15.82:46570) 2024-05-20T16:14:20.075 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:20.073+0000 7f0315d9b640 1 -- 172.21.15.82:0/3195455044 learned_addr learned my addr 172.21.15.82:0/3195455044 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:14:20.076 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:20.073+0000 7f0315d9b640 1 --2- 172.21.15.82:0/3195455044 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5580d9e2b1b0 0x5580da1cae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:14:20.076 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:20.073+0000 7f031659c640 1 --2- 172.21.15.82:0/3195455044 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5580d9e022f0 0x5580da1cb3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:14:20.076 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:20.073+0000 7f030f7fe640 1 --2- 172.21.15.82:0/1290747199 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f03040a2c10 0x7f03040a5120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:14:20.076 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:20.073+0000 7f0315d9b640 1 -- 172.21.15.82:0/3195455044 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5580d9e2b1b0 msgr2=0x5580da1cae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:14:20.076 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:20.073+0000 7f0315d9b640 1 --2- 172.21.15.82:0/3195455044 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5580d9e2b1b0 0x5580da1cae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:20.076 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:20.073+0000 7f0315d9b640 1 -- 172.21.15.82:0/3195455044 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5580d9e022f0 msgr2=0x5580da1cb3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:14:20.076 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:20.073+0000 7f0315d9b640 1 --2- 172.21.15.82:0/3195455044 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5580d9e022f0 0x5580da1cb3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:20.076 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:20.073+0000 7f0315d9b640 1 -- 172.21.15.82:0/3195455044 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5580da0fb8e0 con 0x5580d9e0a4d0 2024-05-20T16:14:20.076 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:20.073+0000 7f0316d9d640 1 --2- 172.21.15.82:0/1290747199 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f03040a2c10 0x7f03040a5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:14:20.076 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:20.073+0000 7f0315d9b640 1 --2- 172.21.15.82:0/3195455044 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5580d9e0a4d0 0x5580da1d22e0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f0300009230 tx=0x7f0300000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc0b399159b84fc server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:14:20.076 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:20.073+0000 7f030f7fe640 1 -- 172.21.15.82:0/1290747199 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9150+0+0 (secure 0 0 0) 0x7f03100cf180 con 0x7f0318108a10 2024-05-20T16:14:20.076 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:20.074+0000 7f030cff9640 1 -- 172.21.15.82:0/3195455044 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0300044020 con 0x5580d9e0a4d0 2024-05-20T16:14:20.076 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:20.074+0000 7f030cff9640 1 -- 172.21.15.82:0/3195455044 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0300004760 con 0x5580d9e0a4d0 2024-05-20T16:14:20.076 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:20.074+0000 7f030cff9640 1 -- 172.21.15.82:0/3195455044 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0300035040 con 0x5580d9e0a4d0 2024-05-20T16:14:20.077 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:20.075+0000 7f0323a23740 1 -- 172.21.15.82:0/3195455044 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5580d9e0a4d0 msgr2=0x5580da1d22e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:14:20.077 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:20.075+0000 7f0316d9d640 1 --2- 172.21.15.82:0/1290747199 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f03040a2c10 0x7f03040a5120 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f031009b530 tx=0x7f0310086040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:14:20.077 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:20.075+0000 7f0323a23740 1 --2- 172.21.15.82:0/3195455044 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5580d9e0a4d0 0x5580da1d22e0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f0300009230 tx=0x7f0300000bc0 comp rx=0 tx=0).stop 2024-05-20T16:14:20.077 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:20.075+0000 7f0323a23740 1 -- 172.21.15.82:0/3195455044 shutdown_connections 2024-05-20T16:14:20.077 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:20.075+0000 7f0323a23740 1 --2- 172.21.15.82:0/3195455044 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5580d9e0a4d0 0x5580da1d22e0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:20.077 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:20.075+0000 7f0323a23740 1 --2- 172.21.15.82:0/3195455044 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5580d9e022f0 0x5580da1cb3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:20.077 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:20.075+0000 7f0323a23740 1 --2- 172.21.15.82:0/3195455044 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5580d9e2b1b0 0x5580da1cae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:20.077 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:20.075+0000 7f0323a23740 1 -- 172.21.15.82:0/3195455044 >> 172.21.15.82:0/3195455044 conn(0x5580d9e1f300 msgr2=0x5580da1c67f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:14:20.077 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:20.075+0000 7f0323a23740 1 -- 172.21.15.82:0/3195455044 shutdown_connections 2024-05-20T16:14:20.077 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:20.075+0000 7f0323a23740 1 -- 172.21.15.82:0/3195455044 wait complete. 2024-05-20T16:14:20.077 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:20.075+0000 7f0323a23740 1 -- start start 2024-05-20T16:14:20.077 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:20.075+0000 7f0323a23740 1 Processor -- start 2024-05-20T16:14:20.081 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:20.075+0000 7f0323a23740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5580d9e022f0 0x5580da1d4b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:14:20.081 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:20.075+0000 7f0323a23740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5580d9e0a4d0 0x5580da1d5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:14:20.081 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:20.075+0000 7f0323a23740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5580d9e2b1b0 0x5580da1dbfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:14:20.081 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:20.075+0000 7f0323a23740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x5580da1d2cd0 con 0x5580d9e022f0 2024-05-20T16:14:20.081 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:20.075+0000 7f0323a23740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x5580da0ea790 con 0x5580d9e2b1b0 2024-05-20T16:14:20.081 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:20.075+0000 7f0323a23740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5580da1d2e50 con 0x5580d9e0a4d0 2024-05-20T16:14:20.081 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:20.076+0000 7f0323a23740 10 client.0 _ll_register_callbacks cb 0x5580da125a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T16:14:20.081 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:20.076+0000 7f0315d9b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5580d9e022f0 0x5580da1d4b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:14:20.081 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:20.076+0000 7f0315d9b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5580d9e022f0 0x5580da1d4b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:46576/0 (socket says 172.21.15.82:46576) 2024-05-20T16:14:20.081 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:20.076+0000 7f0315d9b640 1 -- 172.21.15.82:0/3593716816 learned_addr learned my addr 172.21.15.82:0/3593716816 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:14:20.082 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:20.076+0000 7f0315d9b640 1 -- 172.21.15.82:0/3593716816 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5580d9e0a4d0 msgr2=0x5580da1d5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:14:20.082 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:20.076+0000 7f031659c640 1 --2- 172.21.15.82:0/3593716816 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5580d9e0a4d0 0x5580da1d5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:14:20.082 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:20.076+0000 7f0315d9b640 1 --2- 172.21.15.82:0/3593716816 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5580d9e0a4d0 0x5580da1d5050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:20.082 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:20.076+0000 7f0315d9b640 1 -- 172.21.15.82:0/3593716816 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5580d9e2b1b0 msgr2=0x5580da1dbfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:14:20.082 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:20.076+0000 7f0315d9b640 1 --2- 172.21.15.82:0/3593716816 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5580d9e2b1b0 0x5580da1dbfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:20.082 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:20.076+0000 7f0315d9b640 1 -- 172.21.15.82:0/3593716816 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5580d9e43210 con 0x5580d9e022f0 2024-05-20T16:14:20.082 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:20.076+0000 7f031659c640 1 --2- 172.21.15.82:0/3593716816 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5580d9e0a4d0 0x5580da1d5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T16:14:20.082 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:20.076+0000 7f0315d9b640 1 --2- 172.21.15.82:0/3593716816 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5580d9e022f0 0x5580da1d4b10 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f0300005ab0 tx=0x7f0300009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da18eab8cb53f4ae server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:14:20.082 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:20.076+0000 7f02fdffb640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-05-20T16:14:20.082 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:20.076+0000 7f02fdffb640 1 -- 172.21.15.82:0/3593716816 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0300035070 con 0x5580d9e022f0 2024-05-20T16:14:20.082 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:20.076+0000 7f02fdffb640 1 -- 172.21.15.82:0/3593716816 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9150+0+0 (secure 0 0 0) 0x7f0300031b10 con 0x5580d9e022f0 2024-05-20T16:14:20.082 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:20.076+0000 7f0323a23740 1 -- 172.21.15.82:0/3593716816 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0300004760 con 0x5580d9e022f0 2024-05-20T16:14:20.082 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:20.076+0000 7f02fdffb640 1 -- 172.21.15.82:0/3593716816 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f0300044040 con 0x5580d9e022f0 2024-05-20T16:14:20.083 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:20.076+0000 7f0323a23740 10 client.15477 fetch_fsmap learned FSMap version 25 2024-05-20T16:14:20.083 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:20.076+0000 7f0323a23740 1 -- 172.21.15.82:0/3593716816 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5580d9f0f620 con 0x5580d9e022f0 2024-05-20T16:14:20.083 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:20.077+0000 7f02fdffb640 1 -- 172.21.15.82:0/3593716816 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0300004760 con 0x5580d9e022f0 2024-05-20T16:14:20.083 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:20.077+0000 7f02fdffb640 1 -- 172.21.15.82:0/3593716816 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f03000038f0 con 0x5580d9e022f0 2024-05-20T16:14:20.083 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:20.077+0000 7f0323a23740 10 client.15477 fetch_fsmap finished waiting for FSMap version 25 2024-05-20T16:14:20.083 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:20.077+0000 7f0323a23740 10 client.15477 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T16:14:20.083 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:20.077+0000 7f0323a23740 20 client.15477 populate_metadata read hostname 'smithi082' 2024-05-20T16:14:20.083 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:20.077+0000 7f0323a23740 1 --2- 172.21.15.82:0/3593716816 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x5580d9de9d80 0x5580da1e2070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:14:20.083 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:20.077+0000 7f0323a23740 4 client.15477 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T16:14:20.083 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:20.077+0000 7f0323a23740 1 -- 172.21.15.82:0/3593716816 --> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5580da13ca70 con 0x5580d9de9d80 2024-05-20T16:14:20.083 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:20.077+0000 7f031659c640 1 --2- 172.21.15.82:0/3593716816 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x5580d9de9d80 0x5580da1e2070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:14:20.083 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:20.078+0000 7f02fdffb640 10 client.15477 ms_handle_connect on v2:172.21.15.160:6838/2765554482 2024-05-20T16:14:20.083 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:20.078+0000 7f031659c640 1 --2- 172.21.15.82:0/3593716816 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x5580d9de9d80 0x5580da1e2070 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=81eeef37537c9bb8 server_cookie=4122c5db9f4dcd5c in_seq=0 out_seq=0 2024-05-20T16:14:20.083 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:20.078+0000 7f02fdffb640 1 -- 172.21.15.82:0/3593716816 <== mds.0 v2:172.21.15.160:6838/2765554482 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5580da13ca70 con 0x5580d9de9d80 2024-05-20T16:14:20.084 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:20.078+0000 7f02fdffb640 10 client.15477 handle_command_reply: tid=0 2024-05-20T16:14:20.084 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:20.078+0000 7f0323a23740 1 client.15477 shutdown 2024-05-20T16:14:20.084 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:20.079+0000 7f0323a23740 1 -- 172.21.15.82:0/3593716816 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5580d9e022f0 msgr2=0x5580da1d4b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:14:20.084 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:20.079+0000 7f0323a23740 1 --2- 172.21.15.82:0/3593716816 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5580d9e022f0 0x5580da1d4b10 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f0300005ab0 tx=0x7f0300009260 comp rx=0 tx=0).stop 2024-05-20T16:14:20.084 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:20.079+0000 7f0323a23740 1 -- 172.21.15.82:0/3593716816 shutdown_connections 2024-05-20T16:14:20.084 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:20.079+0000 7f0323a23740 1 --2- 172.21.15.82:0/3593716816 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x5580d9de9d80 0x5580da1e2070 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:20.084 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:20.079+0000 7f0323a23740 1 --2- 172.21.15.82:0/3593716816 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5580d9e2b1b0 0x5580da1dbfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:20.084 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:20.079+0000 7f02fdffb640 0 client.15477 ms_handle_reset on v2:172.21.15.160:6838/2765554482 2024-05-20T16:14:20.084 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:20.079+0000 7f0323a23740 1 --2- 172.21.15.82:0/3593716816 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5580d9e0a4d0 0x5580da1d5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:20.084 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:20.079+0000 7f0323a23740 1 --2- 172.21.15.82:0/3593716816 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5580d9e022f0 0x5580da1d4b10 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:20.084 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:20.079+0000 7f0323a23740 1 -- 172.21.15.82:0/3593716816 >> 172.21.15.82:0/3593716816 conn(0x5580d9e1f300 msgr2=0x5580da1c5810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:14:20.084 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:20.079+0000 7f0323a23740 1 -- 172.21.15.82:0/3593716816 shutdown_connections 2024-05-20T16:14:20.084 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:20.079+0000 7f0323a23740 1 -- 172.21.15.82:0/3593716816 wait complete. 2024-05-20T16:14:20.084 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:20.079+0000 7f0323a23740 20 client.15477 trim_cache size 0 max 16384 2024-05-20T16:14:20.099 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:20.094+0000 7f0323a23740 1 Processor -- start 2024-05-20T16:14:20.099 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:20.094+0000 7f0323a23740 1 -- start start 2024-05-20T16:14:20.100 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:20.094+0000 7f0323a23740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5580d9de9d80 0x5580da1cf640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:14:20.100 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:20.094+0000 7f0323a23740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5580d9e022f0 0x5580da1d4df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:14:20.100 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:20.094+0000 7f0323a23740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5580d9e0a4d0 0x5580da1dbf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:14:20.100 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:20.094+0000 7f0323a23740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f03181ae2e0 con 0x5580d9e0a4d0 2024-05-20T16:14:20.100 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:20.094+0000 7f0323a23740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f03181abbb0 con 0x5580d9de9d80 2024-05-20T16:14:20.100 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:20.094+0000 7f0323a23740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f03181a9a00 con 0x5580d9e022f0 2024-05-20T16:14:20.100 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:20.094+0000 7f0315d9b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5580d9e0a4d0 0x5580da1dbf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:14:20.100 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:20.094+0000 7f0315d9b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5580d9e0a4d0 0x5580da1dbf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:46584/0 (socket says 172.21.15.82:46584) 2024-05-20T16:14:20.100 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:20.094+0000 7f0315d9b640 1 -- 172.21.15.82:0/3669555647 learned_addr learned my addr 172.21.15.82:0/3669555647 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:14:20.100 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:20.094+0000 7f031659c640 1 --2- 172.21.15.82:0/3669555647 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5580d9e022f0 0x5580da1d4df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:14:20.100 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:20.094+0000 7f0315d9b640 1 -- 172.21.15.82:0/3669555647 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5580d9e022f0 msgr2=0x5580da1d4df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:14:20.101 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:20.094+0000 7f0315d9b640 1 --2- 172.21.15.82:0/3669555647 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5580d9e022f0 0x5580da1d4df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:20.101 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:20.094+0000 7f0315d9b640 1 -- 172.21.15.82:0/3669555647 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5580d9de9d80 msgr2=0x5580da1cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:14:20.101 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:20.094+0000 7f0315d9b640 1 --2- 172.21.15.82:0/3669555647 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5580d9de9d80 0x5580da1cf640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:20.101 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:20.094+0000 7f0315d9b640 1 -- 172.21.15.82:0/3669555647 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5580da1d3bb0 con 0x5580d9e0a4d0 2024-05-20T16:14:20.101 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:20.094+0000 7f0315d9b640 1 --2- 172.21.15.82:0/3669555647 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5580d9e0a4d0 0x5580da1dbf40 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f030002a930 tx=0x7f0300000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d39251b5881ee879 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:14:20.101 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:20.095+0000 7f02fffff640 1 -- 172.21.15.82:0/3669555647 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0300035040 con 0x5580d9e0a4d0 2024-05-20T16:14:20.102 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:20.095+0000 7f02fffff640 1 -- 172.21.15.82:0/3669555647 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f030004e070 con 0x5580d9e0a4d0 2024-05-20T16:14:20.102 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:20.095+0000 7f02fffff640 1 -- 172.21.15.82:0/3669555647 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03000039f0 con 0x5580d9e0a4d0 2024-05-20T16:14:20.102 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:20.095+0000 7f0323a23740 1 -- 172.21.15.82:0/3669555647 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5580d9e0a4d0 msgr2=0x5580da1dbf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:14:20.102 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:20.095+0000 7f0323a23740 1 --2- 172.21.15.82:0/3669555647 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5580d9e0a4d0 0x5580da1dbf40 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f030002a930 tx=0x7f0300000bc0 comp rx=0 tx=0).stop 2024-05-20T16:14:20.102 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:20.095+0000 7f0323a23740 1 -- 172.21.15.82:0/3669555647 shutdown_connections 2024-05-20T16:14:20.102 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:20.095+0000 7f0323a23740 1 --2- 172.21.15.82:0/3669555647 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5580d9e0a4d0 0x5580da1dbf40 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:20.102 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:20.095+0000 7f0323a23740 1 --2- 172.21.15.82:0/3669555647 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5580d9e022f0 0x5580da1d4df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:20.102 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:20.095+0000 7f0323a23740 1 --2- 172.21.15.82:0/3669555647 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5580d9de9d80 0x5580da1cf640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:20.102 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:20.095+0000 7f0323a23740 1 -- 172.21.15.82:0/3669555647 >> 172.21.15.82:0/3669555647 conn(0x5580d9e1f300 msgr2=0x5580da1d46e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:14:20.102 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:20.095+0000 7f0323a23740 1 -- 172.21.15.82:0/3669555647 shutdown_connections 2024-05-20T16:14:20.102 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:20.096+0000 7f0323a23740 1 -- 172.21.15.82:0/3669555647 wait complete. 2024-05-20T16:14:20.102 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:20.096+0000 7f0323a23740 1 -- start start 2024-05-20T16:14:20.103 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:20.096+0000 7f0323a23740 1 Processor -- start 2024-05-20T16:14:20.103 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:20.096+0000 7f0323a23740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5580d9de9d80 0x5580da1d7860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:14:20.103 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:20.096+0000 7f0323a23740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5580d9e022f0 0x5580da1d7da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:14:20.103 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:20.096+0000 7f0323a23740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5580d9e0a4d0 0x5580da1dace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:14:20.103 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:20.096+0000 7f0323a23740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f03181aa070 con 0x5580d9e022f0 2024-05-20T16:14:20.103 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:20.096+0000 7f0323a23740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f03181ab4e0 con 0x5580d9e0a4d0 2024-05-20T16:14:20.103 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:20.096+0000 7f0323a23740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f03181aa740 con 0x5580d9de9d80 2024-05-20T16:14:20.103 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:20.096+0000 7f0323a23740 10 client.0 _ll_register_callbacks cb 0x7f03000043f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T16:14:20.103 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:20.096+0000 7f031659c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5580d9e022f0 0x5580da1d7da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:14:20.103 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:20.096+0000 7f031659c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5580d9e022f0 0x5580da1d7da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:46592/0 (socket says 172.21.15.82:46592) 2024-05-20T16:14:20.103 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:20.096+0000 7f031659c640 1 -- 172.21.15.82:0/1400975542 learned_addr learned my addr 172.21.15.82:0/1400975542 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:14:20.103 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:20.096+0000 7f0315d9b640 1 --2- 172.21.15.82:0/1400975542 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5580d9de9d80 0x5580da1d7860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:14:20.104 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:20.096+0000 7f0315d9b640 1 --2- 172.21.15.82:0/1400975542 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5580d9e0a4d0 0x5580da1dace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:14:20.104 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:20.097+0000 7f031659c640 1 -- 172.21.15.82:0/1400975542 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5580d9de9d80 msgr2=0x5580da1d7860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:14:20.104 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:20.097+0000 7f031659c640 1 --2- 172.21.15.82:0/1400975542 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5580d9de9d80 0x5580da1d7860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:20.104 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:20.097+0000 7f031659c640 1 -- 172.21.15.82:0/1400975542 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5580d9e0a4d0 msgr2=0x5580da1dace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:14:20.104 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:20.097+0000 7f031659c640 1 --2- 172.21.15.82:0/1400975542 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5580d9e0a4d0 0x5580da1dace0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:20.104 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:20.097+0000 7f031659c640 1 -- 172.21.15.82:0/1400975542 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5580da1db2a0 con 0x5580d9e022f0 2024-05-20T16:14:20.104 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:20.097+0000 7f0315d9b640 1 --2- 172.21.15.82:0/1400975542 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5580d9e0a4d0 0x5580da1dace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T16:14:20.104 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:20.097+0000 7f031659c640 1 --2- 172.21.15.82:0/1400975542 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5580d9e022f0 0x5580da1d7da0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f03181023a0 tx=0x7f03080139f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9834f04d96478692 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:14:20.104 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:20.097+0000 7f02fdffb640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-05-20T16:14:20.104 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:20.097+0000 7f02fdffb640 1 -- 172.21.15.82:0/1400975542 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f030801b640 con 0x5580d9e022f0 2024-05-20T16:14:20.104 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:20.097+0000 7f02fdffb640 1 -- 172.21.15.82:0/1400975542 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9150+0+0 (secure 0 0 0) 0x7f030801b7e0 con 0x5580d9e022f0 2024-05-20T16:14:20.104 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:20.097+0000 7f0323a23740 1 -- 172.21.15.82:0/1400975542 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f030004e070 con 0x5580d9e022f0 2024-05-20T16:14:20.104 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:20.097+0000 7f02fdffb640 1 -- 172.21.15.82:0/1400975542 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f030802d050 con 0x5580d9e022f0 2024-05-20T16:14:20.104 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:20.097+0000 7f0323a23740 10 client.15483 fetch_fsmap learned FSMap version 25 2024-05-20T16:14:20.105 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:20.097+0000 7f0323a23740 1 -- 172.21.15.82:0/1400975542 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5580da1dc630 con 0x5580d9e022f0 2024-05-20T16:14:20.105 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:20.097+0000 7f02fdffb640 1 -- 172.21.15.82:0/1400975542 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f030004e070 con 0x5580d9e022f0 2024-05-20T16:14:20.105 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:20.097+0000 7f02fdffb640 1 -- 172.21.15.82:0/1400975542 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f030800f040 con 0x5580d9e022f0 2024-05-20T16:14:20.105 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:20.097+0000 7f0323a23740 10 client.15483 fetch_fsmap finished waiting for FSMap version 25 2024-05-20T16:14:20.105 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:20.097+0000 7f0323a23740 10 client.15483 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T16:14:20.105 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:20.097+0000 7f0323a23740 20 client.15483 populate_metadata read hostname 'smithi082' 2024-05-20T16:14:20.105 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:20.097+0000 7f0323a23740 1 --2- 172.21.15.82:0/1400975542 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x5580d9e2b1b0 0x5580da24c680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:14:20.105 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:20.097+0000 7f0323a23740 4 client.15483 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-20T16:14:20.105 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:20.097+0000 7f0323a23740 1 -- 172.21.15.82:0/1400975542 --> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5580da136dc0 con 0x5580d9e2b1b0 2024-05-20T16:14:20.105 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:20.098+0000 7f0315d9b640 1 --2- 172.21.15.82:0/1400975542 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x5580d9e2b1b0 0x5580da24c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:14:20.105 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:20.098+0000 7f02fdffb640 10 client.15483 ms_handle_connect on v2:172.21.15.160:6838/2765554482 2024-05-20T16:14:20.105 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:20.098+0000 7f0315d9b640 1 --2- 172.21.15.82:0/1400975542 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x5580d9e2b1b0 0x5580da24c680 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e29b0109d259e3d2 server_cookie=73316ca396e3d6d in_seq=0 out_seq=0 2024-05-20T16:14:20.112 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:20.112+0000 7f02fdffb640 1 -- 172.21.15.82:0/1400975542 <== mds.0 v2:172.21.15.160:6838/2765554482 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5580da136dc0 con 0x5580d9e2b1b0 2024-05-20T16:14:20.112 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:20.112+0000 7f02fdffb640 10 client.15483 handle_command_reply: tid=0 2024-05-20T16:14:20.112 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:20.112+0000 7f0323a23740 1 client.15483 shutdown 2024-05-20T16:14:20.112 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:20.112+0000 7f0323a23740 1 -- 172.21.15.82:0/1400975542 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5580d9e022f0 msgr2=0x5580da1d7da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:14:20.113 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:20.112+0000 7f0323a23740 1 --2- 172.21.15.82:0/1400975542 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5580d9e022f0 0x5580da1d7da0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f03181023a0 tx=0x7f03080139f0 comp rx=0 tx=0).stop 2024-05-20T16:14:20.113 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:20.112+0000 7f0323a23740 1 -- 172.21.15.82:0/1400975542 shutdown_connections 2024-05-20T16:14:20.113 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:20.112+0000 7f0323a23740 1 --2- 172.21.15.82:0/1400975542 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x5580d9e2b1b0 0x5580da24c680 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:20.113 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:20.112+0000 7f0323a23740 1 --2- 172.21.15.82:0/1400975542 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5580d9e022f0 0x5580da1d7da0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:20.113 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:20.112+0000 7f0323a23740 1 --2- 172.21.15.82:0/1400975542 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5580d9e0a4d0 0x5580da1dace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:20.113 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:20.112+0000 7f02fdffb640 0 client.15483 ms_handle_reset on v2:172.21.15.160:6838/2765554482 2024-05-20T16:14:20.113 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:20.112+0000 7f0323a23740 1 --2- 172.21.15.82:0/1400975542 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5580d9de9d80 0x5580da1d7860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:20.113 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:20.112+0000 7f0323a23740 1 -- 172.21.15.82:0/1400975542 >> 172.21.15.82:0/1400975542 conn(0x5580d9e1f300 msgr2=0x5580da1c9690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:14:20.113 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:20.112+0000 7f0323a23740 1 -- 172.21.15.82:0/1400975542 shutdown_connections 2024-05-20T16:14:20.113 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:20.112+0000 7f0323a23740 1 -- 172.21.15.82:0/1400975542 wait complete. 2024-05-20T16:14:20.113 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:20.112+0000 7f0323a23740 20 client.15483 trim_cache size 0 max 16384 2024-05-20T16:14:20.114 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:{ 2024-05-20T16:14:20.114 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "message": "", 2024-05-20T16:14:20.114 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "return_code": 0 2024-05-20T16:14:20.114 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:} 2024-05-20T16:14:20.115 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:20.113+0000 7f031caa2640 1 -- 172.21.15.82:0/1290747199 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f03040a2c10 msgr2=0x7f03040a5120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:14:20.115 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:20.113+0000 7f031caa2640 1 --2- 172.21.15.82:0/1290747199 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f03040a2c10 0x7f03040a5120 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f031009b530 tx=0x7f0310086040 comp rx=0 tx=0).stop 2024-05-20T16:14:20.115 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:20.113+0000 7f031caa2640 1 -- 172.21.15.82:0/1290747199 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0318108a10 msgr2=0x7f03181a0780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:14:20.115 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:20.113+0000 7f031caa2640 1 --2- 172.21.15.82:0/1290747199 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0318108a10 0x7f03181a0780 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f031000c970 tx=0x7f031000ce30 comp rx=0 tx=0).stop 2024-05-20T16:14:20.115 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:20.113+0000 7f031caa2640 1 -- 172.21.15.82:0/1290747199 shutdown_connections 2024-05-20T16:14:20.115 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:20.113+0000 7f031caa2640 1 --2- 172.21.15.82:0/1290747199 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f03040a2c10 0x7f03040a5120 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:20.115 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:20.113+0000 7f031caa2640 1 --2- 172.21.15.82:0/1290747199 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0318108a10 0x7f03181a0780 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:20.115 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:20.113+0000 7f031caa2640 1 --2- 172.21.15.82:0/1290747199 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0318103d10 0x7f03181997f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:20.115 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:20.113+0000 7f031caa2640 1 --2- 172.21.15.82:0/1290747199 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0318102ad0 0x7f03181992b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:20.115 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:20.113+0000 7f031caa2640 1 -- 172.21.15.82:0/1290747199 >> 172.21.15.82:0/1290747199 conn(0x7f03180fdcc0 msgr2=0x7f03181049f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:14:20.116 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:20.113+0000 7f031caa2640 1 -- 172.21.15.82:0/1290747199 shutdown_connections 2024-05-20T16:14:20.116 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:20.113+0000 7f031caa2640 1 -- 172.21.15.82:0/1290747199 wait complete. 2024-05-20T16:14:21.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:14:21 smithi160 ceph-mon[36347]: pgmap v373: 129 pgs: 129 active+clean; 655 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 597 B/s wr, 6 op/s 2024-05-20T16:14:21.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:14:21 smithi082 ceph-mon[32238]: pgmap v373: 129 pgs: 129 active+clean; 655 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 597 B/s wr, 6 op/s 2024-05-20T16:14:21.885 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:14:21 smithi184 ceph-mon[36209]: pgmap v373: 129 pgs: 129 active+clean; 655 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 597 B/s wr, 6 op/s 2024-05-20T16:14:23.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:14:23 smithi160 ceph-mon[36347]: pgmap v374: 129 pgs: 129 active+clean; 656 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-05-20T16:14:23.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:14:23 smithi082 ceph-mon[32238]: pgmap v374: 129 pgs: 129 active+clean; 656 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-05-20T16:14:23.885 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:14:23 smithi184 ceph-mon[36209]: pgmap v374: 129 pgs: 129 active+clean; 656 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-05-20T16:14:24.682 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:14:24 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T16:14:24.703 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:14:24 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T16:14:24.743 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:14:24 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T16:14:25.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:14:25 smithi160 ceph-mon[36347]: pgmap v375: 129 pgs: 129 active+clean; 656 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.6 KiB/s wr, 10 op/s 2024-05-20T16:14:25.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:14:25 smithi082 ceph-mon[32238]: pgmap v375: 129 pgs: 129 active+clean; 656 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.6 KiB/s wr, 10 op/s 2024-05-20T16:14:25.885 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:14:25 smithi184 ceph-mon[36209]: pgmap v375: 129 pgs: 129 active+clean; 656 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.6 KiB/s wr, 10 op/s 2024-05-20T16:14:26.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:14:26 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:14:26.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:14:26 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:14:26.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:14:26 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:14:26.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:14:26 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:14:26.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:14:26 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:14:26.813 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:14:26 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:14:26.813 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:14:26 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:14:26.813 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:14:26 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T16:14:26.813 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:14:26 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:14:26.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:14:26 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:14:26.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:14:26 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:14:26.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:14:26 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:14:26.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:14:26 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:14:26.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:14:26 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:14:26.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:14:26 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:14:26.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:14:26 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:14:26.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:14:26 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T16:14:26.860 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:14:26 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:14:26.885 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:14:26 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:14:26.885 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:14:26 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:14:26.885 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:14:26 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:14:26.886 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:14:26 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:14:26.886 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:14:26 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:14:26.886 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:14:26 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:14:26.886 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:14:26 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:14:26.886 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:14:26 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T16:14:26.886 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:14:26 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:14:27.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:14:27 smithi160 ceph-mon[36347]: pgmap v376: 129 pgs: 129 active+clean; 656 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.6 KiB/s wr, 9 op/s 2024-05-20T16:14:27.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:14:27 smithi082 ceph-mon[32238]: pgmap v376: 129 pgs: 129 active+clean; 656 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.6 KiB/s wr, 9 op/s 2024-05-20T16:14:27.885 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:14:27 smithi184 ceph-mon[36209]: pgmap v376: 129 pgs: 129 active+clean; 656 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.6 KiB/s wr, 9 op/s 2024-05-20T16:14:29.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:14:29 smithi160 ceph-mon[36347]: pgmap v377: 129 pgs: 129 active+clean; 656 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.6 KiB/s wr, 12 op/s 2024-05-20T16:14:29.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:14:29 smithi082 ceph-mon[32238]: pgmap v377: 129 pgs: 129 active+clean; 656 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.6 KiB/s wr, 12 op/s 2024-05-20T16:14:29.885 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:14:29 smithi184 ceph-mon[36209]: pgmap v377: 129 pgs: 129 active+clean; 656 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.6 KiB/s wr, 12 op/s 2024-05-20T16:14:31.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:14:31 smithi160 ceph-mon[36347]: pgmap v378: 129 pgs: 129 active+clean; 656 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.4 KiB/s wr, 9 op/s 2024-05-20T16:14:31.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:14:31 smithi082 ceph-mon[32238]: pgmap v378: 129 pgs: 129 active+clean; 656 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.4 KiB/s wr, 9 op/s 2024-05-20T16:14:31.885 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:14:31 smithi184 ceph-mon[36209]: pgmap v378: 129 pgs: 129 active+clean; 656 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.4 KiB/s wr, 9 op/s 2024-05-20T16:14:33.224 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:33.222+0000 7fafe60f7640 1 -- 172.21.15.82:0/3749579631 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fafe0101f40 msgr2=0x7fafe0102350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:14:33.224 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:33.222+0000 7fafe60f7640 1 --2- 172.21.15.82:0/3749579631 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fafe0101f40 0x7fafe0102350 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fafd0009900 tx=0x7fafd002a410 comp rx=0 tx=0).stop 2024-05-20T16:14:33.224 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:33.222+0000 7fafe60f7640 1 -- 172.21.15.82:0/3749579631 shutdown_connections 2024-05-20T16:14:33.225 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:33.222+0000 7fafe60f7640 1 --2- 172.21.15.82:0/3749579631 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fafe0103460 0x7fafe0109ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:33.225 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:33.222+0000 7fafe60f7640 1 --2- 172.21.15.82:0/3749579631 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fafe0102920 0x7fafe0102dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:33.225 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:33.222+0000 7fafe60f7640 1 --2- 172.21.15.82:0/3749579631 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fafe0101f40 0x7fafe0102350 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:33.225 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:33.222+0000 7fafe60f7640 1 -- 172.21.15.82:0/3749579631 >> 172.21.15.82:0/3749579631 conn(0x7fafe00fc340 msgr2=0x7fafe00fe810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:14:33.225 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:33.222+0000 7fafe60f7640 1 -- 172.21.15.82:0/3749579631 shutdown_connections 2024-05-20T16:14:33.225 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:33.222+0000 7fafe60f7640 1 -- 172.21.15.82:0/3749579631 wait complete. 2024-05-20T16:14:33.225 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:33.223+0000 7fafe60f7640 1 Processor -- start 2024-05-20T16:14:33.225 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:33.223+0000 7fafe60f7640 1 -- start start 2024-05-20T16:14:33.225 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:33.223+0000 7fafe60f7640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fafe0101f40 0x7fafe01992e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:14:33.225 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:33.223+0000 7fafe60f7640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fafe0102920 0x7fafe0199820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:14:33.225 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:33.223+0000 7fafe60f7640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fafe0103460 0x7fafe01a07b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:14:33.225 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:33.223+0000 7fafe60f7640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fafe010d640 con 0x7fafe0101f40 2024-05-20T16:14:33.225 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:33.223+0000 7fafe60f7640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fafe010d4c0 con 0x7fafe0102920 2024-05-20T16:14:33.226 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:33.223+0000 7fafe60f7640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fafe010d7c0 con 0x7fafe0103460 2024-05-20T16:14:33.226 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:33.223+0000 7fafdf7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fafe0101f40 0x7fafe01992e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:14:33.226 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:33.223+0000 7fafdf7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fafe0101f40 0x7fafe01992e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:53624/0 (socket says 172.21.15.82:53624) 2024-05-20T16:14:33.226 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:33.223+0000 7fafdf7fe640 1 -- 172.21.15.82:0/3959120107 learned_addr learned my addr 172.21.15.82:0/3959120107 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:14:33.226 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:33.223+0000 7fafd7fff640 1 --2- 172.21.15.82:0/3959120107 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fafe0102920 0x7fafe0199820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:14:33.226 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:33.224+0000 7fafdffff640 1 --2- 172.21.15.82:0/3959120107 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fafe0103460 0x7fafe01a07b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:14:33.226 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:33.224+0000 7fafdf7fe640 1 -- 172.21.15.82:0/3959120107 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fafe0103460 msgr2=0x7fafe01a07b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:14:33.226 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:33.224+0000 7fafdf7fe640 1 --2- 172.21.15.82:0/3959120107 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fafe0103460 0x7fafe01a07b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:33.226 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:33.224+0000 7fafdf7fe640 1 -- 172.21.15.82:0/3959120107 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fafe0102920 msgr2=0x7fafe0199820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:14:33.226 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:33.224+0000 7fafdf7fe640 1 --2- 172.21.15.82:0/3959120107 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fafe0102920 0x7fafe0199820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:33.226 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:33.224+0000 7fafdf7fe640 1 -- 172.21.15.82:0/3959120107 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafe01a0e20 con 0x7fafe0101f40 2024-05-20T16:14:33.226 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:33.224+0000 7fafdf7fe640 1 --2- 172.21.15.82:0/3959120107 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fafe0101f40 0x7fafe01992e0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fafd0006010 tx=0x7fafd00043b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:14:33.226 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:33.224+0000 7fafdd7fa640 1 -- 172.21.15.82:0/3959120107 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafd003f030 con 0x7fafe0101f40 2024-05-20T16:14:33.227 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:33.224+0000 7fafe60f7640 1 -- 172.21.15.82:0/3959120107 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fafe01a10b0 con 0x7fafe0101f40 2024-05-20T16:14:33.227 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:33.224+0000 7fafdd7fa640 1 -- 172.21.15.82:0/3959120107 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fafd0003ce0 con 0x7fafe0101f40 2024-05-20T16:14:33.227 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:33.224+0000 7fafdd7fa640 1 -- 172.21.15.82:0/3959120107 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafd0035070 con 0x7fafe0101f40 2024-05-20T16:14:33.227 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:33.224+0000 7fafe60f7640 1 -- 172.21.15.82:0/3959120107 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fafe01a15a0 con 0x7fafe0101f40 2024-05-20T16:14:33.232 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:33.226+0000 7fafdd7fa640 1 -- 172.21.15.82:0/3959120107 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 19) ==== 137884+0+0 (secure 0 0 0) 0x7fafd0042030 con 0x7fafe0101f40 2024-05-20T16:14:33.232 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:33.227+0000 7fafecfa0740 1 Processor -- start 2024-05-20T16:14:33.233 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:33.227+0000 7fafecfa0740 1 -- start start 2024-05-20T16:14:33.233 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:33.227+0000 7fafdd7fa640 1 --2- 172.21.15.82:0/3959120107 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fafcc0a2c10 0x7fafcc0a5120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:14:33.233 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:33.227+0000 7fafecfa0740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55aee47491b0 0x55aee4aed270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:14:33.233 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:33.227+0000 7fafecfa0740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55aee47202f0 0x55aee4aed7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:14:33.233 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:33.227+0000 7fafecfa0740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55aee47284d0 0x55aee4af46f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:14:33.233 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:33.227+0000 7fafecfa0740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55aee4a4d340 con 0x55aee47202f0 2024-05-20T16:14:33.233 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:33.227+0000 7fafecfa0740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x55aee49f5290 con 0x55aee47491b0 2024-05-20T16:14:33.233 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:33.227+0000 7fafecfa0740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55aee4a49300 con 0x55aee47284d0 2024-05-20T16:14:33.233 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:33.227+0000 7fafdffff640 1 --2- 172.21.15.82:0/3959120107 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fafcc0a2c10 0x7fafcc0a5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:14:33.233 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:33.227+0000 7fafdd7fa640 1 -- 172.21.15.82:0/3959120107 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9150+0+0 (secure 0 0 0) 0x7fafd00efb70 con 0x7fafe0101f40 2024-05-20T16:14:33.233 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:33.227+0000 7fafd7fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55aee47202f0 0x55aee4aed7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:14:33.233 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:33.227+0000 7fafdf7fe640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55aee47491b0 0x55aee4aed270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:14:33.233 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:33.227+0000 7fafdf7fe640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55aee47491b0 0x55aee4aed270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.82:53242/0 (socket says 172.21.15.82:53242) 2024-05-20T16:14:33.234 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:33.227+0000 7fafd7fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55aee47202f0 0x55aee4aed7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:53630/0 (socket says 172.21.15.82:53630) 2024-05-20T16:14:33.234 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:33.227+0000 7fafdf7fe640 1 -- 172.21.15.82:0/1894963029 learned_addr learned my addr 172.21.15.82:0/1894963029 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:14:33.234 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:33.228+0000 7fafdffff640 1 --2- 172.21.15.82:0/1894963029 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55aee47284d0 0x55aee4af46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:14:33.234 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:33.228+0000 7fafd7fff640 1 -- 172.21.15.82:0/1894963029 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55aee47284d0 msgr2=0x55aee4af46f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:14:33.234 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:33.228+0000 7fafd7fff640 1 --2- 172.21.15.82:0/1894963029 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55aee47284d0 0x55aee4af46f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:33.234 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:33.228+0000 7fafd7fff640 1 -- 172.21.15.82:0/1894963029 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55aee47491b0 msgr2=0x55aee4aed270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:14:33.234 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:33.228+0000 7fafd7fff640 1 --2- 172.21.15.82:0/1894963029 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55aee47491b0 0x55aee4aed270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:33.234 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:33.228+0000 7fafd7fff640 1 -- 172.21.15.82:0/1894963029 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aee4a198e0 con 0x55aee47202f0 2024-05-20T16:14:33.234 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:33.228+0000 7fafdffff640 1 --2- 172.21.15.82:0/1894963029 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55aee47284d0 0x55aee4af46f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T16:14:33.234 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:33.228+0000 7fafdf7fe640 1 --2- 172.21.15.82:0/1894963029 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55aee47491b0 0x55aee4aed270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T16:14:33.234 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:33.228+0000 7fafdffff640 1 --2- 172.21.15.82:0/3959120107 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fafcc0a2c10 0x7fafcc0a5120 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fafd8005fd0 tx=0x7fafd80137d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:14:33.234 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:33.228+0000 7fafd7fff640 1 --2- 172.21.15.82:0/1894963029 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55aee47202f0 0x55aee4aed7f0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fafc800bbc0 tx=0x7fafc80107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=779de1215dd8fb20 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:14:33.234 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:33.228+0000 7fafd5ffb640 1 -- 172.21.15.82:0/1894963029 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafc8004030 con 0x55aee47202f0 2024-05-20T16:14:33.234 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:33.228+0000 7fafd5ffb640 1 -- 172.21.15.82:0/1894963029 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fafc80041d0 con 0x55aee47202f0 2024-05-20T16:14:33.235 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:33.228+0000 7fafd5ffb640 1 -- 172.21.15.82:0/1894963029 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafc80116a0 con 0x55aee47202f0 2024-05-20T16:14:33.235 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:33.229+0000 7fafecfa0740 1 -- 172.21.15.82:0/1894963029 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55aee47202f0 msgr2=0x55aee4aed7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:14:33.235 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:33.229+0000 7fafecfa0740 1 --2- 172.21.15.82:0/1894963029 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55aee47202f0 0x55aee4aed7f0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fafc800bbc0 tx=0x7fafc80107a0 comp rx=0 tx=0).stop 2024-05-20T16:14:33.235 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:33.229+0000 7fafecfa0740 1 -- 172.21.15.82:0/1894963029 shutdown_connections 2024-05-20T16:14:33.235 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:33.229+0000 7fafecfa0740 1 --2- 172.21.15.82:0/1894963029 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55aee47202f0 0x55aee4aed7f0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:33.235 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:33.229+0000 7fafecfa0740 1 --2- 172.21.15.82:0/1894963029 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55aee47284d0 0x55aee4af46f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:33.235 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:33.229+0000 7fafecfa0740 1 --2- 172.21.15.82:0/1894963029 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55aee47491b0 0x55aee4aed270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:33.235 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:33.229+0000 7fafecfa0740 1 -- 172.21.15.82:0/1894963029 >> 172.21.15.82:0/1894963029 conn(0x55aee473d300 msgr2=0x55aee4ae47f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:14:33.235 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:33.229+0000 7fafecfa0740 1 -- 172.21.15.82:0/1894963029 shutdown_connections 2024-05-20T16:14:33.235 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:33.229+0000 7fafecfa0740 1 -- 172.21.15.82:0/1894963029 wait complete. 2024-05-20T16:14:33.235 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:33.229+0000 7fafecfa0740 1 -- start start 2024-05-20T16:14:33.235 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:33.230+0000 7fafecfa0740 1 Processor -- start 2024-05-20T16:14:33.235 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:33.230+0000 7fafecfa0740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55aee47202f0 0x55aee4af6f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:14:33.236 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:33.230+0000 7fafecfa0740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55aee47284d0 0x55aee4af7480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:14:33.236 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:33.230+0000 7fafecfa0740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55aee47491b0 0x55aee4afe3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:14:33.236 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:33.230+0000 7fafecfa0740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55aee4af5140 con 0x55aee47284d0 2024-05-20T16:14:33.236 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:33.230+0000 7fafecfa0740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x55aee4a08790 con 0x55aee47202f0 2024-05-20T16:14:33.236 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:33.230+0000 7fafecfa0740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55aee4af52c0 con 0x55aee47491b0 2024-05-20T16:14:33.236 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:33.230+0000 7fafecfa0740 10 client.0 _ll_register_callbacks cb 0x55aee4a43a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T16:14:33.236 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:33.230+0000 7fafd7fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55aee47284d0 0x55aee4af7480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:14:33.236 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:33.230+0000 7fafd7fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55aee47284d0 0x55aee4af7480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:53638/0 (socket says 172.21.15.82:53638) 2024-05-20T16:14:33.236 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:33.230+0000 7fafd7fff640 1 -- 172.21.15.82:0/3214238820 learned_addr learned my addr 172.21.15.82:0/3214238820 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:14:33.236 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:33.230+0000 7fafdf7fe640 1 --2- 172.21.15.82:0/3214238820 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55aee47202f0 0x55aee4af6f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:14:33.236 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:33.230+0000 7fafdffff640 1 --2- 172.21.15.82:0/3214238820 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55aee47491b0 0x55aee4afe3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:14:33.236 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:33.230+0000 7fafd7fff640 1 -- 172.21.15.82:0/3214238820 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55aee47491b0 msgr2=0x55aee4afe3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:14:33.236 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:33.230+0000 7fafd7fff640 1 --2- 172.21.15.82:0/3214238820 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55aee47491b0 0x55aee4afe3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:33.236 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:33.230+0000 7fafd7fff640 1 -- 172.21.15.82:0/3214238820 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55aee47202f0 msgr2=0x55aee4af6f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:14:33.237 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:33.230+0000 7fafd7fff640 1 --2- 172.21.15.82:0/3214238820 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55aee47202f0 0x55aee4af6f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:33.237 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:33.230+0000 7fafd7fff640 1 -- 172.21.15.82:0/3214238820 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aee4761210 con 0x55aee47284d0 2024-05-20T16:14:33.237 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:33.231+0000 7fafd7fff640 1 --2- 172.21.15.82:0/3214238820 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55aee47284d0 0x55aee4af7480 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fafc8010ca0 tx=0x7fafc8004380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36197d38ae731845 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:14:33.237 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:33.231+0000 7fafbeffd640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-05-20T16:14:33.237 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:33.231+0000 7fafbeffd640 1 -- 172.21.15.82:0/3214238820 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafc802a060 con 0x55aee47284d0 2024-05-20T16:14:33.237 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:33.231+0000 7fafbeffd640 1 -- 172.21.15.82:0/3214238820 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9150+0+0 (secure 0 0 0) 0x7fafc80035f0 con 0x55aee47284d0 2024-05-20T16:14:33.237 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:33.231+0000 7fafecfa0740 1 -- 172.21.15.82:0/3214238820 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fafc80041d0 con 0x55aee47284d0 2024-05-20T16:14:33.237 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:33.231+0000 7fafbeffd640 1 -- 172.21.15.82:0/3214238820 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fafc8031050 con 0x55aee47284d0 2024-05-20T16:14:33.237 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:33.231+0000 7fafecfa0740 10 client.15495 fetch_fsmap learned FSMap version 25 2024-05-20T16:14:33.237 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:33.231+0000 7fafecfa0740 1 -- 172.21.15.82:0/3214238820 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55aee482d620 con 0x55aee47284d0 2024-05-20T16:14:33.237 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:33.231+0000 7fafbeffd640 1 -- 172.21.15.82:0/3214238820 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fafc80041d0 con 0x55aee47284d0 2024-05-20T16:14:33.237 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:33.231+0000 7fafbeffd640 1 -- 172.21.15.82:0/3214238820 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fafc80047e0 con 0x55aee47284d0 2024-05-20T16:14:33.237 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:33.231+0000 7fafecfa0740 10 client.15495 fetch_fsmap finished waiting for FSMap version 25 2024-05-20T16:14:33.237 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:33.231+0000 7fafecfa0740 10 client.15495 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T16:14:33.238 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:33.231+0000 7fafecfa0740 20 client.15495 populate_metadata read hostname 'smithi082' 2024-05-20T16:14:33.238 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:33.231+0000 7fafecfa0740 1 --2- 172.21.15.82:0/3214238820 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x55aee4707d80 0x55aee4b044d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:14:33.238 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:33.231+0000 7fafecfa0740 4 client.15495 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T16:14:33.238 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:33.231+0000 7fafecfa0740 1 -- 172.21.15.82:0/3214238820 --> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55aee4a5aa70 con 0x55aee4707d80 2024-05-20T16:14:33.238 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:33.232+0000 7fafdf7fe640 1 --2- 172.21.15.82:0/3214238820 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x55aee4707d80 0x55aee4b044d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:14:33.238 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:33.232+0000 7fafbeffd640 10 client.15495 ms_handle_connect on v2:172.21.15.160:6838/2765554482 2024-05-20T16:14:33.238 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:33.232+0000 7fafdf7fe640 1 --2- 172.21.15.82:0/3214238820 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x55aee4707d80 0x55aee4b044d0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3e098d46520eacd0 server_cookie=6dd0857e72980af4 in_seq=0 out_seq=0 2024-05-20T16:14:33.238 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:33.233+0000 7fafbeffd640 1 -- 172.21.15.82:0/3214238820 <== mds.0 v2:172.21.15.160:6838/2765554482 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55aee4a5aa70 con 0x55aee4707d80 2024-05-20T16:14:33.238 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:33.233+0000 7fafbeffd640 10 client.15495 handle_command_reply: tid=0 2024-05-20T16:14:33.238 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:33.233+0000 7fafecfa0740 1 client.15495 shutdown 2024-05-20T16:14:33.239 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:33.233+0000 7fafecfa0740 1 -- 172.21.15.82:0/3214238820 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55aee47284d0 msgr2=0x55aee4af7480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:14:33.239 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:33.233+0000 7fafecfa0740 1 --2- 172.21.15.82:0/3214238820 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55aee47284d0 0x55aee4af7480 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fafc8010ca0 tx=0x7fafc8004380 comp rx=0 tx=0).stop 2024-05-20T16:14:33.239 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:33.233+0000 7fafecfa0740 1 -- 172.21.15.82:0/3214238820 shutdown_connections 2024-05-20T16:14:33.239 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:33.233+0000 7fafecfa0740 1 --2- 172.21.15.82:0/3214238820 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x55aee4707d80 0x55aee4b044d0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:33.239 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:33.233+0000 7fafecfa0740 1 --2- 172.21.15.82:0/3214238820 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55aee47284d0 0x55aee4af7480 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:33.239 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:33.233+0000 7fafecfa0740 1 --2- 172.21.15.82:0/3214238820 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55aee47491b0 0x55aee4afe3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:33.239 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:33.233+0000 7fafbeffd640 0 client.15495 ms_handle_reset on v2:172.21.15.160:6838/2765554482 2024-05-20T16:14:33.240 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:33.233+0000 7fafecfa0740 1 --2- 172.21.15.82:0/3214238820 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55aee47202f0 0x55aee4af6f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:33.240 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:33.233+0000 7fafecfa0740 1 -- 172.21.15.82:0/3214238820 >> 172.21.15.82:0/3214238820 conn(0x55aee473d300 msgr2=0x55aee4ae77a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:14:33.240 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:33.233+0000 7fafecfa0740 1 -- 172.21.15.82:0/3214238820 shutdown_connections 2024-05-20T16:14:33.240 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:33.233+0000 7fafecfa0740 1 -- 172.21.15.82:0/3214238820 wait complete. 2024-05-20T16:14:33.240 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:33.234+0000 7fafecfa0740 20 client.15495 trim_cache size 0 max 16384 2024-05-20T16:14:33.249 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:33.248+0000 7fafecfa0740 1 Processor -- start 2024-05-20T16:14:33.249 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:33.248+0000 7fafecfa0740 1 -- start start 2024-05-20T16:14:33.249 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:33.249+0000 7fafecfa0740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55aee4707d80 0x55aee4ae4880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:14:33.249 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:33.249+0000 7fafecfa0740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55aee47202f0 0x55aee4afe300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:14:33.249 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:33.249+0000 7fafecfa0740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55aee47284d0 0x55aee4afe840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:14:33.249 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:33.249+0000 7fafecfa0740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fafe01ac0a0 con 0x55aee4707d80 2024-05-20T16:14:33.250 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:33.249+0000 7fafecfa0740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fafe01a9970 con 0x55aee47202f0 2024-05-20T16:14:33.250 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:33.249+0000 7fafecfa0740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fafe01a77c0 con 0x55aee47284d0 2024-05-20T16:14:33.250 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:33.249+0000 7fafdf7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55aee4707d80 0x55aee4ae4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:14:33.250 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:33.249+0000 7fafdf7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55aee4707d80 0x55aee4ae4880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:53652/0 (socket says 172.21.15.82:53652) 2024-05-20T16:14:33.250 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:33.249+0000 7fafdf7fe640 1 -- 172.21.15.82:0/4050637065 learned_addr learned my addr 172.21.15.82:0/4050637065 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:14:33.250 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:33.249+0000 7fafd7fff640 1 --2- 172.21.15.82:0/4050637065 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55aee47202f0 0x55aee4afe300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:14:33.250 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:33.249+0000 7fafdffff640 1 --2- 172.21.15.82:0/4050637065 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55aee47284d0 0x55aee4afe840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:14:33.250 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:33.249+0000 7fafdf7fe640 1 -- 172.21.15.82:0/4050637065 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55aee47284d0 msgr2=0x55aee4afe840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:14:33.250 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:33.249+0000 7fafdf7fe640 1 --2- 172.21.15.82:0/4050637065 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55aee47284d0 0x55aee4afe840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:33.250 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:33.249+0000 7fafdf7fe640 1 -- 172.21.15.82:0/4050637065 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55aee47202f0 msgr2=0x55aee4afe300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:14:33.250 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:33.249+0000 7fafdf7fe640 1 --2- 172.21.15.82:0/4050637065 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55aee47202f0 0x55aee4afe300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:33.250 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:33.249+0000 7fafdf7fe640 1 -- 172.21.15.82:0/4050637065 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aee4aedc40 con 0x55aee4707d80 2024-05-20T16:14:33.250 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:33.249+0000 7fafdffff640 1 --2- 172.21.15.82:0/4050637065 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55aee47284d0 0x55aee4afe840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T16:14:33.251 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:33.249+0000 7fafdf7fe640 1 --2- 172.21.15.82:0/4050637065 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55aee4707d80 0x55aee4ae4880 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fafd0000c00 tx=0x7fafd003b800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73849d4840d34f32 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:14:33.251 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:33.249+0000 7fafd57fa640 1 -- 172.21.15.82:0/4050637065 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafd0049040 con 0x55aee4707d80 2024-05-20T16:14:33.251 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:33.249+0000 7fafd57fa640 1 -- 172.21.15.82:0/4050637065 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fafd00c0290 con 0x55aee4707d80 2024-05-20T16:14:33.251 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:33.249+0000 7fafd57fa640 1 -- 172.21.15.82:0/4050637065 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafd00b8550 con 0x55aee4707d80 2024-05-20T16:14:33.254 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:33.250+0000 7fafecfa0740 1 -- 172.21.15.82:0/4050637065 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55aee4707d80 msgr2=0x55aee4ae4880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:14:33.254 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:33.250+0000 7fafecfa0740 1 --2- 172.21.15.82:0/4050637065 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55aee4707d80 0x55aee4ae4880 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fafd0000c00 tx=0x7fafd003b800 comp rx=0 tx=0).stop 2024-05-20T16:14:33.254 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:33.250+0000 7fafecfa0740 1 -- 172.21.15.82:0/4050637065 shutdown_connections 2024-05-20T16:14:33.254 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:33.250+0000 7fafecfa0740 1 --2- 172.21.15.82:0/4050637065 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55aee47284d0 0x55aee4afe840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:33.254 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:33.250+0000 7fafecfa0740 1 --2- 172.21.15.82:0/4050637065 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55aee47202f0 0x55aee4afe300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:33.255 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:33.250+0000 7fafecfa0740 1 --2- 172.21.15.82:0/4050637065 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55aee4707d80 0x55aee4ae4880 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:33.255 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:33.250+0000 7fafecfa0740 1 -- 172.21.15.82:0/4050637065 >> 172.21.15.82:0/4050637065 conn(0x55aee473d300 msgr2=0x55aee4ae4170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:14:33.255 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:33.250+0000 7fafecfa0740 1 -- 172.21.15.82:0/4050637065 shutdown_connections 2024-05-20T16:14:33.255 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:33.250+0000 7fafecfa0740 1 -- 172.21.15.82:0/4050637065 wait complete. 2024-05-20T16:14:33.255 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:33.250+0000 7fafecfa0740 1 -- start start 2024-05-20T16:14:33.255 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:33.251+0000 7fafecfa0740 1 Processor -- start 2024-05-20T16:14:33.255 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:33.251+0000 7fafecfa0740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55aee4707d80 0x55aee4afc200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:14:33.255 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:33.251+0000 7fafecfa0740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55aee47202f0 0x55aee4afc740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:14:33.255 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:33.251+0000 7fafecfa0740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55aee47284d0 0x55aee4afd600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:14:33.255 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:33.251+0000 7fafecfa0740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fafe01a7e30 con 0x55aee47202f0 2024-05-20T16:14:33.255 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:33.251+0000 7fafecfa0740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fafe01a92a0 con 0x55aee47284d0 2024-05-20T16:14:33.255 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:33.251+0000 7fafecfa0740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fafe01a8500 con 0x55aee4707d80 2024-05-20T16:14:33.255 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:33.251+0000 7fafecfa0740 10 client.0 _ll_register_callbacks cb 0x7fafd0005e50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T16:14:33.255 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:33.251+0000 7fafd7fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55aee47202f0 0x55aee4afc740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:14:33.255 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:33.251+0000 7fafd7fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55aee47202f0 0x55aee4afc740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:53658/0 (socket says 172.21.15.82:53658) 2024-05-20T16:14:33.256 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:33.251+0000 7fafd7fff640 1 -- 172.21.15.82:0/2754813951 learned_addr learned my addr 172.21.15.82:0/2754813951 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:14:33.256 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:33.251+0000 7fafdffff640 1 --2- 172.21.15.82:0/2754813951 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55aee47284d0 0x55aee4afd600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:14:33.256 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:33.251+0000 7fafdf7fe640 1 --2- 172.21.15.82:0/2754813951 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55aee4707d80 0x55aee4afc200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:14:33.256 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:33.251+0000 7fafd7fff640 1 -- 172.21.15.82:0/2754813951 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55aee4707d80 msgr2=0x55aee4afc200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:14:33.256 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:33.251+0000 7fafd7fff640 1 --2- 172.21.15.82:0/2754813951 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55aee4707d80 0x55aee4afc200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:33.256 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:33.251+0000 7fafd7fff640 1 -- 172.21.15.82:0/2754813951 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55aee47284d0 msgr2=0x55aee4afd600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:14:33.256 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:33.251+0000 7fafd7fff640 1 --2- 172.21.15.82:0/2754813951 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55aee47284d0 0x55aee4afd600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:33.256 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:33.251+0000 7fafd7fff640 1 -- 172.21.15.82:0/2754813951 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aee4afdbc0 con 0x55aee47202f0 2024-05-20T16:14:33.256 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:33.252+0000 7fafd7fff640 1 --2- 172.21.15.82:0/2754813951 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55aee47202f0 0x55aee4afc740 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fafc800b3d0 tx=0x7fafc800bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbc1b2580443f6ee server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:14:33.257 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:33.252+0000 7fafbeffd640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-05-20T16:14:33.257 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:33.252+0000 7fafbeffd640 1 -- 172.21.15.82:0/2754813951 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafc802a050 con 0x55aee47202f0 2024-05-20T16:14:33.257 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:33.252+0000 7fafbeffd640 1 -- 172.21.15.82:0/2754813951 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9150+0+0 (secure 0 0 0) 0x7fafc8003be0 con 0x55aee47202f0 2024-05-20T16:14:33.257 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:33.252+0000 7fafecfa0740 1 -- 172.21.15.82:0/2754813951 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fafd00c0290 con 0x55aee47202f0 2024-05-20T16:14:33.257 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:33.252+0000 7fafbeffd640 1 -- 172.21.15.82:0/2754813951 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fafc800ba40 con 0x55aee47202f0 2024-05-20T16:14:33.257 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:33.252+0000 7fafecfa0740 10 client.15501 fetch_fsmap learned FSMap version 25 2024-05-20T16:14:33.257 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:33.252+0000 7fafecfa0740 1 -- 172.21.15.82:0/2754813951 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55aee4afeec0 con 0x55aee47202f0 2024-05-20T16:14:33.257 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:33.252+0000 7fafbeffd640 1 -- 172.21.15.82:0/2754813951 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fafd00c0290 con 0x55aee47202f0 2024-05-20T16:14:33.257 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:33.252+0000 7fafbeffd640 1 -- 172.21.15.82:0/2754813951 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fafc8004ed0 con 0x55aee47202f0 2024-05-20T16:14:33.257 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:33.252+0000 7fafecfa0740 10 client.15501 fetch_fsmap finished waiting for FSMap version 25 2024-05-20T16:14:33.257 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:33.252+0000 7fafecfa0740 10 client.15501 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T16:14:33.257 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:33.252+0000 7fafecfa0740 20 client.15501 populate_metadata read hostname 'smithi082' 2024-05-20T16:14:33.257 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:33.252+0000 7fafecfa0740 1 --2- 172.21.15.82:0/2754813951 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x55aee47491b0 0x55aee4b6f030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:14:33.257 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:33.252+0000 7fafecfa0740 4 client.15501 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-20T16:14:33.258 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:33.252+0000 7fafecfa0740 1 -- 172.21.15.82:0/2754813951 --> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55aee4a54dc0 con 0x55aee47491b0 2024-05-20T16:14:33.258 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:33.253+0000 7fafdf7fe640 1 --2- 172.21.15.82:0/2754813951 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x55aee47491b0 0x55aee4b6f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:14:33.258 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:33.253+0000 7fafbeffd640 10 client.15501 ms_handle_connect on v2:172.21.15.160:6838/2765554482 2024-05-20T16:14:33.258 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:33.253+0000 7fafdf7fe640 1 --2- 172.21.15.82:0/2754813951 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x55aee47491b0 0x55aee4b6f030 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bb9e81595804fe03 server_cookie=51a4cee0ef824a62 in_seq=0 out_seq=0 2024-05-20T16:14:33.259 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:33.257+0000 7fafbeffd640 1 -- 172.21.15.82:0/2754813951 <== mds.0 v2:172.21.15.160:6838/2765554482 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55aee4a54dc0 con 0x55aee47491b0 2024-05-20T16:14:33.259 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:33.257+0000 7fafbeffd640 10 client.15501 handle_command_reply: tid=0 2024-05-20T16:14:33.259 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:33.257+0000 7fafecfa0740 1 client.15501 shutdown 2024-05-20T16:14:33.259 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:33.257+0000 7fafecfa0740 1 -- 172.21.15.82:0/2754813951 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55aee47202f0 msgr2=0x55aee4afc740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:14:33.259 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:33.257+0000 7fafecfa0740 1 --2- 172.21.15.82:0/2754813951 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55aee47202f0 0x55aee4afc740 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fafc800b3d0 tx=0x7fafc800bf60 comp rx=0 tx=0).stop 2024-05-20T16:14:33.259 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:33.257+0000 7fafecfa0740 1 -- 172.21.15.82:0/2754813951 shutdown_connections 2024-05-20T16:14:33.259 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:33.257+0000 7fafecfa0740 1 --2- 172.21.15.82:0/2754813951 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x55aee47491b0 0x55aee4b6f030 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:33.259 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:33.257+0000 7fafecfa0740 1 --2- 172.21.15.82:0/2754813951 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55aee47202f0 0x55aee4afc740 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:33.260 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:33.257+0000 7fafbeffd640 0 client.15501 ms_handle_reset on v2:172.21.15.160:6838/2765554482 2024-05-20T16:14:33.260 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:33.257+0000 7fafecfa0740 1 --2- 172.21.15.82:0/2754813951 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55aee47284d0 0x55aee4afd600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:33.260 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:33.257+0000 7fafecfa0740 1 --2- 172.21.15.82:0/2754813951 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55aee4707d80 0x55aee4afc200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:33.260 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:33.257+0000 7fafecfa0740 1 -- 172.21.15.82:0/2754813951 >> 172.21.15.82:0/2754813951 conn(0x55aee473d300 msgr2=0x55aee4b08c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:14:33.260 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:33.257+0000 7fafecfa0740 1 -- 172.21.15.82:0/2754813951 shutdown_connections 2024-05-20T16:14:33.260 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:33.257+0000 7fafecfa0740 1 -- 172.21.15.82:0/2754813951 wait complete. 2024-05-20T16:14:33.260 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:33.257+0000 7fafecfa0740 20 client.15501 trim_cache size 0 max 16384 2024-05-20T16:14:33.260 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:{ 2024-05-20T16:14:33.260 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "message": "", 2024-05-20T16:14:33.260 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "return_code": 0 2024-05-20T16:14:33.260 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:} 2024-05-20T16:14:33.261 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:33.258+0000 7fafe60f7640 1 -- 172.21.15.82:0/3959120107 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fafcc0a2c10 msgr2=0x7fafcc0a5120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:14:33.261 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:33.258+0000 7fafe60f7640 1 --2- 172.21.15.82:0/3959120107 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fafcc0a2c10 0x7fafcc0a5120 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fafd8005fd0 tx=0x7fafd80137d0 comp rx=0 tx=0).stop 2024-05-20T16:14:33.261 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:33.258+0000 7fafe60f7640 1 -- 172.21.15.82:0/3959120107 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fafe0101f40 msgr2=0x7fafe01992e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:14:33.261 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:33.258+0000 7fafe60f7640 1 --2- 172.21.15.82:0/3959120107 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fafe0101f40 0x7fafe01992e0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fafd0006010 tx=0x7fafd00043b0 comp rx=0 tx=0).stop 2024-05-20T16:14:33.261 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:33.258+0000 7fafe60f7640 1 -- 172.21.15.82:0/3959120107 shutdown_connections 2024-05-20T16:14:33.262 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:33.258+0000 7fafe60f7640 1 --2- 172.21.15.82:0/3959120107 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fafcc0a2c10 0x7fafcc0a5120 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:33.262 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:33.258+0000 7fafe60f7640 1 --2- 172.21.15.82:0/3959120107 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fafe0103460 0x7fafe01a07b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:33.262 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:33.258+0000 7fafe60f7640 1 --2- 172.21.15.82:0/3959120107 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fafe0102920 0x7fafe0199820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:33.262 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:33.258+0000 7fafe60f7640 1 --2- 172.21.15.82:0/3959120107 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fafe0101f40 0x7fafe01992e0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:33.262 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:33.258+0000 7fafe60f7640 1 -- 172.21.15.82:0/3959120107 >> 172.21.15.82:0/3959120107 conn(0x7fafe00fc340 msgr2=0x7fafe0108da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:14:33.262 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:33.258+0000 7fafe60f7640 1 -- 172.21.15.82:0/3959120107 shutdown_connections 2024-05-20T16:14:33.262 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:33.258+0000 7fafe60f7640 1 -- 172.21.15.82:0/3959120107 wait complete. 2024-05-20T16:14:33.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:14:33 smithi160 ceph-mon[36347]: pgmap v379: 129 pgs: 129 active+clean; 656 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-05-20T16:14:33.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:14:33 smithi082 ceph-mon[32238]: pgmap v379: 129 pgs: 129 active+clean; 656 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-05-20T16:14:33.885 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:14:33 smithi184 ceph-mon[36209]: pgmap v379: 129 pgs: 129 active+clean; 656 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-05-20T16:14:35.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:14:35 smithi160 ceph-mon[36347]: pgmap v380: 129 pgs: 129 active+clean; 656 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 426 B/s wr, 7 op/s 2024-05-20T16:14:35.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:14:35 smithi082 ceph-mon[32238]: pgmap v380: 129 pgs: 129 active+clean; 656 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 426 B/s wr, 7 op/s 2024-05-20T16:14:35.885 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:14:35 smithi184 ceph-mon[36209]: pgmap v380: 129 pgs: 129 active+clean; 656 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 426 B/s wr, 7 op/s 2024-05-20T16:14:37.635 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:14:37 smithi184 ceph-mon[36209]: pgmap v381: 129 pgs: 129 active+clean; 656 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-05-20T16:14:37.814 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:14:37 smithi160 ceph-mon[36347]: pgmap v381: 129 pgs: 129 active+clean; 656 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-05-20T16:14:37.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:14:37 smithi082 ceph-mon[32238]: pgmap v381: 129 pgs: 129 active+clean; 656 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-05-20T16:14:39.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:14:39 smithi160 ceph-mon[36347]: pgmap v382: 129 pgs: 129 active+clean; 657 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-20T16:14:39.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:14:39 smithi082 ceph-mon[32238]: pgmap v382: 129 pgs: 129 active+clean; 657 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-20T16:14:39.885 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:14:39 smithi184 ceph-mon[36209]: pgmap v382: 129 pgs: 129 active+clean; 657 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-20T16:14:41.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:14:41 smithi160 ceph-mon[36347]: pgmap v383: 129 pgs: 129 active+clean; 657 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-20T16:14:41.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:14:41 smithi082 ceph-mon[32238]: pgmap v383: 129 pgs: 129 active+clean; 657 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-20T16:14:41.885 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:14:41 smithi184 ceph-mon[36209]: pgmap v383: 129 pgs: 129 active+clean; 657 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-20T16:14:43.661 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:14:43 smithi082 ceph-mon[32238]: pgmap v384: 129 pgs: 129 active+clean; 657 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-20T16:14:43.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:14:43 smithi160 ceph-mon[36347]: pgmap v384: 129 pgs: 129 active+clean; 657 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-20T16:14:43.885 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:14:43 smithi184 ceph-mon[36209]: pgmap v384: 129 pgs: 129 active+clean; 657 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-20T16:14:45.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:14:45 smithi160 ceph-mon[36347]: pgmap v385: 129 pgs: 129 active+clean; 657 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-20T16:14:45.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:14:45 smithi082 ceph-mon[32238]: pgmap v385: 129 pgs: 129 active+clean; 657 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-20T16:14:45.885 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:14:45 smithi184 ceph-mon[36209]: pgmap v385: 129 pgs: 129 active+clean; 657 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-20T16:14:46.368 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:46.368+0000 7f6a5cdc3640 1 -- 172.21.15.82:0/4070148752 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6a58102ae0 msgr2=0x7f6a581037e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:14:46.368 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:46.368+0000 7f6a5cdc3640 1 --2- 172.21.15.82:0/4070148752 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6a58102ae0 0x7f6a581037e0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f6a48009900 tx=0x7f6a4802a410 comp rx=0 tx=0).stop 2024-05-20T16:14:46.369 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:46.368+0000 7f6a5cdc3640 1 -- 172.21.15.82:0/4070148752 shutdown_connections 2024-05-20T16:14:46.369 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:46.368+0000 7f6a5cdc3640 1 --2- 172.21.15.82:0/4070148752 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6a58108a20 0x7f6a5810ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:46.369 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:46.368+0000 7f6a5cdc3640 1 --2- 172.21.15.82:0/4070148752 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6a58103d20 0x7f6a58104130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:46.369 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:46.368+0000 7f6a5cdc3640 1 --2- 172.21.15.82:0/4070148752 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6a58102ae0 0x7f6a581037e0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:46.369 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:46.368+0000 7f6a5cdc3640 1 -- 172.21.15.82:0/4070148752 >> 172.21.15.82:0/4070148752 conn(0x7f6a580fdcf0 msgr2=0x7f6a58100180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:14:46.369 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:46.368+0000 7f6a5cdc3640 1 -- 172.21.15.82:0/4070148752 shutdown_connections 2024-05-20T16:14:46.369 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:46.368+0000 7f6a5cdc3640 1 -- 172.21.15.82:0/4070148752 wait complete. 2024-05-20T16:14:46.371 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:46.369+0000 7f6a5cdc3640 1 Processor -- start 2024-05-20T16:14:46.372 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:46.369+0000 7f6a5cdc3640 1 -- start start 2024-05-20T16:14:46.372 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:46.369+0000 7f6a5cdc3640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6a58102ae0 0x7f6a581992f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:14:46.372 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:46.369+0000 7f6a5cdc3640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6a58103d20 0x7f6a58199830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:14:46.372 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:46.369+0000 7f6a5cdc3640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6a58108a20 0x7f6a581a07c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:14:46.372 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:46.369+0000 7f6a5cdc3640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f6a5810e780 con 0x7f6a58108a20 2024-05-20T16:14:46.372 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:46.369+0000 7f6a5cdc3640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f6a5810e600 con 0x7f6a58102ae0 2024-05-20T16:14:46.372 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:46.369+0000 7f6a5cdc3640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f6a5810e900 con 0x7f6a58103d20 2024-05-20T16:14:46.372 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:46.369+0000 7f6a56d9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6a58108a20 0x7f6a581a07c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:14:46.372 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:46.370+0000 7f6a56d9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6a58108a20 0x7f6a581a07c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:57290/0 (socket says 172.21.15.82:57290) 2024-05-20T16:14:46.372 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:46.370+0000 7f6a56d9d640 1 -- 172.21.15.82:0/149053672 learned_addr learned my addr 172.21.15.82:0/149053672 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:14:46.372 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:46.370+0000 7f6a4ffff640 1 --2- 172.21.15.82:0/149053672 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6a58103d20 0x7f6a58199830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:14:46.373 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:46.370+0000 7f6a5659c640 1 --2- 172.21.15.82:0/149053672 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6a58102ae0 0x7f6a581992f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:14:46.373 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:46.370+0000 7f6a56d9d640 1 -- 172.21.15.82:0/149053672 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6a58103d20 msgr2=0x7f6a58199830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:14:46.373 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:46.370+0000 7f6a56d9d640 1 --2- 172.21.15.82:0/149053672 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6a58103d20 0x7f6a58199830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:46.373 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:46.370+0000 7f6a56d9d640 1 -- 172.21.15.82:0/149053672 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6a58102ae0 msgr2=0x7f6a581992f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:14:46.373 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:46.370+0000 7f6a56d9d640 1 --2- 172.21.15.82:0/149053672 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6a58102ae0 0x7f6a581992f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:46.373 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:46.370+0000 7f6a56d9d640 1 -- 172.21.15.82:0/149053672 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a581a0e30 con 0x7f6a58108a20 2024-05-20T16:14:46.373 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:46.370+0000 7f6a5659c640 1 --2- 172.21.15.82:0/149053672 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6a58102ae0 0x7f6a581992f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T16:14:46.373 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:46.370+0000 7f6a56d9d640 1 --2- 172.21.15.82:0/149053672 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6a58108a20 0x7f6a581a07c0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f6a5000cad0 tx=0x7f6a5000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:14:46.373 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:46.370+0000 7f6a4f7fe640 1 -- 172.21.15.82:0/149053672 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a5000ae20 con 0x7f6a58108a20 2024-05-20T16:14:46.373 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:46.370+0000 7f6a4f7fe640 1 -- 172.21.15.82:0/149053672 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a50005020 con 0x7f6a58108a20 2024-05-20T16:14:46.373 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:46.370+0000 7f6a5cdc3640 1 -- 172.21.15.82:0/149053672 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a581a1120 con 0x7f6a58108a20 2024-05-20T16:14:46.373 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:46.370+0000 7f6a4f7fe640 1 -- 172.21.15.82:0/149053672 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a5000f6f0 con 0x7f6a58108a20 2024-05-20T16:14:46.373 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:46.370+0000 7f6a5cdc3640 1 -- 172.21.15.82:0/149053672 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a581a1620 con 0x7f6a58108a20 2024-05-20T16:14:46.380 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:46.373+0000 7f6a4f7fe640 1 -- 172.21.15.82:0/149053672 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 19) ==== 137884+0+0 (secure 0 0 0) 0x7f6a5000f8f0 con 0x7f6a58108a20 2024-05-20T16:14:46.380 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:46.373+0000 7f6a63c6b740 1 Processor -- start 2024-05-20T16:14:46.380 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:46.373+0000 7f6a63c6b740 1 -- start start 2024-05-20T16:14:46.380 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:46.373+0000 7f6a63c6b740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56128bdd31b0 0x56128c172e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:14:46.380 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:46.373+0000 7f6a63c6b740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56128bdaa2f0 0x56128c1733e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:14:46.380 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:46.373+0000 7f6a63c6b740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56128bdb24d0 0x56128c17a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:14:46.381 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:46.373+0000 7f6a63c6b740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x56128c0d7340 con 0x56128bdaa2f0 2024-05-20T16:14:46.381 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:46.373+0000 7f6a63c6b740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x56128c07f290 con 0x56128bdb24d0 2024-05-20T16:14:46.381 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:46.373+0000 7f6a63c6b740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x56128c0d3300 con 0x56128bdd31b0 2024-05-20T16:14:46.381 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:46.373+0000 7f6a5659c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56128bdaa2f0 0x56128c1733e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:14:46.381 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:46.374+0000 7f6a5659c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56128bdaa2f0 0x56128c1733e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:57296/0 (socket says 172.21.15.82:57296) 2024-05-20T16:14:46.381 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:46.374+0000 7f6a5659c640 1 -- 172.21.15.82:0/981725767 learned_addr learned my addr 172.21.15.82:0/981725767 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:14:46.381 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:46.374+0000 7f6a4ffff640 1 --2- 172.21.15.82:0/981725767 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56128bdd31b0 0x56128c172e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:14:46.381 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:46.374+0000 7f6a4f7fe640 1 --2- 172.21.15.82:0/149053672 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f6a440a2c10 0x7f6a440a5120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:14:46.381 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:46.374+0000 7f6a4ffff640 1 --2- 172.21.15.82:0/981725767 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56128bdb24d0 0x56128c17a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:14:46.381 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:46.374+0000 7f6a5659c640 1 -- 172.21.15.82:0/981725767 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56128bdd31b0 msgr2=0x56128c172e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:14:46.382 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:46.374+0000 7f6a5659c640 1 --2- 172.21.15.82:0/981725767 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56128bdd31b0 0x56128c172e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:46.382 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:46.374+0000 7f6a5659c640 1 -- 172.21.15.82:0/981725767 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56128bdb24d0 msgr2=0x56128c17a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:14:46.382 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:46.374+0000 7f6a5659c640 1 --2- 172.21.15.82:0/981725767 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56128bdb24d0 0x56128c17a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:46.382 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:46.374+0000 7f6a5659c640 1 -- 172.21.15.82:0/981725767 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56128c0a38e0 con 0x56128bdaa2f0 2024-05-20T16:14:46.382 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:46.374+0000 7f6a4ffff640 1 --2- 172.21.15.82:0/981725767 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56128bdb24d0 0x56128c17a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T16:14:46.382 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:46.374+0000 7f6a56d9d640 1 --2- 172.21.15.82:0/149053672 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f6a440a2c10 0x7f6a440a5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:14:46.382 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:46.374+0000 7f6a4f7fe640 1 -- 172.21.15.82:0/149053672 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9150+0+0 (secure 0 0 0) 0x7f6a500cf5f0 con 0x7f6a58108a20 2024-05-20T16:14:46.382 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:46.374+0000 7f6a5659c640 1 --2- 172.21.15.82:0/981725767 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56128bdaa2f0 0x56128c1733e0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f6a48006010 tx=0x7f6a48004200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3316c563eb97f29c server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:14:46.382 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:46.374+0000 7f6a4cff9640 1 -- 172.21.15.82:0/981725767 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a4803f070 con 0x56128bdaa2f0 2024-05-20T16:14:46.382 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:46.374+0000 7f6a56d9d640 1 --2- 172.21.15.82:0/149053672 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f6a440a2c10 0x7f6a440a5120 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f6a5009b660 tx=0x7f6a50096380 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:14:46.382 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:46.374+0000 7f6a4cff9640 1 -- 172.21.15.82:0/981725767 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a480044b0 con 0x56128bdaa2f0 2024-05-20T16:14:46.382 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:46.374+0000 7f6a4cff9640 1 -- 172.21.15.82:0/981725767 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a48035030 con 0x56128bdaa2f0 2024-05-20T16:14:46.382 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:46.375+0000 7f6a63c6b740 1 -- 172.21.15.82:0/981725767 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56128bdaa2f0 msgr2=0x56128c1733e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:14:46.383 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:46.375+0000 7f6a63c6b740 1 --2- 172.21.15.82:0/981725767 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56128bdaa2f0 0x56128c1733e0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f6a48006010 tx=0x7f6a48004200 comp rx=0 tx=0).stop 2024-05-20T16:14:46.383 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:46.375+0000 7f6a63c6b740 1 -- 172.21.15.82:0/981725767 shutdown_connections 2024-05-20T16:14:46.383 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:46.375+0000 7f6a63c6b740 1 --2- 172.21.15.82:0/981725767 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56128bdaa2f0 0x56128c1733e0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:46.383 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:46.375+0000 7f6a63c6b740 1 --2- 172.21.15.82:0/981725767 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56128bdb24d0 0x56128c17a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:46.383 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:46.375+0000 7f6a63c6b740 1 --2- 172.21.15.82:0/981725767 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56128bdd31b0 0x56128c172e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:46.383 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:46.375+0000 7f6a63c6b740 1 -- 172.21.15.82:0/981725767 >> 172.21.15.82:0/981725767 conn(0x56128bdc7300 msgr2=0x56128c16e7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:14:46.383 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:46.376+0000 7f6a63c6b740 1 -- 172.21.15.82:0/981725767 shutdown_connections 2024-05-20T16:14:46.383 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:46.376+0000 7f6a63c6b740 1 -- 172.21.15.82:0/981725767 wait complete. 2024-05-20T16:14:46.383 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:46.376+0000 7f6a63c6b740 1 -- start start 2024-05-20T16:14:46.383 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:46.376+0000 7f6a63c6b740 1 Processor -- start 2024-05-20T16:14:46.383 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:46.376+0000 7f6a63c6b740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56128bdaa2f0 0x56128c17cb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:14:46.383 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:46.376+0000 7f6a63c6b740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56128bdb24d0 0x56128c17d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:14:46.383 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:46.376+0000 7f6a63c6b740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56128bdd31b0 0x56128c183fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:14:46.384 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:46.376+0000 7f6a63c6b740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x56128c17acd0 con 0x56128bdaa2f0 2024-05-20T16:14:46.384 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:46.376+0000 7f6a63c6b740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x56128c092790 con 0x56128bdb24d0 2024-05-20T16:14:46.384 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:46.376+0000 7f6a63c6b740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x56128c17ae50 con 0x56128bdd31b0 2024-05-20T16:14:46.384 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:46.376+0000 7f6a63c6b740 10 client.0 _ll_register_callbacks cb 0x56128c0cda20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T16:14:46.384 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:46.376+0000 7f6a4ffff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56128bdaa2f0 0x56128c17cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:14:46.384 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:46.376+0000 7f6a4ffff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56128bdaa2f0 0x56128c17cb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:57304/0 (socket says 172.21.15.82:57304) 2024-05-20T16:14:46.384 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:46.376+0000 7f6a4ffff640 1 -- 172.21.15.82:0/4134223641 learned_addr learned my addr 172.21.15.82:0/4134223641 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:14:46.384 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:46.377+0000 7f6a5659c640 1 --2- 172.21.15.82:0/4134223641 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56128bdb24d0 0x56128c17d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:14:46.384 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:46.377+0000 7f6a4ffff640 1 -- 172.21.15.82:0/4134223641 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56128bdd31b0 msgr2=0x56128c183fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:14:46.384 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:46.377+0000 7f6a4ffff640 1 --2- 172.21.15.82:0/4134223641 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56128bdd31b0 0x56128c183fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:46.384 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:46.377+0000 7f6a4ffff640 1 -- 172.21.15.82:0/4134223641 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56128bdb24d0 msgr2=0x56128c17d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:14:46.384 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:46.377+0000 7f6a4ffff640 1 --2- 172.21.15.82:0/4134223641 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56128bdb24d0 0x56128c17d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:46.384 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:46.377+0000 7f6a4ffff640 1 -- 172.21.15.82:0/4134223641 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56128bdeb210 con 0x56128bdaa2f0 2024-05-20T16:14:46.385 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:46.377+0000 7f6a4ffff640 1 --2- 172.21.15.82:0/4134223641 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56128bdaa2f0 0x56128c17cb00 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f6a40013e40 tx=0x7f6a4000f7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c459baae2c7a5a31 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:14:46.385 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:46.377+0000 7f6a3dffb640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-05-20T16:14:46.385 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:46.377+0000 7f6a3dffb640 1 -- 172.21.15.82:0/4134223641 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a4001d440 con 0x56128bdaa2f0 2024-05-20T16:14:46.385 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:46.377+0000 7f6a3dffb640 1 -- 172.21.15.82:0/4134223641 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9150+0+0 (secure 0 0 0) 0x7f6a40010430 con 0x56128bdaa2f0 2024-05-20T16:14:46.385 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:46.377+0000 7f6a63c6b740 1 -- 172.21.15.82:0/4134223641 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6a480044b0 con 0x56128bdaa2f0 2024-05-20T16:14:46.385 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:46.377+0000 7f6a3dffb640 1 -- 172.21.15.82:0/4134223641 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f6a40010650 con 0x56128bdaa2f0 2024-05-20T16:14:46.385 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:46.377+0000 7f6a63c6b740 10 client.15513 fetch_fsmap learned FSMap version 25 2024-05-20T16:14:46.385 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:46.377+0000 7f6a63c6b740 1 -- 172.21.15.82:0/4134223641 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56128beb7620 con 0x56128bdaa2f0 2024-05-20T16:14:46.385 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:46.378+0000 7f6a3dffb640 1 -- 172.21.15.82:0/4134223641 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a480044b0 con 0x56128bdaa2f0 2024-05-20T16:14:46.385 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:46.378+0000 7f6a3dffb640 1 -- 172.21.15.82:0/4134223641 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f6a4001b070 con 0x56128bdaa2f0 2024-05-20T16:14:46.385 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:46.378+0000 7f6a63c6b740 10 client.15513 fetch_fsmap finished waiting for FSMap version 25 2024-05-20T16:14:46.385 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:46.378+0000 7f6a63c6b740 10 client.15513 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T16:14:46.385 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:46.378+0000 7f6a63c6b740 20 client.15513 populate_metadata read hostname 'smithi082' 2024-05-20T16:14:46.386 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:46.378+0000 7f6a63c6b740 1 --2- 172.21.15.82:0/4134223641 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x56128bd91d80 0x56128c18a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:14:46.386 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:46.378+0000 7f6a63c6b740 4 client.15513 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T16:14:46.386 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:46.378+0000 7f6a63c6b740 1 -- 172.21.15.82:0/4134223641 --> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56128c0e4a70 con 0x56128bd91d80 2024-05-20T16:14:46.386 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:46.378+0000 7f6a5659c640 1 --2- 172.21.15.82:0/4134223641 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x56128bd91d80 0x56128c18a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:14:46.386 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:46.379+0000 7f6a3dffb640 10 client.15513 ms_handle_connect on v2:172.21.15.160:6838/2765554482 2024-05-20T16:14:46.386 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:46.379+0000 7f6a5659c640 1 --2- 172.21.15.82:0/4134223641 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x56128bd91d80 0x56128c18a4a0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c56ff073da69f6e5 server_cookie=b1a9a5702ab4e5cd in_seq=0 out_seq=0 2024-05-20T16:14:46.386 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:46.379+0000 7f6a3dffb640 1 -- 172.21.15.82:0/4134223641 <== mds.0 v2:172.21.15.160:6838/2765554482 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56128c0e4a70 con 0x56128bd91d80 2024-05-20T16:14:46.386 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:46.379+0000 7f6a3dffb640 10 client.15513 handle_command_reply: tid=0 2024-05-20T16:14:46.386 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:46.379+0000 7f6a63c6b740 1 client.15513 shutdown 2024-05-20T16:14:46.386 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:46.380+0000 7f6a63c6b740 1 -- 172.21.15.82:0/4134223641 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56128bdaa2f0 msgr2=0x56128c17cb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:14:46.386 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:46.380+0000 7f6a63c6b740 1 --2- 172.21.15.82:0/4134223641 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56128bdaa2f0 0x56128c17cb00 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f6a40013e40 tx=0x7f6a4000f7f0 comp rx=0 tx=0).stop 2024-05-20T16:14:46.386 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:46.380+0000 7f6a63c6b740 1 -- 172.21.15.82:0/4134223641 shutdown_connections 2024-05-20T16:14:46.386 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:46.380+0000 7f6a63c6b740 1 --2- 172.21.15.82:0/4134223641 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x56128bd91d80 0x56128c18a4a0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:46.387 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:46.380+0000 7f6a63c6b740 1 --2- 172.21.15.82:0/4134223641 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56128bdd31b0 0x56128c183fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:46.387 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:46.380+0000 7f6a63c6b740 1 --2- 172.21.15.82:0/4134223641 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56128bdb24d0 0x56128c17d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:46.387 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:46.380+0000 7f6a63c6b740 1 --2- 172.21.15.82:0/4134223641 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56128bdaa2f0 0x56128c17cb00 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:46.387 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:46.380+0000 7f6a3dffb640 0 client.15513 ms_handle_reset on v2:172.21.15.160:6838/2765554482 2024-05-20T16:14:46.387 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:46.380+0000 7f6a63c6b740 1 -- 172.21.15.82:0/4134223641 >> 172.21.15.82:0/4134223641 conn(0x56128bdc7300 msgr2=0x56128c1717a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:14:46.387 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:46.380+0000 7f6a63c6b740 1 -- 172.21.15.82:0/4134223641 shutdown_connections 2024-05-20T16:14:46.387 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:46.380+0000 7f6a63c6b740 1 -- 172.21.15.82:0/4134223641 wait complete. 2024-05-20T16:14:46.387 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:46.380+0000 7f6a63c6b740 20 client.15513 trim_cache size 0 max 16384 2024-05-20T16:14:46.402 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:46.395+0000 7f6a63c6b740 1 Processor -- start 2024-05-20T16:14:46.402 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:46.395+0000 7f6a63c6b740 1 -- start start 2024-05-20T16:14:46.402 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:46.395+0000 7f6a63c6b740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56128bdaa2f0 0x56128c17cde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:14:46.402 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:46.395+0000 7f6a63c6b740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56128bdb24d0 0x56128c184320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:14:46.402 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:46.395+0000 7f6a63c6b740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56128bdd31b0 0x56128c16cde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:14:46.402 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:46.395+0000 7f6a63c6b740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f6a581ae370 con 0x56128bdd31b0 2024-05-20T16:14:46.402 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:46.395+0000 7f6a63c6b740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f6a581abc40 con 0x56128bdaa2f0 2024-05-20T16:14:46.402 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:46.395+0000 7f6a63c6b740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f6a581a9a90 con 0x56128bdb24d0 2024-05-20T16:14:46.402 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:46.395+0000 7f6a4ffff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56128bdd31b0 0x56128c16cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:14:46.402 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:46.395+0000 7f6a4ffff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56128bdd31b0 0x56128c16cde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:57314/0 (socket says 172.21.15.82:57314) 2024-05-20T16:14:46.403 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:46.395+0000 7f6a4ffff640 1 -- 172.21.15.82:0/4187697383 learned_addr learned my addr 172.21.15.82:0/4187697383 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:14:46.403 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:46.395+0000 7f6a4ffff640 1 --2- 172.21.15.82:0/4187697383 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56128bdaa2f0 0x56128c17cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:14:46.403 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:46.395+0000 7f6a5659c640 1 --2- 172.21.15.82:0/4187697383 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56128bdb24d0 0x56128c184320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:14:46.403 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:46.395+0000 7f6a4ffff640 1 -- 172.21.15.82:0/4187697383 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56128bdb24d0 msgr2=0x56128c184320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:14:46.403 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:46.395+0000 7f6a4ffff640 1 --2- 172.21.15.82:0/4187697383 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56128bdb24d0 0x56128c184320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:46.403 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:46.395+0000 7f6a4ffff640 1 -- 172.21.15.82:0/4187697383 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56128bdaa2f0 msgr2=0x56128c17cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:14:46.403 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:46.395+0000 7f6a4ffff640 1 --2- 172.21.15.82:0/4187697383 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56128bdaa2f0 0x56128c17cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:46.403 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:46.395+0000 7f6a4ffff640 1 -- 172.21.15.82:0/4187697383 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56128c174030 con 0x56128bdd31b0 2024-05-20T16:14:46.403 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:46.396+0000 7f6a4ffff640 1 --2- 172.21.15.82:0/4187697383 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56128bdd31b0 0x56128c16cde0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f6a40013af0 tx=0x7f6a40012040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e466e7e6ae75ed5 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:14:46.403 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:46.396+0000 7f6a3ffff640 1 -- 172.21.15.82:0/4187697383 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a4000ad00 con 0x56128bdd31b0 2024-05-20T16:14:46.403 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:46.396+0000 7f6a3ffff640 1 -- 172.21.15.82:0/4187697383 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a40010cd0 con 0x56128bdd31b0 2024-05-20T16:14:46.403 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:46.396+0000 7f6a3ffff640 1 -- 172.21.15.82:0/4187697383 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a4001c850 con 0x56128bdd31b0 2024-05-20T16:14:46.403 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:46.396+0000 7f6a63c6b740 1 -- 172.21.15.82:0/4187697383 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56128bdd31b0 msgr2=0x56128c16cde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:14:46.403 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:46.396+0000 7f6a63c6b740 1 --2- 172.21.15.82:0/4187697383 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56128bdd31b0 0x56128c16cde0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f6a40013af0 tx=0x7f6a40012040 comp rx=0 tx=0).stop 2024-05-20T16:14:46.404 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:46.397+0000 7f6a63c6b740 1 -- 172.21.15.82:0/4187697383 shutdown_connections 2024-05-20T16:14:46.404 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:46.397+0000 7f6a63c6b740 1 --2- 172.21.15.82:0/4187697383 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56128bdd31b0 0x56128c16cde0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:46.404 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:46.397+0000 7f6a63c6b740 1 --2- 172.21.15.82:0/4187697383 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56128bdb24d0 0x56128c184320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:46.404 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:46.397+0000 7f6a63c6b740 1 --2- 172.21.15.82:0/4187697383 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56128bdaa2f0 0x56128c17cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:46.404 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:46.397+0000 7f6a63c6b740 1 -- 172.21.15.82:0/4187697383 >> 172.21.15.82:0/4187697383 conn(0x56128bdc7300 msgr2=0x56128c17c6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:14:46.404 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:46.397+0000 7f6a63c6b740 1 -- 172.21.15.82:0/4187697383 shutdown_connections 2024-05-20T16:14:46.404 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:46.397+0000 7f6a63c6b740 1 -- 172.21.15.82:0/4187697383 wait complete. 2024-05-20T16:14:46.404 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:46.397+0000 7f6a63c6b740 1 -- start start 2024-05-20T16:14:46.404 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:46.397+0000 7f6a63c6b740 1 Processor -- start 2024-05-20T16:14:46.404 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:46.397+0000 7f6a63c6b740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56128bdaa2f0 0x56128c1ed650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:14:46.404 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:46.397+0000 7f6a63c6b740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56128bdb24d0 0x56128c1edb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:14:46.404 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:46.397+0000 7f6a63c6b740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56128bdd31b0 0x56128c1f4a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:14:46.404 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:46.397+0000 7f6a63c6b740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f6a581aa100 con 0x56128bdaa2f0 2024-05-20T16:14:46.404 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:46.397+0000 7f6a63c6b740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f6a581ab570 con 0x56128bdb24d0 2024-05-20T16:14:46.405 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:46.397+0000 7f6a63c6b740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f6a581aa7d0 con 0x56128bdd31b0 2024-05-20T16:14:46.405 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:46.397+0000 7f6a4ffff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56128bdaa2f0 0x56128c1ed650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:14:46.405 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:46.397+0000 7f6a63c6b740 10 client.0 _ll_register_callbacks cb 0x7f6a4000ac50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T16:14:46.405 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:46.397+0000 7f6a4ffff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56128bdaa2f0 0x56128c1ed650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:57318/0 (socket says 172.21.15.82:57318) 2024-05-20T16:14:46.405 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:46.397+0000 7f6a4ffff640 1 -- 172.21.15.82:0/531131354 learned_addr learned my addr 172.21.15.82:0/531131354 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:14:46.405 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:46.397+0000 7f6a5659c640 1 --2- 172.21.15.82:0/531131354 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56128bdb24d0 0x56128c1edb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:14:46.405 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:46.398+0000 7f6a4ffff640 1 -- 172.21.15.82:0/531131354 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56128bdd31b0 msgr2=0x56128c1f4a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:14:46.405 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:46.398+0000 7f6a4ffff640 1 --2- 172.21.15.82:0/531131354 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56128bdd31b0 0x56128c1f4a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:46.405 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:46.398+0000 7f6a4ffff640 1 -- 172.21.15.82:0/531131354 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56128bdb24d0 msgr2=0x56128c1edb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:14:46.405 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:46.398+0000 7f6a4ffff640 1 --2- 172.21.15.82:0/531131354 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56128bdb24d0 0x56128c1edb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:46.405 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:46.398+0000 7f6a4ffff640 1 -- 172.21.15.82:0/531131354 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56128c1f5040 con 0x56128bdaa2f0 2024-05-20T16:14:46.406 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:46.398+0000 7f6a5659c640 1 --2- 172.21.15.82:0/531131354 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56128bdb24d0 0x56128c1edb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T16:14:46.406 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:46.398+0000 7f6a4ffff640 1 --2- 172.21.15.82:0/531131354 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56128bdaa2f0 0x56128c1ed650 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f6a400058c0 tx=0x7f6a4000a5e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=303bbddc036d2a5e server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:14:46.406 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:46.398+0000 7f6a3dffb640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-05-20T16:14:46.406 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:46.398+0000 7f6a3dffb640 1 -- 172.21.15.82:0/531131354 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a4003c060 con 0x56128bdaa2f0 2024-05-20T16:14:46.406 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:46.398+0000 7f6a3dffb640 1 -- 172.21.15.82:0/531131354 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9150+0+0 (secure 0 0 0) 0x7f6a400107f0 con 0x56128bdaa2f0 2024-05-20T16:14:46.406 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:46.398+0000 7f6a63c6b740 1 -- 172.21.15.82:0/531131354 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6a40010cd0 con 0x56128bdaa2f0 2024-05-20T16:14:46.406 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:46.398+0000 7f6a3dffb640 1 -- 172.21.15.82:0/531131354 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f6a40010a10 con 0x56128bdaa2f0 2024-05-20T16:14:46.406 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:46.398+0000 7f6a63c6b740 10 client.15519 fetch_fsmap learned FSMap version 25 2024-05-20T16:14:46.406 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:46.398+0000 7f6a63c6b740 1 -- 172.21.15.82:0/531131354 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56128c1f62d0 con 0x56128bdaa2f0 2024-05-20T16:14:46.406 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:46.398+0000 7f6a3dffb640 1 -- 172.21.15.82:0/531131354 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a40010cd0 con 0x56128bdaa2f0 2024-05-20T16:14:46.406 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:46.398+0000 7f6a3dffb640 1 -- 172.21.15.82:0/531131354 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f6a4001b070 con 0x56128bdaa2f0 2024-05-20T16:14:46.406 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:46.398+0000 7f6a63c6b740 10 client.15519 fetch_fsmap finished waiting for FSMap version 25 2024-05-20T16:14:46.406 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:46.398+0000 7f6a63c6b740 10 client.15519 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T16:14:46.407 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:46.398+0000 7f6a63c6b740 20 client.15519 populate_metadata read hostname 'smithi082' 2024-05-20T16:14:46.407 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:46.399+0000 7f6a63c6b740 1 --2- 172.21.15.82:0/531131354 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x56128bd671c0 0x56128c1f8c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:14:46.407 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:46.399+0000 7f6a63c6b740 4 client.15519 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-20T16:14:46.407 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:46.399+0000 7f6a63c6b740 1 -- 172.21.15.82:0/531131354 --> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56128c0dedc0 con 0x56128bd671c0 2024-05-20T16:14:46.407 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:46.399+0000 7f6a5659c640 1 --2- 172.21.15.82:0/531131354 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x56128bd671c0 0x56128c1f8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:14:46.407 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:46.399+0000 7f6a3dffb640 10 client.15519 ms_handle_connect on v2:172.21.15.160:6838/2765554482 2024-05-20T16:14:46.407 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:46.399+0000 7f6a5659c640 1 --2- 172.21.15.82:0/531131354 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x56128bd671c0 0x56128c1f8c10 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bcc691212dace703 server_cookie=60215bdcdba80e16 in_seq=0 out_seq=0 2024-05-20T16:14:46.408 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:46.406+0000 7f6a3dffb640 1 -- 172.21.15.82:0/531131354 <== mds.0 v2:172.21.15.160:6838/2765554482 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56128c0dedc0 con 0x56128bd671c0 2024-05-20T16:14:46.408 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:46.406+0000 7f6a3dffb640 10 client.15519 handle_command_reply: tid=0 2024-05-20T16:14:46.409 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:46.406+0000 7f6a63c6b740 1 client.15519 shutdown 2024-05-20T16:14:46.409 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:46.407+0000 7f6a63c6b740 1 -- 172.21.15.82:0/531131354 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56128bdaa2f0 msgr2=0x56128c1ed650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:14:46.409 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:46.407+0000 7f6a63c6b740 1 --2- 172.21.15.82:0/531131354 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56128bdaa2f0 0x56128c1ed650 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f6a400058c0 tx=0x7f6a4000a5e0 comp rx=0 tx=0).stop 2024-05-20T16:14:46.409 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:46.407+0000 7f6a63c6b740 1 -- 172.21.15.82:0/531131354 shutdown_connections 2024-05-20T16:14:46.409 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:46.407+0000 7f6a63c6b740 1 --2- 172.21.15.82:0/531131354 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x56128bd671c0 0x56128c1f8c10 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:46.409 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:46.407+0000 7f6a63c6b740 1 --2- 172.21.15.82:0/531131354 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56128bdd31b0 0x56128c1f4a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:46.409 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:46.407+0000 7f6a63c6b740 1 --2- 172.21.15.82:0/531131354 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56128bdb24d0 0x56128c1edb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:46.409 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:46.407+0000 7f6a3dffb640 0 client.15519 ms_handle_reset on v2:172.21.15.160:6838/2765554482 2024-05-20T16:14:46.409 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:46.407+0000 7f6a63c6b740 1 --2- 172.21.15.82:0/531131354 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56128bdaa2f0 0x56128c1ed650 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:46.409 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:46.407+0000 7f6a63c6b740 1 -- 172.21.15.82:0/531131354 >> 172.21.15.82:0/531131354 conn(0x56128bdc7300 msgr2=0x56128c171690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:14:46.409 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:46.407+0000 7f6a63c6b740 1 -- 172.21.15.82:0/531131354 shutdown_connections 2024-05-20T16:14:46.409 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:46.407+0000 7f6a63c6b740 1 -- 172.21.15.82:0/531131354 wait complete. 2024-05-20T16:14:46.409 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:46.407+0000 7f6a63c6b740 20 client.15519 trim_cache size 0 max 16384 2024-05-20T16:14:46.410 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:{ 2024-05-20T16:14:46.410 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "message": "", 2024-05-20T16:14:46.410 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "return_code": 0 2024-05-20T16:14:46.410 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:} 2024-05-20T16:14:46.411 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:46.407+0000 7f6a5cdc3640 1 -- 172.21.15.82:0/149053672 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f6a440a2c10 msgr2=0x7f6a440a5120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:14:46.411 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:46.407+0000 7f6a5cdc3640 1 --2- 172.21.15.82:0/149053672 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f6a440a2c10 0x7f6a440a5120 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f6a5009b660 tx=0x7f6a50096380 comp rx=0 tx=0).stop 2024-05-20T16:14:46.411 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:46.407+0000 7f6a5cdc3640 1 -- 172.21.15.82:0/149053672 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6a58108a20 msgr2=0x7f6a581a07c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:14:46.411 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:46.407+0000 7f6a5cdc3640 1 --2- 172.21.15.82:0/149053672 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6a58108a20 0x7f6a581a07c0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f6a5000cad0 tx=0x7f6a5000cf90 comp rx=0 tx=0).stop 2024-05-20T16:14:46.411 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:46.407+0000 7f6a5cdc3640 1 -- 172.21.15.82:0/149053672 shutdown_connections 2024-05-20T16:14:46.411 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:46.407+0000 7f6a5cdc3640 1 --2- 172.21.15.82:0/149053672 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f6a440a2c10 0x7f6a440a5120 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:46.412 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:46.407+0000 7f6a5cdc3640 1 --2- 172.21.15.82:0/149053672 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6a58108a20 0x7f6a581a07c0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:46.412 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:46.407+0000 7f6a5cdc3640 1 --2- 172.21.15.82:0/149053672 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6a58103d20 0x7f6a58199830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:46.412 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:46.407+0000 7f6a5cdc3640 1 --2- 172.21.15.82:0/149053672 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6a58102ae0 0x7f6a581992f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:46.412 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:46.407+0000 7f6a5cdc3640 1 -- 172.21.15.82:0/149053672 >> 172.21.15.82:0/149053672 conn(0x7f6a580fdcf0 msgr2=0x7f6a58104a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:14:46.412 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:46.408+0000 7f6a5cdc3640 1 -- 172.21.15.82:0/149053672 shutdown_connections 2024-05-20T16:14:46.412 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:46.408+0000 7f6a5cdc3640 1 -- 172.21.15.82:0/149053672 wait complete. 2024-05-20T16:14:47.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:14:47 smithi160 ceph-mon[36347]: pgmap v386: 129 pgs: 129 active+clean; 657 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 511 B/s wr, 6 op/s 2024-05-20T16:14:47.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:14:47 smithi082 ceph-mon[32238]: pgmap v386: 129 pgs: 129 active+clean; 657 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 511 B/s wr, 6 op/s 2024-05-20T16:14:47.885 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:14:47 smithi184 ceph-mon[36209]: pgmap v386: 129 pgs: 129 active+clean; 657 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 511 B/s wr, 6 op/s 2024-05-20T16:14:49.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:14:49 smithi160 ceph-mon[36347]: pgmap v387: 129 pgs: 129 active+clean; 668 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.8 KiB/s wr, 9 op/s 2024-05-20T16:14:49.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:14:49 smithi082 ceph-mon[32238]: pgmap v387: 129 pgs: 129 active+clean; 668 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.8 KiB/s wr, 9 op/s 2024-05-20T16:14:49.885 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:14:49 smithi184 ceph-mon[36209]: pgmap v387: 129 pgs: 129 active+clean; 668 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.8 KiB/s wr, 9 op/s 2024-05-20T16:14:51.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:14:51 smithi160 ceph-mon[36347]: pgmap v388: 129 pgs: 129 active+clean; 668 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-05-20T16:14:51.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:14:51 smithi082 ceph-mon[32238]: pgmap v388: 129 pgs: 129 active+clean; 668 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-05-20T16:14:51.885 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:14:51 smithi184 ceph-mon[36209]: pgmap v388: 129 pgs: 129 active+clean; 668 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-05-20T16:14:53.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:14:53 smithi160 ceph-mon[36347]: pgmap v389: 129 pgs: 129 active+clean; 668 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-05-20T16:14:53.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:14:53 smithi082 ceph-mon[32238]: pgmap v389: 129 pgs: 129 active+clean; 668 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-05-20T16:14:53.885 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:14:53 smithi184 ceph-mon[36209]: pgmap v389: 129 pgs: 129 active+clean; 668 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-05-20T16:14:55.688 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:14:55 smithi184 ceph-mon[36209]: pgmap v390: 129 pgs: 129 active+clean; 668 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-05-20T16:14:55.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:14:55 smithi160 ceph-mon[36347]: pgmap v390: 129 pgs: 129 active+clean; 668 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-05-20T16:14:55.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:14:55 smithi082 ceph-mon[32238]: pgmap v390: 129 pgs: 129 active+clean; 668 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-05-20T16:14:57.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:14:57 smithi160 ceph-mon[36347]: pgmap v391: 129 pgs: 129 active+clean; 668 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-05-20T16:14:57.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:14:57 smithi082 ceph-mon[32238]: pgmap v391: 129 pgs: 129 active+clean; 668 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-05-20T16:14:57.885 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:14:57 smithi184 ceph-mon[36209]: pgmap v391: 129 pgs: 129 active+clean; 668 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-05-20T16:14:59.512 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:59.510+0000 7f71a3bea640 1 -- 172.21.15.82:0/886774372 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f719c102c40 msgr2=0x7f719c103050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:14:59.512 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:59.510+0000 7f71a3bea640 1 --2- 172.21.15.82:0/886774372 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f719c102c40 0x7f719c103050 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f718c009900 tx=0x7f718c02a410 comp rx=0 tx=0).stop 2024-05-20T16:14:59.512 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:59.510+0000 7f71a3bea640 1 -- 172.21.15.82:0/886774372 shutdown_connections 2024-05-20T16:14:59.512 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:59.510+0000 7f71a3bea640 1 --2- 172.21.15.82:0/886774372 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f719c102c40 0x7f719c103050 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:59.512 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:59.510+0000 7f71a3bea640 1 --2- 172.21.15.82:0/886774372 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f719c107940 0x7f719c109d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:59.512 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:59.510+0000 7f71a3bea640 1 --2- 172.21.15.82:0/886774372 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f719c101a00 0x7f719c102700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:59.512 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:59.510+0000 7f71a3bea640 1 -- 172.21.15.82:0/886774372 >> 172.21.15.82:0/886774372 conn(0x7f719c0fcc10 msgr2=0x7f719c0ff0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:14:59.512 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:59.510+0000 7f71a3bea640 1 -- 172.21.15.82:0/886774372 shutdown_connections 2024-05-20T16:14:59.513 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:59.510+0000 7f71a3bea640 1 -- 172.21.15.82:0/886774372 wait complete. 2024-05-20T16:14:59.513 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:59.511+0000 7f71a3bea640 1 Processor -- start 2024-05-20T16:14:59.513 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:59.511+0000 7f71a3bea640 1 -- start start 2024-05-20T16:14:59.513 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:59.511+0000 7f71a3bea640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f719c101a00 0x7f719c06ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:14:59.513 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:59.511+0000 7f71a3bea640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f719c102c40 0x7f719c19e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:14:59.513 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:59.511+0000 7f71a3bea640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f719c107940 0x7f719c1a0600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:14:59.513 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:59.511+0000 7f71a3bea640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f719c10d6c0 con 0x7f719c102c40 2024-05-20T16:14:59.513 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:59.511+0000 7f71a3bea640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f719c10d540 con 0x7f719c107940 2024-05-20T16:14:59.513 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:59.511+0000 7f71a3bea640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f719c10d840 con 0x7f719c101a00 2024-05-20T16:14:59.513 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:59.511+0000 7f71a1185640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f719c102c40 0x7f719c19e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:14:59.513 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:59.511+0000 7f71a1185640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f719c102c40 0x7f719c19e0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:54166/0 (socket says 172.21.15.82:54166) 2024-05-20T16:14:59.513 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:59.511+0000 7f71a1185640 1 -- 172.21.15.82:0/3963361528 learned_addr learned my addr 172.21.15.82:0/3963361528 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:14:59.513 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:59.511+0000 7f71a2187640 1 --2- 172.21.15.82:0/3963361528 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f719c107940 0x7f719c1a0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:14:59.513 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:59.511+0000 7f71a1986640 1 --2- 172.21.15.82:0/3963361528 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f719c101a00 0x7f719c06ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:14:59.514 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:59.511+0000 7f71a1185640 1 -- 172.21.15.82:0/3963361528 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f719c101a00 msgr2=0x7f719c06ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:14:59.514 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:59.511+0000 7f71a1185640 1 --2- 172.21.15.82:0/3963361528 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f719c101a00 0x7f719c06ac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:59.514 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:59.511+0000 7f71a1185640 1 -- 172.21.15.82:0/3963361528 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f719c107940 msgr2=0x7f719c1a0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:14:59.514 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:59.511+0000 7f71a1185640 1 --2- 172.21.15.82:0/3963361528 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f719c107940 0x7f719c1a0600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:59.514 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:59.511+0000 7f71a1185640 1 -- 172.21.15.82:0/3963361528 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f719c1a0e20 con 0x7f719c102c40 2024-05-20T16:14:59.514 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:59.511+0000 7f71a1185640 1 --2- 172.21.15.82:0/3963361528 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f719c102c40 0x7f719c19e0b0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f718c031440 tx=0x7f718c032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:14:59.514 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:59.512+0000 7f7192ffd640 1 -- 172.21.15.82:0/3963361528 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f718c03f030 con 0x7f719c102c40 2024-05-20T16:14:59.514 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:59.512+0000 7f7192ffd640 1 -- 172.21.15.82:0/3963361528 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f718c003ce0 con 0x7f719c102c40 2024-05-20T16:14:59.514 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:59.512+0000 7f71a3bea640 1 -- 172.21.15.82:0/3963361528 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f719c1a10b0 con 0x7f719c102c40 2024-05-20T16:14:59.514 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:59.512+0000 7f7192ffd640 1 -- 172.21.15.82:0/3963361528 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f718c035070 con 0x7f719c102c40 2024-05-20T16:14:59.514 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:59.512+0000 7f71a3bea640 1 -- 172.21.15.82:0/3963361528 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f719c1a15f0 con 0x7f719c102c40 2024-05-20T16:14:59.517 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:59.515+0000 7f71aab6c740 1 Processor -- start 2024-05-20T16:14:59.517 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:59.515+0000 7f71aab6c740 1 -- start start 2024-05-20T16:14:59.518 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:59.515+0000 7f71aab6c740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d4071511b0 0x55d4074f0e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:14:59.518 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:59.515+0000 7f7192ffd640 1 -- 172.21.15.82:0/3963361528 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 19) ==== 137884+0+0 (secure 0 0 0) 0x7f718c042030 con 0x7f719c102c40 2024-05-20T16:14:59.518 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:59.515+0000 7f71aab6c740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d4071282f0 0x55d4074f13e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:14:59.518 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:59.515+0000 7f71aab6c740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d4071304d0 0x55d4074f82e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:14:59.518 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:59.515+0000 7f71aab6c740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55d407455340 con 0x55d4071282f0 2024-05-20T16:14:59.518 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:59.515+0000 7f71aab6c740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x55d4073fd290 con 0x55d4071304d0 2024-05-20T16:14:59.518 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:59.515+0000 7f71aab6c740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55d407451300 con 0x55d4071511b0 2024-05-20T16:14:59.518 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:59.515+0000 7f71a1986640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d4071282f0 0x55d4074f13e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:14:59.518 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:59.515+0000 7f71a1986640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d4071282f0 0x55d4074f13e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:54178/0 (socket says 172.21.15.82:54178) 2024-05-20T16:14:59.518 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:59.515+0000 7f71a1986640 1 -- 172.21.15.82:0/177160981 learned_addr learned my addr 172.21.15.82:0/177160981 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:14:59.518 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:59.515+0000 7f71a2187640 1 --2- 172.21.15.82:0/177160981 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d4071511b0 0x55d4074f0e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:14:59.519 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:59.516+0000 7f71a1185640 1 --2- 172.21.15.82:0/177160981 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d4071304d0 0x55d4074f82e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:14:59.519 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:59.516+0000 7f71a1986640 1 -- 172.21.15.82:0/177160981 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d4071511b0 msgr2=0x55d4074f0e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:14:59.519 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:59.516+0000 7f71a1986640 1 --2- 172.21.15.82:0/177160981 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d4071511b0 0x55d4074f0e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:59.519 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:59.516+0000 7f71a1986640 1 -- 172.21.15.82:0/177160981 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d4071304d0 msgr2=0x55d4074f82e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:14:59.519 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:59.516+0000 7f71a1986640 1 --2- 172.21.15.82:0/177160981 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d4071304d0 0x55d4074f82e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:59.519 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:59.516+0000 7f71a1986640 1 -- 172.21.15.82:0/177160981 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d4074218e0 con 0x55d4071282f0 2024-05-20T16:14:59.519 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:59.516+0000 7f71a1986640 1 --2- 172.21.15.82:0/177160981 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d4071282f0 0x55d4074f13e0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f719800bc40 tx=0x7f719800f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15077880e6d32f83 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:14:59.519 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:59.516+0000 7f7192ffd640 1 --2- 172.21.15.82:0/3963361528 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f71880a2c90 0x7f71880a51a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:14:59.519 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:59.516+0000 7f7187fff640 1 -- 172.21.15.82:0/177160981 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7198015070 con 0x55d4071282f0 2024-05-20T16:14:59.519 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:59.516+0000 7f7187fff640 1 -- 172.21.15.82:0/177160981 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f719801a940 con 0x55d4071282f0 2024-05-20T16:14:59.519 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:59.516+0000 7f7187fff640 1 -- 172.21.15.82:0/177160981 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71980119d0 con 0x55d4071282f0 2024-05-20T16:14:59.519 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:59.516+0000 7f71a2187640 1 --2- 172.21.15.82:0/3963361528 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f71880a2c90 0x7f71880a51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:14:59.519 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:59.516+0000 7f7192ffd640 1 -- 172.21.15.82:0/3963361528 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9150+0+0 (secure 0 0 0) 0x7f718c0efa10 con 0x7f719c102c40 2024-05-20T16:14:59.519 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:59.517+0000 7f71aab6c740 1 -- 172.21.15.82:0/177160981 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d4071282f0 msgr2=0x55d4074f13e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:14:59.520 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:59.517+0000 7f71a2187640 1 --2- 172.21.15.82:0/3963361528 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f71880a2c90 0x7f71880a51a0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x55d4074f2030 tx=0x7f7194009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:14:59.520 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:59.517+0000 7f71aab6c740 1 --2- 172.21.15.82:0/177160981 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d4071282f0 0x55d4074f13e0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f719800bc40 tx=0x7f719800f730 comp rx=0 tx=0).stop 2024-05-20T16:14:59.520 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:59.517+0000 7f71aab6c740 1 -- 172.21.15.82:0/177160981 shutdown_connections 2024-05-20T16:14:59.520 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:59.517+0000 7f71aab6c740 1 --2- 172.21.15.82:0/177160981 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d4071282f0 0x55d4074f13e0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:59.520 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:59.517+0000 7f71aab6c740 1 --2- 172.21.15.82:0/177160981 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d4071304d0 0x55d4074f82e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:59.520 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:59.517+0000 7f71aab6c740 1 --2- 172.21.15.82:0/177160981 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d4071511b0 0x55d4074f0e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:59.520 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:59.517+0000 7f71aab6c740 1 -- 172.21.15.82:0/177160981 >> 172.21.15.82:0/177160981 conn(0x55d407145300 msgr2=0x55d4074ec7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:14:59.520 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:59.517+0000 7f71aab6c740 1 -- 172.21.15.82:0/177160981 shutdown_connections 2024-05-20T16:14:59.520 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:59.517+0000 7f71aab6c740 1 -- 172.21.15.82:0/177160981 wait complete. 2024-05-20T16:14:59.520 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:59.518+0000 7f71aab6c740 1 -- start start 2024-05-20T16:14:59.520 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:59.518+0000 7f71aab6c740 1 Processor -- start 2024-05-20T16:14:59.523 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:59.518+0000 7f71aab6c740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d4071282f0 0x55d4074fac20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:14:59.524 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:59.518+0000 7f71aab6c740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d4071304d0 0x55d4074fb160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:14:59.524 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:59.518+0000 7f71aab6c740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d4071511b0 0x55d407502050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:14:59.524 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:59.518+0000 7f71aab6c740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55d4074f8d30 con 0x55d4071282f0 2024-05-20T16:14:59.524 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:59.518+0000 7f71aab6c740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x55d407410790 con 0x55d4071511b0 2024-05-20T16:14:59.524 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:59.518+0000 7f71aab6c740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55d4074f8eb0 con 0x55d4071304d0 2024-05-20T16:14:59.524 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:59.518+0000 7f71aab6c740 10 client.0 _ll_register_callbacks cb 0x55d40744ba20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T16:14:59.524 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:59.519+0000 7f71a1986640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d4071282f0 0x55d4074fac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:14:59.524 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:59.519+0000 7f71a1986640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d4071282f0 0x55d4074fac20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:54182/0 (socket says 172.21.15.82:54182) 2024-05-20T16:14:59.524 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:59.519+0000 7f71a1986640 1 -- 172.21.15.82:0/1230164409 learned_addr learned my addr 172.21.15.82:0/1230164409 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:14:59.524 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:59.519+0000 7f71a2187640 1 --2- 172.21.15.82:0/1230164409 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d4071511b0 0x55d407502050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:14:59.524 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:59.519+0000 7f71a1185640 1 --2- 172.21.15.82:0/1230164409 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d4071304d0 0x55d4074fb160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:14:59.524 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:59.519+0000 7f71a1986640 1 -- 172.21.15.82:0/1230164409 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d4071304d0 msgr2=0x55d4074fb160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:14:59.525 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:59.519+0000 7f71a1986640 1 --2- 172.21.15.82:0/1230164409 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d4071304d0 0x55d4074fb160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:59.525 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:59.519+0000 7f71a1986640 1 -- 172.21.15.82:0/1230164409 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d4071511b0 msgr2=0x55d407502050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:14:59.525 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:59.519+0000 7f71a1986640 1 --2- 172.21.15.82:0/1230164409 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d4071511b0 0x55d407502050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:59.525 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:59.519+0000 7f71a1986640 1 -- 172.21.15.82:0/1230164409 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d407169210 con 0x55d4071282f0 2024-05-20T16:14:59.525 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:59.519+0000 7f71a1986640 1 --2- 172.21.15.82:0/1230164409 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d4071282f0 0x55d4074fac20 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f719800bf60 tx=0x7f719800bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c182ed2def3d00d server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:14:59.525 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:59.519+0000 7f71857fa640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-05-20T16:14:59.525 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:59.519+0000 7f71857fa640 1 -- 172.21.15.82:0/1230164409 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7198015070 con 0x55d4071282f0 2024-05-20T16:14:59.525 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:59.519+0000 7f71857fa640 1 -- 172.21.15.82:0/1230164409 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9150+0+0 (secure 0 0 0) 0x7f719801aaf0 con 0x55d4071282f0 2024-05-20T16:14:59.525 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:59.519+0000 7f71aab6c740 1 -- 172.21.15.82:0/1230164409 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f719801a940 con 0x55d4071282f0 2024-05-20T16:14:59.525 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:59.520+0000 7f71857fa640 1 -- 172.21.15.82:0/1230164409 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f719801a760 con 0x55d4071282f0 2024-05-20T16:14:59.525 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:59.520+0000 7f71aab6c740 10 client.15531 fetch_fsmap learned FSMap version 25 2024-05-20T16:14:59.525 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:59.520+0000 7f71aab6c740 1 -- 172.21.15.82:0/1230164409 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55d407235620 con 0x55d4071282f0 2024-05-20T16:14:59.525 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:59.520+0000 7f71857fa640 1 -- 172.21.15.82:0/1230164409 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f719801a940 con 0x55d4071282f0 2024-05-20T16:14:59.525 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:59.520+0000 7f71857fa640 1 -- 172.21.15.82:0/1230164409 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f71980117b0 con 0x55d4071282f0 2024-05-20T16:14:59.526 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:59.520+0000 7f71aab6c740 10 client.15531 fetch_fsmap finished waiting for FSMap version 25 2024-05-20T16:14:59.526 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:59.520+0000 7f71aab6c740 10 client.15531 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T16:14:59.526 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:59.520+0000 7f71aab6c740 20 client.15531 populate_metadata read hostname 'smithi082' 2024-05-20T16:14:59.526 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:59.520+0000 7f71aab6c740 1 --2- 172.21.15.82:0/1230164409 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x55d40710fd80 0x55d4075082b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:14:59.526 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:59.520+0000 7f71aab6c740 4 client.15531 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T16:14:59.526 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:59.520+0000 7f71aab6c740 1 -- 172.21.15.82:0/1230164409 --> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d407462a70 con 0x55d40710fd80 2024-05-20T16:14:59.526 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:59.520+0000 7f71a1185640 1 --2- 172.21.15.82:0/1230164409 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x55d40710fd80 0x55d4075082b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:14:59.526 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:59.521+0000 7f71857fa640 10 client.15531 ms_handle_connect on v2:172.21.15.160:6838/2765554482 2024-05-20T16:14:59.526 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:59.521+0000 7f71a1185640 1 --2- 172.21.15.82:0/1230164409 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x55d40710fd80 0x55d4075082b0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f1e079b3492dd2a1 server_cookie=7b0caa6c8ce08f53 in_seq=0 out_seq=0 2024-05-20T16:14:59.526 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:59.521+0000 7f71857fa640 1 -- 172.21.15.82:0/1230164409 <== mds.0 v2:172.21.15.160:6838/2765554482 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55d407462a70 con 0x55d40710fd80 2024-05-20T16:14:59.526 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:59.521+0000 7f71857fa640 10 client.15531 handle_command_reply: tid=0 2024-05-20T16:14:59.526 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:59.522+0000 7f71aab6c740 1 client.15531 shutdown 2024-05-20T16:14:59.527 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:59.522+0000 7f71aab6c740 1 -- 172.21.15.82:0/1230164409 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d4071282f0 msgr2=0x55d4074fac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:14:59.527 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:59.522+0000 7f71aab6c740 1 --2- 172.21.15.82:0/1230164409 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d4071282f0 0x55d4074fac20 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f719800bf60 tx=0x7f719800bf90 comp rx=0 tx=0).stop 2024-05-20T16:14:59.527 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:59.522+0000 7f71aab6c740 1 -- 172.21.15.82:0/1230164409 shutdown_connections 2024-05-20T16:14:59.527 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:59.522+0000 7f71aab6c740 1 --2- 172.21.15.82:0/1230164409 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x55d40710fd80 0x55d4075082b0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:59.527 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:59.522+0000 7f71aab6c740 1 --2- 172.21.15.82:0/1230164409 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d4071511b0 0x55d407502050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:59.527 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:59.522+0000 7f71aab6c740 1 --2- 172.21.15.82:0/1230164409 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d4071304d0 0x55d4074fb160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:59.527 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:59.522+0000 7f71aab6c740 1 --2- 172.21.15.82:0/1230164409 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d4071282f0 0x55d4074fac20 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:59.527 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:59.522+0000 7f71857fa640 0 client.15531 ms_handle_reset on v2:172.21.15.160:6838/2765554482 2024-05-20T16:14:59.527 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:59.522+0000 7f71aab6c740 1 -- 172.21.15.82:0/1230164409 >> 172.21.15.82:0/1230164409 conn(0x55d407145300 msgr2=0x55d4074ef7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:14:59.527 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:59.522+0000 7f71aab6c740 1 -- 172.21.15.82:0/1230164409 shutdown_connections 2024-05-20T16:14:59.527 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:59.522+0000 7f71aab6c740 1 -- 172.21.15.82:0/1230164409 wait complete. 2024-05-20T16:14:59.527 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:59.522+0000 7f71aab6c740 20 client.15531 trim_cache size 0 max 16384 2024-05-20T16:14:59.539 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:59.538+0000 7f71aab6c740 1 Processor -- start 2024-05-20T16:14:59.540 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:59.539+0000 7f71aab6c740 1 -- start start 2024-05-20T16:14:59.540 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:59.539+0000 7f71aab6c740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d4071282f0 0x55d4074ec920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:14:59.540 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:59.539+0000 7f71aab6c740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d4071304d0 0x55d4074f4070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:14:59.540 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:59.539+0000 7f71aab6c740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d4071511b0 0x55d4074f45b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:14:59.540 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:59.539+0000 7f71aab6c740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f719c1ae350 con 0x55d4071304d0 2024-05-20T16:14:59.540 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:59.539+0000 7f71aab6c740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f719c1abc20 con 0x55d4071511b0 2024-05-20T16:14:59.540 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:59.539+0000 7f71aab6c740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f719c1a9a70 con 0x55d4071282f0 2024-05-20T16:14:59.540 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:59.539+0000 7f71a1185640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d4071304d0 0x55d4074f4070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:14:59.540 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:59.539+0000 7f71a1185640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d4071304d0 0x55d4074f4070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58266/0 (socket says 172.21.15.82:58266) 2024-05-20T16:14:59.540 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:59.539+0000 7f71a1185640 1 -- 172.21.15.82:0/127247892 learned_addr learned my addr 172.21.15.82:0/127247892 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:14:59.540 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:59.539+0000 7f71a2187640 1 --2- 172.21.15.82:0/127247892 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d4071511b0 0x55d4074f45b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:14:59.540 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:59.539+0000 7f71a1986640 1 --2- 172.21.15.82:0/127247892 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d4071282f0 0x55d4074ec920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:14:59.541 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:59.539+0000 7f71a1185640 1 -- 172.21.15.82:0/127247892 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d4071282f0 msgr2=0x55d4074ec920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:14:59.541 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:59.539+0000 7f71a1185640 1 --2- 172.21.15.82:0/127247892 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d4071282f0 0x55d4074ec920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:59.541 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:59.539+0000 7f71a1185640 1 -- 172.21.15.82:0/127247892 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d4071511b0 msgr2=0x55d4074f45b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:14:59.541 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:59.539+0000 7f71a1185640 1 --2- 172.21.15.82:0/127247892 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d4071511b0 0x55d4074f45b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:59.541 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:59.539+0000 7f71a1185640 1 -- 172.21.15.82:0/127247892 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d4074f5a80 con 0x55d4071304d0 2024-05-20T16:14:59.541 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:59.539+0000 7f71a2187640 1 --2- 172.21.15.82:0/127247892 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d4071511b0 0x55d4074f45b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T16:14:59.541 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:59.539+0000 7f71a1185640 1 --2- 172.21.15.82:0/127247892 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d4071304d0 0x55d4074f4070 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f718c005ab0 tx=0x7f718c0f1f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8db0e6ce7b40c013 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:14:59.541 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:59.540+0000 7f71877fe640 1 -- 172.21.15.82:0/127247892 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f718c049040 con 0x55d4071304d0 2024-05-20T16:14:59.541 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:59.540+0000 7f71877fe640 1 -- 172.21.15.82:0/127247892 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f718c0bc480 con 0x55d4071304d0 2024-05-20T16:14:59.541 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:59.540+0000 7f71877fe640 1 -- 172.21.15.82:0/127247892 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f718c0c49f0 con 0x55d4071304d0 2024-05-20T16:14:59.544 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:59.540+0000 7f71aab6c740 1 -- 172.21.15.82:0/127247892 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d4071304d0 msgr2=0x55d4074f4070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:14:59.545 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:59.540+0000 7f71aab6c740 1 --2- 172.21.15.82:0/127247892 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d4071304d0 0x55d4074f4070 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f718c005ab0 tx=0x7f718c0f1f00 comp rx=0 tx=0).stop 2024-05-20T16:14:59.545 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:59.541+0000 7f71aab6c740 1 -- 172.21.15.82:0/127247892 shutdown_connections 2024-05-20T16:14:59.545 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:59.541+0000 7f71aab6c740 1 --2- 172.21.15.82:0/127247892 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d4071304d0 0x55d4074f4070 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:59.545 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:59.541+0000 7f71aab6c740 1 --2- 172.21.15.82:0/127247892 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d4071511b0 0x55d4074f45b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:59.545 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:59.541+0000 7f71aab6c740 1 --2- 172.21.15.82:0/127247892 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d4071282f0 0x55d4074ec920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:59.545 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:59.541+0000 7f71aab6c740 1 -- 172.21.15.82:0/127247892 >> 172.21.15.82:0/127247892 conn(0x55d407145300 msgr2=0x55d4074ec210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:14:59.545 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:59.541+0000 7f71aab6c740 1 -- 172.21.15.82:0/127247892 shutdown_connections 2024-05-20T16:14:59.545 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:59.541+0000 7f71aab6c740 1 -- 172.21.15.82:0/127247892 wait complete. 2024-05-20T16:14:59.545 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:59.541+0000 7f71aab6c740 1 -- start start 2024-05-20T16:14:59.545 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:59.541+0000 7f71aab6c740 1 Processor -- start 2024-05-20T16:14:59.545 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:59.541+0000 7f71aab6c740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d4071282f0 0x55d4074fe290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:14:59.545 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:59.541+0000 7f71aab6c740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d4071304d0 0x55d4074fe7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:14:59.546 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:59.541+0000 7f71aab6c740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d4071511b0 0x55d40756e650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:14:59.546 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:59.541+0000 7f71aab6c740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f719c1aa0e0 con 0x55d4071304d0 2024-05-20T16:14:59.546 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:59.541+0000 7f71aab6c740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f719c1ab550 con 0x55d4071511b0 2024-05-20T16:14:59.546 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:59.541+0000 7f71aab6c740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f719c1aa7b0 con 0x55d4071282f0 2024-05-20T16:14:59.546 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:59.541+0000 7f71aab6c740 10 client.0 _ll_register_callbacks cb 0x7f718c0f3430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T16:14:59.546 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:59.542+0000 7f71a1185640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d4071304d0 0x55d4074fe7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:14:59.546 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:59.542+0000 7f71a1185640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d4071304d0 0x55d4074fe7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58270/0 (socket says 172.21.15.82:58270) 2024-05-20T16:14:59.546 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:59.542+0000 7f71a1185640 1 -- 172.21.15.82:0/3067939699 learned_addr learned my addr 172.21.15.82:0/3067939699 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:14:59.546 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:59.542+0000 7f71a2187640 1 --2- 172.21.15.82:0/3067939699 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d4071511b0 0x55d40756e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:14:59.546 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:59.542+0000 7f71a1986640 1 --2- 172.21.15.82:0/3067939699 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d4071282f0 0x55d4074fe290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:14:59.546 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:59.542+0000 7f71a1185640 1 -- 172.21.15.82:0/3067939699 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d4071282f0 msgr2=0x55d4074fe290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:14:59.546 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:59.542+0000 7f71a1185640 1 --2- 172.21.15.82:0/3067939699 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d4071282f0 0x55d4074fe290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:59.546 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:59.542+0000 7f71a1185640 1 -- 172.21.15.82:0/3067939699 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d4071511b0 msgr2=0x55d40756e650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:14:59.547 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:59.542+0000 7f71a1185640 1 --2- 172.21.15.82:0/3067939699 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d4071511b0 0x55d40756e650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:59.547 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:59.542+0000 7f71a1185640 1 -- 172.21.15.82:0/3067939699 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d407501790 con 0x55d4071304d0 2024-05-20T16:14:59.547 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:59.542+0000 7f71a1185640 1 --2- 172.21.15.82:0/3067939699 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d4071304d0 0x55d4074fe7d0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f718c0bba80 tx=0x7f718c03b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6b57100e9499a04 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:14:59.547 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:59.542+0000 7f71857fa640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-05-20T16:14:59.547 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:59.542+0000 7f71857fa640 1 -- 172.21.15.82:0/3067939699 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f718c0bb8c0 con 0x55d4071304d0 2024-05-20T16:14:59.547 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:59.542+0000 7f71857fa640 1 -- 172.21.15.82:0/3067939699 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9150+0+0 (secure 0 0 0) 0x7f718c03b460 con 0x55d4071304d0 2024-05-20T16:14:59.547 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:59.542+0000 7f71aab6c740 1 -- 172.21.15.82:0/3067939699 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f718c0bc480 con 0x55d4071304d0 2024-05-20T16:14:59.547 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:59.542+0000 7f71857fa640 1 -- 172.21.15.82:0/3067939699 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f718c0c49f0 con 0x55d4071304d0 2024-05-20T16:14:59.547 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:59.542+0000 7f71aab6c740 10 client.15537 fetch_fsmap learned FSMap version 25 2024-05-20T16:14:59.547 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:59.542+0000 7f71aab6c740 1 -- 172.21.15.82:0/3067939699 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55d407501790 con 0x55d4071304d0 2024-05-20T16:14:59.547 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:59.543+0000 7f71857fa640 1 -- 172.21.15.82:0/3067939699 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f718c0bc480 con 0x55d4071304d0 2024-05-20T16:14:59.547 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:59.543+0000 7f71857fa640 1 -- 172.21.15.82:0/3067939699 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f718c03bc80 con 0x55d4071304d0 2024-05-20T16:14:59.547 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:59.543+0000 7f71aab6c740 10 client.15537 fetch_fsmap finished waiting for FSMap version 25 2024-05-20T16:14:59.547 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:59.543+0000 7f71aab6c740 10 client.15537 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T16:14:59.547 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:59.543+0000 7f71aab6c740 20 client.15537 populate_metadata read hostname 'smithi082' 2024-05-20T16:14:59.548 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:59.543+0000 7f71aab6c740 1 --2- 172.21.15.82:0/3067939699 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x55d4070e51c0 0x55d407571fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:14:59.548 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:59.543+0000 7f71aab6c740 4 client.15537 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-20T16:14:59.548 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:59.543+0000 7f71aab6c740 1 -- 172.21.15.82:0/3067939699 --> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d40745cdc0 con 0x55d4070e51c0 2024-05-20T16:14:59.548 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:59.543+0000 7f71a1986640 1 --2- 172.21.15.82:0/3067939699 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x55d4070e51c0 0x55d407571fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:14:59.548 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:59.544+0000 7f71857fa640 10 client.15537 ms_handle_connect on v2:172.21.15.160:6838/2765554482 2024-05-20T16:14:59.548 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:59.544+0000 7f71a1986640 1 --2- 172.21.15.82:0/3067939699 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x55d4070e51c0 0x55d407571fd0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d8ae03592e07331f server_cookie=e2921e2943af6d64 in_seq=0 out_seq=0 2024-05-20T16:14:59.549 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:59.548+0000 7f71857fa640 1 -- 172.21.15.82:0/3067939699 <== mds.0 v2:172.21.15.160:6838/2765554482 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d40745cdc0 con 0x55d4070e51c0 2024-05-20T16:14:59.549 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:59.548+0000 7f71857fa640 10 client.15537 handle_command_reply: tid=0 2024-05-20T16:14:59.549 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:59.548+0000 7f71aab6c740 1 client.15537 shutdown 2024-05-20T16:14:59.549 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:59.548+0000 7f71aab6c740 1 -- 172.21.15.82:0/3067939699 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d4071304d0 msgr2=0x55d4074fe7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:14:59.549 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:59.548+0000 7f71aab6c740 1 --2- 172.21.15.82:0/3067939699 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d4071304d0 0x55d4074fe7d0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f718c0bba80 tx=0x7f718c03b000 comp rx=0 tx=0).stop 2024-05-20T16:14:59.549 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:59.548+0000 7f71aab6c740 1 -- 172.21.15.82:0/3067939699 shutdown_connections 2024-05-20T16:14:59.549 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:59.548+0000 7f71aab6c740 1 --2- 172.21.15.82:0/3067939699 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x55d4070e51c0 0x55d407571fd0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:59.549 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:59.548+0000 7f71aab6c740 1 --2- 172.21.15.82:0/3067939699 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d4071304d0 0x55d4074fe7d0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:59.549 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:59.548+0000 7f71aab6c740 1 --2- 172.21.15.82:0/3067939699 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d4071511b0 0x55d40756e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:59.550 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:59.548+0000 7f71aab6c740 1 --2- 172.21.15.82:0/3067939699 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d4071282f0 0x55d4074fe290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:59.550 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:59.548+0000 7f71857fa640 0 client.15537 ms_handle_reset on v2:172.21.15.160:6838/2765554482 2024-05-20T16:14:59.550 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:59.548+0000 7f71aab6c740 1 -- 172.21.15.82:0/3067939699 >> 172.21.15.82:0/3067939699 conn(0x55d407145300 msgr2=0x55d4074fa760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:14:59.550 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:59.548+0000 7f71aab6c740 1 -- 172.21.15.82:0/3067939699 shutdown_connections 2024-05-20T16:14:59.550 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:59.548+0000 7f71aab6c740 1 -- 172.21.15.82:0/3067939699 wait complete. 2024-05-20T16:14:59.550 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:59.548+0000 7f71aab6c740 20 client.15537 trim_cache size 0 max 16384 2024-05-20T16:14:59.550 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:{ 2024-05-20T16:14:59.550 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "message": "", 2024-05-20T16:14:59.550 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "return_code": 0 2024-05-20T16:14:59.550 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:} 2024-05-20T16:14:59.551 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:59.549+0000 7f71a3bea640 1 -- 172.21.15.82:0/3963361528 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f71880a2c90 msgr2=0x7f71880a51a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:14:59.551 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:59.549+0000 7f71a3bea640 1 --2- 172.21.15.82:0/3963361528 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f71880a2c90 0x7f71880a51a0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x55d4074f2030 tx=0x7f7194009500 comp rx=0 tx=0).stop 2024-05-20T16:14:59.551 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:59.549+0000 7f71a3bea640 1 -- 172.21.15.82:0/3963361528 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f719c102c40 msgr2=0x7f719c19e0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:14:59.551 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:59.549+0000 7f71a3bea640 1 --2- 172.21.15.82:0/3963361528 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f719c102c40 0x7f719c19e0b0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f718c031440 tx=0x7f718c032ce0 comp rx=0 tx=0).stop 2024-05-20T16:14:59.551 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:59.549+0000 7f71a3bea640 1 -- 172.21.15.82:0/3963361528 shutdown_connections 2024-05-20T16:14:59.551 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:59.549+0000 7f71a3bea640 1 --2- 172.21.15.82:0/3963361528 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f71880a2c90 0x7f71880a51a0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:59.552 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:59.549+0000 7f71a3bea640 1 --2- 172.21.15.82:0/3963361528 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f719c102c40 0x7f719c19e0b0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:59.552 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:59.549+0000 7f71a3bea640 1 --2- 172.21.15.82:0/3963361528 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f719c107940 0x7f719c1a0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:59.552 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:59.549+0000 7f71a3bea640 1 --2- 172.21.15.82:0/3963361528 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f719c101a00 0x7f719c06ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:14:59.552 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:59.549+0000 7f71a3bea640 1 -- 172.21.15.82:0/3963361528 >> 172.21.15.82:0/3963361528 conn(0x7f719c0fcc10 msgr2=0x7f719c103920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:14:59.552 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:59.549+0000 7f71a3bea640 1 -- 172.21.15.82:0/3963361528 shutdown_connections 2024-05-20T16:14:59.552 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:14:59.549+0000 7f71a3bea640 1 -- 172.21.15.82:0/3963361528 wait complete. 2024-05-20T16:14:59.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:14:59 smithi160 ceph-mon[36347]: pgmap v392: 129 pgs: 129 active+clean; 668 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.4 KiB/s wr, 8 op/s 2024-05-20T16:14:59.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:14:59 smithi082 ceph-mon[32238]: pgmap v392: 129 pgs: 129 active+clean; 668 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.4 KiB/s wr, 8 op/s 2024-05-20T16:14:59.885 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:14:59 smithi184 ceph-mon[36209]: pgmap v392: 129 pgs: 129 active+clean; 668 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.4 KiB/s wr, 8 op/s 2024-05-20T16:15:01.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:15:01 smithi160 ceph-mon[36347]: pgmap v393: 129 pgs: 129 active+clean; 668 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2024-05-20T16:15:01.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:15:01 smithi082 ceph-mon[32238]: pgmap v393: 129 pgs: 129 active+clean; 668 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2024-05-20T16:15:01.885 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:15:01 smithi184 ceph-mon[36209]: pgmap v393: 129 pgs: 129 active+clean; 668 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2024-05-20T16:15:03.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:15:03 smithi160 ceph-mon[36347]: pgmap v394: 129 pgs: 129 active+clean; 668 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-20T16:15:03.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:15:03 smithi082 ceph-mon[32238]: pgmap v394: 129 pgs: 129 active+clean; 668 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-20T16:15:03.885 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:15:03 smithi184 ceph-mon[36209]: pgmap v394: 129 pgs: 129 active+clean; 668 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-20T16:15:05.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:15:05 smithi160 ceph-mon[36347]: pgmap v395: 129 pgs: 129 active+clean; 668 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-20T16:15:05.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:15:05 smithi082 ceph-mon[32238]: pgmap v395: 129 pgs: 129 active+clean; 668 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-20T16:15:05.885 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:15:05 smithi184 ceph-mon[36209]: pgmap v395: 129 pgs: 129 active+clean; 668 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-20T16:15:07.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:15:07 smithi160 ceph-mon[36347]: pgmap v396: 129 pgs: 129 active+clean; 668 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-20T16:15:07.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:15:07 smithi082 ceph-mon[32238]: pgmap v396: 129 pgs: 129 active+clean; 668 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-20T16:15:07.885 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:15:07 smithi184 ceph-mon[36209]: pgmap v396: 129 pgs: 129 active+clean; 668 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-20T16:15:09.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:15:09 smithi160 ceph-mon[36347]: pgmap v397: 129 pgs: 129 active+clean; 668 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-20T16:15:09.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:15:09 smithi082 ceph-mon[32238]: pgmap v397: 129 pgs: 129 active+clean; 668 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-20T16:15:09.885 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:15:09 smithi184 ceph-mon[36209]: pgmap v397: 129 pgs: 129 active+clean; 668 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-20T16:15:11.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:15:11 smithi160 ceph-mon[36347]: pgmap v398: 129 pgs: 129 active+clean; 668 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-20T16:15:11.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:15:11 smithi082 ceph-mon[32238]: pgmap v398: 129 pgs: 129 active+clean; 668 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-20T16:15:11.885 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:15:11 smithi184 ceph-mon[36209]: pgmap v398: 129 pgs: 129 active+clean; 668 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-20T16:15:12.700 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:12.700+0000 7fa5eec79640 1 -- 172.21.15.82:0/253771855 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa5e8108780 msgr2=0x7fa5e810ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:15:12.700 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:12.700+0000 7fa5eec79640 1 --2- 172.21.15.82:0/253771855 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa5e8108780 0x7fa5e810ac90 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fa5e0009ca0 tx=0x7fa5e002a8c0 comp rx=0 tx=0).stop 2024-05-20T16:15:12.700 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:12.700+0000 7fa5eec79640 1 -- 172.21.15.82:0/253771855 shutdown_connections 2024-05-20T16:15:12.700 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:12.700+0000 7fa5eec79640 1 --2- 172.21.15.82:0/253771855 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa5e8108780 0x7fa5e810ac90 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:15:12.700 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:12.700+0000 7fa5eec79640 1 --2- 172.21.15.82:0/253771855 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa5e8105c10 0x7fa5e81080f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:15:12.700 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:12.700+0000 7fa5eec79640 1 --2- 172.21.15.82:0/253771855 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa5e8100e30 0x7fa5e8101240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:15:12.700 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:12.700+0000 7fa5eec79640 1 -- 172.21.15.82:0/253771855 >> 172.21.15.82:0/253771855 conn(0x7fa5e80fc340 msgr2=0x7fa5e80fe810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:15:12.701 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:12.700+0000 7fa5eec79640 1 -- 172.21.15.82:0/253771855 shutdown_connections 2024-05-20T16:15:12.701 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:12.700+0000 7fa5eec79640 1 -- 172.21.15.82:0/253771855 wait complete. 2024-05-20T16:15:12.703 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:12.701+0000 7fa5eec79640 1 Processor -- start 2024-05-20T16:15:12.703 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:12.701+0000 7fa5eec79640 1 -- start start 2024-05-20T16:15:12.703 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:12.701+0000 7fa5eec79640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa5e8100e30 0x7fa5e8199270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:15:12.703 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:12.701+0000 7fa5eec79640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa5e8105c10 0x7fa5e81997b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:15:12.703 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:12.701+0000 7fa5eec79640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa5e8108780 0x7fa5e81a0740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:15:12.703 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:12.701+0000 7fa5eec79640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fa5e810e760 con 0x7fa5e8100e30 2024-05-20T16:15:12.703 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:12.701+0000 7fa5eec79640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fa5e810e5e0 con 0x7fa5e8108780 2024-05-20T16:15:12.703 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:12.701+0000 7fa5eec79640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa5e810e8e0 con 0x7fa5e8105c10 2024-05-20T16:15:12.703 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:12.701+0000 7fa5eca15640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa5e8100e30 0x7fa5e8199270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:15:12.703 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:12.701+0000 7fa5eca15640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa5e8100e30 0x7fa5e8199270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41882/0 (socket says 172.21.15.82:41882) 2024-05-20T16:15:12.703 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:12.701+0000 7fa5eca15640 1 -- 172.21.15.82:0/3613853651 learned_addr learned my addr 172.21.15.82:0/3613853651 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:15:12.703 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:12.701+0000 7fa5e7fff640 1 --2- 172.21.15.82:0/3613853651 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa5e8105c10 0x7fa5e81997b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:15:12.703 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:12.701+0000 7fa5ed216640 1 --2- 172.21.15.82:0/3613853651 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa5e8108780 0x7fa5e81a0740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:15:12.703 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:12.701+0000 7fa5eca15640 1 -- 172.21.15.82:0/3613853651 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa5e8105c10 msgr2=0x7fa5e81997b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:15:12.704 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:12.701+0000 7fa5eca15640 1 --2- 172.21.15.82:0/3613853651 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa5e8105c10 0x7fa5e81997b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:15:12.704 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:12.701+0000 7fa5eca15640 1 -- 172.21.15.82:0/3613853651 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa5e8108780 msgr2=0x7fa5e81a0740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:15:12.704 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:12.701+0000 7fa5eca15640 1 --2- 172.21.15.82:0/3613853651 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa5e8108780 0x7fa5e81a0740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:15:12.704 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:12.701+0000 7fa5eca15640 1 -- 172.21.15.82:0/3613853651 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5e81a0db0 con 0x7fa5e8100e30 2024-05-20T16:15:12.704 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:12.702+0000 7fa5eca15640 1 --2- 172.21.15.82:0/3613853651 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa5e8100e30 0x7fa5e8199270 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fa5dc00e9f0 tx=0x7fa5dc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:15:12.704 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:12.702+0000 7fa5e5ffb640 1 -- 172.21.15.82:0/3613853651 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5dc00ccc0 con 0x7fa5e8100e30 2024-05-20T16:15:12.704 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:12.702+0000 7fa5eec79640 1 -- 172.21.15.82:0/3613853651 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5e81018e0 con 0x7fa5e8100e30 2024-05-20T16:15:12.704 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:12.702+0000 7fa5e5ffb640 1 -- 172.21.15.82:0/3613853651 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa5dc005500 con 0x7fa5e8100e30 2024-05-20T16:15:12.704 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:12.702+0000 7fa5e5ffb640 1 -- 172.21.15.82:0/3613853651 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5dc010690 con 0x7fa5e8100e30 2024-05-20T16:15:12.704 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:12.702+0000 7fa5eec79640 1 -- 172.21.15.82:0/3613853651 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5e8101de0 con 0x7fa5e8100e30 2024-05-20T16:15:12.705 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:12.705+0000 7fa5e5ffb640 1 -- 172.21.15.82:0/3613853651 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 19) ==== 137884+0+0 (secure 0 0 0) 0x7fa5dc0046c0 con 0x7fa5e8100e30 2024-05-20T16:15:12.705 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:12.706+0000 7fa5e5ffb640 1 --2- 172.21.15.82:0/3613853651 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fa5d80a2e10 0x7fa5d80a5320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:15:12.706 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:12.706+0000 7fa5f5bf5740 1 Processor -- start 2024-05-20T16:15:12.706 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:12.706+0000 7fa5e5ffb640 1 -- 172.21.15.82:0/3613853651 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9150+0+0 (secure 0 0 0) 0x7fa5dc010430 con 0x7fa5e8100e30 2024-05-20T16:15:12.706 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:12.706+0000 7fa5e7fff640 1 --2- 172.21.15.82:0/3613853651 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fa5d80a2e10 0x7fa5d80a5320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:15:12.706 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:12.706+0000 7fa5f5bf5740 1 -- start start 2024-05-20T16:15:12.707 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:12.706+0000 7fa5f5bf5740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563e5cee01b0 0x563e5d288670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:15:12.707 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:12.706+0000 7fa5f5bf5740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563e5ceb72f0 0x563e5d288bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:15:12.707 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:12.706+0000 7fa5f5bf5740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563e5cebf4d0 0x563e5d28faf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:15:12.707 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:12.706+0000 7fa5f5bf5740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x563e5d1e4340 con 0x563e5cee01b0 2024-05-20T16:15:12.707 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:12.706+0000 7fa5f5bf5740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x563e5d18c290 con 0x563e5ceb72f0 2024-05-20T16:15:12.707 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:12.706+0000 7fa5f5bf5740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x563e5d1e0300 con 0x563e5cebf4d0 2024-05-20T16:15:12.707 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:12.706+0000 7fa5eca15640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563e5cee01b0 0x563e5d288670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:15:12.707 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:12.706+0000 7fa5eca15640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563e5cee01b0 0x563e5d288670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41898/0 (socket says 172.21.15.82:41898) 2024-05-20T16:15:12.708 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:12.707+0000 7fa5eca15640 1 -- 172.21.15.82:0/1651391301 learned_addr learned my addr 172.21.15.82:0/1651391301 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:15:12.708 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:12.707+0000 7fa5e7fff640 1 --2- 172.21.15.82:0/1651391301 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563e5ceb72f0 0x563e5d288bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:15:12.708 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:12.707+0000 7fa5e7fff640 1 --2- 172.21.15.82:0/3613853651 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fa5d80a2e10 0x7fa5d80a5320 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fa5e819a310 tx=0x7fa5d4005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:15:12.708 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:12.707+0000 7fa5ed216640 1 --2- 172.21.15.82:0/1651391301 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563e5cebf4d0 0x563e5d28faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:15:12.708 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:12.707+0000 7fa5eca15640 1 -- 172.21.15.82:0/1651391301 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563e5cebf4d0 msgr2=0x563e5d28faf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:15:12.708 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:12.707+0000 7fa5eca15640 1 --2- 172.21.15.82:0/1651391301 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563e5cebf4d0 0x563e5d28faf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:15:12.708 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:12.707+0000 7fa5eca15640 1 -- 172.21.15.82:0/1651391301 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563e5ceb72f0 msgr2=0x563e5d288bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:15:12.708 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:12.707+0000 7fa5eca15640 1 --2- 172.21.15.82:0/1651391301 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563e5ceb72f0 0x563e5d288bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:15:12.708 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:12.707+0000 7fa5eca15640 1 -- 172.21.15.82:0/1651391301 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563e5d1b08e0 con 0x563e5cee01b0 2024-05-20T16:15:12.708 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:12.707+0000 7fa5eca15640 1 --2- 172.21.15.82:0/1651391301 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563e5cee01b0 0x563e5d288670 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fa5dc00cfd0 tx=0x7fa5dc09b530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50ac7b4e94f6298a server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:15:12.708 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:12.707+0000 7fa5d2ffd640 1 -- 172.21.15.82:0/1651391301 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5dc09bb10 con 0x563e5cee01b0 2024-05-20T16:15:12.708 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:12.707+0000 7fa5d2ffd640 1 -- 172.21.15.82:0/1651391301 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa5dc09fec0 con 0x563e5cee01b0 2024-05-20T16:15:12.708 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:12.707+0000 7fa5d2ffd640 1 -- 172.21.15.82:0/1651391301 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5dc09af60 con 0x563e5cee01b0 2024-05-20T16:15:12.712 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:12.708+0000 7fa5f5bf5740 1 -- 172.21.15.82:0/1651391301 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563e5cee01b0 msgr2=0x563e5d288670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:15:12.712 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:12.708+0000 7fa5f5bf5740 1 --2- 172.21.15.82:0/1651391301 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563e5cee01b0 0x563e5d288670 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fa5dc00cfd0 tx=0x7fa5dc09b530 comp rx=0 tx=0).stop 2024-05-20T16:15:12.712 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:12.708+0000 7fa5f5bf5740 1 -- 172.21.15.82:0/1651391301 shutdown_connections 2024-05-20T16:15:12.712 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:12.708+0000 7fa5f5bf5740 1 --2- 172.21.15.82:0/1651391301 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563e5cebf4d0 0x563e5d28faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:15:12.712 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:12.708+0000 7fa5f5bf5740 1 --2- 172.21.15.82:0/1651391301 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563e5ceb72f0 0x563e5d288bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:15:12.712 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:12.708+0000 7fa5f5bf5740 1 --2- 172.21.15.82:0/1651391301 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563e5cee01b0 0x563e5d288670 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:15:12.712 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:12.708+0000 7fa5f5bf5740 1 -- 172.21.15.82:0/1651391301 >> 172.21.15.82:0/1651391301 conn(0x563e5ced4300 msgr2=0x563e5d27b7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:15:12.712 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:12.708+0000 7fa5f5bf5740 1 -- 172.21.15.82:0/1651391301 shutdown_connections 2024-05-20T16:15:12.712 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:12.708+0000 7fa5f5bf5740 1 -- 172.21.15.82:0/1651391301 wait complete. 2024-05-20T16:15:12.713 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:12.709+0000 7fa5f5bf5740 1 -- start start 2024-05-20T16:15:12.713 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:12.709+0000 7fa5f5bf5740 1 Processor -- start 2024-05-20T16:15:12.713 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:12.709+0000 7fa5f5bf5740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563e5ceb72f0 0x563e5d2923a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:15:12.713 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:12.709+0000 7fa5f5bf5740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563e5cebf4d0 0x563e5d2928e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:15:12.713 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:12.709+0000 7fa5f5bf5740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563e5cee01b0 0x563e5d299840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:15:12.713 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:12.709+0000 7fa5f5bf5740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x563e5d290480 con 0x563e5cee01b0 2024-05-20T16:15:12.713 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:12.709+0000 7fa5f5bf5740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x563e5d19f790 con 0x563e5ceb72f0 2024-05-20T16:15:12.713 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:12.709+0000 7fa5f5bf5740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x563e5d290600 con 0x563e5cebf4d0 2024-05-20T16:15:12.713 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:12.709+0000 7fa5f5bf5740 10 client.0 _ll_register_callbacks cb 0x563e5d1daa20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T16:15:12.714 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:12.709+0000 7fa5ed216640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563e5cee01b0 0x563e5d299840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:15:12.714 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:12.709+0000 7fa5ed216640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563e5cee01b0 0x563e5d299840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41908/0 (socket says 172.21.15.82:41908) 2024-05-20T16:15:12.714 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:12.709+0000 7fa5ed216640 1 -- 172.21.15.82:0/1135048552 learned_addr learned my addr 172.21.15.82:0/1135048552 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:15:12.714 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:12.710+0000 7fa5eca15640 1 --2- 172.21.15.82:0/1135048552 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563e5ceb72f0 0x563e5d2923a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:15:12.714 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:12.710+0000 7fa5e7fff640 1 --2- 172.21.15.82:0/1135048552 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563e5cebf4d0 0x563e5d2928e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:15:12.714 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:12.710+0000 7fa5ed216640 1 -- 172.21.15.82:0/1135048552 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563e5cebf4d0 msgr2=0x563e5d2928e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:15:12.714 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:12.710+0000 7fa5ed216640 1 --2- 172.21.15.82:0/1135048552 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563e5cebf4d0 0x563e5d2928e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:15:12.715 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:12.710+0000 7fa5ed216640 1 -- 172.21.15.82:0/1135048552 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563e5ceb72f0 msgr2=0x563e5d2923a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:15:12.715 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:12.710+0000 7fa5ed216640 1 --2- 172.21.15.82:0/1135048552 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563e5ceb72f0 0x563e5d2923a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:15:12.715 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:12.710+0000 7fa5ed216640 1 -- 172.21.15.82:0/1135048552 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563e5cef8210 con 0x563e5cee01b0 2024-05-20T16:15:12.715 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:12.710+0000 7fa5eca15640 1 --2- 172.21.15.82:0/1135048552 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563e5ceb72f0 0x563e5d2923a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T16:15:12.715 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:12.710+0000 7fa5ed216640 1 --2- 172.21.15.82:0/1135048552 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563e5cee01b0 0x563e5d299840 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fa5e0035000 tx=0x7fa5e0031910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=664b400493847ff9 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:15:12.715 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:12.710+0000 7fa5c7fff640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-05-20T16:15:12.716 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:12.710+0000 7fa5c7fff640 1 -- 172.21.15.82:0/1135048552 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5e0049060 con 0x563e5cee01b0 2024-05-20T16:15:12.716 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:12.710+0000 7fa5c7fff640 1 -- 172.21.15.82:0/1135048552 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9150+0+0 (secure 0 0 0) 0x7fa5e0003970 con 0x563e5cee01b0 2024-05-20T16:15:12.716 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:12.710+0000 7fa5f5bf5740 1 -- 172.21.15.82:0/1135048552 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa5dc09fec0 con 0x563e5cee01b0 2024-05-20T16:15:12.716 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:12.711+0000 7fa5c7fff640 1 -- 172.21.15.82:0/1135048552 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa5e0050050 con 0x563e5cee01b0 2024-05-20T16:15:12.716 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:12.711+0000 7fa5f5bf5740 10 client.15549 fetch_fsmap learned FSMap version 25 2024-05-20T16:15:12.716 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:12.711+0000 7fa5f5bf5740 1 -- 172.21.15.82:0/1135048552 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x563e5cfc4620 con 0x563e5cee01b0 2024-05-20T16:15:12.716 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:12.711+0000 7fa5c7fff640 1 -- 172.21.15.82:0/1135048552 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa5dc09fec0 con 0x563e5cee01b0 2024-05-20T16:15:12.716 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:12.711+0000 7fa5c7fff640 1 -- 172.21.15.82:0/1135048552 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fa5e002f610 con 0x563e5cee01b0 2024-05-20T16:15:12.716 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:12.711+0000 7fa5f5bf5740 10 client.15549 fetch_fsmap finished waiting for FSMap version 25 2024-05-20T16:15:12.716 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:12.711+0000 7fa5f5bf5740 10 client.15549 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T16:15:12.716 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:12.711+0000 7fa5f5bf5740 20 client.15549 populate_metadata read hostname 'smithi082' 2024-05-20T16:15:12.716 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:12.711+0000 7fa5f5bf5740 1 --2- 172.21.15.82:0/1135048552 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x563e5ce9ed80 0x563e5d29fd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:15:12.716 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:12.711+0000 7fa5f5bf5740 4 client.15549 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T16:15:12.717 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:12.711+0000 7fa5f5bf5740 1 -- 172.21.15.82:0/1135048552 --> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563e5d1f1a70 con 0x563e5ce9ed80 2024-05-20T16:15:12.717 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:12.711+0000 7fa5eca15640 1 --2- 172.21.15.82:0/1135048552 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x563e5ce9ed80 0x563e5d29fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:15:12.717 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:12.712+0000 7fa5c7fff640 10 client.15549 ms_handle_connect on v2:172.21.15.160:6838/2765554482 2024-05-20T16:15:12.717 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:12.712+0000 7fa5eca15640 1 --2- 172.21.15.82:0/1135048552 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x563e5ce9ed80 0x563e5d29fd80 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d88f3d3b65c09453 server_cookie=e89308e2f8107b51 in_seq=0 out_seq=0 2024-05-20T16:15:12.718 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:12.713+0000 7fa5c7fff640 1 -- 172.21.15.82:0/1135048552 <== mds.0 v2:172.21.15.160:6838/2765554482 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x563e5d1f1a70 con 0x563e5ce9ed80 2024-05-20T16:15:12.718 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:12.713+0000 7fa5c7fff640 10 client.15549 handle_command_reply: tid=0 2024-05-20T16:15:12.718 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:12.713+0000 7fa5f5bf5740 1 client.15549 shutdown 2024-05-20T16:15:12.718 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:12.713+0000 7fa5f5bf5740 1 -- 172.21.15.82:0/1135048552 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563e5cee01b0 msgr2=0x563e5d299840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:15:12.718 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:12.713+0000 7fa5f5bf5740 1 --2- 172.21.15.82:0/1135048552 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563e5cee01b0 0x563e5d299840 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fa5e0035000 tx=0x7fa5e0031910 comp rx=0 tx=0).stop 2024-05-20T16:15:12.718 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:12.713+0000 7fa5f5bf5740 1 -- 172.21.15.82:0/1135048552 shutdown_connections 2024-05-20T16:15:12.718 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:12.713+0000 7fa5f5bf5740 1 --2- 172.21.15.82:0/1135048552 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x563e5ce9ed80 0x563e5d29fd80 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:15:12.718 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:12.713+0000 7fa5f5bf5740 1 --2- 172.21.15.82:0/1135048552 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563e5cee01b0 0x563e5d299840 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:15:12.719 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:12.713+0000 7fa5f5bf5740 1 --2- 172.21.15.82:0/1135048552 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563e5cebf4d0 0x563e5d2928e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:15:12.719 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:12.713+0000 7fa5c7fff640 0 client.15549 ms_handle_reset on v2:172.21.15.160:6838/2765554482 2024-05-20T16:15:12.719 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:12.713+0000 7fa5f5bf5740 1 --2- 172.21.15.82:0/1135048552 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563e5ceb72f0 0x563e5d2923a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:15:12.719 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:12.713+0000 7fa5f5bf5740 1 -- 172.21.15.82:0/1135048552 >> 172.21.15.82:0/1135048552 conn(0x563e5ced4300 msgr2=0x563e5d27a810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:15:12.719 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:12.713+0000 7fa5f5bf5740 1 -- 172.21.15.82:0/1135048552 shutdown_connections 2024-05-20T16:15:12.719 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:12.713+0000 7fa5f5bf5740 1 -- 172.21.15.82:0/1135048552 wait complete. 2024-05-20T16:15:12.720 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:12.714+0000 7fa5f5bf5740 20 client.15549 trim_cache size 0 max 16384 2024-05-20T16:15:12.730 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:12.729+0000 7fa5f5bf5740 1 Processor -- start 2024-05-20T16:15:12.730 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:12.729+0000 7fa5f5bf5740 1 -- start start 2024-05-20T16:15:12.730 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:12.729+0000 7fa5f5bf5740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563e5ceb72f0 0x563e5d292ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:15:12.730 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:12.729+0000 7fa5f5bf5740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563e5cebf4d0 0x563e5d299760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:15:12.730 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:12.729+0000 7fa5f5bf5740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563e5cee01b0 0x563e5d299ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:15:12.730 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:12.729+0000 7fa5f5bf5740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fa5e81ac120 con 0x563e5cee01b0 2024-05-20T16:15:12.730 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:12.729+0000 7fa5f5bf5740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fa5e81a99f0 con 0x563e5cebf4d0 2024-05-20T16:15:12.730 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:12.729+0000 7fa5f5bf5740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa5e81a7840 con 0x563e5ceb72f0 2024-05-20T16:15:12.730 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:12.730+0000 7fa5ed216640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563e5cee01b0 0x563e5d299ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:15:12.730 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:12.730+0000 7fa5ed216640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563e5cee01b0 0x563e5d299ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41920/0 (socket says 172.21.15.82:41920) 2024-05-20T16:15:12.731 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:12.730+0000 7fa5ed216640 1 -- 172.21.15.82:0/563696028 learned_addr learned my addr 172.21.15.82:0/563696028 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:15:12.731 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:12.730+0000 7fa5eca15640 1 --2- 172.21.15.82:0/563696028 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563e5ceb72f0 0x563e5d292ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:15:12.731 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:12.730+0000 7fa5e7fff640 1 --2- 172.21.15.82:0/563696028 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563e5cebf4d0 0x563e5d299760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:15:12.731 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:12.730+0000 7fa5ed216640 1 -- 172.21.15.82:0/563696028 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563e5ceb72f0 msgr2=0x563e5d292ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:15:12.731 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:12.730+0000 7fa5ed216640 1 --2- 172.21.15.82:0/563696028 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563e5ceb72f0 0x563e5d292ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:15:12.731 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:12.730+0000 7fa5ed216640 1 -- 172.21.15.82:0/563696028 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563e5cebf4d0 msgr2=0x563e5d299760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:15:12.731 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:12.730+0000 7fa5ed216640 1 --2- 172.21.15.82:0/563696028 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563e5cebf4d0 0x563e5d299760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:15:12.731 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:12.730+0000 7fa5ed216640 1 -- 172.21.15.82:0/563696028 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563e5d292ff0 con 0x563e5cee01b0 2024-05-20T16:15:12.736 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:12.730+0000 7fa5eca15640 1 --2- 172.21.15.82:0/563696028 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563e5ceb72f0 0x563e5d292ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T16:15:12.736 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:12.730+0000 7fa5ed216640 1 --2- 172.21.15.82:0/563696028 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563e5cee01b0 0x563e5d299ca0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fa5e002a900 tx=0x7fa5e0009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59dfebd6106a10a5 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:15:12.736 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:12.731+0000 7fa5d27fc640 1 -- 172.21.15.82:0/563696028 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5e0059070 con 0x563e5cee01b0 2024-05-20T16:15:12.736 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:12.731+0000 7fa5d27fc640 1 -- 172.21.15.82:0/563696028 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa5e0036040 con 0x563e5cee01b0 2024-05-20T16:15:12.736 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:12.731+0000 7fa5d27fc640 1 -- 172.21.15.82:0/563696028 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5e0049050 con 0x563e5cee01b0 2024-05-20T16:15:12.737 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:12.732+0000 7fa5f5bf5740 1 -- 172.21.15.82:0/563696028 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563e5cee01b0 msgr2=0x563e5d299ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:15:12.737 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:12.732+0000 7fa5f5bf5740 1 --2- 172.21.15.82:0/563696028 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563e5cee01b0 0x563e5d299ca0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fa5e002a900 tx=0x7fa5e0009b20 comp rx=0 tx=0).stop 2024-05-20T16:15:12.737 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:12.732+0000 7fa5f5bf5740 1 -- 172.21.15.82:0/563696028 shutdown_connections 2024-05-20T16:15:12.737 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:12.732+0000 7fa5f5bf5740 1 --2- 172.21.15.82:0/563696028 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563e5cee01b0 0x563e5d299ca0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:15:12.737 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:12.732+0000 7fa5f5bf5740 1 --2- 172.21.15.82:0/563696028 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563e5cebf4d0 0x563e5d299760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:15:12.737 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:12.732+0000 7fa5f5bf5740 1 --2- 172.21.15.82:0/563696028 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563e5ceb72f0 0x563e5d292ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:15:12.737 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:12.732+0000 7fa5f5bf5740 1 -- 172.21.15.82:0/563696028 >> 172.21.15.82:0/563696028 conn(0x563e5ced4300 msgr2=0x563e5d2923a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:15:12.737 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:12.732+0000 7fa5f5bf5740 1 -- 172.21.15.82:0/563696028 shutdown_connections 2024-05-20T16:15:12.737 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:12.732+0000 7fa5f5bf5740 1 -- 172.21.15.82:0/563696028 wait complete. 2024-05-20T16:15:12.737 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:12.732+0000 7fa5f5bf5740 1 -- start start 2024-05-20T16:15:12.737 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:12.733+0000 7fa5f5bf5740 1 Processor -- start 2024-05-20T16:15:12.737 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:12.733+0000 7fa5f5bf5740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563e5ceb72f0 0x563e5d299f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:15:12.737 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:12.733+0000 7fa5f5bf5740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563e5cebf4d0 0x563e5d296bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:15:12.737 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:12.733+0000 7fa5f5bf5740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563e5cee01b0 0x563e5d297240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:15:12.738 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:12.733+0000 7fa5f5bf5740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fa5e81a7eb0 con 0x563e5cee01b0 2024-05-20T16:15:12.738 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:12.733+0000 7fa5f5bf5740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fa5e81a9320 con 0x563e5ceb72f0 2024-05-20T16:15:12.738 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:12.733+0000 7fa5f5bf5740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa5e81a8580 con 0x563e5cebf4d0 2024-05-20T16:15:12.738 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:12.733+0000 7fa5eca15640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563e5ceb72f0 0x563e5d299f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:15:12.738 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:12.733+0000 7fa5f5bf5740 10 client.0 _ll_register_callbacks cb 0x7fa5d4005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T16:15:12.738 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:12.733+0000 7fa5eca15640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563e5ceb72f0 0x563e5d299f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.82:45664/0 (socket says 172.21.15.82:45664) 2024-05-20T16:15:12.738 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:12.733+0000 7fa5eca15640 1 -- 172.21.15.82:0/694428661 learned_addr learned my addr 172.21.15.82:0/694428661 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:15:12.738 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:12.733+0000 7fa5ed216640 1 --2- 172.21.15.82:0/694428661 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563e5cee01b0 0x563e5d297240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:15:12.738 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:12.734+0000 7fa5eca15640 1 -- 172.21.15.82:0/694428661 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563e5cebf4d0 msgr2=0x563e5d296bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:15:12.738 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:12.734+0000 7fa5e7fff640 1 --2- 172.21.15.82:0/694428661 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563e5cebf4d0 0x563e5d296bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:15:12.738 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:12.734+0000 7fa5eca15640 1 --2- 172.21.15.82:0/694428661 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563e5cebf4d0 0x563e5d296bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:15:12.738 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:12.734+0000 7fa5eca15640 1 -- 172.21.15.82:0/694428661 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563e5cee01b0 msgr2=0x563e5d297240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:15:12.738 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:12.734+0000 7fa5eca15640 1 --2- 172.21.15.82:0/694428661 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563e5cee01b0 0x563e5d297240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:15:12.738 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:12.734+0000 7fa5eca15640 1 -- 172.21.15.82:0/694428661 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563e5d306110 con 0x563e5ceb72f0 2024-05-20T16:15:12.739 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:12.734+0000 7fa5ed216640 1 --2- 172.21.15.82:0/694428661 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563e5cee01b0 0x563e5d297240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T16:15:12.739 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:12.734+0000 7fa5e7fff640 1 --2- 172.21.15.82:0/694428661 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563e5cebf4d0 0x563e5d296bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T16:15:12.739 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:12.734+0000 7fa5eca15640 1 --2- 172.21.15.82:0/694428661 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563e5ceb72f0 0x563e5d299f90 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fa5dc00e9c0 tx=0x7fa5dc09b5a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dc110c675649fea4 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:15:12.739 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:12.734+0000 7fa5c7fff640 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-20T16:15:12.739 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:12.734+0000 7fa5c7fff640 1 -- 172.21.15.82:0/694428661 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5dc098300 con 0x563e5ceb72f0 2024-05-20T16:15:12.739 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:12.735+0000 7fa5c7fff640 1 -- 172.21.15.82:0/694428661 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9150+0+0 (secure 0 0 0) 0x7fa5dc0ba2b0 con 0x563e5ceb72f0 2024-05-20T16:15:12.739 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:12.735+0000 7fa5f5bf5740 1 -- 172.21.15.82:0/694428661 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa5e0036040 con 0x563e5ceb72f0 2024-05-20T16:15:12.739 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:12.735+0000 7fa5c7fff640 1 -- 172.21.15.82:0/694428661 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa5dc09bb10 con 0x563e5ceb72f0 2024-05-20T16:15:12.739 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:12.735+0000 7fa5f5bf5740 10 client.25088 fetch_fsmap learned FSMap version 25 2024-05-20T16:15:12.739 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:12.735+0000 7fa5f5bf5740 1 -- 172.21.15.82:0/694428661 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x563e5d3073d0 con 0x563e5ceb72f0 2024-05-20T16:15:12.739 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:12.735+0000 7fa5c7fff640 1 -- 172.21.15.82:0/694428661 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa5e0036040 con 0x563e5ceb72f0 2024-05-20T16:15:12.739 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:12.735+0000 7fa5c7fff640 1 -- 172.21.15.82:0/694428661 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fa5dc0a42f0 con 0x563e5ceb72f0 2024-05-20T16:15:12.741 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:12.736+0000 7fa5f5bf5740 10 client.25088 fetch_fsmap finished waiting for FSMap version 25 2024-05-20T16:15:12.741 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:12.736+0000 7fa5f5bf5740 10 client.25088 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T16:15:12.741 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:12.736+0000 7fa5f5bf5740 20 client.25088 populate_metadata read hostname 'smithi082' 2024-05-20T16:15:12.741 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:12.736+0000 7fa5f5bf5740 1 --2- 172.21.15.82:0/694428661 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x563e5ce741c0 0x563e5d309950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:15:12.742 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:12.736+0000 7fa5f5bf5740 4 client.25088 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-20T16:15:12.742 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:12.736+0000 7fa5f5bf5740 1 -- 172.21.15.82:0/694428661 --> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563e5d1ebdc0 con 0x563e5ce741c0 2024-05-20T16:15:12.742 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:12.737+0000 7fa5e7fff640 1 --2- 172.21.15.82:0/694428661 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x563e5ce741c0 0x563e5d309950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:15:12.742 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:12.737+0000 7fa5e7fff640 1 --2- 172.21.15.82:0/694428661 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x563e5ce741c0 0x563e5d309950 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2cd57b1f33b7253f server_cookie=ac8e4605aef7cc47 in_seq=0 out_seq=0 2024-05-20T16:15:12.742 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:12.737+0000 7fa5c7fff640 10 client.25088 ms_handle_connect on v2:172.21.15.160:6838/2765554482 2024-05-20T16:15:12.742 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:12.741+0000 7fa5c7fff640 1 -- 172.21.15.82:0/694428661 <== mds.0 v2:172.21.15.160:6838/2765554482 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x563e5d1ebdc0 con 0x563e5ce741c0 2024-05-20T16:15:12.742 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:12.741+0000 7fa5c7fff640 10 client.25088 handle_command_reply: tid=0 2024-05-20T16:15:12.742 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:12.741+0000 7fa5f5bf5740 1 client.25088 shutdown 2024-05-20T16:15:12.742 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:12.741+0000 7fa5f5bf5740 1 -- 172.21.15.82:0/694428661 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563e5ceb72f0 msgr2=0x563e5d299f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:15:12.742 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:12.741+0000 7fa5f5bf5740 1 --2- 172.21.15.82:0/694428661 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563e5ceb72f0 0x563e5d299f90 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fa5dc00e9c0 tx=0x7fa5dc09b5a0 comp rx=0 tx=0).stop 2024-05-20T16:15:12.742 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:12.741+0000 7fa5f5bf5740 1 -- 172.21.15.82:0/694428661 shutdown_connections 2024-05-20T16:15:12.742 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:12.741+0000 7fa5f5bf5740 1 --2- 172.21.15.82:0/694428661 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x563e5ce741c0 0x563e5d309950 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:15:12.742 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:12.741+0000 7fa5f5bf5740 1 --2- 172.21.15.82:0/694428661 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563e5cee01b0 0x563e5d297240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:15:12.743 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:12.741+0000 7fa5f5bf5740 1 --2- 172.21.15.82:0/694428661 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563e5cebf4d0 0x563e5d296bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:15:12.743 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:12.741+0000 7fa5f5bf5740 1 --2- 172.21.15.82:0/694428661 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563e5ceb72f0 0x563e5d299f90 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:15:12.743 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:12.741+0000 7fa5c7fff640 0 client.25088 ms_handle_reset on v2:172.21.15.160:6838/2765554482 2024-05-20T16:15:12.743 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:12.741+0000 7fa5f5bf5740 1 -- 172.21.15.82:0/694428661 >> 172.21.15.82:0/694428661 conn(0x563e5ced4300 msgr2=0x563e5d2909d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:15:12.743 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:12.741+0000 7fa5f5bf5740 1 -- 172.21.15.82:0/694428661 shutdown_connections 2024-05-20T16:15:12.743 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:12.741+0000 7fa5f5bf5740 1 -- 172.21.15.82:0/694428661 wait complete. 2024-05-20T16:15:12.743 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:12.742+0000 7fa5f5bf5740 20 client.25088 trim_cache size 0 max 16384 2024-05-20T16:15:12.743 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:{ 2024-05-20T16:15:12.743 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "message": "", 2024-05-20T16:15:12.743 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "return_code": 0 2024-05-20T16:15:12.743 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:} 2024-05-20T16:15:12.744 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:12.742+0000 7fa5eec79640 1 -- 172.21.15.82:0/3613853651 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fa5d80a2e10 msgr2=0x7fa5d80a5320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:15:12.744 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:12.742+0000 7fa5eec79640 1 --2- 172.21.15.82:0/3613853651 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fa5d80a2e10 0x7fa5d80a5320 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fa5e819a310 tx=0x7fa5d4005f90 comp rx=0 tx=0).stop 2024-05-20T16:15:12.744 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:12.742+0000 7fa5eec79640 1 -- 172.21.15.82:0/3613853651 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa5e8100e30 msgr2=0x7fa5e8199270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:15:12.744 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:12.742+0000 7fa5eec79640 1 --2- 172.21.15.82:0/3613853651 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa5e8100e30 0x7fa5e8199270 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fa5dc00e9f0 tx=0x7fa5dc00eeb0 comp rx=0 tx=0).stop 2024-05-20T16:15:12.744 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:12.742+0000 7fa5eec79640 1 -- 172.21.15.82:0/3613853651 shutdown_connections 2024-05-20T16:15:12.744 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:12.742+0000 7fa5eec79640 1 --2- 172.21.15.82:0/3613853651 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fa5d80a2e10 0x7fa5d80a5320 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:15:12.744 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:12.742+0000 7fa5eec79640 1 --2- 172.21.15.82:0/3613853651 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa5e8108780 0x7fa5e81a0740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:15:12.745 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:12.742+0000 7fa5eec79640 1 --2- 172.21.15.82:0/3613853651 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa5e8105c10 0x7fa5e81997b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:15:12.745 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:12.742+0000 7fa5eec79640 1 --2- 172.21.15.82:0/3613853651 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa5e8100e30 0x7fa5e8199270 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:15:12.745 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:12.742+0000 7fa5eec79640 1 -- 172.21.15.82:0/3613853651 >> 172.21.15.82:0/3613853651 conn(0x7fa5e80fc340 msgr2=0x7fa5e8106790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:15:12.745 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:12.743+0000 7fa5eec79640 1 -- 172.21.15.82:0/3613853651 shutdown_connections 2024-05-20T16:15:12.745 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:12.743+0000 7fa5eec79640 1 -- 172.21.15.82:0/3613853651 wait complete. 2024-05-20T16:15:13.661 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:15:13 smithi082 ceph-mon[32238]: pgmap v399: 129 pgs: 129 active+clean; 668 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-20T16:15:13.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:15:13 smithi160 ceph-mon[36347]: pgmap v399: 129 pgs: 129 active+clean; 668 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-20T16:15:13.885 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:15:13 smithi184 ceph-mon[36209]: pgmap v399: 129 pgs: 129 active+clean; 668 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-20T16:15:15.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:15:15 smithi160 ceph-mon[36347]: pgmap v400: 129 pgs: 129 active+clean; 668 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-20T16:15:15.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:15:15 smithi082 ceph-mon[32238]: pgmap v400: 129 pgs: 129 active+clean; 668 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-20T16:15:15.886 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:15:15 smithi184 ceph-mon[36209]: pgmap v400: 129 pgs: 129 active+clean; 668 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-20T16:15:17.635 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:15:17 smithi184 ceph-mon[36209]: mgrmap e20: x(active, since 10m), standbys: z, y 2024-05-20T16:15:17.635 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:15:17 smithi184 ceph-mon[36209]: pgmap v401: 129 pgs: 129 active+clean; 668 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2024-05-20T16:15:17.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:15:17 smithi160 ceph-mon[36347]: mgrmap e20: x(active, since 10m), standbys: z, y 2024-05-20T16:15:17.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:15:17 smithi160 ceph-mon[36347]: pgmap v401: 129 pgs: 129 active+clean; 668 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2024-05-20T16:15:17.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:15:17 smithi082 ceph-mon[32238]: mgrmap e20: x(active, since 10m), standbys: z, y 2024-05-20T16:15:17.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:15:17 smithi082 ceph-mon[32238]: pgmap v401: 129 pgs: 129 active+clean; 668 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2024-05-20T16:15:19.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:15:19 smithi160 ceph-mon[36347]: pgmap v402: 129 pgs: 129 active+clean; 670 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 597 B/s wr, 8 op/s 2024-05-20T16:15:19.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:15:19 smithi082 ceph-mon[32238]: pgmap v402: 129 pgs: 129 active+clean; 670 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 597 B/s wr, 8 op/s 2024-05-20T16:15:19.885 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:15:19 smithi184 ceph-mon[36209]: pgmap v402: 129 pgs: 129 active+clean; 670 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 597 B/s wr, 8 op/s 2024-05-20T16:15:21.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:15:21 smithi160 ceph-mon[36347]: pgmap v403: 129 pgs: 129 active+clean; 670 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 6 op/s 2024-05-20T16:15:21.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:15:21 smithi082 ceph-mon[32238]: pgmap v403: 129 pgs: 129 active+clean; 670 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 6 op/s 2024-05-20T16:15:21.885 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:15:21 smithi184 ceph-mon[36209]: pgmap v403: 129 pgs: 129 active+clean; 670 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 6 op/s 2024-05-20T16:15:23.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:15:23 smithi160 ceph-mon[36347]: pgmap v404: 129 pgs: 129 active+clean; 670 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 597 B/s wr, 7 op/s 2024-05-20T16:15:23.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:15:23 smithi082 ceph-mon[32238]: pgmap v404: 129 pgs: 129 active+clean; 670 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 597 B/s wr, 7 op/s 2024-05-20T16:15:23.885 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:15:23 smithi184 ceph-mon[36209]: pgmap v404: 129 pgs: 129 active+clean; 670 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 597 B/s wr, 7 op/s 2024-05-20T16:15:25.688 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:15:25 smithi184 ceph-mon[36209]: pgmap v405: 129 pgs: 129 active+clean; 670 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 597 B/s wr, 6 op/s 2024-05-20T16:15:25.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:15:25 smithi160 ceph-mon[36347]: pgmap v405: 129 pgs: 129 active+clean; 670 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 597 B/s wr, 6 op/s 2024-05-20T16:15:25.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:15:25 smithi082 ceph-mon[32238]: pgmap v405: 129 pgs: 129 active+clean; 670 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 597 B/s wr, 6 op/s 2024-05-20T16:15:25.894 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:25.894+0000 7f3ba783d640 1 -- 172.21.15.82:0/3440346385 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3ba0102c10 msgr2=0x7f3ba0103020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:15:25.894 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:25.894+0000 7f3ba783d640 1 --2- 172.21.15.82:0/3440346385 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3ba0102c10 0x7f3ba0103020 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f3b90009900 tx=0x7f3b9002a410 comp rx=0 tx=0).stop 2024-05-20T16:15:25.894 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:25.894+0000 7f3ba783d640 1 -- 172.21.15.82:0/3440346385 shutdown_connections 2024-05-20T16:15:25.894 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:25.894+0000 7f3ba783d640 1 --2- 172.21.15.82:0/3440346385 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3ba0102c10 0x7f3ba0103020 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:15:25.895 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:25.894+0000 7f3ba783d640 1 --2- 172.21.15.82:0/3440346385 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3ba0107910 0x7f3ba0109d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:15:25.895 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:25.894+0000 7f3ba783d640 1 --2- 172.21.15.82:0/3440346385 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3ba01019d0 0x7f3ba01026d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:15:25.895 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:25.894+0000 7f3ba783d640 1 -- 172.21.15.82:0/3440346385 >> 172.21.15.82:0/3440346385 conn(0x7f3ba00fcbc0 msgr2=0x7f3ba00ff070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:15:25.895 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:25.894+0000 7f3ba783d640 1 -- 172.21.15.82:0/3440346385 shutdown_connections 2024-05-20T16:15:25.895 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:25.894+0000 7f3ba783d640 1 -- 172.21.15.82:0/3440346385 wait complete. 2024-05-20T16:15:25.897 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:25.895+0000 7f3ba783d640 1 Processor -- start 2024-05-20T16:15:25.897 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:25.895+0000 7f3ba783d640 1 -- start start 2024-05-20T16:15:25.897 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:25.895+0000 7f3ba783d640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3ba01019d0 0x7f3ba01992c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:15:25.897 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:25.895+0000 7f3ba783d640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3ba0102c10 0x7f3ba0199800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:15:25.897 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:25.895+0000 7f3ba783d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3ba0107910 0x7f3ba01a0790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:15:25.897 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:25.895+0000 7f3ba783d640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f3ba010e750 con 0x7f3ba0107910 2024-05-20T16:15:25.897 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:25.895+0000 7f3ba783d640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f3ba010e5d0 con 0x7f3ba01019d0 2024-05-20T16:15:25.897 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:25.896+0000 7f3ba783d640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f3ba010e8d0 con 0x7f3ba0102c10 2024-05-20T16:15:25.897 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:25.896+0000 7f3ba5dda640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3ba0107910 0x7f3ba01a0790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:15:25.897 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:25.896+0000 7f3ba5dda640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3ba0107910 0x7f3ba01a0790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:35190/0 (socket says 172.21.15.82:35190) 2024-05-20T16:15:25.897 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:25.896+0000 7f3ba5dda640 1 -- 172.21.15.82:0/2244017410 learned_addr learned my addr 172.21.15.82:0/2244017410 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:15:25.897 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:25.896+0000 7f3ba55d9640 1 --2- 172.21.15.82:0/2244017410 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3ba01019d0 0x7f3ba01992c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:15:25.897 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:25.896+0000 7f3ba4dd8640 1 --2- 172.21.15.82:0/2244017410 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3ba0102c10 0x7f3ba0199800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:15:25.898 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:25.896+0000 7f3ba5dda640 1 -- 172.21.15.82:0/2244017410 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3ba0102c10 msgr2=0x7f3ba0199800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:15:25.898 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:25.896+0000 7f3ba5dda640 1 --2- 172.21.15.82:0/2244017410 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3ba0102c10 0x7f3ba0199800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:15:25.898 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:25.896+0000 7f3ba5dda640 1 -- 172.21.15.82:0/2244017410 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3ba01019d0 msgr2=0x7f3ba01992c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:15:25.898 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:25.896+0000 7f3ba5dda640 1 --2- 172.21.15.82:0/2244017410 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3ba01019d0 0x7f3ba01992c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:15:25.898 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:25.896+0000 7f3ba5dda640 1 -- 172.21.15.82:0/2244017410 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ba01a0e00 con 0x7f3ba0107910 2024-05-20T16:15:25.898 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:25.896+0000 7f3ba5dda640 1 --2- 172.21.15.82:0/2244017410 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3ba0107910 0x7f3ba01a0790 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f3b9800ea20 tx=0x7f3b9800eee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:15:25.898 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:25.896+0000 7f3b967fc640 1 -- 172.21.15.82:0/2244017410 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b98004d20 con 0x7f3ba0107910 2024-05-20T16:15:25.898 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:25.896+0000 7f3b967fc640 1 -- 172.21.15.82:0/2244017410 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3b98017e40 con 0x7f3ba0107910 2024-05-20T16:15:25.898 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:25.896+0000 7f3b967fc640 1 -- 172.21.15.82:0/2244017410 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b980059a0 con 0x7f3ba0107910 2024-05-20T16:15:25.898 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:25.896+0000 7f3ba783d640 1 -- 172.21.15.82:0/2244017410 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ba010a880 con 0x7f3ba0107910 2024-05-20T16:15:25.898 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:25.897+0000 7f3ba783d640 1 -- 172.21.15.82:0/2244017410 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ba010acc0 con 0x7f3ba0107910 2024-05-20T16:15:25.899 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:25.900+0000 7f3bae7be740 1 Processor -- start 2024-05-20T16:15:25.900 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:25.900+0000 7f3b967fc640 1 -- 172.21.15.82:0/2244017410 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f3b98004ec0 con 0x7f3ba0107910 2024-05-20T16:15:25.900 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:25.900+0000 7f3bae7be740 1 -- start start 2024-05-20T16:15:25.901 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:25.901+0000 7f3bae7be740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bfa33781b0 0x55bfa3717e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:15:25.901 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:25.901+0000 7f3bae7be740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bfa334f2f0 0x55bfa37183e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:15:25.901 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:25.901+0000 7f3bae7be740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bfa33574d0 0x55bfa371f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:15:25.901 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:25.901+0000 7f3bae7be740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55bfa367c340 con 0x55bfa33781b0 2024-05-20T16:15:25.901 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:25.901+0000 7f3bae7be740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x55bfa3624290 con 0x55bfa33574d0 2024-05-20T16:15:25.901 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:25.901+0000 7f3bae7be740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55bfa3678300 con 0x55bfa334f2f0 2024-05-20T16:15:25.902 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:25.901+0000 7f3ba4dd8640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bfa33781b0 0x55bfa3717e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:15:25.902 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:25.901+0000 7f3ba4dd8640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bfa33781b0 0x55bfa3717e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:35196/0 (socket says 172.21.15.82:35196) 2024-05-20T16:15:25.902 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:25.901+0000 7f3ba4dd8640 1 -- 172.21.15.82:0/2037456119 learned_addr learned my addr 172.21.15.82:0/2037456119 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:15:25.902 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:25.901+0000 7f3ba55d9640 1 --2- 172.21.15.82:0/2037456119 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bfa334f2f0 0x55bfa37183e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:15:25.902 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:25.901+0000 7f3ba4dd8640 1 --2- 172.21.15.82:0/2037456119 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bfa33574d0 0x55bfa371f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:15:25.902 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:25.901+0000 7f3b967fc640 1 --2- 172.21.15.82:0/2244017410 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f3b8c0a2b80 0x7f3b8c0a5090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:15:25.902 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:25.901+0000 7f3ba4dd8640 1 -- 172.21.15.82:0/2037456119 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bfa334f2f0 msgr2=0x55bfa37183e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:15:25.902 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:25.901+0000 7f3ba4dd8640 1 --2- 172.21.15.82:0/2037456119 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bfa334f2f0 0x55bfa37183e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:15:25.902 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:25.901+0000 7f3ba4dd8640 1 -- 172.21.15.82:0/2037456119 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bfa33574d0 msgr2=0x55bfa371f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:15:25.902 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:25.901+0000 7f3ba4dd8640 1 --2- 172.21.15.82:0/2037456119 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bfa33574d0 0x55bfa371f2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:15:25.902 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:25.901+0000 7f3ba4dd8640 1 -- 172.21.15.82:0/2037456119 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bfa36488e0 con 0x55bfa33781b0 2024-05-20T16:15:25.902 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:25.902+0000 7f3b967fc640 1 -- 172.21.15.82:0/2244017410 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9150+0+0 (secure 0 0 0) 0x7f3b980d7ae0 con 0x7f3ba0107910 2024-05-20T16:15:25.903 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:25.902+0000 7f3ba4dd8640 1 --2- 172.21.15.82:0/2037456119 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bfa33781b0 0x55bfa3717e60 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f3b900096c0 tx=0x7f3b900095a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a83df0985bf6 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:15:25.906 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:25.902+0000 7f3ba5dda640 1 --2- 172.21.15.82:0/2244017410 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f3b8c0a2b80 0x7f3b8c0a5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:15:25.906 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:25.902+0000 7f3ba5dda640 1 --2- 172.21.15.82:0/2244017410 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f3b8c0a2b80 0x7f3b8c0a5090 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f3b980a0410 tx=0x7f3b98005b40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:15:25.906 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:25.902+0000 7f3b8b7fe640 1 -- 172.21.15.82:0/2037456119 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b90035030 con 0x55bfa33781b0 2024-05-20T16:15:25.906 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:25.902+0000 7f3b8b7fe640 1 -- 172.21.15.82:0/2037456119 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3b90031830 con 0x55bfa33781b0 2024-05-20T16:15:25.906 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:25.902+0000 7f3b8b7fe640 1 -- 172.21.15.82:0/2037456119 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b900046c0 con 0x55bfa33781b0 2024-05-20T16:15:25.906 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:25.903+0000 7f3bae7be740 1 -- 172.21.15.82:0/2037456119 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bfa33781b0 msgr2=0x55bfa3717e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:15:25.907 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:25.903+0000 7f3bae7be740 1 --2- 172.21.15.82:0/2037456119 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bfa33781b0 0x55bfa3717e60 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f3b900096c0 tx=0x7f3b900095a0 comp rx=0 tx=0).stop 2024-05-20T16:15:25.907 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:25.903+0000 7f3bae7be740 1 -- 172.21.15.82:0/2037456119 shutdown_connections 2024-05-20T16:15:25.907 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:25.903+0000 7f3bae7be740 1 --2- 172.21.15.82:0/2037456119 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bfa33574d0 0x55bfa371f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:15:25.907 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:25.903+0000 7f3bae7be740 1 --2- 172.21.15.82:0/2037456119 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bfa334f2f0 0x55bfa37183e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:15:25.907 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:25.903+0000 7f3bae7be740 1 --2- 172.21.15.82:0/2037456119 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bfa33781b0 0x55bfa3717e60 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:15:25.907 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:25.903+0000 7f3bae7be740 1 -- 172.21.15.82:0/2037456119 >> 172.21.15.82:0/2037456119 conn(0x55bfa336c300 msgr2=0x55bfa37137f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:15:25.907 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:25.904+0000 7f3bae7be740 1 -- 172.21.15.82:0/2037456119 shutdown_connections 2024-05-20T16:15:25.907 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:25.904+0000 7f3bae7be740 1 -- 172.21.15.82:0/2037456119 wait complete. 2024-05-20T16:15:25.907 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:25.904+0000 7f3bae7be740 1 -- start start 2024-05-20T16:15:25.907 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:25.904+0000 7f3bae7be740 1 Processor -- start 2024-05-20T16:15:25.907 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:25.904+0000 7f3bae7be740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bfa334f2f0 0x55bfa3721c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:15:25.907 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:25.904+0000 7f3bae7be740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bfa33574d0 0x55bfa3722140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:15:25.907 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:25.904+0000 7f3bae7be740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bfa33781b0 0x55bfa37290a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:15:25.907 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:25.905+0000 7f3bae7be740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55bfa371fcd0 con 0x55bfa33574d0 2024-05-20T16:15:25.907 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:25.905+0000 7f3bae7be740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x55bfa3637790 con 0x55bfa334f2f0 2024-05-20T16:15:25.908 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:25.905+0000 7f3bae7be740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55bfa371fe50 con 0x55bfa33781b0 2024-05-20T16:15:25.908 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:25.905+0000 7f3bae7be740 10 client.0 _ll_register_callbacks cb 0x55bfa3672a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T16:15:25.908 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:25.905+0000 7f3ba55d9640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bfa33574d0 0x55bfa3722140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:15:25.908 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:25.905+0000 7f3ba55d9640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bfa33574d0 0x55bfa3722140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:35208/0 (socket says 172.21.15.82:35208) 2024-05-20T16:15:25.908 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:25.905+0000 7f3ba55d9640 1 -- 172.21.15.82:0/4173802880 learned_addr learned my addr 172.21.15.82:0/4173802880 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:15:25.908 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:25.905+0000 7f3ba4dd8640 1 --2- 172.21.15.82:0/4173802880 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bfa33781b0 0x55bfa37290a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:15:25.908 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:25.905+0000 7f3ba4dd8640 1 --2- 172.21.15.82:0/4173802880 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bfa334f2f0 0x55bfa3721c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:15:25.908 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:25.905+0000 7f3ba55d9640 1 -- 172.21.15.82:0/4173802880 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bfa33781b0 msgr2=0x55bfa37290a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:15:25.908 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:25.905+0000 7f3ba55d9640 1 --2- 172.21.15.82:0/4173802880 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bfa33781b0 0x55bfa37290a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:15:25.908 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:25.905+0000 7f3ba55d9640 1 -- 172.21.15.82:0/4173802880 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bfa334f2f0 msgr2=0x55bfa3721c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:15:25.908 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:25.905+0000 7f3ba55d9640 1 --2- 172.21.15.82:0/4173802880 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bfa334f2f0 0x55bfa3721c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:15:25.908 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:25.905+0000 7f3ba55d9640 1 -- 172.21.15.82:0/4173802880 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bfa3390210 con 0x55bfa33574d0 2024-05-20T16:15:25.908 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:25.905+0000 7f3ba4dd8640 1 --2- 172.21.15.82:0/4173802880 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bfa33781b0 0x55bfa37290a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T16:15:25.908 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:25.905+0000 7f3ba55d9640 1 --2- 172.21.15.82:0/4173802880 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bfa33574d0 0x55bfa3722140 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f3b9c011760 tx=0x7f3b9c00ba30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b30a7aee1df217a3 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:15:25.908 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:25.905+0000 7f3b88ff9640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-05-20T16:15:25.909 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:25.905+0000 7f3b88ff9640 1 -- 172.21.15.82:0/4173802880 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b9c005220 con 0x55bfa33574d0 2024-05-20T16:15:25.909 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:25.905+0000 7f3b88ff9640 1 -- 172.21.15.82:0/4173802880 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9150+0+0 (secure 0 0 0) 0x7f3b9c0053c0 con 0x55bfa33574d0 2024-05-20T16:15:25.909 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:25.905+0000 7f3bae7be740 1 -- 172.21.15.82:0/4173802880 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3b90031830 con 0x55bfa33574d0 2024-05-20T16:15:25.909 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:25.906+0000 7f3b88ff9640 1 -- 172.21.15.82:0/4173802880 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f3b9c0055c0 con 0x55bfa33574d0 2024-05-20T16:15:25.909 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:25.906+0000 7f3bae7be740 10 client.15567 fetch_fsmap learned FSMap version 25 2024-05-20T16:15:25.909 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:25.906+0000 7f3bae7be740 1 -- 172.21.15.82:0/4173802880 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55bfa345c620 con 0x55bfa33574d0 2024-05-20T16:15:25.909 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:25.906+0000 7f3b88ff9640 1 -- 172.21.15.82:0/4173802880 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3b9c004c80 con 0x55bfa33574d0 2024-05-20T16:15:25.909 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:25.906+0000 7f3b88ff9640 1 -- 172.21.15.82:0/4173802880 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f3b9c018070 con 0x55bfa33574d0 2024-05-20T16:15:25.909 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:25.906+0000 7f3bae7be740 10 client.15567 fetch_fsmap finished waiting for FSMap version 25 2024-05-20T16:15:25.909 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:25.906+0000 7f3bae7be740 10 client.15567 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T16:15:25.909 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:25.906+0000 7f3bae7be740 20 client.15567 populate_metadata read hostname 'smithi082' 2024-05-20T16:15:25.909 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:25.906+0000 7f3bae7be740 1 --2- 172.21.15.82:0/4173802880 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x55bfa3336d80 0x55bfa372f670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:15:25.909 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:25.906+0000 7f3bae7be740 4 client.15567 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T16:15:25.909 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:25.906+0000 7f3bae7be740 1 -- 172.21.15.82:0/4173802880 --> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f3b90031830 con 0x55bfa3336d80 2024-05-20T16:15:25.909 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:25.906+0000 7f3ba4dd8640 1 --2- 172.21.15.82:0/4173802880 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x55bfa3336d80 0x55bfa372f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:15:25.911 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:25.907+0000 7f3b88ff9640 10 client.15567 ms_handle_connect on v2:172.21.15.160:6838/2765554482 2024-05-20T16:15:25.911 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:25.907+0000 7f3ba4dd8640 1 --2- 172.21.15.82:0/4173802880 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x55bfa3336d80 0x55bfa372f670 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1cb8cf27df6db8ad server_cookie=56829619548fcf35 in_seq=0 out_seq=0 2024-05-20T16:15:25.911 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:25.908+0000 7f3b88ff9640 1 -- 172.21.15.82:0/4173802880 <== mds.0 v2:172.21.15.160:6838/2765554482 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7f3b90031830 con 0x55bfa3336d80 2024-05-20T16:15:25.911 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:25.908+0000 7f3b88ff9640 10 client.15567 handle_command_reply: tid=0 2024-05-20T16:15:25.911 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:25.908+0000 7f3bae7be740 1 client.15567 shutdown 2024-05-20T16:15:25.911 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:25.908+0000 7f3bae7be740 1 -- 172.21.15.82:0/4173802880 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bfa33574d0 msgr2=0x55bfa3722140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:15:25.911 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:25.908+0000 7f3bae7be740 1 --2- 172.21.15.82:0/4173802880 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bfa33574d0 0x55bfa3722140 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f3b9c011760 tx=0x7f3b9c00ba30 comp rx=0 tx=0).stop 2024-05-20T16:15:25.911 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:25.908+0000 7f3bae7be740 1 -- 172.21.15.82:0/4173802880 shutdown_connections 2024-05-20T16:15:25.911 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:25.908+0000 7f3bae7be740 1 --2- 172.21.15.82:0/4173802880 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x55bfa3336d80 0x55bfa372f670 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:15:25.911 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:25.908+0000 7f3bae7be740 1 --2- 172.21.15.82:0/4173802880 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bfa33574d0 0x55bfa3722140 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:15:25.911 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:25.908+0000 7f3bae7be740 1 --2- 172.21.15.82:0/4173802880 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bfa33781b0 0x55bfa37290a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:15:25.911 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:25.908+0000 7f3bae7be740 1 --2- 172.21.15.82:0/4173802880 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bfa334f2f0 0x55bfa3721c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:15:25.911 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:25.908+0000 7f3bae7be740 1 -- 172.21.15.82:0/4173802880 >> 172.21.15.82:0/4173802880 conn(0x55bfa336c300 msgr2=0x55bfa37167a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:15:25.911 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:25.908+0000 7f3b88ff9640 0 client.15567 ms_handle_reset on v2:172.21.15.160:6838/2765554482 2024-05-20T16:15:25.912 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:25.908+0000 7f3bae7be740 1 -- 172.21.15.82:0/4173802880 shutdown_connections 2024-05-20T16:15:25.912 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:25.908+0000 7f3bae7be740 1 -- 172.21.15.82:0/4173802880 wait complete. 2024-05-20T16:15:25.912 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:25.909+0000 7f3bae7be740 20 client.15567 trim_cache size 0 max 16384 2024-05-20T16:15:25.939 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:25.938+0000 7f3bae7be740 1 Processor -- start 2024-05-20T16:15:25.939 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:25.938+0000 7f3bae7be740 1 -- start start 2024-05-20T16:15:25.939 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:25.938+0000 7f3bae7be740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bfa334f2f0 0x55bfa3734e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:15:25.939 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:25.938+0000 7f3bae7be740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bfa33574d0 0x55bfa37353b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:15:25.939 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:25.938+0000 7f3bae7be740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bfa33781b0 0x55bfa3728fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:15:25.939 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:25.938+0000 7f3bae7be740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f3ba01ac010 con 0x55bfa33781b0 2024-05-20T16:15:25.940 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:25.938+0000 7f3bae7be740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f3ba01a98e0 con 0x55bfa33574d0 2024-05-20T16:15:25.940 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:25.938+0000 7f3bae7be740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f3ba01a7730 con 0x55bfa334f2f0 2024-05-20T16:15:25.940 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:25.939+0000 7f3ba4dd8640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bfa33781b0 0x55bfa3728fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:15:25.940 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:25.939+0000 7f3ba4dd8640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bfa33781b0 0x55bfa3728fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:35216/0 (socket says 172.21.15.82:35216) 2024-05-20T16:15:25.940 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:25.939+0000 7f3ba4dd8640 1 -- 172.21.15.82:0/1260310421 learned_addr learned my addr 172.21.15.82:0/1260310421 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:15:25.940 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:25.939+0000 7f3ba55d9640 1 --2- 172.21.15.82:0/1260310421 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bfa33574d0 0x55bfa37353b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:15:25.940 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:25.939+0000 7f3ba55d9640 1 -- 172.21.15.82:0/1260310421 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bfa334f2f0 msgr2=0x55bfa3734e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:15:25.940 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:25.939+0000 7f3ba55d9640 1 --2- 172.21.15.82:0/1260310421 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bfa334f2f0 0x55bfa3734e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:15:25.940 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:25.939+0000 7f3ba55d9640 1 -- 172.21.15.82:0/1260310421 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bfa33781b0 msgr2=0x55bfa3728fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:15:25.940 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:25.939+0000 7f3ba55d9640 1 --2- 172.21.15.82:0/1260310421 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bfa33781b0 0x55bfa3728fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:15:25.940 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:25.939+0000 7f3ba55d9640 1 -- 172.21.15.82:0/1260310421 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bfa3717a10 con 0x55bfa33574d0 2024-05-20T16:15:25.940 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:25.939+0000 7f3ba4dd8640 1 --2- 172.21.15.82:0/1260310421 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bfa33781b0 0x55bfa3728fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T16:15:25.940 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:25.939+0000 7f3ba55d9640 1 --2- 172.21.15.82:0/1260310421 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bfa33574d0 0x55bfa37353b0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f3b9c0057c0 tx=0x7f3b9c011b30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4dc2027641586e77 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:15:25.940 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:25.939+0000 7f3b8affd640 1 -- 172.21.15.82:0/1260310421 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b9c02b070 con 0x55bfa33574d0 2024-05-20T16:15:25.940 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:25.940+0000 7f3b8affd640 1 -- 172.21.15.82:0/1260310421 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3b9c018d70 con 0x55bfa33574d0 2024-05-20T16:15:25.941 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:25.940+0000 7f3b8affd640 1 -- 172.21.15.82:0/1260310421 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b9c004e30 con 0x55bfa33574d0 2024-05-20T16:15:25.943 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:25.941+0000 7f3bae7be740 1 -- 172.21.15.82:0/1260310421 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bfa33574d0 msgr2=0x55bfa37353b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:15:25.943 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:25.941+0000 7f3bae7be740 1 --2- 172.21.15.82:0/1260310421 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bfa33574d0 0x55bfa37353b0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f3b9c0057c0 tx=0x7f3b9c011b30 comp rx=0 tx=0).stop 2024-05-20T16:15:25.943 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:25.941+0000 7f3bae7be740 1 -- 172.21.15.82:0/1260310421 shutdown_connections 2024-05-20T16:15:25.943 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:25.941+0000 7f3bae7be740 1 --2- 172.21.15.82:0/1260310421 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bfa33781b0 0x55bfa3728fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:15:25.944 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:25.941+0000 7f3bae7be740 1 --2- 172.21.15.82:0/1260310421 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bfa33574d0 0x55bfa37353b0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:15:25.944 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:25.941+0000 7f3bae7be740 1 --2- 172.21.15.82:0/1260310421 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bfa334f2f0 0x55bfa3734e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:15:25.944 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:25.941+0000 7f3bae7be740 1 -- 172.21.15.82:0/1260310421 >> 172.21.15.82:0/1260310421 conn(0x55bfa336c300 msgr2=0x55bfa371c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:15:25.944 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:25.941+0000 7f3bae7be740 1 -- 172.21.15.82:0/1260310421 shutdown_connections 2024-05-20T16:15:25.944 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:25.941+0000 7f3bae7be740 1 -- 172.21.15.82:0/1260310421 wait complete. 2024-05-20T16:15:25.944 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:25.941+0000 7f3bae7be740 1 -- start start 2024-05-20T16:15:25.944 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:25.941+0000 7f3bae7be740 1 Processor -- start 2024-05-20T16:15:25.944 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:25.942+0000 7f3bae7be740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bfa334f2f0 0x55bfa3728d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:15:25.944 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:25.942+0000 7f3bae7be740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bfa33574d0 0x55bfa3729290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:15:25.944 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:25.942+0000 7f3bae7be740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bfa33781b0 0x55bfa3795e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:15:25.944 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:25.942+0000 7f3bae7be740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f3ba01a7da0 con 0x55bfa33781b0 2024-05-20T16:15:25.944 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:25.942+0000 7f3bae7be740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f3ba01a9210 con 0x55bfa334f2f0 2024-05-20T16:15:25.944 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:25.942+0000 7f3bae7be740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f3ba01a8470 con 0x55bfa33574d0 2024-05-20T16:15:25.944 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:25.942+0000 7f3bae7be740 10 client.0 _ll_register_callbacks cb 0x7f3b9c00b5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T16:15:25.944 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:25.942+0000 7f3ba4dd8640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bfa33781b0 0x55bfa3795e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:15:25.944 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:25.942+0000 7f3ba55d9640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bfa33574d0 0x55bfa3729290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:15:25.945 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:25.942+0000 7f3ba55d9640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bfa33574d0 0x55bfa3729290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.82:51226/0 (socket says 172.21.15.82:51226) 2024-05-20T16:15:25.945 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:25.942+0000 7f3ba4dd8640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bfa33781b0 0x55bfa3795e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:35224/0 (socket says 172.21.15.82:35224) 2024-05-20T16:15:25.945 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:25.942+0000 7f3ba55d9640 1 -- 172.21.15.82:0/460103373 learned_addr learned my addr 172.21.15.82:0/460103373 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:15:25.945 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:25.942+0000 7f3ba4dd8640 1 --2- 172.21.15.82:0/460103373 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bfa334f2f0 0x55bfa3728d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:15:25.945 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:25.943+0000 7f3ba4dd8640 1 -- 172.21.15.82:0/460103373 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bfa33574d0 msgr2=0x55bfa3729290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:15:25.945 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:25.943+0000 7f3ba4dd8640 1 --2- 172.21.15.82:0/460103373 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bfa33574d0 0x55bfa3729290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:15:25.945 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:25.943+0000 7f3ba4dd8640 1 -- 172.21.15.82:0/460103373 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bfa334f2f0 msgr2=0x55bfa3728d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:15:25.945 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:25.943+0000 7f3ba4dd8640 1 --2- 172.21.15.82:0/460103373 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bfa334f2f0 0x55bfa3728d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:15:25.945 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:25.943+0000 7f3ba4dd8640 1 -- 172.21.15.82:0/460103373 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bfa3796410 con 0x55bfa33781b0 2024-05-20T16:15:25.945 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:25.943+0000 7f3ba55d9640 1 --2- 172.21.15.82:0/460103373 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bfa33574d0 0x55bfa3729290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T16:15:25.945 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:25.943+0000 7f3ba4dd8640 1 --2- 172.21.15.82:0/460103373 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bfa33781b0 0x55bfa3795e20 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f3b90033000 tx=0x7f3b9004d040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c72bafc2edb0a63 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:15:25.945 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:25.943+0000 7f3b88ff9640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-05-20T16:15:25.945 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:25.943+0000 7f3b88ff9640 1 -- 172.21.15.82:0/460103373 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b90035070 con 0x55bfa33781b0 2024-05-20T16:15:25.945 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:25.943+0000 7f3b88ff9640 1 -- 172.21.15.82:0/460103373 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9150+0+0 (secure 0 0 0) 0x7f3b900329b0 con 0x55bfa33781b0 2024-05-20T16:15:25.945 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:25.943+0000 7f3bae7be740 1 -- 172.21.15.82:0/460103373 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3b9c018d70 con 0x55bfa33781b0 2024-05-20T16:15:25.946 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:25.943+0000 7f3b88ff9640 1 -- 172.21.15.82:0/460103373 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f3b90004030 con 0x55bfa33781b0 2024-05-20T16:15:25.946 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:25.943+0000 7f3bae7be740 10 client.15573 fetch_fsmap learned FSMap version 25 2024-05-20T16:15:25.946 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:25.943+0000 7f3bae7be740 1 -- 172.21.15.82:0/460103373 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55bfa3797680 con 0x55bfa33781b0 2024-05-20T16:15:25.946 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:25.944+0000 7f3b88ff9640 1 -- 172.21.15.82:0/460103373 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3b9c018d70 con 0x55bfa33781b0 2024-05-20T16:15:25.946 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:25.944+0000 7f3b88ff9640 1 -- 172.21.15.82:0/460103373 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f3b9003b460 con 0x55bfa33781b0 2024-05-20T16:15:25.947 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:25.944+0000 7f3bae7be740 10 client.15573 fetch_fsmap finished waiting for FSMap version 25 2024-05-20T16:15:25.947 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:25.944+0000 7f3bae7be740 10 client.15573 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T16:15:25.947 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:25.944+0000 7f3bae7be740 20 client.15573 populate_metadata read hostname 'smithi082' 2024-05-20T16:15:25.947 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:25.944+0000 7f3bae7be740 1 --2- 172.21.15.82:0/460103373 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x55bfa330c1c0 0x55bfa3799b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:15:25.947 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:25.944+0000 7f3bae7be740 4 client.15573 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-20T16:15:25.947 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:25.944+0000 7f3bae7be740 1 -- 172.21.15.82:0/460103373 --> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55bfa3689a70 con 0x55bfa330c1c0 2024-05-20T16:15:25.947 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:25.945+0000 7f3ba55d9640 1 --2- 172.21.15.82:0/460103373 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x55bfa330c1c0 0x55bfa3799b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:15:25.947 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:25.945+0000 7f3b88ff9640 10 client.15573 ms_handle_connect on v2:172.21.15.160:6838/2765554482 2024-05-20T16:15:25.947 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:25.945+0000 7f3ba55d9640 1 --2- 172.21.15.82:0/460103373 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x55bfa330c1c0 0x55bfa3799b70 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f582b842dc175a56 server_cookie=30924b67e6e0b92f in_seq=0 out_seq=0 2024-05-20T16:15:25.951 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:25.951+0000 7f3b88ff9640 1 -- 172.21.15.82:0/460103373 <== mds.0 v2:172.21.15.160:6838/2765554482 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55bfa3689a70 con 0x55bfa330c1c0 2024-05-20T16:15:25.951 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:25.951+0000 7f3b88ff9640 10 client.15573 handle_command_reply: tid=0 2024-05-20T16:15:25.952 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:25.951+0000 7f3bae7be740 1 client.15573 shutdown 2024-05-20T16:15:25.952 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:25.951+0000 7f3bae7be740 1 -- 172.21.15.82:0/460103373 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bfa33781b0 msgr2=0x55bfa3795e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:15:25.952 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:25.951+0000 7f3bae7be740 1 --2- 172.21.15.82:0/460103373 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bfa33781b0 0x55bfa3795e20 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f3b90033000 tx=0x7f3b9004d040 comp rx=0 tx=0).stop 2024-05-20T16:15:25.952 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:25.951+0000 7f3bae7be740 1 -- 172.21.15.82:0/460103373 shutdown_connections 2024-05-20T16:15:25.952 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:25.951+0000 7f3bae7be740 1 --2- 172.21.15.82:0/460103373 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x55bfa330c1c0 0x55bfa3799b70 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:15:25.952 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:25.951+0000 7f3b88ff9640 0 client.15573 ms_handle_reset on v2:172.21.15.160:6838/2765554482 2024-05-20T16:15:25.952 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:25.951+0000 7f3bae7be740 1 --2- 172.21.15.82:0/460103373 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bfa33781b0 0x55bfa3795e20 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:15:25.952 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:25.951+0000 7f3bae7be740 1 --2- 172.21.15.82:0/460103373 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bfa33574d0 0x55bfa3729290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:15:25.952 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:25.951+0000 7f3bae7be740 1 --2- 172.21.15.82:0/460103373 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55bfa334f2f0 0x55bfa3728d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:15:25.952 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:25.951+0000 7f3bae7be740 1 -- 172.21.15.82:0/460103373 >> 172.21.15.82:0/460103373 conn(0x55bfa336c300 msgr2=0x55bfa3716740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:15:25.952 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:25.952+0000 7f3bae7be740 1 -- 172.21.15.82:0/460103373 shutdown_connections 2024-05-20T16:15:25.952 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:25.952+0000 7f3bae7be740 1 -- 172.21.15.82:0/460103373 wait complete. 2024-05-20T16:15:25.952 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:25.952+0000 7f3bae7be740 20 client.15573 trim_cache size 0 max 16384 2024-05-20T16:15:25.953 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:{ 2024-05-20T16:15:25.953 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "message": "", 2024-05-20T16:15:25.953 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "return_code": 0 2024-05-20T16:15:25.953 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:} 2024-05-20T16:15:25.953 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:25.952+0000 7f3ba783d640 1 -- 172.21.15.82:0/2244017410 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f3b8c0a2b80 msgr2=0x7f3b8c0a5090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:15:25.954 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:25.952+0000 7f3ba783d640 1 --2- 172.21.15.82:0/2244017410 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f3b8c0a2b80 0x7f3b8c0a5090 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f3b980a0410 tx=0x7f3b98005b40 comp rx=0 tx=0).stop 2024-05-20T16:15:25.954 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:25.952+0000 7f3ba783d640 1 -- 172.21.15.82:0/2244017410 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3ba0107910 msgr2=0x7f3ba01a0790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:15:25.954 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:25.952+0000 7f3ba783d640 1 --2- 172.21.15.82:0/2244017410 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3ba0107910 0x7f3ba01a0790 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f3b9800ea20 tx=0x7f3b9800eee0 comp rx=0 tx=0).stop 2024-05-20T16:15:25.954 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:25.952+0000 7f3ba783d640 1 -- 172.21.15.82:0/2244017410 shutdown_connections 2024-05-20T16:15:25.954 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:25.952+0000 7f3ba783d640 1 --2- 172.21.15.82:0/2244017410 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f3b8c0a2b80 0x7f3b8c0a5090 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:15:25.954 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:25.952+0000 7f3ba783d640 1 --2- 172.21.15.82:0/2244017410 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3ba0107910 0x7f3ba01a0790 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:15:25.954 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:25.952+0000 7f3ba783d640 1 --2- 172.21.15.82:0/2244017410 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3ba0102c10 0x7f3ba0199800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:15:25.954 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:25.952+0000 7f3ba783d640 1 --2- 172.21.15.82:0/2244017410 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3ba01019d0 0x7f3ba01992c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:15:25.954 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:25.952+0000 7f3ba783d640 1 -- 172.21.15.82:0/2244017410 >> 172.21.15.82:0/2244017410 conn(0x7f3ba00fcbc0 msgr2=0x7f3ba01038f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:15:25.954 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:25.952+0000 7f3ba783d640 1 -- 172.21.15.82:0/2244017410 shutdown_connections 2024-05-20T16:15:25.954 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:25.952+0000 7f3ba783d640 1 -- 172.21.15.82:0/2244017410 wait complete. 2024-05-20T16:15:26.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:15:26 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T16:15:26.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:15:26 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T16:15:26.885 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:15:26 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T16:15:28.312 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:15:27 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:15:28.312 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:15:27 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:15:28.312 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:15:27 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:15:28.313 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:15:27 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:15:28.313 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:15:27 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:15:28.313 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:15:27 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:15:28.313 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:15:27 smithi160 ceph-mon[36347]: pgmap v406: 129 pgs: 129 active+clean; 670 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 6 op/s 2024-05-20T16:15:28.313 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:15:27 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:15:28.313 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:15:27 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T16:15:28.313 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:15:27 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:15:28.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:15:27 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:15:28.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:15:27 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:15:28.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:15:27 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:15:28.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:15:27 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:15:28.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:15:27 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:15:28.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:15:27 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:15:28.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:15:27 smithi082 ceph-mon[32238]: pgmap v406: 129 pgs: 129 active+clean; 670 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 6 op/s 2024-05-20T16:15:28.360 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:15:27 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:15:28.360 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:15:27 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T16:15:28.360 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:15:27 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:15:28.385 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:15:27 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:15:28.386 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:15:27 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:15:28.386 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:15:27 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:15:28.386 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:15:27 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:15:28.386 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:15:27 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:15:28.386 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:15:27 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:15:28.386 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:15:27 smithi184 ceph-mon[36209]: pgmap v406: 129 pgs: 129 active+clean; 670 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 6 op/s 2024-05-20T16:15:28.386 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:15:27 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:15:28.386 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:15:27 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T16:15:28.386 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:15:27 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:15:29.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:15:29 smithi160 ceph-mon[36347]: pgmap v407: 129 pgs: 129 active+clean; 671 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 767 B/s wr, 8 op/s 2024-05-20T16:15:29.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:15:29 smithi082 ceph-mon[32238]: pgmap v407: 129 pgs: 129 active+clean; 671 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 767 B/s wr, 8 op/s 2024-05-20T16:15:29.885 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:15:29 smithi184 ceph-mon[36209]: pgmap v407: 129 pgs: 129 active+clean; 671 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 767 B/s wr, 8 op/s 2024-05-20T16:15:31.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:15:31 smithi160 ceph-mon[36347]: pgmap v408: 129 pgs: 129 active+clean; 671 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-20T16:15:31.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:15:31 smithi082 ceph-mon[32238]: pgmap v408: 129 pgs: 129 active+clean; 671 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-20T16:15:31.885 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:15:31 smithi184 ceph-mon[36209]: pgmap v408: 129 pgs: 129 active+clean; 671 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-20T16:15:33.805 INFO:tasks.workunit.client.0.smithi082.stderr:Note: switching to 'd2b507c724ef6c8295ee0a35d185cdab167fd61d'. 2024-05-20T16:15:33.805 INFO:tasks.workunit.client.0.smithi082.stderr: 2024-05-20T16:15:33.805 INFO:tasks.workunit.client.0.smithi082.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-05-20T16:15:33.805 INFO:tasks.workunit.client.0.smithi082.stderr:changes and commit them, and you can discard any commits you make in this 2024-05-20T16:15:33.805 INFO:tasks.workunit.client.0.smithi082.stderr:state without impacting any branches by switching back to a branch. 2024-05-20T16:15:33.805 INFO:tasks.workunit.client.0.smithi082.stderr: 2024-05-20T16:15:33.805 INFO:tasks.workunit.client.0.smithi082.stderr:If you want to create a new branch to retain commits you create, you may 2024-05-20T16:15:33.805 INFO:tasks.workunit.client.0.smithi082.stderr:do so (now or later) by using -c with the switch command. Example: 2024-05-20T16:15:33.805 INFO:tasks.workunit.client.0.smithi082.stderr: 2024-05-20T16:15:33.806 INFO:tasks.workunit.client.0.smithi082.stderr: git switch -c 2024-05-20T16:15:33.806 INFO:tasks.workunit.client.0.smithi082.stderr: 2024-05-20T16:15:33.806 INFO:tasks.workunit.client.0.smithi082.stderr:Or undo this operation with: 2024-05-20T16:15:33.806 INFO:tasks.workunit.client.0.smithi082.stderr: 2024-05-20T16:15:33.806 INFO:tasks.workunit.client.0.smithi082.stderr: git switch - 2024-05-20T16:15:33.806 INFO:tasks.workunit.client.0.smithi082.stderr: 2024-05-20T16:15:33.806 INFO:tasks.workunit.client.0.smithi082.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-05-20T16:15:33.806 INFO:tasks.workunit.client.0.smithi082.stderr: 2024-05-20T16:15:33.806 INFO:tasks.workunit.client.0.smithi082.stderr:HEAD is now at d2b507c724e Merge PR #56984 into wip-pdonnell-testing-20240517.203521-squid-debug 2024-05-20T16:15:33.811 DEBUG:teuthology.orchestra.run.smithi082:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-05-20T16:15:33.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:15:33 smithi160 ceph-mon[36347]: pgmap v409: 129 pgs: 129 active+clean; 671 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-20T16:15:33.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:15:33 smithi082 ceph-mon[32238]: pgmap v409: 129 pgs: 129 active+clean; 671 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-20T16:15:33.875 INFO:tasks.workunit.client.0.smithi082.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-05-20T16:15:33.878 INFO:tasks.workunit.client.0.smithi082.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-05-20T16:15:33.878 INFO:tasks.workunit.client.0.smithi082.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-05-20T16:15:33.885 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:15:33 smithi184 ceph-mon[36209]: pgmap v409: 129 pgs: 129 active+clean; 671 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-20T16:15:34.565 INFO:tasks.workunit.client.0.smithi082.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-05-20T16:15:34.619 INFO:tasks.workunit.client.0.smithi082.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-05-20T16:15:34.658 INFO:tasks.workunit.client.0.smithi082.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-05-20T16:15:34.660 INFO:tasks.workunit.client.0.smithi082.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-05-20T16:15:34.660 INFO:tasks.workunit.client.0.smithi082.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-05-20T16:15:34.699 INFO:tasks.workunit.client.0.smithi082.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-05-20T16:15:34.702 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-05-20T16:15:34.702 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-05-20T16:15:34.759 INFO:tasks.workunit:Running workunits matching suites/iogen.sh on client.0... 2024-05-20T16:15:34.761 INFO:tasks.workunit:Running workunit suites/iogen.sh... 2024-05-20T16:15:34.761 DEBUG:teuthology.orchestra.run.smithi082:workunit test suites/iogen.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=d2b507c724ef6c8295ee0a35d185cdab167fd61d TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/iogen.sh 2024-05-20T16:15:34.822 INFO:tasks.workunit.client.0.smithi082.stdout:getting iogen 2024-05-20T16:15:34.822 INFO:tasks.workunit.client.0.smithi082.stderr:+ echo 'getting iogen' 2024-05-20T16:15:34.823 INFO:tasks.workunit.client.0.smithi082.stderr:+ wget http://download.ceph.com/qa/iogen_3.1p0.tar 2024-05-20T16:15:34.845 INFO:tasks.workunit.client.0.smithi082.stderr:--2024-05-20 16:15:34-- http://download.ceph.com/qa/iogen_3.1p0.tar 2024-05-20T16:15:34.846 INFO:tasks.workunit.client.0.smithi082.stderr:Resolving download.ceph.com (download.ceph.com)... 158.69.68.124, 2607:5300:201:2000::3:58a1 2024-05-20T16:15:34.939 INFO:tasks.workunit.client.0.smithi082.stderr:Connecting to download.ceph.com (download.ceph.com)|158.69.68.124|:80... connected. 2024-05-20T16:15:35.014 INFO:tasks.workunit.client.0.smithi082.stderr:HTTP request sent, awaiting response... 200 OK 2024-05-20T16:15:35.014 INFO:tasks.workunit.client.0.smithi082.stderr:Length: 8071 (7.9K) [application/octet-stream] 2024-05-20T16:15:35.015 INFO:tasks.workunit.client.0.smithi082.stderr:Saving to: ‘iogen_3.1p0.tar’ 2024-05-20T16:15:35.018 INFO:tasks.workunit.client.0.smithi082.stderr: 2024-05-20T16:15:35.018 INFO:tasks.workunit.client.0.smithi082.stderr: 0K ....... 100% 1.12G=0s 2024-05-20T16:15:35.018 INFO:tasks.workunit.client.0.smithi082.stderr: 2024-05-20T16:15:35.019 INFO:tasks.workunit.client.0.smithi082.stderr:2024-05-20 16:15:35 (1.12 GB/s) - ‘iogen_3.1p0.tar’ saved [8071/8071] 2024-05-20T16:15:35.019 INFO:tasks.workunit.client.0.smithi082.stderr: 2024-05-20T16:15:35.019 INFO:tasks.workunit.client.0.smithi082.stderr:+ tar -xvzf iogen_3.1p0.tar 2024-05-20T16:15:35.036 INFO:tasks.workunit.client.0.smithi082.stdout:iogen_3.1p0 2024-05-20T16:15:35.037 INFO:tasks.workunit.client.0.smithi082.stdout:iogen_3.1p0/src 2024-05-20T16:15:35.038 INFO:tasks.workunit.client.0.smithi082.stdout:iogen_3.1p0/src/iogen.8 2024-05-20T16:15:35.040 INFO:tasks.workunit.client.0.smithi082.stdout:iogen_3.1p0/src/iogen.c 2024-05-20T16:15:35.042 INFO:tasks.workunit.client.0.smithi082.stdout:iogen_3.1p0/README 2024-05-20T16:15:35.043 INFO:tasks.workunit.client.0.smithi082.stdout:iogen_3.1p0/Makefile.bsd 2024-05-20T16:15:35.045 INFO:tasks.workunit.client.0.smithi082.stdout:iogen_3.1p0/Makefile 2024-05-20T16:15:35.046 INFO:tasks.workunit.client.0.smithi082.stderr:+ cd iogen_3.1p0 2024-05-20T16:15:35.047 INFO:tasks.workunit.client.0.smithi082.stdout:making iogen 2024-05-20T16:15:35.047 INFO:tasks.workunit.client.0.smithi082.stderr:+ echo 'making iogen' 2024-05-20T16:15:35.047 INFO:tasks.workunit.client.0.smithi082.stderr:+ make 2024-05-20T16:15:35.055 INFO:tasks.workunit.client.0.smithi082.stdout:gcc -D_FILE_OFFSET_BITS=64 -DVERSION=\"3.1p0\" -Dstrlcpy=strncpy -Darc4random=rand -O2 src/iogen.c -o iogen 2024-05-20T16:15:35.275 INFO:tasks.workunit.client.0.smithi082.stdout:nroff -mandoc -Tascii src/iogen.8 > iogen.cat8 2024-05-20T16:15:35.358 INFO:tasks.workunit.client.0.smithi082.stdout:running iogen 2024-05-20T16:15:35.359 INFO:tasks.workunit.client.0.smithi082.stderr:+ echo 'running iogen' 2024-05-20T16:15:35.359 INFO:tasks.workunit.client.0.smithi082.stderr:+ ./iogen -n 5 -s 2g 2024-05-20T16:15:35.359 INFO:tasks.workunit.client.0.smithi082.stderr:child spawned 86714 2024-05-20T16:15:35.359 INFO:tasks.workunit.client.0.smithi082.stderr:child spawned 86715 2024-05-20T16:15:35.359 INFO:tasks.workunit.client.0.smithi082.stderr:child spawned 86716 2024-05-20T16:15:35.359 INFO:tasks.workunit.client.0.smithi082.stderr:child spawned 86717 2024-05-20T16:15:35.359 INFO:tasks.workunit.client.0.smithi082.stderr:child spawned 86718 2024-05-20T16:15:35.359 INFO:tasks.workunit.client.0.smithi082.stdout:sleep for 10 min 2024-05-20T16:15:35.360 INFO:tasks.workunit.client.0.smithi082.stderr:+ echo 'sleep for 10 min' 2024-05-20T16:15:35.360 INFO:tasks.workunit.client.0.smithi082.stderr:+ sleep 600 2024-05-20T16:15:35.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:15:35 smithi082 ceph-mon[32238]: pgmap v410: 129 pgs: 129 active+clean; 671 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-20T16:15:35.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:15:35 smithi160 ceph-mon[36347]: pgmap v410: 129 pgs: 129 active+clean; 671 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-20T16:15:35.885 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:15:35 smithi184 ceph-mon[36209]: pgmap v410: 129 pgs: 129 active+clean; 671 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-20T16:15:37.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:15:37 smithi160 ceph-mon[36347]: pgmap v411: 129 pgs: 129 active+clean; 181 MiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 15 MiB/s wr, 9 op/s 2024-05-20T16:15:37.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:15:37 smithi082 ceph-mon[32238]: pgmap v411: 129 pgs: 129 active+clean; 181 MiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 15 MiB/s wr, 9 op/s 2024-05-20T16:15:37.885 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:15:37 smithi184 ceph-mon[36209]: pgmap v411: 129 pgs: 129 active+clean; 181 MiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 15 MiB/s wr, 9 op/s 2024-05-20T16:15:39.177 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:39.177+0000 7f7c367ce640 1 -- 172.21.15.82:0/992995403 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7c30101170 msgr2=0x7f7c30101e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:15:39.177 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:39.177+0000 7f7c367ce640 1 --2- 172.21.15.82:0/992995403 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7c30101170 0x7f7c30101e70 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f7c20009900 tx=0x7f7c2002a410 comp rx=0 tx=0).stop 2024-05-20T16:15:39.177 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:39.177+0000 7f7c367ce640 1 -- 172.21.15.82:0/992995403 shutdown_connections 2024-05-20T16:15:39.177 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:39.177+0000 7f7c367ce640 1 --2- 172.21.15.82:0/992995403 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7c301070b0 0x7f7c301094d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:15:39.177 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:39.177+0000 7f7c367ce640 1 --2- 172.21.15.82:0/992995403 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7c301023b0 0x7f7c301027c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:15:39.178 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:39.177+0000 7f7c367ce640 1 --2- 172.21.15.82:0/992995403 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7c30101170 0x7f7c30101e70 secure :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f7c20009900 tx=0x7f7c2002a410 comp rx=0 tx=0).stop 2024-05-20T16:15:39.178 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:39.177+0000 7f7c367ce640 1 -- 172.21.15.82:0/992995403 >> 172.21.15.82:0/992995403 conn(0x7f7c300fc340 msgr2=0x7f7c300fe810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:15:39.180 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:39.180+0000 7f7c367ce640 1 -- 172.21.15.82:0/992995403 shutdown_connections 2024-05-20T16:15:39.180 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:39.180+0000 7f7c367ce640 1 -- 172.21.15.82:0/992995403 wait complete. 2024-05-20T16:15:39.182 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:39.181+0000 7f7c367ce640 1 Processor -- start 2024-05-20T16:15:39.182 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:39.182+0000 7f7c367ce640 1 -- start start 2024-05-20T16:15:39.182 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:39.182+0000 7f7c367ce640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7c301023b0 0x7f7c30197350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:15:39.182 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:39.182+0000 7f7c367ce640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7c301070b0 0x7f7c30197910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:15:39.183 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:39.182+0000 7f7c367ce640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7c3019c810 0x7f7c3019cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:15:39.183 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:39.182+0000 7f7c367ce640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f7c3010c680 con 0x7f7c301070b0 2024-05-20T16:15:39.183 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:39.182+0000 7f7c367ce640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f7c3010c500 con 0x7f7c3019c810 2024-05-20T16:15:39.183 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:39.182+0000 7f7c367ce640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f7c3010c800 con 0x7f7c301023b0 2024-05-20T16:15:39.183 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:39.183+0000 7f7c2f7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7c301070b0 0x7f7c30197910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:15:39.184 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:39.183+0000 7f7c2f7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7c301070b0 0x7f7c30197910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:55256/0 (socket says 172.21.15.82:55256) 2024-05-20T16:15:39.184 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:39.183+0000 7f7c2f7fe640 1 -- 172.21.15.82:0/1907759906 learned_addr learned my addr 172.21.15.82:0/1907759906 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:15:39.184 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:39.183+0000 7f7c34d6b640 1 --2- 172.21.15.82:0/1907759906 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7c3019c810 0x7f7c3019cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:15:39.185 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:39.185+0000 7f7c2ffff640 1 --2- 172.21.15.82:0/1907759906 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7c301023b0 0x7f7c30197350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:15:39.186 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:39.186+0000 7f7c34d6b640 1 -- 172.21.15.82:0/1907759906 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7c301023b0 msgr2=0x7f7c30197350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:15:39.186 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:39.186+0000 7f7c34d6b640 1 --2- 172.21.15.82:0/1907759906 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7c301023b0 0x7f7c30197350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:15:39.186 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:39.186+0000 7f7c34d6b640 1 -- 172.21.15.82:0/1907759906 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7c301070b0 msgr2=0x7f7c30197910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:15:39.186 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:39.186+0000 7f7c34d6b640 1 --2- 172.21.15.82:0/1907759906 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7c301070b0 0x7f7c30197910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:15:39.186 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:39.186+0000 7f7c34d6b640 1 -- 172.21.15.82:0/1907759906 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c3019d350 con 0x7f7c3019c810 2024-05-20T16:15:39.191 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:39.192+0000 7f7c34d6b640 1 --2- 172.21.15.82:0/1907759906 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7c3019c810 0x7f7c3019cc20 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f7c2800be20 tx=0x7f7c2800bf10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:15:39.193 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:39.193+0000 7f7c2d7fa640 1 -- 172.21.15.82:0/1907759906 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c28019070 con 0x7f7c3019c810 2024-05-20T16:15:39.193 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:39.193+0000 7f7c2d7fa640 1 -- 172.21.15.82:0/1907759906 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c28014920 con 0x7f7c3019c810 2024-05-20T16:15:39.193 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:39.193+0000 7f7c2d7fa640 1 -- 172.21.15.82:0/1907759906 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c28012940 con 0x7f7c3019c810 2024-05-20T16:15:39.195 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:39.195+0000 7f7c367ce640 1 -- 172.21.15.82:0/1907759906 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c3019d640 con 0x7f7c3019c810 2024-05-20T16:15:39.199 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:39.195+0000 7f7c367ce640 1 -- 172.21.15.82:0/1907759906 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c3019db90 con 0x7f7c3019c810 2024-05-20T16:15:39.202 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:39.202+0000 7f7c3d67b740 1 Processor -- start 2024-05-20T16:15:39.207 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:39.204+0000 7f7c3d67b740 1 -- start start 2024-05-20T16:15:39.207 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:39.204+0000 7f7c3d67b740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561c37d5f1b0 0x561c380fee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:15:39.207 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:39.206+0000 7f7c2d7fa640 1 -- 172.21.15.82:0/1907759906 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f7c2801c070 con 0x7f7c3019c810 2024-05-20T16:15:39.208 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:39.208+0000 7f7c2d7fa640 1 --2- 172.21.15.82:0/1907759906 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f7c180a2cc0 0x7f7c180a51d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:15:39.210 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:39.209+0000 7f7c2d7fa640 1 -- 172.21.15.82:0/1907759906 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9150+0+0 (secure 0 0 0) 0x7f7c280cecb0 con 0x7f7c3019c810 2024-05-20T16:15:39.211 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:39.210+0000 7f7c2ffff640 1 --2- 172.21.15.82:0/1907759906 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f7c180a2cc0 0x7f7c180a51d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:15:39.211 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:39.211+0000 7f7c3d67b740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561c37d362f0 0x561c380ff3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:15:39.212 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:39.211+0000 7f7c2ffff640 1 --2- 172.21.15.82:0/1907759906 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f7c180a2cc0 0x7f7c180a51d0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f7c20008000 tx=0x7f7c20031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:15:39.212 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:39.211+0000 7f7c3d67b740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561c37d3e4d0 0x561c381062e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:15:39.212 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:39.211+0000 7f7c3d67b740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x561c38063340 con 0x561c37d5f1b0 2024-05-20T16:15:39.212 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:39.211+0000 7f7c3d67b740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x561c3800b290 con 0x561c37d3e4d0 2024-05-20T16:15:39.212 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:39.211+0000 7f7c3d67b740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x561c3805f300 con 0x561c37d362f0 2024-05-20T16:15:39.216 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:39.212+0000 7f7c34d6b640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561c37d3e4d0 0x561c381062e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:15:39.216 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:39.212+0000 7f7c34d6b640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561c37d3e4d0 0x561c381062e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.82:59492/0 (socket says 172.21.15.82:59492) 2024-05-20T16:15:39.216 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:39.212+0000 7f7c34d6b640 1 -- 172.21.15.82:0/3715489692 learned_addr learned my addr 172.21.15.82:0/3715489692 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:15:39.216 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:39.212+0000 7f7c2f7fe640 1 --2- 172.21.15.82:0/3715489692 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561c37d362f0 0x561c380ff3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:15:39.216 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:39.213+0000 7f7c2f7fe640 1 -- 172.21.15.82:0/3715489692 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561c37d3e4d0 msgr2=0x561c381062e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:15:39.216 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:39.213+0000 7f7c2f7fe640 1 --2- 172.21.15.82:0/3715489692 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561c37d3e4d0 0x561c381062e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:15:39.216 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:39.213+0000 7f7c2f7fe640 1 -- 172.21.15.82:0/3715489692 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561c37d5f1b0 msgr2=0x561c380fee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:15:39.216 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:39.213+0000 7f7c2f7fe640 1 --2- 172.21.15.82:0/3715489692 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561c37d5f1b0 0x561c380fee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:15:39.216 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:39.214+0000 7f7c2f7fe640 1 -- 172.21.15.82:0/3715489692 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561c3802f8e0 con 0x561c37d362f0 2024-05-20T16:15:39.216 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:39.215+0000 7f7c2f7fe640 1 --2- 172.21.15.82:0/3715489692 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561c37d362f0 0x561c380ff3e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f7c24009ab0 tx=0x7f7c24009f70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f516f6bd98ea18d0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:15:39.218 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:39.217+0000 7f7c167fc640 1 -- 172.21.15.82:0/3715489692 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c24018020 con 0x561c37d362f0 2024-05-20T16:15:39.231 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:39.217+0000 7f7c167fc640 1 -- 172.21.15.82:0/3715489692 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c24010c20 con 0x561c37d362f0 2024-05-20T16:15:39.233 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:39.233+0000 7f7c3d67b740 1 -- 172.21.15.82:0/3715489692 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561c37d362f0 msgr2=0x561c380ff3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:15:39.233 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:39.233+0000 7f7c3d67b740 1 --2- 172.21.15.82:0/3715489692 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561c37d362f0 0x561c380ff3e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f7c24009ab0 tx=0x7f7c24009f70 comp rx=0 tx=0).stop 2024-05-20T16:15:39.234 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:39.234+0000 7f7c3d67b740 1 -- 172.21.15.82:0/3715489692 shutdown_connections 2024-05-20T16:15:39.234 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:39.234+0000 7f7c3d67b740 1 --2- 172.21.15.82:0/3715489692 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561c37d3e4d0 0x561c381062e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:15:39.234 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:39.234+0000 7f7c3d67b740 1 --2- 172.21.15.82:0/3715489692 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561c37d362f0 0x561c380ff3e0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:15:39.234 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:39.234+0000 7f7c3d67b740 1 --2- 172.21.15.82:0/3715489692 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561c37d5f1b0 0x561c380fee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:15:39.234 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:39.234+0000 7f7c3d67b740 1 -- 172.21.15.82:0/3715489692 >> 172.21.15.82:0/3715489692 conn(0x561c37d53300 msgr2=0x561c380fa7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:15:39.234 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:39.234+0000 7f7c3d67b740 1 -- 172.21.15.82:0/3715489692 shutdown_connections 2024-05-20T16:15:39.234 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:39.234+0000 7f7c3d67b740 1 -- 172.21.15.82:0/3715489692 wait complete. 2024-05-20T16:15:39.235 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:39.235+0000 7f7c3d67b740 1 -- start start 2024-05-20T16:15:39.241 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:39.239+0000 7f7c3d67b740 1 Processor -- start 2024-05-20T16:15:39.242 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:39.242+0000 7f7c3d67b740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561c37d3e4d0 0x561c38108a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:15:39.242 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:39.243+0000 7f7c3d67b740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561c37d5f1b0 0x561c38108f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:15:39.242 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:39.243+0000 7f7c3d67b740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561c37d1dd80 0x561c3810feb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:15:39.243 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:39.243+0000 7f7c3d67b740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x561c38106dd0 con 0x561c37d1dd80 2024-05-20T16:15:39.243 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:39.243+0000 7f7c3d67b740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x561c3801e790 con 0x561c37d3e4d0 2024-05-20T16:15:39.243 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:39.243+0000 7f7c3d67b740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x561c38106f50 con 0x561c37d5f1b0 2024-05-20T16:15:39.244 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:39.244+0000 7f7c3d67b740 10 client.0 _ll_register_callbacks cb 0x561c38059a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T16:15:39.244 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:39.244+0000 7f7c2f7fe640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561c37d5f1b0 0x561c38108f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:15:39.244 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:39.244+0000 7f7c2f7fe640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561c37d5f1b0 0x561c38108f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.82:47028/0 (socket says 172.21.15.82:47028) 2024-05-20T16:15:39.244 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:39.244+0000 7f7c2f7fe640 1 -- 172.21.15.82:0/1174131347 learned_addr learned my addr 172.21.15.82:0/1174131347 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:15:39.244 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:39.244+0000 7f7c2ffff640 1 --2- 172.21.15.82:0/1174131347 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561c37d3e4d0 0x561c38108a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:15:39.244 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:39.245+0000 7f7c34d6b640 1 --2- 172.21.15.82:0/1174131347 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561c37d1dd80 0x561c3810feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:15:39.245 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:39.245+0000 7f7c34d6b640 1 -- 172.21.15.82:0/1174131347 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561c37d5f1b0 msgr2=0x561c38108f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:15:39.245 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:39.245+0000 7f7c34d6b640 1 --2- 172.21.15.82:0/1174131347 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561c37d5f1b0 0x561c38108f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:15:39.245 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:39.245+0000 7f7c34d6b640 1 -- 172.21.15.82:0/1174131347 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561c37d3e4d0 msgr2=0x561c38108a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:15:39.246 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:39.245+0000 7f7c34d6b640 1 --2- 172.21.15.82:0/1174131347 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561c37d3e4d0 0x561c38108a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:15:39.246 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:39.246+0000 7f7c34d6b640 1 -- 172.21.15.82:0/1174131347 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561c37d77210 con 0x561c37d1dd80 2024-05-20T16:15:39.246 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:39.246+0000 7f7c34d6b640 1 --2- 172.21.15.82:0/1174131347 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561c37d1dd80 0x561c3810feb0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f7c2809b160 tx=0x7f7c2800ce70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b64b5f45438ce2c5 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:15:39.246 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:39.246+0000 7f7c0f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-05-20T16:15:39.246 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:39.247+0000 7f7c0f7fe640 1 -- 172.21.15.82:0/1174131347 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c28097840 con 0x561c37d1dd80 2024-05-20T16:15:39.247 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:39.247+0000 7f7c3d67b740 1 -- 172.21.15.82:0/1174131347 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x561c38070a70 con 0x561c37d1dd80 2024-05-20T16:15:39.247 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:39.247+0000 7f7c0f7fe640 1 -- 172.21.15.82:0/1174131347 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9150+0+0 (secure 0 0 0) 0x7f7c28014ad0 con 0x561c37d1dd80 2024-05-20T16:15:39.248 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:39.248+0000 7f7c0f7fe640 1 -- 172.21.15.82:0/1174131347 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f7c280979e0 con 0x561c37d1dd80 2024-05-20T16:15:39.249 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:39.250+0000 7f7c3d67b740 10 client.15582 fetch_fsmap learned FSMap version 25 2024-05-20T16:15:39.249 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:39.250+0000 7f7c3d67b740 1 -- 172.21.15.82:0/1174131347 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x561c37e43620 con 0x561c37d1dd80 2024-05-20T16:15:39.254 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:39.255+0000 7f7c0f7fe640 1 -- 172.21.15.82:0/1174131347 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x561c38070a70 con 0x561c37d1dd80 2024-05-20T16:15:39.256 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:39.257+0000 7f7c0f7fe640 1 -- 172.21.15.82:0/1174131347 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f7c280a3900 con 0x561c37d1dd80 2024-05-20T16:15:39.260 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:39.259+0000 7f7c3d67b740 10 client.15582 fetch_fsmap finished waiting for FSMap version 25 2024-05-20T16:15:39.260 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:39.259+0000 7f7c3d67b740 10 client.15582 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T16:15:39.260 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:39.259+0000 7f7c3d67b740 20 client.15582 populate_metadata read hostname 'smithi082' 2024-05-20T16:15:39.260 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:39.259+0000 7f7c3d67b740 1 --2- 172.21.15.82:0/1174131347 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x561c37cf31c0 0x561c38116230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:15:39.260 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:39.259+0000 7f7c3d67b740 4 client.15582 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T16:15:39.260 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:39.259+0000 7f7c3d67b740 1 -- 172.21.15.82:0/1174131347 --> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561c3806adc0 con 0x561c37cf31c0 2024-05-20T16:15:39.261 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:39.261+0000 7f7c2ffff640 1 --2- 172.21.15.82:0/1174131347 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x561c37cf31c0 0x561c38116230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:15:39.264 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:39.264+0000 7f7c2ffff640 1 --2- 172.21.15.82:0/1174131347 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x561c37cf31c0 0x561c38116230 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6078ae60a41f7498 server_cookie=2b2bf003f6861cbd in_seq=0 out_seq=0 2024-05-20T16:15:39.264 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:39.264+0000 7f7c0f7fe640 10 client.15582 ms_handle_connect on v2:172.21.15.160:6838/2765554482 2024-05-20T16:15:39.268 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:39.268+0000 7f7c0f7fe640 1 -- 172.21.15.82:0/1174131347 <== mds.0 v2:172.21.15.160:6838/2765554482 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x561c3806adc0 con 0x561c37cf31c0 2024-05-20T16:15:39.268 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:39.268+0000 7f7c0f7fe640 10 client.15582 handle_command_reply: tid=0 2024-05-20T16:15:39.268 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:39.268+0000 7f7c3d67b740 1 client.15582 shutdown 2024-05-20T16:15:39.280 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:39.279+0000 7f7c3d67b740 1 -- 172.21.15.82:0/1174131347 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561c37d1dd80 msgr2=0x561c3810feb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:15:39.280 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:39.279+0000 7f7c3d67b740 1 --2- 172.21.15.82:0/1174131347 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561c37d1dd80 0x561c3810feb0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f7c2809b160 tx=0x7f7c2800ce70 comp rx=0 tx=0).stop 2024-05-20T16:15:39.281 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:39.280+0000 7f7c3d67b740 1 -- 172.21.15.82:0/1174131347 shutdown_connections 2024-05-20T16:15:39.281 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:39.280+0000 7f7c3d67b740 1 --2- 172.21.15.82:0/1174131347 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x561c37cf31c0 0x561c38116230 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:15:39.281 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:39.280+0000 7f7c3d67b740 1 --2- 172.21.15.82:0/1174131347 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561c37d1dd80 0x561c3810feb0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:15:39.281 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:39.280+0000 7f7c3d67b740 1 --2- 172.21.15.82:0/1174131347 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561c37d5f1b0 0x561c38108f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:15:39.281 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:39.280+0000 7f7c3d67b740 1 --2- 172.21.15.82:0/1174131347 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561c37d3e4d0 0x561c38108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:15:39.281 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:39.280+0000 7f7c3d67b740 1 -- 172.21.15.82:0/1174131347 >> 172.21.15.82:0/1174131347 conn(0x561c37d53300 msgr2=0x561c380f9810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:15:39.282 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:39.280+0000 7f7c0f7fe640 0 client.15582 ms_handle_reset on v2:172.21.15.160:6838/2765554482 2024-05-20T16:15:39.282 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:39.280+0000 7f7c3d67b740 1 -- 172.21.15.82:0/1174131347 shutdown_connections 2024-05-20T16:15:39.282 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:39.280+0000 7f7c3d67b740 1 -- 172.21.15.82:0/1174131347 wait complete. 2024-05-20T16:15:39.282 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:39.280+0000 7f7c3d67b740 20 client.15582 trim_cache size 0 max 16384 2024-05-20T16:15:39.317 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:39.317+0000 7f7c3d67b740 1 Processor -- start 2024-05-20T16:15:39.317 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:39.317+0000 7f7c3d67b740 1 -- start start 2024-05-20T16:15:39.318 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:39.318+0000 7f7c3d67b740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561c37d1dd80 0x561c38107340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:15:39.318 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:39.318+0000 7f7c3d67b740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561c37d3e4d0 0x561c38107880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:15:39.318 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:39.318+0000 7f7c3d67b740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561c37d5f1b0 0x561c3811aa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:15:39.319 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:39.318+0000 7f7c3d67b740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f7c301c60e0 con 0x561c37d1dd80 2024-05-20T16:15:39.319 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:39.319+0000 7f7c3d67b740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f7c301c39b0 con 0x561c37d3e4d0 2024-05-20T16:15:39.319 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:39.319+0000 7f7c2ffff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561c37d1dd80 0x561c38107340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:15:39.319 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:39.319+0000 7f7c2ffff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561c37d1dd80 0x561c38107340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:55284/0 (socket says 172.21.15.82:55284) 2024-05-20T16:15:39.319 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:39.319+0000 7f7c2ffff640 1 -- 172.21.15.82:0/74484545 learned_addr learned my addr 172.21.15.82:0/74484545 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:15:39.319 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:39.319+0000 7f7c2f7fe640 1 --2- 172.21.15.82:0/74484545 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561c37d3e4d0 0x561c38107880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:15:39.319 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:39.319+0000 7f7c3d67b740 1 -- 172.21.15.82:0/74484545 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f7c3019e780 con 0x561c37d5f1b0 2024-05-20T16:15:39.319 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:39.319+0000 7f7c34d6b640 1 --2- 172.21.15.82:0/74484545 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561c37d5f1b0 0x561c3811aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:15:39.320 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:39.320+0000 7f7c2ffff640 1 -- 172.21.15.82:0/74484545 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561c37d5f1b0 msgr2=0x561c3811aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:15:39.320 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:39.320+0000 7f7c2ffff640 1 --2- 172.21.15.82:0/74484545 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561c37d5f1b0 0x561c3811aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:15:39.320 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:39.320+0000 7f7c2ffff640 1 -- 172.21.15.82:0/74484545 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561c37d3e4d0 msgr2=0x561c38107880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:15:39.321 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:39.320+0000 7f7c2ffff640 1 --2- 172.21.15.82:0/74484545 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561c37d3e4d0 0x561c38107880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:15:39.321 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:39.320+0000 7f7c2ffff640 1 -- 172.21.15.82:0/74484545 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561c380ffec0 con 0x561c37d1dd80 2024-05-20T16:15:39.321 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:39.320+0000 7f7c34d6b640 1 --2- 172.21.15.82:0/74484545 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561c37d5f1b0 0x561c3811aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T16:15:39.321 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:39.320+0000 7f7c2ffff640 1 --2- 172.21.15.82:0/74484545 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561c37d1dd80 0x561c38107340 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f7c20009900 tx=0x7f7c2003c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a23a1c0521255c0a server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:15:39.321 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:39.320+0000 7f7c15ffb640 1 -- 172.21.15.82:0/74484545 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c20035050 con 0x561c37d1dd80 2024-05-20T16:15:39.321 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:39.321+0000 7f7c15ffb640 1 -- 172.21.15.82:0/74484545 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c20056070 con 0x561c37d1dd80 2024-05-20T16:15:39.321 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:39.321+0000 7f7c15ffb640 1 -- 172.21.15.82:0/74484545 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c200035f0 con 0x561c37d1dd80 2024-05-20T16:15:39.322 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:39.322+0000 7f7c3d67b740 1 -- 172.21.15.82:0/74484545 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561c37d1dd80 msgr2=0x561c38107340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:15:39.322 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:39.322+0000 7f7c3d67b740 1 --2- 172.21.15.82:0/74484545 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561c37d1dd80 0x561c38107340 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f7c20009900 tx=0x7f7c2003c000 comp rx=0 tx=0).stop 2024-05-20T16:15:39.322 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:39.322+0000 7f7c3d67b740 1 -- 172.21.15.82:0/74484545 shutdown_connections 2024-05-20T16:15:39.322 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:39.322+0000 7f7c3d67b740 1 --2- 172.21.15.82:0/74484545 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561c37d5f1b0 0x561c3811aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:15:39.322 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:39.322+0000 7f7c3d67b740 1 --2- 172.21.15.82:0/74484545 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561c37d3e4d0 0x561c38107880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:15:39.323 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:39.322+0000 7f7c3d67b740 1 --2- 172.21.15.82:0/74484545 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561c37d1dd80 0x561c38107340 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:15:39.323 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:39.322+0000 7f7c3d67b740 1 -- 172.21.15.82:0/74484545 >> 172.21.15.82:0/74484545 conn(0x561c37d53300 msgr2=0x561c38102da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:15:39.323 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:39.323+0000 7f7c3d67b740 1 -- 172.21.15.82:0/74484545 shutdown_connections 2024-05-20T16:15:39.323 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:39.323+0000 7f7c3d67b740 1 -- 172.21.15.82:0/74484545 wait complete. 2024-05-20T16:15:39.323 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:39.324+0000 7f7c3d67b740 1 -- start start 2024-05-20T16:15:39.324 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:39.324+0000 7f7c3d67b740 1 Processor -- start 2024-05-20T16:15:39.324 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:39.324+0000 7f7c3d67b740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561c37d1dd80 0x561c3810dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:15:39.324 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:39.325+0000 7f7c3d67b740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561c37d3e4d0 0x561c3810e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:15:39.325 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:39.325+0000 7f7c2f7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561c37d3e4d0 0x561c3810e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:15:39.325 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:39.325+0000 7f7c2f7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561c37d3e4d0 0x561c3810e4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:55300/0 (socket says 172.21.15.82:55300) 2024-05-20T16:15:39.325 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:39.325+0000 7f7c2f7fe640 1 -- 172.21.15.82:0/192507998 learned_addr learned my addr 172.21.15.82:0/192507998 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:15:39.325 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:39.325+0000 7f7c3d67b740 1 --2- 172.21.15.82:0/192507998 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561c37d5f1b0 0x561c3810ea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:15:39.325 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:39.325+0000 7f7c3d67b740 1 -- 172.21.15.82:0/192507998 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f7c30073240 con 0x561c37d3e4d0 2024-05-20T16:15:39.326 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:39.325+0000 7f7c3d67b740 1 -- 172.21.15.82:0/192507998 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f7c301c32e0 con 0x561c37d1dd80 2024-05-20T16:15:39.326 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:39.325+0000 7f7c3d67b740 1 -- 172.21.15.82:0/192507998 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f7c30073910 con 0x561c37d5f1b0 2024-05-20T16:15:39.326 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:39.325+0000 7f7c3d67b740 10 client.0 _ll_register_callbacks cb 0x7f7c2809ade0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T16:15:39.326 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:39.325+0000 7f7c2ffff640 1 --2- 172.21.15.82:0/192507998 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561c37d1dd80 0x561c3810dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:15:39.326 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:39.326+0000 7f7c2f7fe640 1 -- 172.21.15.82:0/192507998 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561c37d5f1b0 msgr2=0x561c3810ea20 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-20T16:15:39.326 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:39.326+0000 7f7c2f7fe640 1 --2- 172.21.15.82:0/192507998 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561c37d5f1b0 0x561c3810ea20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:15:39.326 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:39.326+0000 7f7c2f7fe640 1 -- 172.21.15.82:0/192507998 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561c37d1dd80 msgr2=0x561c3810dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:15:39.326 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:39.326+0000 7f7c2f7fe640 1 --2- 172.21.15.82:0/192507998 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561c37d1dd80 0x561c3810dfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:15:39.326 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:39.326+0000 7f7c2f7fe640 1 -- 172.21.15.82:0/192507998 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561c3817c700 con 0x561c37d3e4d0 2024-05-20T16:15:39.328 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:39.326+0000 7f7c2f7fe640 1 --2- 172.21.15.82:0/192507998 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561c37d3e4d0 0x561c3810e4e0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f7c24001eb0 tx=0x7f7c24013a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e375bbe6ff45765 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:15:39.328 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:39.326+0000 7f7c0f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-05-20T16:15:39.328 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:39.326+0000 7f7c0f7fe640 1 -- 172.21.15.82:0/192507998 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c24018040 con 0x561c37d3e4d0 2024-05-20T16:15:39.328 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:39.326+0000 7f7c0f7fe640 1 -- 172.21.15.82:0/192507998 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9150+0+0 (secure 0 0 0) 0x7f7c24013d90 con 0x561c37d3e4d0 2024-05-20T16:15:39.328 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:39.326+0000 7f7c3d67b740 1 -- 172.21.15.82:0/192507998 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7c20056070 con 0x561c37d3e4d0 2024-05-20T16:15:39.328 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:39.326+0000 7f7c0f7fe640 1 -- 172.21.15.82:0/192507998 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f7c24030050 con 0x561c37d3e4d0 2024-05-20T16:15:39.328 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:39.326+0000 7f7c3d67b740 10 client.15588 fetch_fsmap learned FSMap version 25 2024-05-20T16:15:39.328 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:39.326+0000 7f7c3d67b740 1 -- 172.21.15.82:0/192507998 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x561c3817d9a0 con 0x561c37d3e4d0 2024-05-20T16:15:39.328 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:39.327+0000 7f7c0f7fe640 1 -- 172.21.15.82:0/192507998 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c24014950 con 0x561c37d3e4d0 2024-05-20T16:15:39.328 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:39.327+0000 7f7c0f7fe640 1 -- 172.21.15.82:0/192507998 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f7c24027860 con 0x561c37d3e4d0 2024-05-20T16:15:39.328 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:39.327+0000 7f7c3d67b740 10 client.15588 fetch_fsmap finished waiting for FSMap version 25 2024-05-20T16:15:39.328 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:39.327+0000 7f7c3d67b740 10 client.15588 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T16:15:39.328 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:39.327+0000 7f7c3d67b740 20 client.15588 populate_metadata read hostname 'smithi082' 2024-05-20T16:15:39.328 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:39.327+0000 7f7c3d67b740 1 --2- 172.21.15.82:0/192507998 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x561c3817e050 0x561c38180470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:15:39.329 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:39.327+0000 7f7c3d67b740 4 client.15588 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-20T16:15:39.329 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:39.327+0000 7f7c3d67b740 1 -- 172.21.15.82:0/192507998 --> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f7c20056070 con 0x561c3817e050 2024-05-20T16:15:39.329 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:39.328+0000 7f7c2ffff640 1 --2- 172.21.15.82:0/192507998 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x561c3817e050 0x561c38180470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:15:39.330 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:39.331+0000 7f7c2ffff640 1 --2- 172.21.15.82:0/192507998 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x561c3817e050 0x561c38180470 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=26636a36781741ee server_cookie=71b3b75ad4c84e2e in_seq=0 out_seq=0 2024-05-20T16:15:39.330 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:39.331+0000 7f7c0f7fe640 10 client.15588 ms_handle_connect on v2:172.21.15.160:6838/2765554482 2024-05-20T16:15:39.672 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:39.671+0000 7f7c0f7fe640 1 -- 172.21.15.82:0/192507998 <== mds.0 v2:172.21.15.160:6838/2765554482 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f7c20056070 con 0x561c3817e050 2024-05-20T16:15:39.672 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:39.671+0000 7f7c0f7fe640 10 client.15588 handle_command_reply: tid=0 2024-05-20T16:15:39.672 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:39.671+0000 7f7c3d67b740 1 client.15588 shutdown 2024-05-20T16:15:39.673 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:39.672+0000 7f7c3d67b740 1 -- 172.21.15.82:0/192507998 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561c37d3e4d0 msgr2=0x561c3810e4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:15:39.673 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:39.672+0000 7f7c3d67b740 1 --2- 172.21.15.82:0/192507998 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561c37d3e4d0 0x561c3810e4e0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f7c24001eb0 tx=0x7f7c24013a90 comp rx=0 tx=0).stop 2024-05-20T16:15:39.673 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:39.672+0000 7f7c3d67b740 1 -- 172.21.15.82:0/192507998 shutdown_connections 2024-05-20T16:15:39.673 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:39.672+0000 7f7c3d67b740 1 --2- 172.21.15.82:0/192507998 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x561c3817e050 0x561c38180470 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:15:39.673 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:39.672+0000 7f7c3d67b740 1 --2- 172.21.15.82:0/192507998 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561c37d3e4d0 0x561c3810e4e0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:15:39.673 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:39.672+0000 7f7c3d67b740 1 --2- 172.21.15.82:0/192507998 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561c37d5f1b0 0x561c3810ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:15:39.673 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:39.672+0000 7f7c3d67b740 1 --2- 172.21.15.82:0/192507998 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561c37d1dd80 0x561c3810dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:15:39.673 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:39.672+0000 7f7c3d67b740 1 -- 172.21.15.82:0/192507998 >> 172.21.15.82:0/192507998 conn(0x561c37d53300 msgr2=0x561c38107340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:15:39.673 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:39.672+0000 7f7c0f7fe640 0 client.15588 ms_handle_reset on v2:172.21.15.160:6838/2765554482 2024-05-20T16:15:39.674 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:39.672+0000 7f7c3d67b740 1 -- 172.21.15.82:0/192507998 shutdown_connections 2024-05-20T16:15:39.674 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:39.672+0000 7f7c3d67b740 1 -- 172.21.15.82:0/192507998 wait complete. 2024-05-20T16:15:39.674 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:39.673+0000 7f7c3d67b740 20 client.15588 trim_cache size 0 max 16384 2024-05-20T16:15:39.675 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:{ 2024-05-20T16:15:39.675 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "message": "", 2024-05-20T16:15:39.675 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "return_code": 0 2024-05-20T16:15:39.675 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:} 2024-05-20T16:15:39.678 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:39.677+0000 7f7c367ce640 1 -- 172.21.15.82:0/1907759906 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f7c180a2cc0 msgr2=0x7f7c180a51d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:15:39.678 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:39.677+0000 7f7c367ce640 1 --2- 172.21.15.82:0/1907759906 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f7c180a2cc0 0x7f7c180a51d0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f7c20008000 tx=0x7f7c20031400 comp rx=0 tx=0).stop 2024-05-20T16:15:39.678 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:39.677+0000 7f7c367ce640 1 -- 172.21.15.82:0/1907759906 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7c3019c810 msgr2=0x7f7c3019cc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:15:39.678 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:39.677+0000 7f7c367ce640 1 --2- 172.21.15.82:0/1907759906 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7c3019c810 0x7f7c3019cc20 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f7c2800be20 tx=0x7f7c2800bf10 comp rx=0 tx=0).stop 2024-05-20T16:15:39.678 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:39.678+0000 7f7c367ce640 1 -- 172.21.15.82:0/1907759906 shutdown_connections 2024-05-20T16:15:39.679 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:39.678+0000 7f7c367ce640 1 --2- 172.21.15.82:0/1907759906 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f7c180a2cc0 0x7f7c180a51d0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:15:39.679 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:39.678+0000 7f7c367ce640 1 --2- 172.21.15.82:0/1907759906 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7c301070b0 0x7f7c30197910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:15:39.679 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:39.678+0000 7f7c367ce640 1 --2- 172.21.15.82:0/1907759906 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7c3019c810 0x7f7c3019cc20 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:15:39.679 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:39.678+0000 7f7c367ce640 1 --2- 172.21.15.82:0/1907759906 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7c301023b0 0x7f7c30197350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:15:39.679 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:39.678+0000 7f7c367ce640 1 -- 172.21.15.82:0/1907759906 >> 172.21.15.82:0/1907759906 conn(0x7f7c300fc340 msgr2=0x7f7c30103090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:15:39.679 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:39.680+0000 7f7c367ce640 1 -- 172.21.15.82:0/1907759906 shutdown_connections 2024-05-20T16:15:39.680 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:39.680+0000 7f7c367ce640 1 -- 172.21.15.82:0/1907759906 wait complete. 2024-05-20T16:15:39.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:15:39 smithi160 ceph-mon[36347]: pgmap v412: 129 pgs: 129 active+clean; 1.6 GiB data, 500 MiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 133 MiB/s wr, 96 op/s 2024-05-20T16:15:39.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:15:39 smithi082 ceph-mon[32238]: pgmap v412: 129 pgs: 129 active+clean; 1.6 GiB data, 500 MiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 133 MiB/s wr, 96 op/s 2024-05-20T16:15:39.885 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:15:39 smithi184 ceph-mon[36209]: pgmap v412: 129 pgs: 129 active+clean; 1.6 GiB data, 500 MiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 133 MiB/s wr, 96 op/s 2024-05-20T16:15:41.434 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:15:41 smithi082 ceph-mon[32238]: pgmap v413: 129 pgs: 129 active+clean; 2.6 GiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 218 MiB/s wr, 116 op/s 2024-05-20T16:15:41.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:15:41 smithi160 ceph-mon[36347]: pgmap v413: 129 pgs: 129 active+clean; 2.6 GiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 218 MiB/s wr, 116 op/s 2024-05-20T16:15:41.885 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:15:41 smithi184 ceph-mon[36209]: pgmap v413: 129 pgs: 129 active+clean; 2.6 GiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 218 MiB/s wr, 116 op/s 2024-05-20T16:15:43.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:15:43 smithi160 ceph-mon[36347]: pgmap v414: 129 pgs: 129 active+clean; 4.2 GiB data, 926 MiB used, 1.0 TiB / 1.0 TiB avail; 679 KiB/s rd, 357 MiB/s wr, 278 op/s 2024-05-20T16:15:43.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:15:43 smithi082 ceph-mon[32238]: pgmap v414: 129 pgs: 129 active+clean; 4.2 GiB data, 926 MiB used, 1.0 TiB / 1.0 TiB avail; 679 KiB/s rd, 357 MiB/s wr, 278 op/s 2024-05-20T16:15:43.885 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:15:43 smithi184 ceph-mon[36209]: pgmap v414: 129 pgs: 129 active+clean; 4.2 GiB data, 926 MiB used, 1.0 TiB / 1.0 TiB avail; 679 KiB/s rd, 357 MiB/s wr, 278 op/s 2024-05-20T16:15:45.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:15:45 smithi160 ceph-mon[36347]: pgmap v415: 129 pgs: 129 active+clean; 7.0 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1020 KiB/s rd, 616 MiB/s wr, 664 op/s 2024-05-20T16:15:45.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:15:45 smithi082 ceph-mon[32238]: pgmap v415: 129 pgs: 129 active+clean; 7.0 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1020 KiB/s rd, 616 MiB/s wr, 664 op/s 2024-05-20T16:15:45.885 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:15:45 smithi184 ceph-mon[36209]: pgmap v415: 129 pgs: 129 active+clean; 7.0 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1020 KiB/s rd, 616 MiB/s wr, 664 op/s 2024-05-20T16:15:47.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:15:47 smithi160 ceph-mon[36347]: pgmap v416: 129 pgs: 129 active+clean; 8.1 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 729 MiB/s wr, 838 op/s 2024-05-20T16:15:47.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:15:47 smithi082 ceph-mon[32238]: pgmap v416: 129 pgs: 129 active+clean; 8.1 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 729 MiB/s wr, 838 op/s 2024-05-20T16:15:47.885 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:15:47 smithi184 ceph-mon[36209]: pgmap v416: 129 pgs: 129 active+clean; 8.1 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 729 MiB/s wr, 838 op/s 2024-05-20T16:15:49.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:15:49 smithi160 ceph-mon[36347]: pgmap v417: 129 pgs: 129 active+clean; 9.7 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 947 MiB/s wr, 1.20k op/s 2024-05-20T16:15:49.867 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:15:49 smithi082 ceph-mon[32238]: pgmap v417: 129 pgs: 129 active+clean; 9.7 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 947 MiB/s wr, 1.20k op/s 2024-05-20T16:15:49.885 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:15:49 smithi184 ceph-mon[36209]: pgmap v417: 129 pgs: 129 active+clean; 9.7 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 947 MiB/s wr, 1.20k op/s 2024-05-20T16:15:51.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:15:51 smithi160 ceph-mon[36347]: pgmap v418: 129 pgs: 129 active+clean; 9.9 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 912 MiB/s wr, 1.31k op/s 2024-05-20T16:15:51.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:15:51 smithi082 ceph-mon[32238]: pgmap v418: 129 pgs: 129 active+clean; 9.9 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 912 MiB/s wr, 1.31k op/s 2024-05-20T16:15:51.885 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:15:51 smithi184 ceph-mon[36209]: pgmap v418: 129 pgs: 129 active+clean; 9.9 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 912 MiB/s wr, 1.31k op/s 2024-05-20T16:15:53.022 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:53.022+0000 7f8bd5338640 1 -- 172.21.15.82:0/2260150324 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8bd010a340 msgr2=0x7f8bd010a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:15:53.023 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:53.022+0000 7f8bd5338640 1 --2- 172.21.15.82:0/2260150324 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8bd010a340 0x7f8bd010a750 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f8bc8009ca0 tx=0x7f8bc802a8e0 comp rx=0 tx=0).stop 2024-05-20T16:15:53.030 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:53.027+0000 7f8bd5338640 1 -- 172.21.15.82:0/2260150324 shutdown_connections 2024-05-20T16:15:53.030 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:53.027+0000 7f8bd5338640 1 --2- 172.21.15.82:0/2260150324 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8bd010f040 0x7f8bd0111460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:15:53.030 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:53.027+0000 7f8bd5338640 1 --2- 172.21.15.82:0/2260150324 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8bd010a340 0x7f8bd010a750 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:15:53.030 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:53.027+0000 7f8bd5338640 1 --2- 172.21.15.82:0/2260150324 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8bd0109100 0x7f8bd0109e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:15:53.030 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:53.027+0000 7f8bd5338640 1 -- 172.21.15.82:0/2260150324 >> 172.21.15.82:0/2260150324 conn(0x7f8bd0077d90 msgr2=0x7f8bd0106970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:15:53.033 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:53.034+0000 7f8bd5338640 1 -- 172.21.15.82:0/2260150324 shutdown_connections 2024-05-20T16:15:53.035 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:53.036+0000 7f8bd5338640 1 -- 172.21.15.82:0/2260150324 wait complete. 2024-05-20T16:15:53.037 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:53.037+0000 7f8bd5338640 1 Processor -- start 2024-05-20T16:15:53.037 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:53.037+0000 7f8bd5338640 1 -- start start 2024-05-20T16:15:53.037 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:53.038+0000 7f8bd5338640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8bd0109100 0x7f8bd00717c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:15:53.038 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:53.038+0000 7f8bd5338640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8bd010a340 0x7f8bd0071d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:15:53.038 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:53.038+0000 7f8bd5338640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8bd010f040 0x7f8bd0072c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:15:53.039 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:53.039+0000 7f8bd5338640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f8bd006baa0 con 0x7f8bd010f040 2024-05-20T16:15:53.039 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:53.039+0000 7f8bd5338640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f8bd006b920 con 0x7f8bd0109100 2024-05-20T16:15:53.039 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:53.039+0000 7f8bd5338640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f8bd006bc20 con 0x7f8bd010a340 2024-05-20T16:15:53.039 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:53.039+0000 7f8bceffd640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8bd0109100 0x7f8bd00717c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:15:53.039 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:53.039+0000 7f8bceffd640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8bd0109100 0x7f8bd00717c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.82:35808/0 (socket says 172.21.15.82:35808) 2024-05-20T16:15:53.039 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:53.039+0000 7f8bceffd640 1 -- 172.21.15.82:0/2685029215 learned_addr learned my addr 172.21.15.82:0/2685029215 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:15:53.042 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:53.042+0000 7f8bcf7fe640 1 --2- 172.21.15.82:0/2685029215 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8bd010f040 0x7f8bd0072c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:15:53.044 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:53.044+0000 7f8bceffd640 1 -- 172.21.15.82:0/2685029215 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8bd010a340 msgr2=0x7f8bd0071d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:15:53.044 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:53.044+0000 7f8bceffd640 1 --2- 172.21.15.82:0/2685029215 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8bd010a340 0x7f8bd0071d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:15:53.044 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:53.044+0000 7f8bceffd640 1 -- 172.21.15.82:0/2685029215 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8bd010f040 msgr2=0x7f8bd0072c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:15:53.044 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:53.044+0000 7f8bceffd640 1 --2- 172.21.15.82:0/2685029215 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8bd010f040 0x7f8bd0072c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:15:53.044 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:53.044+0000 7f8bceffd640 1 -- 172.21.15.82:0/2685029215 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bd00732d0 con 0x7f8bd0109100 2024-05-20T16:15:53.046 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:53.047+0000 7f8bceffd640 1 --2- 172.21.15.82:0/2685029215 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8bd0109100 0x7f8bd00717c0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f8bc000d740 tx=0x7f8bc000dc00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:15:53.051 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:53.052+0000 7f8bb7fff640 1 -- 172.21.15.82:0/2685029215 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bc0017070 con 0x7f8bd0109100 2024-05-20T16:15:53.052 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:53.052+0000 7f8bb7fff640 1 -- 172.21.15.82:0/2685029215 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8bc000de50 con 0x7f8bd0109100 2024-05-20T16:15:53.052 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:53.052+0000 7f8bb7fff640 1 -- 172.21.15.82:0/2685029215 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bc0003e00 con 0x7f8bd0109100 2024-05-20T16:15:53.055 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:53.053+0000 7f8bd5338640 1 -- 172.21.15.82:0/2685029215 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8bd00735c0 con 0x7f8bd0109100 2024-05-20T16:15:53.055 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:53.053+0000 7f8bd5338640 1 -- 172.21.15.82:0/2685029215 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8bd0073ac0 con 0x7f8bd0109100 2024-05-20T16:15:53.061 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:53.062+0000 7f8bb7fff640 1 -- 172.21.15.82:0/2685029215 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f8bc0010460 con 0x7f8bd0109100 2024-05-20T16:15:53.063 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:53.063+0000 7f8bb7fff640 1 --2- 172.21.15.82:0/2685029215 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f8bb80a2d90 0x7f8bb80a52a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:15:53.064 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:53.064+0000 7f8bb7fff640 1 -- 172.21.15.82:0/2685029215 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9150+0+0 (secure 0 0 0) 0x7f8bc00cfd00 con 0x7f8bd0109100 2024-05-20T16:15:53.068 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:53.067+0000 7f8bdc1e4740 1 Processor -- start 2024-05-20T16:15:53.068 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:53.068+0000 7f8bdc1e4740 1 -- start start 2024-05-20T16:15:53.068 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:53.068+0000 7f8bce7fc640 1 --2- 172.21.15.82:0/2685029215 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f8bb80a2d90 0x7f8bb80a52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:15:53.068 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:53.068+0000 7f8bdc1e4740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56369d8191b0 0x56369dbb8e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:15:53.068 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:53.068+0000 7f8bdc1e4740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56369d7f02f0 0x56369dbb93e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:15:53.068 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:53.068+0000 7f8bdc1e4740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56369d7f84d0 0x56369dbc02e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:15:53.068 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:53.068+0000 7f8bdc1e4740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x56369db1d340 con 0x56369d7f84d0 2024-05-20T16:15:53.068 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:53.068+0000 7f8bdc1e4740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x56369dac5290 con 0x56369d8191b0 2024-05-20T16:15:53.068 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:53.068+0000 7f8bdc1e4740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x56369db19300 con 0x56369d7f02f0 2024-05-20T16:15:53.070 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:53.071+0000 7f8bce7fc640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56369d7f02f0 0x56369dbb93e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:15:53.071 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:53.071+0000 7f8bce7fc640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56369d7f02f0 0x56369dbb93e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.82:48012/0 (socket says 172.21.15.82:48012) 2024-05-20T16:15:53.071 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:53.071+0000 7f8bce7fc640 1 -- 172.21.15.82:0/954024564 learned_addr learned my addr 172.21.15.82:0/954024564 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:15:53.072 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:53.073+0000 7f8bceffd640 1 --2- 172.21.15.82:0/954024564 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56369d8191b0 0x56369dbb8e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:15:53.073 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:53.073+0000 7f8bce7fc640 1 -- 172.21.15.82:0/954024564 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56369d8191b0 msgr2=0x56369dbb8e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:15:53.073 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:53.073+0000 7f8bce7fc640 1 --2- 172.21.15.82:0/954024564 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56369d8191b0 0x56369dbb8e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:15:53.073 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:53.073+0000 7f8bce7fc640 1 -- 172.21.15.82:0/954024564 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56369d7f84d0 msgr2=0x56369dbc02e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:15:53.073 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:53.073+0000 7f8bce7fc640 1 --2- 172.21.15.82:0/954024564 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56369d7f84d0 0x56369dbc02e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:15:53.073 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:53.073+0000 7f8bce7fc640 1 -- 172.21.15.82:0/954024564 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56369dae98e0 con 0x56369d7f02f0 2024-05-20T16:15:53.074 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:53.075+0000 7f8bce7fc640 1 --2- 172.21.15.82:0/2685029215 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f8bb80a2d90 0x7f8bb80a52a0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f8bd0072860 tx=0x7f8bc8005df0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:15:53.076 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:53.077+0000 7f8bce7fc640 1 --2- 172.21.15.82:0/954024564 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56369d7f02f0 0x56369dbb93e0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f8bc8009ca0 tx=0x7f8bc8043000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8663f3f4febd3937 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:15:53.083 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:53.082+0000 7f8bb57fa640 1 -- 172.21.15.82:0/954024564 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bc8004040 con 0x56369d7f02f0 2024-05-20T16:15:53.083 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:53.082+0000 7f8bb57fa640 1 -- 172.21.15.82:0/954024564 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8bc8036040 con 0x56369d7f02f0 2024-05-20T16:15:53.083 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:53.082+0000 7f8bb57fa640 1 -- 172.21.15.82:0/954024564 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bc8030ce0 con 0x56369d7f02f0 2024-05-20T16:15:53.085 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:53.085+0000 7f8bdc1e4740 1 -- 172.21.15.82:0/954024564 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56369d7f02f0 msgr2=0x56369dbb93e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:15:53.085 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:53.085+0000 7f8bdc1e4740 1 --2- 172.21.15.82:0/954024564 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56369d7f02f0 0x56369dbb93e0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f8bc8009ca0 tx=0x7f8bc8043000 comp rx=0 tx=0).stop 2024-05-20T16:15:53.085 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:53.086+0000 7f8bdc1e4740 1 -- 172.21.15.82:0/954024564 shutdown_connections 2024-05-20T16:15:53.085 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:53.086+0000 7f8bdc1e4740 1 --2- 172.21.15.82:0/954024564 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56369d7f84d0 0x56369dbc02e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:15:53.085 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:53.086+0000 7f8bdc1e4740 1 --2- 172.21.15.82:0/954024564 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56369d7f02f0 0x56369dbb93e0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:15:53.086 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:53.086+0000 7f8bdc1e4740 1 --2- 172.21.15.82:0/954024564 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56369d8191b0 0x56369dbb8e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:15:53.086 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:53.086+0000 7f8bdc1e4740 1 -- 172.21.15.82:0/954024564 >> 172.21.15.82:0/954024564 conn(0x56369d80d300 msgr2=0x56369dbb47f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:15:53.088 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:53.088+0000 7f8bdc1e4740 1 -- 172.21.15.82:0/954024564 shutdown_connections 2024-05-20T16:15:53.089 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:53.089+0000 7f8bdc1e4740 1 -- 172.21.15.82:0/954024564 wait complete. 2024-05-20T16:15:53.089 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:53.089+0000 7f8bdc1e4740 1 -- start start 2024-05-20T16:15:53.089 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:53.090+0000 7f8bdc1e4740 1 Processor -- start 2024-05-20T16:15:53.090 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:53.090+0000 7f8bdc1e4740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56369d7f02f0 0x56369dbc2c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:15:53.090 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:53.090+0000 7f8bdc1e4740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56369d7f84d0 0x56369dbc31c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:15:53.090 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:53.090+0000 7f8bdc1e4740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56369d8191b0 0x56369dbca120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:15:53.090 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:53.090+0000 7f8bdc1e4740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x56369dbc0cd0 con 0x56369d7f84d0 2024-05-20T16:15:53.090 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:53.090+0000 7f8bdc1e4740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x56369dad8790 con 0x56369d7f02f0 2024-05-20T16:15:53.091 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:53.090+0000 7f8bdc1e4740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x56369dbc0e50 con 0x56369d8191b0 2024-05-20T16:15:53.091 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:53.090+0000 7f8bdc1e4740 10 client.0 _ll_register_callbacks cb 0x56369db13a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T16:15:53.092 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:53.092+0000 7f8bce7fc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56369d7f84d0 0x56369dbc31c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:15:53.092 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:53.092+0000 7f8bce7fc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56369d7f84d0 0x56369dbc31c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:47466/0 (socket says 172.21.15.82:47466) 2024-05-20T16:15:53.092 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:53.092+0000 7f8bce7fc640 1 -- 172.21.15.82:0/3713248910 learned_addr learned my addr 172.21.15.82:0/3713248910 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:15:53.092 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:53.092+0000 7f8bcf7fe640 1 --2- 172.21.15.82:0/3713248910 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56369d8191b0 0x56369dbca120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:15:53.094 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:53.094+0000 7f8bceffd640 1 --2- 172.21.15.82:0/3713248910 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56369d7f02f0 0x56369dbc2c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:15:53.094 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:53.095+0000 7f8bcf7fe640 1 -- 172.21.15.82:0/3713248910 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56369d7f02f0 msgr2=0x56369dbc2c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:15:53.095 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:53.095+0000 7f8bcf7fe640 1 --2- 172.21.15.82:0/3713248910 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56369d7f02f0 0x56369dbc2c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:15:53.095 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:53.095+0000 7f8bcf7fe640 1 -- 172.21.15.82:0/3713248910 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56369d7f84d0 msgr2=0x56369dbc31c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:15:53.095 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:53.095+0000 7f8bcf7fe640 1 --2- 172.21.15.82:0/3713248910 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56369d7f84d0 0x56369dbc31c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:15:53.095 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:53.095+0000 7f8bcf7fe640 1 -- 172.21.15.82:0/3713248910 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56369d831210 con 0x56369d8191b0 2024-05-20T16:15:53.096 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:53.096+0000 7f8bcf7fe640 1 --2- 172.21.15.82:0/3713248910 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56369d8191b0 0x56369dbca120 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x56369dbb5620 tx=0x7f8bc4011b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ac2763026b0fa400 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:15:53.096 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:53.096+0000 7f8bb2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-05-20T16:15:53.096 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:53.096+0000 7f8bb2ff5640 1 -- 172.21.15.82:0/3713248910 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bc400fd70 con 0x56369d8191b0 2024-05-20T16:15:53.096 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:53.097+0000 7f8bdc1e4740 1 -- 172.21.15.82:0/3713248910 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8bc8036040 con 0x56369d8191b0 2024-05-20T16:15:53.096 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:53.097+0000 7f8bb2ff5640 1 -- 172.21.15.82:0/3713248910 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9150+0+0 (secure 0 0 0) 0x7f8bc4005020 con 0x56369d8191b0 2024-05-20T16:15:53.098 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:53.099+0000 7f8bb2ff5640 1 -- 172.21.15.82:0/3713248910 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f8bc4024050 con 0x56369d8191b0 2024-05-20T16:15:53.100 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:53.100+0000 7f8bdc1e4740 10 client.25150 fetch_fsmap learned FSMap version 25 2024-05-20T16:15:53.100 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:53.100+0000 7f8bdc1e4740 1 -- 172.21.15.82:0/3713248910 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56369d8fd620 con 0x56369d8191b0 2024-05-20T16:15:53.102 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:53.102+0000 7f8bb2ff5640 1 -- 172.21.15.82:0/3713248910 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8bc8036040 con 0x56369d8191b0 2024-05-20T16:15:53.102 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:53.102+0000 7f8bb2ff5640 1 -- 172.21.15.82:0/3713248910 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f8bc4019430 con 0x56369d8191b0 2024-05-20T16:15:53.102 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:53.102+0000 7f8bdc1e4740 10 client.25150 fetch_fsmap finished waiting for FSMap version 25 2024-05-20T16:15:53.102 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:53.103+0000 7f8bdc1e4740 10 client.25150 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T16:15:53.103 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:53.103+0000 7f8bdc1e4740 20 client.25150 populate_metadata read hostname 'smithi082' 2024-05-20T16:15:53.103 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:53.103+0000 7f8bdc1e4740 1 --2- 172.21.15.82:0/3713248910 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x56369d7d7d80 0x56369dbd0b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:15:53.103 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:53.103+0000 7f8bdc1e4740 4 client.25150 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T16:15:53.103 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:53.103+0000 7f8bdc1e4740 1 -- 172.21.15.82:0/3713248910 --> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56369db2aa70 con 0x56369d7d7d80 2024-05-20T16:15:53.106 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:53.106+0000 7f8bceffd640 1 --2- 172.21.15.82:0/3713248910 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x56369d7d7d80 0x56369dbd0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:15:53.113 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:53.112+0000 7f8bceffd640 1 --2- 172.21.15.82:0/3713248910 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x56369d7d7d80 0x56369dbd0b40 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7f0c7a979c590281 server_cookie=da49b23bcf85a9be in_seq=0 out_seq=0 2024-05-20T16:15:53.113 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:53.112+0000 7f8bb2ff5640 10 client.25150 ms_handle_connect on v2:172.21.15.160:6838/2765554482 2024-05-20T16:15:53.115 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:53.115+0000 7f8bb2ff5640 1 -- 172.21.15.82:0/3713248910 <== mds.0 v2:172.21.15.160:6838/2765554482 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56369db2aa70 con 0x56369d7d7d80 2024-05-20T16:15:53.115 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:53.115+0000 7f8bb2ff5640 10 client.25150 handle_command_reply: tid=0 2024-05-20T16:15:53.115 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:53.115+0000 7f8bdc1e4740 1 client.25150 shutdown 2024-05-20T16:15:53.115 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:53.115+0000 7f8bdc1e4740 1 -- 172.21.15.82:0/3713248910 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56369d8191b0 msgr2=0x56369dbca120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:15:53.115 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:53.115+0000 7f8bdc1e4740 1 --2- 172.21.15.82:0/3713248910 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56369d8191b0 0x56369dbca120 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x56369dbb5620 tx=0x7f8bc4011b50 comp rx=0 tx=0).stop 2024-05-20T16:15:53.120 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:53.120+0000 7f8bdc1e4740 1 -- 172.21.15.82:0/3713248910 shutdown_connections 2024-05-20T16:15:53.120 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:53.120+0000 7f8bdc1e4740 1 --2- 172.21.15.82:0/3713248910 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x56369d7d7d80 0x56369dbd0b40 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:15:53.120 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:53.120+0000 7f8bdc1e4740 1 --2- 172.21.15.82:0/3713248910 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56369d7f84d0 0x56369dbc31c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:15:53.120 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:53.120+0000 7f8bdc1e4740 1 --2- 172.21.15.82:0/3713248910 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56369d8191b0 0x56369dbca120 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:15:53.120 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:53.120+0000 7f8bdc1e4740 1 --2- 172.21.15.82:0/3713248910 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56369d7f02f0 0x56369dbc2c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:15:53.120 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:53.120+0000 7f8bdc1e4740 1 -- 172.21.15.82:0/3713248910 >> 172.21.15.82:0/3713248910 conn(0x56369d80d300 msgr2=0x56369dbb77a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:15:53.120 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:53.120+0000 7f8bb2ff5640 0 client.25150 ms_handle_reset on v2:172.21.15.160:6838/2765554482 2024-05-20T16:15:53.120 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:53.120+0000 7f8bdc1e4740 1 -- 172.21.15.82:0/3713248910 shutdown_connections 2024-05-20T16:15:53.120 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:53.120+0000 7f8bdc1e4740 1 -- 172.21.15.82:0/3713248910 wait complete. 2024-05-20T16:15:53.123 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:53.122+0000 7f8bdc1e4740 20 client.25150 trim_cache size 0 max 16384 2024-05-20T16:15:53.157 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:53.158+0000 7f8bdc1e4740 1 Processor -- start 2024-05-20T16:15:53.157 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:53.158+0000 7f8bdc1e4740 1 -- start start 2024-05-20T16:15:53.159 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:53.158+0000 7f8bdc1e4740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56369d7f02f0 0x56369dbc18f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:15:53.159 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:53.159+0000 7f8bdc1e4740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56369d7f84d0 0x56369dbc3390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:15:53.159 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:53.159+0000 7f8bdc1e4740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56369d8191b0 0x56369dbd6340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:15:53.159 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:53.159+0000 7f8bdc1e4740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f8bbc0125b0 con 0x56369d7f84d0 2024-05-20T16:15:53.159 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:53.159+0000 7f8bdc1e4740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f8bbc00fe80 con 0x56369d8191b0 2024-05-20T16:15:53.159 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:53.159+0000 7f8bdc1e4740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f8bbc00dcd0 con 0x56369d7f02f0 2024-05-20T16:15:53.159 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:53.159+0000 7f8bceffd640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56369d7f02f0 0x56369dbc18f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:15:53.159 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:53.159+0000 7f8bceffd640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56369d7f02f0 0x56369dbc18f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.82:48022/0 (socket says 172.21.15.82:48022) 2024-05-20T16:15:53.159 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:53.159+0000 7f8bceffd640 1 -- 172.21.15.82:0/2117554949 learned_addr learned my addr 172.21.15.82:0/2117554949 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:15:53.159 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:53.159+0000 7f8bcf7fe640 1 --2- 172.21.15.82:0/2117554949 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56369d8191b0 0x56369dbd6340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:15:53.160 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:53.160+0000 7f8bce7fc640 1 --2- 172.21.15.82:0/2117554949 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56369d7f84d0 0x56369dbc3390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:15:53.161 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:53.161+0000 7f8bcf7fe640 1 -- 172.21.15.82:0/2117554949 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56369d7f02f0 msgr2=0x56369dbc18f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:15:53.161 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:53.161+0000 7f8bcf7fe640 1 --2- 172.21.15.82:0/2117554949 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56369d7f02f0 0x56369dbc18f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:15:53.161 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:53.161+0000 7f8bcf7fe640 1 -- 172.21.15.82:0/2117554949 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56369d7f84d0 msgr2=0x56369dbc3390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:15:53.161 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:53.161+0000 7f8bcf7fe640 1 --2- 172.21.15.82:0/2117554949 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56369d7f84d0 0x56369dbc3390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:15:53.161 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:53.161+0000 7f8bcf7fe640 1 -- 172.21.15.82:0/2117554949 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56369dbc0480 con 0x56369d8191b0 2024-05-20T16:15:53.161 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:53.162+0000 7f8bcf7fe640 1 --2- 172.21.15.82:0/2117554949 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56369d8191b0 0x56369dbd6340 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f8bc4019800 tx=0x7f8bc401afe0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b6f1bbd114ff0642 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:15:53.163 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:53.163+0000 7f8bb4ff9640 1 -- 172.21.15.82:0/2117554949 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bc4000e60 con 0x56369d8191b0 2024-05-20T16:15:53.163 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:53.164+0000 7f8bb4ff9640 1 -- 172.21.15.82:0/2117554949 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8bc400fb70 con 0x56369d8191b0 2024-05-20T16:15:53.163 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:53.164+0000 7f8bb4ff9640 1 -- 172.21.15.82:0/2117554949 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bc40287c0 con 0x56369d8191b0 2024-05-20T16:15:53.166 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:53.165+0000 7f8bdc1e4740 1 -- 172.21.15.82:0/2117554949 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56369d8191b0 msgr2=0x56369dbd6340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:15:53.166 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:53.165+0000 7f8bdc1e4740 1 --2- 172.21.15.82:0/2117554949 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56369d8191b0 0x56369dbd6340 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f8bc4019800 tx=0x7f8bc401afe0 comp rx=0 tx=0).stop 2024-05-20T16:15:53.166 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:53.166+0000 7f8bdc1e4740 1 -- 172.21.15.82:0/2117554949 shutdown_connections 2024-05-20T16:15:53.166 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:53.166+0000 7f8bdc1e4740 1 --2- 172.21.15.82:0/2117554949 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56369d7f84d0 0x56369dbc3390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:15:53.166 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:53.166+0000 7f8bdc1e4740 1 --2- 172.21.15.82:0/2117554949 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56369d8191b0 0x56369dbd6340 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:15:53.166 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:53.166+0000 7f8bdc1e4740 1 --2- 172.21.15.82:0/2117554949 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56369d7f02f0 0x56369dbc18f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:15:53.166 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:53.166+0000 7f8bdc1e4740 1 -- 172.21.15.82:0/2117554949 >> 172.21.15.82:0/2117554949 conn(0x56369d80d300 msgr2=0x56369dbc2c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:15:53.168 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:53.168+0000 7f8bdc1e4740 1 -- 172.21.15.82:0/2117554949 shutdown_connections 2024-05-20T16:15:53.168 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:53.168+0000 7f8bdc1e4740 1 -- 172.21.15.82:0/2117554949 wait complete. 2024-05-20T16:15:53.168 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:53.169+0000 7f8bdc1e4740 1 -- start start 2024-05-20T16:15:53.169 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:53.169+0000 7f8bdc1e4740 1 Processor -- start 2024-05-20T16:15:53.169 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:53.170+0000 7f8bdc1e4740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56369d7f02f0 0x56369dbca6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:15:53.169 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:53.170+0000 7f8bdc1e4740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56369d7f84d0 0x56369dbc8dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:15:53.170 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:53.170+0000 7f8bdc1e4740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56369d8191b0 0x56369dbc9310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:15:53.170 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:53.170+0000 7f8bdc1e4740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f8bbc00e340 con 0x56369d8191b0 2024-05-20T16:15:53.170 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:53.170+0000 7f8bdc1e4740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f8bbc00f7b0 con 0x56369d7f02f0 2024-05-20T16:15:53.170 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:53.170+0000 7f8bdc1e4740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f8bbc00ea10 con 0x56369d7f84d0 2024-05-20T16:15:53.170 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:53.170+0000 7f8bdc1e4740 10 client.0 _ll_register_callbacks cb 0x7f8bc4009d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T16:15:53.171 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:53.170+0000 7f8bce7fc640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56369d7f84d0 0x56369dbc8dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:15:53.171 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:53.170+0000 7f8bce7fc640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56369d7f84d0 0x56369dbc8dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.82:48024/0 (socket says 172.21.15.82:48024) 2024-05-20T16:15:53.171 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:53.170+0000 7f8bce7fc640 1 -- 172.21.15.82:0/462986528 learned_addr learned my addr 172.21.15.82:0/462986528 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:15:53.171 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:53.171+0000 7f8bce7fc640 1 -- 172.21.15.82:0/462986528 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56369d7f02f0 msgr2=0x56369dbca6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:15:53.171 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:53.171+0000 7f8bce7fc640 1 --2- 172.21.15.82:0/462986528 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56369d7f02f0 0x56369dbca6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:15:53.172 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:53.171+0000 7f8bce7fc640 1 -- 172.21.15.82:0/462986528 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56369d8191b0 msgr2=0x56369dbc9310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:15:53.172 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:53.171+0000 7f8bce7fc640 1 --2- 172.21.15.82:0/462986528 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56369d8191b0 0x56369dbc9310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:15:53.172 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:53.171+0000 7f8bce7fc640 1 -- 172.21.15.82:0/462986528 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56369dbc9900 con 0x56369d7f84d0 2024-05-20T16:15:53.172 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:53.171+0000 7f8bce7fc640 1 --2- 172.21.15.82:0/462986528 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56369d7f84d0 0x56369dbc8dd0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f8bc8030cb0 tx=0x7f8bc8005fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1f996454a0aee1a3 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:15:53.172 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:53.171+0000 7f8bb2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-05-20T16:15:53.174 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:53.173+0000 7f8bb2ff5640 1 -- 172.21.15.82:0/462986528 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bc803e840 con 0x56369d7f84d0 2024-05-20T16:15:53.174 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:53.173+0000 7f8bb2ff5640 1 -- 172.21.15.82:0/462986528 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9150+0+0 (secure 0 0 0) 0x7f8bc803e9e0 con 0x56369d7f84d0 2024-05-20T16:15:53.178 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:53.179+0000 7f8bdc1e4740 1 -- 172.21.15.82:0/462986528 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8bc400fb70 con 0x56369d7f84d0 2024-05-20T16:15:53.181 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:53.181+0000 7f8bb2ff5640 1 -- 172.21.15.82:0/462986528 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f8bc803ebe0 con 0x56369d7f84d0 2024-05-20T16:15:53.183 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:53.183+0000 7f8bdc1e4740 10 client.25156 fetch_fsmap learned FSMap version 25 2024-05-20T16:15:53.183 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:53.184+0000 7f8bdc1e4740 1 -- 172.21.15.82:0/462986528 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56369dc37cc0 con 0x56369d7f84d0 2024-05-20T16:15:53.184 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:53.184+0000 7f8bb2ff5640 1 -- 172.21.15.82:0/462986528 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8bc400fb70 con 0x56369d7f84d0 2024-05-20T16:15:53.184 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:53.184+0000 7f8bb2ff5640 1 -- 172.21.15.82:0/462986528 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f8bc8063020 con 0x56369d7f84d0 2024-05-20T16:15:53.184 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:53.184+0000 7f8bdc1e4740 10 client.25156 fetch_fsmap finished waiting for FSMap version 25 2024-05-20T16:15:53.184 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:53.184+0000 7f8bdc1e4740 10 client.25156 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T16:15:53.185 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:53.184+0000 7f8bdc1e4740 20 client.25156 populate_metadata read hostname 'smithi082' 2024-05-20T16:15:53.185 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:53.185+0000 7f8bdc1e4740 1 --2- 172.21.15.82:0/462986528 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x56369d7ad1c0 0x56369dc3a310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:15:53.185 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:53.185+0000 7f8bdc1e4740 4 client.25156 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-20T16:15:53.185 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:53.185+0000 7f8bdc1e4740 1 -- 172.21.15.82:0/462986528 --> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56369db24dc0 con 0x56369d7ad1c0 2024-05-20T16:15:53.186 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:53.186+0000 7f8bceffd640 1 --2- 172.21.15.82:0/462986528 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x56369d7ad1c0 0x56369dc3a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:15:53.189 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:53.189+0000 7f8bceffd640 1 --2- 172.21.15.82:0/462986528 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x56369d7ad1c0 0x56369dc3a310 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e8254c62b0e23c3b server_cookie=bda346e75e7dc521 in_seq=0 out_seq=0 2024-05-20T16:15:53.189 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:53.189+0000 7f8bb2ff5640 10 client.25156 ms_handle_connect on v2:172.21.15.160:6838/2765554482 2024-05-20T16:15:53.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:15:53 smithi160 ceph-mon[36347]: pgmap v419: 129 pgs: 129 active+clean; 10 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 942 MiB/s wr, 1.58k op/s 2024-05-20T16:15:53.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:15:53 smithi082 ceph-mon[32238]: pgmap v419: 129 pgs: 129 active+clean; 10 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 942 MiB/s wr, 1.58k op/s 2024-05-20T16:15:53.885 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:15:53 smithi184 ceph-mon[36209]: pgmap v419: 129 pgs: 129 active+clean; 10 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 942 MiB/s wr, 1.58k op/s 2024-05-20T16:15:55.490 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:55.490+0000 7f8bb2ff5640 1 -- 172.21.15.82:0/462986528 <== mds.0 v2:172.21.15.160:6838/2765554482 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56369db24dc0 con 0x56369d7ad1c0 2024-05-20T16:15:55.490 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:55.490+0000 7f8bb2ff5640 10 client.25156 handle_command_reply: tid=0 2024-05-20T16:15:55.490 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:55.490+0000 7f8bdc1e4740 1 client.25156 shutdown 2024-05-20T16:15:55.491 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:55.491+0000 7f8bdc1e4740 1 -- 172.21.15.82:0/462986528 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56369d7f84d0 msgr2=0x56369dbc8dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:15:55.491 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:55.491+0000 7f8bdc1e4740 1 --2- 172.21.15.82:0/462986528 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56369d7f84d0 0x56369dbc8dd0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f8bc8030cb0 tx=0x7f8bc8005fd0 comp rx=0 tx=0).stop 2024-05-20T16:15:55.492 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:55.491+0000 7f8bdc1e4740 1 -- 172.21.15.82:0/462986528 shutdown_connections 2024-05-20T16:15:55.492 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:55.491+0000 7f8bdc1e4740 1 --2- 172.21.15.82:0/462986528 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x56369d7ad1c0 0x56369dc3a310 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:15:55.492 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:55.491+0000 7f8bdc1e4740 1 --2- 172.21.15.82:0/462986528 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56369d8191b0 0x56369dbc9310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:15:55.492 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:55.491+0000 7f8bdc1e4740 1 --2- 172.21.15.82:0/462986528 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56369d7f84d0 0x56369dbc8dd0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:15:55.492 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:55.491+0000 7f8bdc1e4740 1 --2- 172.21.15.82:0/462986528 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56369d7f02f0 0x56369dbca6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:15:55.492 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:55.491+0000 7f8bdc1e4740 1 -- 172.21.15.82:0/462986528 >> 172.21.15.82:0/462986528 conn(0x56369d80d300 msgr2=0x56369dbc2940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:15:55.492 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:55.492+0000 7f8bb2ff5640 0 client.25156 ms_handle_reset on v2:172.21.15.160:6838/2765554482 2024-05-20T16:15:55.492 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:55.492+0000 7f8bdc1e4740 1 -- 172.21.15.82:0/462986528 shutdown_connections 2024-05-20T16:15:55.492 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:55.492+0000 7f8bdc1e4740 1 -- 172.21.15.82:0/462986528 wait complete. 2024-05-20T16:15:55.492 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:55.492+0000 7f8bdc1e4740 20 client.25156 trim_cache size 0 max 16384 2024-05-20T16:15:55.493 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:{ 2024-05-20T16:15:55.493 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "message": "", 2024-05-20T16:15:55.493 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "return_code": 0 2024-05-20T16:15:55.493 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:} 2024-05-20T16:15:55.494 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:55.494+0000 7f8bd5338640 1 -- 172.21.15.82:0/2685029215 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f8bb80a2d90 msgr2=0x7f8bb80a52a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:15:55.494 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:55.494+0000 7f8bd5338640 1 --2- 172.21.15.82:0/2685029215 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f8bb80a2d90 0x7f8bb80a52a0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f8bd0072860 tx=0x7f8bc8005df0 comp rx=0 tx=0).stop 2024-05-20T16:15:55.494 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:55.494+0000 7f8bd5338640 1 -- 172.21.15.82:0/2685029215 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8bd0109100 msgr2=0x7f8bd00717c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:15:55.494 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:55.494+0000 7f8bd5338640 1 --2- 172.21.15.82:0/2685029215 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8bd0109100 0x7f8bd00717c0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f8bc000d740 tx=0x7f8bc000dc00 comp rx=0 tx=0).stop 2024-05-20T16:15:55.496 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:55.495+0000 7f8bd5338640 1 -- 172.21.15.82:0/2685029215 shutdown_connections 2024-05-20T16:15:55.496 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:55.495+0000 7f8bd5338640 1 --2- 172.21.15.82:0/2685029215 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f8bb80a2d90 0x7f8bb80a52a0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:15:55.496 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:55.495+0000 7f8bd5338640 1 --2- 172.21.15.82:0/2685029215 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8bd010f040 0x7f8bd0072c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:15:55.496 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:55.496+0000 7f8bd5338640 1 --2- 172.21.15.82:0/2685029215 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8bd010a340 0x7f8bd0071d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:15:55.496 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:55.496+0000 7f8bd5338640 1 --2- 172.21.15.82:0/2685029215 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8bd0109100 0x7f8bd00717c0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:15:55.496 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:55.496+0000 7f8bd5338640 1 -- 172.21.15.82:0/2685029215 >> 172.21.15.82:0/2685029215 conn(0x7f8bd0077d90 msgr2=0x7f8bd010b020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:15:55.496 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:55.496+0000 7f8bd5338640 1 -- 172.21.15.82:0/2685029215 shutdown_connections 2024-05-20T16:15:55.496 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:15:55.496+0000 7f8bd5338640 1 -- 172.21.15.82:0/2685029215 wait complete. 2024-05-20T16:15:55.687 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:15:55 smithi184 ceph-mon[36209]: pgmap v420: 129 pgs: 129 active+clean; 10 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 994 MiB/s wr, 1.89k op/s 2024-05-20T16:15:55.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:15:55 smithi160 ceph-mon[36347]: pgmap v420: 129 pgs: 129 active+clean; 10 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 994 MiB/s wr, 1.89k op/s 2024-05-20T16:15:55.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:15:55 smithi082 ceph-mon[32238]: pgmap v420: 129 pgs: 129 active+clean; 10 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 994 MiB/s wr, 1.89k op/s 2024-05-20T16:15:57.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:15:57 smithi160 ceph-mon[36347]: pgmap v421: 129 pgs: 129 active+clean; 10 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 794 MiB/s wr, 1.61k op/s 2024-05-20T16:15:57.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:15:57 smithi082 ceph-mon[32238]: pgmap v421: 129 pgs: 129 active+clean; 10 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 794 MiB/s wr, 1.61k op/s 2024-05-20T16:15:57.885 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:15:57 smithi184 ceph-mon[36209]: pgmap v421: 129 pgs: 129 active+clean; 10 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 794 MiB/s wr, 1.61k op/s 2024-05-20T16:15:59.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:15:59 smithi160 ceph-mon[36347]: pgmap v422: 129 pgs: 129 active+clean; 10 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 969 MiB/s wr, 2.09k op/s 2024-05-20T16:15:59.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:15:59 smithi082 ceph-mon[32238]: pgmap v422: 129 pgs: 129 active+clean; 10 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 969 MiB/s wr, 2.09k op/s 2024-05-20T16:15:59.885 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:15:59 smithi184 ceph-mon[36209]: pgmap v422: 129 pgs: 129 active+clean; 10 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 969 MiB/s wr, 2.09k op/s 2024-05-20T16:16:01.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:16:01 smithi160 ceph-mon[36347]: pgmap v423: 129 pgs: 129 active+clean; 10 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 807 MiB/s wr, 1.94k op/s 2024-05-20T16:16:01.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:16:01 smithi082 ceph-mon[32238]: pgmap v423: 129 pgs: 129 active+clean; 10 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 807 MiB/s wr, 1.94k op/s 2024-05-20T16:16:01.885 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:16:01 smithi184 ceph-mon[36209]: pgmap v423: 129 pgs: 129 active+clean; 10 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 807 MiB/s wr, 1.94k op/s 2024-05-20T16:16:03.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:16:03 smithi160 ceph-mon[36347]: pgmap v424: 129 pgs: 129 active+clean; 10 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 823 MiB/s wr, 2.05k op/s 2024-05-20T16:16:03.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:16:03 smithi082 ceph-mon[32238]: pgmap v424: 129 pgs: 129 active+clean; 10 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 823 MiB/s wr, 2.05k op/s 2024-05-20T16:16:03.885 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:16:03 smithi184 ceph-mon[36209]: pgmap v424: 129 pgs: 129 active+clean; 10 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 823 MiB/s wr, 2.05k op/s 2024-05-20T16:16:05.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:16:05 smithi160 ceph-mon[36347]: pgmap v425: 129 pgs: 129 active+clean; 10 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 893 MiB/s wr, 2.21k op/s 2024-05-20T16:16:05.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:16:05 smithi082 ceph-mon[32238]: pgmap v425: 129 pgs: 129 active+clean; 10 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 893 MiB/s wr, 2.21k op/s 2024-05-20T16:16:05.885 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:16:05 smithi184 ceph-mon[36209]: pgmap v425: 129 pgs: 129 active+clean; 10 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 893 MiB/s wr, 2.21k op/s 2024-05-20T16:16:07.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:16:07 smithi160 ceph-mon[36347]: pgmap v426: 129 pgs: 129 active+clean; 10 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 752 MiB/s wr, 1.84k op/s 2024-05-20T16:16:07.860 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:16:07 smithi082 ceph-mon[32238]: pgmap v426: 129 pgs: 129 active+clean; 10 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 752 MiB/s wr, 1.84k op/s 2024-05-20T16:16:07.885 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:16:07 smithi184 ceph-mon[36209]: pgmap v426: 129 pgs: 129 active+clean; 10 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 752 MiB/s wr, 1.84k op/s 2024-05-20T16:16:08.710 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:08.709+0000 7fa051f57640 1 -- 172.21.15.82:0/2643549845 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa04c1093d0 msgr2=0x7fa04c109dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:16:08.710 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:08.709+0000 7fa051f57640 1 --2- 172.21.15.82:0/2643549845 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa04c1093d0 0x7fa04c109dc0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fa03c01a910 tx=0x7fa03c03b430 comp rx=0 tx=0).stop 2024-05-20T16:16:08.711 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:08.711+0000 7fa051f57640 1 -- 172.21.15.82:0/2643549845 shutdown_connections 2024-05-20T16:16:08.711 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:08.711+0000 7fa051f57640 1 --2- 172.21.15.82:0/2643549845 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa04c10f020 0x7fa04c10f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:16:08.711 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:08.711+0000 7fa051f57640 1 --2- 172.21.15.82:0/2643549845 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa04c10a300 0x7fa04c10a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:16:08.711 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:08.711+0000 7fa051f57640 1 --2- 172.21.15.82:0/2643549845 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa04c1093d0 0x7fa04c109dc0 secure :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fa03c01a910 tx=0x7fa03c03b430 comp rx=0 tx=0).stop 2024-05-20T16:16:08.711 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:08.711+0000 7fa051f57640 1 -- 172.21.15.82:0/2643549845 >> 172.21.15.82:0/2643549845 conn(0x7fa04c06f4f0 msgr2=0x7fa04c071980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:16:08.721 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:08.721+0000 7fa051f57640 1 -- 172.21.15.82:0/2643549845 shutdown_connections 2024-05-20T16:16:08.724 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:08.721+0000 7fa051f57640 1 -- 172.21.15.82:0/2643549845 wait complete. 2024-05-20T16:16:08.724 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:08.723+0000 7fa051f57640 1 Processor -- start 2024-05-20T16:16:08.727 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:08.725+0000 7fa051f57640 1 -- start start 2024-05-20T16:16:08.727 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:08.725+0000 7fa051f57640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa04c10a300 0x7fa04c19fab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:16:08.727 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:08.725+0000 7fa051f57640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa04c10f020 0x7fa04c1a0070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:16:08.727 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:08.725+0000 7fa051f57640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa04c1a4f70 0x7fa04c1a7390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:16:08.727 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:08.725+0000 7fa051f57640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fa04c112e40 con 0x7fa04c1a4f70 2024-05-20T16:16:08.727 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:08.725+0000 7fa051f57640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fa04c112cc0 con 0x7fa04c10f020 2024-05-20T16:16:08.727 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:08.725+0000 7fa051f57640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa04c112fc0 con 0x7fa04c10a300 2024-05-20T16:16:08.727 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:08.726+0000 7fa04affd640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa04c10f020 0x7fa04c1a0070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:16:08.727 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:08.726+0000 7fa04affd640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa04c10f020 0x7fa04c1a0070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.82:45842/0 (socket says 172.21.15.82:45842) 2024-05-20T16:16:08.727 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:08.726+0000 7fa04affd640 1 -- 172.21.15.82:0/76826917 learned_addr learned my addr 172.21.15.82:0/76826917 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:16:08.727 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:08.726+0000 7fa04affd640 1 -- 172.21.15.82:0/76826917 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa04c10a300 msgr2=0x7fa04c19fab0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-20T16:16:08.727 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:08.726+0000 7fa04affd640 1 --2- 172.21.15.82:0/76826917 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa04c10a300 0x7fa04c19fab0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:16:08.728 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:08.726+0000 7fa04affd640 1 -- 172.21.15.82:0/76826917 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa04c1a4f70 msgr2=0x7fa04c1a7390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:16:08.728 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:08.726+0000 7fa04affd640 1 --2- 172.21.15.82:0/76826917 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa04c1a4f70 0x7fa04c1a7390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:16:08.728 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:08.726+0000 7fa04affd640 1 -- 172.21.15.82:0/76826917 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa04c1a7ac0 con 0x7fa04c10f020 2024-05-20T16:16:08.730 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:08.727+0000 7fa04affd640 1 --2- 172.21.15.82:0/76826917 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa04c10f020 0x7fa04c1a0070 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fa04400ba50 tx=0x7fa04400bf10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:16:08.732 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:08.732+0000 7fa048ff9640 1 -- 172.21.15.82:0/76826917 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa044019070 con 0x7fa04c10f020 2024-05-20T16:16:08.732 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:08.732+0000 7fa048ff9640 1 -- 172.21.15.82:0/76826917 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa04400cbe0 con 0x7fa04c10f020 2024-05-20T16:16:08.732 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:08.732+0000 7fa048ff9640 1 -- 172.21.15.82:0/76826917 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0440076e0 con 0x7fa04c10f020 2024-05-20T16:16:08.732 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:08.732+0000 7fa051f57640 1 -- 172.21.15.82:0/76826917 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa04c1a7db0 con 0x7fa04c10f020 2024-05-20T16:16:08.732 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:08.732+0000 7fa051f57640 1 -- 172.21.15.82:0/76826917 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa04c1a82a0 con 0x7fa04c10f020 2024-05-20T16:16:08.759 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:08.747+0000 7fa048ff9640 1 -- 172.21.15.82:0/76826917 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fa04401c070 con 0x7fa04c10f020 2024-05-20T16:16:08.759 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:08.757+0000 7fa058e00740 1 Processor -- start 2024-05-20T16:16:08.759 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:08.758+0000 7fa048ff9640 1 --2- 172.21.15.82:0/76826917 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fa0340a2d90 0x7fa0340a52a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:16:08.759 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:08.759+0000 7fa048ff9640 1 -- 172.21.15.82:0/76826917 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9150+0+0 (secure 0 0 0) 0x7fa0440ce930 con 0x7fa04c10f020 2024-05-20T16:16:08.761 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:08.760+0000 7fa058e00740 1 -- start start 2024-05-20T16:16:08.761 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:08.760+0000 7fa058e00740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55935b5e91b0 0x55935b988e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:16:08.761 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:08.760+0000 7fa058e00740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55935b5c02f0 0x55935b9893e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:16:08.761 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:08.760+0000 7fa058e00740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55935b5c84d0 0x55935b9902e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:16:08.761 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:08.760+0000 7fa058e00740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55935b8ed340 con 0x55935b5c02f0 2024-05-20T16:16:08.761 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:08.760+0000 7fa058e00740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x55935b895290 con 0x55935b5c84d0 2024-05-20T16:16:08.761 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:08.760+0000 7fa058e00740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55935b8e9300 con 0x55935b5e91b0 2024-05-20T16:16:08.762 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:08.761+0000 7fa04bfff640 1 --2- 172.21.15.82:0/76826917 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fa0340a2d90 0x7fa0340a52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:16:08.763 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:08.763+0000 7fa04bfff640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55935b5c84d0 0x55935b9902e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:16:08.763 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:08.763+0000 7fa04bfff640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55935b5c84d0 0x55935b9902e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.82:45852/0 (socket says 172.21.15.82:45852) 2024-05-20T16:16:08.763 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:08.763+0000 7fa04bfff640 1 -- 172.21.15.82:0/3884988545 learned_addr learned my addr 172.21.15.82:0/3884988545 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:16:08.763 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:08.763+0000 7fa04affd640 1 --2- 172.21.15.82:0/3884988545 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55935b5c02f0 0x55935b9893e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:16:08.763 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:08.764+0000 7fa04b7fe640 1 --2- 172.21.15.82:0/3884988545 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55935b5e91b0 0x55935b988e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:16:08.765 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:08.765+0000 7fa04bfff640 1 --2- 172.21.15.82:0/76826917 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fa0340a2d90 0x7fa0340a52a0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fa040009f50 tx=0x7fa0400127c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:16:08.765 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:08.765+0000 7fa04bfff640 1 -- 172.21.15.82:0/3884988545 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55935b5e91b0 msgr2=0x55935b988e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:16:08.765 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:08.765+0000 7fa04bfff640 1 --2- 172.21.15.82:0/3884988545 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55935b5e91b0 0x55935b988e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:16:08.765 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:08.765+0000 7fa04bfff640 1 -- 172.21.15.82:0/3884988545 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55935b5c02f0 msgr2=0x55935b9893e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:16:08.766 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:08.765+0000 7fa04bfff640 1 --2- 172.21.15.82:0/3884988545 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55935b5c02f0 0x55935b9893e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:16:08.766 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:08.765+0000 7fa04bfff640 1 -- 172.21.15.82:0/3884988545 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55935b8b98e0 con 0x55935b5c84d0 2024-05-20T16:16:08.767 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:08.767+0000 7fa04bfff640 1 --2- 172.21.15.82:0/3884988545 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55935b5c84d0 0x55935b9902e0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fa040010670 tx=0x7fa04001d000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c2f7aa8417dc2ea2 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:16:08.768 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:08.768+0000 7fa031ffb640 1 -- 172.21.15.82:0/3884988545 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa040004220 con 0x55935b5c84d0 2024-05-20T16:16:08.769 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:08.769+0000 7fa031ffb640 1 -- 172.21.15.82:0/3884988545 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa0400043c0 con 0x55935b5c84d0 2024-05-20T16:16:08.770 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:08.771+0000 7fa058e00740 1 -- 172.21.15.82:0/3884988545 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55935b5c84d0 msgr2=0x55935b9902e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:16:08.771 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:08.771+0000 7fa058e00740 1 --2- 172.21.15.82:0/3884988545 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55935b5c84d0 0x55935b9902e0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fa040010670 tx=0x7fa04001d000 comp rx=0 tx=0).stop 2024-05-20T16:16:08.777 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:08.777+0000 7fa058e00740 1 -- 172.21.15.82:0/3884988545 shutdown_connections 2024-05-20T16:16:08.777 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:08.777+0000 7fa058e00740 1 --2- 172.21.15.82:0/3884988545 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55935b5c02f0 0x55935b9893e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:16:08.777 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:08.777+0000 7fa058e00740 1 --2- 172.21.15.82:0/3884988545 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55935b5c84d0 0x55935b9902e0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:16:08.777 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:08.777+0000 7fa058e00740 1 --2- 172.21.15.82:0/3884988545 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55935b5e91b0 0x55935b988e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:16:08.777 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:08.777+0000 7fa058e00740 1 -- 172.21.15.82:0/3884988545 >> 172.21.15.82:0/3884988545 conn(0x55935b5dd300 msgr2=0x55935b9847f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:16:08.783 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:08.783+0000 7fa058e00740 1 -- 172.21.15.82:0/3884988545 shutdown_connections 2024-05-20T16:16:08.785 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:08.784+0000 7fa058e00740 1 -- 172.21.15.82:0/3884988545 wait complete. 2024-05-20T16:16:08.785 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:08.784+0000 7fa058e00740 1 -- start start 2024-05-20T16:16:08.785 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:08.785+0000 7fa058e00740 1 Processor -- start 2024-05-20T16:16:08.786 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:08.786+0000 7fa058e00740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55935b5c02f0 0x55935b992aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:16:08.786 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:08.786+0000 7fa058e00740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55935b5c84d0 0x55935b992fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:16:08.787 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:08.786+0000 7fa058e00740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55935b5e91b0 0x55935b999f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:16:08.787 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:08.786+0000 7fa058e00740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55935b990d70 con 0x55935b5e91b0 2024-05-20T16:16:08.787 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:08.786+0000 7fa058e00740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x55935b8a8790 con 0x55935b5c84d0 2024-05-20T16:16:08.787 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:08.786+0000 7fa058e00740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55935b990ef0 con 0x55935b5c02f0 2024-05-20T16:16:08.787 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:08.786+0000 7fa058e00740 10 client.0 _ll_register_callbacks cb 0x55935b8e3a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T16:16:08.787 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:08.788+0000 7fa04bfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55935b5e91b0 0x55935b999f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:16:08.788 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:08.788+0000 7fa04bfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55935b5e91b0 0x55935b999f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:49430/0 (socket says 172.21.15.82:49430) 2024-05-20T16:16:08.788 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:08.788+0000 7fa04bfff640 1 -- 172.21.15.82:0/432663148 learned_addr learned my addr 172.21.15.82:0/432663148 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:16:08.788 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:08.789+0000 7fa04affd640 1 --2- 172.21.15.82:0/432663148 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55935b5c84d0 0x55935b992fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:16:08.790 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:08.789+0000 7fa04bfff640 1 -- 172.21.15.82:0/432663148 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55935b5c02f0 msgr2=0x55935b992aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:16:08.790 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:08.789+0000 7fa04bfff640 1 --2- 172.21.15.82:0/432663148 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55935b5c02f0 0x55935b992aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:16:08.790 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:08.789+0000 7fa04bfff640 1 -- 172.21.15.82:0/432663148 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55935b5c84d0 msgr2=0x55935b992fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:16:08.790 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:08.789+0000 7fa04bfff640 1 --2- 172.21.15.82:0/432663148 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55935b5c84d0 0x55935b992fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:16:08.790 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:08.789+0000 7fa04bfff640 1 -- 172.21.15.82:0/432663148 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55935b601210 con 0x55935b5e91b0 2024-05-20T16:16:08.797 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:08.796+0000 7fa04bfff640 1 --2- 172.21.15.82:0/432663148 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55935b5e91b0 0x55935b999f40 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fa040009460 tx=0x7fa040004570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e828839a571198c9 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:16:08.797 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:08.798+0000 7fa02affd640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-05-20T16:16:08.802 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:08.802+0000 7fa02affd640 1 -- 172.21.15.82:0/432663148 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa040005200 con 0x55935b5e91b0 2024-05-20T16:16:08.802 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:08.802+0000 7fa02affd640 1 -- 172.21.15.82:0/432663148 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9150+0+0 (secure 0 0 0) 0x7fa0400053a0 con 0x55935b5e91b0 2024-05-20T16:16:08.803 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:08.803+0000 7fa058e00740 1 -- 172.21.15.82:0/432663148 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa0400043c0 con 0x55935b5e91b0 2024-05-20T16:16:08.810 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:08.810+0000 7fa02affd640 1 -- 172.21.15.82:0/432663148 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa0400055a0 con 0x55935b5e91b0 2024-05-20T16:16:08.812 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:08.813+0000 7fa058e00740 10 client.15609 fetch_fsmap learned FSMap version 25 2024-05-20T16:16:08.812 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:08.813+0000 7fa058e00740 1 -- 172.21.15.82:0/432663148 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55935b6cd620 con 0x55935b5e91b0 2024-05-20T16:16:08.814 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:08.814+0000 7fa02affd640 1 -- 172.21.15.82:0/432663148 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa0400043c0 con 0x55935b5e91b0 2024-05-20T16:16:08.815 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:08.815+0000 7fa02affd640 1 -- 172.21.15.82:0/432663148 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fa040029660 con 0x55935b5e91b0 2024-05-20T16:16:08.816 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:08.816+0000 7fa058e00740 10 client.15609 fetch_fsmap finished waiting for FSMap version 25 2024-05-20T16:16:08.816 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:08.816+0000 7fa058e00740 10 client.15609 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T16:16:08.816 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:08.816+0000 7fa058e00740 20 client.15609 populate_metadata read hostname 'smithi082' 2024-05-20T16:16:08.816 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:08.816+0000 7fa058e00740 1 --2- 172.21.15.82:0/432663148 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x55935b5a7d80 0x55935b9a0340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:16:08.816 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:08.816+0000 7fa058e00740 4 client.15609 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T16:16:08.816 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:08.816+0000 7fa058e00740 1 -- 172.21.15.82:0/432663148 --> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55935b8faa70 con 0x55935b5a7d80 2024-05-20T16:16:08.818 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:08.818+0000 7fa04b7fe640 1 --2- 172.21.15.82:0/432663148 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x55935b5a7d80 0x55935b9a0340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:16:08.827 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:08.827+0000 7fa04b7fe640 1 --2- 172.21.15.82:0/432663148 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x55935b5a7d80 0x55935b9a0340 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=64f84fb44091d5d8 server_cookie=84b06c51226f1610 in_seq=0 out_seq=0 2024-05-20T16:16:08.827 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:08.827+0000 7fa02affd640 10 client.15609 ms_handle_connect on v2:172.21.15.160:6838/2765554482 2024-05-20T16:16:08.832 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:08.832+0000 7fa02affd640 1 -- 172.21.15.82:0/432663148 <== mds.0 v2:172.21.15.160:6838/2765554482 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55935b8faa70 con 0x55935b5a7d80 2024-05-20T16:16:08.832 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:08.832+0000 7fa02affd640 10 client.15609 handle_command_reply: tid=0 2024-05-20T16:16:08.832 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:08.832+0000 7fa058e00740 1 client.15609 shutdown 2024-05-20T16:16:08.834 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:08.834+0000 7fa058e00740 1 -- 172.21.15.82:0/432663148 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55935b5e91b0 msgr2=0x55935b999f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:16:08.835 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:08.834+0000 7fa058e00740 1 --2- 172.21.15.82:0/432663148 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55935b5e91b0 0x55935b999f40 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fa040009460 tx=0x7fa040004570 comp rx=0 tx=0).stop 2024-05-20T16:16:08.835 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:08.834+0000 7fa058e00740 1 -- 172.21.15.82:0/432663148 shutdown_connections 2024-05-20T16:16:08.835 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:08.834+0000 7fa058e00740 1 --2- 172.21.15.82:0/432663148 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x55935b5a7d80 0x55935b9a0340 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:16:08.835 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:08.834+0000 7fa058e00740 1 --2- 172.21.15.82:0/432663148 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55935b5e91b0 0x55935b999f40 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:16:08.835 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:08.834+0000 7fa058e00740 1 --2- 172.21.15.82:0/432663148 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55935b5c84d0 0x55935b992fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:16:08.835 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:08.834+0000 7fa058e00740 1 --2- 172.21.15.82:0/432663148 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55935b5c02f0 0x55935b992aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:16:08.835 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:08.834+0000 7fa058e00740 1 -- 172.21.15.82:0/432663148 >> 172.21.15.82:0/432663148 conn(0x55935b5dd300 msgr2=0x55935b983810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:16:08.835 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:08.835+0000 7fa02affd640 0 client.15609 ms_handle_reset on v2:172.21.15.160:6838/2765554482 2024-05-20T16:16:08.835 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:08.835+0000 7fa058e00740 1 -- 172.21.15.82:0/432663148 shutdown_connections 2024-05-20T16:16:08.835 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:08.835+0000 7fa058e00740 1 -- 172.21.15.82:0/432663148 wait complete. 2024-05-20T16:16:08.838 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:08.835+0000 7fa058e00740 20 client.15609 trim_cache size 0 max 16384 2024-05-20T16:16:08.890 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:08.890+0000 7fa058e00740 1 Processor -- start 2024-05-20T16:16:08.891 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:08.891+0000 7fa058e00740 1 -- start start 2024-05-20T16:16:08.891 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:08.891+0000 7fa058e00740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55935b5dd300 0x55935b9841d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:16:08.891 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:08.891+0000 7fa058e00740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55935b5c02f0 0x55935b984710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:16:08.891 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:08.891+0000 7fa058e00740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55935b5c84d0 0x55935b999e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:16:08.891 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:08.891+0000 7fa058e00740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fa0380125f0 con 0x55935b5c84d0 2024-05-20T16:16:08.891 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:08.891+0000 7fa058e00740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fa03800fec0 con 0x55935b5c02f0 2024-05-20T16:16:08.891 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:08.891+0000 7fa058e00740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa03800dd10 con 0x55935b5dd300 2024-05-20T16:16:08.892 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:08.893+0000 7fa04affd640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55935b5c02f0 0x55935b984710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:16:08.892 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:08.893+0000 7fa04b7fe640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55935b5dd300 0x55935b9841d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:16:08.893 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:08.893+0000 7fa04b7fe640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55935b5dd300 0x55935b9841d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.82:52334/0 (socket says 172.21.15.82:52334) 2024-05-20T16:16:08.893 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:08.893+0000 7fa04b7fe640 1 -- 172.21.15.82:0/171028570 learned_addr learned my addr 172.21.15.82:0/171028570 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:16:08.893 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:08.893+0000 7fa04bfff640 1 --2- 172.21.15.82:0/171028570 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55935b5c84d0 0x55935b999e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:16:08.893 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:08.894+0000 7fa04affd640 1 -- 172.21.15.82:0/171028570 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55935b5dd300 msgr2=0x55935b9841d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:16:08.893 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:08.894+0000 7fa04affd640 1 --2- 172.21.15.82:0/171028570 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55935b5dd300 0x55935b9841d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:16:08.894 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:08.894+0000 7fa04affd640 1 -- 172.21.15.82:0/171028570 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55935b5c84d0 msgr2=0x55935b999e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:16:08.894 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:08.894+0000 7fa04affd640 1 --2- 172.21.15.82:0/171028570 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55935b5c84d0 0x55935b999e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:16:08.894 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:08.894+0000 7fa04affd640 1 -- 172.21.15.82:0/171028570 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55935b97fa60 con 0x55935b5c02f0 2024-05-20T16:16:08.894 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:08.894+0000 7fa04affd640 1 --2- 172.21.15.82:0/171028570 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55935b5c02f0 0x55935b984710 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fa0440cc040 tx=0x7fa04400e040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=97b145a58fe4397b server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:16:08.895 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:08.896+0000 7fa0317fa640 1 -- 172.21.15.82:0/171028570 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0440990a0 con 0x55935b5c02f0 2024-05-20T16:16:08.895 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:08.896+0000 7fa0317fa640 1 -- 172.21.15.82:0/171028570 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa0440cc150 con 0x55935b5c02f0 2024-05-20T16:16:08.896 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:08.896+0000 7fa0317fa640 1 -- 172.21.15.82:0/171028570 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa04409a450 con 0x55935b5c02f0 2024-05-20T16:16:08.899 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:08.899+0000 7fa058e00740 1 -- 172.21.15.82:0/171028570 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55935b5c02f0 msgr2=0x55935b984710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:16:08.899 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:08.899+0000 7fa058e00740 1 --2- 172.21.15.82:0/171028570 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55935b5c02f0 0x55935b984710 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fa0440cc040 tx=0x7fa04400e040 comp rx=0 tx=0).stop 2024-05-20T16:16:08.902 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:08.903+0000 7fa058e00740 1 -- 172.21.15.82:0/171028570 shutdown_connections 2024-05-20T16:16:08.902 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:08.903+0000 7fa058e00740 1 --2- 172.21.15.82:0/171028570 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55935b5c84d0 0x55935b999e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:16:08.902 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:08.903+0000 7fa058e00740 1 --2- 172.21.15.82:0/171028570 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55935b5c02f0 0x55935b984710 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:16:08.903 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:08.903+0000 7fa058e00740 1 --2- 172.21.15.82:0/171028570 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55935b5dd300 0x55935b9841d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:16:08.903 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:08.903+0000 7fa058e00740 1 -- 172.21.15.82:0/171028570 >> 172.21.15.82:0/171028570 conn(0x55935b9895f0 msgr2=0x55935b991d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:16:08.914 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:08.913+0000 7fa058e00740 1 -- 172.21.15.82:0/171028570 shutdown_connections 2024-05-20T16:16:08.915 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:08.916+0000 7fa058e00740 1 -- 172.21.15.82:0/171028570 wait complete. 2024-05-20T16:16:08.916 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:08.916+0000 7fa058e00740 1 -- start start 2024-05-20T16:16:08.917 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:08.917+0000 7fa058e00740 1 Processor -- start 2024-05-20T16:16:08.918 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:08.918+0000 7fa058e00740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55935b5c02f0 0x55935b997390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:16:08.918 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:08.918+0000 7fa058e00740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55935b5c84d0 0x55935b9978d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:16:08.918 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:08.918+0000 7fa058e00740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55935b5dd300 0x55935ba066a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:16:08.918 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:08.918+0000 7fa058e00740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fa03800e380 con 0x55935b5c02f0 2024-05-20T16:16:08.918 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:08.918+0000 7fa058e00740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fa03800f7f0 con 0x55935b5c84d0 2024-05-20T16:16:08.919 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:08.918+0000 7fa058e00740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa03800ea50 con 0x55935b5dd300 2024-05-20T16:16:08.919 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:08.918+0000 7fa058e00740 10 client.0 _ll_register_callbacks cb 0x7fa04409ab70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T16:16:08.920 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:08.920+0000 7fa04bfff640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55935b5dd300 0x55935ba066a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:16:08.920 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:08.920+0000 7fa04bfff640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55935b5dd300 0x55935ba066a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.82:52344/0 (socket says 172.21.15.82:52344) 2024-05-20T16:16:08.920 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:08.920+0000 7fa04bfff640 1 -- 172.21.15.82:0/393746659 learned_addr learned my addr 172.21.15.82:0/393746659 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:16:08.922 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:08.922+0000 7fa04bfff640 1 -- 172.21.15.82:0/393746659 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55935b5c84d0 msgr2=0x55935b9978d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:16:08.922 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:08.922+0000 7fa04bfff640 1 --2- 172.21.15.82:0/393746659 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55935b5c84d0 0x55935b9978d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:16:08.922 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:08.922+0000 7fa04bfff640 1 -- 172.21.15.82:0/393746659 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55935b5c02f0 msgr2=0x55935b997390 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-20T16:16:08.922 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:08.922+0000 7fa04bfff640 1 --2- 172.21.15.82:0/393746659 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55935b5c02f0 0x55935b997390 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:16:08.922 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:08.922+0000 7fa04bfff640 1 -- 172.21.15.82:0/393746659 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55935b99a890 con 0x55935b5dd300 2024-05-20T16:16:08.933 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:08.933+0000 7fa04bfff640 1 --2- 172.21.15.82:0/393746659 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55935b5dd300 0x55935ba066a0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fa040009460 tx=0x7fa040004810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9a4ef318813a3868 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:16:08.934 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:08.934+0000 7fa02affd640 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-05-20T16:16:08.937 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:08.937+0000 7fa02affd640 1 -- 172.21.15.82:0/393746659 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa040044060 con 0x55935b5dd300 2024-05-20T16:16:08.938 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:08.938+0000 7fa058e00740 1 -- 172.21.15.82:0/393746659 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa0440cc150 con 0x55935b5dd300 2024-05-20T16:16:08.938 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:08.938+0000 7fa02affd640 1 -- 172.21.15.82:0/393746659 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9150+0+0 (secure 0 0 0) 0x7fa0400048f0 con 0x55935b5dd300 2024-05-20T16:16:08.939 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:08.939+0000 7fa02affd640 1 -- 172.21.15.82:0/393746659 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa04004b050 con 0x55935b5dd300 2024-05-20T16:16:08.940 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:08.940+0000 7fa058e00740 10 client.25168 fetch_fsmap learned FSMap version 25 2024-05-20T16:16:08.940 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:08.940+0000 7fa058e00740 1 -- 172.21.15.82:0/393746659 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55935ba07ae0 con 0x55935b5dd300 2024-05-20T16:16:08.943 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:08.943+0000 7fa02affd640 1 -- 172.21.15.82:0/393746659 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa0440cc150 con 0x55935b5dd300 2024-05-20T16:16:08.944 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:08.943+0000 7fa02affd640 1 -- 172.21.15.82:0/393746659 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fa040005150 con 0x55935b5dd300 2024-05-20T16:16:08.944 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:08.944+0000 7fa058e00740 10 client.25168 fetch_fsmap finished waiting for FSMap version 25 2024-05-20T16:16:08.944 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:08.944+0000 7fa058e00740 10 client.25168 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T16:16:08.944 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:08.944+0000 7fa058e00740 20 client.25168 populate_metadata read hostname 'smithi082' 2024-05-20T16:16:08.944 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:08.944+0000 7fa058e00740 1 --2- 172.21.15.82:0/393746659 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x55935b5e91b0 0x55935ba0a450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:16:08.944 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:08.944+0000 7fa058e00740 4 client.25168 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-20T16:16:08.944 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:08.944+0000 7fa058e00740 1 -- 172.21.15.82:0/393746659 --> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55935b8f4dc0 con 0x55935b5e91b0 2024-05-20T16:16:08.948 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:08.947+0000 7fa04b7fe640 1 --2- 172.21.15.82:0/393746659 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x55935b5e91b0 0x55935ba0a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:16:08.950 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:08.950+0000 7fa04b7fe640 1 --2- 172.21.15.82:0/393746659 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x55935b5e91b0 0x55935ba0a450 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=af88341d70103fc server_cookie=6fed248eb446cee in_seq=0 out_seq=0 2024-05-20T16:16:08.950 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:08.950+0000 7fa02affd640 10 client.25168 ms_handle_connect on v2:172.21.15.160:6838/2765554482 2024-05-20T16:16:09.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:16:09 smithi160 ceph-mon[36347]: pgmap v427: 129 pgs: 129 active+clean; 10 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 930 MiB/s wr, 2.19k op/s 2024-05-20T16:16:09.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:16:09 smithi082 ceph-mon[32238]: pgmap v427: 129 pgs: 129 active+clean; 10 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 930 MiB/s wr, 2.19k op/s 2024-05-20T16:16:09.885 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:16:09 smithi184 ceph-mon[36209]: pgmap v427: 129 pgs: 129 active+clean; 10 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 930 MiB/s wr, 2.19k op/s 2024-05-20T16:16:11.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:16:11 smithi160 ceph-mon[36347]: pgmap v428: 129 pgs: 129 active+clean; 10 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 743 MiB/s wr, 1.71k op/s 2024-05-20T16:16:11.860 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:16:11 smithi082 ceph-mon[32238]: pgmap v428: 129 pgs: 129 active+clean; 10 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 743 MiB/s wr, 1.71k op/s 2024-05-20T16:16:11.885 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:16:11 smithi184 ceph-mon[36209]: pgmap v428: 129 pgs: 129 active+clean; 10 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 743 MiB/s wr, 1.71k op/s 2024-05-20T16:16:13.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:16:13 smithi160 ceph-mon[36347]: pgmap v429: 129 pgs: 129 active+clean; 10 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 792 MiB/s wr, 1.65k op/s 2024-05-20T16:16:13.879 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:16:13 smithi082 ceph-mon[32238]: pgmap v429: 129 pgs: 129 active+clean; 10 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 792 MiB/s wr, 1.65k op/s 2024-05-20T16:16:13.885 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:16:13 smithi184 ceph-mon[36209]: pgmap v429: 129 pgs: 129 active+clean; 10 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 792 MiB/s wr, 1.65k op/s 2024-05-20T16:16:15.109 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:15.109+0000 7fa02affd640 1 -- 172.21.15.82:0/393746659 <== mds.0 v2:172.21.15.160:6838/2765554482 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55935b8f4dc0 con 0x55935b5e91b0 2024-05-20T16:16:15.109 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:15.109+0000 7fa02affd640 10 client.25168 handle_command_reply: tid=0 2024-05-20T16:16:15.110 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:15.109+0000 7fa058e00740 1 client.25168 shutdown 2024-05-20T16:16:15.114 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:15.114+0000 7fa058e00740 1 -- 172.21.15.82:0/393746659 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55935b5dd300 msgr2=0x55935ba066a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:16:15.114 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:15.114+0000 7fa058e00740 1 --2- 172.21.15.82:0/393746659 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55935b5dd300 0x55935ba066a0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fa040009460 tx=0x7fa040004810 comp rx=0 tx=0).stop 2024-05-20T16:16:15.115 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:15.114+0000 7fa058e00740 1 -- 172.21.15.82:0/393746659 shutdown_connections 2024-05-20T16:16:15.115 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:15.114+0000 7fa058e00740 1 --2- 172.21.15.82:0/393746659 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x55935b5e91b0 0x55935ba0a450 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:16:15.115 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:15.114+0000 7fa058e00740 1 --2- 172.21.15.82:0/393746659 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55935b5dd300 0x55935ba066a0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:16:15.115 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:15.114+0000 7fa058e00740 1 --2- 172.21.15.82:0/393746659 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55935b5c84d0 0x55935b9978d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:16:15.115 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:15.114+0000 7fa058e00740 1 --2- 172.21.15.82:0/393746659 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55935b5c02f0 0x55935b997390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:16:15.115 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:15.114+0000 7fa058e00740 1 -- 172.21.15.82:0/393746659 >> 172.21.15.82:0/393746659 conn(0x55935b9895f0 msgr2=0x55935b9841d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:16:15.115 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:15.114+0000 7fa02affd640 0 client.25168 ms_handle_reset on v2:172.21.15.160:6838/2765554482 2024-05-20T16:16:15.115 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:15.115+0000 7fa058e00740 1 -- 172.21.15.82:0/393746659 shutdown_connections 2024-05-20T16:16:15.115 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:15.115+0000 7fa058e00740 1 -- 172.21.15.82:0/393746659 wait complete. 2024-05-20T16:16:15.115 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:15.115+0000 7fa058e00740 20 client.25168 trim_cache size 0 max 16384 2024-05-20T16:16:15.116 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:{ 2024-05-20T16:16:15.116 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "message": "", 2024-05-20T16:16:15.116 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "return_code": 0 2024-05-20T16:16:15.116 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:} 2024-05-20T16:16:15.118 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:15.118+0000 7fa051f57640 1 -- 172.21.15.82:0/76826917 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fa0340a2d90 msgr2=0x7fa0340a52a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:16:15.118 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:15.118+0000 7fa051f57640 1 --2- 172.21.15.82:0/76826917 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fa0340a2d90 0x7fa0340a52a0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fa040009f50 tx=0x7fa0400127c0 comp rx=0 tx=0).stop 2024-05-20T16:16:15.118 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:15.118+0000 7fa051f57640 1 -- 172.21.15.82:0/76826917 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa04c10f020 msgr2=0x7fa04c1a0070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:16:15.118 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:15.118+0000 7fa051f57640 1 --2- 172.21.15.82:0/76826917 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa04c10f020 0x7fa04c1a0070 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fa04400ba50 tx=0x7fa04400bf10 comp rx=0 tx=0).stop 2024-05-20T16:16:15.119 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:15.119+0000 7fa051f57640 1 -- 172.21.15.82:0/76826917 shutdown_connections 2024-05-20T16:16:15.120 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:15.120+0000 7fa051f57640 1 --2- 172.21.15.82:0/76826917 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fa0340a2d90 0x7fa0340a52a0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:16:15.120 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:15.120+0000 7fa051f57640 1 --2- 172.21.15.82:0/76826917 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa04c1a4f70 0x7fa04c1a7390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:16:15.120 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:15.120+0000 7fa051f57640 1 --2- 172.21.15.82:0/76826917 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa04c10f020 0x7fa04c1a0070 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:16:15.120 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:15.120+0000 7fa051f57640 1 --2- 172.21.15.82:0/76826917 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa04c10a300 0x7fa04c19fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:16:15.120 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:15.120+0000 7fa051f57640 1 -- 172.21.15.82:0/76826917 >> 172.21.15.82:0/76826917 conn(0x7fa04c06f4f0 msgr2=0x7fa04c10d010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:16:15.120 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:15.120+0000 7fa051f57640 1 -- 172.21.15.82:0/76826917 shutdown_connections 2024-05-20T16:16:15.121 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:15.120+0000 7fa051f57640 1 -- 172.21.15.82:0/76826917 wait complete. 2024-05-20T16:16:15.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:16:15 smithi160 ceph-mon[36347]: pgmap v430: 129 pgs: 129 active+clean; 10 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 890 MiB/s wr, 1.61k op/s 2024-05-20T16:16:15.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:16:15 smithi082 ceph-mon[32238]: pgmap v430: 129 pgs: 129 active+clean; 10 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 890 MiB/s wr, 1.61k op/s 2024-05-20T16:16:15.885 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:16:15 smithi184 ceph-mon[36209]: pgmap v430: 129 pgs: 129 active+clean; 10 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 890 MiB/s wr, 1.61k op/s 2024-05-20T16:16:17.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:16:17 smithi160 ceph-mon[36347]: pgmap v431: 129 pgs: 129 active+clean; 10 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 756 MiB/s wr, 1.24k op/s 2024-05-20T16:16:17.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:16:17 smithi082 ceph-mon[32238]: pgmap v431: 129 pgs: 129 active+clean; 10 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 756 MiB/s wr, 1.24k op/s 2024-05-20T16:16:17.885 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:16:17 smithi184 ceph-mon[36209]: pgmap v431: 129 pgs: 129 active+clean; 10 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 756 MiB/s wr, 1.24k op/s 2024-05-20T16:16:19.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:16:19 smithi160 ceph-mon[36347]: pgmap v432: 129 pgs: 129 active+clean; 10 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 919 MiB/s wr, 1.56k op/s 2024-05-20T16:16:19.860 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:16:19 smithi082 ceph-mon[32238]: pgmap v432: 129 pgs: 129 active+clean; 10 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 919 MiB/s wr, 1.56k op/s 2024-05-20T16:16:19.885 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:16:19 smithi184 ceph-mon[36209]: pgmap v432: 129 pgs: 129 active+clean; 10 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 919 MiB/s wr, 1.56k op/s 2024-05-20T16:16:21.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:16:21 smithi160 ceph-mon[36347]: pgmap v433: 129 pgs: 129 active+clean; 10 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 784 MiB/s wr, 1.30k op/s 2024-05-20T16:16:21.860 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:16:21 smithi082 ceph-mon[32238]: pgmap v433: 129 pgs: 129 active+clean; 10 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 784 MiB/s wr, 1.30k op/s 2024-05-20T16:16:21.885 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:16:21 smithi184 ceph-mon[36209]: pgmap v433: 129 pgs: 129 active+clean; 10 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 784 MiB/s wr, 1.30k op/s 2024-05-20T16:16:23.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:16:23 smithi160 ceph-mon[36347]: pgmap v434: 129 pgs: 129 active+clean; 10 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 795 MiB/s wr, 1.22k op/s 2024-05-20T16:16:23.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:16:23 smithi082 ceph-mon[32238]: pgmap v434: 129 pgs: 129 active+clean; 10 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 795 MiB/s wr, 1.22k op/s 2024-05-20T16:16:23.885 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:16:23 smithi184 ceph-mon[36209]: pgmap v434: 129 pgs: 129 active+clean; 10 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 795 MiB/s wr, 1.22k op/s 2024-05-20T16:16:25.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:16:25 smithi160 ceph-mon[36347]: pgmap v435: 129 pgs: 129 active+clean; 10 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 879 MiB/s wr, 1.27k op/s 2024-05-20T16:16:25.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:16:25 smithi082 ceph-mon[32238]: pgmap v435: 129 pgs: 129 active+clean; 10 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 879 MiB/s wr, 1.27k op/s 2024-05-20T16:16:25.885 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:16:25 smithi184 ceph-mon[36209]: pgmap v435: 129 pgs: 129 active+clean; 10 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 879 MiB/s wr, 1.27k op/s 2024-05-20T16:16:27.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:16:27 smithi160 ceph-mon[36347]: pgmap v436: 129 pgs: 129 active+clean; 10 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 735 MiB/s wr, 1.09k op/s 2024-05-20T16:16:27.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:16:27 smithi082 ceph-mon[32238]: pgmap v436: 129 pgs: 129 active+clean; 10 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 735 MiB/s wr, 1.09k op/s 2024-05-20T16:16:27.885 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:16:27 smithi184 ceph-mon[36209]: pgmap v436: 129 pgs: 129 active+clean; 10 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 735 MiB/s wr, 1.09k op/s 2024-05-20T16:16:28.364 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:28.364+0000 7fb6d0a7a640 1 -- 172.21.15.82:0/937838106 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb6cc109300 msgr2=0x7fb6cc109c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:16:28.364 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:28.364+0000 7fb6d0a7a640 1 --2- 172.21.15.82:0/937838106 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb6cc109300 0x7fb6cc109c40 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fb6c0009860 tx=0x7fb6c002a380 comp rx=0 tx=0).stop 2024-05-20T16:16:28.364 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:28.365+0000 7fb6d0a7a640 1 -- 172.21.15.82:0/937838106 shutdown_connections 2024-05-20T16:16:28.365 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:28.365+0000 7fb6d0a7a640 1 --2- 172.21.15.82:0/937838106 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb6cc10eea0 0x7fb6cc10f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:16:28.365 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:28.365+0000 7fb6d0a7a640 1 --2- 172.21.15.82:0/937838106 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb6cc10a180 0x7fb6cc10a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:16:28.365 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:28.365+0000 7fb6d0a7a640 1 --2- 172.21.15.82:0/937838106 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb6cc109300 0x7fb6cc109c40 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:16:28.365 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:28.365+0000 7fb6d0a7a640 1 -- 172.21.15.82:0/937838106 >> 172.21.15.82:0/937838106 conn(0x7fb6cc06f340 msgr2=0x7fb6cc0717d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:16:28.365 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:28.365+0000 7fb6d0a7a640 1 -- 172.21.15.82:0/937838106 shutdown_connections 2024-05-20T16:16:28.366 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:28.365+0000 7fb6d0a7a640 1 -- 172.21.15.82:0/937838106 wait complete. 2024-05-20T16:16:28.367 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:28.367+0000 7fb6d0a7a640 1 Processor -- start 2024-05-20T16:16:28.367 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:28.367+0000 7fb6d0a7a640 1 -- start start 2024-05-20T16:16:28.367 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:28.367+0000 7fb6d0a7a640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb6cc109300 0x7fb6cc1a7f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:16:28.367 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:28.367+0000 7fb6d0a7a640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb6cc10a180 0x7fb6cc1a8480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:16:28.367 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:28.367+0000 7fb6d0a7a640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb6cc10eea0 0x7fb6cc1af410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:16:28.368 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:28.367+0000 7fb6d0a7a640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fb6cc11d400 con 0x7fb6cc109300 2024-05-20T16:16:28.368 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:28.367+0000 7fb6d0a7a640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fb6cc11d280 con 0x7fb6cc10a180 2024-05-20T16:16:28.368 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:28.367+0000 7fb6d0a7a640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fb6cc11d580 con 0x7fb6cc10eea0 2024-05-20T16:16:28.368 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:28.368+0000 7fb6cad9d640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb6cc10eea0 0x7fb6cc1af410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:16:28.368 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:28.368+0000 7fb6cad9d640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb6cc10eea0 0x7fb6cc1af410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.82:53074/0 (socket says 172.21.15.82:53074) 2024-05-20T16:16:28.368 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:28.368+0000 7fb6cad9d640 1 -- 172.21.15.82:0/967359408 learned_addr learned my addr 172.21.15.82:0/967359408 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:16:28.368 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:28.368+0000 7fb6c9d9b640 1 --2- 172.21.15.82:0/967359408 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb6cc10a180 0x7fb6cc1a8480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:16:28.368 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:28.368+0000 7fb6cad9d640 1 -- 172.21.15.82:0/967359408 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb6cc10a180 msgr2=0x7fb6cc1a8480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:16:28.369 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:28.368+0000 7fb6cad9d640 1 --2- 172.21.15.82:0/967359408 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb6cc10a180 0x7fb6cc1a8480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:16:28.369 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:28.368+0000 7fb6cad9d640 1 -- 172.21.15.82:0/967359408 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb6cc109300 msgr2=0x7fb6cc1a7f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:16:28.369 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:28.368+0000 7fb6cad9d640 1 --2- 172.21.15.82:0/967359408 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb6cc109300 0x7fb6cc1a7f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:16:28.369 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:28.368+0000 7fb6cad9d640 1 -- 172.21.15.82:0/967359408 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6cc1afa80 con 0x7fb6cc10eea0 2024-05-20T16:16:28.369 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:28.368+0000 7fb6c9d9b640 1 --2- 172.21.15.82:0/967359408 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb6cc10a180 0x7fb6cc1a8480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T16:16:28.369 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:28.369+0000 7fb6cad9d640 1 --2- 172.21.15.82:0/967359408 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb6cc10eea0 0x7fb6cc1af410 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fb6bc009fd0 tx=0x7fb6bc00b8e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:16:28.369 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:28.369+0000 7fb6bb7fe640 1 -- 172.21.15.82:0/967359408 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6bc011830 con 0x7fb6cc10eea0 2024-05-20T16:16:28.371 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:28.370+0000 7fb6d0a7a640 1 -- 172.21.15.82:0/967359408 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6cc1afd70 con 0x7fb6cc10eea0 2024-05-20T16:16:28.371 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:28.370+0000 7fb6d0a7a640 1 -- 172.21.15.82:0/967359408 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6cc1b0200 con 0x7fb6cc10eea0 2024-05-20T16:16:28.374 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:28.375+0000 7fb6d7a01740 1 Processor -- start 2024-05-20T16:16:28.375 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:28.375+0000 7fb6d7a01740 1 -- start start 2024-05-20T16:16:28.376 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:28.376+0000 7fb6d7a01740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5582aadf51b0 0x5582ab195050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:16:28.376 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:28.376+0000 7fb6d7a01740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5582aadcc2f0 0x5582ab1955d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:16:28.376 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:28.376+0000 7fb6d7a01740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5582aadd44d0 0x5582ab19c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:16:28.376 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:28.376+0000 7fb6d7a01740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x5582ab0f9340 con 0x5582aadd44d0 2024-05-20T16:16:28.377 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:28.373+0000 7fb6bb7fe640 1 -- 172.21.15.82:0/967359408 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb6bc016070 con 0x7fb6cc10eea0 2024-05-20T16:16:28.377 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:28.377+0000 7fb6ca59c640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5582aadcc2f0 0x5582ab1955d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:16:28.377 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:28.377+0000 7fb6ca59c640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5582aadcc2f0 0x5582ab1955d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.82:53082/0 (socket says 172.21.15.82:53082) 2024-05-20T16:16:28.377 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:28.377+0000 7fb6ca59c640 1 -- 172.21.15.82:0/94309632 learned_addr learned my addr 172.21.15.82:0/94309632 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:16:28.377 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:28.377+0000 7fb6c9d9b640 1 --2- 172.21.15.82:0/94309632 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5582aadd44d0 0x5582ab19c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:16:28.377 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:28.377+0000 7fb6d7a01740 1 -- 172.21.15.82:0/94309632 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x5582ab0a1290 con 0x5582aadf51b0 2024-05-20T16:16:28.377 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:28.377+0000 7fb6d7a01740 1 -- 172.21.15.82:0/94309632 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5582ab0f5300 con 0x5582aadcc2f0 2024-05-20T16:16:28.378 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:28.377+0000 7fb6c9d9b640 1 --2- 172.21.15.82:0/94309632 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5582aadf51b0 0x5582ab195050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:16:28.378 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:28.377+0000 7fb6bb7fe640 1 -- 172.21.15.82:0/967359408 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6bc00f2e0 con 0x7fb6cc10eea0 2024-05-20T16:16:28.378 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:28.377+0000 7fb6bb7fe640 1 -- 172.21.15.82:0/967359408 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fb6bc00f540 con 0x7fb6cc10eea0 2024-05-20T16:16:28.378 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:28.377+0000 7fb6ca59c640 1 -- 172.21.15.82:0/94309632 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5582aadf51b0 msgr2=0x5582ab195050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:16:28.378 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:28.377+0000 7fb6ca59c640 1 --2- 172.21.15.82:0/94309632 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5582aadf51b0 0x5582ab195050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:16:28.378 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:28.377+0000 7fb6ca59c640 1 -- 172.21.15.82:0/94309632 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5582aadd44d0 msgr2=0x5582ab19c4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:16:28.378 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:28.377+0000 7fb6ca59c640 1 --2- 172.21.15.82:0/94309632 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5582aadd44d0 0x5582ab19c4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:16:28.378 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:28.377+0000 7fb6ca59c640 1 -- 172.21.15.82:0/94309632 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5582ab0c58e0 con 0x5582aadcc2f0 2024-05-20T16:16:28.379 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:28.378+0000 7fb6ca59c640 1 --2- 172.21.15.82:0/94309632 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5582aadcc2f0 0x5582ab1955d0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fb6c0035040 tx=0x7fb6c0030430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=be8b84b6f940d28a server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:16:28.379 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:28.379+0000 7fb6bb7fe640 1 --2- 172.21.15.82:0/967359408 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fb6b40a2dd0 0x7fb6b40a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:16:28.379 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:28.380+0000 7fb6bb7fe640 1 -- 172.21.15.82:0/967359408 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9150+0+0 (secure 0 0 0) 0x7fb6bc0cfb10 con 0x7fb6cc10eea0 2024-05-20T16:16:28.386 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:28.387+0000 7fb6c9d9b640 1 --2- 172.21.15.82:0/967359408 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fb6b40a2dd0 0x7fb6b40a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:16:28.391 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:28.390+0000 7fb6b8ff9640 1 -- 172.21.15.82:0/94309632 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6c0037040 con 0x5582aadcc2f0 2024-05-20T16:16:28.391 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:28.390+0000 7fb6b8ff9640 1 -- 172.21.15.82:0/94309632 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb6c0040070 con 0x5582aadcc2f0 2024-05-20T16:16:28.391 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:28.390+0000 7fb6b8ff9640 1 -- 172.21.15.82:0/94309632 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6c0003970 con 0x5582aadcc2f0 2024-05-20T16:16:28.393 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:28.393+0000 7fb6d7a01740 1 -- 172.21.15.82:0/94309632 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5582aadcc2f0 msgr2=0x5582ab1955d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:16:28.393 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:28.393+0000 7fb6d7a01740 1 --2- 172.21.15.82:0/94309632 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5582aadcc2f0 0x5582ab1955d0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fb6c0035040 tx=0x7fb6c0030430 comp rx=0 tx=0).stop 2024-05-20T16:16:28.394 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:28.394+0000 7fb6d7a01740 1 -- 172.21.15.82:0/94309632 shutdown_connections 2024-05-20T16:16:28.394 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:28.394+0000 7fb6d7a01740 1 --2- 172.21.15.82:0/94309632 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5582aadd44d0 0x5582ab19c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:16:28.394 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:28.394+0000 7fb6d7a01740 1 --2- 172.21.15.82:0/94309632 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5582aadcc2f0 0x5582ab1955d0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:16:28.394 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:28.394+0000 7fb6d7a01740 1 --2- 172.21.15.82:0/94309632 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5582aadf51b0 0x5582ab195050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:16:28.394 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:28.394+0000 7fb6d7a01740 1 -- 172.21.15.82:0/94309632 >> 172.21.15.82:0/94309632 conn(0x5582aade9300 msgr2=0x5582ab1909e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:16:28.401 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:28.400+0000 7fb6d7a01740 1 -- 172.21.15.82:0/94309632 shutdown_connections 2024-05-20T16:16:28.402 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:28.401+0000 7fb6d7a01740 1 -- 172.21.15.82:0/94309632 wait complete. 2024-05-20T16:16:28.402 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:28.402+0000 7fb6d7a01740 1 -- start start 2024-05-20T16:16:28.402 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:28.402+0000 7fb6d7a01740 1 Processor -- start 2024-05-20T16:16:28.406 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:28.404+0000 7fb6d7a01740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5582aadcc2f0 0x5582ab19ed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:16:28.406 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:28.404+0000 7fb6d7a01740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5582aadd44d0 0x5582ab19f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:16:28.406 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:28.404+0000 7fb6d7a01740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5582aadf51b0 0x5582ab1a6200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:16:28.406 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:28.404+0000 7fb6d7a01740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x5582ab19cec0 con 0x5582aadd44d0 2024-05-20T16:16:28.406 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:28.404+0000 7fb6d7a01740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x5582ab0b4790 con 0x5582aadf51b0 2024-05-20T16:16:28.406 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:28.404+0000 7fb6d7a01740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5582ab19d040 con 0x5582aadcc2f0 2024-05-20T16:16:28.406 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:28.404+0000 7fb6d7a01740 10 client.0 _ll_register_callbacks cb 0x5582ab0efa20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T16:16:28.406 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:28.405+0000 7fb6c9d9b640 1 --2- 172.21.15.82:0/967359408 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fb6b40a2dd0 0x7fb6b40a52e0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fb6c4005c40 tx=0x7fb6c4009a80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:16:28.407 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:28.406+0000 7fb6c9d9b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5582aadd44d0 0x5582ab19f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:16:28.407 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:28.406+0000 7fb6c9d9b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5582aadd44d0 0x5582ab19f2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:48282/0 (socket says 172.21.15.82:48282) 2024-05-20T16:16:28.407 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:28.406+0000 7fb6c9d9b640 1 -- 172.21.15.82:0/156781590 learned_addr learned my addr 172.21.15.82:0/156781590 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:16:28.407 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:28.407+0000 7fb6cad9d640 1 --2- 172.21.15.82:0/156781590 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5582aadf51b0 0x5582ab1a6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:16:28.409 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:28.409+0000 7fb6ca59c640 1 --2- 172.21.15.82:0/156781590 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5582aadcc2f0 0x5582ab19ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:16:28.411 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:28.410+0000 7fb6cad9d640 1 -- 172.21.15.82:0/156781590 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5582aadcc2f0 msgr2=0x5582ab19ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:16:28.411 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:28.410+0000 7fb6cad9d640 1 --2- 172.21.15.82:0/156781590 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5582aadcc2f0 0x5582ab19ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:16:28.412 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:28.410+0000 7fb6cad9d640 1 -- 172.21.15.82:0/156781590 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5582aadd44d0 msgr2=0x5582ab19f2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:16:28.412 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:28.410+0000 7fb6cad9d640 1 --2- 172.21.15.82:0/156781590 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5582aadd44d0 0x5582ab19f2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:16:28.412 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:28.410+0000 7fb6cad9d640 1 -- 172.21.15.82:0/156781590 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5582aae0d210 con 0x5582aadf51b0 2024-05-20T16:16:28.413 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:28.413+0000 7fb6cad9d640 1 --2- 172.21.15.82:0/156781590 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5582aadf51b0 0x5582ab1a6200 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fb6bc09bd00 tx=0x7fb6bc00ff40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3558a5144819ef72 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:16:28.415 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:28.415+0000 7fb6b1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-20T16:16:28.415 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:28.415+0000 7fb6b1ffb640 1 -- 172.21.15.82:0/156781590 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6bc0a03b0 con 0x5582aadf51b0 2024-05-20T16:16:28.415 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:28.415+0000 7fb6b1ffb640 1 -- 172.21.15.82:0/156781590 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9150+0+0 (secure 0 0 0) 0x7fb6bc098670 con 0x5582aadf51b0 2024-05-20T16:16:28.416 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:28.417+0000 7fb6d7a01740 1 -- 172.21.15.82:0/156781590 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb6c0040070 con 0x5582aadf51b0 2024-05-20T16:16:28.421 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:28.421+0000 7fb6b1ffb640 1 -- 172.21.15.82:0/156781590 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fb6bc0a4c70 con 0x5582aadf51b0 2024-05-20T16:16:28.421 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:28.421+0000 7fb6d7a01740 10 client.25154 fetch_fsmap learned FSMap version 25 2024-05-20T16:16:28.421 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:28.421+0000 7fb6d7a01740 1 -- 172.21.15.82:0/156781590 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5582aaed9620 con 0x5582aadf51b0 2024-05-20T16:16:28.426 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:28.424+0000 7fb6b1ffb640 1 -- 172.21.15.82:0/156781590 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb6c0040070 con 0x5582aadf51b0 2024-05-20T16:16:28.427 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:28.426+0000 7fb6b1ffb640 1 -- 172.21.15.82:0/156781590 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fb6bc097d50 con 0x5582aadf51b0 2024-05-20T16:16:28.428 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:28.427+0000 7fb6d7a01740 10 client.25154 fetch_fsmap finished waiting for FSMap version 25 2024-05-20T16:16:28.429 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:28.427+0000 7fb6d7a01740 10 client.25154 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T16:16:28.429 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:28.427+0000 7fb6d7a01740 20 client.25154 populate_metadata read hostname 'smithi082' 2024-05-20T16:16:28.429 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:28.427+0000 7fb6d7a01740 1 --2- 172.21.15.82:0/156781590 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x5582aadb3d80 0x5582ab1acc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:16:28.429 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:28.427+0000 7fb6d7a01740 4 client.25154 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T16:16:28.429 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:28.427+0000 7fb6d7a01740 1 -- 172.21.15.82:0/156781590 --> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5582ab106a70 con 0x5582aadb3d80 2024-05-20T16:16:28.431 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:28.431+0000 7fb6ca59c640 1 --2- 172.21.15.82:0/156781590 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x5582aadb3d80 0x5582ab1acc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:16:28.445 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:28.443+0000 7fb6ca59c640 1 --2- 172.21.15.82:0/156781590 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x5582aadb3d80 0x5582ab1acc90 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b80a350a6c1a7e81 server_cookie=ff84eb1660bc503d in_seq=0 out_seq=0 2024-05-20T16:16:28.446 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:28.446+0000 7fb6b1ffb640 10 client.25154 ms_handle_connect on v2:172.21.15.160:6838/2765554482 2024-05-20T16:16:28.449 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:28.448+0000 7fb6b1ffb640 1 -- 172.21.15.82:0/156781590 <== mds.0 v2:172.21.15.160:6838/2765554482 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5582ab106a70 con 0x5582aadb3d80 2024-05-20T16:16:28.450 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:28.449+0000 7fb6b1ffb640 10 client.25154 handle_command_reply: tid=0 2024-05-20T16:16:28.450 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:28.449+0000 7fb6d7a01740 1 client.25154 shutdown 2024-05-20T16:16:28.457 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:28.452+0000 7fb6d7a01740 1 -- 172.21.15.82:0/156781590 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5582aadf51b0 msgr2=0x5582ab1a6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:16:28.457 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:28.452+0000 7fb6d7a01740 1 --2- 172.21.15.82:0/156781590 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5582aadf51b0 0x5582ab1a6200 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fb6bc09bd00 tx=0x7fb6bc00ff40 comp rx=0 tx=0).stop 2024-05-20T16:16:28.457 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:28.452+0000 7fb6d7a01740 1 -- 172.21.15.82:0/156781590 shutdown_connections 2024-05-20T16:16:28.457 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:28.452+0000 7fb6d7a01740 1 --2- 172.21.15.82:0/156781590 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x5582aadb3d80 0x5582ab1acc90 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:16:28.457 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:28.452+0000 7fb6d7a01740 1 --2- 172.21.15.82:0/156781590 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5582aadd44d0 0x5582ab19f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:16:28.457 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:28.452+0000 7fb6d7a01740 1 --2- 172.21.15.82:0/156781590 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5582aadf51b0 0x5582ab1a6200 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:16:28.457 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:28.452+0000 7fb6d7a01740 1 --2- 172.21.15.82:0/156781590 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5582aadcc2f0 0x5582ab19ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:16:28.457 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:28.452+0000 7fb6d7a01740 1 -- 172.21.15.82:0/156781590 >> 172.21.15.82:0/156781590 conn(0x5582aade9300 msgr2=0x5582ab193990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:16:28.457 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:28.452+0000 7fb6b1ffb640 0 client.25154 ms_handle_reset on v2:172.21.15.160:6838/2765554482 2024-05-20T16:16:28.459 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:28.456+0000 7fb6d7a01740 1 -- 172.21.15.82:0/156781590 shutdown_connections 2024-05-20T16:16:28.459 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:28.456+0000 7fb6d7a01740 1 -- 172.21.15.82:0/156781590 wait complete. 2024-05-20T16:16:28.459 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:28.458+0000 7fb6d7a01740 20 client.25154 trim_cache size 0 max 16384 2024-05-20T16:16:28.539 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:28.539+0000 7fb6d7a01740 1 Processor -- start 2024-05-20T16:16:28.539 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:28.539+0000 7fb6d7a01740 1 -- start start 2024-05-20T16:16:28.540 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:28.540+0000 7fb6d7a01740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5582aadcc2f0 0x5582ab19e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:16:28.540 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:28.540+0000 7fb6d7a01740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5582aadd44d0 0x5582ab1a6120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:16:28.541 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:28.540+0000 7fb6ca59c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5582aadcc2f0 0x5582ab19e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:16:28.541 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:28.541+0000 7fb6ca59c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5582aadcc2f0 0x5582ab19e720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:48284/0 (socket says 172.21.15.82:48284) 2024-05-20T16:16:28.542 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:28.542+0000 7fb6ca59c640 1 -- 172.21.15.82:0/2823917283 learned_addr learned my addr 172.21.15.82:0/2823917283 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:16:28.542 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:28.542+0000 7fb6d7a01740 1 --2- 172.21.15.82:0/2823917283 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5582aadf51b0 0x5582ab1b2450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:16:28.542 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:28.542+0000 7fb6d7a01740 1 -- 172.21.15.82:0/2823917283 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fb6ac012330 con 0x5582aadcc2f0 2024-05-20T16:16:28.542 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:28.542+0000 7fb6d7a01740 1 -- 172.21.15.82:0/2823917283 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fb6ac00fc00 con 0x5582aadf51b0 2024-05-20T16:16:28.542 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:28.542+0000 7fb6d7a01740 1 -- 172.21.15.82:0/2823917283 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fb6ac00db00 con 0x5582aadd44d0 2024-05-20T16:16:28.547 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:28.547+0000 7fb6c9d9b640 1 --2- 172.21.15.82:0/2823917283 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5582aadd44d0 0x5582ab1a6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:16:28.547 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:28.547+0000 7fb6cad9d640 1 --2- 172.21.15.82:0/2823917283 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5582aadf51b0 0x5582ab1b2450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:16:28.547 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:28.547+0000 7fb6ca59c640 1 -- 172.21.15.82:0/2823917283 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5582aadd44d0 msgr2=0x5582ab1a6120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:16:28.547 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:28.547+0000 7fb6ca59c640 1 --2- 172.21.15.82:0/2823917283 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5582aadd44d0 0x5582ab1a6120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:16:28.548 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:28.547+0000 7fb6ca59c640 1 -- 172.21.15.82:0/2823917283 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5582aadf51b0 msgr2=0x5582ab1b2450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:16:28.548 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:28.547+0000 7fb6ca59c640 1 --2- 172.21.15.82:0/2823917283 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5582aadf51b0 0x5582ab1b2450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:16:28.548 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:28.547+0000 7fb6ca59c640 1 -- 172.21.15.82:0/2823917283 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5582ab194c00 con 0x5582aadcc2f0 2024-05-20T16:16:28.548 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:28.547+0000 7fb6cad9d640 1 --2- 172.21.15.82:0/2823917283 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5582aadf51b0 0x5582ab1b2450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T16:16:28.548 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:28.547+0000 7fb6ca59c640 1 --2- 172.21.15.82:0/2823917283 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5582aadcc2f0 0x5582ab19e720 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fb6c0009430 tx=0x7fb6c0000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5cae2f55ecc0321 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:16:28.549 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:28.548+0000 7fb6b3fff640 1 -- 172.21.15.82:0/2823917283 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6c0037040 con 0x5582aadcc2f0 2024-05-20T16:16:28.549 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:28.549+0000 7fb6b3fff640 1 -- 172.21.15.82:0/2823917283 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb6c0040040 con 0x5582aadcc2f0 2024-05-20T16:16:28.549 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:28.549+0000 7fb6b3fff640 1 -- 172.21.15.82:0/2823917283 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6c004d750 con 0x5582aadcc2f0 2024-05-20T16:16:28.554 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:28.550+0000 7fb6d7a01740 1 -- 172.21.15.82:0/2823917283 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5582aadcc2f0 msgr2=0x5582ab19e720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:16:28.555 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:28.550+0000 7fb6d7a01740 1 --2- 172.21.15.82:0/2823917283 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5582aadcc2f0 0x5582ab19e720 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fb6c0009430 tx=0x7fb6c0000bc0 comp rx=0 tx=0).stop 2024-05-20T16:16:28.555 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:28.552+0000 7fb6d7a01740 1 -- 172.21.15.82:0/2823917283 shutdown_connections 2024-05-20T16:16:28.555 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:28.552+0000 7fb6d7a01740 1 --2- 172.21.15.82:0/2823917283 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5582aadf51b0 0x5582ab1b2450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:16:28.555 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:28.552+0000 7fb6d7a01740 1 --2- 172.21.15.82:0/2823917283 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5582aadd44d0 0x5582ab1a6120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:16:28.555 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:28.552+0000 7fb6d7a01740 1 --2- 172.21.15.82:0/2823917283 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5582aadcc2f0 0x5582ab19e720 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:16:28.555 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:28.552+0000 7fb6d7a01740 1 -- 172.21.15.82:0/2823917283 >> 172.21.15.82:0/2823917283 conn(0x5582aade9300 msgr2=0x5582ab19e010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:16:28.558 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:28.554+0000 7fb6d7a01740 1 -- 172.21.15.82:0/2823917283 shutdown_connections 2024-05-20T16:16:28.559 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:28.558+0000 7fb6d7a01740 1 -- 172.21.15.82:0/2823917283 wait complete. 2024-05-20T16:16:28.559 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:28.559+0000 7fb6d7a01740 1 -- start start 2024-05-20T16:16:28.559 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:28.559+0000 7fb6d7a01740 1 Processor -- start 2024-05-20T16:16:28.559 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:28.560+0000 7fb6d7a01740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5582aadcc2f0 0x5582ab1a5400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:16:28.559 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:28.560+0000 7fb6d7a01740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5582aadd44d0 0x5582ab1a5940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:16:28.560 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:28.560+0000 7fb6d7a01740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5582aadf51b0 0x5582ab213120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:16:28.561 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:28.560+0000 7fb6d7a01740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fb6ac00e0c0 con 0x5582aadd44d0 2024-05-20T16:16:28.561 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:28.560+0000 7fb6d7a01740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fb6ac00f530 con 0x5582aadf51b0 2024-05-20T16:16:28.561 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:28.560+0000 7fb6d7a01740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fb6ac00e790 con 0x5582aadcc2f0 2024-05-20T16:16:28.561 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:28.561+0000 7fb6d7a01740 10 client.0 _ll_register_callbacks cb 0x7fb6bc010fa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T16:16:28.561 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:28.561+0000 7fb6cad9d640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5582aadf51b0 0x5582ab213120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:16:28.561 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:28.561+0000 7fb6cad9d640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5582aadf51b0 0x5582ab213120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.82:51312/0 (socket says 172.21.15.82:51312) 2024-05-20T16:16:28.561 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:28.561+0000 7fb6cad9d640 1 -- 172.21.15.82:0/4165486703 learned_addr learned my addr 172.21.15.82:0/4165486703 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:16:28.561 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:28.561+0000 7fb6ca59c640 1 --2- 172.21.15.82:0/4165486703 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5582aadcc2f0 0x5582ab1a5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:16:28.562 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:28.562+0000 7fb6ca59c640 1 -- 172.21.15.82:0/4165486703 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5582aadf51b0 msgr2=0x5582ab213120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:16:28.562 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:28.562+0000 7fb6ca59c640 1 --2- 172.21.15.82:0/4165486703 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5582aadf51b0 0x5582ab213120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:16:28.562 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:28.562+0000 7fb6ca59c640 1 -- 172.21.15.82:0/4165486703 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5582aadd44d0 msgr2=0x5582ab1a5940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:16:28.562 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:28.562+0000 7fb6ca59c640 1 --2- 172.21.15.82:0/4165486703 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5582aadd44d0 0x5582ab1a5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:16:28.562 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:28.562+0000 7fb6ca59c640 1 -- 172.21.15.82:0/4165486703 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5582ab213710 con 0x5582aadcc2f0 2024-05-20T16:16:28.562 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:28.562+0000 7fb6ca59c640 1 --2- 172.21.15.82:0/4165486703 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5582aadcc2f0 0x5582ab1a5400 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x5582ab1aa460 tx=0x7fb6c0005d30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cf70e7c8bd41feba server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:16:28.564 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:28.564+0000 7fb6b1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-05-20T16:16:28.564 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:28.564+0000 7fb6b1ffb640 1 -- 172.21.15.82:0/4165486703 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6c0037030 con 0x5582aadcc2f0 2024-05-20T16:16:28.564 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:28.564+0000 7fb6d7a01740 1 -- 172.21.15.82:0/4165486703 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb6c0040040 con 0x5582aadcc2f0 2024-05-20T16:16:28.564 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:28.564+0000 7fb6b1ffb640 1 -- 172.21.15.82:0/4165486703 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9150+0+0 (secure 0 0 0) 0x7fb6c0030700 con 0x5582aadcc2f0 2024-05-20T16:16:28.566 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:28.567+0000 7fb6b1ffb640 1 -- 172.21.15.82:0/4165486703 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fb6c004d4d0 con 0x5582aadcc2f0 2024-05-20T16:16:28.570 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:28.571+0000 7fb6d7a01740 10 client.25186 fetch_fsmap learned FSMap version 25 2024-05-20T16:16:28.570 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:28.571+0000 7fb6d7a01740 1 -- 172.21.15.82:0/4165486703 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5582ab2148f0 con 0x5582aadcc2f0 2024-05-20T16:16:28.574 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:28.575+0000 7fb6b1ffb640 1 -- 172.21.15.82:0/4165486703 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb6c0040040 con 0x5582aadcc2f0 2024-05-20T16:16:28.575 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:28.575+0000 7fb6b1ffb640 1 -- 172.21.15.82:0/4165486703 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fb6c003a070 con 0x5582aadcc2f0 2024-05-20T16:16:28.575 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:28.575+0000 7fb6d7a01740 10 client.25186 fetch_fsmap finished waiting for FSMap version 25 2024-05-20T16:16:28.575 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:28.575+0000 7fb6d7a01740 10 client.25186 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T16:16:28.575 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:28.575+0000 7fb6d7a01740 20 client.25186 populate_metadata read hostname 'smithi082' 2024-05-20T16:16:28.575 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:28.575+0000 7fb6d7a01740 1 --2- 172.21.15.82:0/4165486703 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x5582aad891c0 0x5582ab216cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:16:28.575 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:28.575+0000 7fb6d7a01740 4 client.25186 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-20T16:16:28.575 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:28.575+0000 7fb6d7a01740 1 -- 172.21.15.82:0/4165486703 --> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5582ab18baa0 con 0x5582aad891c0 2024-05-20T16:16:28.578 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:28.578+0000 7fb6c9d9b640 1 --2- 172.21.15.82:0/4165486703 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x5582aad891c0 0x5582ab216cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:16:28.589 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:28.589+0000 7fb6b1ffb640 10 client.25186 ms_handle_connect on v2:172.21.15.160:6838/2765554482 2024-05-20T16:16:28.589 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:28.589+0000 7fb6c9d9b640 1 --2- 172.21.15.82:0/4165486703 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x5582aad891c0 0x5582ab216cc0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8ded08fcd2206d42 server_cookie=77b87422fec13e2a in_seq=0 out_seq=0 2024-05-20T16:16:28.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:16:28 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T16:16:28.861 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:16:28 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T16:16:28.885 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:16:28 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T16:16:29.713 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:16:29 smithi160 ceph-mon[36347]: pgmap v437: 129 pgs: 129 active+clean; 10 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 909 MiB/s wr, 1.50k op/s 2024-05-20T16:16:29.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:16:29 smithi082 ceph-mon[32238]: pgmap v437: 129 pgs: 129 active+clean; 10 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 909 MiB/s wr, 1.50k op/s 2024-05-20T16:16:29.885 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:16:29 smithi184 ceph-mon[36209]: pgmap v437: 129 pgs: 129 active+clean; 10 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 909 MiB/s wr, 1.50k op/s 2024-05-20T16:16:30.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:16:30 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:16:30.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:16:30 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T16:16:30.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:16:30 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:16:30.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:16:30 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:16:30.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:16:30 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T16:16:30.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:16:30 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:16:30.885 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:16:30 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:16:30.885 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:16:30 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T16:16:30.885 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:16:30 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:16:31.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:16:31 smithi160 ceph-mon[36347]: pgmap v438: 129 pgs: 129 active+clean; 10 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 806 MiB/s wr, 1.30k op/s 2024-05-20T16:16:31.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:16:31 smithi082 ceph-mon[32238]: pgmap v438: 129 pgs: 129 active+clean; 10 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 806 MiB/s wr, 1.30k op/s 2024-05-20T16:16:31.886 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:16:31 smithi184 ceph-mon[36209]: pgmap v438: 129 pgs: 129 active+clean; 10 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 806 MiB/s wr, 1.30k op/s 2024-05-20T16:16:33.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:16:33 smithi160 ceph-mon[36347]: pgmap v439: 129 pgs: 129 active+clean; 10 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 815 MiB/s wr, 1.24k op/s 2024-05-20T16:16:33.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:16:33 smithi082 ceph-mon[32238]: pgmap v439: 129 pgs: 129 active+clean; 10 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 815 MiB/s wr, 1.24k op/s 2024-05-20T16:16:33.885 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:16:33 smithi184 ceph-mon[36209]: pgmap v439: 129 pgs: 129 active+clean; 10 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 815 MiB/s wr, 1.24k op/s 2024-05-20T16:16:34.226 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:34.226+0000 7fb6b1ffb640 1 -- 172.21.15.82:0/4165486703 <== mds.0 v2:172.21.15.160:6838/2765554482 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5582ab18baa0 con 0x5582aad891c0 2024-05-20T16:16:34.226 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:34.226+0000 7fb6b1ffb640 10 client.25186 handle_command_reply: tid=0 2024-05-20T16:16:34.226 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:34.226+0000 7fb6d7a01740 1 client.25186 shutdown 2024-05-20T16:16:34.226 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:34.226+0000 7fb6d7a01740 1 -- 172.21.15.82:0/4165486703 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5582aadcc2f0 msgr2=0x5582ab1a5400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:16:34.226 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:34.226+0000 7fb6d7a01740 1 --2- 172.21.15.82:0/4165486703 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5582aadcc2f0 0x5582ab1a5400 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x5582ab1aa460 tx=0x7fb6c0005d30 comp rx=0 tx=0).stop 2024-05-20T16:16:34.227 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:34.226+0000 7fb6d7a01740 1 -- 172.21.15.82:0/4165486703 shutdown_connections 2024-05-20T16:16:34.227 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:34.226+0000 7fb6d7a01740 1 --2- 172.21.15.82:0/4165486703 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x5582aad891c0 0x5582ab216cc0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:16:34.227 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:34.226+0000 7fb6d7a01740 1 --2- 172.21.15.82:0/4165486703 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5582aadd44d0 0x5582ab1a5940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:16:34.227 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:34.226+0000 7fb6d7a01740 1 --2- 172.21.15.82:0/4165486703 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5582aadf51b0 0x5582ab213120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:16:34.227 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:34.226+0000 7fb6d7a01740 1 --2- 172.21.15.82:0/4165486703 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5582aadcc2f0 0x5582ab1a5400 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:16:34.227 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:34.226+0000 7fb6d7a01740 1 -- 172.21.15.82:0/4165486703 >> 172.21.15.82:0/4165486703 conn(0x5582aade9300 msgr2=0x5582ab193880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:16:34.228 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:34.227+0000 7fb6b1ffb640 0 client.25186 ms_handle_reset on v2:172.21.15.160:6838/2765554482 2024-05-20T16:16:34.228 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:34.227+0000 7fb6d7a01740 1 -- 172.21.15.82:0/4165486703 shutdown_connections 2024-05-20T16:16:34.228 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:34.227+0000 7fb6d7a01740 1 -- 172.21.15.82:0/4165486703 wait complete. 2024-05-20T16:16:34.228 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:34.227+0000 7fb6d7a01740 20 client.25186 trim_cache size 0 max 16384 2024-05-20T16:16:34.228 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:{ 2024-05-20T16:16:34.228 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "message": "", 2024-05-20T16:16:34.228 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "return_code": 0 2024-05-20T16:16:34.228 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:} 2024-05-20T16:16:34.233 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:34.234+0000 7fb6d0a7a640 1 -- 172.21.15.82:0/967359408 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fb6b40a2dd0 msgr2=0x7fb6b40a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:16:34.233 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:34.234+0000 7fb6d0a7a640 1 --2- 172.21.15.82:0/967359408 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fb6b40a2dd0 0x7fb6b40a52e0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fb6c4005c40 tx=0x7fb6c4009a80 comp rx=0 tx=0).stop 2024-05-20T16:16:34.234 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:34.234+0000 7fb6d0a7a640 1 -- 172.21.15.82:0/967359408 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb6cc10eea0 msgr2=0x7fb6cc1af410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:16:34.234 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:34.234+0000 7fb6d0a7a640 1 --2- 172.21.15.82:0/967359408 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb6cc10eea0 0x7fb6cc1af410 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fb6bc009fd0 tx=0x7fb6bc00b8e0 comp rx=0 tx=0).stop 2024-05-20T16:16:34.238 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:34.238+0000 7fb6d0a7a640 1 -- 172.21.15.82:0/967359408 shutdown_connections 2024-05-20T16:16:34.238 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:34.238+0000 7fb6d0a7a640 1 --2- 172.21.15.82:0/967359408 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fb6b40a2dd0 0x7fb6b40a52e0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:16:34.238 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:34.238+0000 7fb6d0a7a640 1 --2- 172.21.15.82:0/967359408 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb6cc10eea0 0x7fb6cc1af410 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:16:34.238 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:34.238+0000 7fb6d0a7a640 1 --2- 172.21.15.82:0/967359408 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb6cc10a180 0x7fb6cc1a8480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:16:34.238 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:34.238+0000 7fb6d0a7a640 1 --2- 172.21.15.82:0/967359408 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb6cc109300 0x7fb6cc1a7f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:16:34.238 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:34.238+0000 7fb6d0a7a640 1 -- 172.21.15.82:0/967359408 >> 172.21.15.82:0/967359408 conn(0x7fb6cc06f340 msgr2=0x7fb6cc10ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:16:34.239 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:34.239+0000 7fb6d0a7a640 1 -- 172.21.15.82:0/967359408 shutdown_connections 2024-05-20T16:16:34.239 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:34.239+0000 7fb6d0a7a640 1 -- 172.21.15.82:0/967359408 wait complete. 2024-05-20T16:16:35.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:16:35 smithi160 ceph-mon[36347]: pgmap v440: 129 pgs: 129 active+clean; 10 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 879 MiB/s wr, 1.37k op/s 2024-05-20T16:16:35.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:16:35 smithi082 ceph-mon[32238]: pgmap v440: 129 pgs: 129 active+clean; 10 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 879 MiB/s wr, 1.37k op/s 2024-05-20T16:16:35.885 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:16:35 smithi184 ceph-mon[36209]: pgmap v440: 129 pgs: 129 active+clean; 10 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 879 MiB/s wr, 1.37k op/s 2024-05-20T16:16:37.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:16:37 smithi160 ceph-mon[36347]: pgmap v441: 129 pgs: 129 active+clean; 10 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 726 MiB/s wr, 1.24k op/s 2024-05-20T16:16:37.864 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:16:37 smithi082 ceph-mon[32238]: pgmap v441: 129 pgs: 129 active+clean; 10 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 726 MiB/s wr, 1.24k op/s 2024-05-20T16:16:37.885 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:16:37 smithi184 ceph-mon[36209]: pgmap v441: 129 pgs: 129 active+clean; 10 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 726 MiB/s wr, 1.24k op/s 2024-05-20T16:16:39.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:16:39 smithi160 ceph-mon[36347]: pgmap v442: 129 pgs: 129 active+clean; 10 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 904 MiB/s wr, 1.39k op/s 2024-05-20T16:16:39.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:16:39 smithi082 ceph-mon[32238]: pgmap v442: 129 pgs: 129 active+clean; 10 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 904 MiB/s wr, 1.39k op/s 2024-05-20T16:16:39.885 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:16:39 smithi184 ceph-mon[36209]: pgmap v442: 129 pgs: 129 active+clean; 10 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 904 MiB/s wr, 1.39k op/s 2024-05-20T16:16:41.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:16:41 smithi160 ceph-mon[36347]: pgmap v443: 129 pgs: 129 active+clean; 10 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 805 MiB/s wr, 1.05k op/s 2024-05-20T16:16:41.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:16:41 smithi082 ceph-mon[32238]: pgmap v443: 129 pgs: 129 active+clean; 10 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 805 MiB/s wr, 1.05k op/s 2024-05-20T16:16:41.885 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:16:41 smithi184 ceph-mon[36209]: pgmap v443: 129 pgs: 129 active+clean; 10 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 805 MiB/s wr, 1.05k op/s 2024-05-20T16:16:43.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:16:43 smithi160 ceph-mon[36347]: pgmap v444: 129 pgs: 129 active+clean; 10 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 799 MiB/s wr, 962 op/s 2024-05-20T16:16:43.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:16:43 smithi082 ceph-mon[32238]: pgmap v444: 129 pgs: 129 active+clean; 10 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 799 MiB/s wr, 962 op/s 2024-05-20T16:16:43.885 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:16:43 smithi184 ceph-mon[36209]: pgmap v444: 129 pgs: 129 active+clean; 10 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 799 MiB/s wr, 962 op/s 2024-05-20T16:16:45.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:16:45 smithi160 ceph-mon[36347]: pgmap v445: 129 pgs: 129 active+clean; 10 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 846 MiB/s wr, 1.02k op/s 2024-05-20T16:16:45.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:16:45 smithi082 ceph-mon[32238]: pgmap v445: 129 pgs: 129 active+clean; 10 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 846 MiB/s wr, 1.02k op/s 2024-05-20T16:16:45.885 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:16:45 smithi184 ceph-mon[36209]: pgmap v445: 129 pgs: 129 active+clean; 10 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 846 MiB/s wr, 1.02k op/s 2024-05-20T16:16:47.508 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:47.508+0000 7f58c0a4d640 1 -- 172.21.15.82:0/2358403502 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f58bc10ee60 msgr2=0x7f58bc10f270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:16:47.508 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:47.508+0000 7f58c0a4d640 1 --2- 172.21.15.82:0/2358403502 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f58bc10ee60 0x7f58bc10f270 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f58b4009ca0 tx=0x7f58b402a8f0 comp rx=0 tx=0).stop 2024-05-20T16:16:47.512 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:47.512+0000 7f58c0a4d640 1 -- 172.21.15.82:0/2358403502 shutdown_connections 2024-05-20T16:16:47.512 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:47.512+0000 7f58c0a4d640 1 --2- 172.21.15.82:0/2358403502 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f58bc10ee60 0x7f58bc10f270 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:16:47.512 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:47.512+0000 7f58c0a4d640 1 --2- 172.21.15.82:0/2358403502 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f58bc10a140 0x7f58bc10a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:16:47.513 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:47.512+0000 7f58c0a4d640 1 --2- 172.21.15.82:0/2358403502 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f58bc1092f0 0x7f58bc109c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:16:47.513 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:47.513+0000 7f58c0a4d640 1 -- 172.21.15.82:0/2358403502 >> 172.21.15.82:0/2358403502 conn(0x7f58bc06f2e0 msgr2=0x7f58bc071790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:16:47.513 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:47.514+0000 7f58c0a4d640 1 -- 172.21.15.82:0/2358403502 shutdown_connections 2024-05-20T16:16:47.514 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:47.514+0000 7f58c0a4d640 1 -- 172.21.15.82:0/2358403502 wait complete. 2024-05-20T16:16:47.515 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:47.515+0000 7f58c0a4d640 1 Processor -- start 2024-05-20T16:16:47.518 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:47.518+0000 7f58c0a4d640 1 -- start start 2024-05-20T16:16:47.518 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:47.518+0000 7f58c0a4d640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f58bc1092f0 0x7f58bc1a7e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:16:47.518 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:47.518+0000 7f58c0a4d640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f58bc10a140 0x7f58bc1a8350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:16:47.519 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:47.518+0000 7f58c0a4d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f58bc10ee60 0x7f58bc1af2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:16:47.519 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:47.518+0000 7f58c0a4d640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f58bc11d390 con 0x7f58bc10ee60 2024-05-20T16:16:47.519 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:47.518+0000 7f58c0a4d640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f58bc11d210 con 0x7f58bc10a140 2024-05-20T16:16:47.519 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:47.518+0000 7f58c0a4d640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f58bc11d510 con 0x7f58bc1092f0 2024-05-20T16:16:47.524 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:47.524+0000 7f58baffd640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f58bc10a140 0x7f58bc1a8350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:16:47.524 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:47.524+0000 7f58bb7fe640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f58bc1092f0 0x7f58bc1a7e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:16:47.530 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:47.526+0000 7f58bb7fe640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f58bc1092f0 0x7f58bc1a7e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.82:58276/0 (socket says 172.21.15.82:58276) 2024-05-20T16:16:47.530 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:47.526+0000 7f58bb7fe640 1 -- 172.21.15.82:0/1038080821 learned_addr learned my addr 172.21.15.82:0/1038080821 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:16:47.530 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:47.526+0000 7f58bbfff640 1 --2- 172.21.15.82:0/1038080821 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f58bc10ee60 0x7f58bc1af2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:16:47.532 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:47.531+0000 7f58bb7fe640 1 -- 172.21.15.82:0/1038080821 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f58bc10a140 msgr2=0x7f58bc1a8350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:16:47.532 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:47.531+0000 7f58bb7fe640 1 --2- 172.21.15.82:0/1038080821 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f58bc10a140 0x7f58bc1a8350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:16:47.533 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:47.531+0000 7f58bb7fe640 1 -- 172.21.15.82:0/1038080821 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f58bc10ee60 msgr2=0x7f58bc1af2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:16:47.533 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:47.531+0000 7f58bb7fe640 1 --2- 172.21.15.82:0/1038080821 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f58bc10ee60 0x7f58bc1af2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:16:47.533 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:47.531+0000 7f58bb7fe640 1 -- 172.21.15.82:0/1038080821 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58bc110c40 con 0x7f58bc1092f0 2024-05-20T16:16:47.543 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:47.536+0000 7f58bb7fe640 1 --2- 172.21.15.82:0/1038080821 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f58bc1092f0 0x7f58bc1a7e10 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f58a400d770 tx=0x7f58a400dc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:16:47.543 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:47.540+0000 7f58b8ff9640 1 -- 172.21.15.82:0/1038080821 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58a4017070 con 0x7f58bc1092f0 2024-05-20T16:16:47.544 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:47.540+0000 7f58c0a4d640 1 -- 172.21.15.82:0/1038080821 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58bc110f30 con 0x7f58bc1092f0 2024-05-20T16:16:47.544 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:47.540+0000 7f58b8ff9640 1 -- 172.21.15.82:0/1038080821 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f58a4009c30 con 0x7f58bc1092f0 2024-05-20T16:16:47.545 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:47.540+0000 7f58b8ff9640 1 -- 172.21.15.82:0/1038080821 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58a4017070 con 0x7f58bc1092f0 2024-05-20T16:16:47.545 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:47.540+0000 7f58c0a4d640 1 -- 172.21.15.82:0/1038080821 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58bc111480 con 0x7f58bc1092f0 2024-05-20T16:16:47.546 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:47.547+0000 7f58c8c32740 1 Processor -- start 2024-05-20T16:16:47.551 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:47.551+0000 7f58b8ff9640 1 -- 172.21.15.82:0/1038080821 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f58a401a070 con 0x7f58bc1092f0 2024-05-20T16:16:47.552 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:47.552+0000 7f58c8c32740 1 -- start start 2024-05-20T16:16:47.552 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:47.552+0000 7f58c8c32740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562b322801b0 0x562b32620050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:16:47.553 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:47.552+0000 7f58c8c32740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562b322572f0 0x562b326205d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:16:47.553 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:47.552+0000 7f58c8c32740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562b3225f4d0 0x562b326274d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:16:47.553 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:47.552+0000 7f58c8c32740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x562b32584340 con 0x562b322801b0 2024-05-20T16:16:47.553 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:47.552+0000 7f58c8c32740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x562b3252c290 con 0x562b3225f4d0 2024-05-20T16:16:47.553 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:47.552+0000 7f58c8c32740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x562b32580300 con 0x562b322572f0 2024-05-20T16:16:47.556 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:47.556+0000 7f58b8ff9640 1 --2- 172.21.15.82:0/1038080821 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f58a80a2fc0 0x7f58a80a54d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:16:47.556 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:47.556+0000 7f58baffd640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562b3225f4d0 0x562b326274d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:16:47.556 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:47.556+0000 7f58baffd640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562b3225f4d0 0x562b326274d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.82:36154/0 (socket says 172.21.15.82:36154) 2024-05-20T16:16:47.556 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:47.556+0000 7f58baffd640 1 -- 172.21.15.82:0/612980303 learned_addr learned my addr 172.21.15.82:0/612980303 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:16:47.557 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:47.557+0000 7f58bbfff640 1 --2- 172.21.15.82:0/612980303 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562b322801b0 0x562b32620050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:16:47.557 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:47.557+0000 7f58b8ff9640 1 -- 172.21.15.82:0/1038080821 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9150+0+0 (secure 0 0 0) 0x7f58a4009a30 con 0x7f58bc1092f0 2024-05-20T16:16:47.562 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:47.562+0000 7f58baffd640 1 -- 172.21.15.82:0/612980303 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562b322572f0 msgr2=0x562b326205d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:16:47.562 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:47.562+0000 7f58baffd640 1 --2- 172.21.15.82:0/612980303 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562b322572f0 0x562b326205d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:16:47.562 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:47.562+0000 7f58baffd640 1 -- 172.21.15.82:0/612980303 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562b322801b0 msgr2=0x562b32620050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:16:47.562 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:47.562+0000 7f58baffd640 1 --2- 172.21.15.82:0/612980303 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562b322801b0 0x562b32620050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:16:47.562 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:47.562+0000 7f58baffd640 1 -- 172.21.15.82:0/612980303 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562b325508e0 con 0x562b3225f4d0 2024-05-20T16:16:47.562 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:47.562+0000 7f58bbfff640 1 --2- 172.21.15.82:0/1038080821 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f58a80a2fc0 0x7f58a80a54d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:16:47.563 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:47.563+0000 7f58baffd640 1 --2- 172.21.15.82:0/612980303 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562b3225f4d0 0x562b326274d0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f58ac009c20 tx=0x7f58ac0117b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ae6521a5a9762a6a server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:16:47.564 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:47.564+0000 7f58bbfff640 1 --2- 172.21.15.82:0/1038080821 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f58a80a2fc0 0x7f58a80a54d0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x562b32621220 tx=0x7f58b402a8f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:16:47.568 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:47.566+0000 7f58b1ffb640 1 -- 172.21.15.82:0/612980303 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58ac012450 con 0x562b3225f4d0 2024-05-20T16:16:47.568 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:47.566+0000 7f58b1ffb640 1 -- 172.21.15.82:0/612980303 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f58ac012ad0 con 0x562b3225f4d0 2024-05-20T16:16:47.568 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:47.566+0000 7f58b1ffb640 1 -- 172.21.15.82:0/612980303 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58ac00fbf0 con 0x562b3225f4d0 2024-05-20T16:16:47.569 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:47.569+0000 7f58c8c32740 1 -- 172.21.15.82:0/612980303 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562b3225f4d0 msgr2=0x562b326274d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:16:47.569 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:47.569+0000 7f58c8c32740 1 --2- 172.21.15.82:0/612980303 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562b3225f4d0 0x562b326274d0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f58ac009c20 tx=0x7f58ac0117b0 comp rx=0 tx=0).stop 2024-05-20T16:16:47.570 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:47.570+0000 7f58c8c32740 1 -- 172.21.15.82:0/612980303 shutdown_connections 2024-05-20T16:16:47.570 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:47.570+0000 7f58c8c32740 1 --2- 172.21.15.82:0/612980303 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562b3225f4d0 0x562b326274d0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:16:47.570 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:47.570+0000 7f58c8c32740 1 --2- 172.21.15.82:0/612980303 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562b322572f0 0x562b326205d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:16:47.570 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:47.570+0000 7f58c8c32740 1 --2- 172.21.15.82:0/612980303 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562b322801b0 0x562b32620050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:16:47.570 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:47.570+0000 7f58c8c32740 1 -- 172.21.15.82:0/612980303 >> 172.21.15.82:0/612980303 conn(0x562b32274300 msgr2=0x562b3261b9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:16:47.574 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:47.575+0000 7f58c8c32740 1 -- 172.21.15.82:0/612980303 shutdown_connections 2024-05-20T16:16:47.575 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:47.575+0000 7f58c8c32740 1 -- 172.21.15.82:0/612980303 wait complete. 2024-05-20T16:16:47.576 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:47.576+0000 7f58c8c32740 1 -- start start 2024-05-20T16:16:47.576 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:47.576+0000 7f58c8c32740 1 Processor -- start 2024-05-20T16:16:47.577 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:47.577+0000 7f58c8c32740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562b322572f0 0x562b32629e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:16:47.577 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:47.577+0000 7f58c8c32740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562b3225f4d0 0x562b3262a350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:16:47.577 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:47.577+0000 7f58c8c32740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562b322801b0 0x562b326312b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:16:47.577 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:47.577+0000 7f58c8c32740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x562b32627f20 con 0x562b3225f4d0 2024-05-20T16:16:47.577 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:47.577+0000 7f58c8c32740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x562b3253f790 con 0x562b322801b0 2024-05-20T16:16:47.577 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:47.577+0000 7f58c8c32740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x562b326280a0 con 0x562b322572f0 2024-05-20T16:16:47.577 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:47.577+0000 7f58c8c32740 10 client.0 _ll_register_callbacks cb 0x562b3257aa20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T16:16:47.578 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:47.577+0000 7f58baffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562b3225f4d0 0x562b3262a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:16:47.578 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:47.578+0000 7f58baffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562b3225f4d0 0x562b3262a350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43228/0 (socket says 172.21.15.82:43228) 2024-05-20T16:16:47.578 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:47.578+0000 7f58baffd640 1 -- 172.21.15.82:0/1747820109 learned_addr learned my addr 172.21.15.82:0/1747820109 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:16:47.579 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:47.578+0000 7f58bb7fe640 1 --2- 172.21.15.82:0/1747820109 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562b322572f0 0x562b32629e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:16:47.579 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:47.579+0000 7f58bbfff640 1 --2- 172.21.15.82:0/1747820109 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562b322801b0 0x562b326312b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:16:47.579 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:47.579+0000 7f58baffd640 1 -- 172.21.15.82:0/1747820109 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562b322572f0 msgr2=0x562b32629e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:16:47.579 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:47.579+0000 7f58baffd640 1 --2- 172.21.15.82:0/1747820109 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562b322572f0 0x562b32629e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:16:47.579 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:47.579+0000 7f58baffd640 1 -- 172.21.15.82:0/1747820109 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562b322801b0 msgr2=0x562b326312b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:16:47.579 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:47.579+0000 7f58baffd640 1 --2- 172.21.15.82:0/1747820109 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562b322801b0 0x562b326312b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:16:47.579 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:47.579+0000 7f58baffd640 1 -- 172.21.15.82:0/1747820109 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562b32298210 con 0x562b3225f4d0 2024-05-20T16:16:47.580 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:47.580+0000 7f58baffd640 1 --2- 172.21.15.82:0/1747820109 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562b3225f4d0 0x562b3262a350 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f58ac005820 tx=0x7f58ac009830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7d51994efb411f3 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:16:47.587 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:47.586+0000 7f589affd640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-05-20T16:16:47.587 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:47.587+0000 7f589affd640 1 -- 172.21.15.82:0/1747820109 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58ac004440 con 0x562b3225f4d0 2024-05-20T16:16:47.587 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:47.587+0000 7f589affd640 1 -- 172.21.15.82:0/1747820109 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9150+0+0 (secure 0 0 0) 0x7f58ac012450 con 0x562b3225f4d0 2024-05-20T16:16:47.587 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:47.587+0000 7f58c8c32740 1 -- 172.21.15.82:0/1747820109 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f58ac012ad0 con 0x562b3225f4d0 2024-05-20T16:16:47.597 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:47.590+0000 7f589affd640 1 -- 172.21.15.82:0/1747820109 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f58ac012650 con 0x562b3225f4d0 2024-05-20T16:16:47.597 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:47.591+0000 7f58c8c32740 10 client.15633 fetch_fsmap learned FSMap version 25 2024-05-20T16:16:47.597 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:47.591+0000 7f58c8c32740 1 -- 172.21.15.82:0/1747820109 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x562b32364620 con 0x562b3225f4d0 2024-05-20T16:16:47.597 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:47.592+0000 7f589affd640 1 -- 172.21.15.82:0/1747820109 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f58ac012ad0 con 0x562b3225f4d0 2024-05-20T16:16:47.598 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:47.594+0000 7f589affd640 1 -- 172.21.15.82:0/1747820109 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f58ac0039c0 con 0x562b3225f4d0 2024-05-20T16:16:47.598 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:47.598+0000 7f58c8c32740 10 client.15633 fetch_fsmap finished waiting for FSMap version 25 2024-05-20T16:16:47.598 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:47.598+0000 7f58c8c32740 10 client.15633 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T16:16:47.598 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:47.598+0000 7f58c8c32740 20 client.15633 populate_metadata read hostname 'smithi082' 2024-05-20T16:16:47.598 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:47.598+0000 7f58c8c32740 1 --2- 172.21.15.82:0/1747820109 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x562b3223ed80 0x562b326374d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:16:47.598 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:47.598+0000 7f58c8c32740 4 client.15633 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T16:16:47.598 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:47.598+0000 7f58c8c32740 1 -- 172.21.15.82:0/1747820109 --> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562b32591a70 con 0x562b3223ed80 2024-05-20T16:16:47.600 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:47.598+0000 7f58bb7fe640 1 --2- 172.21.15.82:0/1747820109 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x562b3223ed80 0x562b326374d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:16:47.607 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:47.606+0000 7f589affd640 10 client.15633 ms_handle_connect on v2:172.21.15.160:6838/2765554482 2024-05-20T16:16:47.607 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:47.607+0000 7f58bb7fe640 1 --2- 172.21.15.82:0/1747820109 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x562b3223ed80 0x562b326374d0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=31e1360729bfe97c server_cookie=97a7c77a3195c004 in_seq=0 out_seq=0 2024-05-20T16:16:47.610 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:47.610+0000 7f589affd640 1 -- 172.21.15.82:0/1747820109 <== mds.0 v2:172.21.15.160:6838/2765554482 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x562b32591a70 con 0x562b3223ed80 2024-05-20T16:16:47.610 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:47.610+0000 7f589affd640 10 client.15633 handle_command_reply: tid=0 2024-05-20T16:16:47.611 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:47.612+0000 7f58c8c32740 1 client.15633 shutdown 2024-05-20T16:16:47.616 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:47.616+0000 7f58c8c32740 1 -- 172.21.15.82:0/1747820109 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562b3225f4d0 msgr2=0x562b3262a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:16:47.616 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:47.616+0000 7f58c8c32740 1 --2- 172.21.15.82:0/1747820109 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562b3225f4d0 0x562b3262a350 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f58ac005820 tx=0x7f58ac009830 comp rx=0 tx=0).stop 2024-05-20T16:16:47.616 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:47.616+0000 7f58c8c32740 1 -- 172.21.15.82:0/1747820109 shutdown_connections 2024-05-20T16:16:47.616 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:47.616+0000 7f58c8c32740 1 --2- 172.21.15.82:0/1747820109 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x562b3223ed80 0x562b326374d0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:16:47.616 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:47.616+0000 7f58c8c32740 1 --2- 172.21.15.82:0/1747820109 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562b3225f4d0 0x562b3262a350 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:16:47.616 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:47.616+0000 7f58c8c32740 1 --2- 172.21.15.82:0/1747820109 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562b322801b0 0x562b326312b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:16:47.616 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:47.616+0000 7f58c8c32740 1 --2- 172.21.15.82:0/1747820109 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562b322572f0 0x562b32629e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:16:47.617 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:47.616+0000 7f589affd640 0 client.15633 ms_handle_reset on v2:172.21.15.160:6838/2765554482 2024-05-20T16:16:47.617 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:47.616+0000 7f58c8c32740 1 -- 172.21.15.82:0/1747820109 >> 172.21.15.82:0/1747820109 conn(0x562b32274300 msgr2=0x562b3261aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:16:47.617 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:47.617+0000 7f58c8c32740 1 -- 172.21.15.82:0/1747820109 shutdown_connections 2024-05-20T16:16:47.618 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:47.618+0000 7f58c8c32740 1 -- 172.21.15.82:0/1747820109 wait complete. 2024-05-20T16:16:47.618 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:47.619+0000 7f58c8c32740 20 client.15633 trim_cache size 0 max 16384 2024-05-20T16:16:47.671 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:47.671+0000 7f58c8c32740 1 Processor -- start 2024-05-20T16:16:47.671 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:47.671+0000 7f58c8c32740 1 -- start start 2024-05-20T16:16:47.672 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:47.672+0000 7f58c8c32740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562b322572f0 0x562b3261b400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:16:47.672 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:47.672+0000 7f58c8c32740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562b3225f4d0 0x562b326311d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:16:47.672 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:47.672+0000 7f58c8c32740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562b322801b0 0x562b3261b940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:16:47.672 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:47.672+0000 7f58baffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562b3225f4d0 0x562b326311d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:16:47.673 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:47.672+0000 7f58baffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562b3225f4d0 0x562b326311d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43234/0 (socket says 172.21.15.82:43234) 2024-05-20T16:16:47.673 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:47.672+0000 7f58baffd640 1 -- 172.21.15.82:0/2289799892 learned_addr learned my addr 172.21.15.82:0/2289799892 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:16:47.673 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:47.672+0000 7f58c8c32740 1 -- 172.21.15.82:0/2289799892 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f58bc1c5850 con 0x562b3225f4d0 2024-05-20T16:16:47.673 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:47.672+0000 7f58c8c32740 1 -- 172.21.15.82:0/2289799892 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f58bc1c3120 con 0x562b322801b0 2024-05-20T16:16:47.673 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:47.672+0000 7f58c8c32740 1 -- 172.21.15.82:0/2289799892 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f58bc119150 con 0x562b322572f0 2024-05-20T16:16:47.677 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:47.673+0000 7f58bb7fe640 1 --2- 172.21.15.82:0/2289799892 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562b322572f0 0x562b3261b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:16:47.677 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:47.673+0000 7f58bbfff640 1 --2- 172.21.15.82:0/2289799892 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562b322801b0 0x562b3261b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:16:47.678 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:47.678+0000 7f58baffd640 1 -- 172.21.15.82:0/2289799892 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562b322572f0 msgr2=0x562b3261b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:16:47.678 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:47.678+0000 7f58baffd640 1 --2- 172.21.15.82:0/2289799892 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562b322572f0 0x562b3261b400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:16:47.678 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:47.678+0000 7f58baffd640 1 -- 172.21.15.82:0/2289799892 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562b322801b0 msgr2=0x562b3261b940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:16:47.679 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:47.678+0000 7f58baffd640 1 --2- 172.21.15.82:0/2289799892 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562b322801b0 0x562b3261b940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:16:47.679 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:47.678+0000 7f58baffd640 1 -- 172.21.15.82:0/2289799892 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562b32621250 con 0x562b3225f4d0 2024-05-20T16:16:47.679 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:47.678+0000 7f58bb7fe640 1 --2- 172.21.15.82:0/2289799892 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562b322572f0 0x562b3261b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T16:16:47.681 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:47.681+0000 7f58baffd640 1 --2- 172.21.15.82:0/2289799892 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562b3225f4d0 0x562b326311d0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f58ac003f30 tx=0x7f58ac023000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f72750c80fb78177 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:16:47.682 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:47.682+0000 7f58b17fa640 1 -- 172.21.15.82:0/2289799892 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58ac009670 con 0x562b3225f4d0 2024-05-20T16:16:47.682 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:47.682+0000 7f58b17fa640 1 -- 172.21.15.82:0/2289799892 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f58ac029070 con 0x562b3225f4d0 2024-05-20T16:16:47.684 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:47.684+0000 7f58c8c32740 1 -- 172.21.15.82:0/2289799892 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562b3225f4d0 msgr2=0x562b326311d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:16:47.684 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:47.684+0000 7f58c8c32740 1 --2- 172.21.15.82:0/2289799892 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562b3225f4d0 0x562b326311d0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f58ac003f30 tx=0x7f58ac023000 comp rx=0 tx=0).stop 2024-05-20T16:16:47.688 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:47.688+0000 7f58c8c32740 1 -- 172.21.15.82:0/2289799892 shutdown_connections 2024-05-20T16:16:47.688 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:47.688+0000 7f58c8c32740 1 --2- 172.21.15.82:0/2289799892 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562b3225f4d0 0x562b326311d0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:16:47.688 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:47.688+0000 7f58c8c32740 1 --2- 172.21.15.82:0/2289799892 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562b322801b0 0x562b3261b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:16:47.688 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:47.688+0000 7f58c8c32740 1 --2- 172.21.15.82:0/2289799892 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562b322572f0 0x562b3261b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:16:47.688 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:47.688+0000 7f58c8c32740 1 -- 172.21.15.82:0/2289799892 >> 172.21.15.82:0/2289799892 conn(0x562b32274300 msgr2=0x562b32624270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:16:47.688 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:47.688+0000 7f58c8c32740 1 -- 172.21.15.82:0/2289799892 shutdown_connections 2024-05-20T16:16:47.688 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:47.688+0000 7f58c8c32740 1 -- 172.21.15.82:0/2289799892 wait complete. 2024-05-20T16:16:47.689 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:47.689+0000 7f58c8c32740 1 -- start start 2024-05-20T16:16:47.689 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:47.689+0000 7f58c8c32740 1 Processor -- start 2024-05-20T16:16:47.690 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:47.690+0000 7f58c8c32740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562b322572f0 0x562b3262e370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:16:47.690 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:47.690+0000 7f58c8c32740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562b3225f4d0 0x562b3262e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:16:47.690 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:47.690+0000 7f58c8c32740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562b322801b0 0x562b3262f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:16:47.690 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:47.690+0000 7f58c8c32740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f58bc1c15e0 con 0x562b3225f4d0 2024-05-20T16:16:47.690 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:47.690+0000 7f58c8c32740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f58bc1c2a50 con 0x562b322572f0 2024-05-20T16:16:47.690 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:47.690+0000 7f58c8c32740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f58bc1c1cb0 con 0x562b322801b0 2024-05-20T16:16:47.690 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:47.690+0000 7f58c8c32740 10 client.0 _ll_register_callbacks cb 0x7f58ac009aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T16:16:47.691 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:47.691+0000 7f58baffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562b3225f4d0 0x562b3262e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:16:47.691 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:47.691+0000 7f58baffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562b3225f4d0 0x562b3262e8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43238/0 (socket says 172.21.15.82:43238) 2024-05-20T16:16:47.691 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:47.691+0000 7f58baffd640 1 -- 172.21.15.82:0/1290120270 learned_addr learned my addr 172.21.15.82:0/1290120270 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:16:47.691 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:47.692+0000 7f58baffd640 1 -- 172.21.15.82:0/1290120270 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562b322801b0 msgr2=0x562b3262f7e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-20T16:16:47.692 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:47.692+0000 7f58baffd640 1 --2- 172.21.15.82:0/1290120270 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562b322801b0 0x562b3262f7e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:16:47.692 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:47.692+0000 7f58baffd640 1 -- 172.21.15.82:0/1290120270 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562b322572f0 msgr2=0x562b3262e370 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-20T16:16:47.692 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:47.692+0000 7f58baffd640 1 --2- 172.21.15.82:0/1290120270 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562b322572f0 0x562b3262e370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:16:47.692 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:47.692+0000 7f58baffd640 1 -- 172.21.15.82:0/1290120270 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562b3262fda0 con 0x562b3225f4d0 2024-05-20T16:16:47.693 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:47.692+0000 7f58baffd640 1 --2- 172.21.15.82:0/1290120270 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562b3225f4d0 0x562b3262e8b0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f58ac024530 tx=0x7f58ac0098a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e674f57af770a515 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:16:47.693 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:47.693+0000 7f589affd640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-05-20T16:16:47.693 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:47.693+0000 7f589affd640 1 -- 172.21.15.82:0/1290120270 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58ac0054d0 con 0x562b3225f4d0 2024-05-20T16:16:47.693 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:47.693+0000 7f58c8c32740 1 -- 172.21.15.82:0/1290120270 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f58ac029070 con 0x562b3225f4d0 2024-05-20T16:16:47.693 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:47.693+0000 7f589affd640 1 -- 172.21.15.82:0/1290120270 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9150+0+0 (secure 0 0 0) 0x7f58ac0035f0 con 0x562b3225f4d0 2024-05-20T16:16:47.694 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:47.694+0000 7f589affd640 1 -- 172.21.15.82:0/1290120270 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f58ac0037f0 con 0x562b3225f4d0 2024-05-20T16:16:47.694 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:47.694+0000 7f58c8c32740 10 client.15639 fetch_fsmap learned FSMap version 25 2024-05-20T16:16:47.694 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:47.694+0000 7f58c8c32740 1 -- 172.21.15.82:0/1290120270 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x562b32631030 con 0x562b3225f4d0 2024-05-20T16:16:47.695 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:47.695+0000 7f589affd640 1 -- 172.21.15.82:0/1290120270 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f58ac029070 con 0x562b3225f4d0 2024-05-20T16:16:47.696 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:47.697+0000 7f589affd640 1 -- 172.21.15.82:0/1290120270 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f58ac036040 con 0x562b3225f4d0 2024-05-20T16:16:47.697 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:47.697+0000 7f58c8c32740 10 client.15639 fetch_fsmap finished waiting for FSMap version 25 2024-05-20T16:16:47.697 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:47.697+0000 7f58c8c32740 10 client.15639 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T16:16:47.697 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:47.697+0000 7f58c8c32740 20 client.15639 populate_metadata read hostname 'smithi082' 2024-05-20T16:16:47.697 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:47.697+0000 7f58c8c32740 1 --2- 172.21.15.82:0/1290120270 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x562b322141c0 0x562b326a1a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:16:47.697 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:47.697+0000 7f58c8c32740 4 client.15639 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-20T16:16:47.697 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:47.697+0000 7f58c8c32740 1 -- 172.21.15.82:0/1290120270 --> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562b32623d80 con 0x562b322141c0 2024-05-20T16:16:47.698 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:47.698+0000 7f58bb7fe640 1 --2- 172.21.15.82:0/1290120270 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x562b322141c0 0x562b326a1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:16:47.710 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:47.710+0000 7f58bb7fe640 1 --2- 172.21.15.82:0/1290120270 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x562b322141c0 0x562b326a1a60 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dd9ca89db9950b7b server_cookie=aab86998b75a4be3 in_seq=0 out_seq=0 2024-05-20T16:16:47.710 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:47.711+0000 7f589affd640 10 client.15639 ms_handle_connect on v2:172.21.15.160:6838/2765554482 2024-05-20T16:16:47.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:16:47 smithi160 ceph-mon[36347]: pgmap v446: 129 pgs: 129 active+clean; 10 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 758 MiB/s wr, 972 op/s 2024-05-20T16:16:47.883 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:16:47 smithi082 ceph-mon[32238]: pgmap v446: 129 pgs: 129 active+clean; 10 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 758 MiB/s wr, 972 op/s 2024-05-20T16:16:47.885 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:16:47 smithi184 ceph-mon[36209]: pgmap v446: 129 pgs: 129 active+clean; 10 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 758 MiB/s wr, 972 op/s 2024-05-20T16:16:49.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:16:49 smithi160 ceph-mon[36347]: pgmap v447: 129 pgs: 129 active+clean; 10 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 895 MiB/s wr, 1.36k op/s 2024-05-20T16:16:49.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:16:49 smithi082 ceph-mon[32238]: pgmap v447: 129 pgs: 129 active+clean; 10 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 895 MiB/s wr, 1.36k op/s 2024-05-20T16:16:49.885 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:16:49 smithi184 ceph-mon[36209]: pgmap v447: 129 pgs: 129 active+clean; 10 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 895 MiB/s wr, 1.36k op/s 2024-05-20T16:16:51.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:16:51 smithi160 ceph-mon[36347]: pgmap v448: 129 pgs: 129 active+clean; 10 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 819 MiB/s wr, 1.39k op/s 2024-05-20T16:16:51.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:16:51 smithi082 ceph-mon[32238]: pgmap v448: 129 pgs: 129 active+clean; 10 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 819 MiB/s wr, 1.39k op/s 2024-05-20T16:16:51.885 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:16:51 smithi184 ceph-mon[36209]: pgmap v448: 129 pgs: 129 active+clean; 10 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 819 MiB/s wr, 1.39k op/s 2024-05-20T16:16:53.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:16:53 smithi160 ceph-mon[36347]: pgmap v449: 129 pgs: 129 active+clean; 10 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 741 MiB/s wr, 1.29k op/s 2024-05-20T16:16:53.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:16:53 smithi082 ceph-mon[32238]: pgmap v449: 129 pgs: 129 active+clean; 10 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 741 MiB/s wr, 1.29k op/s 2024-05-20T16:16:53.885 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:16:53 smithi184 ceph-mon[36209]: pgmap v449: 129 pgs: 129 active+clean; 10 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 741 MiB/s wr, 1.29k op/s 2024-05-20T16:16:54.703 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:54.704+0000 7f589affd640 1 -- 172.21.15.82:0/1290120270 <== mds.0 v2:172.21.15.160:6838/2765554482 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x562b32623d80 con 0x562b322141c0 2024-05-20T16:16:54.704 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:54.704+0000 7f589affd640 10 client.15639 handle_command_reply: tid=0 2024-05-20T16:16:54.704 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:54.704+0000 7f58c8c32740 1 client.15639 shutdown 2024-05-20T16:16:54.705 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:54.705+0000 7f58c8c32740 1 -- 172.21.15.82:0/1290120270 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562b3225f4d0 msgr2=0x562b3262e8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:16:54.705 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:54.705+0000 7f58c8c32740 1 --2- 172.21.15.82:0/1290120270 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562b3225f4d0 0x562b3262e8b0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f58ac024530 tx=0x7f58ac0098a0 comp rx=0 tx=0).stop 2024-05-20T16:16:54.705 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:54.705+0000 7f58c8c32740 1 -- 172.21.15.82:0/1290120270 shutdown_connections 2024-05-20T16:16:54.705 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:54.705+0000 7f58c8c32740 1 --2- 172.21.15.82:0/1290120270 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x562b322141c0 0x562b326a1a60 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:16:54.705 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:54.705+0000 7f58c8c32740 1 --2- 172.21.15.82:0/1290120270 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562b3225f4d0 0x562b3262e8b0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:16:54.705 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:54.705+0000 7f58c8c32740 1 --2- 172.21.15.82:0/1290120270 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562b322801b0 0x562b3262f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:16:54.705 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:54.705+0000 7f58c8c32740 1 --2- 172.21.15.82:0/1290120270 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562b322572f0 0x562b3262e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:16:54.705 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:54.705+0000 7f589affd640 0 client.15639 ms_handle_reset on v2:172.21.15.160:6838/2765554482 2024-05-20T16:16:54.705 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:54.705+0000 7f58c8c32740 1 -- 172.21.15.82:0/1290120270 >> 172.21.15.82:0/1290120270 conn(0x562b32274300 msgr2=0x562b3261b400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:16:54.706 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:54.706+0000 7f58c8c32740 1 -- 172.21.15.82:0/1290120270 shutdown_connections 2024-05-20T16:16:54.706 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:54.706+0000 7f58c8c32740 1 -- 172.21.15.82:0/1290120270 wait complete. 2024-05-20T16:16:54.706 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:54.706+0000 7f58c8c32740 20 client.15639 trim_cache size 0 max 16384 2024-05-20T16:16:54.706 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:{ 2024-05-20T16:16:54.706 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "message": "", 2024-05-20T16:16:54.706 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "return_code": 0 2024-05-20T16:16:54.707 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:} 2024-05-20T16:16:54.708 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:54.709+0000 7f58c0a4d640 1 -- 172.21.15.82:0/1038080821 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f58a80a2fc0 msgr2=0x7f58a80a54d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:16:54.709 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:54.709+0000 7f58c0a4d640 1 --2- 172.21.15.82:0/1038080821 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f58a80a2fc0 0x7f58a80a54d0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x562b32621220 tx=0x7f58b402a8f0 comp rx=0 tx=0).stop 2024-05-20T16:16:54.709 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:54.709+0000 7f58c0a4d640 1 -- 172.21.15.82:0/1038080821 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f58bc1092f0 msgr2=0x7f58bc1a7e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:16:54.709 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:54.709+0000 7f58c0a4d640 1 --2- 172.21.15.82:0/1038080821 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f58bc1092f0 0x7f58bc1a7e10 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f58a400d770 tx=0x7f58a400dc30 comp rx=0 tx=0).stop 2024-05-20T16:16:54.709 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:54.709+0000 7f58c0a4d640 1 -- 172.21.15.82:0/1038080821 shutdown_connections 2024-05-20T16:16:54.709 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:54.709+0000 7f58c0a4d640 1 --2- 172.21.15.82:0/1038080821 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f58a80a2fc0 0x7f58a80a54d0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:16:54.709 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:54.709+0000 7f58c0a4d640 1 --2- 172.21.15.82:0/1038080821 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f58bc10ee60 0x7f58bc1af2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:16:54.709 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:54.709+0000 7f58c0a4d640 1 --2- 172.21.15.82:0/1038080821 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f58bc10a140 0x7f58bc1a8350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:16:54.710 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:54.709+0000 7f58c0a4d640 1 --2- 172.21.15.82:0/1038080821 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f58bc1092f0 0x7f58bc1a7e10 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:16:54.710 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:54.709+0000 7f58c0a4d640 1 -- 172.21.15.82:0/1038080821 >> 172.21.15.82:0/1038080821 conn(0x7f58bc06f2e0 msgr2=0x7f58bc071770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:16:54.710 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:54.709+0000 7f58c0a4d640 1 -- 172.21.15.82:0/1038080821 shutdown_connections 2024-05-20T16:16:54.710 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:16:54.710+0000 7f58c0a4d640 1 -- 172.21.15.82:0/1038080821 wait complete. 2024-05-20T16:16:55.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:16:55 smithi160 ceph-mon[36347]: pgmap v450: 129 pgs: 129 active+clean; 10 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 850 MiB/s wr, 1.42k op/s 2024-05-20T16:16:55.860 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:16:55 smithi082 ceph-mon[32238]: pgmap v450: 129 pgs: 129 active+clean; 10 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 850 MiB/s wr, 1.42k op/s 2024-05-20T16:16:55.885 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:16:55 smithi184 ceph-mon[36209]: pgmap v450: 129 pgs: 129 active+clean; 10 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 850 MiB/s wr, 1.42k op/s 2024-05-20T16:16:57.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:16:57 smithi160 ceph-mon[36347]: pgmap v451: 129 pgs: 129 active+clean; 10 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 805 MiB/s wr, 1.40k op/s 2024-05-20T16:16:57.860 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:16:57 smithi082 ceph-mon[32238]: pgmap v451: 129 pgs: 129 active+clean; 10 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 805 MiB/s wr, 1.40k op/s 2024-05-20T16:16:57.885 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:16:57 smithi184 ceph-mon[36209]: pgmap v451: 129 pgs: 129 active+clean; 10 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 805 MiB/s wr, 1.40k op/s 2024-05-20T16:16:59.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:16:59 smithi160 ceph-mon[36347]: pgmap v452: 129 pgs: 129 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 909 MiB/s wr, 1.56k op/s 2024-05-20T16:16:59.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:16:59 smithi082 ceph-mon[32238]: pgmap v452: 129 pgs: 129 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 909 MiB/s wr, 1.56k op/s 2024-05-20T16:16:59.885 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:16:59 smithi184 ceph-mon[36209]: pgmap v452: 129 pgs: 129 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 909 MiB/s wr, 1.56k op/s 2024-05-20T16:17:01.746 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:17:01 smithi082 ceph-mon[32238]: pgmap v453: 129 pgs: 129 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 837 MiB/s wr, 1.26k op/s 2024-05-20T16:17:01.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:17:01 smithi160 ceph-mon[36347]: pgmap v453: 129 pgs: 129 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 837 MiB/s wr, 1.26k op/s 2024-05-20T16:17:01.885 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:17:01 smithi184 ceph-mon[36209]: pgmap v453: 129 pgs: 129 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 837 MiB/s wr, 1.26k op/s 2024-05-20T16:17:03.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:17:03 smithi160 ceph-mon[36347]: pgmap v454: 129 pgs: 129 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 812 MiB/s wr, 1.16k op/s 2024-05-20T16:17:03.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:17:03 smithi082 ceph-mon[32238]: pgmap v454: 129 pgs: 129 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 812 MiB/s wr, 1.16k op/s 2024-05-20T16:17:03.885 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:17:03 smithi184 ceph-mon[36209]: pgmap v454: 129 pgs: 129 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 812 MiB/s wr, 1.16k op/s 2024-05-20T16:17:05.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:17:05 smithi160 ceph-mon[36347]: pgmap v455: 129 pgs: 129 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 940 MiB/s wr, 1.35k op/s 2024-05-20T16:17:05.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:17:05 smithi082 ceph-mon[32238]: pgmap v455: 129 pgs: 129 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 940 MiB/s wr, 1.35k op/s 2024-05-20T16:17:05.885 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:17:05 smithi184 ceph-mon[36209]: pgmap v455: 129 pgs: 129 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 940 MiB/s wr, 1.35k op/s 2024-05-20T16:17:07.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:17:07 smithi160 ceph-mon[36347]: pgmap v456: 129 pgs: 129 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 685 KiB/s rd, 837 MiB/s wr, 1.25k op/s 2024-05-20T16:17:07.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:17:07 smithi082 ceph-mon[32238]: pgmap v456: 129 pgs: 129 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 685 KiB/s rd, 837 MiB/s wr, 1.25k op/s 2024-05-20T16:17:07.885 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:17:07 smithi184 ceph-mon[36209]: pgmap v456: 129 pgs: 129 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 685 KiB/s rd, 837 MiB/s wr, 1.25k op/s 2024-05-20T16:17:07.990 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:07.990+0000 7ff560d7e640 1 -- 172.21.15.82:0/591362723 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff55c10f6d0 msgr2=0x7ff55c111af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:17:07.990 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:07.990+0000 7ff560d7e640 1 --2- 172.21.15.82:0/591362723 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff55c10f6d0 0x7ff55c111af0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7ff554009ca0 tx=0x7ff55402a8c0 comp rx=0 tx=0).stop 2024-05-20T16:17:07.991 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:07.991+0000 7ff560d7e640 1 -- 172.21.15.82:0/591362723 shutdown_connections 2024-05-20T16:17:07.991 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:07.991+0000 7ff560d7e640 1 --2- 172.21.15.82:0/591362723 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff55c10f6d0 0x7ff55c111af0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:17:07.991 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:07.991+0000 7ff560d7e640 1 --2- 172.21.15.82:0/591362723 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff55c10a9d0 0x7ff55c10ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:17:07.991 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:07.991+0000 7ff560d7e640 1 --2- 172.21.15.82:0/591362723 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff55c109bd0 0x7ff55c10a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:17:07.991 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:07.991+0000 7ff560d7e640 1 -- 172.21.15.82:0/591362723 >> 172.21.15.82:0/591362723 conn(0x7ff55c06f2e0 msgr2=0x7ff55c071790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:17:07.991 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:07.992+0000 7ff560d7e640 1 -- 172.21.15.82:0/591362723 shutdown_connections 2024-05-20T16:17:07.992 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:07.992+0000 7ff560d7e640 1 -- 172.21.15.82:0/591362723 wait complete. 2024-05-20T16:17:07.993 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:07.993+0000 7ff560d7e640 1 Processor -- start 2024-05-20T16:17:07.994 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:07.994+0000 7ff560d7e640 1 -- start start 2024-05-20T16:17:07.994 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:07.994+0000 7ff560d7e640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff55c109bd0 0x7ff55c11aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:17:07.994 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:07.994+0000 7ff560d7e640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff55c10a9d0 0x7ff55c11af70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:17:07.994 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:07.994+0000 7ff560d7e640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff55c10f6d0 0x7ff55c11de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:17:07.995 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:07.994+0000 7ff560d7e640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7ff55c1142c0 con 0x7ff55c10f6d0 2024-05-20T16:17:07.995 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:07.994+0000 7ff560d7e640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7ff55c114140 con 0x7ff55c10a9d0 2024-05-20T16:17:07.995 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:07.994+0000 7ff560d7e640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ff55c114440 con 0x7ff55c109bd0 2024-05-20T16:17:07.995 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:07.995+0000 7ff55affd640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff55c10a9d0 0x7ff55c11af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:17:07.995 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:07.995+0000 7ff55affd640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff55c10a9d0 0x7ff55c11af70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.82:57318/0 (socket says 172.21.15.82:57318) 2024-05-20T16:17:07.995 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:07.995+0000 7ff55affd640 1 -- 172.21.15.82:0/1239982144 learned_addr learned my addr 172.21.15.82:0/1239982144 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:17:07.995 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:07.995+0000 7ff55bfff640 1 --2- 172.21.15.82:0/1239982144 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff55c10f6d0 0x7ff55c11de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:17:07.996 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:07.995+0000 7ff55affd640 1 -- 172.21.15.82:0/1239982144 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff55c109bd0 msgr2=0x7ff55c11aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:17:07.996 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:07.995+0000 7ff55affd640 1 --2- 172.21.15.82:0/1239982144 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff55c109bd0 0x7ff55c11aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:17:07.996 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:07.995+0000 7ff55affd640 1 -- 172.21.15.82:0/1239982144 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff55c10f6d0 msgr2=0x7ff55c11de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:17:07.996 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:07.995+0000 7ff55affd640 1 --2- 172.21.15.82:0/1239982144 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff55c10f6d0 0x7ff55c11de90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:17:07.996 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:07.995+0000 7ff55affd640 1 -- 172.21.15.82:0/1239982144 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff55c11e500 con 0x7ff55c10a9d0 2024-05-20T16:17:07.996 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:07.996+0000 7ff55affd640 1 --2- 172.21.15.82:0/1239982144 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff55c10a9d0 0x7ff55c11af70 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7ff5500097d0 tx=0x7ff550009c90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:17:07.996 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:07.996+0000 7ff558ff9640 1 -- 172.21.15.82:0/1239982144 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff550005020 con 0x7ff55c10a9d0 2024-05-20T16:17:07.999 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:07.997+0000 7ff560d7e640 1 -- 172.21.15.82:0/1239982144 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff55c11e7f0 con 0x7ff55c10a9d0 2024-05-20T16:17:07.999 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:07.997+0000 7ff560d7e640 1 -- 172.21.15.82:0/1239982144 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff55c11ed40 con 0x7ff55c10a9d0 2024-05-20T16:17:08.001 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:08.001+0000 7ff568b22740 1 Processor -- start 2024-05-20T16:17:08.001 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:08.001+0000 7ff558ff9640 1 -- 172.21.15.82:0/1239982144 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff5500056a0 con 0x7ff55c10a9d0 2024-05-20T16:17:08.003 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:08.004+0000 7ff568b22740 1 -- start start 2024-05-20T16:17:08.005 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:08.005+0000 7ff568b22740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5615af93e1b0 0x5615afcde050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:17:08.005 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:08.005+0000 7ff55bfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5615af93e1b0 0x5615afcde050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:17:08.005 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:08.005+0000 7ff55bfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5615af93e1b0 0x5615afcde050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:49614/0 (socket says 172.21.15.82:49614) 2024-05-20T16:17:08.005 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:08.005+0000 7ff55bfff640 1 -- 172.21.15.82:0/2336169829 learned_addr learned my addr 172.21.15.82:0/2336169829 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:17:08.005 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:08.006+0000 7ff568b22740 1 --2- 172.21.15.82:0/2336169829 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5615af9152f0 0x5615afcde5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:17:08.006 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:08.006+0000 7ff568b22740 1 --2- 172.21.15.82:0/2336169829 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5615af91d4d0 0x5615afce54d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:17:08.006 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:08.006+0000 7ff568b22740 1 -- 172.21.15.82:0/2336169829 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x5615afc42340 con 0x5615af93e1b0 2024-05-20T16:17:08.006 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:08.006+0000 7ff568b22740 1 -- 172.21.15.82:0/2336169829 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x5615afbea290 con 0x5615af91d4d0 2024-05-20T16:17:08.006 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:08.006+0000 7ff55affd640 1 --2- 172.21.15.82:0/2336169829 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5615af91d4d0 0x5615afce54d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:17:08.007 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:08.006+0000 7ff568b22740 1 -- 172.21.15.82:0/2336169829 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5615afc3e300 con 0x5615af9152f0 2024-05-20T16:17:08.008 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:08.008+0000 7ff558ff9640 1 -- 172.21.15.82:0/1239982144 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff550004b10 con 0x7ff55c10a9d0 2024-05-20T16:17:08.008 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:08.008+0000 7ff55b7fe640 1 --2- 172.21.15.82:0/2336169829 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5615af9152f0 0x5615afcde5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:17:08.008 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:08.008+0000 7ff558ff9640 1 -- 172.21.15.82:0/1239982144 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7ff55001b020 con 0x7ff55c10a9d0 2024-05-20T16:17:08.009 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:08.009+0000 7ff55b7fe640 1 -- 172.21.15.82:0/2336169829 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5615af91d4d0 msgr2=0x5615afce54d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:17:08.009 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:08.009+0000 7ff55b7fe640 1 --2- 172.21.15.82:0/2336169829 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5615af91d4d0 0x5615afce54d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:17:08.009 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:08.009+0000 7ff55b7fe640 1 -- 172.21.15.82:0/2336169829 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5615af93e1b0 msgr2=0x5615afcde050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:17:08.009 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:08.009+0000 7ff55b7fe640 1 --2- 172.21.15.82:0/2336169829 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5615af93e1b0 0x5615afcde050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:17:08.009 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:08.009+0000 7ff55b7fe640 1 -- 172.21.15.82:0/2336169829 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5615afc0e8e0 con 0x5615af9152f0 2024-05-20T16:17:08.009 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:08.009+0000 7ff55b7fe640 1 --2- 172.21.15.82:0/2336169829 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5615af9152f0 0x5615afcde5d0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7ff55c073650 tx=0x7ff54c00af80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e3a42167ca8d5a0e server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:17:08.010 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:08.010+0000 7ff541ffb640 1 -- 172.21.15.82:0/2336169829 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff54c00ed80 con 0x5615af9152f0 2024-05-20T16:17:08.011 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:08.010+0000 7ff541ffb640 1 -- 172.21.15.82:0/2336169829 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff54c005020 con 0x5615af9152f0 2024-05-20T16:17:08.011 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:08.010+0000 7ff541ffb640 1 -- 172.21.15.82:0/2336169829 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff54c004c10 con 0x5615af9152f0 2024-05-20T16:17:08.011 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:08.010+0000 7ff558ff9640 1 --2- 172.21.15.82:0/1239982144 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7ff5440a2bc0 0x7ff5440a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:17:08.011 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:08.011+0000 7ff558ff9640 1 -- 172.21.15.82:0/1239982144 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9150+0+0 (secure 0 0 0) 0x7ff5500cfcf0 con 0x7ff55c10a9d0 2024-05-20T16:17:08.013 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:08.013+0000 7ff568b22740 1 -- 172.21.15.82:0/2336169829 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5615af9152f0 msgr2=0x5615afcde5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:17:08.013 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:08.013+0000 7ff568b22740 1 --2- 172.21.15.82:0/2336169829 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5615af9152f0 0x5615afcde5d0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7ff55c073650 tx=0x7ff54c00af80 comp rx=0 tx=0).stop 2024-05-20T16:17:08.014 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:08.013+0000 7ff55bfff640 1 --2- 172.21.15.82:0/1239982144 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7ff5440a2bc0 0x7ff5440a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:17:08.014 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:08.014+0000 7ff568b22740 1 -- 172.21.15.82:0/2336169829 shutdown_connections 2024-05-20T16:17:08.014 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:08.014+0000 7ff568b22740 1 --2- 172.21.15.82:0/2336169829 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5615af91d4d0 0x5615afce54d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:17:08.014 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:08.014+0000 7ff568b22740 1 --2- 172.21.15.82:0/2336169829 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5615af9152f0 0x5615afcde5d0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:17:08.014 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:08.014+0000 7ff568b22740 1 --2- 172.21.15.82:0/2336169829 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5615af93e1b0 0x5615afcde050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:17:08.014 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:08.014+0000 7ff568b22740 1 -- 172.21.15.82:0/2336169829 >> 172.21.15.82:0/2336169829 conn(0x5615af932300 msgr2=0x5615afcd99e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:17:08.016 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:08.015+0000 7ff568b22740 1 -- 172.21.15.82:0/2336169829 shutdown_connections 2024-05-20T16:17:08.018 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:08.018+0000 7ff568b22740 1 -- 172.21.15.82:0/2336169829 wait complete. 2024-05-20T16:17:08.018 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:08.018+0000 7ff568b22740 1 -- start start 2024-05-20T16:17:08.018 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:08.019+0000 7ff55bfff640 1 --2- 172.21.15.82:0/1239982144 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7ff5440a2bc0 0x7ff5440a50d0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x5615afcdf220 tx=0x7ff5540093e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:17:08.019 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:08.019+0000 7ff568b22740 1 Processor -- start 2024-05-20T16:17:08.019 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:08.019+0000 7ff568b22740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5615af9152f0 0x5615afce7e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:17:08.019 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:08.019+0000 7ff568b22740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5615af91d4d0 0x5615afce8360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:17:08.019 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:08.019+0000 7ff568b22740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5615af93e1b0 0x5615afcef2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:17:08.020 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:08.019+0000 7ff568b22740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x5615afce5f20 con 0x5615af9152f0 2024-05-20T16:17:08.020 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:08.019+0000 7ff568b22740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x5615afbfd790 con 0x5615af93e1b0 2024-05-20T16:17:08.020 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:08.019+0000 7ff568b22740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5615afce60a0 con 0x5615af91d4d0 2024-05-20T16:17:08.020 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:08.019+0000 7ff568b22740 10 client.0 _ll_register_callbacks cb 0x5615afc38a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T16:17:08.021 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:08.020+0000 7ff55b7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5615af9152f0 0x5615afce7e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:17:08.021 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:08.020+0000 7ff55b7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5615af9152f0 0x5615afce7e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:49628/0 (socket says 172.21.15.82:49628) 2024-05-20T16:17:08.021 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:08.020+0000 7ff55b7fe640 1 -- 172.21.15.82:0/4028749838 learned_addr learned my addr 172.21.15.82:0/4028749838 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:17:08.021 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:08.021+0000 7ff55affd640 1 --2- 172.21.15.82:0/4028749838 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5615af91d4d0 0x5615afce8360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:17:08.021 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:08.021+0000 7ff55b7fe640 1 -- 172.21.15.82:0/4028749838 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5615af91d4d0 msgr2=0x5615afce8360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:17:08.021 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:08.021+0000 7ff55b7fe640 1 --2- 172.21.15.82:0/4028749838 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5615af91d4d0 0x5615afce8360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:17:08.021 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:08.021+0000 7ff55b7fe640 1 -- 172.21.15.82:0/4028749838 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5615af93e1b0 msgr2=0x5615afcef2c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-20T16:17:08.021 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:08.021+0000 7ff55b7fe640 1 --2- 172.21.15.82:0/4028749838 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5615af93e1b0 0x5615afcef2c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:17:08.021 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:08.021+0000 7ff55b7fe640 1 -- 172.21.15.82:0/4028749838 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5615af956210 con 0x5615af9152f0 2024-05-20T16:17:08.021 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:08.021+0000 7ff55b7fe640 1 --2- 172.21.15.82:0/4028749838 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5615af9152f0 0x5615afce7e20 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7ff54c005e00 tx=0x7ff54c005400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3170740ad5e5ef7 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:17:08.021 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:08.021+0000 7ff53affd640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-05-20T16:17:08.024 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:08.024+0000 7ff53affd640 1 -- 172.21.15.82:0/4028749838 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff54c004910 con 0x5615af9152f0 2024-05-20T16:17:08.024 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:08.025+0000 7ff568b22740 1 -- 172.21.15.82:0/4028749838 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff54c005020 con 0x5615af9152f0 2024-05-20T16:17:08.024 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:08.025+0000 7ff53affd640 1 -- 172.21.15.82:0/4028749838 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9150+0+0 (secure 0 0 0) 0x7ff54c004ab0 con 0x5615af9152f0 2024-05-20T16:17:08.025 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:08.025+0000 7ff53affd640 1 -- 172.21.15.82:0/4028749838 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ff54c02e050 con 0x5615af9152f0 2024-05-20T16:17:08.028 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:08.028+0000 7ff568b22740 10 client.15651 fetch_fsmap learned FSMap version 25 2024-05-20T16:17:08.028 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:08.028+0000 7ff568b22740 1 -- 172.21.15.82:0/4028749838 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5615afa22620 con 0x5615af9152f0 2024-05-20T16:17:08.033 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:08.033+0000 7ff53affd640 1 -- 172.21.15.82:0/4028749838 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff54c005020 con 0x5615af9152f0 2024-05-20T16:17:08.033 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:08.033+0000 7ff53affd640 1 -- 172.21.15.82:0/4028749838 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7ff54c00b040 con 0x5615af9152f0 2024-05-20T16:17:08.033 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:08.033+0000 7ff568b22740 10 client.15651 fetch_fsmap finished waiting for FSMap version 25 2024-05-20T16:17:08.033 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:08.033+0000 7ff568b22740 10 client.15651 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T16:17:08.033 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:08.033+0000 7ff568b22740 20 client.15651 populate_metadata read hostname 'smithi082' 2024-05-20T16:17:08.033 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:08.033+0000 7ff568b22740 1 --2- 172.21.15.82:0/4028749838 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x5615af8fcd80 0x5615afcf54f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:17:08.034 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:08.033+0000 7ff568b22740 4 client.15651 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T16:17:08.034 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:08.033+0000 7ff568b22740 1 -- 172.21.15.82:0/4028749838 --> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5615afc4fa70 con 0x5615af8fcd80 2024-05-20T16:17:08.035 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:08.036+0000 7ff55affd640 1 --2- 172.21.15.82:0/4028749838 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x5615af8fcd80 0x5615afcf54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:17:08.037 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:08.037+0000 7ff55affd640 1 --2- 172.21.15.82:0/4028749838 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x5615af8fcd80 0x5615afcf54f0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5e2a9112008b8c64 server_cookie=cd25750877ba90e0 in_seq=0 out_seq=0 2024-05-20T16:17:08.039 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:08.039+0000 7ff53affd640 10 client.15651 ms_handle_connect on v2:172.21.15.160:6838/2765554482 2024-05-20T16:17:08.039 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:08.039+0000 7ff53affd640 1 -- 172.21.15.82:0/4028749838 <== mds.0 v2:172.21.15.160:6838/2765554482 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5615afc4fa70 con 0x5615af8fcd80 2024-05-20T16:17:08.039 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:08.039+0000 7ff53affd640 10 client.15651 handle_command_reply: tid=0 2024-05-20T16:17:08.039 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:08.040+0000 7ff568b22740 1 client.15651 shutdown 2024-05-20T16:17:08.041 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:08.041+0000 7ff568b22740 1 -- 172.21.15.82:0/4028749838 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5615af9152f0 msgr2=0x5615afce7e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:17:08.041 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:08.041+0000 7ff568b22740 1 --2- 172.21.15.82:0/4028749838 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5615af9152f0 0x5615afce7e20 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7ff54c005e00 tx=0x7ff54c005400 comp rx=0 tx=0).stop 2024-05-20T16:17:08.041 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:08.042+0000 7ff568b22740 1 -- 172.21.15.82:0/4028749838 shutdown_connections 2024-05-20T16:17:08.041 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:08.042+0000 7ff568b22740 1 --2- 172.21.15.82:0/4028749838 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x5615af8fcd80 0x5615afcf54f0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:17:08.042 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:08.042+0000 7ff568b22740 1 --2- 172.21.15.82:0/4028749838 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5615af93e1b0 0x5615afcef2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:17:08.042 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:08.042+0000 7ff568b22740 1 --2- 172.21.15.82:0/4028749838 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5615af91d4d0 0x5615afce8360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:17:08.042 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:08.042+0000 7ff568b22740 1 --2- 172.21.15.82:0/4028749838 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5615af9152f0 0x5615afce7e20 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:17:08.042 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:08.042+0000 7ff568b22740 1 -- 172.21.15.82:0/4028749838 >> 172.21.15.82:0/4028749838 conn(0x5615af932300 msgr2=0x5615afcdc990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:17:08.042 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:08.042+0000 7ff53affd640 0 client.15651 ms_handle_reset on v2:172.21.15.160:6838/2765554482 2024-05-20T16:17:08.044 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:08.044+0000 7ff568b22740 1 -- 172.21.15.82:0/4028749838 shutdown_connections 2024-05-20T16:17:08.044 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:08.044+0000 7ff568b22740 1 -- 172.21.15.82:0/4028749838 wait complete. 2024-05-20T16:17:08.044 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:08.044+0000 7ff568b22740 20 client.15651 trim_cache size 0 max 16384 2024-05-20T16:17:08.093 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:08.092+0000 7ff568b22740 1 Processor -- start 2024-05-20T16:17:08.094 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:08.092+0000 7ff568b22740 1 -- start start 2024-05-20T16:17:08.094 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:08.093+0000 7ff568b22740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5615af9152f0 0x5615afcd9a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:17:08.094 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:08.093+0000 7ff568b22740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5615af91d4d0 0x5615afce79f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:17:08.094 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:08.093+0000 7ff568b22740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5615af93e1b0 0x5615afce7f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:17:08.094 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:08.093+0000 7ff568b22740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7ff5480125b0 con 0x5615af93e1b0 2024-05-20T16:17:08.094 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:08.093+0000 7ff568b22740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7ff54800fe80 con 0x5615af9152f0 2024-05-20T16:17:08.094 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:08.093+0000 7ff568b22740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ff54800dcd0 con 0x5615af91d4d0 2024-05-20T16:17:08.096 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:08.096+0000 7ff55bfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5615af93e1b0 0x5615afce7f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:17:08.096 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:08.096+0000 7ff55bfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5615af93e1b0 0x5615afce7f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:49644/0 (socket says 172.21.15.82:49644) 2024-05-20T16:17:08.096 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:08.096+0000 7ff55bfff640 1 -- 172.21.15.82:0/3340506755 learned_addr learned my addr 172.21.15.82:0/3340506755 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:17:08.096 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:08.096+0000 7ff55b7fe640 1 --2- 172.21.15.82:0/3340506755 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5615af9152f0 0x5615afcd9a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:17:08.096 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:08.096+0000 7ff55affd640 1 --2- 172.21.15.82:0/3340506755 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5615af91d4d0 0x5615afce79f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:17:08.097 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:08.096+0000 7ff55bfff640 1 -- 172.21.15.82:0/3340506755 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5615af91d4d0 msgr2=0x5615afce79f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:17:08.097 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:08.097+0000 7ff55bfff640 1 --2- 172.21.15.82:0/3340506755 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5615af91d4d0 0x5615afce79f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:17:08.097 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:08.097+0000 7ff55bfff640 1 -- 172.21.15.82:0/3340506755 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5615af9152f0 msgr2=0x5615afcd9a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:17:08.097 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:08.097+0000 7ff55bfff640 1 --2- 172.21.15.82:0/3340506755 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5615af9152f0 0x5615afcd9a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:17:08.098 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:08.097+0000 7ff55bfff640 1 -- 172.21.15.82:0/3340506755 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5615afcddc00 con 0x5615af93e1b0 2024-05-20T16:17:08.098 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:08.097+0000 7ff55bfff640 1 --2- 172.21.15.82:0/3340506755 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5615af93e1b0 0x5615afce7f30 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7ff554003c10 tx=0x7ff554040000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d691cdfaa290683 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:17:08.106 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:08.106+0000 7ff5417fa640 1 -- 172.21.15.82:0/3340506755 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff554003c40 con 0x5615af93e1b0 2024-05-20T16:17:08.106 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:08.106+0000 7ff5417fa640 1 -- 172.21.15.82:0/3340506755 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff554050070 con 0x5615af93e1b0 2024-05-20T16:17:08.106 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:08.106+0000 7ff5417fa640 1 -- 172.21.15.82:0/3340506755 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff55402fa10 con 0x5615af93e1b0 2024-05-20T16:17:08.108 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:08.107+0000 7ff568b22740 1 -- 172.21.15.82:0/3340506755 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5615af93e1b0 msgr2=0x5615afce7f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:17:08.108 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:08.108+0000 7ff568b22740 1 --2- 172.21.15.82:0/3340506755 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5615af93e1b0 0x5615afce7f30 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7ff554003c10 tx=0x7ff554040000 comp rx=0 tx=0).stop 2024-05-20T16:17:08.108 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:08.108+0000 7ff568b22740 1 -- 172.21.15.82:0/3340506755 shutdown_connections 2024-05-20T16:17:08.108 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:08.108+0000 7ff568b22740 1 --2- 172.21.15.82:0/3340506755 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5615af93e1b0 0x5615afce7f30 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:17:08.108 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:08.108+0000 7ff568b22740 1 --2- 172.21.15.82:0/3340506755 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5615af91d4d0 0x5615afce79f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:17:08.108 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:08.108+0000 7ff568b22740 1 --2- 172.21.15.82:0/3340506755 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5615af9152f0 0x5615afcd9a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:17:08.108 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:08.108+0000 7ff568b22740 1 -- 172.21.15.82:0/3340506755 >> 172.21.15.82:0/3340506755 conn(0x5615af932300 msgr2=0x5615afcd9370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:17:08.108 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:08.108+0000 7ff568b22740 1 -- 172.21.15.82:0/3340506755 shutdown_connections 2024-05-20T16:17:08.109 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:08.108+0000 7ff568b22740 1 -- 172.21.15.82:0/3340506755 wait complete. 2024-05-20T16:17:08.109 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:08.109+0000 7ff568b22740 1 -- start start 2024-05-20T16:17:08.109 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:08.110+0000 7ff568b22740 1 Processor -- start 2024-05-20T16:17:08.112 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:08.112+0000 7ff568b22740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5615af9152f0 0x5615afceba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:17:08.112 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:08.112+0000 7ff568b22740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5615af91d4d0 0x5615afcebf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:17:08.112 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:08.112+0000 7ff568b22740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5615af93e1b0 0x5615afcece60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:17:08.112 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:08.113+0000 7ff568b22740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7ff54800e340 con 0x5615af93e1b0 2024-05-20T16:17:08.113 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:08.113+0000 7ff568b22740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7ff54800f7b0 con 0x5615af91d4d0 2024-05-20T16:17:08.113 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:08.113+0000 7ff568b22740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ff54800ea10 con 0x5615af9152f0 2024-05-20T16:17:08.114 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:08.113+0000 7ff55affd640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5615af91d4d0 0x5615afcebf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:17:08.114 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:08.113+0000 7ff55affd640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5615af91d4d0 0x5615afcebf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.82:57356/0 (socket says 172.21.15.82:57356) 2024-05-20T16:17:08.114 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:08.113+0000 7ff55affd640 1 -- 172.21.15.82:0/3521108599 learned_addr learned my addr 172.21.15.82:0/3521108599 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:17:08.114 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:08.113+0000 7ff568b22740 10 client.0 _ll_register_callbacks cb 0x7ff550005240 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T16:17:08.114 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:08.113+0000 7ff55bfff640 1 --2- 172.21.15.82:0/3521108599 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5615af93e1b0 0x5615afcece60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:17:08.114 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:08.114+0000 7ff55b7fe640 1 --2- 172.21.15.82:0/3521108599 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5615af9152f0 0x5615afceba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:17:08.115 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:08.114+0000 7ff55affd640 1 -- 172.21.15.82:0/3521108599 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5615af9152f0 msgr2=0x5615afceba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:17:08.115 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:08.114+0000 7ff55affd640 1 --2- 172.21.15.82:0/3521108599 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5615af9152f0 0x5615afceba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:17:08.115 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:08.114+0000 7ff55affd640 1 -- 172.21.15.82:0/3521108599 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5615af93e1b0 msgr2=0x5615afcece60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:17:08.115 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:08.114+0000 7ff55affd640 1 --2- 172.21.15.82:0/3521108599 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5615af93e1b0 0x5615afcece60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:17:08.115 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:08.114+0000 7ff55affd640 1 -- 172.21.15.82:0/3521108599 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5615afced420 con 0x5615af91d4d0 2024-05-20T16:17:08.115 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:08.115+0000 7ff55affd640 1 --2- 172.21.15.82:0/3521108599 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5615af91d4d0 0x5615afcebf80 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7ff55000eed0 tx=0x7ff550011000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3be2ba4d54e043a1 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:17:08.116 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:08.115+0000 7ff53affd640 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-20T16:17:08.116 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:08.116+0000 7ff53affd640 1 -- 172.21.15.82:0/3521108599 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff55009c8e0 con 0x5615af91d4d0 2024-05-20T16:17:08.116 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:08.117+0000 7ff53affd640 1 -- 172.21.15.82:0/3521108599 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9150+0+0 (secure 0 0 0) 0x7ff550098850 con 0x5615af91d4d0 2024-05-20T16:17:08.121 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:08.119+0000 7ff568b22740 1 -- 172.21.15.82:0/3521108599 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff554050070 con 0x5615af91d4d0 2024-05-20T16:17:08.121 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:08.121+0000 7ff53affd640 1 -- 172.21.15.82:0/3521108599 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ff55009c1e0 con 0x5615af91d4d0 2024-05-20T16:17:08.123 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:08.123+0000 7ff568b22740 10 client.25184 fetch_fsmap learned FSMap version 25 2024-05-20T16:17:08.124 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:08.124+0000 7ff568b22740 1 -- 172.21.15.82:0/3521108599 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5615afcee780 con 0x5615af91d4d0 2024-05-20T16:17:08.134 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:08.132+0000 7ff53affd640 1 -- 172.21.15.82:0/3521108599 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff5500a0590 con 0x5615af91d4d0 2024-05-20T16:17:08.134 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:08.132+0000 7ff53affd640 1 -- 172.21.15.82:0/3521108599 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7ff550004cb0 con 0x5615af91d4d0 2024-05-20T16:17:08.134 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:08.134+0000 7ff568b22740 10 client.25184 fetch_fsmap finished waiting for FSMap version 25 2024-05-20T16:17:08.134 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:08.134+0000 7ff568b22740 10 client.25184 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T16:17:08.134 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:08.134+0000 7ff568b22740 20 client.25184 populate_metadata read hostname 'smithi082' 2024-05-20T16:17:08.134 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:08.134+0000 7ff568b22740 1 --2- 172.21.15.82:0/3521108599 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x5615af8d21c0 0x5615afd5faf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:17:08.135 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:08.134+0000 7ff568b22740 4 client.25184 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-20T16:17:08.135 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:08.134+0000 7ff568b22740 1 -- 172.21.15.82:0/3521108599 --> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7ff554050070 con 0x5615af8d21c0 2024-05-20T16:17:08.140 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:08.137+0000 7ff55b7fe640 1 --2- 172.21.15.82:0/3521108599 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x5615af8d21c0 0x5615afd5faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:17:08.151 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:08.151+0000 7ff53affd640 10 client.25184 ms_handle_connect on v2:172.21.15.160:6838/2765554482 2024-05-20T16:17:08.151 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:08.151+0000 7ff55b7fe640 1 --2- 172.21.15.82:0/3521108599 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x5615af8d21c0 0x5615afd5faf0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=293186a14035fb17 server_cookie=e70e42c9164fe83e in_seq=0 out_seq=0 2024-05-20T16:17:10.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:17:10 smithi160 ceph-mon[36347]: pgmap v457: 129 pgs: 129 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 685 KiB/s rd, 917 MiB/s wr, 1.38k op/s 2024-05-20T16:17:10.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:17:10 smithi082 ceph-mon[32238]: pgmap v457: 129 pgs: 129 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 685 KiB/s rd, 917 MiB/s wr, 1.38k op/s 2024-05-20T16:17:10.885 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:17:10 smithi184 ceph-mon[36209]: pgmap v457: 129 pgs: 129 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 685 KiB/s rd, 917 MiB/s wr, 1.38k op/s 2024-05-20T16:17:11.498 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:11.499+0000 7ff53affd640 1 -- 172.21.15.82:0/3521108599 <== mds.0 v2:172.21.15.160:6838/2765554482 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7ff554050070 con 0x5615af8d21c0 2024-05-20T16:17:11.499 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:11.499+0000 7ff53affd640 10 client.25184 handle_command_reply: tid=0 2024-05-20T16:17:11.499 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:11.499+0000 7ff568b22740 1 client.25184 shutdown 2024-05-20T16:17:11.501 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:11.502+0000 7ff568b22740 1 -- 172.21.15.82:0/3521108599 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5615af91d4d0 msgr2=0x5615afcebf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:17:11.501 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:11.502+0000 7ff568b22740 1 --2- 172.21.15.82:0/3521108599 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5615af91d4d0 0x5615afcebf80 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7ff55000eed0 tx=0x7ff550011000 comp rx=0 tx=0).stop 2024-05-20T16:17:11.502 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:11.502+0000 7ff568b22740 1 -- 172.21.15.82:0/3521108599 shutdown_connections 2024-05-20T16:17:11.502 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:11.502+0000 7ff568b22740 1 --2- 172.21.15.82:0/3521108599 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x5615af8d21c0 0x5615afd5faf0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:17:11.502 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:11.502+0000 7ff568b22740 1 --2- 172.21.15.82:0/3521108599 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5615af93e1b0 0x5615afcece60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:17:11.502 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:11.502+0000 7ff568b22740 1 --2- 172.21.15.82:0/3521108599 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5615af91d4d0 0x5615afcebf80 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:17:11.502 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:11.502+0000 7ff568b22740 1 --2- 172.21.15.82:0/3521108599 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5615af9152f0 0x5615afceba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:17:11.502 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:11.502+0000 7ff568b22740 1 -- 172.21.15.82:0/3521108599 >> 172.21.15.82:0/3521108599 conn(0x5615af932300 msgr2=0x5615afcdc880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:17:11.503 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:11.502+0000 7ff53affd640 0 client.25184 ms_handle_reset on v2:172.21.15.160:6838/2765554482 2024-05-20T16:17:11.503 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:11.502+0000 7ff568b22740 1 -- 172.21.15.82:0/3521108599 shutdown_connections 2024-05-20T16:17:11.503 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:11.503+0000 7ff568b22740 1 -- 172.21.15.82:0/3521108599 wait complete. 2024-05-20T16:17:11.503 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:11.503+0000 7ff568b22740 20 client.25184 trim_cache size 0 max 16384 2024-05-20T16:17:11.504 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:{ 2024-05-20T16:17:11.504 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "message": "", 2024-05-20T16:17:11.504 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "return_code": 0 2024-05-20T16:17:11.504 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:} 2024-05-20T16:17:11.505 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:11.506+0000 7ff560d7e640 1 -- 172.21.15.82:0/1239982144 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7ff5440a2bc0 msgr2=0x7ff5440a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:17:11.506 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:11.506+0000 7ff560d7e640 1 --2- 172.21.15.82:0/1239982144 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7ff5440a2bc0 0x7ff5440a50d0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x5615afcdf220 tx=0x7ff5540093e0 comp rx=0 tx=0).stop 2024-05-20T16:17:11.506 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:11.506+0000 7ff560d7e640 1 -- 172.21.15.82:0/1239982144 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff55c10a9d0 msgr2=0x7ff55c11af70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:17:11.506 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:11.506+0000 7ff560d7e640 1 --2- 172.21.15.82:0/1239982144 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff55c10a9d0 0x7ff55c11af70 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7ff5500097d0 tx=0x7ff550009c90 comp rx=0 tx=0).stop 2024-05-20T16:17:11.506 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:11.506+0000 7ff560d7e640 1 -- 172.21.15.82:0/1239982144 shutdown_connections 2024-05-20T16:17:11.506 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:11.506+0000 7ff560d7e640 1 --2- 172.21.15.82:0/1239982144 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7ff5440a2bc0 0x7ff5440a50d0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:17:11.506 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:11.506+0000 7ff560d7e640 1 --2- 172.21.15.82:0/1239982144 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff55c10f6d0 0x7ff55c11de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:17:11.507 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:11.506+0000 7ff560d7e640 1 --2- 172.21.15.82:0/1239982144 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff55c10a9d0 0x7ff55c11af70 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:17:11.507 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:11.506+0000 7ff560d7e640 1 --2- 172.21.15.82:0/1239982144 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff55c109bd0 0x7ff55c11aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:17:11.507 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:11.506+0000 7ff560d7e640 1 -- 172.21.15.82:0/1239982144 >> 172.21.15.82:0/1239982144 conn(0x7ff55c06f2e0 msgr2=0x7ff55c071700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:17:11.507 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:11.507+0000 7ff560d7e640 1 -- 172.21.15.82:0/1239982144 shutdown_connections 2024-05-20T16:17:11.507 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:11.507+0000 7ff560d7e640 1 -- 172.21.15.82:0/1239982144 wait complete. 2024-05-20T16:17:11.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:17:11 smithi160 ceph-mon[36347]: pgmap v458: 129 pgs: 129 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 343 KiB/s rd, 797 MiB/s wr, 1.17k op/s 2024-05-20T16:17:11.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:17:11 smithi082 ceph-mon[32238]: pgmap v458: 129 pgs: 129 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 343 KiB/s rd, 797 MiB/s wr, 1.17k op/s 2024-05-20T16:17:11.885 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:17:11 smithi184 ceph-mon[36209]: pgmap v458: 129 pgs: 129 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 343 KiB/s rd, 797 MiB/s wr, 1.17k op/s 2024-05-20T16:17:13.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:17:13 smithi160 ceph-mon[36347]: pgmap v459: 129 pgs: 129 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 343 KiB/s rd, 827 MiB/s wr, 1.41k op/s 2024-05-20T16:17:13.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:17:13 smithi082 ceph-mon[32238]: pgmap v459: 129 pgs: 129 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 343 KiB/s rd, 827 MiB/s wr, 1.41k op/s 2024-05-20T16:17:13.885 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:17:13 smithi184 ceph-mon[36209]: pgmap v459: 129 pgs: 129 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 343 KiB/s rd, 827 MiB/s wr, 1.41k op/s 2024-05-20T16:17:15.362 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:17:15 smithi082 ceph-mon[32238]: daemon mds.c restarted 2024-05-20T16:17:15.385 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:17:15 smithi184 ceph-mon[36209]: daemon mds.c restarted 2024-05-20T16:17:15.386 INFO:journalctl@ceph.mds.c.smithi184.stdout:May 20 16:17:14 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-c[67402]: ignoring --setuser ceph since I am not root 2024-05-20T16:17:15.386 INFO:journalctl@ceph.mds.c.smithi184.stdout:May 20 16:17:14 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-c[67402]: ignoring --setgroup ceph since I am not root 2024-05-20T16:17:15.386 INFO:journalctl@ceph.mds.c.smithi184.stdout:May 20 16:17:14 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-c[67402]: starting mds.c at 2024-05-20T16:17:15.562 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:17:15 smithi160 ceph-mon[36347]: daemon mds.c restarted 2024-05-20T16:17:16.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:17:16 smithi082 ceph-mon[32238]: osdmap e96: 12 total, 12 up, 12 in 2024-05-20T16:17:16.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:17:16 smithi082 ceph-mon[32238]: mds.? [v2:172.21.15.184:6836/1347621608,v1:172.21.15.184:6837/1347621608] up:boot 2024-05-20T16:17:16.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:17:16 smithi082 ceph-mon[32238]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 2 up:standby-replay 7 up:standby 2024-05-20T16:17:16.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:17:16 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-20T16:17:16.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:17:16 smithi082 ceph-mon[32238]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-05-20T16:17:16.360 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:17:16 smithi082 ceph-mon[32238]: pgmap v461: 129 pgs: 129 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 810 MiB/s wr, 1.70k op/s 2024-05-20T16:17:16.385 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:17:16 smithi184 ceph-mon[36209]: osdmap e96: 12 total, 12 up, 12 in 2024-05-20T16:17:16.385 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:17:16 smithi184 ceph-mon[36209]: mds.? [v2:172.21.15.184:6836/1347621608,v1:172.21.15.184:6837/1347621608] up:boot 2024-05-20T16:17:16.386 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:17:16 smithi184 ceph-mon[36209]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 2 up:standby-replay 7 up:standby 2024-05-20T16:17:16.386 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:17:16 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-20T16:17:16.386 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:17:16 smithi184 ceph-mon[36209]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-05-20T16:17:16.386 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:17:16 smithi184 ceph-mon[36209]: pgmap v461: 129 pgs: 129 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 810 MiB/s wr, 1.70k op/s 2024-05-20T16:17:16.562 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:17:16 smithi160 ceph-mon[36347]: osdmap e96: 12 total, 12 up, 12 in 2024-05-20T16:17:16.562 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:17:16 smithi160 ceph-mon[36347]: mds.? [v2:172.21.15.184:6836/1347621608,v1:172.21.15.184:6837/1347621608] up:boot 2024-05-20T16:17:16.562 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:17:16 smithi160 ceph-mon[36347]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 2 up:standby-replay 7 up:standby 2024-05-20T16:17:16.562 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:17:16 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-20T16:17:16.562 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:17:16 smithi160 ceph-mon[36347]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-05-20T16:17:16.562 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:17:16 smithi160 ceph-mon[36347]: pgmap v461: 129 pgs: 129 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 810 MiB/s wr, 1.70k op/s 2024-05-20T16:17:17.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:17:17 smithi160 ceph-mon[36347]: pgmap v462: 129 pgs: 129 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 749 MiB/s wr, 1.60k op/s 2024-05-20T16:17:17.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:17:17 smithi082 ceph-mon[32238]: pgmap v462: 129 pgs: 129 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 749 MiB/s wr, 1.60k op/s 2024-05-20T16:17:17.885 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:17:17 smithi184 ceph-mon[36209]: pgmap v462: 129 pgs: 129 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 749 MiB/s wr, 1.60k op/s 2024-05-20T16:17:19.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:17:19 smithi160 ceph-mon[36347]: pgmap v463: 129 pgs: 129 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 791 MiB/s wr, 1.47k op/s 2024-05-20T16:17:19.860 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:17:19 smithi082 ceph-mon[32238]: pgmap v463: 129 pgs: 129 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 791 MiB/s wr, 1.47k op/s 2024-05-20T16:17:19.885 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:17:19 smithi184 ceph-mon[36209]: pgmap v463: 129 pgs: 129 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 791 MiB/s wr, 1.47k op/s 2024-05-20T16:17:21.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:17:21 smithi160 ceph-mon[36347]: pgmap v464: 129 pgs: 129 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 797 MiB/s wr, 1.46k op/s 2024-05-20T16:17:21.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:17:21 smithi082 ceph-mon[32238]: pgmap v464: 129 pgs: 129 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 797 MiB/s wr, 1.46k op/s 2024-05-20T16:17:21.885 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:17:21 smithi184 ceph-mon[36209]: pgmap v464: 129 pgs: 129 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 797 MiB/s wr, 1.46k op/s 2024-05-20T16:17:23.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:17:23 smithi160 ceph-mon[36347]: pgmap v465: 129 pgs: 129 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 923 MiB/s wr, 1.76k op/s 2024-05-20T16:17:23.860 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:17:23 smithi082 ceph-mon[32238]: pgmap v465: 129 pgs: 129 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 923 MiB/s wr, 1.76k op/s 2024-05-20T16:17:23.885 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:17:23 smithi184 ceph-mon[36209]: pgmap v465: 129 pgs: 129 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 923 MiB/s wr, 1.76k op/s 2024-05-20T16:17:24.788 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:24.787+0000 7f4a7727c640 1 -- 172.21.15.82:0/1915533204 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4a70106490 msgr2=0x7f4a701068a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:17:24.788 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:24.787+0000 7f4a7727c640 1 --2- 172.21.15.82:0/1915533204 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4a70106490 0x7f4a701068a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f4a6c009900 tx=0x7f4a6c02a440 comp rx=0 tx=0).stop 2024-05-20T16:17:24.791 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:24.791+0000 7f4a7727c640 1 -- 172.21.15.82:0/1915533204 shutdown_connections 2024-05-20T16:17:24.791 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:24.791+0000 7f4a7727c640 1 --2- 172.21.15.82:0/1915533204 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4a70103b30 0x7f4a70105f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:17:24.791 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:24.791+0000 7f4a7727c640 1 --2- 172.21.15.82:0/1915533204 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4a70106490 0x7f4a701068a0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:17:24.791 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:24.791+0000 7f4a7727c640 1 --2- 172.21.15.82:0/1915533204 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4a701011d0 0x7f4a701035f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:17:24.791 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:24.791+0000 7f4a7727c640 1 -- 172.21.15.82:0/1915533204 >> 172.21.15.82:0/1915533204 conn(0x7f4a700fc9e0 msgr2=0x7f4a700fee90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:17:24.794 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:24.794+0000 7f4a7727c640 1 -- 172.21.15.82:0/1915533204 shutdown_connections 2024-05-20T16:17:24.794 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:24.795+0000 7f4a7727c640 1 -- 172.21.15.82:0/1915533204 wait complete. 2024-05-20T16:17:24.796 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:24.796+0000 7f4a7727c640 1 Processor -- start 2024-05-20T16:17:24.796 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:24.796+0000 7f4a7727c640 1 -- start start 2024-05-20T16:17:24.796 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:24.796+0000 7f4a7727c640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4a701011d0 0x7f4a7019f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:17:24.796 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:24.796+0000 7f4a7727c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4a70103b30 0x7f4a7019fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:17:24.796 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:24.796+0000 7f4a7727c640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4a70106490 0x7f4a701a6cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:17:24.797 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:24.796+0000 7f4a7727c640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f4a70114c90 con 0x7f4a70103b30 2024-05-20T16:17:24.797 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:24.796+0000 7f4a7727c640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f4a70114b10 con 0x7f4a701011d0 2024-05-20T16:17:24.797 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:24.796+0000 7f4a7727c640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f4a70114e10 con 0x7f4a70106490 2024-05-20T16:17:24.797 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:24.797+0000 7f4a7627a640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4a701011d0 0x7f4a7019f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:17:24.798 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:24.797+0000 7f4a7627a640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4a701011d0 0x7f4a7019f800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.82:43518/0 (socket says 172.21.15.82:43518) 2024-05-20T16:17:24.798 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:24.797+0000 7f4a7627a640 1 -- 172.21.15.82:0/1734350440 learned_addr learned my addr 172.21.15.82:0/1734350440 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:17:24.798 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:24.797+0000 7f4a76a7b640 1 --2- 172.21.15.82:0/1734350440 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4a70106490 0x7f4a701a6cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:17:24.798 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:24.797+0000 7f4a75a79640 1 --2- 172.21.15.82:0/1734350440 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4a70103b30 0x7f4a7019fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:17:24.799 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:24.798+0000 7f4a7627a640 1 -- 172.21.15.82:0/1734350440 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4a70106490 msgr2=0x7f4a701a6cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:17:24.799 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:24.798+0000 7f4a7627a640 1 --2- 172.21.15.82:0/1734350440 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4a70106490 0x7f4a701a6cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:17:24.799 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:24.798+0000 7f4a7627a640 1 -- 172.21.15.82:0/1734350440 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4a70103b30 msgr2=0x7f4a7019fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:17:24.799 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:24.798+0000 7f4a7627a640 1 --2- 172.21.15.82:0/1734350440 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4a70103b30 0x7f4a7019fd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:17:24.799 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:24.798+0000 7f4a7627a640 1 -- 172.21.15.82:0/1734350440 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a701a7340 con 0x7f4a701011d0 2024-05-20T16:17:24.799 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:24.799+0000 7f4a7627a640 1 --2- 172.21.15.82:0/1734350440 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4a701011d0 0x7f4a7019f800 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f4a6800b9e0 tx=0x7f4a6800bde0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:17:24.800 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:24.800+0000 7f4a677fe640 1 -- 172.21.15.82:0/1734350440 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a68019070 con 0x7f4a701011d0 2024-05-20T16:17:24.800 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:24.800+0000 7f4a677fe640 1 -- 172.21.15.82:0/1734350440 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4a68014440 con 0x7f4a701011d0 2024-05-20T16:17:24.800 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:24.800+0000 7f4a677fe640 1 -- 172.21.15.82:0/1734350440 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a680125e0 con 0x7f4a701011d0 2024-05-20T16:17:24.800 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:24.800+0000 7f4a7727c640 1 -- 172.21.15.82:0/1734350440 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a701a7630 con 0x7f4a701011d0 2024-05-20T16:17:24.800 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:24.800+0000 7f4a7727c640 1 -- 172.21.15.82:0/1734350440 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a701a7bb0 con 0x7f4a701011d0 2024-05-20T16:17:24.812 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:24.808+0000 7f4a7f37e740 1 Processor -- start 2024-05-20T16:17:24.812 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:24.808+0000 7f4a677fe640 1 -- 172.21.15.82:0/1734350440 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f4a6801c070 con 0x7f4a701011d0 2024-05-20T16:17:24.812 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:24.810+0000 7f4a677fe640 1 --2- 172.21.15.82:0/1734350440 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f4a5c0a2d50 0x7f4a5c0a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:17:24.812 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:24.810+0000 7f4a677fe640 1 -- 172.21.15.82:0/1734350440 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9150+0+0 (secure 0 0 0) 0x7f4a680ce8d0 con 0x7f4a701011d0 2024-05-20T16:17:24.812 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:24.812+0000 7f4a7f37e740 1 -- start start 2024-05-20T16:17:24.812 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:24.812+0000 7f4a7f37e740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564523ffc1b0 0x56452439c060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:17:24.812 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:24.812+0000 7f4a76a7b640 1 --2- 172.21.15.82:0/1734350440 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f4a5c0a2d50 0x7f4a5c0a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:17:24.813 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:24.812+0000 7f4a7f37e740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564523fd32f0 0x56452439c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:17:24.813 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:24.812+0000 7f4a7f37e740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564523fdb4d0 0x5645243a34e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:17:24.813 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:24.812+0000 7f4a7f37e740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x564524300340 con 0x564523fd32f0 2024-05-20T16:17:24.813 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:24.812+0000 7f4a7f37e740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x5645242a8290 con 0x564523fdb4d0 2024-05-20T16:17:24.813 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:24.812+0000 7f4a7f37e740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5645242fc300 con 0x564523ffc1b0 2024-05-20T16:17:24.813 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:24.812+0000 7f4a7627a640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564523ffc1b0 0x56452439c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:17:24.813 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:24.812+0000 7f4a75a79640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564523fd32f0 0x56452439c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:17:24.813 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:24.812+0000 7f4a7627a640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564523ffc1b0 0x56452439c060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.82:49204/0 (socket says 172.21.15.82:49204) 2024-05-20T16:17:24.813 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:24.812+0000 7f4a7627a640 1 -- 172.21.15.82:0/1258333729 learned_addr learned my addr 172.21.15.82:0/1258333729 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:17:24.813 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:24.812+0000 7f4a76a7b640 1 --2- 172.21.15.82:0/1258333729 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564523fdb4d0 0x5645243a34e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:17:24.814 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:24.813+0000 7f4a75a79640 1 -- 172.21.15.82:0/1258333729 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564523ffc1b0 msgr2=0x56452439c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:17:24.814 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:24.813+0000 7f4a75a79640 1 --2- 172.21.15.82:0/1258333729 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564523ffc1b0 0x56452439c060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:17:24.814 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:24.813+0000 7f4a75a79640 1 -- 172.21.15.82:0/1258333729 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564523fdb4d0 msgr2=0x5645243a34e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:17:24.815 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:24.813+0000 7f4a75a79640 1 --2- 172.21.15.82:0/1258333729 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564523fdb4d0 0x5645243a34e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:17:24.815 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:24.813+0000 7f4a75a79640 1 -- 172.21.15.82:0/1258333729 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5645242cc8e0 con 0x564523fd32f0 2024-05-20T16:17:24.815 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:24.814+0000 7f4a75a79640 1 --2- 172.21.15.82:0/1258333729 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564523fd32f0 0x56452439c5e0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f4a60009970 tx=0x7f4a60009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eda55f7ad1ad5075 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:17:24.815 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:24.814+0000 7f4a64bfe640 1 -- 172.21.15.82:0/1258333729 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a60017070 con 0x564523fd32f0 2024-05-20T16:17:24.815 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:24.815+0000 7f4a64bfe640 1 -- 172.21.15.82:0/1258333729 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4a60013470 con 0x564523fd32f0 2024-05-20T16:17:24.816 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:24.816+0000 7f4a64bfe640 1 -- 172.21.15.82:0/1258333729 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a60017070 con 0x564523fd32f0 2024-05-20T16:17:24.816 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:24.816+0000 7f4a7f37e740 1 -- 172.21.15.82:0/1258333729 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564523fd32f0 msgr2=0x56452439c5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:17:24.816 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:24.816+0000 7f4a76a7b640 1 --2- 172.21.15.82:0/1734350440 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f4a5c0a2d50 0x7f4a5c0a5260 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f4a6c031610 tx=0x7f4a6c004430 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:17:24.816 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:24.816+0000 7f4a7f37e740 1 --2- 172.21.15.82:0/1258333729 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564523fd32f0 0x56452439c5e0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f4a60009970 tx=0x7f4a60009e30 comp rx=0 tx=0).stop 2024-05-20T16:17:24.816 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:24.816+0000 7f4a7f37e740 1 -- 172.21.15.82:0/1258333729 shutdown_connections 2024-05-20T16:17:24.816 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:24.816+0000 7f4a7f37e740 1 --2- 172.21.15.82:0/1258333729 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564523fd32f0 0x56452439c5e0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:17:24.816 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:24.816+0000 7f4a7f37e740 1 --2- 172.21.15.82:0/1258333729 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564523fdb4d0 0x5645243a34e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:17:24.816 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:24.816+0000 7f4a7f37e740 1 --2- 172.21.15.82:0/1258333729 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564523ffc1b0 0x56452439c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:17:24.816 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:24.816+0000 7f4a7f37e740 1 -- 172.21.15.82:0/1258333729 >> 172.21.15.82:0/1258333729 conn(0x564523ff0300 msgr2=0x5645243979f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:17:24.816 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:24.816+0000 7f4a7f37e740 1 -- 172.21.15.82:0/1258333729 shutdown_connections 2024-05-20T16:17:24.816 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:24.816+0000 7f4a7f37e740 1 -- 172.21.15.82:0/1258333729 wait complete. 2024-05-20T16:17:24.817 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:24.817+0000 7f4a7f37e740 1 -- start start 2024-05-20T16:17:24.817 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:24.817+0000 7f4a7f37e740 1 Processor -- start 2024-05-20T16:17:24.818 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:24.818+0000 7f4a7f37e740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564523fd32f0 0x5645243a5d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:17:24.819 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:24.818+0000 7f4a7f37e740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564523fdb4d0 0x5645243a6280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:17:24.819 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:24.818+0000 7f4a7f37e740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564523ffc1b0 0x5645243ad1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:17:24.819 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:24.818+0000 7f4a7f37e740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x5645243a3f80 con 0x564523ffc1b0 2024-05-20T16:17:24.819 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:24.818+0000 7f4a7f37e740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x5645242bb790 con 0x564523fdb4d0 2024-05-20T16:17:24.819 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:24.818+0000 7f4a7f37e740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5645243a4100 con 0x564523fd32f0 2024-05-20T16:17:24.819 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:24.818+0000 7f4a7f37e740 10 client.0 _ll_register_callbacks cb 0x5645242f6a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T16:17:24.819 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:24.819+0000 7f4a76a7b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564523ffc1b0 0x5645243ad1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:17:24.819 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:24.819+0000 7f4a76a7b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564523ffc1b0 0x5645243ad1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43636/0 (socket says 172.21.15.82:43636) 2024-05-20T16:17:24.819 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:24.819+0000 7f4a76a7b640 1 -- 172.21.15.82:0/1947882427 learned_addr learned my addr 172.21.15.82:0/1947882427 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:17:24.819 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:24.819+0000 7f4a7627a640 1 --2- 172.21.15.82:0/1947882427 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564523fd32f0 0x5645243a5d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:17:24.819 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:24.819+0000 7f4a75a79640 1 --2- 172.21.15.82:0/1947882427 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564523fdb4d0 0x5645243a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:17:24.820 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:24.819+0000 7f4a7627a640 1 -- 172.21.15.82:0/1947882427 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564523fdb4d0 msgr2=0x5645243a6280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:17:24.820 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:24.819+0000 7f4a7627a640 1 --2- 172.21.15.82:0/1947882427 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564523fdb4d0 0x5645243a6280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:17:24.820 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:24.819+0000 7f4a7627a640 1 -- 172.21.15.82:0/1947882427 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564523ffc1b0 msgr2=0x5645243ad1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:17:24.820 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:24.819+0000 7f4a7627a640 1 --2- 172.21.15.82:0/1947882427 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564523ffc1b0 0x5645243ad1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:17:24.821 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:24.819+0000 7f4a7627a640 1 -- 172.21.15.82:0/1947882427 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564524014210 con 0x564523fd32f0 2024-05-20T16:17:24.821 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:24.819+0000 7f4a7627a640 1 --2- 172.21.15.82:0/1947882427 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564523fd32f0 0x5645243a5d40 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f4a68097710 tx=0x7f4a6809f230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f48e74ba96ecfa66 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:17:24.821 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:24.819+0000 7f4a59ffb640 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-05-20T16:17:24.821 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:24.820+0000 7f4a59ffb640 1 -- 172.21.15.82:0/1947882427 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a6809a190 con 0x564523fd32f0 2024-05-20T16:17:24.821 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:24.820+0000 7f4a59ffb640 1 -- 172.21.15.82:0/1947882427 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9150+0+0 (secure 0 0 0) 0x7f4a6809a330 con 0x564523fd32f0 2024-05-20T16:17:24.821 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:24.820+0000 7f4a7f37e740 1 -- 172.21.15.82:0/1947882427 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4a60013470 con 0x564523fd32f0 2024-05-20T16:17:24.821 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:24.820+0000 7f4a59ffb640 1 -- 172.21.15.82:0/1947882427 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f4a6800cb20 con 0x564523fd32f0 2024-05-20T16:17:24.821 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:24.820+0000 7f4a7f37e740 10 client.25222 fetch_fsmap learned FSMap version 27 2024-05-20T16:17:24.821 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:24.820+0000 7f4a7f37e740 1 -- 172.21.15.82:0/1947882427 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5645240e0620 con 0x564523fd32f0 2024-05-20T16:17:24.821 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:24.821+0000 7f4a59ffb640 1 -- 172.21.15.82:0/1947882427 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4a60013470 con 0x564523fd32f0 2024-05-20T16:17:24.822 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:24.821+0000 7f4a59ffb640 1 -- 172.21.15.82:0/1947882427 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f4a680a0280 con 0x564523fd32f0 2024-05-20T16:17:24.822 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:24.821+0000 7f4a7f37e740 10 client.25222 fetch_fsmap finished waiting for FSMap version 27 2024-05-20T16:17:24.822 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:24.821+0000 7f4a7f37e740 10 client.25222 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T16:17:24.822 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:24.821+0000 7f4a7f37e740 20 client.25222 populate_metadata read hostname 'smithi082' 2024-05-20T16:17:24.822 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:24.821+0000 7f4a7f37e740 1 --2- 172.21.15.82:0/1947882427 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x564523fbad80 0x5645243b37e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:17:24.822 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:24.821+0000 7f4a7f37e740 4 client.25222 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T16:17:24.822 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:24.821+0000 7f4a7f37e740 1 -- 172.21.15.82:0/1947882427 --> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56452430da70 con 0x564523fbad80 2024-05-20T16:17:24.823 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:24.822+0000 7f4a75a79640 1 --2- 172.21.15.82:0/1947882427 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x564523fbad80 0x5645243b37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:17:24.824 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:24.825+0000 7f4a75a79640 1 --2- 172.21.15.82:0/1947882427 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x564523fbad80 0x5645243b37e0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f5c4d4de43d7320d server_cookie=902d1f8844305664 in_seq=0 out_seq=0 2024-05-20T16:17:24.825 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:24.825+0000 7f4a59ffb640 10 client.25222 ms_handle_connect on v2:172.21.15.160:6838/2765554482 2024-05-20T16:17:24.827 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:24.828+0000 7f4a59ffb640 1 -- 172.21.15.82:0/1947882427 <== mds.0 v2:172.21.15.160:6838/2765554482 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56452430da70 con 0x564523fbad80 2024-05-20T16:17:24.827 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:24.828+0000 7f4a59ffb640 10 client.25222 handle_command_reply: tid=0 2024-05-20T16:17:24.827 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:24.828+0000 7f4a7f37e740 1 client.25222 shutdown 2024-05-20T16:17:24.829 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:24.829+0000 7f4a7f37e740 1 -- 172.21.15.82:0/1947882427 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564523fd32f0 msgr2=0x5645243a5d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:17:24.829 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:24.829+0000 7f4a7f37e740 1 --2- 172.21.15.82:0/1947882427 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564523fd32f0 0x5645243a5d40 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f4a68097710 tx=0x7f4a6809f230 comp rx=0 tx=0).stop 2024-05-20T16:17:24.829 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:24.829+0000 7f4a7f37e740 1 -- 172.21.15.82:0/1947882427 shutdown_connections 2024-05-20T16:17:24.829 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:24.829+0000 7f4a7f37e740 1 --2- 172.21.15.82:0/1947882427 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x564523fbad80 0x5645243b37e0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:17:24.830 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:24.829+0000 7f4a7f37e740 1 --2- 172.21.15.82:0/1947882427 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564523ffc1b0 0x5645243ad1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:17:24.830 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:24.829+0000 7f4a7f37e740 1 --2- 172.21.15.82:0/1947882427 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564523fdb4d0 0x5645243a6280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:17:24.830 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:24.829+0000 7f4a7f37e740 1 --2- 172.21.15.82:0/1947882427 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564523fd32f0 0x5645243a5d40 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:17:24.830 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:24.829+0000 7f4a7f37e740 1 -- 172.21.15.82:0/1947882427 >> 172.21.15.82:0/1947882427 conn(0x564523ff0300 msgr2=0x56452439a9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:17:24.830 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:24.829+0000 7f4a59ffb640 0 client.25222 ms_handle_reset on v2:172.21.15.160:6838/2765554482 2024-05-20T16:17:24.830 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:24.830+0000 7f4a7f37e740 1 -- 172.21.15.82:0/1947882427 shutdown_connections 2024-05-20T16:17:24.830 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:24.830+0000 7f4a7f37e740 1 -- 172.21.15.82:0/1947882427 wait complete. 2024-05-20T16:17:24.830 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:24.830+0000 7f4a7f37e740 20 client.25222 trim_cache size 0 max 16384 2024-05-20T16:17:24.856 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:24.856+0000 7f4a7f37e740 1 Processor -- start 2024-05-20T16:17:24.856 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:24.856+0000 7f4a7f37e740 1 -- start start 2024-05-20T16:17:24.856 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:24.856+0000 7f4a7f37e740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564523fd32f0 0x5645243a5fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:17:24.856 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:24.856+0000 7f4a7f37e740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564523fdb4d0 0x56452439afa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:17:24.857 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:24.857+0000 7f4a7f37e740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564523ffc1b0 0x56452439b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:17:24.857 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:24.857+0000 7f4a7f37e740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f4a54012490 con 0x564523fdb4d0 2024-05-20T16:17:24.857 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:24.857+0000 7f4a7f37e740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f4a5400fd60 con 0x564523ffc1b0 2024-05-20T16:17:24.857 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:24.857+0000 7f4a7f37e740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f4a5400dbb0 con 0x564523fd32f0 2024-05-20T16:17:24.857 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:24.857+0000 7f4a7627a640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564523fd32f0 0x5645243a5fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:17:24.858 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:24.857+0000 7f4a7627a640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564523fd32f0 0x5645243a5fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.82:49218/0 (socket says 172.21.15.82:49218) 2024-05-20T16:17:24.858 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:24.857+0000 7f4a7627a640 1 -- 172.21.15.82:0/1255338043 learned_addr learned my addr 172.21.15.82:0/1255338043 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:17:24.858 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:24.857+0000 7f4a75a79640 1 --2- 172.21.15.82:0/1255338043 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564523fdb4d0 0x56452439afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:17:24.858 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:24.857+0000 7f4a76a7b640 1 --2- 172.21.15.82:0/1255338043 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564523ffc1b0 0x56452439b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:17:24.858 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:24.857+0000 7f4a7627a640 1 -- 172.21.15.82:0/1255338043 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564523ffc1b0 msgr2=0x56452439b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:17:24.858 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:24.857+0000 7f4a7627a640 1 --2- 172.21.15.82:0/1255338043 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564523ffc1b0 0x56452439b4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:17:24.858 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:24.857+0000 7f4a7627a640 1 -- 172.21.15.82:0/1255338043 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564523fdb4d0 msgr2=0x56452439afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:17:24.858 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:24.857+0000 7f4a7627a640 1 --2- 172.21.15.82:0/1255338043 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564523fdb4d0 0x56452439afa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:17:24.858 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:24.857+0000 7f4a7627a640 1 -- 172.21.15.82:0/1255338043 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56452439bc10 con 0x564523fd32f0 2024-05-20T16:17:24.858 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:24.857+0000 7f4a7627a640 1 --2- 172.21.15.82:0/1255338043 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564523fd32f0 0x5645243a5fe0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f4a6809acd0 tx=0x7f4a6809a880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fd37e6decd50af56 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:17:24.858 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:24.858+0000 7f4a5bfff640 1 -- 172.21.15.82:0/1255338043 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a68096fe0 con 0x564523fd32f0 2024-05-20T16:17:24.858 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:24.858+0000 7f4a5bfff640 1 -- 172.21.15.82:0/1255338043 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4a6809ad40 con 0x564523fd32f0 2024-05-20T16:17:24.859 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:24.858+0000 7f4a5bfff640 1 -- 172.21.15.82:0/1255338043 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a6809a3f0 con 0x564523fd32f0 2024-05-20T16:17:24.860 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:24.859+0000 7f4a7f37e740 1 -- 172.21.15.82:0/1255338043 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564523fd32f0 msgr2=0x5645243a5fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:17:24.860 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:24.859+0000 7f4a7f37e740 1 --2- 172.21.15.82:0/1255338043 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564523fd32f0 0x5645243a5fe0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f4a6809acd0 tx=0x7f4a6809a880 comp rx=0 tx=0).stop 2024-05-20T16:17:24.860 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:24.859+0000 7f4a7f37e740 1 -- 172.21.15.82:0/1255338043 shutdown_connections 2024-05-20T16:17:24.860 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:24.859+0000 7f4a7f37e740 1 --2- 172.21.15.82:0/1255338043 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564523fdb4d0 0x56452439afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:17:24.860 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:24.859+0000 7f4a7f37e740 1 --2- 172.21.15.82:0/1255338043 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564523ffc1b0 0x56452439b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:17:24.860 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:24.859+0000 7f4a7f37e740 1 --2- 172.21.15.82:0/1255338043 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564523fd32f0 0x5645243a5fe0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:17:24.860 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:24.859+0000 7f4a7f37e740 1 -- 172.21.15.82:0/1255338043 >> 172.21.15.82:0/1255338043 conn(0x564523ff0300 msgr2=0x56452439a890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:17:24.860 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:24.859+0000 7f4a7f37e740 1 -- 172.21.15.82:0/1255338043 shutdown_connections 2024-05-20T16:17:24.861 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:24.859+0000 7f4a7f37e740 1 -- 172.21.15.82:0/1255338043 wait complete. 2024-05-20T16:17:24.861 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:24.860+0000 7f4a7f37e740 1 -- start start 2024-05-20T16:17:24.861 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:24.860+0000 7f4a7f37e740 1 Processor -- start 2024-05-20T16:17:24.861 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:24.860+0000 7f4a7f37e740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564523fd32f0 0x5645243aa2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:17:24.861 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:24.861+0000 7f4a7627a640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564523fd32f0 0x5645243aa2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:17:24.861 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:24.861+0000 7f4a7627a640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564523fd32f0 0x5645243aa2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.82:49220/0 (socket says 172.21.15.82:49220) 2024-05-20T16:17:24.863 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:24.862+0000 7f4a7f37e740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564523fdb4d0 0x5645243aa7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:17:24.863 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:24.862+0000 7f4a7f37e740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564523ffc1b0 0x5645243ab720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:17:24.863 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:24.862+0000 7f4a7f37e740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f4a5400e220 con 0x564523ffc1b0 2024-05-20T16:17:24.863 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:24.862+0000 7f4a7f37e740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f4a5400f690 con 0x564523fdb4d0 2024-05-20T16:17:24.863 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:24.862+0000 7f4a7f37e740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f4a5400e8f0 con 0x564523fd32f0 2024-05-20T16:17:24.864 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:24.862+0000 7f4a7f37e740 10 client.0 _ll_register_callbacks cb 0x7f4a6809b440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T16:17:24.864 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:24.864+0000 7f4a7627a640 1 -- 172.21.15.82:0/3482849737 learned_addr learned my addr 172.21.15.82:0/3482849737 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:17:24.864 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:24.864+0000 7f4a7627a640 1 -- 172.21.15.82:0/3482849737 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564523fdb4d0 msgr2=0x5645243aa7f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-20T16:17:24.864 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:24.864+0000 7f4a7627a640 1 --2- 172.21.15.82:0/3482849737 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564523fdb4d0 0x5645243aa7f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:17:24.864 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:24.864+0000 7f4a7627a640 1 -- 172.21.15.82:0/3482849737 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564523ffc1b0 msgr2=0x5645243ab720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:17:24.864 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:24.864+0000 7f4a7627a640 1 --2- 172.21.15.82:0/3482849737 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564523ffc1b0 0x5645243ab720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:17:24.864 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:24.864+0000 7f4a7627a640 1 -- 172.21.15.82:0/3482849737 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5645243abce0 con 0x564523fd32f0 2024-05-20T16:17:24.865 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:24.865+0000 7f4a7627a640 1 --2- 172.21.15.82:0/3482849737 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564523fd32f0 0x5645243aa2b0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f4a6800b9b0 tx=0x7f4a6809a3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6de2481e0e51eba1 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:17:24.865 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:24.865+0000 7f4a59ffb640 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-05-20T16:17:24.865 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:24.865+0000 7f4a59ffb640 1 -- 172.21.15.82:0/3482849737 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a6807e060 con 0x564523fd32f0 2024-05-20T16:17:24.865 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:24.865+0000 7f4a59ffb640 1 -- 172.21.15.82:0/3482849737 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9150+0+0 (secure 0 0 0) 0x7f4a68096a90 con 0x564523fd32f0 2024-05-20T16:17:24.865 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:24.865+0000 7f4a7f37e740 1 -- 172.21.15.82:0/3482849737 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4a6809ad40 con 0x564523fd32f0 2024-05-20T16:17:24.866 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:24.866+0000 7f4a59ffb640 1 -- 172.21.15.82:0/3482849737 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f4a68096fe0 con 0x564523fd32f0 2024-05-20T16:17:24.866 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:24.866+0000 7f4a7f37e740 10 client.25228 fetch_fsmap learned FSMap version 27 2024-05-20T16:17:24.866 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:24.866+0000 7f4a7f37e740 1 -- 172.21.15.82:0/3482849737 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56452441bd40 con 0x564523fd32f0 2024-05-20T16:17:24.868 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:24.867+0000 7f4a59ffb640 1 -- 172.21.15.82:0/3482849737 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4a6809ad40 con 0x564523fd32f0 2024-05-20T16:17:24.868 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:24.867+0000 7f4a59ffb640 1 -- 172.21.15.82:0/3482849737 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f4a680aa020 con 0x564523fd32f0 2024-05-20T16:17:24.868 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:24.868+0000 7f4a7f37e740 10 client.25228 fetch_fsmap finished waiting for FSMap version 27 2024-05-20T16:17:24.868 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:24.868+0000 7f4a7f37e740 10 client.25228 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T16:17:24.868 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:24.868+0000 7f4a7f37e740 20 client.25228 populate_metadata read hostname 'smithi082' 2024-05-20T16:17:24.868 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:24.868+0000 7f4a7f37e740 1 --2- 172.21.15.82:0/3482849737 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x564523f901c0 0x56452441e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:17:24.868 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:24.868+0000 7f4a7f37e740 4 client.25228 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-20T16:17:24.868 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:24.868+0000 7f4a7f37e740 1 -- 172.21.15.82:0/3482849737 --> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564524307dc0 con 0x564523f901c0 2024-05-20T16:17:24.868 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:24.868+0000 7f4a75a79640 1 --2- 172.21.15.82:0/3482849737 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x564523f901c0 0x56452441e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:17:24.869 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:24.869+0000 7f4a75a79640 1 --2- 172.21.15.82:0/3482849737 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x564523f901c0 0x56452441e4c0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4238b1fd277934a8 server_cookie=817688859422176e in_seq=0 out_seq=0 2024-05-20T16:17:24.869 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:24.869+0000 7f4a59ffb640 10 client.25228 ms_handle_connect on v2:172.21.15.160:6838/2765554482 2024-05-20T16:17:25.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:17:25 smithi160 ceph-mon[36347]: pgmap v466: 129 pgs: 129 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 809 MiB/s wr, 1.43k op/s 2024-05-20T16:17:25.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:17:25 smithi082 ceph-mon[32238]: pgmap v466: 129 pgs: 129 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 809 MiB/s wr, 1.43k op/s 2024-05-20T16:17:25.885 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:17:25 smithi184 ceph-mon[36209]: pgmap v466: 129 pgs: 129 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 809 MiB/s wr, 1.43k op/s 2024-05-20T16:17:27.811 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:17:27 smithi160 ceph-mon[36347]: pgmap v467: 129 pgs: 129 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 760 MiB/s wr, 1.26k op/s 2024-05-20T16:17:27.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:17:27 smithi082 ceph-mon[32238]: pgmap v467: 129 pgs: 129 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 760 MiB/s wr, 1.26k op/s 2024-05-20T16:17:27.885 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:17:27 smithi184 ceph-mon[36209]: pgmap v467: 129 pgs: 129 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 760 MiB/s wr, 1.26k op/s 2024-05-20T16:17:29.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:17:29 smithi160 ceph-mon[36347]: pgmap v468: 129 pgs: 129 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 928 MiB/s wr, 1.46k op/s 2024-05-20T16:17:29.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:17:29 smithi082 ceph-mon[32238]: pgmap v468: 129 pgs: 129 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 928 MiB/s wr, 1.46k op/s 2024-05-20T16:17:29.885 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:17:29 smithi184 ceph-mon[36209]: pgmap v468: 129 pgs: 129 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 928 MiB/s wr, 1.46k op/s 2024-05-20T16:17:30.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:17:30 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T16:17:30.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:17:30 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T16:17:30.885 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:17:30 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T16:17:31.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:17:31 smithi160 ceph-mon[36347]: pgmap v469: 129 pgs: 129 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 778 MiB/s wr, 1.40k op/s 2024-05-20T16:17:31.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:17:31 smithi082 ceph-mon[32238]: pgmap v469: 129 pgs: 129 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 778 MiB/s wr, 1.40k op/s 2024-05-20T16:17:31.885 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:17:31 smithi184 ceph-mon[36209]: pgmap v469: 129 pgs: 129 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 778 MiB/s wr, 1.40k op/s 2024-05-20T16:17:32.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:17:32 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:17:32.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:17:32 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T16:17:32.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:17:32 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:17:32.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:17:32 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:17:32.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:17:32 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T16:17:32.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:17:32 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:17:32.885 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:17:32 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:17:32.885 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:17:32 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T16:17:32.885 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:17:32 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:17:33.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:17:33 smithi082 ceph-mon[32238]: pgmap v470: 129 pgs: 129 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 899 MiB/s wr, 1.65k op/s 2024-05-20T16:17:33.885 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:17:33 smithi184 ceph-mon[36209]: pgmap v470: 129 pgs: 129 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 899 MiB/s wr, 1.65k op/s 2024-05-20T16:17:34.062 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:17:33 smithi160 ceph-mon[36347]: pgmap v470: 129 pgs: 129 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 899 MiB/s wr, 1.65k op/s 2024-05-20T16:17:35.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:17:35 smithi160 ceph-mon[36347]: pgmap v471: 129 pgs: 129 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 748 MiB/s wr, 1.17k op/s 2024-05-20T16:17:35.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:17:35 smithi082 ceph-mon[32238]: pgmap v471: 129 pgs: 129 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 748 MiB/s wr, 1.17k op/s 2024-05-20T16:17:35.885 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:17:35 smithi184 ceph-mon[36209]: pgmap v471: 129 pgs: 129 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 748 MiB/s wr, 1.17k op/s 2024-05-20T16:17:37.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:17:37 smithi160 ceph-mon[36347]: pgmap v472: 129 pgs: 129 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 785 MiB/s wr, 1.19k op/s 2024-05-20T16:17:37.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:17:37 smithi082 ceph-mon[32238]: pgmap v472: 129 pgs: 129 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 785 MiB/s wr, 1.19k op/s 2024-05-20T16:17:37.885 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:17:37 smithi184 ceph-mon[36209]: pgmap v472: 129 pgs: 129 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 785 MiB/s wr, 1.19k op/s 2024-05-20T16:17:39.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:17:39 smithi160 ceph-mon[36347]: pgmap v473: 129 pgs: 129 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 934 MiB/s wr, 1.43k op/s 2024-05-20T16:17:39.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:17:39 smithi082 ceph-mon[32238]: pgmap v473: 129 pgs: 129 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 934 MiB/s wr, 1.43k op/s 2024-05-20T16:17:39.885 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:17:39 smithi184 ceph-mon[36209]: pgmap v473: 129 pgs: 129 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 934 MiB/s wr, 1.43k op/s 2024-05-20T16:17:41.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:17:41 smithi160 ceph-mon[36347]: pgmap v474: 129 pgs: 129 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 773 MiB/s wr, 1.25k op/s 2024-05-20T16:17:41.860 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:17:41 smithi082 ceph-mon[32238]: pgmap v474: 129 pgs: 129 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 773 MiB/s wr, 1.25k op/s 2024-05-20T16:17:41.885 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:17:41 smithi184 ceph-mon[36209]: pgmap v474: 129 pgs: 129 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 773 MiB/s wr, 1.25k op/s 2024-05-20T16:17:42.986 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:42.986+0000 7f4a59ffb640 1 -- 172.21.15.82:0/3482849737 <== mds.0 v2:172.21.15.160:6838/2765554482 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564524307dc0 con 0x564523f901c0 2024-05-20T16:17:42.986 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:42.986+0000 7f4a59ffb640 10 client.25228 handle_command_reply: tid=0 2024-05-20T16:17:42.986 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:42.986+0000 7f4a7f37e740 1 client.25228 shutdown 2024-05-20T16:17:42.989 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:42.989+0000 7f4a7f37e740 1 -- 172.21.15.82:0/3482849737 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564523fd32f0 msgr2=0x5645243aa2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:17:42.989 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:42.989+0000 7f4a7f37e740 1 --2- 172.21.15.82:0/3482849737 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564523fd32f0 0x5645243aa2b0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f4a6800b9b0 tx=0x7f4a6809a3c0 comp rx=0 tx=0).stop 2024-05-20T16:17:42.992 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:42.992+0000 7f4a7f37e740 1 -- 172.21.15.82:0/3482849737 shutdown_connections 2024-05-20T16:17:42.992 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:42.992+0000 7f4a7f37e740 1 --2- 172.21.15.82:0/3482849737 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x564523f901c0 0x56452441e4c0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:17:42.992 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:42.992+0000 7f4a7f37e740 1 --2- 172.21.15.82:0/3482849737 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564523ffc1b0 0x5645243ab720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:17:42.992 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:42.992+0000 7f4a7f37e740 1 --2- 172.21.15.82:0/3482849737 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564523fdb4d0 0x5645243aa7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:17:42.992 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:42.992+0000 7f4a7f37e740 1 --2- 172.21.15.82:0/3482849737 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564523fd32f0 0x5645243aa2b0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:17:42.992 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:42.992+0000 7f4a7f37e740 1 -- 172.21.15.82:0/3482849737 >> 172.21.15.82:0/3482849737 conn(0x564523ff0300 msgr2=0x56452439b2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:17:42.992 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:42.992+0000 7f4a59ffb640 0 client.25228 ms_handle_reset on v2:172.21.15.160:6838/2765554482 2024-05-20T16:17:42.992 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:42.992+0000 7f4a7f37e740 1 -- 172.21.15.82:0/3482849737 shutdown_connections 2024-05-20T16:17:42.992 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:42.992+0000 7f4a7f37e740 1 -- 172.21.15.82:0/3482849737 wait complete. 2024-05-20T16:17:42.993 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:42.993+0000 7f4a7f37e740 20 client.25228 trim_cache size 0 max 16384 2024-05-20T16:17:42.994 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:{ 2024-05-20T16:17:42.994 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "message": "", 2024-05-20T16:17:42.994 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "return_code": 0 2024-05-20T16:17:42.994 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:} 2024-05-20T16:17:43.006 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:43.006+0000 7f4a7727c640 1 -- 172.21.15.82:0/1734350440 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f4a5c0a2d50 msgr2=0x7f4a5c0a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:17:43.007 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:43.006+0000 7f4a7727c640 1 --2- 172.21.15.82:0/1734350440 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f4a5c0a2d50 0x7f4a5c0a5260 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f4a6c031610 tx=0x7f4a6c004430 comp rx=0 tx=0).stop 2024-05-20T16:17:43.007 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:43.006+0000 7f4a7727c640 1 -- 172.21.15.82:0/1734350440 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4a701011d0 msgr2=0x7f4a7019f800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:17:43.007 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:43.006+0000 7f4a7727c640 1 --2- 172.21.15.82:0/1734350440 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4a701011d0 0x7f4a7019f800 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f4a6800b9e0 tx=0x7f4a6800bde0 comp rx=0 tx=0).stop 2024-05-20T16:17:43.007 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:43.006+0000 7f4a7727c640 1 -- 172.21.15.82:0/1734350440 shutdown_connections 2024-05-20T16:17:43.007 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:43.006+0000 7f4a7727c640 1 --2- 172.21.15.82:0/1734350440 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f4a5c0a2d50 0x7f4a5c0a5260 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:17:43.007 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:43.006+0000 7f4a7727c640 1 --2- 172.21.15.82:0/1734350440 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4a70103b30 0x7f4a7019fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:17:43.007 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:43.006+0000 7f4a7727c640 1 --2- 172.21.15.82:0/1734350440 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4a70106490 0x7f4a701a6cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:17:43.007 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:43.007+0000 7f4a7727c640 1 --2- 172.21.15.82:0/1734350440 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4a701011d0 0x7f4a7019f800 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:17:43.007 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:43.007+0000 7f4a7727c640 1 -- 172.21.15.82:0/1734350440 >> 172.21.15.82:0/1734350440 conn(0x7f4a700fc9e0 msgr2=0x7f4a70102010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:17:43.007 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:43.007+0000 7f4a7727c640 1 -- 172.21.15.82:0/1734350440 shutdown_connections 2024-05-20T16:17:43.007 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:43.007+0000 7f4a7727c640 1 -- 172.21.15.82:0/1734350440 wait complete. 2024-05-20T16:17:43.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:17:43 smithi160 ceph-mon[36347]: pgmap v475: 129 pgs: 129 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 906 MiB/s wr, 1.51k op/s 2024-05-20T16:17:43.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:17:43 smithi082 ceph-mon[32238]: pgmap v475: 129 pgs: 129 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 906 MiB/s wr, 1.51k op/s 2024-05-20T16:17:43.885 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:17:43 smithi184 ceph-mon[36209]: pgmap v475: 129 pgs: 129 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 906 MiB/s wr, 1.51k op/s 2024-05-20T16:17:45.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:17:45 smithi160 ceph-mon[36347]: pgmap v476: 129 pgs: 129 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 817 MiB/s wr, 1.35k op/s 2024-05-20T16:17:45.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:17:45 smithi082 ceph-mon[32238]: pgmap v476: 129 pgs: 129 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 817 MiB/s wr, 1.35k op/s 2024-05-20T16:17:45.885 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:17:45 smithi184 ceph-mon[36209]: pgmap v476: 129 pgs: 129 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 817 MiB/s wr, 1.35k op/s 2024-05-20T16:17:47.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:17:47 smithi160 ceph-mon[36347]: pgmap v477: 129 pgs: 129 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 764 MiB/s wr, 1.26k op/s 2024-05-20T16:17:47.867 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:17:47 smithi082 ceph-mon[32238]: pgmap v477: 129 pgs: 129 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 764 MiB/s wr, 1.26k op/s 2024-05-20T16:17:47.886 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:17:47 smithi184 ceph-mon[36209]: pgmap v477: 129 pgs: 129 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 764 MiB/s wr, 1.26k op/s 2024-05-20T16:17:49.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:17:49 smithi160 ceph-mon[36347]: pgmap v478: 129 pgs: 129 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 907 MiB/s wr, 1.48k op/s 2024-05-20T16:17:49.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:17:49 smithi082 ceph-mon[32238]: pgmap v478: 129 pgs: 129 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 907 MiB/s wr, 1.48k op/s 2024-05-20T16:17:49.885 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:17:49 smithi184 ceph-mon[36209]: pgmap v478: 129 pgs: 129 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 907 MiB/s wr, 1.48k op/s 2024-05-20T16:17:51.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:17:51 smithi160 ceph-mon[36347]: pgmap v479: 129 pgs: 129 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 777 MiB/s wr, 1.29k op/s 2024-05-20T16:17:51.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:17:51 smithi082 ceph-mon[32238]: pgmap v479: 129 pgs: 129 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 777 MiB/s wr, 1.29k op/s 2024-05-20T16:17:51.885 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:17:51 smithi184 ceph-mon[36209]: pgmap v479: 129 pgs: 129 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 777 MiB/s wr, 1.29k op/s 2024-05-20T16:17:53.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:17:53 smithi160 ceph-mon[36347]: pgmap v480: 129 pgs: 129 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 890 MiB/s wr, 1.48k op/s 2024-05-20T16:17:53.864 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:17:53 smithi082 ceph-mon[32238]: pgmap v480: 129 pgs: 129 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 890 MiB/s wr, 1.48k op/s 2024-05-20T16:17:53.886 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:17:53 smithi184 ceph-mon[36209]: pgmap v480: 129 pgs: 129 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 890 MiB/s wr, 1.48k op/s 2024-05-20T16:17:55.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:17:55 smithi160 ceph-mon[36347]: pgmap v481: 129 pgs: 129 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 824 MiB/s wr, 1.32k op/s 2024-05-20T16:17:55.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:17:55 smithi082 ceph-mon[32238]: pgmap v481: 129 pgs: 129 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 824 MiB/s wr, 1.32k op/s 2024-05-20T16:17:55.885 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:17:55 smithi184 ceph-mon[36209]: pgmap v481: 129 pgs: 129 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 824 MiB/s wr, 1.32k op/s 2024-05-20T16:17:57.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:17:57 smithi160 ceph-mon[36347]: pgmap v482: 129 pgs: 129 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 764 MiB/s wr, 1.18k op/s 2024-05-20T16:17:57.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:17:57 smithi082 ceph-mon[32238]: pgmap v482: 129 pgs: 129 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 764 MiB/s wr, 1.18k op/s 2024-05-20T16:17:57.885 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:17:57 smithi184 ceph-mon[36209]: pgmap v482: 129 pgs: 129 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 764 MiB/s wr, 1.18k op/s 2024-05-20T16:17:59.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:17:59 smithi160 ceph-mon[36347]: pgmap v483: 129 pgs: 129 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 934 MiB/s wr, 1.43k op/s 2024-05-20T16:17:59.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:17:59 smithi082 ceph-mon[32238]: pgmap v483: 129 pgs: 129 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 934 MiB/s wr, 1.43k op/s 2024-05-20T16:17:59.885 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:17:59 smithi184 ceph-mon[36209]: pgmap v483: 129 pgs: 129 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 934 MiB/s wr, 1.43k op/s 2024-05-20T16:17:59.939 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:59.939+0000 7f6b92236640 1 -- 172.21.15.82:0/2781218683 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6b840ad6c0 msgr2=0x7f6b840afbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:17:59.939 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:59.939+0000 7f6b92236640 1 --2- 172.21.15.82:0/2781218683 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6b840ad6c0 0x7f6b840afbb0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f6b8c067640 tx=0x7f6b8c08dff0 comp rx=0 tx=0).stop 2024-05-20T16:17:59.944 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:59.943+0000 7f6b92236640 1 -- 172.21.15.82:0/2781218683 shutdown_connections 2024-05-20T16:17:59.944 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:59.943+0000 7f6b92236640 1 --2- 172.21.15.82:0/2781218683 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6b840ad6c0 0x7f6b840afbb0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:17:59.944 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:59.943+0000 7f6b92236640 1 --2- 172.21.15.82:0/2781218683 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6b840aad10 0x7f6b840ad130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:17:59.944 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:59.943+0000 7f6b92236640 1 --2- 172.21.15.82:0/2781218683 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6b840a5750 0x7f6b840a5b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:17:59.944 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:59.943+0000 7f6b92236640 1 -- 172.21.15.82:0/2781218683 >> 172.21.15.82:0/2781218683 conn(0x7f6b840a09b0 msgr2=0x7f6b840a2e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:17:59.951 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:59.949+0000 7f6b92236640 1 -- 172.21.15.82:0/2781218683 shutdown_connections 2024-05-20T16:17:59.951 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:59.949+0000 7f6b92236640 1 -- 172.21.15.82:0/2781218683 wait complete. 2024-05-20T16:17:59.951 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:59.951+0000 7f6b92236640 1 Processor -- start 2024-05-20T16:17:59.955 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:59.955+0000 7f6b92236640 1 -- start start 2024-05-20T16:17:59.955 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:59.955+0000 7f6b92236640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6b840a5750 0x7f6b8413bf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:17:59.956 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:59.955+0000 7f6b92236640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6b840aad10 0x7f6b8413c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:17:59.956 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:59.955+0000 7f6b92236640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6b840ad6c0 0x7f6b841546a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:17:59.956 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:59.955+0000 7f6b92236640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f6b840b57a0 con 0x7f6b840ad6c0 2024-05-20T16:17:59.956 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:59.955+0000 7f6b92236640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f6b840b5620 con 0x7f6b840aad10 2024-05-20T16:17:59.956 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:59.955+0000 7f6b92236640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f6b840b5920 con 0x7f6b840a5750 2024-05-20T16:17:59.961 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:59.956+0000 7f6b91234640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6b840a5750 0x7f6b8413bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:17:59.961 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:59.956+0000 7f6b91234640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6b840a5750 0x7f6b8413bf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.82:37532/0 (socket says 172.21.15.82:37532) 2024-05-20T16:17:59.961 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:59.956+0000 7f6b91a35640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6b840ad6c0 0x7f6b841546a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:17:59.961 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:59.956+0000 7f6b91234640 1 -- 172.21.15.82:0/1980581834 learned_addr learned my addr 172.21.15.82:0/1980581834 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:17:59.961 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:59.956+0000 7f6b90a33640 1 --2- 172.21.15.82:0/1980581834 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6b840aad10 0x7f6b8413c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:17:59.962 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:59.957+0000 7f6b91234640 1 -- 172.21.15.82:0/1980581834 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6b840aad10 msgr2=0x7f6b8413c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:17:59.962 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:59.957+0000 7f6b91234640 1 --2- 172.21.15.82:0/1980581834 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6b840aad10 0x7f6b8413c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:17:59.962 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:59.957+0000 7f6b91234640 1 -- 172.21.15.82:0/1980581834 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6b840ad6c0 msgr2=0x7f6b841546a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:17:59.962 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:59.957+0000 7f6b91234640 1 --2- 172.21.15.82:0/1980581834 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6b840ad6c0 0x7f6b841546a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:17:59.962 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:59.957+0000 7f6b91234640 1 -- 172.21.15.82:0/1980581834 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b84143440 con 0x7f6b840a5750 2024-05-20T16:17:59.962 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:59.957+0000 7f6b91234640 1 --2- 172.21.15.82:0/1980581834 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6b840a5750 0x7f6b8413bf40 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f6b8800d740 tx=0x7f6b8800db40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:17:59.962 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:59.957+0000 7f6b827fc640 1 -- 172.21.15.82:0/1980581834 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b88017070 con 0x7f6b840a5750 2024-05-20T16:17:59.962 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:59.957+0000 7f6b827fc640 1 -- 172.21.15.82:0/1980581834 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b88010460 con 0x7f6b840a5750 2024-05-20T16:17:59.962 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:59.957+0000 7f6b827fc640 1 -- 172.21.15.82:0/1980581834 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b8800f620 con 0x7f6b840a5750 2024-05-20T16:17:59.962 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:59.960+0000 7f6b92236640 1 -- 172.21.15.82:0/1980581834 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b84154ca0 con 0x7f6b840a5750 2024-05-20T16:17:59.963 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:59.960+0000 7f6b92236640 1 -- 172.21.15.82:0/1980581834 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b841551a0 con 0x7f6b840a5750 2024-05-20T16:17:59.967 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:59.964+0000 7f6b827fc640 1 -- 172.21.15.82:0/1980581834 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f6b8801a070 con 0x7f6b840a5750 2024-05-20T16:17:59.967 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:59.965+0000 7f6b827fc640 1 --2- 172.21.15.82:0/1980581834 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f6b780a2c90 0x7f6b780a51a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:17:59.967 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:59.966+0000 7f6b90a33640 1 --2- 172.21.15.82:0/1980581834 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f6b780a2c90 0x7f6b780a51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:17:59.967 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:59.966+0000 7f6b90a33640 1 --2- 172.21.15.82:0/1980581834 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f6b780a2c90 0x7f6b780a51a0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f6b8413cf90 tx=0x7f6b7c00a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:17:59.967 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:17:59.967+0000 7f6b827fc640 1 -- 172.21.15.82:0/1980581834 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9150+0+0 (secure 0 0 0) 0x7f6b880ced30 con 0x7f6b840a5750 2024-05-20T16:18:00.038 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:00.038+0000 7f6b9a416740 1 Processor -- start 2024-05-20T16:18:00.039 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:00.038+0000 7f6b9a416740 1 -- start start 2024-05-20T16:18:00.039 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:00.038+0000 7f6b9a416740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5579f64361b0 0x5579f67d6050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:18:00.039 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:00.038+0000 7f6b9a416740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5579f640d2f0 0x5579f67d65d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:18:00.043 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:00.038+0000 7f6b9a416740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5579f64154d0 0x5579f67dd4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:18:00.043 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:00.038+0000 7f6b9a416740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x5579f673a340 con 0x5579f64361b0 2024-05-20T16:18:00.043 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:00.038+0000 7f6b9a416740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x5579f66e2290 con 0x5579f64154d0 2024-05-20T16:18:00.043 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:00.038+0000 7f6b9a416740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5579f6736300 con 0x5579f640d2f0 2024-05-20T16:18:00.043 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:00.039+0000 7f6b90a33640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5579f640d2f0 0x5579f67d65d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:18:00.043 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:00.039+0000 7f6b90a33640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5579f640d2f0 0x5579f67d65d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.82:37544/0 (socket says 172.21.15.82:37544) 2024-05-20T16:18:00.043 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:00.039+0000 7f6b90a33640 1 -- 172.21.15.82:0/590484357 learned_addr learned my addr 172.21.15.82:0/590484357 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:18:00.043 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:00.039+0000 7f6b91a35640 1 --2- 172.21.15.82:0/590484357 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5579f64154d0 0x5579f67dd4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:18:00.044 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:00.040+0000 7f6b91234640 1 --2- 172.21.15.82:0/590484357 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5579f64361b0 0x5579f67d6050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:18:00.044 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:00.041+0000 7f6b90a33640 1 -- 172.21.15.82:0/590484357 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5579f64154d0 msgr2=0x5579f67dd4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:18:00.044 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:00.041+0000 7f6b90a33640 1 --2- 172.21.15.82:0/590484357 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5579f64154d0 0x5579f67dd4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:18:00.044 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:00.041+0000 7f6b90a33640 1 -- 172.21.15.82:0/590484357 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5579f64361b0 msgr2=0x5579f67d6050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:18:00.044 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:00.041+0000 7f6b90a33640 1 --2- 172.21.15.82:0/590484357 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5579f64361b0 0x5579f67d6050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:18:00.044 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:00.041+0000 7f6b90a33640 1 -- 172.21.15.82:0/590484357 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5579f67068e0 con 0x5579f640d2f0 2024-05-20T16:18:00.044 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:00.042+0000 7f6b90a33640 1 --2- 172.21.15.82:0/590484357 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5579f640d2f0 0x5579f67d65d0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f6b7c00ff90 tx=0x7f6b7c012000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=24c77bc035fdae86 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:18:00.046 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:00.045+0000 7f6b777fe640 1 -- 172.21.15.82:0/590484357 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b7c016070 con 0x5579f640d2f0 2024-05-20T16:18:00.047 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:00.045+0000 7f6b777fe640 1 -- 172.21.15.82:0/590484357 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b7c01b900 con 0x5579f640d2f0 2024-05-20T16:18:00.047 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:00.045+0000 7f6b777fe640 1 -- 172.21.15.82:0/590484357 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b7c01c9c0 con 0x5579f640d2f0 2024-05-20T16:18:00.047 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:00.046+0000 7f6b9a416740 1 -- 172.21.15.82:0/590484357 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5579f640d2f0 msgr2=0x5579f67d65d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:18:00.047 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:00.046+0000 7f6b9a416740 1 --2- 172.21.15.82:0/590484357 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5579f640d2f0 0x5579f67d65d0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f6b7c00ff90 tx=0x7f6b7c012000 comp rx=0 tx=0).stop 2024-05-20T16:18:00.048 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:00.048+0000 7f6b9a416740 1 -- 172.21.15.82:0/590484357 shutdown_connections 2024-05-20T16:18:00.048 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:00.048+0000 7f6b9a416740 1 --2- 172.21.15.82:0/590484357 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5579f64154d0 0x5579f67dd4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:18:00.048 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:00.048+0000 7f6b9a416740 1 --2- 172.21.15.82:0/590484357 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5579f640d2f0 0x5579f67d65d0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:18:00.048 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:00.048+0000 7f6b9a416740 1 --2- 172.21.15.82:0/590484357 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5579f64361b0 0x5579f67d6050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:18:00.048 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:00.048+0000 7f6b9a416740 1 -- 172.21.15.82:0/590484357 >> 172.21.15.82:0/590484357 conn(0x5579f642a300 msgr2=0x5579f67d19e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:18:00.051 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:00.051+0000 7f6b9a416740 1 -- 172.21.15.82:0/590484357 shutdown_connections 2024-05-20T16:18:00.051 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:00.051+0000 7f6b9a416740 1 -- 172.21.15.82:0/590484357 wait complete. 2024-05-20T16:18:00.056 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:00.052+0000 7f6b9a416740 1 -- start start 2024-05-20T16:18:00.056 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:00.052+0000 7f6b9a416740 1 Processor -- start 2024-05-20T16:18:00.056 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:00.053+0000 7f6b9a416740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5579f640d2f0 0x5579f67dfd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:18:00.056 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:00.053+0000 7f6b9a416740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5579f64154d0 0x5579f67e02c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:18:00.056 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:00.053+0000 7f6b9a416740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5579f64361b0 0x5579f67e7220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:18:00.056 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:00.053+0000 7f6b9a416740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x5579f67ddf20 con 0x5579f64154d0 2024-05-20T16:18:00.056 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:00.053+0000 7f6b9a416740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x5579f66f5790 con 0x5579f640d2f0 2024-05-20T16:18:00.056 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:00.053+0000 7f6b9a416740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5579f67de0a0 con 0x5579f64361b0 2024-05-20T16:18:00.056 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:00.053+0000 7f6b9a416740 10 client.0 _ll_register_callbacks cb 0x5579f6730a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T16:18:00.057 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:00.054+0000 7f6b91a35640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5579f64361b0 0x5579f67e7220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:18:00.057 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:00.054+0000 7f6b91a35640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5579f64361b0 0x5579f67e7220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.82:37558/0 (socket says 172.21.15.82:37558) 2024-05-20T16:18:00.057 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:00.054+0000 7f6b91a35640 1 -- 172.21.15.82:0/1096190514 learned_addr learned my addr 172.21.15.82:0/1096190514 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:18:00.057 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:00.055+0000 7f6b91a35640 1 -- 172.21.15.82:0/1096190514 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5579f640d2f0 msgr2=0x5579f67dfd80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-20T16:18:00.057 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:00.055+0000 7f6b91a35640 1 --2- 172.21.15.82:0/1096190514 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5579f640d2f0 0x5579f67dfd80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:18:00.057 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:00.055+0000 7f6b91a35640 1 -- 172.21.15.82:0/1096190514 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5579f64154d0 msgr2=0x5579f67e02c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:18:00.057 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:00.055+0000 7f6b91a35640 1 --2- 172.21.15.82:0/1096190514 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5579f64154d0 0x5579f67e02c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:18:00.057 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:00.055+0000 7f6b91a35640 1 -- 172.21.15.82:0/1096190514 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5579f644e210 con 0x5579f64361b0 2024-05-20T16:18:00.057 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:00.055+0000 7f6b91a35640 1 --2- 172.21.15.82:0/1096190514 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5579f64361b0 0x5579f67e7220 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f6b8c067560 tx=0x7f6b8c067f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8524c3884cb3c05 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:18:00.068 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:00.067+0000 7f6b74ff9640 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-05-20T16:18:00.068 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:00.067+0000 7f6b74ff9640 1 -- 172.21.15.82:0/1096190514 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b8c06a460 con 0x5579f64361b0 2024-05-20T16:18:00.068 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:00.067+0000 7f6b9a416740 1 -- 172.21.15.82:0/1096190514 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6b7c01b900 con 0x5579f64361b0 2024-05-20T16:18:00.068 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:00.067+0000 7f6b74ff9640 1 -- 172.21.15.82:0/1096190514 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9150+0+0 (secure 0 0 0) 0x7f6b8c08e8f0 con 0x5579f64361b0 2024-05-20T16:18:00.069 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:00.069+0000 7f6b74ff9640 1 -- 172.21.15.82:0/1096190514 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f6b8c08eb30 con 0x5579f64361b0 2024-05-20T16:18:00.076 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:00.072+0000 7f6b9a416740 10 client.25237 fetch_fsmap learned FSMap version 27 2024-05-20T16:18:00.076 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:00.072+0000 7f6b9a416740 1 -- 172.21.15.82:0/1096190514 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5579f651a620 con 0x5579f64361b0 2024-05-20T16:18:00.080 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:00.080+0000 7f6b74ff9640 1 -- 172.21.15.82:0/1096190514 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b7c01b900 con 0x5579f64361b0 2024-05-20T16:18:00.080 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:00.080+0000 7f6b74ff9640 1 -- 172.21.15.82:0/1096190514 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f6b8c0ac7e0 con 0x5579f64361b0 2024-05-20T16:18:00.085 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:00.084+0000 7f6b9a416740 10 client.25237 fetch_fsmap finished waiting for FSMap version 27 2024-05-20T16:18:00.085 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:00.084+0000 7f6b9a416740 10 client.25237 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T16:18:00.085 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:00.084+0000 7f6b9a416740 20 client.25237 populate_metadata read hostname 'smithi082' 2024-05-20T16:18:00.085 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:00.084+0000 7f6b9a416740 1 --2- 172.21.15.82:0/1096190514 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x5579f63f4d80 0x5579f67ed780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:18:00.085 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:00.084+0000 7f6b9a416740 4 client.25237 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T16:18:00.085 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:00.084+0000 7f6b9a416740 1 -- 172.21.15.82:0/1096190514 --> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5579f6747a70 con 0x5579f63f4d80 2024-05-20T16:18:00.090 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:00.088+0000 7f6b91234640 1 --2- 172.21.15.82:0/1096190514 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x5579f63f4d80 0x5579f67ed780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:18:00.107 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:00.106+0000 7f6b91234640 1 --2- 172.21.15.82:0/1096190514 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x5579f63f4d80 0x5579f67ed780 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7b39162cd4db36a6 server_cookie=84fd86122ce64b29 in_seq=0 out_seq=0 2024-05-20T16:18:00.108 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:00.107+0000 7f6b74ff9640 10 client.25237 ms_handle_connect on v2:172.21.15.160:6838/2765554482 2024-05-20T16:18:00.116 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:00.115+0000 7f6b74ff9640 1 -- 172.21.15.82:0/1096190514 <== mds.0 v2:172.21.15.160:6838/2765554482 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5579f6747a70 con 0x5579f63f4d80 2024-05-20T16:18:00.116 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:00.115+0000 7f6b74ff9640 10 client.25237 handle_command_reply: tid=0 2024-05-20T16:18:00.116 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:00.116+0000 7f6b9a416740 1 client.25237 shutdown 2024-05-20T16:18:00.117 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:00.116+0000 7f6b9a416740 1 -- 172.21.15.82:0/1096190514 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5579f64361b0 msgr2=0x5579f67e7220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:18:00.117 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:00.116+0000 7f6b9a416740 1 --2- 172.21.15.82:0/1096190514 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5579f64361b0 0x5579f67e7220 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f6b8c067560 tx=0x7f6b8c067f90 comp rx=0 tx=0).stop 2024-05-20T16:18:00.117 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:00.116+0000 7f6b9a416740 1 -- 172.21.15.82:0/1096190514 shutdown_connections 2024-05-20T16:18:00.117 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:00.116+0000 7f6b9a416740 1 --2- 172.21.15.82:0/1096190514 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x5579f63f4d80 0x5579f67ed780 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:18:00.117 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:00.116+0000 7f6b9a416740 1 --2- 172.21.15.82:0/1096190514 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5579f64154d0 0x5579f67e02c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:18:00.117 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:00.116+0000 7f6b9a416740 1 --2- 172.21.15.82:0/1096190514 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5579f64361b0 0x5579f67e7220 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:18:00.117 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:00.116+0000 7f6b9a416740 1 --2- 172.21.15.82:0/1096190514 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5579f640d2f0 0x5579f67dfd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:18:00.118 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:00.116+0000 7f6b9a416740 1 -- 172.21.15.82:0/1096190514 >> 172.21.15.82:0/1096190514 conn(0x5579f642a300 msgr2=0x5579f67d4990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:18:00.118 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:00.117+0000 7f6b74ff9640 0 client.25237 ms_handle_reset on v2:172.21.15.160:6838/2765554482 2024-05-20T16:18:00.120 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:00.121+0000 7f6b9a416740 1 -- 172.21.15.82:0/1096190514 shutdown_connections 2024-05-20T16:18:00.124 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:00.124+0000 7f6b9a416740 1 -- 172.21.15.82:0/1096190514 wait complete. 2024-05-20T16:18:00.126 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:00.126+0000 7f6b9a416740 20 client.25237 trim_cache size 0 max 16384 2024-05-20T16:18:00.188 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:00.188+0000 7f6b9a416740 1 Processor -- start 2024-05-20T16:18:00.189 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:00.189+0000 7f6b9a416740 1 -- start start 2024-05-20T16:18:00.190 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:00.190+0000 7f6b9a416740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5579f640d2f0 0x5579f67d4930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:18:00.190 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:00.190+0000 7f6b9a416740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5579f64154d0 0x5579f67e7140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:18:00.190 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:00.190+0000 7f6b9a416740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5579f64361b0 0x5579f67d4e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:18:00.190 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:00.190+0000 7f6b9a416740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f6b84162060 con 0x5579f64361b0 2024-05-20T16:18:00.190 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:00.190+0000 7f6b9a416740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f6b8415f930 con 0x5579f64154d0 2024-05-20T16:18:00.190 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:00.190+0000 7f6b9a416740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f6b8415d780 con 0x5579f640d2f0 2024-05-20T16:18:00.191 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:00.191+0000 7f6b91a35640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5579f64361b0 0x5579f67d4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:18:00.191 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:00.191+0000 7f6b91a35640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5579f64361b0 0x5579f67d4e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:52574/0 (socket says 172.21.15.82:52574) 2024-05-20T16:18:00.191 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:00.191+0000 7f6b91a35640 1 -- 172.21.15.82:0/4165333534 learned_addr learned my addr 172.21.15.82:0/4165333534 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:18:00.192 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:00.192+0000 7f6b90a33640 1 --2- 172.21.15.82:0/4165333534 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5579f64154d0 0x5579f67e7140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:18:00.192 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:00.192+0000 7f6b91a35640 1 -- 172.21.15.82:0/4165333534 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5579f640d2f0 msgr2=0x5579f67d4930 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-20T16:18:00.192 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:00.192+0000 7f6b91a35640 1 --2- 172.21.15.82:0/4165333534 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5579f640d2f0 0x5579f67d4930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:18:00.193 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:00.192+0000 7f6b91a35640 1 -- 172.21.15.82:0/4165333534 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5579f64154d0 msgr2=0x5579f67e7140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:18:00.193 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:00.192+0000 7f6b91a35640 1 --2- 172.21.15.82:0/4165333534 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5579f64154d0 0x5579f67e7140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:18:00.193 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:00.192+0000 7f6b91a35640 1 -- 172.21.15.82:0/4165333534 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5579f67d6e40 con 0x5579f64361b0 2024-05-20T16:18:00.200 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:00.200+0000 7f6b91a35640 1 --2- 172.21.15.82:0/4165333534 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5579f64361b0 0x5579f67d4e70 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f6b8c0674d0 tx=0x7f6b8c069fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c5c89468d002737 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:18:00.201 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:00.201+0000 7f6b76ffd640 1 -- 172.21.15.82:0/4165333534 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b8c06a460 con 0x5579f64361b0 2024-05-20T16:18:00.201 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:00.202+0000 7f6b76ffd640 1 -- 172.21.15.82:0/4165333534 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b8c092040 con 0x5579f64361b0 2024-05-20T16:18:00.203 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:00.203+0000 7f6b9a416740 1 -- 172.21.15.82:0/4165333534 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5579f64361b0 msgr2=0x5579f67d4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:18:00.203 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:00.203+0000 7f6b9a416740 1 --2- 172.21.15.82:0/4165333534 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5579f64361b0 0x5579f67d4e70 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f6b8c0674d0 tx=0x7f6b8c069fe0 comp rx=0 tx=0).stop 2024-05-20T16:18:00.204 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:00.203+0000 7f6b9a416740 1 -- 172.21.15.82:0/4165333534 shutdown_connections 2024-05-20T16:18:00.204 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:00.203+0000 7f6b9a416740 1 --2- 172.21.15.82:0/4165333534 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5579f64361b0 0x5579f67d4e70 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:18:00.204 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:00.204+0000 7f6b9a416740 1 --2- 172.21.15.82:0/4165333534 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5579f64154d0 0x5579f67e7140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:18:00.204 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:00.204+0000 7f6b9a416740 1 --2- 172.21.15.82:0/4165333534 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5579f640d2f0 0x5579f67d4930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:18:00.204 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:00.204+0000 7f6b9a416740 1 -- 172.21.15.82:0/4165333534 >> 172.21.15.82:0/4165333534 conn(0x5579f642a300 msgr2=0x5579f67da1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:18:00.211 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:00.211+0000 7f6b9a416740 1 -- 172.21.15.82:0/4165333534 shutdown_connections 2024-05-20T16:18:00.215 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:00.215+0000 7f6b9a416740 1 -- 172.21.15.82:0/4165333534 wait complete. 2024-05-20T16:18:00.216 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:00.215+0000 7f6b9a416740 1 -- start start 2024-05-20T16:18:00.216 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:00.216+0000 7f6b9a416740 1 Processor -- start 2024-05-20T16:18:00.217 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:00.217+0000 7f6b9a416740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5579f640d2f0 0x5579f67e43b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:18:00.218 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:00.217+0000 7f6b9a416740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5579f64154d0 0x5579f67e48f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:18:00.218 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:00.217+0000 7f6b9a416740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5579f63ca1c0 0x5579f67e5820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:18:00.218 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:00.218+0000 7f6b9a416740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f6b8415ddf0 con 0x5579f640d2f0 2024-05-20T16:18:00.218 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:00.218+0000 7f6b9a416740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f6b8415f260 con 0x5579f63ca1c0 2024-05-20T16:18:00.218 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:00.218+0000 7f6b9a416740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f6b8415e4c0 con 0x5579f64154d0 2024-05-20T16:18:00.219 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:00.218+0000 7f6b9a416740 10 client.0 _ll_register_callbacks cb 0x7f6b8c068220 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T16:18:00.219 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:00.219+0000 7f6b91234640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5579f640d2f0 0x5579f67e43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:18:00.219 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:00.219+0000 7f6b91234640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5579f640d2f0 0x5579f67e43b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:52584/0 (socket says 172.21.15.82:52584) 2024-05-20T16:18:00.219 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:00.219+0000 7f6b91234640 1 -- 172.21.15.82:0/4182919767 learned_addr learned my addr 172.21.15.82:0/4182919767 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:18:00.220 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:00.219+0000 7f6b91234640 1 -- 172.21.15.82:0/4182919767 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5579f64154d0 msgr2=0x5579f67e48f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:18:00.220 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:00.219+0000 7f6b91234640 1 --2- 172.21.15.82:0/4182919767 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5579f64154d0 0x5579f67e48f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:18:00.220 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:00.219+0000 7f6b91234640 1 -- 172.21.15.82:0/4182919767 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5579f63ca1c0 msgr2=0x5579f67e5820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:18:00.220 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:00.219+0000 7f6b91234640 1 --2- 172.21.15.82:0/4182919767 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5579f63ca1c0 0x5579f67e5820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:18:00.220 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:00.220+0000 7f6b91234640 1 -- 172.21.15.82:0/4182919767 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5579f67e5de0 con 0x5579f640d2f0 2024-05-20T16:18:00.220 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:00.220+0000 7f6b91234640 1 --2- 172.21.15.82:0/4182919767 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5579f640d2f0 0x5579f67e43b0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f6b880cc440 tx=0x7f6b880108f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3395de401a0db6b6 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:18:00.220 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:00.220+0000 7f6b74ff9640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-05-20T16:18:00.225 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:00.225+0000 7f6b74ff9640 1 -- 172.21.15.82:0/4182919767 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b880b4060 con 0x5579f640d2f0 2024-05-20T16:18:00.225 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:00.225+0000 7f6b9a416740 1 -- 172.21.15.82:0/4182919767 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5579f6741dc0 con 0x5579f640d2f0 2024-05-20T16:18:00.225 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:00.225+0000 7f6b74ff9640 1 -- 172.21.15.82:0/4182919767 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9150+0+0 (secure 0 0 0) 0x7f6b88010610 con 0x5579f640d2f0 2024-05-20T16:18:00.227 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:00.227+0000 7f6b74ff9640 1 -- 172.21.15.82:0/4182919767 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f6b88014030 con 0x5579f640d2f0 2024-05-20T16:18:00.227 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:00.228+0000 7f6b9a416740 10 client.15687 fetch_fsmap learned FSMap version 27 2024-05-20T16:18:00.227 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:00.228+0000 7f6b9a416740 1 -- 172.21.15.82:0/4182919767 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5579f67e7170 con 0x5579f640d2f0 2024-05-20T16:18:00.230 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:00.230+0000 7f6b74ff9640 1 -- 172.21.15.82:0/4182919767 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5579f6741dc0 con 0x5579f640d2f0 2024-05-20T16:18:00.230 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:00.230+0000 7f6b74ff9640 1 -- 172.21.15.82:0/4182919767 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f6b8809f170 con 0x5579f640d2f0 2024-05-20T16:18:00.232 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:00.232+0000 7f6b9a416740 10 client.15687 fetch_fsmap finished waiting for FSMap version 27 2024-05-20T16:18:00.232 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:00.232+0000 7f6b9a416740 10 client.15687 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T16:18:00.232 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:00.232+0000 7f6b9a416740 20 client.15687 populate_metadata read hostname 'smithi082' 2024-05-20T16:18:00.232 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:00.232+0000 7f6b9a416740 1 --2- 172.21.15.82:0/4182919767 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x5579f6855fc0 0x5579f6858450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:18:00.232 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:00.232+0000 7f6b9a416740 4 client.15687 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-20T16:18:00.232 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:00.232+0000 7f6b9a416740 1 -- 172.21.15.82:0/4182919767 --> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5579f66bf8e0 con 0x5579f6855fc0 2024-05-20T16:18:00.240 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:00.239+0000 7f6b90a33640 1 --2- 172.21.15.82:0/4182919767 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x5579f6855fc0 0x5579f6858450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:18:00.259 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:00.259+0000 7f6b90a33640 1 --2- 172.21.15.82:0/4182919767 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x5579f6855fc0 0x5579f6858450 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a6fef6423db9ad90 server_cookie=9ed894d13ec4a78a in_seq=0 out_seq=0 2024-05-20T16:18:00.261 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:00.260+0000 7f6b74ff9640 10 client.15687 ms_handle_connect on v2:172.21.15.160:6838/2765554482 2024-05-20T16:18:01.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:18:01 smithi160 ceph-mon[36347]: pgmap v484: 129 pgs: 129 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 790 MiB/s wr, 1.24k op/s 2024-05-20T16:18:01.873 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:18:01 smithi082 ceph-mon[32238]: pgmap v484: 129 pgs: 129 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 790 MiB/s wr, 1.24k op/s 2024-05-20T16:18:01.885 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:18:01 smithi184 ceph-mon[36209]: pgmap v484: 129 pgs: 129 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 790 MiB/s wr, 1.24k op/s 2024-05-20T16:18:03.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:18:03 smithi160 ceph-mon[36347]: pgmap v485: 129 pgs: 129 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 832 MiB/s wr, 1.42k op/s 2024-05-20T16:18:03.860 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:18:03 smithi082 ceph-mon[32238]: pgmap v485: 129 pgs: 129 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 832 MiB/s wr, 1.42k op/s 2024-05-20T16:18:03.885 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:18:03 smithi184 ceph-mon[36209]: pgmap v485: 129 pgs: 129 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 832 MiB/s wr, 1.42k op/s 2024-05-20T16:18:05.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:18:05 smithi160 ceph-mon[36347]: pgmap v486: 129 pgs: 129 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 750 MiB/s wr, 1.36k op/s 2024-05-20T16:18:05.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:18:05 smithi082 ceph-mon[32238]: pgmap v486: 129 pgs: 129 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 750 MiB/s wr, 1.36k op/s 2024-05-20T16:18:05.885 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:18:05 smithi184 ceph-mon[36209]: pgmap v486: 129 pgs: 129 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 750 MiB/s wr, 1.36k op/s 2024-05-20T16:18:07.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:18:07 smithi082 ceph-mon[32238]: pgmap v487: 129 pgs: 129 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 688 MiB/s wr, 1.29k op/s 2024-05-20T16:18:07.885 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:18:07 smithi184 ceph-mon[36209]: pgmap v487: 129 pgs: 129 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 688 MiB/s wr, 1.29k op/s 2024-05-20T16:18:08.062 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:18:07 smithi160 ceph-mon[36347]: pgmap v487: 129 pgs: 129 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 688 MiB/s wr, 1.29k op/s 2024-05-20T16:18:09.862 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:18:09 smithi082 ceph-mon[32238]: pgmap v488: 129 pgs: 129 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 846 MiB/s wr, 1.46k op/s 2024-05-20T16:18:09.885 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:18:09 smithi184 ceph-mon[36209]: pgmap v488: 129 pgs: 129 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 846 MiB/s wr, 1.46k op/s 2024-05-20T16:18:10.062 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:18:09 smithi160 ceph-mon[36347]: pgmap v488: 129 pgs: 129 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 846 MiB/s wr, 1.46k op/s 2024-05-20T16:18:11.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:18:11 smithi160 ceph-mon[36347]: pgmap v489: 129 pgs: 129 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 695 MiB/s wr, 1.21k op/s 2024-05-20T16:18:11.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:18:11 smithi082 ceph-mon[32238]: pgmap v489: 129 pgs: 129 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 695 MiB/s wr, 1.21k op/s 2024-05-20T16:18:11.885 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:18:11 smithi184 ceph-mon[36209]: pgmap v489: 129 pgs: 129 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 695 MiB/s wr, 1.21k op/s 2024-05-20T16:18:13.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:18:13 smithi082 ceph-mon[32238]: pgmap v490: 129 pgs: 129 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 844 MiB/s wr, 1.45k op/s 2024-05-20T16:18:13.885 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:18:13 smithi184 ceph-mon[36209]: pgmap v490: 129 pgs: 129 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 844 MiB/s wr, 1.45k op/s 2024-05-20T16:18:14.062 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:18:13 smithi160 ceph-mon[36347]: pgmap v490: 129 pgs: 129 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 844 MiB/s wr, 1.45k op/s 2024-05-20T16:18:15.860 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:18:15 smithi082 ceph-mon[32238]: pgmap v491: 129 pgs: 129 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 765 MiB/s wr, 1.26k op/s 2024-05-20T16:18:15.885 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:18:15 smithi184 ceph-mon[36209]: pgmap v491: 129 pgs: 129 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 765 MiB/s wr, 1.26k op/s 2024-05-20T16:18:16.062 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:18:15 smithi160 ceph-mon[36347]: pgmap v491: 129 pgs: 129 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 765 MiB/s wr, 1.26k op/s 2024-05-20T16:18:17.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:18:17 smithi082 ceph-mon[32238]: pgmap v492: 129 pgs: 129 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 778 MiB/s wr, 1.23k op/s 2024-05-20T16:18:17.885 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:18:17 smithi184 ceph-mon[36209]: pgmap v492: 129 pgs: 129 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 778 MiB/s wr, 1.23k op/s 2024-05-20T16:18:18.062 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:18:17 smithi160 ceph-mon[36347]: pgmap v492: 129 pgs: 129 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 778 MiB/s wr, 1.23k op/s 2024-05-20T16:18:19.860 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:18:19 smithi082 ceph-mon[32238]: pgmap v493: 129 pgs: 129 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 877 MiB/s wr, 1.57k op/s 2024-05-20T16:18:19.885 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:18:19 smithi184 ceph-mon[36209]: pgmap v493: 129 pgs: 129 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 877 MiB/s wr, 1.57k op/s 2024-05-20T16:18:20.062 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:18:19 smithi160 ceph-mon[36347]: pgmap v493: 129 pgs: 129 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 877 MiB/s wr, 1.57k op/s 2024-05-20T16:18:21.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:18:21 smithi082 ceph-mon[32238]: pgmap v494: 129 pgs: 129 active+clean; 10 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 726 MiB/s wr, 1.48k op/s 2024-05-20T16:18:21.885 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:18:21 smithi184 ceph-mon[36209]: pgmap v494: 129 pgs: 129 active+clean; 10 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 726 MiB/s wr, 1.48k op/s 2024-05-20T16:18:22.062 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:18:21 smithi160 ceph-mon[36347]: pgmap v494: 129 pgs: 129 active+clean; 10 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 726 MiB/s wr, 1.48k op/s 2024-05-20T16:18:23.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:18:23 smithi082 ceph-mon[32238]: pgmap v495: 129 pgs: 129 active+clean; 10 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 865 MiB/s wr, 1.67k op/s 2024-05-20T16:18:23.885 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:18:23 smithi184 ceph-mon[36209]: pgmap v495: 129 pgs: 129 active+clean; 10 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 865 MiB/s wr, 1.67k op/s 2024-05-20T16:18:24.062 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:18:23 smithi160 ceph-mon[36347]: pgmap v495: 129 pgs: 129 active+clean; 10 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 865 MiB/s wr, 1.67k op/s 2024-05-20T16:18:25.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:18:25 smithi082 ceph-mon[32238]: pgmap v496: 129 pgs: 129 active+clean; 10 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 695 MiB/s wr, 1.34k op/s 2024-05-20T16:18:25.885 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:18:25 smithi184 ceph-mon[36209]: pgmap v496: 129 pgs: 129 active+clean; 10 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 695 MiB/s wr, 1.34k op/s 2024-05-20T16:18:26.062 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:18:25 smithi160 ceph-mon[36347]: pgmap v496: 129 pgs: 129 active+clean; 10 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 695 MiB/s wr, 1.34k op/s 2024-05-20T16:18:27.426 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:27.426+0000 7f6b74ff9640 1 -- 172.21.15.82:0/4182919767 <== mds.0 v2:172.21.15.160:6838/2765554482 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5579f66bf8e0 con 0x5579f6855fc0 2024-05-20T16:18:27.426 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:27.426+0000 7f6b74ff9640 10 client.15687 handle_command_reply: tid=0 2024-05-20T16:18:27.426 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:27.426+0000 7f6b9a416740 1 client.15687 shutdown 2024-05-20T16:18:27.427 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:27.426+0000 7f6b9a416740 1 -- 172.21.15.82:0/4182919767 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5579f640d2f0 msgr2=0x5579f67e43b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:18:27.427 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:27.426+0000 7f6b9a416740 1 --2- 172.21.15.82:0/4182919767 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5579f640d2f0 0x5579f67e43b0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f6b880cc440 tx=0x7f6b880108f0 comp rx=0 tx=0).stop 2024-05-20T16:18:27.427 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:27.427+0000 7f6b9a416740 1 -- 172.21.15.82:0/4182919767 shutdown_connections 2024-05-20T16:18:27.427 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:27.427+0000 7f6b9a416740 1 --2- 172.21.15.82:0/4182919767 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x5579f6855fc0 0x5579f6858450 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:18:27.427 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:27.427+0000 7f6b9a416740 1 --2- 172.21.15.82:0/4182919767 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5579f63ca1c0 0x5579f67e5820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:18:27.427 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:27.427+0000 7f6b9a416740 1 --2- 172.21.15.82:0/4182919767 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5579f64154d0 0x5579f67e48f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:18:27.428 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:27.427+0000 7f6b9a416740 1 --2- 172.21.15.82:0/4182919767 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5579f640d2f0 0x5579f67e43b0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:18:27.428 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:27.427+0000 7f6b9a416740 1 -- 172.21.15.82:0/4182919767 >> 172.21.15.82:0/4182919767 conn(0x5579f642a300 msgr2=0x5579f67d4930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:18:27.428 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:27.427+0000 7f6b74ff9640 0 client.15687 ms_handle_reset on v2:172.21.15.160:6838/2765554482 2024-05-20T16:18:27.428 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:27.427+0000 7f6b9a416740 1 -- 172.21.15.82:0/4182919767 shutdown_connections 2024-05-20T16:18:27.428 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:27.427+0000 7f6b9a416740 1 -- 172.21.15.82:0/4182919767 wait complete. 2024-05-20T16:18:27.428 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:27.428+0000 7f6b9a416740 20 client.15687 trim_cache size 0 max 16384 2024-05-20T16:18:27.428 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:{ 2024-05-20T16:18:27.428 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "message": "", 2024-05-20T16:18:27.428 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "return_code": 0 2024-05-20T16:18:27.428 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:} 2024-05-20T16:18:27.433 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:27.433+0000 7f6b92236640 1 -- 172.21.15.82:0/1980581834 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f6b780a2c90 msgr2=0x7f6b780a51a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:18:27.433 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:27.433+0000 7f6b92236640 1 --2- 172.21.15.82:0/1980581834 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f6b780a2c90 0x7f6b780a51a0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f6b8413cf90 tx=0x7f6b7c00a2d0 comp rx=0 tx=0).stop 2024-05-20T16:18:27.434 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:27.433+0000 7f6b92236640 1 -- 172.21.15.82:0/1980581834 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6b840a5750 msgr2=0x7f6b8413bf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:18:27.434 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:27.433+0000 7f6b92236640 1 --2- 172.21.15.82:0/1980581834 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6b840a5750 0x7f6b8413bf40 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f6b8800d740 tx=0x7f6b8800db40 comp rx=0 tx=0).stop 2024-05-20T16:18:27.434 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:27.433+0000 7f6b92236640 1 -- 172.21.15.82:0/1980581834 shutdown_connections 2024-05-20T16:18:27.434 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:27.433+0000 7f6b92236640 1 --2- 172.21.15.82:0/1980581834 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f6b780a2c90 0x7f6b780a51a0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:18:27.434 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:27.434+0000 7f6b92236640 1 --2- 172.21.15.82:0/1980581834 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6b840ad6c0 0x7f6b841546a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:18:27.434 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:27.434+0000 7f6b92236640 1 --2- 172.21.15.82:0/1980581834 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6b840aad10 0x7f6b8413c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:18:27.434 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:27.434+0000 7f6b92236640 1 --2- 172.21.15.82:0/1980581834 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6b840a5750 0x7f6b8413bf40 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:18:27.434 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:27.434+0000 7f6b92236640 1 -- 172.21.15.82:0/1980581834 >> 172.21.15.82:0/1980581834 conn(0x7f6b840a09b0 msgr2=0x7f6b840abbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:18:27.435 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:27.435+0000 7f6b92236640 1 -- 172.21.15.82:0/1980581834 shutdown_connections 2024-05-20T16:18:27.435 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:27.435+0000 7f6b92236640 1 -- 172.21.15.82:0/1980581834 wait complete. 2024-05-20T16:18:27.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:18:27 smithi082 ceph-mon[32238]: pgmap v497: 129 pgs: 129 active+clean; 10 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 693 MiB/s wr, 1.26k op/s 2024-05-20T16:18:27.885 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:18:27 smithi184 ceph-mon[36209]: pgmap v497: 129 pgs: 129 active+clean; 10 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 693 MiB/s wr, 1.26k op/s 2024-05-20T16:18:28.062 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:18:27 smithi160 ceph-mon[36347]: pgmap v497: 129 pgs: 129 active+clean; 10 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 693 MiB/s wr, 1.26k op/s 2024-05-20T16:18:29.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:18:29 smithi082 ceph-mon[32238]: pgmap v498: 129 pgs: 129 active+clean; 10 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 818 MiB/s wr, 1.42k op/s 2024-05-20T16:18:29.863 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:18:29 smithi160 ceph-mon[36347]: pgmap v498: 129 pgs: 129 active+clean; 10 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 818 MiB/s wr, 1.42k op/s 2024-05-20T16:18:29.885 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:18:29 smithi184 ceph-mon[36209]: pgmap v498: 129 pgs: 129 active+clean; 10 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 818 MiB/s wr, 1.42k op/s 2024-05-20T16:18:31.861 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:18:31 smithi082 ceph-mon[32238]: pgmap v499: 129 pgs: 129 active+clean; 10 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 725 MiB/s wr, 1.08k op/s 2024-05-20T16:18:31.885 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:18:31 smithi184 ceph-mon[36209]: pgmap v499: 129 pgs: 129 active+clean; 10 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 725 MiB/s wr, 1.08k op/s 2024-05-20T16:18:32.062 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:18:31 smithi160 ceph-mon[36347]: pgmap v499: 129 pgs: 129 active+clean; 10 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 725 MiB/s wr, 1.08k op/s 2024-05-20T16:18:32.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:18:32 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T16:18:32.886 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:18:32 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T16:18:33.062 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:18:32 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T16:18:34.062 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:18:33 smithi160 ceph-mon[36347]: pgmap v500: 129 pgs: 129 active+clean; 10 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 876 MiB/s wr, 1.26k op/s 2024-05-20T16:18:34.119 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:18:33 smithi082 ceph-mon[32238]: pgmap v500: 129 pgs: 129 active+clean; 10 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 876 MiB/s wr, 1.26k op/s 2024-05-20T16:18:34.138 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:18:33 smithi184 ceph-mon[36209]: pgmap v500: 129 pgs: 129 active+clean; 10 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 876 MiB/s wr, 1.26k op/s 2024-05-20T16:18:35.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:18:35 smithi082 ceph-mon[32238]: pgmap v501: 129 pgs: 129 active+clean; 10 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 777 MiB/s wr, 1.14k op/s 2024-05-20T16:18:35.885 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:18:35 smithi184 ceph-mon[36209]: pgmap v501: 129 pgs: 129 active+clean; 10 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 777 MiB/s wr, 1.14k op/s 2024-05-20T16:18:36.062 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:18:35 smithi160 ceph-mon[36347]: pgmap v501: 129 pgs: 129 active+clean; 10 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 777 MiB/s wr, 1.14k op/s 2024-05-20T16:18:37.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:18:37 smithi082 ceph-mon[32238]: pgmap v502: 129 pgs: 129 active+clean; 10 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 752 MiB/s wr, 1.14k op/s 2024-05-20T16:18:37.885 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:18:37 smithi184 ceph-mon[36209]: pgmap v502: 129 pgs: 129 active+clean; 10 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 752 MiB/s wr, 1.14k op/s 2024-05-20T16:18:38.062 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:18:37 smithi160 ceph-mon[36347]: pgmap v502: 129 pgs: 129 active+clean; 10 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 752 MiB/s wr, 1.14k op/s 2024-05-20T16:18:39.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:18:39 smithi082 ceph-mon[32238]: pgmap v503: 129 pgs: 129 active+clean; 10 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 891 MiB/s wr, 1.36k op/s 2024-05-20T16:18:39.885 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:18:39 smithi184 ceph-mon[36209]: pgmap v503: 129 pgs: 129 active+clean; 10 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 891 MiB/s wr, 1.36k op/s 2024-05-20T16:18:40.062 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:18:39 smithi160 ceph-mon[36347]: pgmap v503: 129 pgs: 129 active+clean; 10 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 891 MiB/s wr, 1.36k op/s 2024-05-20T16:18:40.839 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:40.837+0000 7ffa50e3e640 1 -- 172.21.15.82:0/2100909855 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffa4c10ee60 msgr2=0x7ffa4c10f270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:18:40.839 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:40.837+0000 7ffa50e3e640 1 --2- 172.21.15.82:0/2100909855 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffa4c10ee60 0x7ffa4c10f270 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7ffa44009900 tx=0x7ffa4402a440 comp rx=0 tx=0).stop 2024-05-20T16:18:40.839 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:40.839+0000 7ffa50e3e640 1 -- 172.21.15.82:0/2100909855 shutdown_connections 2024-05-20T16:18:40.839 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:40.839+0000 7ffa50e3e640 1 --2- 172.21.15.82:0/2100909855 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffa4c10ee60 0x7ffa4c10f270 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:18:40.839 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:40.839+0000 7ffa50e3e640 1 --2- 172.21.15.82:0/2100909855 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffa4c10a140 0x7ffa4c10a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:18:40.840 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:40.839+0000 7ffa50e3e640 1 --2- 172.21.15.82:0/2100909855 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ffa4c1092f0 0x7ffa4c109c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:18:40.840 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:40.839+0000 7ffa50e3e640 1 -- 172.21.15.82:0/2100909855 >> 172.21.15.82:0/2100909855 conn(0x7ffa4c06f2e0 msgr2=0x7ffa4c071790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:18:40.853 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:40.854+0000 7ffa50e3e640 1 -- 172.21.15.82:0/2100909855 shutdown_connections 2024-05-20T16:18:40.857 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:40.856+0000 7ffa50e3e640 1 -- 172.21.15.82:0/2100909855 wait complete. 2024-05-20T16:18:40.857 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:40.857+0000 7ffa50e3e640 1 Processor -- start 2024-05-20T16:18:40.861 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:40.861+0000 7ffa50e3e640 1 -- start start 2024-05-20T16:18:40.862 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:40.862+0000 7ffa50e3e640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ffa4c1092f0 0x7ffa4c115fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:18:40.862 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:40.862+0000 7ffa50e3e640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffa4c10a140 0x7ffa4c1164e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:18:40.862 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:40.862+0000 7ffa50e3e640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffa4c10ee60 0x7ffa4c117440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:18:40.862 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:40.862+0000 7ffa50e3e640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7ffa4c11d390 con 0x7ffa4c10a140 2024-05-20T16:18:40.862 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:40.862+0000 7ffa50e3e640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7ffa4c11d210 con 0x7ffa4c1092f0 2024-05-20T16:18:40.862 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:40.862+0000 7ffa50e3e640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ffa4c11d510 con 0x7ffa4c10ee60 2024-05-20T16:18:40.863 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:40.863+0000 7ffa4affd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffa4c10a140 0x7ffa4c1164e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:18:40.863 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:40.863+0000 7ffa4affd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffa4c10a140 0x7ffa4c1164e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:59496/0 (socket says 172.21.15.82:59496) 2024-05-20T16:18:40.863 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:40.863+0000 7ffa4affd640 1 -- 172.21.15.82:0/765285026 learned_addr learned my addr 172.21.15.82:0/765285026 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:18:40.863 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:40.863+0000 7ffa4b7fe640 1 --2- 172.21.15.82:0/765285026 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ffa4c1092f0 0x7ffa4c115fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:18:40.863 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:40.863+0000 7ffa4bfff640 1 --2- 172.21.15.82:0/765285026 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffa4c10ee60 0x7ffa4c117440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:18:40.864 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:40.864+0000 7ffa4affd640 1 -- 172.21.15.82:0/765285026 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffa4c10ee60 msgr2=0x7ffa4c117440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:18:40.864 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:40.864+0000 7ffa4affd640 1 --2- 172.21.15.82:0/765285026 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffa4c10ee60 0x7ffa4c117440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:18:40.864 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:40.864+0000 7ffa4affd640 1 -- 172.21.15.82:0/765285026 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ffa4c1092f0 msgr2=0x7ffa4c115fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:18:40.864 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:40.864+0000 7ffa4affd640 1 --2- 172.21.15.82:0/765285026 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ffa4c1092f0 0x7ffa4c115fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:18:40.864 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:40.864+0000 7ffa4affd640 1 -- 172.21.15.82:0/765285026 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa4c117ab0 con 0x7ffa4c10a140 2024-05-20T16:18:40.864 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:40.864+0000 7ffa4b7fe640 1 --2- 172.21.15.82:0/765285026 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ffa4c1092f0 0x7ffa4c115fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T16:18:40.865 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:40.865+0000 7ffa4affd640 1 --2- 172.21.15.82:0/765285026 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffa4c10a140 0x7ffa4c1164e0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7ffa4000e9f0 tx=0x7ffa4000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:18:40.865 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:40.866+0000 7ffa48ff9640 1 -- 172.21.15.82:0/765285026 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa4000cd80 con 0x7ffa4c10a140 2024-05-20T16:18:40.867 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:40.866+0000 7ffa50e3e640 1 -- 172.21.15.82:0/765285026 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa4c117da0 con 0x7ffa4c10a140 2024-05-20T16:18:40.867 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:40.866+0000 7ffa50e3e640 1 -- 172.21.15.82:0/765285026 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa4c1182f0 con 0x7ffa4c10a140 2024-05-20T16:18:40.870 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:40.870+0000 7ffa59022740 1 Processor -- start 2024-05-20T16:18:40.871 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:40.871+0000 7ffa59022740 1 -- start start 2024-05-20T16:18:40.871 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:40.871+0000 7ffa59022740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a680b3e1b0 0x55a680ede060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:18:40.871 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:40.871+0000 7ffa59022740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a680b152f0 0x55a680ede5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:18:40.871 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:40.871+0000 7ffa59022740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a680b1d4d0 0x55a680ee54e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:18:40.872 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:40.872+0000 7ffa59022740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55a680e42340 con 0x55a680b3e1b0 2024-05-20T16:18:40.872 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:40.872+0000 7ffa59022740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x55a680dea290 con 0x55a680b1d4d0 2024-05-20T16:18:40.872 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:40.872+0000 7ffa59022740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55a680e3e300 con 0x55a680b152f0 2024-05-20T16:18:40.872 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:40.872+0000 7ffa4b7fe640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a680b152f0 0x55a680ede5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:18:40.872 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:40.872+0000 7ffa4b7fe640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a680b152f0 0x55a680ede5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.82:47416/0 (socket says 172.21.15.82:47416) 2024-05-20T16:18:40.872 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:40.872+0000 7ffa4b7fe640 1 -- 172.21.15.82:0/1487104602 learned_addr learned my addr 172.21.15.82:0/1487104602 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:18:40.875 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:40.872+0000 7ffa4affd640 1 --2- 172.21.15.82:0/1487104602 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a680b1d4d0 0x55a680ee54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:18:40.878 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:40.878+0000 7ffa4affd640 1 -- 172.21.15.82:0/1487104602 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a680b152f0 msgr2=0x55a680ede5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:18:40.878 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:40.878+0000 7ffa4affd640 1 --2- 172.21.15.82:0/1487104602 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a680b152f0 0x55a680ede5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:18:40.878 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:40.878+0000 7ffa4affd640 1 -- 172.21.15.82:0/1487104602 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a680b3e1b0 msgr2=0x55a680ede060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:18:40.878 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:40.878+0000 7ffa4affd640 1 --2- 172.21.15.82:0/1487104602 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a680b3e1b0 0x55a680ede060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:18:40.878 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:40.878+0000 7ffa4affd640 1 -- 172.21.15.82:0/1487104602 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a680e0e8e0 con 0x55a680b1d4d0 2024-05-20T16:18:40.878 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:40.878+0000 7ffa48ff9640 1 -- 172.21.15.82:0/765285026 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa40010910 con 0x7ffa4c10a140 2024-05-20T16:18:40.880 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:40.880+0000 7ffa48ff9640 1 -- 172.21.15.82:0/765285026 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa40018990 con 0x7ffa4c10a140 2024-05-20T16:18:40.880 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:40.880+0000 7ffa48ff9640 1 -- 172.21.15.82:0/765285026 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7ffa400213f0 con 0x7ffa4c10a140 2024-05-20T16:18:40.880 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:40.881+0000 7ffa4affd640 1 --2- 172.21.15.82:0/1487104602 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a680b1d4d0 0x55a680ee54e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ffa400d4dd0 tx=0x7ffa400d8230 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b1fc8836acb60703 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:18:40.881 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:40.881+0000 7ffa31ffb640 1 -- 172.21.15.82:0/1487104602 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa400de070 con 0x55a680b1d4d0 2024-05-20T16:18:40.881 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:40.881+0000 7ffa31ffb640 1 -- 172.21.15.82:0/1487104602 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa400d8a30 con 0x55a680b1d4d0 2024-05-20T16:18:40.883 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:40.883+0000 7ffa59022740 1 -- 172.21.15.82:0/1487104602 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a680b1d4d0 msgr2=0x55a680ee54e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:18:40.883 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:40.883+0000 7ffa59022740 1 --2- 172.21.15.82:0/1487104602 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a680b1d4d0 0x55a680ee54e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ffa400d4dd0 tx=0x7ffa400d8230 comp rx=0 tx=0).stop 2024-05-20T16:18:40.884 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:40.884+0000 7ffa48ff9640 1 --2- 172.21.15.82:0/765285026 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7ffa340a2ca0 0x7ffa340a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:18:40.884 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:40.885+0000 7ffa48ff9640 1 -- 172.21.15.82:0/765285026 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9150+0+0 (secure 0 0 0) 0x7ffa400d76a0 con 0x7ffa4c10a140 2024-05-20T16:18:40.885 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:40.885+0000 7ffa59022740 1 -- 172.21.15.82:0/1487104602 shutdown_connections 2024-05-20T16:18:40.885 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:40.885+0000 7ffa59022740 1 --2- 172.21.15.82:0/1487104602 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a680b1d4d0 0x55a680ee54e0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:18:40.885 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:40.885+0000 7ffa59022740 1 --2- 172.21.15.82:0/1487104602 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a680b152f0 0x55a680ede5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:18:40.885 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:40.885+0000 7ffa59022740 1 --2- 172.21.15.82:0/1487104602 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a680b3e1b0 0x55a680ede060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:18:40.885 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:40.885+0000 7ffa59022740 1 -- 172.21.15.82:0/1487104602 >> 172.21.15.82:0/1487104602 conn(0x55a680b32300 msgr2=0x55a680ed99f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:18:40.886 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:40.886+0000 7ffa4bfff640 1 --2- 172.21.15.82:0/765285026 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7ffa340a2ca0 0x7ffa340a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:18:40.886 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:40.886+0000 7ffa59022740 1 -- 172.21.15.82:0/1487104602 shutdown_connections 2024-05-20T16:18:40.889 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:40.888+0000 7ffa59022740 1 -- 172.21.15.82:0/1487104602 wait complete. 2024-05-20T16:18:40.889 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:40.889+0000 7ffa59022740 1 -- start start 2024-05-20T16:18:40.889 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:40.889+0000 7ffa59022740 1 Processor -- start 2024-05-20T16:18:40.889 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:40.889+0000 7ffa59022740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a680b152f0 0x55a680ee7d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:18:40.890 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:40.889+0000 7ffa59022740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a680b1d4d0 0x55a680ee82c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:18:40.890 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:40.889+0000 7ffa59022740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a680b3e1b0 0x55a680eef220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:18:40.890 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:40.890+0000 7ffa59022740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55a680ee5f80 con 0x55a680b3e1b0 2024-05-20T16:18:40.890 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:40.890+0000 7ffa59022740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x55a680dfd790 con 0x55a680b152f0 2024-05-20T16:18:40.890 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:40.890+0000 7ffa59022740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55a680ee6100 con 0x55a680b1d4d0 2024-05-20T16:18:40.890 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:40.890+0000 7ffa59022740 10 client.0 _ll_register_callbacks cb 0x55a680e38a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T16:18:40.891 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:40.890+0000 7ffa4b7fe640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a680b152f0 0x55a680ee7d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:18:40.891 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:40.890+0000 7ffa4b7fe640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a680b152f0 0x55a680ee7d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.82:41504/0 (socket says 172.21.15.82:41504) 2024-05-20T16:18:40.891 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:40.890+0000 7ffa4b7fe640 1 -- 172.21.15.82:0/3431349921 learned_addr learned my addr 172.21.15.82:0/3431349921 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:18:40.891 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:40.891+0000 7ffa4bfff640 1 --2- 172.21.15.82:0/765285026 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7ffa340a2ca0 0x7ffa340a51b0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7ffa4402a960 tx=0x7ffa44005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:18:40.891 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:40.891+0000 7ffa4bfff640 1 --2- 172.21.15.82:0/3431349921 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a680b3e1b0 0x55a680eef220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:18:40.891 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:40.891+0000 7ffa4affd640 1 --2- 172.21.15.82:0/3431349921 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a680b1d4d0 0x55a680ee82c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:18:40.892 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:40.891+0000 7ffa4b7fe640 1 -- 172.21.15.82:0/3431349921 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a680b1d4d0 msgr2=0x55a680ee82c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:18:40.892 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:40.891+0000 7ffa4b7fe640 1 --2- 172.21.15.82:0/3431349921 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a680b1d4d0 0x55a680ee82c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:18:40.892 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:40.891+0000 7ffa4b7fe640 1 -- 172.21.15.82:0/3431349921 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a680b3e1b0 msgr2=0x55a680eef220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:18:40.892 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:40.891+0000 7ffa4b7fe640 1 --2- 172.21.15.82:0/3431349921 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a680b3e1b0 0x55a680eef220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:18:40.892 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:40.891+0000 7ffa4b7fe640 1 -- 172.21.15.82:0/3431349921 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a680b56210 con 0x55a680b152f0 2024-05-20T16:18:40.892 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:40.892+0000 7ffa4b7fe640 1 --2- 172.21.15.82:0/3431349921 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a680b152f0 0x55a680ee7d80 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ffa3c010f10 tx=0x7ffa3c00f7f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=24d93f4ad2301421 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:18:40.898 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:40.896+0000 7ffa2f7f6640 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-20T16:18:40.898 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:40.898+0000 7ffa2f7f6640 1 -- 172.21.15.82:0/3431349921 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa3c004b20 con 0x55a680b152f0 2024-05-20T16:18:40.898 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:40.898+0000 7ffa59022740 1 -- 172.21.15.82:0/3431349921 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffa400d8a30 con 0x55a680b152f0 2024-05-20T16:18:40.898 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:40.898+0000 7ffa2f7f6640 1 -- 172.21.15.82:0/3431349921 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9150+0+0 (secure 0 0 0) 0x7ffa3c004cc0 con 0x55a680b152f0 2024-05-20T16:18:40.899 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:40.899+0000 7ffa2f7f6640 1 -- 172.21.15.82:0/3431349921 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ffa3c023050 con 0x55a680b152f0 2024-05-20T16:18:40.902 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:40.902+0000 7ffa59022740 10 client.25220 fetch_fsmap learned FSMap version 27 2024-05-20T16:18:40.902 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:40.902+0000 7ffa59022740 1 -- 172.21.15.82:0/3431349921 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a680c22620 con 0x55a680b152f0 2024-05-20T16:18:40.907 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:40.907+0000 7ffa2f7f6640 1 -- 172.21.15.82:0/3431349921 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa400d8a30 con 0x55a680b152f0 2024-05-20T16:18:40.907 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:40.907+0000 7ffa2f7f6640 1 -- 172.21.15.82:0/3431349921 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7ffa3c0054b0 con 0x55a680b152f0 2024-05-20T16:18:40.907 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:40.907+0000 7ffa59022740 10 client.25220 fetch_fsmap finished waiting for FSMap version 27 2024-05-20T16:18:40.907 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:40.907+0000 7ffa59022740 10 client.25220 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T16:18:40.907 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:40.907+0000 7ffa59022740 20 client.25220 populate_metadata read hostname 'smithi082' 2024-05-20T16:18:40.907 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:40.908+0000 7ffa59022740 1 --2- 172.21.15.82:0/3431349921 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x55a680afcd80 0x55a680ef5c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:18:40.907 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:40.908+0000 7ffa59022740 4 client.25220 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T16:18:40.908 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:40.908+0000 7ffa59022740 1 -- 172.21.15.82:0/3431349921 --> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a680e4fa70 con 0x55a680afcd80 2024-05-20T16:18:40.908 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:40.908+0000 7ffa4affd640 1 --2- 172.21.15.82:0/3431349921 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x55a680afcd80 0x55a680ef5c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:18:40.910 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:40.910+0000 7ffa4affd640 1 --2- 172.21.15.82:0/3431349921 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x55a680afcd80 0x55a680ef5c20 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a2e51522500b361b server_cookie=e5dea44be071de3f in_seq=0 out_seq=0 2024-05-20T16:18:40.910 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:40.911+0000 7ffa2f7f6640 10 client.25220 ms_handle_connect on v2:172.21.15.160:6838/2765554482 2024-05-20T16:18:40.913 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:40.913+0000 7ffa2f7f6640 1 -- 172.21.15.82:0/3431349921 <== mds.0 v2:172.21.15.160:6838/2765554482 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55a680e4fa70 con 0x55a680afcd80 2024-05-20T16:18:40.913 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:40.913+0000 7ffa2f7f6640 10 client.25220 handle_command_reply: tid=0 2024-05-20T16:18:40.913 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:40.913+0000 7ffa59022740 1 client.25220 shutdown 2024-05-20T16:18:40.924 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:40.923+0000 7ffa59022740 1 -- 172.21.15.82:0/3431349921 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a680b152f0 msgr2=0x55a680ee7d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:18:40.924 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:40.923+0000 7ffa59022740 1 --2- 172.21.15.82:0/3431349921 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a680b152f0 0x55a680ee7d80 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ffa3c010f10 tx=0x7ffa3c00f7f0 comp rx=0 tx=0).stop 2024-05-20T16:18:40.924 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:40.923+0000 7ffa59022740 1 -- 172.21.15.82:0/3431349921 shutdown_connections 2024-05-20T16:18:40.924 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:40.923+0000 7ffa59022740 1 --2- 172.21.15.82:0/3431349921 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x55a680afcd80 0x55a680ef5c20 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:18:40.924 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:40.924+0000 7ffa59022740 1 --2- 172.21.15.82:0/3431349921 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a680b3e1b0 0x55a680eef220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:18:40.924 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:40.924+0000 7ffa59022740 1 --2- 172.21.15.82:0/3431349921 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a680b1d4d0 0x55a680ee82c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:18:40.925 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:40.924+0000 7ffa59022740 1 --2- 172.21.15.82:0/3431349921 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a680b152f0 0x55a680ee7d80 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:18:40.925 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:40.924+0000 7ffa59022740 1 -- 172.21.15.82:0/3431349921 >> 172.21.15.82:0/3431349921 conn(0x55a680b32300 msgr2=0x55a680ed88d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:18:40.925 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:40.924+0000 7ffa2f7f6640 0 client.25220 ms_handle_reset on v2:172.21.15.160:6838/2765554482 2024-05-20T16:18:40.925 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:40.924+0000 7ffa59022740 1 -- 172.21.15.82:0/3431349921 shutdown_connections 2024-05-20T16:18:40.925 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:40.924+0000 7ffa59022740 1 -- 172.21.15.82:0/3431349921 wait complete. 2024-05-20T16:18:40.925 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:40.925+0000 7ffa59022740 20 client.25220 trim_cache size 0 max 16384 2024-05-20T16:18:40.985 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:40.982+0000 7ffa59022740 1 Processor -- start 2024-05-20T16:18:40.985 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:40.983+0000 7ffa59022740 1 -- start start 2024-05-20T16:18:40.986 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:40.984+0000 7ffa59022740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a680b152f0 0x55a680ed9260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:18:40.986 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:40.984+0000 7ffa59022740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a680b1d4d0 0x55a680ed97a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:18:40.986 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:40.984+0000 7ffa59022740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a680b3e1b0 0x55a680eef1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:18:40.986 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:40.984+0000 7ffa59022740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7ffa380125a0 con 0x55a680b1d4d0 2024-05-20T16:18:40.986 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:40.984+0000 7ffa59022740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7ffa3800fe70 con 0x55a680b152f0 2024-05-20T16:18:40.986 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:40.984+0000 7ffa59022740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ffa3800dcc0 con 0x55a680b3e1b0 2024-05-20T16:18:40.986 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:40.985+0000 7ffa4bfff640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a680b3e1b0 0x55a680eef1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:18:40.986 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:40.985+0000 7ffa4bfff640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a680b3e1b0 0x55a680eef1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.82:47434/0 (socket says 172.21.15.82:47434) 2024-05-20T16:18:40.986 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:40.985+0000 7ffa4bfff640 1 -- 172.21.15.82:0/2908317452 learned_addr learned my addr 172.21.15.82:0/2908317452 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:18:40.988 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:40.986+0000 7ffa4b7fe640 1 --2- 172.21.15.82:0/2908317452 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a680b152f0 0x55a680ed9260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:18:40.988 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:40.986+0000 7ffa4bfff640 1 -- 172.21.15.82:0/2908317452 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a680b152f0 msgr2=0x55a680ed9260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:18:40.988 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:40.986+0000 7ffa4bfff640 1 --2- 172.21.15.82:0/2908317452 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a680b152f0 0x55a680ed9260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:18:40.988 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:40.986+0000 7ffa4bfff640 1 -- 172.21.15.82:0/2908317452 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a680b1d4d0 msgr2=0x55a680ed97a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:18:40.988 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:40.986+0000 7ffa4bfff640 1 --2- 172.21.15.82:0/2908317452 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a680b1d4d0 0x55a680ed97a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:18:40.988 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:40.986+0000 7ffa4bfff640 1 -- 172.21.15.82:0/2908317452 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a680ed9ce0 con 0x55a680b3e1b0 2024-05-20T16:18:40.988 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:40.987+0000 7ffa4bfff640 1 --2- 172.21.15.82:0/2908317452 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a680b3e1b0 0x55a680eef1b0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7ffa44004fd0 tx=0x7ffa4403d000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=62eda5abb47a53cb server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:18:40.990 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:40.988+0000 7ffa317fa640 1 -- 172.21.15.82:0/2908317452 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa44004240 con 0x55a680b3e1b0 2024-05-20T16:18:40.991 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:40.989+0000 7ffa317fa640 1 -- 172.21.15.82:0/2908317452 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa440055a0 con 0x55a680b3e1b0 2024-05-20T16:18:40.991 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:40.989+0000 7ffa317fa640 1 -- 172.21.15.82:0/2908317452 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa44003e10 con 0x55a680b3e1b0 2024-05-20T16:18:40.991 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:40.990+0000 7ffa59022740 1 -- 172.21.15.82:0/2908317452 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a680b3e1b0 msgr2=0x55a680eef1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:18:40.991 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:40.990+0000 7ffa59022740 1 --2- 172.21.15.82:0/2908317452 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a680b3e1b0 0x55a680eef1b0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7ffa44004fd0 tx=0x7ffa4403d000 comp rx=0 tx=0).stop 2024-05-20T16:18:40.994 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:40.993+0000 7ffa59022740 1 -- 172.21.15.82:0/2908317452 shutdown_connections 2024-05-20T16:18:40.994 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:40.993+0000 7ffa59022740 1 --2- 172.21.15.82:0/2908317452 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a680b1d4d0 0x55a680ed97a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:18:40.994 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:40.993+0000 7ffa59022740 1 --2- 172.21.15.82:0/2908317452 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a680b3e1b0 0x55a680eef1b0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:18:40.994 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:40.993+0000 7ffa59022740 1 --2- 172.21.15.82:0/2908317452 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a680b152f0 0x55a680ed9260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:18:40.994 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:40.993+0000 7ffa59022740 1 -- 172.21.15.82:0/2908317452 >> 172.21.15.82:0/2908317452 conn(0x55a680b32300 msgr2=0x55a680ee20b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:18:40.994 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:40.994+0000 7ffa59022740 1 -- 172.21.15.82:0/2908317452 shutdown_connections 2024-05-20T16:18:40.995 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:40.994+0000 7ffa59022740 1 -- 172.21.15.82:0/2908317452 wait complete. 2024-05-20T16:18:40.995 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:40.995+0000 7ffa59022740 1 -- start start 2024-05-20T16:18:40.995 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:40.996+0000 7ffa59022740 1 Processor -- start 2024-05-20T16:18:40.996 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:40.996+0000 7ffa59022740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a680b152f0 0x55a680eef9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:18:40.996 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:40.997+0000 7ffa59022740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a680b1d4d0 0x55a680eec590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:18:40.997 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:40.997+0000 7ffa59022740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a680b3e1b0 0x55a680eecbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:18:40.997 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:40.997+0000 7ffa59022740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7ffa3800e330 con 0x55a680b152f0 2024-05-20T16:18:40.997 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:40.997+0000 7ffa59022740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7ffa3800f7a0 con 0x55a680b1d4d0 2024-05-20T16:18:40.997 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:40.998+0000 7ffa59022740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ffa3800ea00 con 0x55a680b3e1b0 2024-05-20T16:18:40.998 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:40.998+0000 7ffa4affd640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a680b1d4d0 0x55a680eec590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:18:40.998 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:40.998+0000 7ffa4affd640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a680b1d4d0 0x55a680eec590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.82:41512/0 (socket says 172.21.15.82:41512) 2024-05-20T16:18:40.998 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:40.998+0000 7ffa4affd640 1 -- 172.21.15.82:0/112183559 learned_addr learned my addr 172.21.15.82:0/112183559 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:18:40.998 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:40.998+0000 7ffa4bfff640 1 --2- 172.21.15.82:0/112183559 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a680b3e1b0 0x55a680eecbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:18:40.998 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:40.998+0000 7ffa59022740 10 client.0 _ll_register_callbacks cb 0x7ffa3c00af50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T16:18:40.999 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:40.999+0000 7ffa4b7fe640 1 --2- 172.21.15.82:0/112183559 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a680b152f0 0x55a680eef9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:18:40.999 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:40.999+0000 7ffa4affd640 1 -- 172.21.15.82:0/112183559 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a680b3e1b0 msgr2=0x55a680eecbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:18:40.999 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:40.999+0000 7ffa4affd640 1 --2- 172.21.15.82:0/112183559 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a680b3e1b0 0x55a680eecbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:18:41.000 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:40.999+0000 7ffa4affd640 1 -- 172.21.15.82:0/112183559 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a680b152f0 msgr2=0x55a680eef9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:18:41.000 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:40.999+0000 7ffa4affd640 1 --2- 172.21.15.82:0/112183559 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a680b152f0 0x55a680eef9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:18:41.000 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:40.999+0000 7ffa4affd640 1 -- 172.21.15.82:0/112183559 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a680f5bff0 con 0x55a680b1d4d0 2024-05-20T16:18:41.003 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:41.003+0000 7ffa4affd640 1 --2- 172.21.15.82:0/112183559 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a680b1d4d0 0x55a680eec590 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ffa400057f0 tx=0x7ffa40018d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4a9cb67742acf687 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:18:41.003 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:41.004+0000 7ffa2f7f6640 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-20T16:18:41.009 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:41.009+0000 7ffa2f7f6640 1 -- 172.21.15.82:0/112183559 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa400de040 con 0x55a680b1d4d0 2024-05-20T16:18:41.009 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:41.009+0000 7ffa2f7f6640 1 -- 172.21.15.82:0/112183559 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9150+0+0 (secure 0 0 0) 0x7ffa40010640 con 0x55a680b1d4d0 2024-05-20T16:18:41.009 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:41.009+0000 7ffa59022740 1 -- 172.21.15.82:0/112183559 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffa440055a0 con 0x55a680b1d4d0 2024-05-20T16:18:41.010 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:41.011+0000 7ffa2f7f6640 1 -- 172.21.15.82:0/112183559 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ffa400a36d0 con 0x55a680b1d4d0 2024-05-20T16:18:41.014 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:41.015+0000 7ffa59022740 10 client.25226 fetch_fsmap learned FSMap version 27 2024-05-20T16:18:41.014 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:41.015+0000 7ffa59022740 1 -- 172.21.15.82:0/112183559 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a680f5d310 con 0x55a680b1d4d0 2024-05-20T16:18:41.017 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:41.017+0000 7ffa2f7f6640 1 -- 172.21.15.82:0/112183559 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa400a4290 con 0x55a680b1d4d0 2024-05-20T16:18:41.019 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:41.019+0000 7ffa2f7f6640 1 -- 172.21.15.82:0/112183559 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7ffa400a0170 con 0x55a680b1d4d0 2024-05-20T16:18:41.019 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:41.019+0000 7ffa59022740 10 client.25226 fetch_fsmap finished waiting for FSMap version 27 2024-05-20T16:18:41.019 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:41.019+0000 7ffa59022740 10 client.25226 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T16:18:41.020 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:41.019+0000 7ffa59022740 20 client.25226 populate_metadata read hostname 'smithi082' 2024-05-20T16:18:41.020 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:41.020+0000 7ffa59022740 1 --2- 172.21.15.82:0/112183559 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x55a680ad21c0 0x55a680f5f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:18:41.020 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:41.020+0000 7ffa59022740 4 client.25226 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-20T16:18:41.020 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:41.020+0000 7ffa59022740 1 -- 172.21.15.82:0/112183559 --> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7ffa440055a0 con 0x55a680ad21c0 2024-05-20T16:18:41.020 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:41.020+0000 7ffa4b7fe640 1 --2- 172.21.15.82:0/112183559 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x55a680ad21c0 0x55a680f5f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:18:41.029 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:41.030+0000 7ffa4b7fe640 1 --2- 172.21.15.82:0/112183559 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x55a680ad21c0 0x55a680f5f8a0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5d7db0d057d13444 server_cookie=b3b78a995efd6268 in_seq=0 out_seq=0 2024-05-20T16:18:41.029 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:18:41.030+0000 7ffa2f7f6640 10 client.25226 ms_handle_connect on v2:172.21.15.160:6838/2765554482 2024-05-20T16:18:41.863 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:18:41 smithi082 ceph-mon[32238]: pgmap v504: 129 pgs: 129 active+clean; 10 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 747 MiB/s wr, 1.12k op/s 2024-05-20T16:18:41.885 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:18:41 smithi184 ceph-mon[36209]: pgmap v504: 129 pgs: 129 active+clean; 10 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 747 MiB/s wr, 1.12k op/s 2024-05-20T16:18:42.062 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:18:41 smithi160 ceph-mon[36347]: pgmap v504: 129 pgs: 129 active+clean; 10 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 747 MiB/s wr, 1.12k op/s 2024-05-20T16:18:43.860 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:18:43 smithi082 ceph-mon[32238]: pgmap v505: 129 pgs: 129 active+clean; 10 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 753 MiB/s wr, 1.09k op/s 2024-05-20T16:18:43.885 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:18:43 smithi184 ceph-mon[36209]: pgmap v505: 129 pgs: 129 active+clean; 10 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 753 MiB/s wr, 1.09k op/s 2024-05-20T16:18:44.062 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:18:43 smithi160 ceph-mon[36347]: pgmap v505: 129 pgs: 129 active+clean; 10 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 753 MiB/s wr, 1.09k op/s 2024-05-20T16:18:45.863 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:18:45 smithi082 ceph-mon[32238]: pgmap v506: 129 pgs: 129 active+clean; 10 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 766 MiB/s wr, 1.13k op/s 2024-05-20T16:18:45.885 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:18:45 smithi184 ceph-mon[36209]: pgmap v506: 129 pgs: 129 active+clean; 10 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 766 MiB/s wr, 1.13k op/s 2024-05-20T16:18:46.062 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:18:45 smithi160 ceph-mon[36347]: pgmap v506: 129 pgs: 129 active+clean; 10 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 766 MiB/s wr, 1.13k op/s 2024-05-20T16:18:47.312 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:18:46 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:18:47.312 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:18:46 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T16:18:47.312 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:18:46 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:18:47.360 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:18:46 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:18:47.360 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:18:46 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T16:18:47.360 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:18:46 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:18:47.385 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:18:46 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:18:47.385 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:18:46 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T16:18:47.385 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:18:46 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:18:48.312 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:18:48 smithi160 ceph-mon[36347]: pgmap v507: 129 pgs: 129 active+clean; 10 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 700 MiB/s wr, 1.06k op/s 2024-05-20T16:18:48.358 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:18:48 smithi082 ceph-mon[32238]: pgmap v507: 129 pgs: 129 active+clean; 10 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 700 MiB/s wr, 1.06k op/s 2024-05-20T16:18:48.385 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:18:48 smithi184 ceph-mon[36209]: pgmap v507: 129 pgs: 129 active+clean; 10 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 700 MiB/s wr, 1.06k op/s 2024-05-20T16:18:49.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:18:49 smithi082 ceph-mon[32238]: pgmap v508: 129 pgs: 129 active+clean; 10 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 819 MiB/s wr, 1.42k op/s 2024-05-20T16:18:49.887 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:18:49 smithi184 ceph-mon[36209]: pgmap v508: 129 pgs: 129 active+clean; 10 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 819 MiB/s wr, 1.42k op/s 2024-05-20T16:18:50.062 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:18:49 smithi160 ceph-mon[36347]: pgmap v508: 129 pgs: 129 active+clean; 10 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 819 MiB/s wr, 1.42k op/s 2024-05-20T16:18:51.861 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:18:51 smithi082 ceph-mon[32238]: pgmap v509: 129 pgs: 129 active+clean; 10 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 738 MiB/s wr, 1.42k op/s 2024-05-20T16:18:51.885 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:18:51 smithi184 ceph-mon[36209]: pgmap v509: 129 pgs: 129 active+clean; 10 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 738 MiB/s wr, 1.42k op/s 2024-05-20T16:18:52.062 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:18:51 smithi160 ceph-mon[36347]: pgmap v509: 129 pgs: 129 active+clean; 10 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 738 MiB/s wr, 1.42k op/s 2024-05-20T16:18:53.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:18:53 smithi082 ceph-mon[32238]: pgmap v510: 129 pgs: 129 active+clean; 10 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 750 MiB/s wr, 1.41k op/s 2024-05-20T16:18:53.885 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:18:53 smithi184 ceph-mon[36209]: pgmap v510: 129 pgs: 129 active+clean; 10 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 750 MiB/s wr, 1.41k op/s 2024-05-20T16:18:54.062 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:18:53 smithi160 ceph-mon[36347]: pgmap v510: 129 pgs: 129 active+clean; 10 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 750 MiB/s wr, 1.41k op/s 2024-05-20T16:18:55.885 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:18:55 smithi184 ceph-mon[36209]: pgmap v511: 129 pgs: 129 active+clean; 10 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 795 MiB/s wr, 1.43k op/s 2024-05-20T16:18:56.062 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:18:55 smithi160 ceph-mon[36347]: pgmap v511: 129 pgs: 129 active+clean; 10 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 795 MiB/s wr, 1.43k op/s 2024-05-20T16:18:56.132 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:18:55 smithi082 ceph-mon[32238]: pgmap v511: 129 pgs: 129 active+clean; 10 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 795 MiB/s wr, 1.43k op/s 2024-05-20T16:18:57.860 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:18:57 smithi082 ceph-mon[32238]: pgmap v512: 129 pgs: 129 active+clean; 10 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 706 MiB/s wr, 1.31k op/s 2024-05-20T16:18:57.885 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:18:57 smithi184 ceph-mon[36209]: pgmap v512: 129 pgs: 129 active+clean; 10 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 706 MiB/s wr, 1.31k op/s 2024-05-20T16:18:58.062 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:18:57 smithi160 ceph-mon[36347]: pgmap v512: 129 pgs: 129 active+clean; 10 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 706 MiB/s wr, 1.31k op/s 2024-05-20T16:18:59.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:18:59 smithi082 ceph-mon[32238]: pgmap v513: 129 pgs: 129 active+clean; 10 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 843 MiB/s wr, 1.66k op/s 2024-05-20T16:18:59.863 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:18:59 smithi160 ceph-mon[36347]: pgmap v513: 129 pgs: 129 active+clean; 10 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 843 MiB/s wr, 1.66k op/s 2024-05-20T16:18:59.885 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:18:59 smithi184 ceph-mon[36209]: pgmap v513: 129 pgs: 129 active+clean; 10 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 843 MiB/s wr, 1.66k op/s 2024-05-20T16:19:01.860 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:19:01 smithi082 ceph-mon[32238]: pgmap v514: 129 pgs: 129 active+clean; 10 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 977 KiB/s rd, 826 MiB/s wr, 1.52k op/s 2024-05-20T16:19:01.885 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:19:01 smithi184 ceph-mon[36209]: pgmap v514: 129 pgs: 129 active+clean; 10 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 977 KiB/s rd, 826 MiB/s wr, 1.52k op/s 2024-05-20T16:19:02.062 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:19:01 smithi160 ceph-mon[36347]: pgmap v514: 129 pgs: 129 active+clean; 10 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 977 KiB/s rd, 826 MiB/s wr, 1.52k op/s 2024-05-20T16:19:03.543 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:03.543+0000 7ffa2f7f6640 1 -- 172.21.15.82:0/112183559 <== mds.0 v2:172.21.15.160:6838/2765554482 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7ffa440055a0 con 0x55a680ad21c0 2024-05-20T16:19:03.544 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:03.543+0000 7ffa2f7f6640 10 client.25226 handle_command_reply: tid=0 2024-05-20T16:19:03.544 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:03.543+0000 7ffa59022740 1 client.25226 shutdown 2024-05-20T16:19:03.545 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:03.544+0000 7ffa59022740 1 -- 172.21.15.82:0/112183559 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a680b1d4d0 msgr2=0x55a680eec590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:19:03.545 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:03.544+0000 7ffa59022740 1 --2- 172.21.15.82:0/112183559 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a680b1d4d0 0x55a680eec590 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ffa400057f0 tx=0x7ffa40018d50 comp rx=0 tx=0).stop 2024-05-20T16:19:03.545 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:03.544+0000 7ffa59022740 1 -- 172.21.15.82:0/112183559 shutdown_connections 2024-05-20T16:19:03.545 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:03.544+0000 7ffa59022740 1 --2- 172.21.15.82:0/112183559 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x55a680ad21c0 0x55a680f5f8a0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:19:03.545 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:03.544+0000 7ffa59022740 1 --2- 172.21.15.82:0/112183559 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a680b3e1b0 0x55a680eecbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:19:03.545 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:03.544+0000 7ffa59022740 1 --2- 172.21.15.82:0/112183559 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a680b1d4d0 0x55a680eec590 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:19:03.545 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:03.544+0000 7ffa59022740 1 --2- 172.21.15.82:0/112183559 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a680b152f0 0x55a680eef9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:19:03.545 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:03.544+0000 7ffa59022740 1 -- 172.21.15.82:0/112183559 >> 172.21.15.82:0/112183559 conn(0x55a680b32300 msgr2=0x55a680ee20b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:19:03.545 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:03.544+0000 7ffa2f7f6640 0 client.25226 ms_handle_reset on v2:172.21.15.160:6838/2765554482 2024-05-20T16:19:03.546 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:03.546+0000 7ffa59022740 1 -- 172.21.15.82:0/112183559 shutdown_connections 2024-05-20T16:19:03.546 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:03.546+0000 7ffa59022740 1 -- 172.21.15.82:0/112183559 wait complete. 2024-05-20T16:19:03.546 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:03.546+0000 7ffa59022740 20 client.25226 trim_cache size 0 max 16384 2024-05-20T16:19:03.547 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:{ 2024-05-20T16:19:03.547 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "message": "", 2024-05-20T16:19:03.547 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "return_code": 0 2024-05-20T16:19:03.547 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:} 2024-05-20T16:19:03.548 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:03.547+0000 7ffa50e3e640 1 -- 172.21.15.82:0/765285026 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7ffa340a2ca0 msgr2=0x7ffa340a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:19:03.548 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:03.547+0000 7ffa50e3e640 1 --2- 172.21.15.82:0/765285026 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7ffa340a2ca0 0x7ffa340a51b0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7ffa4402a960 tx=0x7ffa44005850 comp rx=0 tx=0).stop 2024-05-20T16:19:03.548 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:03.547+0000 7ffa50e3e640 1 -- 172.21.15.82:0/765285026 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffa4c10a140 msgr2=0x7ffa4c1164e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:19:03.548 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:03.547+0000 7ffa50e3e640 1 --2- 172.21.15.82:0/765285026 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffa4c10a140 0x7ffa4c1164e0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7ffa4000e9f0 tx=0x7ffa4000eeb0 comp rx=0 tx=0).stop 2024-05-20T16:19:03.548 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:03.547+0000 7ffa50e3e640 1 -- 172.21.15.82:0/765285026 shutdown_connections 2024-05-20T16:19:03.549 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:03.547+0000 7ffa50e3e640 1 --2- 172.21.15.82:0/765285026 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7ffa340a2ca0 0x7ffa340a51b0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:19:03.549 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:03.547+0000 7ffa50e3e640 1 --2- 172.21.15.82:0/765285026 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffa4c10a140 0x7ffa4c1164e0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:19:03.549 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:03.547+0000 7ffa50e3e640 1 --2- 172.21.15.82:0/765285026 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffa4c10ee60 0x7ffa4c117440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:19:03.549 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:03.547+0000 7ffa50e3e640 1 --2- 172.21.15.82:0/765285026 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ffa4c1092f0 0x7ffa4c115fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:19:03.549 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:03.547+0000 7ffa50e3e640 1 -- 172.21.15.82:0/765285026 >> 172.21.15.82:0/765285026 conn(0x7ffa4c06f2e0 msgr2=0x7ffa4c10ae40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:19:03.549 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:03.548+0000 7ffa50e3e640 1 -- 172.21.15.82:0/765285026 shutdown_connections 2024-05-20T16:19:03.549 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:03.548+0000 7ffa50e3e640 1 -- 172.21.15.82:0/765285026 wait complete. 2024-05-20T16:19:03.862 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:19:03 smithi082 ceph-mon[32238]: pgmap v515: 129 pgs: 129 active+clean; 10 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 797 MiB/s wr, 1.39k op/s 2024-05-20T16:19:03.885 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:19:03 smithi184 ceph-mon[36209]: pgmap v515: 129 pgs: 129 active+clean; 10 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 797 MiB/s wr, 1.39k op/s 2024-05-20T16:19:04.062 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:19:03 smithi160 ceph-mon[36347]: pgmap v515: 129 pgs: 129 active+clean; 10 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 797 MiB/s wr, 1.39k op/s 2024-05-20T16:19:05.860 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:19:05 smithi082 ceph-mon[32238]: pgmap v516: 129 pgs: 129 active+clean; 10 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 836 MiB/s wr, 1.51k op/s 2024-05-20T16:19:05.885 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:19:05 smithi184 ceph-mon[36209]: pgmap v516: 129 pgs: 129 active+clean; 10 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 836 MiB/s wr, 1.51k op/s 2024-05-20T16:19:06.061 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:19:05 smithi160 ceph-mon[36347]: pgmap v516: 129 pgs: 129 active+clean; 10 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 836 MiB/s wr, 1.51k op/s 2024-05-20T16:19:07.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:19:07 smithi082 ceph-mon[32238]: pgmap v517: 129 pgs: 129 active+clean; 10 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 803 MiB/s wr, 1.53k op/s 2024-05-20T16:19:07.885 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:19:07 smithi184 ceph-mon[36209]: pgmap v517: 129 pgs: 129 active+clean; 10 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 803 MiB/s wr, 1.53k op/s 2024-05-20T16:19:08.062 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:19:07 smithi160 ceph-mon[36347]: pgmap v517: 129 pgs: 129 active+clean; 10 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 803 MiB/s wr, 1.53k op/s 2024-05-20T16:19:09.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:19:09 smithi082 ceph-mon[32238]: pgmap v518: 129 pgs: 129 active+clean; 10 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 848 MiB/s wr, 1.77k op/s 2024-05-20T16:19:09.885 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:19:09 smithi184 ceph-mon[36209]: pgmap v518: 129 pgs: 129 active+clean; 10 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 848 MiB/s wr, 1.77k op/s 2024-05-20T16:19:10.062 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:19:09 smithi160 ceph-mon[36347]: pgmap v518: 129 pgs: 129 active+clean; 10 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 848 MiB/s wr, 1.77k op/s 2024-05-20T16:19:11.885 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:19:11 smithi184 ceph-mon[36209]: pgmap v519: 129 pgs: 129 active+clean; 10 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 757 MiB/s wr, 1.53k op/s 2024-05-20T16:19:12.062 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:19:11 smithi160 ceph-mon[36347]: pgmap v519: 129 pgs: 129 active+clean; 10 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 757 MiB/s wr, 1.53k op/s 2024-05-20T16:19:12.110 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:19:11 smithi082 ceph-mon[32238]: pgmap v519: 129 pgs: 129 active+clean; 10 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 757 MiB/s wr, 1.53k op/s 2024-05-20T16:19:13.885 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:19:13 smithi184 ceph-mon[36209]: pgmap v520: 129 pgs: 129 active+clean; 10 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 757 MiB/s wr, 1.47k op/s 2024-05-20T16:19:13.917 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:19:13 smithi082 ceph-mon[32238]: pgmap v520: 129 pgs: 129 active+clean; 10 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 757 MiB/s wr, 1.47k op/s 2024-05-20T16:19:14.062 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:19:13 smithi160 ceph-mon[36347]: pgmap v520: 129 pgs: 129 active+clean; 10 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 757 MiB/s wr, 1.47k op/s 2024-05-20T16:19:16.062 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:19:15 smithi160 ceph-mon[36347]: pgmap v521: 129 pgs: 129 active+clean; 10 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 803 MiB/s wr, 1.45k op/s 2024-05-20T16:19:16.110 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:19:15 smithi082 ceph-mon[32238]: pgmap v521: 129 pgs: 129 active+clean; 10 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 803 MiB/s wr, 1.45k op/s 2024-05-20T16:19:16.135 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:19:15 smithi184 ceph-mon[36209]: pgmap v521: 129 pgs: 129 active+clean; 10 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 803 MiB/s wr, 1.45k op/s 2024-05-20T16:19:16.835 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:16.833+0000 7f30d865a640 1 -- 172.21.15.82:0/1349792364 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f30d0107910 msgr2=0x7f30d0109d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:19:16.835 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:16.833+0000 7f30d865a640 1 --2- 172.21.15.82:0/1349792364 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f30d0107910 0x7f30d0109d30 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f30c8009ca0 tx=0x7f30c802a5c0 comp rx=0 tx=0).stop 2024-05-20T16:19:16.842 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:16.837+0000 7f30d865a640 1 -- 172.21.15.82:0/1349792364 shutdown_connections 2024-05-20T16:19:16.842 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:16.837+0000 7f30d865a640 1 --2- 172.21.15.82:0/1349792364 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f30d0107910 0x7f30d0109d30 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:19:16.842 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:16.837+0000 7f30d865a640 1 --2- 172.21.15.82:0/1349792364 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f30d0102c10 0x7f30d0103020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:19:16.842 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:16.837+0000 7f30d865a640 1 --2- 172.21.15.82:0/1349792364 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f30d01019d0 0x7f30d01026d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:19:16.843 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:16.837+0000 7f30d865a640 1 -- 172.21.15.82:0/1349792364 >> 172.21.15.82:0/1349792364 conn(0x7f30d00fcbc0 msgr2=0x7f30d00ff070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:19:16.847 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:16.839+0000 7f30d865a640 1 -- 172.21.15.82:0/1349792364 shutdown_connections 2024-05-20T16:19:16.847 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:16.839+0000 7f30d865a640 1 -- 172.21.15.82:0/1349792364 wait complete. 2024-05-20T16:19:16.847 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:16.841+0000 7f30d865a640 1 Processor -- start 2024-05-20T16:19:16.847 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:16.841+0000 7f30d865a640 1 -- start start 2024-05-20T16:19:16.848 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:16.841+0000 7f30d865a640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f30d01019d0 0x7f30d0070a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:19:16.848 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:16.841+0000 7f30d865a640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f30d0102c10 0x7f30d0070f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:19:16.848 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:16.841+0000 7f30d865a640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f30d0107910 0x7f30d0071ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:19:16.848 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:16.841+0000 7f30d865a640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f30d0114db0 con 0x7f30d0107910 2024-05-20T16:19:16.848 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:16.841+0000 7f30d865a640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f30d0114c30 con 0x7f30d0102c10 2024-05-20T16:19:16.848 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:16.841+0000 7f30d865a640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f30d0114f30 con 0x7f30d01019d0 2024-05-20T16:19:16.849 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:16.842+0000 7f30d6bf7640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f30d0107910 0x7f30d0071ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:19:16.849 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:16.842+0000 7f30d6bf7640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f30d0107910 0x7f30d0071ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:38696/0 (socket says 172.21.15.82:38696) 2024-05-20T16:19:16.849 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:16.842+0000 7f30d6bf7640 1 -- 172.21.15.82:0/1112699451 learned_addr learned my addr 172.21.15.82:0/1112699451 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:19:16.849 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:16.842+0000 7f30d5bf5640 1 --2- 172.21.15.82:0/1112699451 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f30d0102c10 0x7f30d0070f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:19:16.849 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:16.843+0000 7f30d63f6640 1 --2- 172.21.15.82:0/1112699451 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f30d01019d0 0x7f30d0070a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:19:16.849 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:16.845+0000 7f30d5bf5640 1 -- 172.21.15.82:0/1112699451 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f30d01019d0 msgr2=0x7f30d0070a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:19:16.850 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:16.845+0000 7f30d5bf5640 1 --2- 172.21.15.82:0/1112699451 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f30d01019d0 0x7f30d0070a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:19:16.850 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:16.845+0000 7f30d5bf5640 1 -- 172.21.15.82:0/1112699451 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f30d0107910 msgr2=0x7f30d0071ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:19:16.850 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:16.845+0000 7f30d5bf5640 1 --2- 172.21.15.82:0/1112699451 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f30d0107910 0x7f30d0071ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:19:16.850 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:16.845+0000 7f30d5bf5640 1 -- 172.21.15.82:0/1112699451 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30d0072530 con 0x7f30d0102c10 2024-05-20T16:19:16.850 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:16.846+0000 7f30d5bf5640 1 --2- 172.21.15.82:0/1112699451 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f30d0102c10 0x7f30d0070f60 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f30c000d930 tx=0x7f30c000ddf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:19:16.851 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:16.849+0000 7f30c77fe640 1 -- 172.21.15.82:0/1112699451 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30c000cb80 con 0x7f30d0102c10 2024-05-20T16:19:16.852 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:16.849+0000 7f30c77fe640 1 -- 172.21.15.82:0/1112699451 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f30c000cd20 con 0x7f30d0102c10 2024-05-20T16:19:16.852 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:16.849+0000 7f30c77fe640 1 -- 172.21.15.82:0/1112699451 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30c0015410 con 0x7f30d0102c10 2024-05-20T16:19:16.852 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:16.850+0000 7f30d865a640 1 -- 172.21.15.82:0/1112699451 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30d0072820 con 0x7f30d0102c10 2024-05-20T16:19:16.852 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:16.851+0000 7f30d865a640 1 -- 172.21.15.82:0/1112699451 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30d0072da0 con 0x7f30d0102c10 2024-05-20T16:19:16.863 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:16.859+0000 7f30c77fe640 1 -- 172.21.15.82:0/1112699451 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f30c00155b0 con 0x7f30d0102c10 2024-05-20T16:19:16.863 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:16.859+0000 7f30df5db740 1 Processor -- start 2024-05-20T16:19:16.871 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:16.865+0000 7f30df5db740 1 -- start start 2024-05-20T16:19:16.871 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:16.865+0000 7f30df5db740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dc73df51b0 0x55dc74194e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:19:16.871 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:16.865+0000 7f30df5db740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55dc73dcc2f0 0x55dc741953e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:19:16.872 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:16.865+0000 7f30df5db740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dc73dd44d0 0x55dc7419c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:19:16.872 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:16.865+0000 7f30df5db740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55dc740f9340 con 0x55dc73dd44d0 2024-05-20T16:19:16.872 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:16.865+0000 7f30df5db740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x55dc740a1290 con 0x55dc73dcc2f0 2024-05-20T16:19:16.872 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:16.865+0000 7f30df5db740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55dc740f5300 con 0x55dc73df51b0 2024-05-20T16:19:16.872 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:16.866+0000 7f30c77fe640 1 --2- 172.21.15.82:0/1112699451 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f30bc0a2d90 0x7f30bc0a52a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:19:16.872 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:16.866+0000 7f30c77fe640 1 -- 172.21.15.82:0/1112699451 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9150+0+0 (secure 0 0 0) 0x7f30c00cf9d0 con 0x7f30d0102c10 2024-05-20T16:19:16.872 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:16.868+0000 7f30d5bf5640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dc73dd44d0 0x55dc7419c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:19:16.872 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:16.868+0000 7f30d6bf7640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dc73df51b0 0x55dc74194e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:19:16.872 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:16.869+0000 7f30d63f6640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55dc73dcc2f0 0x55dc741953e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:19:16.872 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:16.870+0000 7f30d6bf7640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dc73df51b0 0x55dc74194e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.82:51786/0 (socket says 172.21.15.82:51786) 2024-05-20T16:19:16.872 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:16.870+0000 7f30d6bf7640 1 -- 172.21.15.82:0/3350150016 learned_addr learned my addr 172.21.15.82:0/3350150016 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:19:16.872 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:16.870+0000 7f30d6bf7640 1 --2- 172.21.15.82:0/1112699451 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f30bc0a2d90 0x7f30bc0a52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:19:16.874 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:16.872+0000 7f30d6bf7640 1 -- 172.21.15.82:0/3350150016 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55dc73dcc2f0 msgr2=0x55dc741953e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:19:16.874 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:16.872+0000 7f30d6bf7640 1 --2- 172.21.15.82:0/3350150016 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55dc73dcc2f0 0x55dc741953e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:19:16.874 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:16.872+0000 7f30d6bf7640 1 -- 172.21.15.82:0/3350150016 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dc73dd44d0 msgr2=0x55dc7419c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:19:16.874 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:16.872+0000 7f30d6bf7640 1 --2- 172.21.15.82:0/3350150016 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dc73dd44d0 0x55dc7419c2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:19:16.874 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:16.872+0000 7f30d6bf7640 1 -- 172.21.15.82:0/3350150016 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc740c58e0 con 0x55dc73df51b0 2024-05-20T16:19:16.876 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:16.876+0000 7f30d6bf7640 1 --2- 172.21.15.82:0/3350150016 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dc73df51b0 0x55dc74194e60 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f30c802f8e0 tx=0x7f30c802f910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=435d18ea1caa3d56 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:19:16.878 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:16.878+0000 7f30d6bf7640 1 --2- 172.21.15.82:0/1112699451 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f30bc0a2d90 0x7f30bc0a52a0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f30c8005df0 tx=0x7f30c804a000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:19:16.880 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:16.879+0000 7f30c4bfe640 1 -- 172.21.15.82:0/3350150016 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30c804f070 con 0x55dc73df51b0 2024-05-20T16:19:16.880 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:16.879+0000 7f30c4bfe640 1 -- 172.21.15.82:0/3350150016 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f30c803fd00 con 0x55dc73df51b0 2024-05-20T16:19:16.883 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:16.880+0000 7f30df5db740 1 -- 172.21.15.82:0/3350150016 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dc73df51b0 msgr2=0x55dc74194e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:19:16.883 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:16.880+0000 7f30df5db740 1 --2- 172.21.15.82:0/3350150016 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dc73df51b0 0x55dc74194e60 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f30c802f8e0 tx=0x7f30c802f910 comp rx=0 tx=0).stop 2024-05-20T16:19:16.886 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:16.885+0000 7f30df5db740 1 -- 172.21.15.82:0/3350150016 shutdown_connections 2024-05-20T16:19:16.886 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:16.885+0000 7f30df5db740 1 --2- 172.21.15.82:0/3350150016 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dc73dd44d0 0x55dc7419c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:19:16.886 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:16.885+0000 7f30df5db740 1 --2- 172.21.15.82:0/3350150016 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55dc73dcc2f0 0x55dc741953e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:19:16.886 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:16.885+0000 7f30df5db740 1 --2- 172.21.15.82:0/3350150016 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dc73df51b0 0x55dc74194e60 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:19:16.886 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:16.885+0000 7f30df5db740 1 -- 172.21.15.82:0/3350150016 >> 172.21.15.82:0/3350150016 conn(0x55dc73de9300 msgr2=0x55dc741907f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:19:16.889 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:16.888+0000 7f30df5db740 1 -- 172.21.15.82:0/3350150016 shutdown_connections 2024-05-20T16:19:16.889 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:16.888+0000 7f30df5db740 1 -- 172.21.15.82:0/3350150016 wait complete. 2024-05-20T16:19:16.889 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:16.888+0000 7f30df5db740 1 -- start start 2024-05-20T16:19:16.889 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:16.888+0000 7f30df5db740 1 Processor -- start 2024-05-20T16:19:16.890 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:16.889+0000 7f30df5db740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55dc73dcc2f0 0x55dc7419eba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:19:16.891 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:16.889+0000 7f30df5db740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dc73dd44d0 0x55dc7419f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:19:16.891 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:16.889+0000 7f30df5db740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dc73df51b0 0x55dc741a6040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:19:16.891 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:16.889+0000 7f30df5db740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55dc7419cd20 con 0x55dc73dd44d0 2024-05-20T16:19:16.892 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:16.889+0000 7f30df5db740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x55dc740b4790 con 0x55dc73dcc2f0 2024-05-20T16:19:16.892 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:16.889+0000 7f30df5db740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55dc7419cea0 con 0x55dc73df51b0 2024-05-20T16:19:16.892 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:16.889+0000 7f30df5db740 10 client.0 _ll_register_callbacks cb 0x55dc740efa20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T16:19:16.892 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:16.890+0000 7f30d63f6640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55dc73dcc2f0 0x55dc7419eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:19:16.892 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:16.890+0000 7f30d63f6640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55dc73dcc2f0 0x55dc7419eba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.82:50284/0 (socket says 172.21.15.82:50284) 2024-05-20T16:19:16.892 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:16.890+0000 7f30d63f6640 1 -- 172.21.15.82:0/2521386011 learned_addr learned my addr 172.21.15.82:0/2521386011 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:19:16.892 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:16.890+0000 7f30d6bf7640 1 --2- 172.21.15.82:0/2521386011 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dc73df51b0 0x55dc741a6040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:19:16.892 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:16.891+0000 7f30d6bf7640 1 -- 172.21.15.82:0/2521386011 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55dc73dcc2f0 msgr2=0x55dc7419eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:19:16.892 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:16.891+0000 7f30d6bf7640 1 --2- 172.21.15.82:0/2521386011 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55dc73dcc2f0 0x55dc7419eba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:19:16.892 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:16.891+0000 7f30d6bf7640 1 -- 172.21.15.82:0/2521386011 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dc73dd44d0 msgr2=0x55dc7419f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:19:16.892 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:16.891+0000 7f30d6bf7640 1 --2- 172.21.15.82:0/2521386011 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dc73dd44d0 0x55dc7419f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:19:16.892 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:16.891+0000 7f30d6bf7640 1 -- 172.21.15.82:0/2521386011 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dc73e0d210 con 0x55dc73df51b0 2024-05-20T16:19:16.894 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:16.894+0000 7f30d6bf7640 1 --2- 172.21.15.82:0/2521386011 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dc73df51b0 0x55dc741a6040 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f30c8004bb0 tx=0x7f30c8005d80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9c3c694f935cccee server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:19:16.894 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:16.894+0000 7f30b9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-05-20T16:19:16.894 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:16.894+0000 7f30b9ffb640 1 -- 172.21.15.82:0/2521386011 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30c804f030 con 0x55dc73df51b0 2024-05-20T16:19:16.894 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:16.894+0000 7f30df5db740 1 -- 172.21.15.82:0/2521386011 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f30c803fd00 con 0x55dc73df51b0 2024-05-20T16:19:16.894 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:16.894+0000 7f30b9ffb640 1 -- 172.21.15.82:0/2521386011 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9150+0+0 (secure 0 0 0) 0x7f30c804ba90 con 0x55dc73df51b0 2024-05-20T16:19:16.895 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:16.895+0000 7f30b9ffb640 1 -- 172.21.15.82:0/2521386011 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f30c8062050 con 0x55dc73df51b0 2024-05-20T16:19:16.897 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:16.895+0000 7f30df5db740 10 client.25261 fetch_fsmap learned FSMap version 27 2024-05-20T16:19:16.897 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:16.895+0000 7f30df5db740 1 -- 172.21.15.82:0/2521386011 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55dc73ed9620 con 0x55dc73df51b0 2024-05-20T16:19:16.901 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:16.898+0000 7f30b9ffb640 1 -- 172.21.15.82:0/2521386011 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f30c803fd00 con 0x55dc73df51b0 2024-05-20T16:19:16.901 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:16.898+0000 7f30b9ffb640 1 -- 172.21.15.82:0/2521386011 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f30c8054930 con 0x55dc73df51b0 2024-05-20T16:19:16.901 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:16.898+0000 7f30df5db740 10 client.25261 fetch_fsmap finished waiting for FSMap version 27 2024-05-20T16:19:16.901 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:16.898+0000 7f30df5db740 10 client.25261 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T16:19:16.901 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:16.898+0000 7f30df5db740 20 client.25261 populate_metadata read hostname 'smithi082' 2024-05-20T16:19:16.901 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:16.898+0000 7f30df5db740 1 --2- 172.21.15.82:0/2521386011 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x55dc73db3d80 0x55dc741ac6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:19:16.902 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:16.898+0000 7f30df5db740 4 client.25261 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T16:19:16.902 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:16.898+0000 7f30df5db740 1 -- 172.21.15.82:0/2521386011 --> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dc74106a70 con 0x55dc73db3d80 2024-05-20T16:19:16.903 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:16.904+0000 7f30d63f6640 1 --2- 172.21.15.82:0/2521386011 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x55dc73db3d80 0x55dc741ac6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:19:16.917 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:16.917+0000 7f30b9ffb640 10 client.25261 ms_handle_connect on v2:172.21.15.160:6838/2765554482 2024-05-20T16:19:16.917 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:16.917+0000 7f30d63f6640 1 --2- 172.21.15.82:0/2521386011 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x55dc73db3d80 0x55dc741ac6c0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f945e4cad24bd582 server_cookie=56185c8d5a688850 in_seq=0 out_seq=0 2024-05-20T16:19:16.925 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:16.925+0000 7f30b9ffb640 1 -- 172.21.15.82:0/2521386011 <== mds.0 v2:172.21.15.160:6838/2765554482 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55dc74106a70 con 0x55dc73db3d80 2024-05-20T16:19:16.925 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:16.925+0000 7f30b9ffb640 10 client.25261 handle_command_reply: tid=0 2024-05-20T16:19:16.925 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:16.925+0000 7f30df5db740 1 client.25261 shutdown 2024-05-20T16:19:16.928 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:16.929+0000 7f30df5db740 1 -- 172.21.15.82:0/2521386011 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dc73df51b0 msgr2=0x55dc741a6040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:19:16.928 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:16.929+0000 7f30df5db740 1 --2- 172.21.15.82:0/2521386011 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dc73df51b0 0x55dc741a6040 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f30c8004bb0 tx=0x7f30c8005d80 comp rx=0 tx=0).stop 2024-05-20T16:19:16.933 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:16.932+0000 7f30df5db740 1 -- 172.21.15.82:0/2521386011 shutdown_connections 2024-05-20T16:19:16.933 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:16.932+0000 7f30df5db740 1 --2- 172.21.15.82:0/2521386011 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x55dc73db3d80 0x55dc741ac6c0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:19:16.933 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:16.933+0000 7f30df5db740 1 --2- 172.21.15.82:0/2521386011 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dc73dd44d0 0x55dc7419f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:19:16.933 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:16.933+0000 7f30df5db740 1 --2- 172.21.15.82:0/2521386011 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dc73df51b0 0x55dc741a6040 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:19:16.933 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:16.933+0000 7f30df5db740 1 --2- 172.21.15.82:0/2521386011 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55dc73dcc2f0 0x55dc7419eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:19:16.933 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:16.933+0000 7f30df5db740 1 -- 172.21.15.82:0/2521386011 >> 172.21.15.82:0/2521386011 conn(0x55dc73de9300 msgr2=0x55dc7418f810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:19:16.934 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:16.935+0000 7f30b9ffb640 0 client.25261 ms_handle_reset on v2:172.21.15.160:6838/2765554482 2024-05-20T16:19:16.936 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:16.935+0000 7f30df5db740 1 -- 172.21.15.82:0/2521386011 shutdown_connections 2024-05-20T16:19:16.936 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:16.935+0000 7f30df5db740 1 -- 172.21.15.82:0/2521386011 wait complete. 2024-05-20T16:19:16.936 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:16.935+0000 7f30df5db740 20 client.25261 trim_cache size 0 max 16384 2024-05-20T16:19:16.981 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:16.981+0000 7f30df5db740 1 Processor -- start 2024-05-20T16:19:16.981 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:16.981+0000 7f30df5db740 1 -- start start 2024-05-20T16:19:16.981 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:16.981+0000 7f30df5db740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dc73dcc2f0 0x55dc74190870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:19:16.981 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:16.981+0000 7f30df5db740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55dc73dd44d0 0x55dc741a5f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:19:16.981 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:16.981+0000 7f30df5db740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dc73df51b0 0x55dc741a64a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:19:16.981 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:16.981+0000 7f30df5db740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f30d01b4af0 con 0x55dc73dcc2f0 2024-05-20T16:19:16.981 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:16.981+0000 7f30df5db740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f30d01b23c0 con 0x55dc73dd44d0 2024-05-20T16:19:16.982 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:16.981+0000 7f30df5db740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f30d01b0210 con 0x55dc73df51b0 2024-05-20T16:19:16.982 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:16.982+0000 7f30d6bf7640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dc73df51b0 0x55dc741a64a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:19:16.982 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:16.982+0000 7f30d6bf7640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dc73df51b0 0x55dc741a64a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.82:51806/0 (socket says 172.21.15.82:51806) 2024-05-20T16:19:16.982 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:16.982+0000 7f30d6bf7640 1 -- 172.21.15.82:0/4193796393 learned_addr learned my addr 172.21.15.82:0/4193796393 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:19:16.982 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:16.982+0000 7f30d6bf7640 1 -- 172.21.15.82:0/4193796393 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55dc73dd44d0 msgr2=0x55dc741a5f60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-20T16:19:16.982 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:16.982+0000 7f30d6bf7640 1 --2- 172.21.15.82:0/4193796393 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55dc73dd44d0 0x55dc741a5f60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:19:16.983 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:16.982+0000 7f30d6bf7640 1 -- 172.21.15.82:0/4193796393 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dc73dcc2f0 msgr2=0x55dc74190870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:19:16.983 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:16.982+0000 7f30d6bf7640 1 --2- 172.21.15.82:0/4193796393 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dc73dcc2f0 0x55dc74190870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:19:16.983 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:16.982+0000 7f30d6bf7640 1 -- 172.21.15.82:0/4193796393 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc741999f0 con 0x55dc73df51b0 2024-05-20T16:19:16.983 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:16.983+0000 7f30d6bf7640 1 --2- 172.21.15.82:0/4193796393 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dc73df51b0 0x55dc741a64a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f30c8004c20 tx=0x7f30c80099e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fd0df91ecc88b5ab server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:19:16.984 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:16.984+0000 7f30bbfff640 1 -- 172.21.15.82:0/4193796393 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30c804f040 con 0x55dc73df51b0 2024-05-20T16:19:16.984 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:16.985+0000 7f30bbfff640 1 -- 172.21.15.82:0/4193796393 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f30c8059070 con 0x55dc73df51b0 2024-05-20T16:19:16.984 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:16.985+0000 7f30bbfff640 1 -- 172.21.15.82:0/4193796393 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30c804baf0 con 0x55dc73df51b0 2024-05-20T16:19:16.986 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:16.986+0000 7f30df5db740 1 -- 172.21.15.82:0/4193796393 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dc73df51b0 msgr2=0x55dc741a64a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:19:16.986 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:16.986+0000 7f30df5db740 1 --2- 172.21.15.82:0/4193796393 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dc73df51b0 0x55dc741a64a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f30c8004c20 tx=0x7f30c80099e0 comp rx=0 tx=0).stop 2024-05-20T16:19:16.990 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:16.991+0000 7f30df5db740 1 -- 172.21.15.82:0/4193796393 shutdown_connections 2024-05-20T16:19:16.991 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:16.991+0000 7f30df5db740 1 --2- 172.21.15.82:0/4193796393 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dc73df51b0 0x55dc741a64a0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:19:16.991 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:16.991+0000 7f30df5db740 1 --2- 172.21.15.82:0/4193796393 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55dc73dd44d0 0x55dc741a5f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:19:16.991 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:16.991+0000 7f30df5db740 1 --2- 172.21.15.82:0/4193796393 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dc73dcc2f0 0x55dc74190870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:19:16.991 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:16.991+0000 7f30df5db740 1 -- 172.21.15.82:0/4193796393 >> 172.21.15.82:0/4193796393 conn(0x55dc73de9300 msgr2=0x55dc74190160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:19:16.993 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:16.993+0000 7f30df5db740 1 -- 172.21.15.82:0/4193796393 shutdown_connections 2024-05-20T16:19:16.993 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:16.993+0000 7f30df5db740 1 -- 172.21.15.82:0/4193796393 wait complete. 2024-05-20T16:19:16.993 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:16.994+0000 7f30df5db740 1 -- start start 2024-05-20T16:19:16.994 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:16.994+0000 7f30df5db740 1 Processor -- start 2024-05-20T16:19:16.998 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:16.998+0000 7f30df5db740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dc73dcc2f0 0x55dc741a35c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:19:16.998 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:16.998+0000 7f30df5db740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55dc73dd44d0 0x55dc741a3b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:19:16.998 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:16.998+0000 7f30df5db740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dc73df51b0 0x55dc74212a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:19:16.998 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:16.998+0000 7f30df5db740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f30d01b0880 con 0x55dc73df51b0 2024-05-20T16:19:16.998 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:16.998+0000 7f30df5db740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f30d01b1cf0 con 0x55dc73dd44d0 2024-05-20T16:19:16.999 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:16.998+0000 7f30df5db740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f30d01b0f50 con 0x55dc73dcc2f0 2024-05-20T16:19:16.999 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:16.998+0000 7f30df5db740 10 client.0 _ll_register_callbacks cb 0x7f30cc006f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T16:19:16.999 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:16.998+0000 7f30d6bf7640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dc73df51b0 0x55dc74212a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:19:16.999 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:16.998+0000 7f30d5bf5640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55dc73dd44d0 0x55dc741a3b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:19:16.999 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:16.998+0000 7f30d6bf7640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dc73df51b0 0x55dc74212a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:38724/0 (socket says 172.21.15.82:38724) 2024-05-20T16:19:16.999 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:16.998+0000 7f30d6bf7640 1 -- 172.21.15.82:0/3716675515 learned_addr learned my addr 172.21.15.82:0/3716675515 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:19:16.999 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:16.998+0000 7f30d63f6640 1 --2- 172.21.15.82:0/3716675515 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dc73dcc2f0 0x55dc741a35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:19:17.000 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:16.999+0000 7f30d6bf7640 1 -- 172.21.15.82:0/3716675515 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dc73dcc2f0 msgr2=0x55dc741a35c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:19:17.000 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:16.999+0000 7f30d6bf7640 1 --2- 172.21.15.82:0/3716675515 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dc73dcc2f0 0x55dc741a35c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:19:17.000 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:16.999+0000 7f30d6bf7640 1 -- 172.21.15.82:0/3716675515 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55dc73dd44d0 msgr2=0x55dc741a3b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:19:17.000 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:16.999+0000 7f30d6bf7640 1 --2- 172.21.15.82:0/3716675515 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55dc73dd44d0 0x55dc741a3b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:19:17.000 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:16.999+0000 7f30d6bf7640 1 -- 172.21.15.82:0/3716675515 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dc741a6ac0 con 0x55dc73df51b0 2024-05-20T16:19:17.000 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:17.000+0000 7f30d6bf7640 1 --2- 172.21.15.82:0/3716675515 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dc73df51b0 0x55dc74212a20 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f30c8004bf0 tx=0x7f30c802ad70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e557552bfbd7e01a server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:19:17.000 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:17.000+0000 7f30b9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-05-20T16:19:17.001 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:17.001+0000 7f30b9ffb640 1 -- 172.21.15.82:0/3716675515 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30c804f070 con 0x55dc73df51b0 2024-05-20T16:19:17.001 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:17.001+0000 7f30df5db740 1 -- 172.21.15.82:0/3716675515 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f30c8059070 con 0x55dc73df51b0 2024-05-20T16:19:17.001 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:17.001+0000 7f30b9ffb640 1 -- 172.21.15.82:0/3716675515 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9150+0+0 (secure 0 0 0) 0x7f30c804ba90 con 0x55dc73df51b0 2024-05-20T16:19:17.002 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:17.002+0000 7f30b9ffb640 1 -- 172.21.15.82:0/3716675515 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f30c8078050 con 0x55dc73df51b0 2024-05-20T16:19:17.002 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:17.003+0000 7f30df5db740 10 client.15711 fetch_fsmap learned FSMap version 27 2024-05-20T16:19:17.003 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:17.003+0000 7f30df5db740 1 -- 172.21.15.82:0/3716675515 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55dc74213f60 con 0x55dc73df51b0 2024-05-20T16:19:17.003 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:17.004+0000 7f30b9ffb640 1 -- 172.21.15.82:0/3716675515 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f30c8059070 con 0x55dc73df51b0 2024-05-20T16:19:17.004 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:17.004+0000 7f30b9ffb640 1 -- 172.21.15.82:0/3716675515 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f30c806f450 con 0x55dc73df51b0 2024-05-20T16:19:17.004 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:17.004+0000 7f30df5db740 10 client.15711 fetch_fsmap finished waiting for FSMap version 27 2024-05-20T16:19:17.004 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:17.004+0000 7f30df5db740 10 client.15711 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T16:19:17.004 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:17.004+0000 7f30df5db740 20 client.15711 populate_metadata read hostname 'smithi082' 2024-05-20T16:19:17.004 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:17.004+0000 7f30df5db740 1 --2- 172.21.15.82:0/3716675515 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x55dc73d891c0 0x55dc742167f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:19:17.004 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:17.004+0000 7f30df5db740 4 client.15711 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-20T16:19:17.004 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:17.004+0000 7f30df5db740 1 -- 172.21.15.82:0/3716675515 --> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55dc74100dc0 con 0x55dc73d891c0 2024-05-20T16:19:17.004 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:17.004+0000 7f30d63f6640 1 --2- 172.21.15.82:0/3716675515 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x55dc73d891c0 0x55dc742167f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:19:17.005 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:17.005+0000 7f30b9ffb640 10 client.15711 ms_handle_connect on v2:172.21.15.160:6838/2765554482 2024-05-20T16:19:17.006 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:17.006+0000 7f30d63f6640 1 --2- 172.21.15.82:0/3716675515 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x55dc73d891c0 0x55dc742167f0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2d068be4dd7b50ae server_cookie=a5df77484ca50a25 in_seq=0 out_seq=0 2024-05-20T16:19:17.885 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:19:17 smithi184 ceph-mon[36209]: pgmap v522: 129 pgs: 129 active+clean; 10 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 722 MiB/s wr, 1.30k op/s 2024-05-20T16:19:18.062 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:19:17 smithi160 ceph-mon[36347]: pgmap v522: 129 pgs: 129 active+clean; 10 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 722 MiB/s wr, 1.30k op/s 2024-05-20T16:19:18.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:19:17 smithi082 ceph-mon[32238]: pgmap v522: 129 pgs: 129 active+clean; 10 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 722 MiB/s wr, 1.30k op/s 2024-05-20T16:19:19.885 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:19:19 smithi184 ceph-mon[36209]: pgmap v523: 129 pgs: 129 active+clean; 10 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 857 MiB/s wr, 1.63k op/s 2024-05-20T16:19:20.062 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:19:19 smithi160 ceph-mon[36347]: pgmap v523: 129 pgs: 129 active+clean; 10 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 857 MiB/s wr, 1.63k op/s 2024-05-20T16:19:20.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:19:19 smithi082 ceph-mon[32238]: pgmap v523: 129 pgs: 129 active+clean; 10 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 857 MiB/s wr, 1.63k op/s 2024-05-20T16:19:22.062 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:19:21 smithi160 ceph-mon[36347]: pgmap v524: 129 pgs: 129 active+clean; 10 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 755 MiB/s wr, 1.31k op/s 2024-05-20T16:19:22.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:19:21 smithi082 ceph-mon[32238]: pgmap v524: 129 pgs: 129 active+clean; 10 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 755 MiB/s wr, 1.31k op/s 2024-05-20T16:19:22.135 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:19:21 smithi184 ceph-mon[36209]: pgmap v524: 129 pgs: 129 active+clean; 10 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 755 MiB/s wr, 1.31k op/s 2024-05-20T16:19:23.885 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:19:23 smithi184 ceph-mon[36209]: pgmap v525: 129 pgs: 129 active+clean; 10 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 831 MiB/s wr, 1.41k op/s 2024-05-20T16:19:24.062 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:19:23 smithi160 ceph-mon[36347]: pgmap v525: 129 pgs: 129 active+clean; 10 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 831 MiB/s wr, 1.41k op/s 2024-05-20T16:19:24.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:19:23 smithi082 ceph-mon[32238]: pgmap v525: 129 pgs: 129 active+clean; 10 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 831 MiB/s wr, 1.41k op/s 2024-05-20T16:19:25.885 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:19:25 smithi184 ceph-mon[36209]: pgmap v526: 129 pgs: 129 active+clean; 10 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 826 MiB/s wr, 1.44k op/s 2024-05-20T16:19:26.062 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:19:25 smithi160 ceph-mon[36347]: pgmap v526: 129 pgs: 129 active+clean; 10 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 826 MiB/s wr, 1.44k op/s 2024-05-20T16:19:26.110 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:19:25 smithi082 ceph-mon[32238]: pgmap v526: 129 pgs: 129 active+clean; 10 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 826 MiB/s wr, 1.44k op/s 2024-05-20T16:19:28.062 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:19:27 smithi160 ceph-mon[36347]: pgmap v527: 129 pgs: 129 active+clean; 10 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 734 MiB/s wr, 1.37k op/s 2024-05-20T16:19:28.119 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:19:27 smithi082 ceph-mon[32238]: pgmap v527: 129 pgs: 129 active+clean; 10 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 734 MiB/s wr, 1.37k op/s 2024-05-20T16:19:28.135 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:19:27 smithi184 ceph-mon[36209]: pgmap v527: 129 pgs: 129 active+clean; 10 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 734 MiB/s wr, 1.37k op/s 2024-05-20T16:19:30.062 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:19:29 smithi160 ceph-mon[36347]: pgmap v528: 129 pgs: 129 active+clean; 10 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 913 MiB/s wr, 1.60k op/s 2024-05-20T16:19:30.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:19:29 smithi082 ceph-mon[32238]: pgmap v528: 129 pgs: 129 active+clean; 10 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 913 MiB/s wr, 1.60k op/s 2024-05-20T16:19:30.135 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:19:29 smithi184 ceph-mon[36209]: pgmap v528: 129 pgs: 129 active+clean; 10 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 913 MiB/s wr, 1.60k op/s 2024-05-20T16:19:32.062 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:19:31 smithi160 ceph-mon[36347]: pgmap v529: 129 pgs: 129 active+clean; 10 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 714 MiB/s wr, 1.15k op/s 2024-05-20T16:19:32.115 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:19:31 smithi082 ceph-mon[32238]: pgmap v529: 129 pgs: 129 active+clean; 10 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 714 MiB/s wr, 1.15k op/s 2024-05-20T16:19:32.135 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:19:31 smithi184 ceph-mon[36209]: pgmap v529: 129 pgs: 129 active+clean; 10 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 714 MiB/s wr, 1.15k op/s 2024-05-20T16:19:34.062 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:19:33 smithi160 ceph-mon[36347]: pgmap v530: 129 pgs: 129 active+clean; 10 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 845 MiB/s wr, 1.33k op/s 2024-05-20T16:19:34.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:19:33 smithi082 ceph-mon[32238]: pgmap v530: 129 pgs: 129 active+clean; 10 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 845 MiB/s wr, 1.33k op/s 2024-05-20T16:19:34.136 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:19:33 smithi184 ceph-mon[36209]: pgmap v530: 129 pgs: 129 active+clean; 10 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 845 MiB/s wr, 1.33k op/s 2024-05-20T16:19:34.736 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:34.736+0000 7f30b9ffb640 1 -- 172.21.15.82:0/3716675515 <== mds.0 v2:172.21.15.160:6838/2765554482 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55dc74100dc0 con 0x55dc73d891c0 2024-05-20T16:19:34.737 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:34.736+0000 7f30b9ffb640 10 client.15711 handle_command_reply: tid=0 2024-05-20T16:19:34.737 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:34.736+0000 7f30df5db740 1 client.15711 shutdown 2024-05-20T16:19:34.737 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:34.737+0000 7f30df5db740 1 -- 172.21.15.82:0/3716675515 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dc73df51b0 msgr2=0x55dc74212a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:19:34.737 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:34.737+0000 7f30df5db740 1 --2- 172.21.15.82:0/3716675515 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dc73df51b0 0x55dc74212a20 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f30c8004bf0 tx=0x7f30c802ad70 comp rx=0 tx=0).stop 2024-05-20T16:19:34.737 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:34.737+0000 7f30df5db740 1 -- 172.21.15.82:0/3716675515 shutdown_connections 2024-05-20T16:19:34.738 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:34.737+0000 7f30df5db740 1 --2- 172.21.15.82:0/3716675515 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x55dc73d891c0 0x55dc742167f0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:19:34.738 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:34.737+0000 7f30df5db740 1 --2- 172.21.15.82:0/3716675515 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dc73df51b0 0x55dc74212a20 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:19:34.738 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:34.737+0000 7f30df5db740 1 --2- 172.21.15.82:0/3716675515 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55dc73dd44d0 0x55dc741a3b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:19:34.738 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:34.737+0000 7f30df5db740 1 --2- 172.21.15.82:0/3716675515 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dc73dcc2f0 0x55dc741a35c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:19:34.738 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:34.737+0000 7f30df5db740 1 -- 172.21.15.82:0/3716675515 >> 172.21.15.82:0/3716675515 conn(0x55dc73de9300 msgr2=0x55dc7418ede0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:19:34.738 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:34.737+0000 7f30b9ffb640 0 client.15711 ms_handle_reset on v2:172.21.15.160:6838/2765554482 2024-05-20T16:19:34.738 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:34.738+0000 7f30df5db740 1 -- 172.21.15.82:0/3716675515 shutdown_connections 2024-05-20T16:19:34.740 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:34.740+0000 7f30df5db740 1 -- 172.21.15.82:0/3716675515 wait complete. 2024-05-20T16:19:34.741 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:34.741+0000 7f30df5db740 20 client.15711 trim_cache size 0 max 16384 2024-05-20T16:19:34.743 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:{ 2024-05-20T16:19:34.743 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "message": "", 2024-05-20T16:19:34.743 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "return_code": 0 2024-05-20T16:19:34.743 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:} 2024-05-20T16:19:34.751 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:34.750+0000 7f30d865a640 1 -- 172.21.15.82:0/1112699451 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f30bc0a2d90 msgr2=0x7f30bc0a52a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:19:34.751 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:34.750+0000 7f30d865a640 1 --2- 172.21.15.82:0/1112699451 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f30bc0a2d90 0x7f30bc0a52a0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f30c8005df0 tx=0x7f30c804a000 comp rx=0 tx=0).stop 2024-05-20T16:19:34.751 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:34.751+0000 7f30d865a640 1 -- 172.21.15.82:0/1112699451 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f30d0102c10 msgr2=0x7f30d0070f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:19:34.751 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:34.751+0000 7f30d865a640 1 --2- 172.21.15.82:0/1112699451 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f30d0102c10 0x7f30d0070f60 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f30c000d930 tx=0x7f30c000ddf0 comp rx=0 tx=0).stop 2024-05-20T16:19:34.756 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:34.756+0000 7f30d865a640 1 -- 172.21.15.82:0/1112699451 shutdown_connections 2024-05-20T16:19:34.756 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:34.756+0000 7f30d865a640 1 --2- 172.21.15.82:0/1112699451 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f30bc0a2d90 0x7f30bc0a52a0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:19:34.756 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:34.756+0000 7f30d865a640 1 --2- 172.21.15.82:0/1112699451 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f30d0107910 0x7f30d0071ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:19:34.756 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:34.756+0000 7f30d865a640 1 --2- 172.21.15.82:0/1112699451 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f30d0102c10 0x7f30d0070f60 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:19:34.757 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:34.756+0000 7f30d865a640 1 --2- 172.21.15.82:0/1112699451 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f30d01019d0 0x7f30d0070a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:19:34.757 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:34.756+0000 7f30d865a640 1 -- 172.21.15.82:0/1112699451 >> 172.21.15.82:0/1112699451 conn(0x7f30d00fcbc0 msgr2=0x7f30d01038f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:19:34.761 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:34.761+0000 7f30d865a640 1 -- 172.21.15.82:0/1112699451 shutdown_connections 2024-05-20T16:19:34.762 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:34.762+0000 7f30d865a640 1 -- 172.21.15.82:0/1112699451 wait complete. 2024-05-20T16:19:36.885 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:19:36 smithi184 ceph-mon[36209]: pgmap v531: 129 pgs: 129 active+clean; 10 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 820 MiB/s wr, 1.26k op/s 2024-05-20T16:19:37.062 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:19:36 smithi160 ceph-mon[36347]: pgmap v531: 129 pgs: 129 active+clean; 10 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 820 MiB/s wr, 1.26k op/s 2024-05-20T16:19:37.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:19:36 smithi082 ceph-mon[32238]: pgmap v531: 129 pgs: 129 active+clean; 10 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 820 MiB/s wr, 1.26k op/s 2024-05-20T16:19:38.062 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:19:37 smithi160 ceph-mon[36347]: pgmap v532: 129 pgs: 129 active+clean; 10 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 743 MiB/s wr, 1.14k op/s 2024-05-20T16:19:38.112 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:19:37 smithi082 ceph-mon[32238]: pgmap v532: 129 pgs: 129 active+clean; 10 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 743 MiB/s wr, 1.14k op/s 2024-05-20T16:19:38.135 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:19:37 smithi184 ceph-mon[36209]: pgmap v532: 129 pgs: 129 active+clean; 10 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 743 MiB/s wr, 1.14k op/s 2024-05-20T16:19:40.885 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:19:40 smithi184 ceph-mon[36209]: pgmap v533: 129 pgs: 129 active+clean; 10 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 909 MiB/s wr, 1.63k op/s 2024-05-20T16:19:41.062 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:19:40 smithi160 ceph-mon[36347]: pgmap v533: 129 pgs: 129 active+clean; 10 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 909 MiB/s wr, 1.63k op/s 2024-05-20T16:19:41.110 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:19:40 smithi082 ceph-mon[32238]: pgmap v533: 129 pgs: 129 active+clean; 10 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 909 MiB/s wr, 1.63k op/s 2024-05-20T16:19:43.062 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:19:42 smithi160 ceph-mon[36347]: pgmap v534: 129 pgs: 129 active+clean; 10 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 709 MiB/s wr, 1.42k op/s 2024-05-20T16:19:43.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:19:42 smithi082 ceph-mon[32238]: pgmap v534: 129 pgs: 129 active+clean; 10 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 709 MiB/s wr, 1.42k op/s 2024-05-20T16:19:43.135 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:19:42 smithi184 ceph-mon[36209]: pgmap v534: 129 pgs: 129 active+clean; 10 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 709 MiB/s wr, 1.42k op/s 2024-05-20T16:19:43.886 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:19:43 smithi184 ceph-mon[36209]: pgmap v535: 129 pgs: 129 active+clean; 10 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 802 MiB/s wr, 1.69k op/s 2024-05-20T16:19:44.010 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:19:43 smithi082 ceph-mon[32238]: pgmap v535: 129 pgs: 129 active+clean; 10 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 802 MiB/s wr, 1.69k op/s 2024-05-20T16:19:44.062 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:19:43 smithi160 ceph-mon[36347]: pgmap v535: 129 pgs: 129 active+clean; 10 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 802 MiB/s wr, 1.69k op/s 2024-05-20T16:19:46.885 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:19:46 smithi184 ceph-mon[36209]: pgmap v536: 129 pgs: 129 active+clean; 10 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 762 MiB/s wr, 1.74k op/s 2024-05-20T16:19:47.062 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:19:46 smithi160 ceph-mon[36347]: pgmap v536: 129 pgs: 129 active+clean; 10 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 762 MiB/s wr, 1.74k op/s 2024-05-20T16:19:47.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:19:46 smithi082 ceph-mon[32238]: pgmap v536: 129 pgs: 129 active+clean; 10 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 762 MiB/s wr, 1.74k op/s 2024-05-20T16:19:48.032 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:48.029+0000 7fb858901640 1 -- 172.21.15.82:0/251601205 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb854101920 msgr2=0x7fb854101d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:19:48.032 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:48.029+0000 7fb858901640 1 --2- 172.21.15.82:0/251601205 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb854101920 0x7fb854101d30 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fb848009900 tx=0x7fb84802a440 comp rx=0 tx=0).stop 2024-05-20T16:19:48.033 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:48.032+0000 7fb858901640 1 -- 172.21.15.82:0/251601205 shutdown_connections 2024-05-20T16:19:48.033 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:48.032+0000 7fb858901640 1 --2- 172.21.15.82:0/251601205 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb854101920 0x7fb854101d30 secure :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fb848009900 tx=0x7fb84802a440 comp rx=0 tx=0).stop 2024-05-20T16:19:48.033 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:48.032+0000 7fb858901640 1 --2- 172.21.15.82:0/251601205 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb854106640 0x7fb854106a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:19:48.033 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:48.032+0000 7fb858901640 1 --2- 172.21.15.82:0/251601205 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb854100820 0x7fb8541013e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:19:48.033 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:48.032+0000 7fb858901640 1 -- 172.21.15.82:0/251601205 >> 172.21.15.82:0/251601205 conn(0x7fb8540fbd20 msgr2=0x7fb8540fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:19:48.043 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:48.038+0000 7fb858901640 1 -- 172.21.15.82:0/251601205 shutdown_connections 2024-05-20T16:19:48.043 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:48.038+0000 7fb858901640 1 -- 172.21.15.82:0/251601205 wait complete. 2024-05-20T16:19:48.043 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:48.039+0000 7fb858901640 1 Processor -- start 2024-05-20T16:19:48.043 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:48.039+0000 7fb858901640 1 -- start start 2024-05-20T16:19:48.043 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:48.040+0000 7fb858901640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb854100820 0x7fb854070a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:19:48.044 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:48.040+0000 7fb858901640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb854106640 0x7fb854070fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:19:48.044 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:48.040+0000 7fb858901640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb854071ea0 0x7fb8540722b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:19:48.044 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:48.040+0000 7fb858901640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fb854114c70 con 0x7fb854071ea0 2024-05-20T16:19:48.044 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:48.040+0000 7fb858901640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fb854114af0 con 0x7fb854100820 2024-05-20T16:19:48.044 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:48.040+0000 7fb858901640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fb854114df0 con 0x7fb854106640 2024-05-20T16:19:48.044 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:48.041+0000 7fb85359e640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb854071ea0 0x7fb8540722b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:19:48.044 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:48.041+0000 7fb85359e640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb854071ea0 0x7fb8540722b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:50284/0 (socket says 172.21.15.82:50284) 2024-05-20T16:19:48.044 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:48.042+0000 7fb852d9d640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb854100820 0x7fb854070a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:19:48.044 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:48.042+0000 7fb852d9d640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb854100820 0x7fb854070a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.82:38580/0 (socket says 172.21.15.82:38580) 2024-05-20T16:19:48.044 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:48.042+0000 7fb852d9d640 1 -- 172.21.15.82:0/2270905894 learned_addr learned my addr 172.21.15.82:0/2270905894 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:19:48.047 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:48.045+0000 7fb852d9d640 1 -- 172.21.15.82:0/2270905894 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb854106640 msgr2=0x7fb854070fc0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-20T16:19:48.047 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:48.045+0000 7fb852d9d640 1 --2- 172.21.15.82:0/2270905894 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb854106640 0x7fb854070fc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:19:48.048 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:48.045+0000 7fb852d9d640 1 -- 172.21.15.82:0/2270905894 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb854071ea0 msgr2=0x7fb8540722b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:19:48.048 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:48.045+0000 7fb852d9d640 1 --2- 172.21.15.82:0/2270905894 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb854071ea0 0x7fb8540722b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:19:48.048 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:48.045+0000 7fb852d9d640 1 -- 172.21.15.82:0/2270905894 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8540729e0 con 0x7fb854100820 2024-05-20T16:19:48.059 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:48.051+0000 7fb852d9d640 1 --2- 172.21.15.82:0/2270905894 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb854100820 0x7fb854070a00 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fb84400b450 tx=0x7fb84400b910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:19:48.059 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:48.055+0000 7fb83bfff640 1 -- 172.21.15.82:0/2270905894 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb844015020 con 0x7fb854100820 2024-05-20T16:19:48.059 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:48.055+0000 7fb83bfff640 1 -- 172.21.15.82:0/2270905894 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb844011d70 con 0x7fb854100820 2024-05-20T16:19:48.059 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:48.055+0000 7fb83bfff640 1 -- 172.21.15.82:0/2270905894 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb84400f340 con 0x7fb854100820 2024-05-20T16:19:48.059 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:48.058+0000 7fb858901640 1 -- 172.21.15.82:0/2270905894 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb854072cd0 con 0x7fb854100820 2024-05-20T16:19:48.059 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:48.058+0000 7fb858901640 1 -- 172.21.15.82:0/2270905894 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8540731a0 con 0x7fb854100820 2024-05-20T16:19:48.061 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:19:47 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T16:19:48.063 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:48.062+0000 7fb85ffaa740 1 Processor -- start 2024-05-20T16:19:48.063 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:48.062+0000 7fb85ffaa740 1 -- start start 2024-05-20T16:19:48.063 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:48.063+0000 7fb85ffaa740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fa240491b0 0x55fa243e9060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:19:48.063 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:48.063+0000 7fb85ffaa740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fa240202f0 0x55fa243e95e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:19:48.063 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:48.063+0000 7fb85ffaa740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fa240284d0 0x55fa243f04e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:19:48.063 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:48.063+0000 7fb85ffaa740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55fa2434d340 con 0x55fa240202f0 2024-05-20T16:19:48.063 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:48.063+0000 7fb85ffaa740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x55fa242f5290 con 0x55fa240491b0 2024-05-20T16:19:48.064 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:48.063+0000 7fb85ffaa740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55fa24349300 con 0x55fa240284d0 2024-05-20T16:19:48.067 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:48.064+0000 7fb85359e640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fa240491b0 0x55fa243e9060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:19:48.067 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:48.064+0000 7fb85359e640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fa240491b0 0x55fa243e9060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.82:38582/0 (socket says 172.21.15.82:38582) 2024-05-20T16:19:48.069 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:48.064+0000 7fb85359e640 1 -- 172.21.15.82:0/3699171134 learned_addr learned my addr 172.21.15.82:0/3699171134 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:19:48.069 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:48.064+0000 7fb852d9d640 1 --2- 172.21.15.82:0/3699171134 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fa240202f0 0x55fa243e95e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:19:48.069 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:48.065+0000 7fb852d9d640 1 -- 172.21.15.82:0/3699171134 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fa240284d0 msgr2=0x55fa243f04e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:19:48.069 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:48.065+0000 7fb852d9d640 1 --2- 172.21.15.82:0/3699171134 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fa240284d0 0x55fa243f04e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:19:48.069 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:48.065+0000 7fb852d9d640 1 -- 172.21.15.82:0/3699171134 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fa240491b0 msgr2=0x55fa243e9060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:19:48.069 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:48.065+0000 7fb852d9d640 1 --2- 172.21.15.82:0/3699171134 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fa240491b0 0x55fa243e9060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:19:48.069 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:48.065+0000 7fb852d9d640 1 -- 172.21.15.82:0/3699171134 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fa243198e0 con 0x55fa240202f0 2024-05-20T16:19:48.069 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:48.066+0000 7fb852d9d640 1 --2- 172.21.15.82:0/3699171134 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fa240202f0 0x55fa243e95e0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fb84400be10 tx=0x7fb844010890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5b75ad81a803625 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:19:48.073 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:48.069+0000 7fb8397fa640 1 -- 172.21.15.82:0/3699171134 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb84400fc40 con 0x55fa240202f0 2024-05-20T16:19:48.073 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:48.070+0000 7fb8397fa640 1 -- 172.21.15.82:0/3699171134 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb84400fde0 con 0x55fa240202f0 2024-05-20T16:19:48.073 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:48.071+0000 7fb83bfff640 1 -- 172.21.15.82:0/2270905894 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fb844018020 con 0x7fb854100820 2024-05-20T16:19:48.074 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:48.072+0000 7fb85ffaa740 1 -- 172.21.15.82:0/3699171134 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fa240202f0 msgr2=0x55fa243e95e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:19:48.074 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:48.072+0000 7fb85ffaa740 1 --2- 172.21.15.82:0/3699171134 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fa240202f0 0x55fa243e95e0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fb84400be10 tx=0x7fb844010890 comp rx=0 tx=0).stop 2024-05-20T16:19:48.074 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:48.073+0000 7fb83bfff640 1 --2- 172.21.15.82:0/2270905894 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fb83c0a2d10 0x7fb83c0a5220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:19:48.074 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:48.073+0000 7fb83bfff640 1 -- 172.21.15.82:0/2270905894 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9150+0+0 (secure 0 0 0) 0x7fb8440d37c0 con 0x7fb854100820 2024-05-20T16:19:48.074 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:48.074+0000 7fb85359e640 1 --2- 172.21.15.82:0/2270905894 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fb83c0a2d10 0x7fb83c0a5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:19:48.075 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:48.075+0000 7fb85ffaa740 1 -- 172.21.15.82:0/3699171134 shutdown_connections 2024-05-20T16:19:48.075 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:48.075+0000 7fb85ffaa740 1 --2- 172.21.15.82:0/3699171134 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fa240202f0 0x55fa243e95e0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:19:48.075 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:48.075+0000 7fb85ffaa740 1 --2- 172.21.15.82:0/3699171134 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fa240284d0 0x55fa243f04e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:19:48.075 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:48.075+0000 7fb85ffaa740 1 --2- 172.21.15.82:0/3699171134 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fa240491b0 0x55fa243e9060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:19:48.075 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:48.075+0000 7fb85ffaa740 1 -- 172.21.15.82:0/3699171134 >> 172.21.15.82:0/3699171134 conn(0x55fa2403d300 msgr2=0x55fa243e49f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:19:48.078 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:48.077+0000 7fb85ffaa740 1 -- 172.21.15.82:0/3699171134 shutdown_connections 2024-05-20T16:19:48.083 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:48.079+0000 7fb85ffaa740 1 -- 172.21.15.82:0/3699171134 wait complete. 2024-05-20T16:19:48.083 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:48.080+0000 7fb85ffaa740 1 -- start start 2024-05-20T16:19:48.083 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:48.080+0000 7fb85ffaa740 1 Processor -- start 2024-05-20T16:19:48.083 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:48.081+0000 7fb85ffaa740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fa240202f0 0x55fa243f2c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:19:48.083 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:48.081+0000 7fb85ffaa740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fa240284d0 0x55fa243f3180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:19:48.084 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:48.081+0000 7fb85ffaa740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fa240491b0 0x55fa243fa0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:19:48.084 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:48.081+0000 7fb85ffaa740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55fa243f0ed0 con 0x55fa240491b0 2024-05-20T16:19:48.084 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:48.081+0000 7fb85ffaa740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x55fa24308790 con 0x55fa240284d0 2024-05-20T16:19:48.084 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:48.081+0000 7fb85ffaa740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55fa243f1050 con 0x55fa240202f0 2024-05-20T16:19:48.084 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:48.081+0000 7fb85ffaa740 10 client.0 _ll_register_callbacks cb 0x55fa24343a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T16:19:48.084 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:48.081+0000 7fb85359e640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fa240491b0 0x55fa243fa0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:19:48.084 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:48.081+0000 7fb85359e640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fa240491b0 0x55fa243fa0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:50304/0 (socket says 172.21.15.82:50304) 2024-05-20T16:19:48.084 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:48.081+0000 7fb85359e640 1 -- 172.21.15.82:0/534675948 learned_addr learned my addr 172.21.15.82:0/534675948 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:19:48.084 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:48.082+0000 7fb85259c640 1 --2- 172.21.15.82:0/534675948 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fa240284d0 0x55fa243f3180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:19:48.084 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:48.082+0000 7fb852d9d640 1 --2- 172.21.15.82:0/534675948 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fa240202f0 0x55fa243f2c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:19:48.084 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:48.083+0000 7fb85259c640 1 -- 172.21.15.82:0/534675948 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fa240202f0 msgr2=0x55fa243f2c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:19:48.084 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:48.083+0000 7fb85259c640 1 --2- 172.21.15.82:0/534675948 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fa240202f0 0x55fa243f2c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:19:48.084 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:48.083+0000 7fb85259c640 1 -- 172.21.15.82:0/534675948 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fa240491b0 msgr2=0x55fa243fa0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:19:48.084 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:48.083+0000 7fb85259c640 1 --2- 172.21.15.82:0/534675948 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fa240491b0 0x55fa243fa0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:19:48.085 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:48.083+0000 7fb85259c640 1 -- 172.21.15.82:0/534675948 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fa24061210 con 0x55fa240284d0 2024-05-20T16:19:48.086 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:48.086+0000 7fb85259c640 1 --2- 172.21.15.82:0/534675948 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fa240284d0 0x55fa243f3180 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fb848008000 tx=0x7fb848032e50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=11b76d80fc0efe1d server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:19:48.089 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:48.088+0000 7fb836ff5640 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-20T16:19:48.090 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:48.090+0000 7fb836ff5640 1 -- 172.21.15.82:0/534675948 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb84803f030 con 0x55fa240284d0 2024-05-20T16:19:48.090 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:48.090+0000 7fb85ffaa740 1 -- 172.21.15.82:0/534675948 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb84400fde0 con 0x55fa240284d0 2024-05-20T16:19:48.090 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:48.090+0000 7fb836ff5640 1 -- 172.21.15.82:0/534675948 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9150+0+0 (secure 0 0 0) 0x7fb848031850 con 0x55fa240284d0 2024-05-20T16:19:48.094 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:48.095+0000 7fb836ff5640 1 -- 172.21.15.82:0/534675948 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb84804b050 con 0x55fa240284d0 2024-05-20T16:19:48.098 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:48.099+0000 7fb85359e640 1 --2- 172.21.15.82:0/2270905894 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fb83c0a2d10 0x7fb83c0a5220 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fb84c00adb0 tx=0x7fb84c009650 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:19:48.107 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:48.107+0000 7fb85ffaa740 10 client.25250 fetch_fsmap learned FSMap version 27 2024-05-20T16:19:48.107 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:48.107+0000 7fb85ffaa740 1 -- 172.21.15.82:0/534675948 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55fa2412d620 con 0x55fa240284d0 2024-05-20T16:19:48.112 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:19:47 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T16:19:48.113 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:48.110+0000 7fb836ff5640 1 -- 172.21.15.82:0/534675948 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb84400fde0 con 0x55fa240284d0 2024-05-20T16:19:48.113 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:48.111+0000 7fb836ff5640 1 -- 172.21.15.82:0/534675948 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fb8480046c0 con 0x55fa240284d0 2024-05-20T16:19:48.113 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:48.112+0000 7fb85ffaa740 10 client.25250 fetch_fsmap finished waiting for FSMap version 27 2024-05-20T16:19:48.113 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:48.112+0000 7fb85ffaa740 10 client.25250 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T16:19:48.113 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:48.112+0000 7fb85ffaa740 20 client.25250 populate_metadata read hostname 'smithi082' 2024-05-20T16:19:48.113 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:48.112+0000 7fb85ffaa740 1 --2- 172.21.15.82:0/534675948 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x55fa24007d80 0x55fa24400a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:19:48.113 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:48.112+0000 7fb85ffaa740 4 client.25250 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T16:19:48.113 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:48.112+0000 7fb85ffaa740 1 -- 172.21.15.82:0/534675948 --> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fa2435aa70 con 0x55fa24007d80 2024-05-20T16:19:48.113 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:48.113+0000 7fb852d9d640 1 --2- 172.21.15.82:0/534675948 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x55fa24007d80 0x55fa24400a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:19:48.123 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:48.123+0000 7fb852d9d640 1 --2- 172.21.15.82:0/534675948 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x55fa24007d80 0x55fa24400a20 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8eefb233f97c7d31 server_cookie=433cce5d8d2ecb18 in_seq=0 out_seq=0 2024-05-20T16:19:48.123 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:48.123+0000 7fb836ff5640 10 client.25250 ms_handle_connect on v2:172.21.15.160:6838/2765554482 2024-05-20T16:19:48.127 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:48.127+0000 7fb836ff5640 1 -- 172.21.15.82:0/534675948 <== mds.0 v2:172.21.15.160:6838/2765554482 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55fa2435aa70 con 0x55fa24007d80 2024-05-20T16:19:48.127 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:48.128+0000 7fb836ff5640 10 client.25250 handle_command_reply: tid=0 2024-05-20T16:19:48.127 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:48.128+0000 7fb85ffaa740 1 client.25250 shutdown 2024-05-20T16:19:48.128 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:48.128+0000 7fb85ffaa740 1 -- 172.21.15.82:0/534675948 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fa240284d0 msgr2=0x55fa243f3180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:19:48.128 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:48.128+0000 7fb85ffaa740 1 --2- 172.21.15.82:0/534675948 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fa240284d0 0x55fa243f3180 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fb848008000 tx=0x7fb848032e50 comp rx=0 tx=0).stop 2024-05-20T16:19:48.128 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:48.128+0000 7fb85ffaa740 1 -- 172.21.15.82:0/534675948 shutdown_connections 2024-05-20T16:19:48.128 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:48.128+0000 7fb85ffaa740 1 --2- 172.21.15.82:0/534675948 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x55fa24007d80 0x55fa24400a20 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:19:48.128 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:48.128+0000 7fb85ffaa740 1 --2- 172.21.15.82:0/534675948 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fa240491b0 0x55fa243fa0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:19:48.129 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:48.128+0000 7fb85ffaa740 1 --2- 172.21.15.82:0/534675948 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fa240284d0 0x55fa243f3180 secure :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fb848008000 tx=0x7fb848032e50 comp rx=0 tx=0).stop 2024-05-20T16:19:48.129 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:48.128+0000 7fb85ffaa740 1 --2- 172.21.15.82:0/534675948 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fa240202f0 0x55fa243f2c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:19:48.129 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:48.128+0000 7fb85ffaa740 1 -- 172.21.15.82:0/534675948 >> 172.21.15.82:0/534675948 conn(0x55fa2403d300 msgr2=0x55fa243e79a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:19:48.129 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:48.128+0000 7fb836ff5640 0 client.25250 ms_handle_reset on v2:172.21.15.160:6838/2765554482 2024-05-20T16:19:48.129 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:48.129+0000 7fb85ffaa740 1 -- 172.21.15.82:0/534675948 shutdown_connections 2024-05-20T16:19:48.129 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:48.129+0000 7fb85ffaa740 1 -- 172.21.15.82:0/534675948 wait complete. 2024-05-20T16:19:48.129 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:48.129+0000 7fb85ffaa740 20 client.25250 trim_cache size 0 max 16384 2024-05-20T16:19:48.135 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:19:47 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T16:19:48.167 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:48.167+0000 7fb85ffaa740 1 Processor -- start 2024-05-20T16:19:48.167 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:48.168+0000 7fb85ffaa740 1 -- start start 2024-05-20T16:19:48.169 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:48.169+0000 7fb85ffaa740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fa240202f0 0x55fa243ed090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:19:48.169 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:48.169+0000 7fb85ffaa740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fa240491b0 0x55fa243ed5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:19:48.170 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:48.170+0000 7fb852d9d640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fa240202f0 0x55fa243ed090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:19:48.170 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:48.170+0000 7fb852d9d640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fa240202f0 0x55fa243ed090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.82:38592/0 (socket says 172.21.15.82:38592) 2024-05-20T16:19:48.170 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:48.170+0000 7fb85ffaa740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fa23fdd1c0 0x55fa244061f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:19:48.170 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:48.170+0000 7fb852d9d640 1 -- 172.21.15.82:0/387509609 learned_addr learned my addr 172.21.15.82:0/387509609 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:19:48.170 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:48.170+0000 7fb85ffaa740 1 -- 172.21.15.82:0/387509609 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fb8541ce540 con 0x55fa240491b0 2024-05-20T16:19:48.171 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:48.170+0000 7fb85ffaa740 1 -- 172.21.15.82:0/387509609 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fb8541a9e80 con 0x55fa240202f0 2024-05-20T16:19:48.171 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:48.170+0000 7fb85ffaa740 1 -- 172.21.15.82:0/387509609 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fb8541a7de0 con 0x55fa23fdd1c0 2024-05-20T16:19:48.174 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:48.173+0000 7fb85259c640 1 --2- 172.21.15.82:0/387509609 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fa240491b0 0x55fa243ed5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:19:48.174 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:48.174+0000 7fb85359e640 1 --2- 172.21.15.82:0/387509609 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fa23fdd1c0 0x55fa244061f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:19:48.178 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:48.178+0000 7fb852d9d640 1 -- 172.21.15.82:0/387509609 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fa23fdd1c0 msgr2=0x55fa244061f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:19:48.178 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:48.178+0000 7fb852d9d640 1 --2- 172.21.15.82:0/387509609 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fa23fdd1c0 0x55fa244061f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:19:48.178 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:48.178+0000 7fb852d9d640 1 -- 172.21.15.82:0/387509609 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fa240491b0 msgr2=0x55fa243ed5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:19:48.178 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:48.178+0000 7fb852d9d640 1 --2- 172.21.15.82:0/387509609 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fa240491b0 0x55fa243ed5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:19:48.178 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:48.178+0000 7fb852d9d640 1 -- 172.21.15.82:0/387509609 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fa243dfa60 con 0x55fa240202f0 2024-05-20T16:19:48.179 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:48.179+0000 7fb852d9d640 1 --2- 172.21.15.82:0/387509609 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fa240202f0 0x55fa243ed090 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fb844009fd0 tx=0x7fb8440d0da0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=679f94e11d51de93 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:19:48.181 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:48.180+0000 7fb838ff9640 1 -- 172.21.15.82:0/387509609 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8440bc070 con 0x55fa240202f0 2024-05-20T16:19:48.181 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:48.180+0000 7fb838ff9640 1 -- 172.21.15.82:0/387509609 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb84409fcc0 con 0x55fa240202f0 2024-05-20T16:19:48.181 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:48.181+0000 7fb838ff9640 1 -- 172.21.15.82:0/387509609 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb84400fc40 con 0x55fa240202f0 2024-05-20T16:19:48.182 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:48.182+0000 7fb85ffaa740 1 -- 172.21.15.82:0/387509609 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fa240202f0 msgr2=0x55fa243ed090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:19:48.182 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:48.182+0000 7fb85ffaa740 1 --2- 172.21.15.82:0/387509609 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fa240202f0 0x55fa243ed090 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fb844009fd0 tx=0x7fb8440d0da0 comp rx=0 tx=0).stop 2024-05-20T16:19:48.186 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:48.182+0000 7fb85ffaa740 1 -- 172.21.15.82:0/387509609 shutdown_connections 2024-05-20T16:19:48.186 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:48.182+0000 7fb85ffaa740 1 --2- 172.21.15.82:0/387509609 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fa240491b0 0x55fa243ed5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:19:48.186 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:48.182+0000 7fb85ffaa740 1 --2- 172.21.15.82:0/387509609 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fa23fdd1c0 0x55fa244061f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:19:48.186 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:48.182+0000 7fb85ffaa740 1 --2- 172.21.15.82:0/387509609 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fa240202f0 0x55fa243ed090 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:19:48.186 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:48.182+0000 7fb85ffaa740 1 -- 172.21.15.82:0/387509609 >> 172.21.15.82:0/387509609 conn(0x55fa2403d300 msgr2=0x55fa243ec370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:19:48.186 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:48.183+0000 7fb85ffaa740 1 -- 172.21.15.82:0/387509609 shutdown_connections 2024-05-20T16:19:48.186 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:48.183+0000 7fb85ffaa740 1 -- 172.21.15.82:0/387509609 wait complete. 2024-05-20T16:19:48.186 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:48.183+0000 7fb85ffaa740 1 -- start start 2024-05-20T16:19:48.186 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:48.183+0000 7fb85ffaa740 1 Processor -- start 2024-05-20T16:19:48.186 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:48.184+0000 7fb85ffaa740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fa23fdd1c0 0x55fa243f6500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:19:48.187 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:48.184+0000 7fb85ffaa740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fa240202f0 0x55fa243f6a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:19:48.187 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:48.184+0000 7fb85ffaa740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fa240491b0 0x55fa24466d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:19:48.187 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:48.184+0000 7fb85ffaa740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fb8541a8370 con 0x55fa240491b0 2024-05-20T16:19:48.187 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:48.184+0000 7fb85ffaa740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fb8541a97b0 con 0x55fa240202f0 2024-05-20T16:19:48.187 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:48.184+0000 7fb85ffaa740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fb8541a8a10 con 0x55fa23fdd1c0 2024-05-20T16:19:48.187 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:48.184+0000 7fb85ffaa740 10 client.0 _ll_register_callbacks cb 0x7fb848009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T16:19:48.187 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:48.184+0000 7fb852d9d640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fa23fdd1c0 0x55fa243f6500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:19:48.187 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:48.184+0000 7fb852d9d640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fa23fdd1c0 0x55fa243f6500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.82:42710/0 (socket says 172.21.15.82:42710) 2024-05-20T16:19:48.187 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:48.184+0000 7fb852d9d640 1 -- 172.21.15.82:0/920163274 learned_addr learned my addr 172.21.15.82:0/920163274 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:19:48.187 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:48.185+0000 7fb85259c640 1 --2- 172.21.15.82:0/920163274 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fa240202f0 0x55fa243f6a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:19:48.187 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:48.186+0000 7fb852d9d640 1 -- 172.21.15.82:0/920163274 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fa240202f0 msgr2=0x55fa243f6a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:19:48.188 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:48.186+0000 7fb852d9d640 1 --2- 172.21.15.82:0/920163274 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fa240202f0 0x55fa243f6a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:19:48.188 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:48.186+0000 7fb852d9d640 1 -- 172.21.15.82:0/920163274 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fa240491b0 msgr2=0x55fa24466d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:19:48.188 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:48.186+0000 7fb852d9d640 1 --2- 172.21.15.82:0/920163274 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fa240491b0 0x55fa24466d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:19:48.188 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:48.186+0000 7fb852d9d640 1 -- 172.21.15.82:0/920163274 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fa243f9990 con 0x55fa23fdd1c0 2024-05-20T16:19:48.188 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:48.186+0000 7fb852d9d640 1 --2- 172.21.15.82:0/920163274 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fa23fdd1c0 0x55fa243f6500 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fb84409c690 tx=0x7fb844010950 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4f7b81e42d2c9008 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:19:48.192 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:48.192+0000 7fb836ff5640 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-05-20T16:19:48.192 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:48.192+0000 7fb836ff5640 1 -- 172.21.15.82:0/920163274 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8440bc050 con 0x55fa23fdd1c0 2024-05-20T16:19:48.192 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:48.192+0000 7fb836ff5640 1 -- 172.21.15.82:0/920163274 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9150+0+0 (secure 0 0 0) 0x7fb84409dfc0 con 0x55fa23fdd1c0 2024-05-20T16:19:48.192 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:48.192+0000 7fb85ffaa740 1 -- 172.21.15.82:0/920163274 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb84409fcc0 con 0x55fa23fdd1c0 2024-05-20T16:19:48.194 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:48.194+0000 7fb836ff5640 1 -- 172.21.15.82:0/920163274 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb84409c330 con 0x55fa23fdd1c0 2024-05-20T16:19:48.194 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:48.195+0000 7fb85ffaa740 10 client.25273 fetch_fsmap learned FSMap version 27 2024-05-20T16:19:48.195 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:48.195+0000 7fb85ffaa740 1 -- 172.21.15.82:0/920163274 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55fa244681e0 con 0x55fa23fdd1c0 2024-05-20T16:19:48.196 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:48.196+0000 7fb836ff5640 1 -- 172.21.15.82:0/920163274 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb84409fcc0 con 0x55fa23fdd1c0 2024-05-20T16:19:48.200 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:48.201+0000 7fb836ff5640 1 -- 172.21.15.82:0/920163274 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fb84402c070 con 0x55fa23fdd1c0 2024-05-20T16:19:48.201 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:48.201+0000 7fb85ffaa740 10 client.25273 fetch_fsmap finished waiting for FSMap version 27 2024-05-20T16:19:48.201 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:48.201+0000 7fb85ffaa740 10 client.25273 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T16:19:48.201 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:48.201+0000 7fb85ffaa740 20 client.25273 populate_metadata read hostname 'smithi082' 2024-05-20T16:19:48.201 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:48.201+0000 7fb85ffaa740 1 --2- 172.21.15.82:0/920163274 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x55fa24468380 0x55fa2446a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:19:48.201 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:48.201+0000 7fb85ffaa740 4 client.25273 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-20T16:19:48.201 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:48.201+0000 7fb85ffaa740 1 -- 172.21.15.82:0/920163274 --> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55fa24354dc0 con 0x55fa24468380 2024-05-20T16:19:48.203 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:48.203+0000 7fb85259c640 1 --2- 172.21.15.82:0/920163274 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x55fa24468380 0x55fa2446a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:19:48.208 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:48.208+0000 7fb85259c640 1 --2- 172.21.15.82:0/920163274 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x55fa24468380 0x55fa2446a7a0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f11d5bb7b870bcf server_cookie=6245af7310ee3d4a in_seq=0 out_seq=0 2024-05-20T16:19:48.208 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:48.208+0000 7fb836ff5640 10 client.25273 ms_handle_connect on v2:172.21.15.160:6838/2765554482 2024-05-20T16:19:49.062 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:19:48 smithi160 ceph-mon[36347]: pgmap v537: 129 pgs: 129 active+clean; 10 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 680 MiB/s wr, 1.61k op/s 2024-05-20T16:19:49.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:19:48 smithi082 ceph-mon[32238]: pgmap v537: 129 pgs: 129 active+clean; 10 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 680 MiB/s wr, 1.61k op/s 2024-05-20T16:19:49.135 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:19:48 smithi184 ceph-mon[36209]: pgmap v537: 129 pgs: 129 active+clean; 10 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 680 MiB/s wr, 1.61k op/s 2024-05-20T16:19:50.063 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:19:49 smithi160 ceph-mon[36347]: pgmap v538: 129 pgs: 129 active+clean; 10 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 812 MiB/s wr, 1.87k op/s 2024-05-20T16:19:50.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:19:49 smithi082 ceph-mon[32238]: pgmap v538: 129 pgs: 129 active+clean; 10 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 812 MiB/s wr, 1.87k op/s 2024-05-20T16:19:50.135 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:19:49 smithi184 ceph-mon[36209]: pgmap v538: 129 pgs: 129 active+clean; 10 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 812 MiB/s wr, 1.87k op/s 2024-05-20T16:19:50.500 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:50.500+0000 7fb836ff5640 1 -- 172.21.15.82:0/920163274 <== mds.0 v2:172.21.15.160:6838/2765554482 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55fa24354dc0 con 0x55fa24468380 2024-05-20T16:19:50.500 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:50.500+0000 7fb836ff5640 10 client.25273 handle_command_reply: tid=0 2024-05-20T16:19:50.500 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:50.500+0000 7fb85ffaa740 1 client.25273 shutdown 2024-05-20T16:19:50.501 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:50.500+0000 7fb85ffaa740 1 -- 172.21.15.82:0/920163274 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fa23fdd1c0 msgr2=0x55fa243f6500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:19:50.501 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:50.500+0000 7fb85ffaa740 1 --2- 172.21.15.82:0/920163274 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fa23fdd1c0 0x55fa243f6500 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fb84409c690 tx=0x7fb844010950 comp rx=0 tx=0).stop 2024-05-20T16:19:50.501 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:50.501+0000 7fb85ffaa740 1 -- 172.21.15.82:0/920163274 shutdown_connections 2024-05-20T16:19:50.501 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:50.501+0000 7fb85ffaa740 1 --2- 172.21.15.82:0/920163274 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x55fa24468380 0x55fa2446a7a0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:19:50.501 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:50.501+0000 7fb85ffaa740 1 --2- 172.21.15.82:0/920163274 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fa240491b0 0x55fa24466d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:19:50.501 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:50.501+0000 7fb85ffaa740 1 --2- 172.21.15.82:0/920163274 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fa240202f0 0x55fa243f6a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:19:50.501 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:50.501+0000 7fb85ffaa740 1 --2- 172.21.15.82:0/920163274 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fa23fdd1c0 0x55fa243f6500 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:19:50.501 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:50.501+0000 7fb85ffaa740 1 -- 172.21.15.82:0/920163274 >> 172.21.15.82:0/920163274 conn(0x55fa2403d300 msgr2=0x55fa243f1420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:19:50.502 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:50.501+0000 7fb836ff5640 0 client.25273 ms_handle_reset on v2:172.21.15.160:6838/2765554482 2024-05-20T16:19:50.502 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:50.501+0000 7fb85ffaa740 1 -- 172.21.15.82:0/920163274 shutdown_connections 2024-05-20T16:19:50.502 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:50.501+0000 7fb85ffaa740 1 -- 172.21.15.82:0/920163274 wait complete. 2024-05-20T16:19:50.502 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:50.501+0000 7fb85ffaa740 20 client.25273 trim_cache size 0 max 16384 2024-05-20T16:19:50.502 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:{ 2024-05-20T16:19:50.502 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "message": "", 2024-05-20T16:19:50.502 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "return_code": 0 2024-05-20T16:19:50.502 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:} 2024-05-20T16:19:50.503 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:50.503+0000 7fb858901640 1 -- 172.21.15.82:0/2270905894 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fb83c0a2d10 msgr2=0x7fb83c0a5220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:19:50.503 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:50.503+0000 7fb858901640 1 --2- 172.21.15.82:0/2270905894 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fb83c0a2d10 0x7fb83c0a5220 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fb84c00adb0 tx=0x7fb84c009650 comp rx=0 tx=0).stop 2024-05-20T16:19:50.503 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:50.503+0000 7fb858901640 1 -- 172.21.15.82:0/2270905894 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb854100820 msgr2=0x7fb854070a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:19:50.503 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:50.503+0000 7fb858901640 1 --2- 172.21.15.82:0/2270905894 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb854100820 0x7fb854070a00 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fb84400b450 tx=0x7fb84400b910 comp rx=0 tx=0).stop 2024-05-20T16:19:50.505 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:50.505+0000 7fb858901640 1 -- 172.21.15.82:0/2270905894 shutdown_connections 2024-05-20T16:19:50.505 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:50.506+0000 7fb858901640 1 --2- 172.21.15.82:0/2270905894 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fb83c0a2d10 0x7fb83c0a5220 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:19:50.505 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:50.506+0000 7fb858901640 1 --2- 172.21.15.82:0/2270905894 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb854071ea0 0x7fb8540722b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:19:50.505 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:50.506+0000 7fb858901640 1 --2- 172.21.15.82:0/2270905894 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb854106640 0x7fb854070fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:19:50.506 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:50.506+0000 7fb858901640 1 --2- 172.21.15.82:0/2270905894 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb854100820 0x7fb854070a00 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:19:50.506 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:50.506+0000 7fb858901640 1 -- 172.21.15.82:0/2270905894 >> 172.21.15.82:0/2270905894 conn(0x7fb8540fbd20 msgr2=0x7fb854102620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:19:50.508 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:50.508+0000 7fb858901640 1 -- 172.21.15.82:0/2270905894 shutdown_connections 2024-05-20T16:19:50.508 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:19:50.508+0000 7fb858901640 1 -- 172.21.15.82:0/2270905894 wait complete. 2024-05-20T16:19:51.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:19:51 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:19:51.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:19:51 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T16:19:51.813 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:19:51 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:19:51.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:19:51 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:19:51.860 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:19:51 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T16:19:51.860 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:19:51 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:19:51.886 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:19:51 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:19:51.886 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:19:51 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T16:19:51.886 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:19:51 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:19:52.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:19:52 smithi160 ceph-mon[36347]: pgmap v539: 129 pgs: 129 active+clean; 10 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 669 MiB/s wr, 1.47k op/s 2024-05-20T16:19:52.863 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:19:52 smithi082 ceph-mon[32238]: pgmap v539: 129 pgs: 129 active+clean; 10 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 669 MiB/s wr, 1.47k op/s 2024-05-20T16:19:52.885 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:19:52 smithi184 ceph-mon[36209]: pgmap v539: 129 pgs: 129 active+clean; 10 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 669 MiB/s wr, 1.47k op/s 2024-05-20T16:19:54.885 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:19:54 smithi184 ceph-mon[36209]: pgmap v540: 129 pgs: 129 active+clean; 10 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 762 MiB/s wr, 1.71k op/s 2024-05-20T16:19:55.062 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:19:54 smithi160 ceph-mon[36347]: pgmap v540: 129 pgs: 129 active+clean; 10 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 762 MiB/s wr, 1.71k op/s 2024-05-20T16:19:55.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:19:54 smithi082 ceph-mon[32238]: pgmap v540: 129 pgs: 129 active+clean; 10 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 762 MiB/s wr, 1.71k op/s 2024-05-20T16:19:56.885 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:19:56 smithi184 ceph-mon[36209]: pgmap v541: 129 pgs: 129 active+clean; 10 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 796 MiB/s wr, 1.74k op/s 2024-05-20T16:19:57.062 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:19:56 smithi160 ceph-mon[36347]: pgmap v541: 129 pgs: 129 active+clean; 10 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 796 MiB/s wr, 1.74k op/s 2024-05-20T16:19:57.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:19:56 smithi082 ceph-mon[32238]: pgmap v541: 129 pgs: 129 active+clean; 10 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 796 MiB/s wr, 1.74k op/s 2024-05-20T16:19:59.062 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:19:58 smithi160 ceph-mon[36347]: pgmap v542: 129 pgs: 129 active+clean; 10 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 652 MiB/s wr, 1.39k op/s 2024-05-20T16:19:59.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:19:58 smithi082 ceph-mon[32238]: pgmap v542: 129 pgs: 129 active+clean; 10 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 652 MiB/s wr, 1.39k op/s 2024-05-20T16:19:59.135 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:19:58 smithi184 ceph-mon[36209]: pgmap v542: 129 pgs: 129 active+clean; 10 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 652 MiB/s wr, 1.39k op/s 2024-05-20T16:19:59.963 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:19:59 smithi160 ceph-mon[36347]: pgmap v543: 129 pgs: 129 active+clean; 10 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 811 MiB/s wr, 1.71k op/s 2024-05-20T16:20:00.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:19:59 smithi082 ceph-mon[32238]: pgmap v543: 129 pgs: 129 active+clean; 10 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 811 MiB/s wr, 1.71k op/s 2024-05-20T16:20:00.135 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:19:59 smithi184 ceph-mon[36209]: pgmap v543: 129 pgs: 129 active+clean; 10 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 811 MiB/s wr, 1.71k op/s 2024-05-20T16:20:01.062 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:20:00 smithi160 ceph-mon[36347]: overall HEALTH_OK 2024-05-20T16:20:01.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:20:00 smithi082 ceph-mon[32238]: overall HEALTH_OK 2024-05-20T16:20:01.135 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:20:00 smithi184 ceph-mon[36209]: overall HEALTH_OK 2024-05-20T16:20:02.062 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:20:01 smithi160 ceph-mon[36347]: pgmap v544: 129 pgs: 129 active+clean; 10 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 708 MiB/s wr, 1.46k op/s 2024-05-20T16:20:02.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:20:01 smithi082 ceph-mon[32238]: pgmap v544: 129 pgs: 129 active+clean; 10 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 708 MiB/s wr, 1.46k op/s 2024-05-20T16:20:02.135 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:20:01 smithi184 ceph-mon[36209]: pgmap v544: 129 pgs: 129 active+clean; 10 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 708 MiB/s wr, 1.46k op/s 2024-05-20T16:20:03.736 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:03.734+0000 7f2d550eb640 1 -- 172.21.15.82:0/3037767388 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2d5010a300 msgr2=0x7f2d5010a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:20:03.736 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:03.734+0000 7f2d550eb640 1 --2- 172.21.15.82:0/3037767388 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2d5010a300 0x7f2d5010a710 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f2d44009900 tx=0x7f2d4402a440 comp rx=0 tx=0).stop 2024-05-20T16:20:03.742 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:03.742+0000 7f2d550eb640 1 -- 172.21.15.82:0/3037767388 shutdown_connections 2024-05-20T16:20:03.742 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:03.742+0000 7f2d550eb640 1 --2- 172.21.15.82:0/3037767388 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2d5010a300 0x7f2d5010a710 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:20:03.742 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:03.742+0000 7f2d550eb640 1 --2- 172.21.15.82:0/3037767388 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2d5010f020 0x7f2d5010f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:20:03.742 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:03.742+0000 7f2d550eb640 1 --2- 172.21.15.82:0/3037767388 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2d501093d0 0x7f2d50109dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:20:03.742 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:03.742+0000 7f2d550eb640 1 -- 172.21.15.82:0/3037767388 >> 172.21.15.82:0/3037767388 conn(0x7f2d5006f4f0 msgr2=0x7f2d50071980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:20:03.746 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:03.746+0000 7f2d550eb640 1 -- 172.21.15.82:0/3037767388 shutdown_connections 2024-05-20T16:20:03.747 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:03.747+0000 7f2d550eb640 1 -- 172.21.15.82:0/3037767388 wait complete. 2024-05-20T16:20:03.747 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:03.748+0000 7f2d550eb640 1 Processor -- start 2024-05-20T16:20:03.749 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:03.749+0000 7f2d550eb640 1 -- start start 2024-05-20T16:20:03.749 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:03.749+0000 7f2d550eb640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2d501093d0 0x7f2d501a82f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:20:03.750 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:03.749+0000 7f2d550eb640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2d5010a300 0x7f2d501a8830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:20:03.750 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:03.749+0000 7f2d550eb640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2d5010f020 0x7f2d501af7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:20:03.750 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:03.750+0000 7f2d550eb640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f2d5011d720 con 0x7f2d5010a300 2024-05-20T16:20:03.750 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:03.750+0000 7f2d550eb640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f2d5011d5a0 con 0x7f2d5010f020 2024-05-20T16:20:03.750 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:03.750+0000 7f2d550eb640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f2d5011d8a0 con 0x7f2d501093d0 2024-05-20T16:20:03.750 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:03.750+0000 7f2d4e59c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2d5010a300 0x7f2d501a8830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:20:03.750 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:03.750+0000 7f2d4e59c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2d5010a300 0x7f2d501a8830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:50560/0 (socket says 172.21.15.82:50560) 2024-05-20T16:20:03.750 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:03.750+0000 7f2d4e59c640 1 -- 172.21.15.82:0/1447548423 learned_addr learned my addr 172.21.15.82:0/1447548423 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:20:03.751 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:03.751+0000 7f2d4e59c640 1 -- 172.21.15.82:0/1447548423 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2d501093d0 msgr2=0x7f2d501a82f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:20:03.751 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:03.751+0000 7f2d4e59c640 1 --2- 172.21.15.82:0/1447548423 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2d501093d0 0x7f2d501a82f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:20:03.751 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:03.751+0000 7f2d4e59c640 1 -- 172.21.15.82:0/1447548423 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2d5010f020 msgr2=0x7f2d501af7c0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-20T16:20:03.751 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:03.751+0000 7f2d4e59c640 1 --2- 172.21.15.82:0/1447548423 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2d5010f020 0x7f2d501af7c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:20:03.751 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:03.751+0000 7f2d4e59c640 1 -- 172.21.15.82:0/1447548423 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d501afe30 con 0x7f2d5010a300 2024-05-20T16:20:03.751 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:03.752+0000 7f2d4e59c640 1 --2- 172.21.15.82:0/1447548423 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2d5010a300 0x7f2d501a8830 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f2d44009a20 tx=0x7f2d44032da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:20:03.754 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:03.754+0000 7f2d37fff640 1 -- 172.21.15.82:0/1447548423 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d4403f030 con 0x7f2d5010a300 2024-05-20T16:20:03.754 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:03.754+0000 7f2d37fff640 1 -- 172.21.15.82:0/1447548423 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d44003ce0 con 0x7f2d5010a300 2024-05-20T16:20:03.754 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:03.754+0000 7f2d37fff640 1 -- 172.21.15.82:0/1447548423 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d44035070 con 0x7f2d5010a300 2024-05-20T16:20:03.754 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:03.754+0000 7f2d550eb640 1 -- 172.21.15.82:0/1447548423 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d501b00c0 con 0x7f2d5010a300 2024-05-20T16:20:03.754 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:03.755+0000 7f2d550eb640 1 -- 172.21.15.82:0/1447548423 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d501b05e0 con 0x7f2d5010a300 2024-05-20T16:20:03.759 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:03.759+0000 7f2d5bf83740 1 Processor -- start 2024-05-20T16:20:03.764 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:03.764+0000 7f2d5bf83740 1 -- start start 2024-05-20T16:20:03.764 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:03.764+0000 7f2d5bf83740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5615f65e31b0 0x5615f6982e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:20:03.764 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:03.764+0000 7f2d5bf83740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5615f65ba2f0 0x5615f69833e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:20:03.764 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:03.764+0000 7f2d5bf83740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5615f65c24d0 0x5615f698a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:20:03.764 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:03.764+0000 7f2d5bf83740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x5615f68e7340 con 0x5615f65e31b0 2024-05-20T16:20:03.764 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:03.764+0000 7f2d5bf83740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x5615f688f290 con 0x5615f65c24d0 2024-05-20T16:20:03.764 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:03.764+0000 7f2d5bf83740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5615f68e3300 con 0x5615f65ba2f0 2024-05-20T16:20:03.764 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:03.764+0000 7f2d37fff640 1 -- 172.21.15.82:0/1447548423 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f2d44042030 con 0x7f2d5010a300 2024-05-20T16:20:03.765 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:03.764+0000 7f2d4ed9d640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5615f65ba2f0 0x5615f69833e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:20:03.765 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:03.764+0000 7f2d4ed9d640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5615f65ba2f0 0x5615f69833e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.82:59716/0 (socket says 172.21.15.82:59716) 2024-05-20T16:20:03.765 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:03.764+0000 7f2d4ed9d640 1 -- 172.21.15.82:0/2376777046 learned_addr learned my addr 172.21.15.82:0/2376777046 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:20:03.769 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:03.765+0000 7f2d4f59e640 1 --2- 172.21.15.82:0/2376777046 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5615f65e31b0 0x5615f6982e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:20:03.769 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:03.765+0000 7f2d4ed9d640 1 -- 172.21.15.82:0/2376777046 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5615f65c24d0 msgr2=0x5615f698a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:20:03.769 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:03.768+0000 7f2d4e59c640 1 --2- 172.21.15.82:0/2376777046 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5615f65c24d0 0x5615f698a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:20:03.769 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:03.768+0000 7f2d4ed9d640 1 --2- 172.21.15.82:0/2376777046 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5615f65c24d0 0x5615f698a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:20:03.769 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:03.768+0000 7f2d4ed9d640 1 -- 172.21.15.82:0/2376777046 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5615f65e31b0 msgr2=0x5615f6982e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:20:03.769 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:03.768+0000 7f2d4ed9d640 1 --2- 172.21.15.82:0/2376777046 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5615f65e31b0 0x5615f6982e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:20:03.769 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:03.768+0000 7f2d4ed9d640 1 -- 172.21.15.82:0/2376777046 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5615f68b38e0 con 0x5615f65ba2f0 2024-05-20T16:20:03.769 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:03.769+0000 7f2d4f59e640 1 --2- 172.21.15.82:0/2376777046 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5615f65e31b0 0x5615f6982e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T16:20:03.769 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:03.769+0000 7f2d4e59c640 1 --2- 172.21.15.82:0/2376777046 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5615f65c24d0 0x5615f698a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T16:20:03.769 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:03.769+0000 7f2d4ed9d640 1 --2- 172.21.15.82:0/2376777046 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5615f65ba2f0 0x5615f69833e0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f2d48009d50 tx=0x7f2d4800b420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d6e38ef20ff094dd server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:20:03.770 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:03.770+0000 7f2d37fff640 1 --2- 172.21.15.82:0/1447548423 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f2d380a2d90 0x7f2d380a52a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:20:03.771 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:03.770+0000 7f2d37fff640 1 -- 172.21.15.82:0/1447548423 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9150+0+0 (secure 0 0 0) 0x7f2d440efc60 con 0x7f2d5010a300 2024-05-20T16:20:03.773 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:03.773+0000 7f2d4f59e640 1 --2- 172.21.15.82:0/1447548423 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f2d380a2d90 0x7f2d380a52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:20:03.774 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:03.774+0000 7f2d4f59e640 1 --2- 172.21.15.82:0/1447548423 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f2d380a2d90 0x7f2d380a52a0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f2d40006fd0 tx=0x7f2d40008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:20:03.776 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:03.776+0000 7f2d357fa640 1 -- 172.21.15.82:0/2376777046 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d4800bb30 con 0x5615f65ba2f0 2024-05-20T16:20:03.776 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:03.776+0000 7f2d357fa640 1 -- 172.21.15.82:0/2376777046 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d48017070 con 0x5615f65ba2f0 2024-05-20T16:20:03.778 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:03.778+0000 7f2d5bf83740 1 -- 172.21.15.82:0/2376777046 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5615f65ba2f0 msgr2=0x5615f69833e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:20:03.778 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:03.778+0000 7f2d5bf83740 1 --2- 172.21.15.82:0/2376777046 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5615f65ba2f0 0x5615f69833e0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f2d48009d50 tx=0x7f2d4800b420 comp rx=0 tx=0).stop 2024-05-20T16:20:03.778 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:03.778+0000 7f2d5bf83740 1 -- 172.21.15.82:0/2376777046 shutdown_connections 2024-05-20T16:20:03.778 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:03.778+0000 7f2d5bf83740 1 --2- 172.21.15.82:0/2376777046 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5615f65c24d0 0x5615f698a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:20:03.778 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:03.778+0000 7f2d5bf83740 1 --2- 172.21.15.82:0/2376777046 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5615f65ba2f0 0x5615f69833e0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:20:03.778 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:03.778+0000 7f2d5bf83740 1 --2- 172.21.15.82:0/2376777046 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5615f65e31b0 0x5615f6982e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:20:03.778 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:03.778+0000 7f2d5bf83740 1 -- 172.21.15.82:0/2376777046 >> 172.21.15.82:0/2376777046 conn(0x5615f65d7300 msgr2=0x5615f697e7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:20:03.778 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:03.778+0000 7f2d5bf83740 1 -- 172.21.15.82:0/2376777046 shutdown_connections 2024-05-20T16:20:03.778 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:03.778+0000 7f2d5bf83740 1 -- 172.21.15.82:0/2376777046 wait complete. 2024-05-20T16:20:03.779 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:03.779+0000 7f2d5bf83740 1 -- start start 2024-05-20T16:20:03.779 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:03.779+0000 7f2d5bf83740 1 Processor -- start 2024-05-20T16:20:03.779 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:03.779+0000 7f2d5bf83740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5615f65ba2f0 0x5615f699de30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:20:03.779 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:03.779+0000 7f2d5bf83740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5615f65c24d0 0x5615f699e370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:20:03.779 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:03.779+0000 7f2d5bf83740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5615f65e31b0 0x5615f69a52d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:20:03.779 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:03.779+0000 7f2d5bf83740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x5615f699bd90 con 0x5615f65c24d0 2024-05-20T16:20:03.779 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:03.779+0000 7f2d5bf83740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x5615f68a2790 con 0x5615f65ba2f0 2024-05-20T16:20:03.780 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:03.779+0000 7f2d5bf83740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5615f699bf10 con 0x5615f65e31b0 2024-05-20T16:20:03.780 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:03.779+0000 7f2d5bf83740 10 client.0 _ll_register_callbacks cb 0x5615f68dda20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T16:20:03.780 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:03.780+0000 7f2d4f59e640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5615f65e31b0 0x5615f69a52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:20:03.780 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:03.780+0000 7f2d4f59e640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5615f65e31b0 0x5615f69a52d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.82:59730/0 (socket says 172.21.15.82:59730) 2024-05-20T16:20:03.780 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:03.780+0000 7f2d4f59e640 1 -- 172.21.15.82:0/2255104364 learned_addr learned my addr 172.21.15.82:0/2255104364 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:20:03.781 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:03.780+0000 7f2d4ed9d640 1 --2- 172.21.15.82:0/2255104364 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5615f65ba2f0 0x5615f699de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:20:03.781 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:03.781+0000 7f2d4ed9d640 1 -- 172.21.15.82:0/2255104364 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5615f65e31b0 msgr2=0x5615f69a52d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:20:03.781 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:03.781+0000 7f2d4ed9d640 1 --2- 172.21.15.82:0/2255104364 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5615f65e31b0 0x5615f69a52d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:20:03.781 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:03.781+0000 7f2d4ed9d640 1 -- 172.21.15.82:0/2255104364 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5615f65c24d0 msgr2=0x5615f699e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:20:03.781 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:03.781+0000 7f2d4ed9d640 1 --2- 172.21.15.82:0/2255104364 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5615f65c24d0 0x5615f699e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:20:03.781 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:03.781+0000 7f2d4ed9d640 1 -- 172.21.15.82:0/2255104364 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5615f65fb210 con 0x5615f65ba2f0 2024-05-20T16:20:03.782 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:03.781+0000 7f2d4ed9d640 1 --2- 172.21.15.82:0/2255104364 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5615f65ba2f0 0x5615f699de30 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x5615f6984240 tx=0x7f2d48011a80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e4a2effdf7ec1615 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:20:03.782 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:03.781+0000 7f2d32ff5640 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-20T16:20:03.782 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:03.781+0000 7f2d32ff5640 1 -- 172.21.15.82:0/2255104364 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d48022020 con 0x5615f65ba2f0 2024-05-20T16:20:03.782 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:03.782+0000 7f2d32ff5640 1 -- 172.21.15.82:0/2255104364 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9150+0+0 (secure 0 0 0) 0x7f2d48012430 con 0x5615f65ba2f0 2024-05-20T16:20:03.782 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:03.782+0000 7f2d5bf83740 1 -- 172.21.15.82:0/2255104364 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5615f68f4a70 con 0x5615f65ba2f0 2024-05-20T16:20:03.782 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:03.782+0000 7f2d32ff5640 1 -- 172.21.15.82:0/2255104364 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f2d480099c0 con 0x5615f65ba2f0 2024-05-20T16:20:03.782 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:03.782+0000 7f2d5bf83740 10 client.25259 fetch_fsmap learned FSMap version 27 2024-05-20T16:20:03.782 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:03.782+0000 7f2d5bf83740 1 -- 172.21.15.82:0/2255104364 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5615f66c7620 con 0x5615f65ba2f0 2024-05-20T16:20:03.785 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:03.785+0000 7f2d32ff5640 1 -- 172.21.15.82:0/2255104364 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5615f68f4a70 con 0x5615f65ba2f0 2024-05-20T16:20:03.785 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:03.785+0000 7f2d32ff5640 1 -- 172.21.15.82:0/2255104364 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f2d48011070 con 0x5615f65ba2f0 2024-05-20T16:20:03.785 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:03.785+0000 7f2d5bf83740 10 client.25259 fetch_fsmap finished waiting for FSMap version 27 2024-05-20T16:20:03.785 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:03.785+0000 7f2d5bf83740 10 client.25259 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T16:20:03.785 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:03.785+0000 7f2d5bf83740 20 client.25259 populate_metadata read hostname 'smithi082' 2024-05-20T16:20:03.785 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:03.786+0000 7f2d5bf83740 1 --2- 172.21.15.82:0/2255104364 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x5615f65a1d80 0x5615f69ab4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:20:03.785 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:03.786+0000 7f2d5bf83740 4 client.25259 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T16:20:03.786 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:03.786+0000 7f2d5bf83740 1 -- 172.21.15.82:0/2255104364 --> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5615f68eedc0 con 0x5615f65a1d80 2024-05-20T16:20:03.786 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:03.786+0000 7f2d4e59c640 1 --2- 172.21.15.82:0/2255104364 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x5615f65a1d80 0x5615f69ab4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:20:03.787 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:03.788+0000 7f2d4e59c640 1 --2- 172.21.15.82:0/2255104364 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x5615f65a1d80 0x5615f69ab4d0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a06d54f6d2176480 server_cookie=a634748262caeb40 in_seq=0 out_seq=0 2024-05-20T16:20:03.788 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:03.788+0000 7f2d32ff5640 10 client.25259 ms_handle_connect on v2:172.21.15.160:6838/2765554482 2024-05-20T16:20:03.792 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:03.792+0000 7f2d32ff5640 1 -- 172.21.15.82:0/2255104364 <== mds.0 v2:172.21.15.160:6838/2765554482 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5615f68eedc0 con 0x5615f65a1d80 2024-05-20T16:20:03.793 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:03.792+0000 7f2d32ff5640 10 client.25259 handle_command_reply: tid=0 2024-05-20T16:20:03.793 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:03.792+0000 7f2d5bf83740 1 client.25259 shutdown 2024-05-20T16:20:03.793 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:03.792+0000 7f2d5bf83740 1 -- 172.21.15.82:0/2255104364 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5615f65ba2f0 msgr2=0x5615f699de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:20:03.793 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:03.792+0000 7f2d5bf83740 1 --2- 172.21.15.82:0/2255104364 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5615f65ba2f0 0x5615f699de30 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x5615f6984240 tx=0x7f2d48011a80 comp rx=0 tx=0).stop 2024-05-20T16:20:03.793 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:03.792+0000 7f2d5bf83740 1 -- 172.21.15.82:0/2255104364 shutdown_connections 2024-05-20T16:20:03.793 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:03.792+0000 7f2d5bf83740 1 --2- 172.21.15.82:0/2255104364 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x5615f65a1d80 0x5615f69ab4d0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:20:03.793 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:03.793+0000 7f2d5bf83740 1 --2- 172.21.15.82:0/2255104364 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5615f65c24d0 0x5615f699e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:20:03.793 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:03.793+0000 7f2d5bf83740 1 --2- 172.21.15.82:0/2255104364 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5615f65e31b0 0x5615f69a52d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:20:03.793 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:03.793+0000 7f2d5bf83740 1 --2- 172.21.15.82:0/2255104364 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5615f65ba2f0 0x5615f699de30 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:20:03.793 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:03.793+0000 7f2d5bf83740 1 -- 172.21.15.82:0/2255104364 >> 172.21.15.82:0/2255104364 conn(0x5615f65d7300 msgr2=0x5615f6981810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:20:03.793 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:03.793+0000 7f2d32ff5640 0 client.25259 ms_handle_reset on v2:172.21.15.160:6838/2765554482 2024-05-20T16:20:03.793 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:03.793+0000 7f2d5bf83740 1 -- 172.21.15.82:0/2255104364 shutdown_connections 2024-05-20T16:20:03.794 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:03.793+0000 7f2d5bf83740 1 -- 172.21.15.82:0/2255104364 wait complete. 2024-05-20T16:20:03.794 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:03.793+0000 7f2d5bf83740 20 client.25259 trim_cache size 0 max 16384 2024-05-20T16:20:03.826 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:03.827+0000 7f2d5bf83740 1 Processor -- start 2024-05-20T16:20:03.828 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:03.827+0000 7f2d5bf83740 1 -- start start 2024-05-20T16:20:03.828 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:03.827+0000 7f2d5bf83740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5615f65ba2f0 0x5615f699e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:20:03.828 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:03.827+0000 7f2d5bf83740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5615f65c24d0 0x5615f69816d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:20:03.829 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:03.827+0000 7f2d5bf83740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5615f65e31b0 0x5615f6981c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:20:03.829 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:03.827+0000 7f2d5bf83740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f2d3c0125a0 con 0x5615f65ba2f0 2024-05-20T16:20:03.829 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:03.827+0000 7f2d5bf83740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f2d3c00fe70 con 0x5615f65e31b0 2024-05-20T16:20:03.829 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:03.827+0000 7f2d5bf83740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f2d3c00dcc0 con 0x5615f65c24d0 2024-05-20T16:20:03.829 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:03.828+0000 7f2d4ed9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5615f65ba2f0 0x5615f699e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:20:03.829 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:03.828+0000 7f2d4ed9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5615f65ba2f0 0x5615f699e0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:50592/0 (socket says 172.21.15.82:50592) 2024-05-20T16:20:03.829 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:03.828+0000 7f2d4ed9d640 1 -- 172.21.15.82:0/1906543404 learned_addr learned my addr 172.21.15.82:0/1906543404 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:20:03.829 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:03.828+0000 7f2d4f59e640 1 --2- 172.21.15.82:0/1906543404 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5615f65e31b0 0x5615f6981c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:20:03.829 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:03.829+0000 7f2d4f59e640 1 -- 172.21.15.82:0/1906543404 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5615f65c24d0 msgr2=0x5615f69816d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-20T16:20:03.829 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:03.829+0000 7f2d4f59e640 1 --2- 172.21.15.82:0/1906543404 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5615f65c24d0 0x5615f69816d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:20:03.829 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:03.829+0000 7f2d4f59e640 1 -- 172.21.15.82:0/1906543404 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5615f65ba2f0 msgr2=0x5615f699e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:20:03.829 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:03.829+0000 7f2d4f59e640 1 --2- 172.21.15.82:0/1906543404 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5615f65ba2f0 0x5615f699e0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:20:03.829 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:03.829+0000 7f2d4f59e640 1 -- 172.21.15.82:0/1906543404 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5615f6982a10 con 0x5615f65e31b0 2024-05-20T16:20:03.830 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:03.829+0000 7f2d4f59e640 1 --2- 172.21.15.82:0/1906543404 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5615f65e31b0 0x5615f6981c10 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f2d4000d8e0 tx=0x7f2d40010000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7359c0b4c59011e3 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:20:03.831 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:03.831+0000 7f2d34ff9640 1 -- 172.21.15.82:0/1906543404 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d40024070 con 0x5615f65e31b0 2024-05-20T16:20:03.831 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:03.831+0000 7f2d34ff9640 1 -- 172.21.15.82:0/1906543404 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d40020440 con 0x5615f65e31b0 2024-05-20T16:20:03.831 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:03.831+0000 7f2d34ff9640 1 -- 172.21.15.82:0/1906543404 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d4001a040 con 0x5615f65e31b0 2024-05-20T16:20:03.834 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:03.833+0000 7f2d5bf83740 1 -- 172.21.15.82:0/1906543404 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5615f65e31b0 msgr2=0x5615f6981c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:20:03.834 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:03.833+0000 7f2d5bf83740 1 --2- 172.21.15.82:0/1906543404 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5615f65e31b0 0x5615f6981c10 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f2d4000d8e0 tx=0x7f2d40010000 comp rx=0 tx=0).stop 2024-05-20T16:20:03.834 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:03.833+0000 7f2d5bf83740 1 -- 172.21.15.82:0/1906543404 shutdown_connections 2024-05-20T16:20:03.834 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:03.833+0000 7f2d5bf83740 1 --2- 172.21.15.82:0/1906543404 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5615f65e31b0 0x5615f6981c10 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:20:03.834 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:03.833+0000 7f2d5bf83740 1 --2- 172.21.15.82:0/1906543404 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5615f65c24d0 0x5615f69816d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:20:03.834 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:03.833+0000 7f2d5bf83740 1 --2- 172.21.15.82:0/1906543404 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5615f65ba2f0 0x5615f699e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:20:03.835 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:03.833+0000 7f2d5bf83740 1 -- 172.21.15.82:0/1906543404 >> 172.21.15.82:0/1906543404 conn(0x5615f65d7300 msgr2=0x5615f699c450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:20:03.835 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:03.834+0000 7f2d5bf83740 1 -- 172.21.15.82:0/1906543404 shutdown_connections 2024-05-20T16:20:03.835 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:03.834+0000 7f2d5bf83740 1 -- 172.21.15.82:0/1906543404 wait complete. 2024-05-20T16:20:03.836 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:03.835+0000 7f2d5bf83740 1 -- start start 2024-05-20T16:20:03.836 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:03.835+0000 7f2d5bf83740 1 Processor -- start 2024-05-20T16:20:03.836 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:03.836+0000 7f2d5bf83740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5615f65ba2f0 0x5615f69a30a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:20:03.836 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:03.836+0000 7f2d5bf83740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5615f65c24d0 0x5615f69a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:20:03.836 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:03.836+0000 7f2d5bf83740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5615f65e31b0 0x5615f69a3b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:20:03.836 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:03.836+0000 7f2d5bf83740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f2d3c00e330 con 0x5615f65e31b0 2024-05-20T16:20:03.836 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:03.836+0000 7f2d5bf83740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f2d3c00f7a0 con 0x5615f65ba2f0 2024-05-20T16:20:03.836 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:03.836+0000 7f2d5bf83740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f2d3c00ea00 con 0x5615f65c24d0 2024-05-20T16:20:03.836 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:03.836+0000 7f2d5bf83740 10 client.0 _ll_register_callbacks cb 0x7f2d48009c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T16:20:03.836 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:03.836+0000 7f2d4ed9d640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5615f65ba2f0 0x5615f69a30a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:20:03.837 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:03.836+0000 7f2d4ed9d640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5615f65ba2f0 0x5615f69a30a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.82:51284/0 (socket says 172.21.15.82:51284) 2024-05-20T16:20:03.837 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:03.836+0000 7f2d4ed9d640 1 -- 172.21.15.82:0/2207185591 learned_addr learned my addr 172.21.15.82:0/2207185591 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:20:03.837 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:03.836+0000 7f2d4f59e640 1 --2- 172.21.15.82:0/2207185591 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5615f65e31b0 0x5615f69a3b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:20:03.838 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:03.836+0000 7f2d4ed9d640 1 -- 172.21.15.82:0/2207185591 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5615f65c24d0 msgr2=0x5615f69a35e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:20:03.838 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:03.836+0000 7f2d4ed9d640 1 --2- 172.21.15.82:0/2207185591 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5615f65c24d0 0x5615f69a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:20:03.838 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:03.837+0000 7f2d4ed9d640 1 -- 172.21.15.82:0/2207185591 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5615f65e31b0 msgr2=0x5615f69a3b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:20:03.838 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:03.837+0000 7f2d4ed9d640 1 --2- 172.21.15.82:0/2207185591 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5615f65e31b0 0x5615f69a3b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:20:03.838 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:03.837+0000 7f2d4ed9d640 1 -- 172.21.15.82:0/2207185591 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5615f6a11830 con 0x5615f65ba2f0 2024-05-20T16:20:03.838 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:03.837+0000 7f2d4ed9d640 1 --2- 172.21.15.82:0/2207185591 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5615f65ba2f0 0x5615f69a30a0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f2d48009d50 tx=0x7f2d4800bf80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5c57d06a210a239b server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:20:03.838 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:03.837+0000 7f2d32ff5640 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-20T16:20:03.838 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:03.838+0000 7f2d32ff5640 1 -- 172.21.15.82:0/2207185591 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d4801f040 con 0x5615f65ba2f0 2024-05-20T16:20:03.838 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:03.838+0000 7f2d32ff5640 1 -- 172.21.15.82:0/2207185591 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9150+0+0 (secure 0 0 0) 0x7f2d4801b420 con 0x5615f65ba2f0 2024-05-20T16:20:03.838 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:03.838+0000 7f2d5bf83740 1 -- 172.21.15.82:0/2207185591 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2d40020440 con 0x5615f65ba2f0 2024-05-20T16:20:03.838 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:03.838+0000 7f2d32ff5640 1 -- 172.21.15.82:0/2207185591 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f2d48022040 con 0x5615f65ba2f0 2024-05-20T16:20:03.839 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:03.838+0000 7f2d5bf83740 10 client.25265 fetch_fsmap learned FSMap version 27 2024-05-20T16:20:03.839 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:03.838+0000 7f2d5bf83740 1 -- 172.21.15.82:0/2207185591 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5615f6a12960 con 0x5615f65ba2f0 2024-05-20T16:20:03.839 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:03.839+0000 7f2d32ff5640 1 -- 172.21.15.82:0/2207185591 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d40020440 con 0x5615f65ba2f0 2024-05-20T16:20:03.840 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:03.839+0000 7f2d32ff5640 1 -- 172.21.15.82:0/2207185591 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f2d48011040 con 0x5615f65ba2f0 2024-05-20T16:20:03.840 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:03.839+0000 7f2d5bf83740 10 client.25265 fetch_fsmap finished waiting for FSMap version 27 2024-05-20T16:20:03.840 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:03.839+0000 7f2d5bf83740 10 client.25265 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T16:20:03.840 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:03.839+0000 7f2d5bf83740 20 client.25265 populate_metadata read hostname 'smithi082' 2024-05-20T16:20:03.840 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:03.839+0000 7f2d5bf83740 1 --2- 172.21.15.82:0/2207185591 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x5615f65771c0 0x5615f6a152f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:20:03.840 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:03.839+0000 7f2d5bf83740 4 client.25265 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-20T16:20:03.840 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:03.839+0000 7f2d5bf83740 1 -- 172.21.15.82:0/2207185591 --> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5615f686c8e0 con 0x5615f65771c0 2024-05-20T16:20:03.840 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:03.839+0000 7f2d4e59c640 1 --2- 172.21.15.82:0/2207185591 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x5615f65771c0 0x5615f6a152f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:20:03.840 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:03.840+0000 7f2d4e59c640 1 --2- 172.21.15.82:0/2207185591 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x5615f65771c0 0x5615f6a152f0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5a0f0b02bafd1ad8 server_cookie=2421624f06f58032 in_seq=0 out_seq=0 2024-05-20T16:20:03.840 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:03.840+0000 7f2d32ff5640 10 client.25265 ms_handle_connect on v2:172.21.15.160:6838/2765554482 2024-05-20T16:20:05.062 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:20:04 smithi160 ceph-mon[36347]: pgmap v545: 129 pgs: 129 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 746 MiB/s wr, 1.56k op/s 2024-05-20T16:20:05.116 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:20:04 smithi082 ceph-mon[32238]: pgmap v545: 129 pgs: 129 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 746 MiB/s wr, 1.56k op/s 2024-05-20T16:20:05.136 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:20:04 smithi184 ceph-mon[36209]: pgmap v545: 129 pgs: 129 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 746 MiB/s wr, 1.56k op/s 2024-05-20T16:20:06.062 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:20:05 smithi160 ceph-mon[36347]: pgmap v546: 129 pgs: 129 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 809 MiB/s wr, 1.75k op/s 2024-05-20T16:20:06.110 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:20:05 smithi082 ceph-mon[32238]: pgmap v546: 129 pgs: 129 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 809 MiB/s wr, 1.75k op/s 2024-05-20T16:20:06.135 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:20:05 smithi184 ceph-mon[36209]: pgmap v546: 129 pgs: 129 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 809 MiB/s wr, 1.75k op/s 2024-05-20T16:20:07.280 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:07.279+0000 7f2d32ff5640 1 -- 172.21.15.82:0/2207185591 <== mds.0 v2:172.21.15.160:6838/2765554482 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5615f686c8e0 con 0x5615f65771c0 2024-05-20T16:20:07.280 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:07.279+0000 7f2d32ff5640 10 client.25265 handle_command_reply: tid=0 2024-05-20T16:20:07.280 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:07.279+0000 7f2d5bf83740 1 client.25265 shutdown 2024-05-20T16:20:07.280 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:07.280+0000 7f2d5bf83740 1 -- 172.21.15.82:0/2207185591 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5615f65ba2f0 msgr2=0x5615f69a30a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:20:07.280 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:07.280+0000 7f2d5bf83740 1 --2- 172.21.15.82:0/2207185591 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5615f65ba2f0 0x5615f69a30a0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f2d48009d50 tx=0x7f2d4800bf80 comp rx=0 tx=0).stop 2024-05-20T16:20:07.280 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:07.280+0000 7f2d5bf83740 1 -- 172.21.15.82:0/2207185591 shutdown_connections 2024-05-20T16:20:07.280 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:07.280+0000 7f2d5bf83740 1 --2- 172.21.15.82:0/2207185591 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x5615f65771c0 0x5615f6a152f0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:20:07.281 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:07.280+0000 7f2d5bf83740 1 --2- 172.21.15.82:0/2207185591 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5615f65e31b0 0x5615f69a3b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:20:07.281 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:07.280+0000 7f2d5bf83740 1 --2- 172.21.15.82:0/2207185591 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5615f65c24d0 0x5615f69a35e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:20:07.281 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:07.280+0000 7f2d5bf83740 1 --2- 172.21.15.82:0/2207185591 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5615f65ba2f0 0x5615f69a30a0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:20:07.281 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:07.280+0000 7f2d5bf83740 1 -- 172.21.15.82:0/2207185591 >> 172.21.15.82:0/2207185591 conn(0x5615f65d7300 msgr2=0x5615f69816d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:20:07.281 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:07.280+0000 7f2d32ff5640 0 client.25265 ms_handle_reset on v2:172.21.15.160:6838/2765554482 2024-05-20T16:20:07.281 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:07.280+0000 7f2d5bf83740 1 -- 172.21.15.82:0/2207185591 shutdown_connections 2024-05-20T16:20:07.281 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:07.280+0000 7f2d5bf83740 1 -- 172.21.15.82:0/2207185591 wait complete. 2024-05-20T16:20:07.281 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:07.280+0000 7f2d5bf83740 20 client.25265 trim_cache size 0 max 16384 2024-05-20T16:20:07.281 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:{ 2024-05-20T16:20:07.281 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "message": "", 2024-05-20T16:20:07.282 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "return_code": 0 2024-05-20T16:20:07.282 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:} 2024-05-20T16:20:07.282 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:07.281+0000 7f2d550eb640 1 -- 172.21.15.82:0/1447548423 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f2d380a2d90 msgr2=0x7f2d380a52a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:20:07.282 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:07.281+0000 7f2d550eb640 1 --2- 172.21.15.82:0/1447548423 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f2d380a2d90 0x7f2d380a52a0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f2d40006fd0 tx=0x7f2d40008040 comp rx=0 tx=0).stop 2024-05-20T16:20:07.282 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:07.281+0000 7f2d550eb640 1 -- 172.21.15.82:0/1447548423 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2d5010a300 msgr2=0x7f2d501a8830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:20:07.283 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:07.281+0000 7f2d550eb640 1 --2- 172.21.15.82:0/1447548423 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2d5010a300 0x7f2d501a8830 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f2d44009a20 tx=0x7f2d44032da0 comp rx=0 tx=0).stop 2024-05-20T16:20:07.283 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:07.281+0000 7f2d550eb640 1 -- 172.21.15.82:0/1447548423 shutdown_connections 2024-05-20T16:20:07.283 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:07.281+0000 7f2d550eb640 1 --2- 172.21.15.82:0/1447548423 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f2d380a2d90 0x7f2d380a52a0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:20:07.283 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:07.281+0000 7f2d550eb640 1 --2- 172.21.15.82:0/1447548423 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2d5010a300 0x7f2d501a8830 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:20:07.283 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:07.281+0000 7f2d550eb640 1 --2- 172.21.15.82:0/1447548423 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2d5010f020 0x7f2d501af7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:20:07.283 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:07.281+0000 7f2d550eb640 1 --2- 172.21.15.82:0/1447548423 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2d501093d0 0x7f2d501a82f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:20:07.283 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:07.281+0000 7f2d550eb640 1 -- 172.21.15.82:0/1447548423 >> 172.21.15.82:0/1447548423 conn(0x7f2d5006f4f0 msgr2=0x7f2d5010b000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:20:07.283 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:07.281+0000 7f2d550eb640 1 -- 172.21.15.82:0/1447548423 shutdown_connections 2024-05-20T16:20:07.283 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:07.282+0000 7f2d550eb640 1 -- 172.21.15.82:0/1447548423 wait complete. 2024-05-20T16:20:09.062 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:20:08 smithi160 ceph-mon[36347]: pgmap v547: 129 pgs: 129 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 641 MiB/s wr, 1.38k op/s 2024-05-20T16:20:09.110 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:20:08 smithi082 ceph-mon[32238]: pgmap v547: 129 pgs: 129 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 641 MiB/s wr, 1.38k op/s 2024-05-20T16:20:09.135 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:20:08 smithi184 ceph-mon[36209]: pgmap v547: 129 pgs: 129 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 641 MiB/s wr, 1.38k op/s 2024-05-20T16:20:11.062 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:20:10 smithi160 ceph-mon[36347]: pgmap v548: 129 pgs: 129 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 843 MiB/s wr, 1.71k op/s 2024-05-20T16:20:11.110 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:20:10 smithi082 ceph-mon[32238]: pgmap v548: 129 pgs: 129 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 843 MiB/s wr, 1.71k op/s 2024-05-20T16:20:11.135 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:20:10 smithi184 ceph-mon[36209]: pgmap v548: 129 pgs: 129 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 843 MiB/s wr, 1.71k op/s 2024-05-20T16:20:12.062 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:20:11 smithi160 ceph-mon[36347]: pgmap v549: 129 pgs: 129 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 711 MiB/s wr, 1.36k op/s 2024-05-20T16:20:12.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:20:11 smithi082 ceph-mon[32238]: pgmap v549: 129 pgs: 129 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 711 MiB/s wr, 1.36k op/s 2024-05-20T16:20:12.135 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:20:11 smithi184 ceph-mon[36209]: pgmap v549: 129 pgs: 129 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 711 MiB/s wr, 1.36k op/s 2024-05-20T16:20:15.062 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:20:14 smithi160 ceph-mon[36347]: pgmap v550: 129 pgs: 129 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 723 MiB/s wr, 1.39k op/s 2024-05-20T16:20:15.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:20:14 smithi082 ceph-mon[32238]: pgmap v550: 129 pgs: 129 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 723 MiB/s wr, 1.39k op/s 2024-05-20T16:20:15.135 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:20:14 smithi184 ceph-mon[36209]: pgmap v550: 129 pgs: 129 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 723 MiB/s wr, 1.39k op/s 2024-05-20T16:20:17.062 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:20:16 smithi160 ceph-mon[36347]: pgmap v551: 129 pgs: 129 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 785 MiB/s wr, 1.40k op/s 2024-05-20T16:20:17.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:20:16 smithi082 ceph-mon[32238]: pgmap v551: 129 pgs: 129 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 785 MiB/s wr, 1.40k op/s 2024-05-20T16:20:17.136 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:20:16 smithi184 ceph-mon[36209]: pgmap v551: 129 pgs: 129 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 785 MiB/s wr, 1.40k op/s 2024-05-20T16:20:18.062 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:20:17 smithi160 ceph-mon[36347]: pgmap v552: 129 pgs: 129 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 636 MiB/s wr, 943 op/s 2024-05-20T16:20:18.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:20:17 smithi082 ceph-mon[32238]: pgmap v552: 129 pgs: 129 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 636 MiB/s wr, 943 op/s 2024-05-20T16:20:18.135 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:20:17 smithi184 ceph-mon[36209]: pgmap v552: 129 pgs: 129 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 636 MiB/s wr, 943 op/s 2024-05-20T16:20:20.543 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:20.543+0000 7f394cbd5640 1 -- 172.21.15.82:0/1274943325 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3948100b40 msgr2=0x7f3948101840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:20:20.543 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:20.543+0000 7f394cbd5640 1 --2- 172.21.15.82:0/1274943325 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3948100b40 0x7f3948101840 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f393801a910 tx=0x7f393803b450 comp rx=0 tx=0).stop 2024-05-20T16:20:20.544 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:20.545+0000 7f394cbd5640 1 -- 172.21.15.82:0/1274943325 shutdown_connections 2024-05-20T16:20:20.544 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:20.545+0000 7f394cbd5640 1 --2- 172.21.15.82:0/1274943325 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3948101d80 0x7f3948102190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:20:20.544 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:20.545+0000 7f394cbd5640 1 --2- 172.21.15.82:0/1274943325 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3948106a80 0x7f3948108ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:20:20.544 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:20.545+0000 7f394cbd5640 1 --2- 172.21.15.82:0/1274943325 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3948100b40 0x7f3948101840 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:20:20.545 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:20.545+0000 7f394cbd5640 1 -- 172.21.15.82:0/1274943325 >> 172.21.15.82:0/1274943325 conn(0x7f39480fbd50 msgr2=0x7f39480fe1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:20:20.545 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:20.546+0000 7f394cbd5640 1 -- 172.21.15.82:0/1274943325 shutdown_connections 2024-05-20T16:20:20.546 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:20.546+0000 7f394cbd5640 1 -- 172.21.15.82:0/1274943325 wait complete. 2024-05-20T16:20:20.546 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:20.547+0000 7f394cbd5640 1 Processor -- start 2024-05-20T16:20:20.546 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:20.547+0000 7f394cbd5640 1 -- start start 2024-05-20T16:20:20.547 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:20.548+0000 7f394cbd5640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3948100b40 0x7f39481973a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:20:20.548 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:20.548+0000 7f394cbd5640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3948101d80 0x7f39481978e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:20:20.548 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:20.548+0000 7f394cbd5640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3948106a80 0x7f394819e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:20:20.548 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:20.548+0000 7f393e59c640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3948100b40 0x7f39481973a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:20:20.548 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:20.548+0000 7f394cbd5640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f394806dd50 con 0x7f3948101d80 2024-05-20T16:20:20.548 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:20.548+0000 7f393e59c640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3948100b40 0x7f39481973a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.82:36976/0 (socket says 172.21.15.82:36976) 2024-05-20T16:20:20.548 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:20.548+0000 7f393e59c640 1 -- 172.21.15.82:0/191137543 learned_addr learned my addr 172.21.15.82:0/191137543 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:20:20.548 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:20.548+0000 7f394cbd5640 1 -- 172.21.15.82:0/191137543 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f394806dbd0 con 0x7f3948100b40 2024-05-20T16:20:20.548 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:20.548+0000 7f394cbd5640 1 -- 172.21.15.82:0/191137543 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f394806ded0 con 0x7f3948106a80 2024-05-20T16:20:20.549 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:20.548+0000 7f3946d9d640 1 --2- 172.21.15.82:0/191137543 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3948106a80 0x7f394819e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:20:20.549 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:20.549+0000 7f3946d9d640 1 -- 172.21.15.82:0/191137543 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3948100b40 msgr2=0x7f39481973a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:20:20.549 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:20.549+0000 7f3946d9d640 1 --2- 172.21.15.82:0/191137543 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3948100b40 0x7f39481973a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:20:20.549 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:20.549+0000 7f3946d9d640 1 -- 172.21.15.82:0/191137543 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3948101d80 msgr2=0x7f39481978e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:20:20.549 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:20.549+0000 7f3946d9d640 1 --2- 172.21.15.82:0/191137543 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3948101d80 0x7f39481978e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:20:20.550 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:20.549+0000 7f3946d9d640 1 -- 172.21.15.82:0/191137543 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f394819e950 con 0x7f3948106a80 2024-05-20T16:20:20.550 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:20.549+0000 7f393e59c640 1 --2- 172.21.15.82:0/191137543 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3948100b40 0x7f39481973a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T16:20:20.550 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:20.549+0000 7f3946d9d640 1 --2- 172.21.15.82:0/191137543 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3948106a80 0x7f394819e2e0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f394000be20 tx=0x7f394000bf10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:20:20.550 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:20.549+0000 7f393ffff640 1 -- 172.21.15.82:0/191137543 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3940019070 con 0x7f3948106a80 2024-05-20T16:20:20.551 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:20.549+0000 7f393ffff640 1 -- 172.21.15.82:0/191137543 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3940014920 con 0x7f3948106a80 2024-05-20T16:20:20.551 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:20.549+0000 7f393ffff640 1 -- 172.21.15.82:0/191137543 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3940012930 con 0x7f3948106a80 2024-05-20T16:20:20.551 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:20.550+0000 7f394cbd5640 1 -- 172.21.15.82:0/191137543 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f394819ec40 con 0x7f3948106a80 2024-05-20T16:20:20.551 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:20.550+0000 7f394cbd5640 1 -- 172.21.15.82:0/191137543 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f394819f140 con 0x7f3948106a80 2024-05-20T16:20:20.555 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:20.555+0000 7f3953a84740 1 Processor -- start 2024-05-20T16:20:20.555 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:20.556+0000 7f393ffff640 1 -- 172.21.15.82:0/191137543 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f394001c070 con 0x7f3948106a80 2024-05-20T16:20:20.555 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:20.556+0000 7f3953a84740 1 -- start start 2024-05-20T16:20:20.562 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:20.556+0000 7f3953a84740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f80df5a1b0 0x55f80e2fa050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:20:20.563 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:20.556+0000 7f3953a84740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f80df312f0 0x55f80e2fa5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:20:20.563 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:20.556+0000 7f3953a84740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f80df394d0 0x55f80e3014d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:20:20.563 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:20.556+0000 7f3953a84740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55f80e25e340 con 0x55f80df312f0 2024-05-20T16:20:20.563 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:20.556+0000 7f3953a84740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x55f80e206290 con 0x55f80df394d0 2024-05-20T16:20:20.563 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:20.556+0000 7f3953a84740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55f80e25a300 con 0x55f80df5a1b0 2024-05-20T16:20:20.563 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:20.563+0000 7f393e59c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f80df312f0 0x55f80e2fa5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:20:20.563 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:20.563+0000 7f393e59c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f80df312f0 0x55f80e2fa5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58082/0 (socket says 172.21.15.82:58082) 2024-05-20T16:20:20.563 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:20.563+0000 7f393e59c640 1 -- 172.21.15.82:0/2086702926 learned_addr learned my addr 172.21.15.82:0/2086702926 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:20:20.564 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:20.564+0000 7f393ffff640 1 --2- 172.21.15.82:0/191137543 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f39340a2c00 0x7f39340a5110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:20:20.565 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:20.565+0000 7f394659c640 1 --2- 172.21.15.82:0/2086702926 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f80df5a1b0 0x55f80e2fa050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:20:20.565 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:20.565+0000 7f394659c640 1 --2- 172.21.15.82:0/2086702926 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f80df394d0 0x55f80e3014d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:20:20.565 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:20.565+0000 7f393ffff640 1 -- 172.21.15.82:0/191137543 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9150+0+0 (secure 0 0 0) 0x7f39400cec30 con 0x7f3948106a80 2024-05-20T16:20:20.565 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:20.565+0000 7f3946d9d640 1 --2- 172.21.15.82:0/191137543 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f39340a2c00 0x7f39340a5110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:20:20.565 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:20.565+0000 7f393e59c640 1 -- 172.21.15.82:0/2086702926 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f80df5a1b0 msgr2=0x55f80e2fa050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:20:20.566 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:20.565+0000 7f393e59c640 1 --2- 172.21.15.82:0/2086702926 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f80df5a1b0 0x55f80e2fa050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:20:20.566 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:20.565+0000 7f393e59c640 1 -- 172.21.15.82:0/2086702926 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f80df394d0 msgr2=0x55f80e3014d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:20:20.566 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:20.565+0000 7f393e59c640 1 --2- 172.21.15.82:0/2086702926 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f80df394d0 0x55f80e3014d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:20:20.566 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:20.565+0000 7f393e59c640 1 -- 172.21.15.82:0/2086702926 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f80e22a8e0 con 0x55f80df312f0 2024-05-20T16:20:20.566 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:20.566+0000 7f3946d9d640 1 --2- 172.21.15.82:0/191137543 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f39340a2c00 0x7f39340a5110 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f394000bdf0 tx=0x7f3940012340 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:20:20.567 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:20.567+0000 7f393e59c640 1 --2- 172.21.15.82:0/2086702926 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f80df312f0 0x55f80e2fa5d0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f3938003810 tx=0x7f39380046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd9f09ac31eee10f server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:20:20.569 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:20.570+0000 7f393cd99640 1 -- 172.21.15.82:0/2086702926 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3938052030 con 0x55f80df312f0 2024-05-20T16:20:20.570 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:20.570+0000 7f393cd99640 1 -- 172.21.15.82:0/2086702926 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3938005de0 con 0x55f80df312f0 2024-05-20T16:20:20.572 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:20.572+0000 7f3953a84740 1 -- 172.21.15.82:0/2086702926 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f80df312f0 msgr2=0x55f80e2fa5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:20:20.572 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:20.572+0000 7f3953a84740 1 --2- 172.21.15.82:0/2086702926 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f80df312f0 0x55f80e2fa5d0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f3938003810 tx=0x7f39380046c0 comp rx=0 tx=0).stop 2024-05-20T16:20:20.572 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:20.572+0000 7f3953a84740 1 -- 172.21.15.82:0/2086702926 shutdown_connections 2024-05-20T16:20:20.572 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:20.572+0000 7f3953a84740 1 --2- 172.21.15.82:0/2086702926 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f80df312f0 0x55f80e2fa5d0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:20:20.572 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:20.572+0000 7f3953a84740 1 --2- 172.21.15.82:0/2086702926 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f80df394d0 0x55f80e3014d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:20:20.572 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:20.572+0000 7f3953a84740 1 --2- 172.21.15.82:0/2086702926 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f80df5a1b0 0x55f80e2fa050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:20:20.573 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:20.572+0000 7f3953a84740 1 -- 172.21.15.82:0/2086702926 >> 172.21.15.82:0/2086702926 conn(0x55f80df4e300 msgr2=0x55f80e2f59e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:20:20.573 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:20.573+0000 7f3953a84740 1 -- 172.21.15.82:0/2086702926 shutdown_connections 2024-05-20T16:20:20.574 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:20.574+0000 7f3953a84740 1 -- 172.21.15.82:0/2086702926 wait complete. 2024-05-20T16:20:20.574 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:20.574+0000 7f3953a84740 1 -- start start 2024-05-20T16:20:20.575 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:20.575+0000 7f3953a84740 1 Processor -- start 2024-05-20T16:20:20.575 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:20.575+0000 7f3953a84740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f80df394d0 0x55f80e303ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:20:20.575 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:20.575+0000 7f3953a84740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f80df5a1b0 0x55f80e304420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:20:20.576 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:20.575+0000 7f3953a84740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f80df18d80 0x55f80e30b310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:20:20.576 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:20.575+0000 7f3953a84740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55f80e301f10 con 0x55f80df5a1b0 2024-05-20T16:20:20.576 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:20.575+0000 7f3953a84740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x55f80e219790 con 0x55f80df18d80 2024-05-20T16:20:20.576 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:20.575+0000 7f3953a84740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55f80e302090 con 0x55f80df394d0 2024-05-20T16:20:20.576 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:20.575+0000 7f3953a84740 10 client.0 _ll_register_callbacks cb 0x55f80e254a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T16:20:20.577 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:20.576+0000 7f393e59c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f80df5a1b0 0x55f80e304420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:20:20.578 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:20.577+0000 7f393e59c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f80df5a1b0 0x55f80e304420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58096/0 (socket says 172.21.15.82:58096) 2024-05-20T16:20:20.578 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:20.577+0000 7f393e59c640 1 -- 172.21.15.82:0/3084455083 learned_addr learned my addr 172.21.15.82:0/3084455083 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:20:20.578 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:20.578+0000 7f394659c640 1 --2- 172.21.15.82:0/3084455083 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f80df394d0 0x55f80e303ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:20:20.579 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:20.578+0000 7f394659c640 1 --2- 172.21.15.82:0/3084455083 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f80df18d80 0x55f80e30b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:20:20.581 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:20.581+0000 7f394659c640 1 -- 172.21.15.82:0/3084455083 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f80df394d0 msgr2=0x55f80e303ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:20:20.581 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:20.581+0000 7f394659c640 1 --2- 172.21.15.82:0/3084455083 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f80df394d0 0x55f80e303ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:20:20.582 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:20.581+0000 7f394659c640 1 -- 172.21.15.82:0/3084455083 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f80df5a1b0 msgr2=0x55f80e304420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:20:20.582 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:20.581+0000 7f394659c640 1 --2- 172.21.15.82:0/3084455083 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f80df5a1b0 0x55f80e304420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:20:20.582 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:20.581+0000 7f394659c640 1 -- 172.21.15.82:0/3084455083 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f80df72210 con 0x55f80df18d80 2024-05-20T16:20:20.588 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:20.587+0000 7f394659c640 1 --2- 172.21.15.82:0/3084455083 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f80df18d80 0x55f80e30b310 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x55f80e2fb220 tx=0x7f3930005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1e521bd347f9a664 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:20:20.588 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:20.588+0000 7f392dffb640 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-20T16:20:20.592 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:20.592+0000 7f392dffb640 1 -- 172.21.15.82:0/3084455083 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3930015110 con 0x55f80df18d80 2024-05-20T16:20:20.592 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:20.592+0000 7f3953a84740 1 -- 172.21.15.82:0/3084455083 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55f80e26ba70 con 0x55f80df18d80 2024-05-20T16:20:20.592 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:20.592+0000 7f392dffb640 1 -- 172.21.15.82:0/3084455083 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9150+0+0 (secure 0 0 0) 0x7f3930009040 con 0x55f80df18d80 2024-05-20T16:20:20.593 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:20.593+0000 7f392dffb640 1 -- 172.21.15.82:0/3084455083 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f3930028050 con 0x55f80df18d80 2024-05-20T16:20:20.594 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:20.593+0000 7f3953a84740 10 client.25274 fetch_fsmap learned FSMap version 27 2024-05-20T16:20:20.594 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:20.594+0000 7f3953a84740 1 -- 172.21.15.82:0/3084455083 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55f80e03e620 con 0x55f80df18d80 2024-05-20T16:20:20.596 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:20.596+0000 7f392dffb640 1 -- 172.21.15.82:0/3084455083 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55f80e26ba70 con 0x55f80df18d80 2024-05-20T16:20:20.596 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:20.596+0000 7f392dffb640 1 -- 172.21.15.82:0/3084455083 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f393001f070 con 0x55f80df18d80 2024-05-20T16:20:20.598 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:20.598+0000 7f3953a84740 10 client.25274 fetch_fsmap finished waiting for FSMap version 27 2024-05-20T16:20:20.598 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:20.598+0000 7f3953a84740 10 client.25274 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T16:20:20.598 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:20.598+0000 7f3953a84740 20 client.25274 populate_metadata read hostname 'smithi082' 2024-05-20T16:20:20.598 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:20.598+0000 7f3953a84740 1 --2- 172.21.15.82:0/3084455083 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x55f80deee1c0 0x55f80e311c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:20:20.598 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:20.598+0000 7f3953a84740 4 client.25274 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T16:20:20.598 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:20.598+0000 7f3953a84740 1 -- 172.21.15.82:0/3084455083 --> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f80e265dc0 con 0x55f80deee1c0 2024-05-20T16:20:20.602 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:20.602+0000 7f393e59c640 1 --2- 172.21.15.82:0/3084455083 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x55f80deee1c0 0x55f80e311c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:20:20.612 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:20.612+0000 7f393e59c640 1 --2- 172.21.15.82:0/3084455083 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x55f80deee1c0 0x55f80e311c20 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8427d08e133d4626 server_cookie=30d6f80f9f145860 in_seq=0 out_seq=0 2024-05-20T16:20:20.612 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:20.613+0000 7f392dffb640 10 client.25274 ms_handle_connect on v2:172.21.15.160:6838/2765554482 2024-05-20T16:20:20.621 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:20.621+0000 7f392dffb640 1 -- 172.21.15.82:0/3084455083 <== mds.0 v2:172.21.15.160:6838/2765554482 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55f80e265dc0 con 0x55f80deee1c0 2024-05-20T16:20:20.621 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:20.621+0000 7f392dffb640 10 client.25274 handle_command_reply: tid=0 2024-05-20T16:20:20.625 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:20.624+0000 7f3953a84740 1 client.25274 shutdown 2024-05-20T16:20:20.626 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:20.626+0000 7f3953a84740 1 -- 172.21.15.82:0/3084455083 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f80df18d80 msgr2=0x55f80e30b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:20:20.626 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:20.626+0000 7f3953a84740 1 --2- 172.21.15.82:0/3084455083 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f80df18d80 0x55f80e30b310 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x55f80e2fb220 tx=0x7f3930005f90 comp rx=0 tx=0).stop 2024-05-20T16:20:20.630 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:20.627+0000 7f3953a84740 1 -- 172.21.15.82:0/3084455083 shutdown_connections 2024-05-20T16:20:20.630 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:20.627+0000 7f3953a84740 1 --2- 172.21.15.82:0/3084455083 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x55f80deee1c0 0x55f80e311c20 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:20:20.631 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:20.628+0000 7f3953a84740 1 --2- 172.21.15.82:0/3084455083 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f80df5a1b0 0x55f80e304420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:20:20.631 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:20.628+0000 7f3953a84740 1 --2- 172.21.15.82:0/3084455083 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f80df18d80 0x55f80e30b310 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:20:20.631 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:20.628+0000 7f3953a84740 1 --2- 172.21.15.82:0/3084455083 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f80df394d0 0x55f80e303ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:20:20.631 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:20.628+0000 7f3953a84740 1 -- 172.21.15.82:0/3084455083 >> 172.21.15.82:0/3084455083 conn(0x55f80df4e300 msgr2=0x55f80e2f8990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:20:20.631 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:20.631+0000 7f392dffb640 0 client.25274 ms_handle_reset on v2:172.21.15.160:6838/2765554482 2024-05-20T16:20:20.631 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:20.631+0000 7f3953a84740 1 -- 172.21.15.82:0/3084455083 shutdown_connections 2024-05-20T16:20:20.631 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:20.632+0000 7f3953a84740 1 -- 172.21.15.82:0/3084455083 wait complete. 2024-05-20T16:20:20.631 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:20.632+0000 7f3953a84740 20 client.25274 trim_cache size 0 max 16384 2024-05-20T16:20:20.679 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:20.679+0000 7f3953a84740 1 Processor -- start 2024-05-20T16:20:20.680 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:20.679+0000 7f3953a84740 1 -- start start 2024-05-20T16:20:20.680 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:20.680+0000 7f3953a84740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f80df18d80 0x55f80e302b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:20:20.681 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:20.680+0000 7f3953a84740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f80df394d0 0x55f80e30b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:20:20.681 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:20.680+0000 7f3953a84740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f80df5a1b0 0x55f80e30b810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:20:20.681 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:20.680+0000 7f3953a84740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f39481abf80 con 0x55f80df394d0 2024-05-20T16:20:20.682 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:20.680+0000 7f3953a84740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f39481a9850 con 0x55f80df5a1b0 2024-05-20T16:20:20.682 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:20.680+0000 7f3953a84740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f39481a76a0 con 0x55f80df18d80 2024-05-20T16:20:20.682 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:20.682+0000 7f394659c640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f80df18d80 0x55f80e302b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:20:20.682 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:20.682+0000 7f394659c640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f80df5a1b0 0x55f80e30b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:20:20.683 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:20.684+0000 7f393e59c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f80df394d0 0x55f80e30b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:20:20.683 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:20.684+0000 7f393e59c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f80df394d0 0x55f80e30b2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58106/0 (socket says 172.21.15.82:58106) 2024-05-20T16:20:20.684 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:20.684+0000 7f393e59c640 1 -- 172.21.15.82:0/542649847 learned_addr learned my addr 172.21.15.82:0/542649847 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:20:20.687 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:20.686+0000 7f394659c640 1 -- 172.21.15.82:0/542649847 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f80df18d80 msgr2=0x55f80e302b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:20:20.687 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:20.686+0000 7f394659c640 1 --2- 172.21.15.82:0/542649847 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f80df18d80 0x55f80e302b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:20:20.687 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:20.686+0000 7f394659c640 1 -- 172.21.15.82:0/542649847 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f80df394d0 msgr2=0x55f80e30b2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:20:20.687 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:20.686+0000 7f394659c640 1 --2- 172.21.15.82:0/542649847 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f80df394d0 0x55f80e30b2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:20:20.687 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:20.686+0000 7f394659c640 1 -- 172.21.15.82:0/542649847 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f80e2f9c00 con 0x55f80df5a1b0 2024-05-20T16:20:20.688 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:20.686+0000 7f394659c640 1 --2- 172.21.15.82:0/542649847 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f80df5a1b0 0x55f80e30b810 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f3930015650 tx=0x7f3930009a00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3973db6164c10cea server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:20:20.688 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:20.688+0000 7f392ffff640 1 -- 172.21.15.82:0/542649847 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3930005bf0 con 0x55f80df5a1b0 2024-05-20T16:20:20.688 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:20.688+0000 7f392ffff640 1 -- 172.21.15.82:0/542649847 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3930004340 con 0x55f80df5a1b0 2024-05-20T16:20:20.688 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:20.688+0000 7f392ffff640 1 -- 172.21.15.82:0/542649847 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39300039c0 con 0x55f80df5a1b0 2024-05-20T16:20:20.693 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:20.689+0000 7f3953a84740 1 -- 172.21.15.82:0/542649847 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f80df5a1b0 msgr2=0x55f80e30b810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:20:20.693 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:20.689+0000 7f3953a84740 1 --2- 172.21.15.82:0/542649847 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f80df5a1b0 0x55f80e30b810 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f3930015650 tx=0x7f3930009a00 comp rx=0 tx=0).stop 2024-05-20T16:20:20.700 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:20.700+0000 7f3953a84740 1 -- 172.21.15.82:0/542649847 shutdown_connections 2024-05-20T16:20:20.700 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:20.700+0000 7f3953a84740 1 --2- 172.21.15.82:0/542649847 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f80df394d0 0x55f80e30b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:20:20.700 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:20.700+0000 7f3953a84740 1 --2- 172.21.15.82:0/542649847 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f80df5a1b0 0x55f80e30b810 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:20:20.700 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:20.700+0000 7f3953a84740 1 --2- 172.21.15.82:0/542649847 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f80df18d80 0x55f80e302b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:20:20.701 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:20.700+0000 7f3953a84740 1 -- 172.21.15.82:0/542649847 >> 172.21.15.82:0/542649847 conn(0x55f80df4e300 msgr2=0x55f80e302480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:20:20.711 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:20.710+0000 7f3953a84740 1 -- 172.21.15.82:0/542649847 shutdown_connections 2024-05-20T16:20:20.715 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:20.715+0000 7f3953a84740 1 -- 172.21.15.82:0/542649847 wait complete. 2024-05-20T16:20:20.715 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:20.716+0000 7f3953a84740 1 -- start start 2024-05-20T16:20:20.715 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:20.716+0000 7f3953a84740 1 Processor -- start 2024-05-20T16:20:20.717 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:20.716+0000 7f3953a84740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f80df18d80 0x55f80e30b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:20:20.717 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:20.716+0000 7f3953a84740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f80df394d0 0x55f80e308030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:20:20.717 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:20.717+0000 7f3953a84740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f80df5a1b0 0x55f80e308570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:20:20.717 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:20.717+0000 7f3953a84740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f39481a7d10 con 0x55f80df394d0 2024-05-20T16:20:20.717 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:20.717+0000 7f3953a84740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f39481a9180 con 0x55f80df18d80 2024-05-20T16:20:20.717 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:20.717+0000 7f3953a84740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f39481a83e0 con 0x55f80df5a1b0 2024-05-20T16:20:20.717 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:20.717+0000 7f3953a84740 10 client.0 _ll_register_callbacks cb 0x7f3930009610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T16:20:20.717 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:20.717+0000 7f394659c640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f80df18d80 0x55f80e30b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:20:20.717 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:20.717+0000 7f394659c640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f80df18d80 0x55f80e30b4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.82:37010/0 (socket says 172.21.15.82:37010) 2024-05-20T16:20:20.717 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:20.717+0000 7f394659c640 1 -- 172.21.15.82:0/4134909471 learned_addr learned my addr 172.21.15.82:0/4134909471 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:20:20.718 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:20.718+0000 7f394659c640 1 -- 172.21.15.82:0/4134909471 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f80df5a1b0 msgr2=0x55f80e308570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:20:20.718 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:20.718+0000 7f394659c640 1 --2- 172.21.15.82:0/4134909471 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f80df5a1b0 0x55f80e308570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:20:20.718 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:20.718+0000 7f394659c640 1 -- 172.21.15.82:0/4134909471 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f80df394d0 msgr2=0x55f80e308030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:20:20.718 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:20.718+0000 7f394659c640 1 --2- 172.21.15.82:0/4134909471 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f80df394d0 0x55f80e308030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:20:20.718 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:20.718+0000 7f394659c640 1 -- 172.21.15.82:0/4134909471 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f80e308ae0 con 0x55f80df18d80 2024-05-20T16:20:20.719 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:20.719+0000 7f394659c640 1 --2- 172.21.15.82:0/4134909471 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f80df18d80 0x55f80e30b4b0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f3930005910 tx=0x7f3930000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=238b192a7418fc7 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:20:20.722 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:20.722+0000 7f392dffb640 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-20T16:20:20.722 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:20.722+0000 7f392dffb640 1 -- 172.21.15.82:0/4134909471 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3930005b90 con 0x55f80df18d80 2024-05-20T16:20:20.722 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:20.723+0000 7f392dffb640 1 -- 172.21.15.82:0/4134909471 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9150+0+0 (secure 0 0 0) 0x7f3930003970 con 0x55f80df18d80 2024-05-20T16:20:20.723 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:20.724+0000 7f3953a84740 1 -- 172.21.15.82:0/4134909471 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3930004340 con 0x55f80df18d80 2024-05-20T16:20:20.727 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:20.727+0000 7f392dffb640 1 -- 172.21.15.82:0/4134909471 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f3930015110 con 0x55f80df18d80 2024-05-20T16:20:20.727 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:20.728+0000 7f3953a84740 10 client.25280 fetch_fsmap learned FSMap version 27 2024-05-20T16:20:20.727 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:20.728+0000 7f3953a84740 1 -- 172.21.15.82:0/4134909471 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55f80e378f80 con 0x55f80df18d80 2024-05-20T16:20:20.728 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:20.728+0000 7f392dffb640 1 -- 172.21.15.82:0/4134909471 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3930004340 con 0x55f80df18d80 2024-05-20T16:20:20.728 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:20.728+0000 7f392dffb640 1 -- 172.21.15.82:0/4134909471 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f393001f070 con 0x55f80df18d80 2024-05-20T16:20:20.728 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:20.729+0000 7f3953a84740 10 client.25280 fetch_fsmap finished waiting for FSMap version 27 2024-05-20T16:20:20.729 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:20.729+0000 7f3953a84740 10 client.25280 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T16:20:20.729 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:20.729+0000 7f3953a84740 20 client.25280 populate_metadata read hostname 'smithi082' 2024-05-20T16:20:20.729 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:20.729+0000 7f3953a84740 1 --2- 172.21.15.82:0/4134909471 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x55f80e3798a0 0x55f80e37bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:20:20.729 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:20.729+0000 7f3953a84740 4 client.25280 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-20T16:20:20.729 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:20.729+0000 7f3953a84740 1 -- 172.21.15.82:0/4134909471 --> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f80e1e38e0 con 0x55f80e3798a0 2024-05-20T16:20:20.732 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:20.732+0000 7f393e59c640 1 --2- 172.21.15.82:0/4134909471 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x55f80e3798a0 0x55f80e37bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:20:20.744 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:20.744+0000 7f392dffb640 10 client.25280 ms_handle_connect on v2:172.21.15.160:6838/2765554482 2024-05-20T16:20:20.744 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:20.744+0000 7f393e59c640 1 --2- 172.21.15.82:0/4134909471 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x55f80e3798a0 0x55f80e37bcf0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d10fa2056e710d06 server_cookie=bd962b4da5943e35 in_seq=0 out_seq=0 2024-05-20T16:20:21.062 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:20:20 smithi160 ceph-mon[36347]: pgmap v553: 129 pgs: 129 active+clean; 10 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 869 MiB/s wr, 1.25k op/s 2024-05-20T16:20:21.110 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:20:20 smithi082 ceph-mon[32238]: pgmap v553: 129 pgs: 129 active+clean; 10 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 869 MiB/s wr, 1.25k op/s 2024-05-20T16:20:21.135 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:20:20 smithi184 ceph-mon[36209]: pgmap v553: 129 pgs: 129 active+clean; 10 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 869 MiB/s wr, 1.25k op/s 2024-05-20T16:20:23.062 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:20:22 smithi160 ceph-mon[36347]: pgmap v554: 129 pgs: 129 active+clean; 10 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 730 MiB/s wr, 979 op/s 2024-05-20T16:20:23.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:20:22 smithi082 ceph-mon[32238]: pgmap v554: 129 pgs: 129 active+clean; 10 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 730 MiB/s wr, 979 op/s 2024-05-20T16:20:23.135 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:20:22 smithi184 ceph-mon[36209]: pgmap v554: 129 pgs: 129 active+clean; 10 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 730 MiB/s wr, 979 op/s 2024-05-20T16:20:24.062 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:20:23 smithi160 ceph-mon[36347]: pgmap v555: 129 pgs: 129 active+clean; 10 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 743 MiB/s wr, 1.05k op/s 2024-05-20T16:20:24.110 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:20:23 smithi082 ceph-mon[32238]: pgmap v555: 129 pgs: 129 active+clean; 10 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 743 MiB/s wr, 1.05k op/s 2024-05-20T16:20:24.135 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:20:23 smithi184 ceph-mon[36209]: pgmap v555: 129 pgs: 129 active+clean; 10 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 743 MiB/s wr, 1.05k op/s 2024-05-20T16:20:24.300 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:24.300+0000 7f392dffb640 1 -- 172.21.15.82:0/4134909471 <== mds.0 v2:172.21.15.160:6838/2765554482 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f80e1e38e0 con 0x55f80e3798a0 2024-05-20T16:20:24.300 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:24.300+0000 7f392dffb640 10 client.25280 handle_command_reply: tid=0 2024-05-20T16:20:24.301 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:24.300+0000 7f3953a84740 1 client.25280 shutdown 2024-05-20T16:20:24.302 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:24.301+0000 7f3953a84740 1 -- 172.21.15.82:0/4134909471 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f80df18d80 msgr2=0x55f80e30b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:20:24.302 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:24.301+0000 7f3953a84740 1 --2- 172.21.15.82:0/4134909471 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f80df18d80 0x55f80e30b4b0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f3930005910 tx=0x7f3930000bc0 comp rx=0 tx=0).stop 2024-05-20T16:20:24.302 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:24.302+0000 7f3953a84740 1 -- 172.21.15.82:0/4134909471 shutdown_connections 2024-05-20T16:20:24.302 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:24.302+0000 7f3953a84740 1 --2- 172.21.15.82:0/4134909471 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x55f80e3798a0 0x55f80e37bcf0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:20:24.302 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:24.302+0000 7f3953a84740 1 --2- 172.21.15.82:0/4134909471 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f80df394d0 0x55f80e308030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:20:24.302 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:24.302+0000 7f3953a84740 1 --2- 172.21.15.82:0/4134909471 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f80df5a1b0 0x55f80e308570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:20:24.302 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:24.302+0000 7f3953a84740 1 --2- 172.21.15.82:0/4134909471 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f80df18d80 0x55f80e30b4b0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:20:24.303 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:24.302+0000 7f3953a84740 1 -- 172.21.15.82:0/4134909471 >> 172.21.15.82:0/4134909471 conn(0x55f80df4e300 msgr2=0x55f80e2f8880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:20:24.303 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:24.302+0000 7f392dffb640 0 client.25280 ms_handle_reset on v2:172.21.15.160:6838/2765554482 2024-05-20T16:20:24.303 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:24.302+0000 7f3953a84740 1 -- 172.21.15.82:0/4134909471 shutdown_connections 2024-05-20T16:20:24.303 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:24.302+0000 7f3953a84740 1 -- 172.21.15.82:0/4134909471 wait complete. 2024-05-20T16:20:24.303 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:24.302+0000 7f3953a84740 20 client.25280 trim_cache size 0 max 16384 2024-05-20T16:20:24.304 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:{ 2024-05-20T16:20:24.304 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "message": "", 2024-05-20T16:20:24.304 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "return_code": 0 2024-05-20T16:20:24.304 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:} 2024-05-20T16:20:24.305 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:24.303+0000 7f394cbd5640 1 -- 172.21.15.82:0/191137543 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f39340a2c00 msgr2=0x7f39340a5110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:20:24.305 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:24.303+0000 7f394cbd5640 1 --2- 172.21.15.82:0/191137543 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f39340a2c00 0x7f39340a5110 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f394000bdf0 tx=0x7f3940012340 comp rx=0 tx=0).stop 2024-05-20T16:20:24.305 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:24.303+0000 7f394cbd5640 1 -- 172.21.15.82:0/191137543 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3948106a80 msgr2=0x7f394819e2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:20:24.305 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:24.303+0000 7f394cbd5640 1 --2- 172.21.15.82:0/191137543 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3948106a80 0x7f394819e2e0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f394000be20 tx=0x7f394000bf10 comp rx=0 tx=0).stop 2024-05-20T16:20:24.305 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:24.303+0000 7f394cbd5640 1 -- 172.21.15.82:0/191137543 shutdown_connections 2024-05-20T16:20:24.305 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:24.303+0000 7f394cbd5640 1 --2- 172.21.15.82:0/191137543 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f39340a2c00 0x7f39340a5110 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:20:24.305 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:24.303+0000 7f394cbd5640 1 --2- 172.21.15.82:0/191137543 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3948101d80 0x7f39481978e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:20:24.305 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:24.303+0000 7f394cbd5640 1 --2- 172.21.15.82:0/191137543 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3948106a80 0x7f394819e2e0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:20:24.305 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:24.303+0000 7f394cbd5640 1 --2- 172.21.15.82:0/191137543 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3948100b40 0x7f39481973a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:20:24.306 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:24.303+0000 7f394cbd5640 1 -- 172.21.15.82:0/191137543 >> 172.21.15.82:0/191137543 conn(0x7f39480fbd50 msgr2=0x7f39480fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:20:24.306 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:24.303+0000 7f394cbd5640 1 -- 172.21.15.82:0/191137543 shutdown_connections 2024-05-20T16:20:24.306 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:24.304+0000 7f394cbd5640 1 -- 172.21.15.82:0/191137543 wait complete. 2024-05-20T16:20:27.062 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:20:26 smithi160 ceph-mon[36347]: pgmap v556: 129 pgs: 129 active+clean; 10 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 824 MiB/s wr, 1.17k op/s 2024-05-20T16:20:27.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:20:26 smithi082 ceph-mon[32238]: pgmap v556: 129 pgs: 129 active+clean; 10 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 824 MiB/s wr, 1.17k op/s 2024-05-20T16:20:27.135 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:20:26 smithi184 ceph-mon[36209]: pgmap v556: 129 pgs: 129 active+clean; 10 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 824 MiB/s wr, 1.17k op/s 2024-05-20T16:20:29.062 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:20:28 smithi160 ceph-mon[36347]: pgmap v557: 129 pgs: 129 active+clean; 10 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 688 MiB/s wr, 948 op/s 2024-05-20T16:20:29.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:20:28 smithi082 ceph-mon[32238]: pgmap v557: 129 pgs: 129 active+clean; 10 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 688 MiB/s wr, 948 op/s 2024-05-20T16:20:29.135 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:20:28 smithi184 ceph-mon[36209]: pgmap v557: 129 pgs: 129 active+clean; 10 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 688 MiB/s wr, 948 op/s 2024-05-20T16:20:31.062 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:20:30 smithi160 ceph-mon[36347]: pgmap v558: 129 pgs: 129 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 860 MiB/s wr, 1.23k op/s 2024-05-20T16:20:31.110 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:20:30 smithi082 ceph-mon[32238]: pgmap v558: 129 pgs: 129 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 860 MiB/s wr, 1.23k op/s 2024-05-20T16:20:31.135 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:20:30 smithi184 ceph-mon[36209]: pgmap v558: 129 pgs: 129 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 860 MiB/s wr, 1.23k op/s 2024-05-20T16:20:32.062 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:20:31 smithi160 ceph-mon[36347]: pgmap v559: 129 pgs: 129 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 698 MiB/s wr, 1.05k op/s 2024-05-20T16:20:32.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:20:31 smithi082 ceph-mon[32238]: pgmap v559: 129 pgs: 129 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 698 MiB/s wr, 1.05k op/s 2024-05-20T16:20:32.135 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:20:31 smithi184 ceph-mon[36209]: pgmap v559: 129 pgs: 129 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 698 MiB/s wr, 1.05k op/s 2024-05-20T16:20:35.062 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:20:34 smithi160 ceph-mon[36347]: pgmap v560: 129 pgs: 129 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 737 MiB/s wr, 1.36k op/s 2024-05-20T16:20:35.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:20:34 smithi082 ceph-mon[32238]: pgmap v560: 129 pgs: 129 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 737 MiB/s wr, 1.36k op/s 2024-05-20T16:20:35.135 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:20:34 smithi184 ceph-mon[36209]: pgmap v560: 129 pgs: 129 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 737 MiB/s wr, 1.36k op/s 2024-05-20T16:20:37.062 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:20:36 smithi160 ceph-mon[36347]: pgmap v561: 129 pgs: 129 active+clean; 10 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 815 MiB/s wr, 1.70k op/s 2024-05-20T16:20:37.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:20:36 smithi082 ceph-mon[32238]: pgmap v561: 129 pgs: 129 active+clean; 10 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 815 MiB/s wr, 1.70k op/s 2024-05-20T16:20:37.135 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:20:36 smithi184 ceph-mon[36209]: pgmap v561: 129 pgs: 129 active+clean; 10 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 815 MiB/s wr, 1.70k op/s 2024-05-20T16:20:37.592 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:37.591+0000 7f1b1afab640 1 -- 172.21.15.82:0/1021507803 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1b141011d0 msgr2=0x7f1b141035f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:20:37.592 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:37.592+0000 7f1b1afab640 1 --2- 172.21.15.82:0/1021507803 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1b141011d0 0x7f1b141035f0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f1b0c009be0 tx=0x7f1b0c02a6d0 comp rx=0 tx=0).stop 2024-05-20T16:20:37.592 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:37.592+0000 7f1b1afab640 1 -- 172.21.15.82:0/1021507803 shutdown_connections 2024-05-20T16:20:37.592 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:37.592+0000 7f1b1afab640 1 --2- 172.21.15.82:0/1021507803 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1b14106490 0x7f1b141068a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:20:37.592 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:37.592+0000 7f1b1afab640 1 --2- 172.21.15.82:0/1021507803 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1b14103b30 0x7f1b14105f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:20:37.592 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:37.592+0000 7f1b1afab640 1 --2- 172.21.15.82:0/1021507803 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1b141011d0 0x7f1b141035f0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:20:37.593 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:37.592+0000 7f1b1afab640 1 -- 172.21.15.82:0/1021507803 >> 172.21.15.82:0/1021507803 conn(0x7f1b140fc9e0 msgr2=0x7f1b140fee90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:20:37.593 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:37.592+0000 7f1b1afab640 1 -- 172.21.15.82:0/1021507803 shutdown_connections 2024-05-20T16:20:37.593 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:37.592+0000 7f1b1afab640 1 -- 172.21.15.82:0/1021507803 wait complete. 2024-05-20T16:20:37.593 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:37.594+0000 7f1b1afab640 1 Processor -- start 2024-05-20T16:20:37.594 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:37.594+0000 7f1b1afab640 1 -- start start 2024-05-20T16:20:37.594 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:37.594+0000 7f1b1afab640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1b141011d0 0x7f1b14070a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:20:37.594 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:37.594+0000 7f1b1afab640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1b14103b30 0x7f1b14070f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:20:37.594 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:37.594+0000 7f1b1afab640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1b14106490 0x7f1b14071ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:20:37.594 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:37.594+0000 7f1b1afab640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f1b14114b60 con 0x7f1b14103b30 2024-05-20T16:20:37.594 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:37.594+0000 7f1b1afab640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f1b141149e0 con 0x7f1b14106490 2024-05-20T16:20:37.594 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:37.594+0000 7f1b1afab640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f1b14114ce0 con 0x7f1b141011d0 2024-05-20T16:20:37.598 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:37.595+0000 7f1b197a8640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1b14103b30 0x7f1b14070f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:20:37.598 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:37.595+0000 7f1b197a8640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1b14103b30 0x7f1b14070f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:38568/0 (socket says 172.21.15.82:38568) 2024-05-20T16:20:37.598 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:37.595+0000 7f1b197a8640 1 -- 172.21.15.82:0/1787842243 learned_addr learned my addr 172.21.15.82:0/1787842243 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:20:37.599 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:37.596+0000 7f1b197a8640 1 -- 172.21.15.82:0/1787842243 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1b141011d0 msgr2=0x7f1b14070a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:20:37.599 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:37.596+0000 7f1b197a8640 1 --2- 172.21.15.82:0/1787842243 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1b141011d0 0x7f1b14070a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:20:37.600 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:37.596+0000 7f1b197a8640 1 -- 172.21.15.82:0/1787842243 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1b14106490 msgr2=0x7f1b14071ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:20:37.600 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:37.596+0000 7f1b197a8640 1 --2- 172.21.15.82:0/1787842243 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1b14106490 0x7f1b14071ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:20:37.600 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:37.596+0000 7f1b197a8640 1 -- 172.21.15.82:0/1787842243 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b14072560 con 0x7f1b14103b30 2024-05-20T16:20:37.600 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:37.596+0000 7f1b197a8640 1 --2- 172.21.15.82:0/1787842243 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1b14103b30 0x7f1b14070f90 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f1b1000e9c0 tx=0x7f1b1000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:20:37.600 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:37.597+0000 7f1b0affd640 1 -- 172.21.15.82:0/1787842243 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b1000ccc0 con 0x7f1b14103b30 2024-05-20T16:20:37.600 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:37.597+0000 7f1b1afab640 1 -- 172.21.15.82:0/1787842243 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b14072850 con 0x7f1b14103b30 2024-05-20T16:20:37.600 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:37.597+0000 7f1b1afab640 1 -- 172.21.15.82:0/1787842243 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b14072da0 con 0x7f1b14103b30 2024-05-20T16:20:37.600 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:37.598+0000 7f1b0affd640 1 -- 172.21.15.82:0/1787842243 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1b10005500 con 0x7f1b14103b30 2024-05-20T16:20:37.600 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:37.599+0000 7f1b0affd640 1 -- 172.21.15.82:0/1787842243 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b10010430 con 0x7f1b14103b30 2024-05-20T16:20:37.602 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:37.602+0000 7f1b0affd640 1 -- 172.21.15.82:0/1787842243 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f1b100046c0 con 0x7f1b14103b30 2024-05-20T16:20:37.603 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:37.604+0000 7f1b0affd640 1 --2- 172.21.15.82:0/1787842243 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f1b000a2d50 0x7f1b000a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:20:37.604 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:37.604+0000 7f1b19fa9640 1 --2- 172.21.15.82:0/1787842243 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f1b000a2d50 0x7f1b000a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:20:37.604 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:37.604+0000 7f1b2318f740 1 Processor -- start 2024-05-20T16:20:37.605 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:37.605+0000 7f1b0affd640 1 -- 172.21.15.82:0/1787842243 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9150+0+0 (secure 0 0 0) 0x7f1b100cf770 con 0x7f1b14103b30 2024-05-20T16:20:37.605 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:37.605+0000 7f1b2318f740 1 -- start start 2024-05-20T16:20:37.605 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:37.605+0000 7f1b2318f740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5570ff4671b0 0x5570ff807060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:20:37.605 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:37.605+0000 7f1b2318f740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5570ff43e2f0 0x5570ff8075e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:20:37.605 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:37.605+0000 7f1b2318f740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5570ff4464d0 0x5570ff80e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:20:37.605 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:37.605+0000 7f1b2318f740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x5570ff76b340 con 0x5570ff43e2f0 2024-05-20T16:20:37.605 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:37.605+0000 7f1b2318f740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x5570ff713290 con 0x5570ff4464d0 2024-05-20T16:20:37.605 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:37.605+0000 7f1b2318f740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5570ff767300 con 0x5570ff4671b0 2024-05-20T16:20:37.605 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:37.605+0000 7f1b19fa9640 1 --2- 172.21.15.82:0/1787842243 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f1b000a2d50 0x7f1b000a5260 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f1b0c004af0 tx=0x7f1b0c000f40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:20:37.606 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:37.605+0000 7f1b19fa9640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5570ff4671b0 0x5570ff807060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:20:37.606 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:37.605+0000 7f1b197a8640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5570ff43e2f0 0x5570ff8075e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:20:37.606 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:37.605+0000 7f1b19fa9640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5570ff4671b0 0x5570ff807060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.82:56058/0 (socket says 172.21.15.82:56058) 2024-05-20T16:20:37.606 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:37.605+0000 7f1b19fa9640 1 -- 172.21.15.82:0/1291985873 learned_addr learned my addr 172.21.15.82:0/1291985873 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:20:37.607 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:37.605+0000 7f1b1a7aa640 1 --2- 172.21.15.82:0/1291985873 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5570ff4464d0 0x5570ff80e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:20:37.607 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:37.605+0000 7f1b197a8640 1 -- 172.21.15.82:0/1291985873 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5570ff4671b0 msgr2=0x5570ff807060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:20:37.607 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:37.606+0000 7f1b197a8640 1 --2- 172.21.15.82:0/1291985873 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5570ff4671b0 0x5570ff807060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:20:37.607 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:37.606+0000 7f1b197a8640 1 -- 172.21.15.82:0/1291985873 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5570ff4464d0 msgr2=0x5570ff80e4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:20:37.607 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:37.606+0000 7f1b197a8640 1 --2- 172.21.15.82:0/1291985873 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5570ff4464d0 0x5570ff80e4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:20:37.607 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:37.606+0000 7f1b197a8640 1 -- 172.21.15.82:0/1291985873 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5570ff7378e0 con 0x5570ff43e2f0 2024-05-20T16:20:37.607 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:37.606+0000 7f1b1a7aa640 1 --2- 172.21.15.82:0/1291985873 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5570ff4464d0 0x5570ff80e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T16:20:37.607 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:37.606+0000 7f1b197a8640 1 --2- 172.21.15.82:0/1291985873 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5570ff43e2f0 0x5570ff8075e0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f1b100cce80 tx=0x7f1b1009b540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=184baa4ebe060269 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:20:37.607 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:37.606+0000 7f1afffff640 1 -- 172.21.15.82:0/1291985873 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b1009ff90 con 0x5570ff43e2f0 2024-05-20T16:20:37.607 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:37.606+0000 7f1afffff640 1 -- 172.21.15.82:0/1291985873 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1b100a4850 con 0x5570ff43e2f0 2024-05-20T16:20:37.607 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:37.606+0000 7f1afffff640 1 -- 172.21.15.82:0/1291985873 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b1009ff90 con 0x5570ff43e2f0 2024-05-20T16:20:37.608 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:37.607+0000 7f1b2318f740 1 -- 172.21.15.82:0/1291985873 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5570ff43e2f0 msgr2=0x5570ff8075e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:20:37.608 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:37.607+0000 7f1b2318f740 1 --2- 172.21.15.82:0/1291985873 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5570ff43e2f0 0x5570ff8075e0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f1b100cce80 tx=0x7f1b1009b540 comp rx=0 tx=0).stop 2024-05-20T16:20:37.608 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:37.608+0000 7f1b2318f740 1 -- 172.21.15.82:0/1291985873 shutdown_connections 2024-05-20T16:20:37.608 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:37.608+0000 7f1b2318f740 1 --2- 172.21.15.82:0/1291985873 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5570ff43e2f0 0x5570ff8075e0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:20:37.608 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:37.608+0000 7f1b2318f740 1 --2- 172.21.15.82:0/1291985873 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5570ff4464d0 0x5570ff80e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:20:37.608 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:37.608+0000 7f1b2318f740 1 --2- 172.21.15.82:0/1291985873 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5570ff4671b0 0x5570ff807060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:20:37.608 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:37.608+0000 7f1b2318f740 1 -- 172.21.15.82:0/1291985873 >> 172.21.15.82:0/1291985873 conn(0x5570ff45b300 msgr2=0x5570ff8029f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:20:37.608 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:37.608+0000 7f1b2318f740 1 -- 172.21.15.82:0/1291985873 shutdown_connections 2024-05-20T16:20:37.608 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:37.608+0000 7f1b2318f740 1 -- 172.21.15.82:0/1291985873 wait complete. 2024-05-20T16:20:37.609 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:37.609+0000 7f1b2318f740 1 -- start start 2024-05-20T16:20:37.609 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:37.609+0000 7f1b2318f740 1 Processor -- start 2024-05-20T16:20:37.609 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:37.609+0000 7f1b2318f740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5570ff43e2f0 0x5570ff810cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:20:37.610 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:37.609+0000 7f1b2318f740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5570ff4464d0 0x5570ff811210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:20:37.610 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:37.609+0000 7f1b2318f740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5570ff4671b0 0x5570ff818170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:20:37.610 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:37.609+0000 7f1b2318f740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x5570ff80eec0 con 0x5570ff4671b0 2024-05-20T16:20:37.610 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:37.609+0000 7f1b2318f740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x5570ff726790 con 0x5570ff43e2f0 2024-05-20T16:20:37.610 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:37.609+0000 7f1b2318f740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5570ff80f040 con 0x5570ff4464d0 2024-05-20T16:20:37.610 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:37.609+0000 7f1b2318f740 10 client.0 _ll_register_callbacks cb 0x5570ff761a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T16:20:37.610 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:37.609+0000 7f1b19fa9640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5570ff43e2f0 0x5570ff810cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:20:37.610 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:37.609+0000 7f1b1a7aa640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5570ff4671b0 0x5570ff818170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:20:37.610 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:37.609+0000 7f1b19fa9640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5570ff43e2f0 0x5570ff810cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.82:52450/0 (socket says 172.21.15.82:52450) 2024-05-20T16:20:37.610 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:37.610+0000 7f1b19fa9640 1 -- 172.21.15.82:0/657053434 learned_addr learned my addr 172.21.15.82:0/657053434 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:20:37.610 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:37.610+0000 7f1b197a8640 1 --2- 172.21.15.82:0/657053434 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5570ff4464d0 0x5570ff811210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:20:37.611 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:37.610+0000 7f1b197a8640 1 -- 172.21.15.82:0/657053434 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5570ff43e2f0 msgr2=0x5570ff810cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:20:37.611 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:37.610+0000 7f1b197a8640 1 --2- 172.21.15.82:0/657053434 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5570ff43e2f0 0x5570ff810cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:20:37.612 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:37.610+0000 7f1b197a8640 1 -- 172.21.15.82:0/657053434 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5570ff4671b0 msgr2=0x5570ff818170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:20:37.612 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:37.610+0000 7f1b197a8640 1 --2- 172.21.15.82:0/657053434 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5570ff4671b0 0x5570ff818170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:20:37.612 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:37.610+0000 7f1b197a8640 1 -- 172.21.15.82:0/657053434 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5570ff47f210 con 0x5570ff4464d0 2024-05-20T16:20:37.612 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:37.611+0000 7f1b197a8640 1 --2- 172.21.15.82:0/657053434 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5570ff4464d0 0x5570ff811210 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f1b100985b0 tx=0x7f1b1009bca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=df539b6956b55e90 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:20:37.612 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:37.611+0000 7f1afd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-05-20T16:20:37.612 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:37.612+0000 7f1afd7fa640 1 -- 172.21.15.82:0/657053434 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b100a8070 con 0x5570ff4464d0 2024-05-20T16:20:37.612 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:37.612+0000 7f1afd7fa640 1 -- 172.21.15.82:0/657053434 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9150+0+0 (secure 0 0 0) 0x7f1b100a45d0 con 0x5570ff4464d0 2024-05-20T16:20:37.614 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:37.612+0000 7f1b2318f740 1 -- 172.21.15.82:0/657053434 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1b100a4850 con 0x5570ff4464d0 2024-05-20T16:20:37.614 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:37.614+0000 7f1afd7fa640 1 -- 172.21.15.82:0/657053434 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f1b1009ff90 con 0x5570ff4464d0 2024-05-20T16:20:37.614 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:37.614+0000 7f1b2318f740 10 client.25303 fetch_fsmap learned FSMap version 27 2024-05-20T16:20:37.614 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:37.614+0000 7f1b2318f740 1 -- 172.21.15.82:0/657053434 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5570ff54b620 con 0x5570ff4464d0 2024-05-20T16:20:37.617 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:37.617+0000 7f1afd7fa640 1 -- 172.21.15.82:0/657053434 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1b100a4850 con 0x5570ff4464d0 2024-05-20T16:20:37.617 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:37.617+0000 7f1afd7fa640 1 -- 172.21.15.82:0/657053434 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f1b10097930 con 0x5570ff4464d0 2024-05-20T16:20:37.618 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:37.617+0000 7f1b2318f740 10 client.25303 fetch_fsmap finished waiting for FSMap version 27 2024-05-20T16:20:37.618 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:37.617+0000 7f1b2318f740 10 client.25303 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T16:20:37.618 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:37.617+0000 7f1b2318f740 20 client.25303 populate_metadata read hostname 'smithi082' 2024-05-20T16:20:37.618 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:37.618+0000 7f1b2318f740 1 --2- 172.21.15.82:0/657053434 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x5570ff425d80 0x5570ff81e230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:20:37.618 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:37.618+0000 7f1b2318f740 4 client.25303 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T16:20:37.618 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:37.618+0000 7f1b2318f740 1 -- 172.21.15.82:0/657053434 --> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5570ff778a70 con 0x5570ff425d80 2024-05-20T16:20:37.618 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:37.618+0000 7f1b19fa9640 1 --2- 172.21.15.82:0/657053434 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x5570ff425d80 0x5570ff81e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:20:37.625 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:37.625+0000 7f1b19fa9640 1 --2- 172.21.15.82:0/657053434 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x5570ff425d80 0x5570ff81e230 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f8fcfd486ec121fe server_cookie=e0447daee020581e in_seq=0 out_seq=0 2024-05-20T16:20:37.625 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:37.625+0000 7f1afd7fa640 10 client.25303 ms_handle_connect on v2:172.21.15.160:6838/2765554482 2024-05-20T16:20:37.627 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:37.628+0000 7f1afd7fa640 1 -- 172.21.15.82:0/657053434 <== mds.0 v2:172.21.15.160:6838/2765554482 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5570ff778a70 con 0x5570ff425d80 2024-05-20T16:20:37.628 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:37.628+0000 7f1afd7fa640 10 client.25303 handle_command_reply: tid=0 2024-05-20T16:20:37.628 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:37.628+0000 7f1b2318f740 1 client.25303 shutdown 2024-05-20T16:20:37.630 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:37.630+0000 7f1b2318f740 1 -- 172.21.15.82:0/657053434 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5570ff4464d0 msgr2=0x5570ff811210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:20:37.631 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:37.631+0000 7f1b2318f740 1 --2- 172.21.15.82:0/657053434 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5570ff4464d0 0x5570ff811210 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f1b100985b0 tx=0x7f1b1009bca0 comp rx=0 tx=0).stop 2024-05-20T16:20:37.631 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:37.631+0000 7f1b2318f740 1 -- 172.21.15.82:0/657053434 shutdown_connections 2024-05-20T16:20:37.631 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:37.631+0000 7f1b2318f740 1 --2- 172.21.15.82:0/657053434 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x5570ff425d80 0x5570ff81e230 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:20:37.631 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:37.631+0000 7f1afd7fa640 0 client.25303 ms_handle_reset on v2:172.21.15.160:6838/2765554482 2024-05-20T16:20:37.631 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:37.632+0000 7f1b2318f740 1 --2- 172.21.15.82:0/657053434 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5570ff4671b0 0x5570ff818170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:20:37.632 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:37.632+0000 7f1b2318f740 1 --2- 172.21.15.82:0/657053434 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5570ff4464d0 0x5570ff811210 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:20:37.632 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:37.632+0000 7f1b2318f740 1 --2- 172.21.15.82:0/657053434 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5570ff43e2f0 0x5570ff810cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:20:37.632 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:37.632+0000 7f1b2318f740 1 -- 172.21.15.82:0/657053434 >> 172.21.15.82:0/657053434 conn(0x5570ff45b300 msgr2=0x5570ff8059a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:20:37.632 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:37.632+0000 7f1b2318f740 1 -- 172.21.15.82:0/657053434 shutdown_connections 2024-05-20T16:20:37.632 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:37.633+0000 7f1b2318f740 1 -- 172.21.15.82:0/657053434 wait complete. 2024-05-20T16:20:37.633 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:37.633+0000 7f1b2318f740 20 client.25303 trim_cache size 0 max 16384 2024-05-20T16:20:37.663 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:37.663+0000 7f1b2318f740 1 Processor -- start 2024-05-20T16:20:37.663 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:37.664+0000 7f1b2318f740 1 -- start start 2024-05-20T16:20:37.664 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:37.664+0000 7f1b2318f740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5570ff425d80 0x5570ff823830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:20:37.664 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:37.664+0000 7f1b2318f740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5570ff43e2f0 0x5570ff818090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:20:37.664 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:37.664+0000 7f1b2318f740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5570ff4464d0 0x5570ff823d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:20:37.665 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:37.665+0000 7f1b19fa9640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5570ff425d80 0x5570ff823830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:20:37.665 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:37.665+0000 7f1b19fa9640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5570ff425d80 0x5570ff823830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.82:56074/0 (socket says 172.21.15.82:56074) 2024-05-20T16:20:37.665 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:37.665+0000 7f1b197a8640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5570ff43e2f0 0x5570ff818090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:20:37.665 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:37.665+0000 7f1b197a8640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5570ff43e2f0 0x5570ff818090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.82:52462/0 (socket says 172.21.15.82:52462) 2024-05-20T16:20:37.665 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:37.665+0000 7f1b19fa9640 1 -- 172.21.15.82:0/2546297903 learned_addr learned my addr 172.21.15.82:0/2546297903 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:20:37.665 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:37.665+0000 7f1b2318f740 1 -- 172.21.15.82:0/2546297903 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f1af8012540 con 0x5570ff4464d0 2024-05-20T16:20:37.665 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:37.665+0000 7f1b2318f740 1 -- 172.21.15.82:0/2546297903 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f1af800fe10 con 0x5570ff43e2f0 2024-05-20T16:20:37.665 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:37.665+0000 7f1b2318f740 1 -- 172.21.15.82:0/2546297903 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f1af800dc20 con 0x5570ff425d80 2024-05-20T16:20:37.666 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:37.666+0000 7f1b1a7aa640 1 --2- 172.21.15.82:0/2546297903 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5570ff4464d0 0x5570ff823d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:20:37.666 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:37.666+0000 7f1b197a8640 1 -- 172.21.15.82:0/2546297903 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5570ff425d80 msgr2=0x5570ff823830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:20:37.666 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:37.666+0000 7f1b197a8640 1 --2- 172.21.15.82:0/2546297903 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5570ff425d80 0x5570ff823830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:20:37.666 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:37.666+0000 7f1b197a8640 1 -- 172.21.15.82:0/2546297903 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5570ff4464d0 msgr2=0x5570ff823d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:20:37.666 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:37.666+0000 7f1b197a8640 1 --2- 172.21.15.82:0/2546297903 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5570ff4464d0 0x5570ff823d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:20:37.666 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:37.666+0000 7f1b197a8640 1 -- 172.21.15.82:0/2546297903 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5570ff80e8a0 con 0x5570ff43e2f0 2024-05-20T16:20:37.666 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:37.667+0000 7f1b197a8640 1 --2- 172.21.15.82:0/2546297903 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5570ff43e2f0 0x5570ff818090 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f1b1000c6a0 tx=0x7f1b1009b540 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7b9b5f5b3cb729fc server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:20:37.669 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:37.669+0000 7f1aff7fe640 1 -- 172.21.15.82:0/2546297903 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b1009ff90 con 0x5570ff43e2f0 2024-05-20T16:20:37.669 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:37.669+0000 7f1aff7fe640 1 -- 172.21.15.82:0/2546297903 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1b1009c2e0 con 0x5570ff43e2f0 2024-05-20T16:20:37.669 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:37.669+0000 7f1aff7fe640 1 -- 172.21.15.82:0/2546297903 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b10005000 con 0x5570ff43e2f0 2024-05-20T16:20:37.670 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:37.671+0000 7f1b2318f740 1 -- 172.21.15.82:0/2546297903 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5570ff43e2f0 msgr2=0x5570ff818090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:20:37.670 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:37.671+0000 7f1b2318f740 1 --2- 172.21.15.82:0/2546297903 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5570ff43e2f0 0x5570ff818090 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f1b1000c6a0 tx=0x7f1b1009b540 comp rx=0 tx=0).stop 2024-05-20T16:20:37.672 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:37.672+0000 7f1b2318f740 1 -- 172.21.15.82:0/2546297903 shutdown_connections 2024-05-20T16:20:37.673 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:37.673+0000 7f1b2318f740 1 --2- 172.21.15.82:0/2546297903 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5570ff4464d0 0x5570ff823d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:20:37.673 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:37.673+0000 7f1b2318f740 1 --2- 172.21.15.82:0/2546297903 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5570ff43e2f0 0x5570ff818090 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:20:37.673 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:37.673+0000 7f1b2318f740 1 --2- 172.21.15.82:0/2546297903 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5570ff425d80 0x5570ff823830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:20:37.673 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:37.673+0000 7f1b2318f740 1 -- 172.21.15.82:0/2546297903 >> 172.21.15.82:0/2546297903 conn(0x5570ff45b300 msgr2=0x5570ff80b140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:20:37.674 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:37.674+0000 7f1b2318f740 1 -- 172.21.15.82:0/2546297903 shutdown_connections 2024-05-20T16:20:37.674 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:37.674+0000 7f1b2318f740 1 -- 172.21.15.82:0/2546297903 wait complete. 2024-05-20T16:20:37.674 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:37.675+0000 7f1b2318f740 1 -- start start 2024-05-20T16:20:37.675 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:37.675+0000 7f1b2318f740 1 Processor -- start 2024-05-20T16:20:37.675 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:37.675+0000 7f1b2318f740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5570ff425d80 0x5570ff817f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:20:37.675 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:37.675+0000 7f1b2318f740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5570ff43e2f0 0x5570ff814af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:20:37.675 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:37.675+0000 7f1b2318f740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5570ff4464d0 0x5570ff815070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:20:37.675 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:37.675+0000 7f1b2318f740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f1af800e2d0 con 0x5570ff43e2f0 2024-05-20T16:20:37.675 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:37.675+0000 7f1b2318f740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f1af800f740 con 0x5570ff4464d0 2024-05-20T16:20:37.676 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:37.675+0000 7f1b2318f740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f1af800e9a0 con 0x5570ff425d80 2024-05-20T16:20:37.676 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:37.676+0000 7f1b2318f740 10 client.0 _ll_register_callbacks cb 0x7f1b0c034070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T16:20:37.676 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:37.676+0000 7f1b19fa9640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5570ff425d80 0x5570ff817f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:20:37.676 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:37.676+0000 7f1b1a7aa640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5570ff4464d0 0x5570ff815070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:20:37.676 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:37.676+0000 7f1b1a7aa640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5570ff4464d0 0x5570ff815070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.82:52470/0 (socket says 172.21.15.82:52470) 2024-05-20T16:20:37.676 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:37.676+0000 7f1b1a7aa640 1 -- 172.21.15.82:0/3961676215 learned_addr learned my addr 172.21.15.82:0/3961676215 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:20:37.677 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:37.677+0000 7f1b1a7aa640 1 -- 172.21.15.82:0/3961676215 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5570ff425d80 msgr2=0x5570ff817f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:20:37.677 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:37.677+0000 7f1b1a7aa640 1 --2- 172.21.15.82:0/3961676215 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5570ff425d80 0x5570ff817f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:20:37.677 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:37.677+0000 7f1b1a7aa640 1 -- 172.21.15.82:0/3961676215 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5570ff43e2f0 msgr2=0x5570ff814af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:20:37.677 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:37.677+0000 7f1b1a7aa640 1 --2- 172.21.15.82:0/3961676215 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5570ff43e2f0 0x5570ff814af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:20:37.678 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:37.677+0000 7f1b1a7aa640 1 -- 172.21.15.82:0/3961676215 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5570ff8843d0 con 0x5570ff4464d0 2024-05-20T16:20:37.679 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:37.680+0000 7f1b1a7aa640 1 --2- 172.21.15.82:0/3961676215 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5570ff4464d0 0x5570ff815070 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f1b0400bfa0 tx=0x7f1b04012820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3b25ba66e74b56be server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:20:37.683 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:37.680+0000 7f1afd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-20T16:20:37.683 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:37.682+0000 7f1afd7fa640 1 -- 172.21.15.82:0/3961676215 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b04005020 con 0x5570ff4464d0 2024-05-20T16:20:37.683 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:37.682+0000 7f1afd7fa640 1 -- 172.21.15.82:0/3961676215 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9150+0+0 (secure 0 0 0) 0x7f1b040051c0 con 0x5570ff4464d0 2024-05-20T16:20:37.683 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:37.684+0000 7f1b2318f740 1 -- 172.21.15.82:0/3961676215 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1b1009c2e0 con 0x5570ff4464d0 2024-05-20T16:20:37.687 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:37.687+0000 7f1afd7fa640 1 -- 172.21.15.82:0/3961676215 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f1b040053c0 con 0x5570ff4464d0 2024-05-20T16:20:37.689 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:37.689+0000 7f1b2318f740 10 client.25295 fetch_fsmap learned FSMap version 27 2024-05-20T16:20:37.689 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:37.689+0000 7f1b2318f740 1 -- 172.21.15.82:0/3961676215 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5570ff885660 con 0x5570ff4464d0 2024-05-20T16:20:37.690 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:37.690+0000 7f1afd7fa640 1 -- 172.21.15.82:0/3961676215 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1b04005560 con 0x5570ff4464d0 2024-05-20T16:20:37.690 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:37.691+0000 7f1afd7fa640 1 -- 172.21.15.82:0/3961676215 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f1b04010c00 con 0x5570ff4464d0 2024-05-20T16:20:37.692 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:37.692+0000 7f1b2318f740 10 client.25295 fetch_fsmap finished waiting for FSMap version 27 2024-05-20T16:20:37.693 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:37.692+0000 7f1b2318f740 10 client.25295 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T16:20:37.693 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:37.692+0000 7f1b2318f740 20 client.25295 populate_metadata read hostname 'smithi082' 2024-05-20T16:20:37.693 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:37.692+0000 7f1b2318f740 1 --2- 172.21.15.82:0/3961676215 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x5570ff4671b0 0x5570ff8880b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:20:37.693 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:37.693+0000 7f1b2318f740 4 client.25295 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-20T16:20:37.693 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:37.693+0000 7f1b2318f740 1 -- 172.21.15.82:0/3961676215 --> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f1b1009c2e0 con 0x5570ff4671b0 2024-05-20T16:20:37.701 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:37.700+0000 7f1b19fa9640 1 --2- 172.21.15.82:0/3961676215 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x5570ff4671b0 0x5570ff8880b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:20:37.733 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:37.728+0000 7f1afd7fa640 10 client.25295 ms_handle_connect on v2:172.21.15.160:6838/2765554482 2024-05-20T16:20:37.733 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:37.728+0000 7f1b19fa9640 1 --2- 172.21.15.82:0/3961676215 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x5570ff4671b0 0x5570ff8880b0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=14669051617fc2a3 server_cookie=63a4ac9d11efead6 in_seq=0 out_seq=0 2024-05-20T16:20:39.062 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:20:38 smithi160 ceph-mon[36347]: pgmap v562: 129 pgs: 129 active+clean; 10 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 631 MiB/s wr, 1.44k op/s 2024-05-20T16:20:39.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:20:38 smithi082 ceph-mon[32238]: pgmap v562: 129 pgs: 129 active+clean; 10 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 631 MiB/s wr, 1.44k op/s 2024-05-20T16:20:39.135 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:20:38 smithi184 ceph-mon[36209]: pgmap v562: 129 pgs: 129 active+clean; 10 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 631 MiB/s wr, 1.44k op/s 2024-05-20T16:20:40.062 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:20:39 smithi160 ceph-mon[36347]: pgmap v563: 129 pgs: 129 active+clean; 10 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 827 MiB/s wr, 1.78k op/s 2024-05-20T16:20:40.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:20:39 smithi082 ceph-mon[32238]: pgmap v563: 129 pgs: 129 active+clean; 10 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 827 MiB/s wr, 1.78k op/s 2024-05-20T16:20:40.145 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:20:39 smithi184 ceph-mon[36209]: pgmap v563: 129 pgs: 129 active+clean; 10 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 827 MiB/s wr, 1.78k op/s 2024-05-20T16:20:40.917 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:40.917+0000 7f1afd7fa640 1 -- 172.21.15.82:0/3961676215 <== mds.0 v2:172.21.15.160:6838/2765554482 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f1b1009c2e0 con 0x5570ff4671b0 2024-05-20T16:20:40.918 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:40.917+0000 7f1afd7fa640 10 client.25295 handle_command_reply: tid=0 2024-05-20T16:20:40.918 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:40.917+0000 7f1b2318f740 1 client.25295 shutdown 2024-05-20T16:20:40.918 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:40.918+0000 7f1b2318f740 1 -- 172.21.15.82:0/3961676215 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5570ff4464d0 msgr2=0x5570ff815070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:20:40.918 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:40.918+0000 7f1b2318f740 1 --2- 172.21.15.82:0/3961676215 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5570ff4464d0 0x5570ff815070 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f1b0400bfa0 tx=0x7f1b04012820 comp rx=0 tx=0).stop 2024-05-20T16:20:40.918 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:40.918+0000 7f1b2318f740 1 -- 172.21.15.82:0/3961676215 shutdown_connections 2024-05-20T16:20:40.918 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:40.918+0000 7f1b2318f740 1 --2- 172.21.15.82:0/3961676215 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x5570ff4671b0 0x5570ff8880b0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:20:40.918 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:40.918+0000 7f1b2318f740 1 --2- 172.21.15.82:0/3961676215 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5570ff43e2f0 0x5570ff814af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:20:40.918 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:40.918+0000 7f1b2318f740 1 --2- 172.21.15.82:0/3961676215 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5570ff4464d0 0x5570ff815070 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:20:40.918 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:40.918+0000 7f1b2318f740 1 --2- 172.21.15.82:0/3961676215 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5570ff425d80 0x5570ff817f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:20:40.918 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:40.918+0000 7f1b2318f740 1 -- 172.21.15.82:0/3961676215 >> 172.21.15.82:0/3961676215 conn(0x5570ff45b300 msgr2=0x5570ff805890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:20:40.918 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:40.918+0000 7f1afd7fa640 0 client.25295 ms_handle_reset on v2:172.21.15.160:6838/2765554482 2024-05-20T16:20:40.919 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:40.918+0000 7f1b2318f740 1 -- 172.21.15.82:0/3961676215 shutdown_connections 2024-05-20T16:20:40.919 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:40.918+0000 7f1b2318f740 1 -- 172.21.15.82:0/3961676215 wait complete. 2024-05-20T16:20:40.919 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:40.918+0000 7f1b2318f740 20 client.25295 trim_cache size 0 max 16384 2024-05-20T16:20:40.920 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:{ 2024-05-20T16:20:40.920 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "message": "", 2024-05-20T16:20:40.920 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "return_code": 0 2024-05-20T16:20:40.920 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:} 2024-05-20T16:20:40.920 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:40.920+0000 7f1b1afab640 1 -- 172.21.15.82:0/1787842243 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f1b000a2d50 msgr2=0x7f1b000a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:20:40.920 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:40.920+0000 7f1b1afab640 1 --2- 172.21.15.82:0/1787842243 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f1b000a2d50 0x7f1b000a5260 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f1b0c004af0 tx=0x7f1b0c000f40 comp rx=0 tx=0).stop 2024-05-20T16:20:40.920 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:40.920+0000 7f1b1afab640 1 -- 172.21.15.82:0/1787842243 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1b14103b30 msgr2=0x7f1b14070f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:20:40.921 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:40.920+0000 7f1b1afab640 1 --2- 172.21.15.82:0/1787842243 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1b14103b30 0x7f1b14070f90 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f1b1000e9c0 tx=0x7f1b1000ee80 comp rx=0 tx=0).stop 2024-05-20T16:20:40.925 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:40.924+0000 7f1b1afab640 1 -- 172.21.15.82:0/1787842243 shutdown_connections 2024-05-20T16:20:40.925 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:40.924+0000 7f1b1afab640 1 --2- 172.21.15.82:0/1787842243 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f1b000a2d50 0x7f1b000a5260 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:20:40.926 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:40.924+0000 7f1b1afab640 1 --2- 172.21.15.82:0/1787842243 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1b14103b30 0x7f1b14070f90 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:20:40.926 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:40.924+0000 7f1b1afab640 1 --2- 172.21.15.82:0/1787842243 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1b14106490 0x7f1b14071ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:20:40.926 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:40.924+0000 7f1b1afab640 1 --2- 172.21.15.82:0/1787842243 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1b141011d0 0x7f1b14070a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:20:40.926 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:40.924+0000 7f1b1afab640 1 -- 172.21.15.82:0/1787842243 >> 172.21.15.82:0/1787842243 conn(0x7f1b140fc9e0 msgr2=0x7f1b14102010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:20:40.926 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:40.924+0000 7f1b1afab640 1 -- 172.21.15.82:0/1787842243 shutdown_connections 2024-05-20T16:20:40.927 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:40.924+0000 7f1b1afab640 1 -- 172.21.15.82:0/1787842243 wait complete. 2024-05-20T16:20:43.062 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:20:42 smithi160 ceph-mon[36347]: pgmap v564: 129 pgs: 129 active+clean; 10 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 676 MiB/s wr, 1.51k op/s 2024-05-20T16:20:43.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:20:42 smithi082 ceph-mon[32238]: pgmap v564: 129 pgs: 129 active+clean; 10 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 676 MiB/s wr, 1.51k op/s 2024-05-20T16:20:43.135 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:20:42 smithi184 ceph-mon[36209]: pgmap v564: 129 pgs: 129 active+clean; 10 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 676 MiB/s wr, 1.51k op/s 2024-05-20T16:20:45.062 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:20:44 smithi160 ceph-mon[36347]: pgmap v565: 129 pgs: 129 active+clean; 10 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 742 MiB/s wr, 1.68k op/s 2024-05-20T16:20:45.111 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:20:44 smithi082 ceph-mon[32238]: pgmap v565: 129 pgs: 129 active+clean; 10 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 742 MiB/s wr, 1.68k op/s 2024-05-20T16:20:45.135 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:20:44 smithi184 ceph-mon[36209]: pgmap v565: 129 pgs: 129 active+clean; 10 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 742 MiB/s wr, 1.68k op/s 2024-05-20T16:20:46.062 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:20:45 smithi160 ceph-mon[36347]: pgmap v566: 129 pgs: 129 active+clean; 10 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 803 MiB/s wr, 1.78k op/s 2024-05-20T16:20:46.110 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:20:45 smithi082 ceph-mon[32238]: pgmap v566: 129 pgs: 129 active+clean; 10 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 803 MiB/s wr, 1.78k op/s 2024-05-20T16:20:46.135 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:20:45 smithi184 ceph-mon[36209]: pgmap v566: 129 pgs: 129 active+clean; 10 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 803 MiB/s wr, 1.78k op/s 2024-05-20T16:20:49.062 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:20:48 smithi160 ceph-mon[36347]: pgmap v567: 129 pgs: 129 active+clean; 10 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 628 MiB/s wr, 1.31k op/s 2024-05-20T16:20:49.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:20:48 smithi082 ceph-mon[32238]: pgmap v567: 129 pgs: 129 active+clean; 10 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 628 MiB/s wr, 1.31k op/s 2024-05-20T16:20:49.135 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:20:48 smithi184 ceph-mon[36209]: pgmap v567: 129 pgs: 129 active+clean; 10 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 628 MiB/s wr, 1.31k op/s 2024-05-20T16:20:51.062 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:20:50 smithi160 ceph-mon[36347]: pgmap v568: 129 pgs: 129 active+clean; 10 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 847 MiB/s wr, 1.68k op/s 2024-05-20T16:20:51.111 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:20:50 smithi082 ceph-mon[32238]: pgmap v568: 129 pgs: 129 active+clean; 10 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 847 MiB/s wr, 1.68k op/s 2024-05-20T16:20:51.135 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:20:50 smithi184 ceph-mon[36209]: pgmap v568: 129 pgs: 129 active+clean; 10 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 847 MiB/s wr, 1.68k op/s 2024-05-20T16:20:52.062 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:20:51 smithi160 ceph-mon[36347]: pgmap v569: 129 pgs: 129 active+clean; 10 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 680 MiB/s wr, 1.36k op/s 2024-05-20T16:20:52.112 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:20:51 smithi082 ceph-mon[32238]: pgmap v569: 129 pgs: 129 active+clean; 10 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 680 MiB/s wr, 1.36k op/s 2024-05-20T16:20:52.136 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:20:51 smithi184 ceph-mon[36209]: pgmap v569: 129 pgs: 129 active+clean; 10 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 680 MiB/s wr, 1.36k op/s 2024-05-20T16:20:53.062 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:20:52 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T16:20:53.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:20:52 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T16:20:53.135 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:20:52 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T16:20:54.062 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:20:53 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-20T16:20:54.062 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:20:53 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-20T16:20:54.062 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:20:53 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-20T16:20:54.062 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:20:53 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-20T16:20:54.062 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:20:53 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-20T16:20:54.062 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:20:53 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-20T16:20:54.062 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:20:53 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-20T16:20:54.062 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:20:53 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-20T16:20:54.063 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:20:53 smithi160 ceph-mon[36347]: Adjusting osd_memory_target on smithi082 to 176.1M 2024-05-20T16:20:54.063 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:20:53 smithi160 ceph-mon[36347]: Adjusting osd_memory_target on smithi160 to 176.2M 2024-05-20T16:20:54.063 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:20:53 smithi160 ceph-mon[36347]: Unable to set osd_memory_target on smithi082 to 184745164: error parsing value: Value '184745164' is below minimum 939524096 2024-05-20T16:20:54.063 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:20:53 smithi160 ceph-mon[36347]: Unable to set osd_memory_target on smithi160 to 184788172: error parsing value: Value '184788172' is below minimum 939524096 2024-05-20T16:20:54.063 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:20:53 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:20:54.063 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:20:53 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T16:20:54.063 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:20:53 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:20:54.063 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:20:53 smithi160 ceph-mon[36347]: pgmap v570: 129 pgs: 129 active+clean; 10 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 707 MiB/s wr, 1.39k op/s 2024-05-20T16:20:54.111 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:20:53 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-20T16:20:54.111 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:20:53 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-20T16:20:54.112 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:20:53 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-20T16:20:54.112 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:20:53 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-20T16:20:54.112 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:20:53 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-20T16:20:54.112 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:20:53 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-20T16:20:54.112 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:20:53 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-20T16:20:54.112 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:20:53 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-20T16:20:54.112 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:20:53 smithi082 ceph-mon[32238]: Adjusting osd_memory_target on smithi082 to 176.1M 2024-05-20T16:20:54.112 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:20:53 smithi082 ceph-mon[32238]: Adjusting osd_memory_target on smithi160 to 176.2M 2024-05-20T16:20:54.112 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:20:53 smithi082 ceph-mon[32238]: Unable to set osd_memory_target on smithi082 to 184745164: error parsing value: Value '184745164' is below minimum 939524096 2024-05-20T16:20:54.112 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:20:53 smithi082 ceph-mon[32238]: Unable to set osd_memory_target on smithi160 to 184788172: error parsing value: Value '184788172' is below minimum 939524096 2024-05-20T16:20:54.112 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:20:53 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:20:54.112 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:20:53 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T16:20:54.112 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:20:53 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:20:54.112 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:20:53 smithi082 ceph-mon[32238]: pgmap v570: 129 pgs: 129 active+clean; 10 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 707 MiB/s wr, 1.39k op/s 2024-05-20T16:20:54.136 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:20:53 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-20T16:20:54.136 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:20:53 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-20T16:20:54.136 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:20:53 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-20T16:20:54.136 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:20:53 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-20T16:20:54.136 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:20:53 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-20T16:20:54.136 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:20:53 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-20T16:20:54.136 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:20:53 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-20T16:20:54.136 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:20:53 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-20T16:20:54.137 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:20:53 smithi184 ceph-mon[36209]: Adjusting osd_memory_target on smithi082 to 176.1M 2024-05-20T16:20:54.137 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:20:53 smithi184 ceph-mon[36209]: Adjusting osd_memory_target on smithi160 to 176.2M 2024-05-20T16:20:54.137 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:20:53 smithi184 ceph-mon[36209]: Unable to set osd_memory_target on smithi082 to 184745164: error parsing value: Value '184745164' is below minimum 939524096 2024-05-20T16:20:54.137 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:20:53 smithi184 ceph-mon[36209]: Unable to set osd_memory_target on smithi160 to 184788172: error parsing value: Value '184788172' is below minimum 939524096 2024-05-20T16:20:54.137 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:20:53 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:20:54.137 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:20:53 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T16:20:54.137 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:20:53 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:20:54.137 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:20:53 smithi184 ceph-mon[36209]: pgmap v570: 129 pgs: 129 active+clean; 10 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 707 MiB/s wr, 1.39k op/s 2024-05-20T16:20:54.173 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:54.173+0000 7fc6337b7640 1 -- 172.21.15.82:0/1477845316 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc62c10eea0 msgr2=0x7fc62c10f2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:20:54.173 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:54.173+0000 7fc6337b7640 1 --2- 172.21.15.82:0/1477845316 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc62c10eea0 0x7fc62c10f2b0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fc624009ca0 tx=0x7fc62402a8f0 comp rx=0 tx=0).stop 2024-05-20T16:20:54.174 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:54.174+0000 7fc6337b7640 1 -- 172.21.15.82:0/1477845316 shutdown_connections 2024-05-20T16:20:54.174 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:54.174+0000 7fc6337b7640 1 --2- 172.21.15.82:0/1477845316 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc62c10eea0 0x7fc62c10f2b0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:20:54.174 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:54.174+0000 7fc6337b7640 1 --2- 172.21.15.82:0/1477845316 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc62c10a180 0x7fc62c10a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:20:54.174 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:54.174+0000 7fc6337b7640 1 --2- 172.21.15.82:0/1477845316 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc62c109300 0x7fc62c109c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:20:54.174 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:54.174+0000 7fc6337b7640 1 -- 172.21.15.82:0/1477845316 >> 172.21.15.82:0/1477845316 conn(0x7fc62c06f340 msgr2=0x7fc62c0717d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:20:54.183 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:54.184+0000 7fc6337b7640 1 -- 172.21.15.82:0/1477845316 shutdown_connections 2024-05-20T16:20:54.184 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:54.184+0000 7fc6337b7640 1 -- 172.21.15.82:0/1477845316 wait complete. 2024-05-20T16:20:54.185 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:54.185+0000 7fc6337b7640 1 Processor -- start 2024-05-20T16:20:54.189 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:54.187+0000 7fc6337b7640 1 -- start start 2024-05-20T16:20:54.189 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:54.188+0000 7fc6337b7640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc62c109300 0x7fc62c1a8050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:20:54.189 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:54.188+0000 7fc6337b7640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc62c10a180 0x7fc62c1a8590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:20:54.190 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:54.188+0000 7fc6337b7640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc62c10eea0 0x7fc62c1af520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:20:54.190 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:54.188+0000 7fc6337b7640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fc62c11d470 con 0x7fc62c109300 2024-05-20T16:20:54.190 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:54.188+0000 7fc6337b7640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fc62c11d2f0 con 0x7fc62c10a180 2024-05-20T16:20:54.190 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:54.188+0000 7fc6337b7640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fc62c11d5f0 con 0x7fc62c10eea0 2024-05-20T16:20:54.190 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:54.191+0000 7fc631d54640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc62c10eea0 0x7fc62c1af520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:20:54.193 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:54.192+0000 7fc631553640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc62c109300 0x7fc62c1a8050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:20:54.193 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:54.192+0000 7fc631553640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc62c109300 0x7fc62c1a8050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:37822/0 (socket says 172.21.15.82:37822) 2024-05-20T16:20:54.193 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:54.192+0000 7fc631553640 1 -- 172.21.15.82:0/751947411 learned_addr learned my addr 172.21.15.82:0/751947411 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:20:54.198 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:54.198+0000 7fc631553640 1 -- 172.21.15.82:0/751947411 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc62c10eea0 msgr2=0x7fc62c1af520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:20:54.198 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:54.198+0000 7fc631553640 1 --2- 172.21.15.82:0/751947411 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc62c10eea0 0x7fc62c1af520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:20:54.199 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:54.198+0000 7fc631553640 1 -- 172.21.15.82:0/751947411 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc62c10a180 msgr2=0x7fc62c1a8590 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-20T16:20:54.199 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:54.198+0000 7fc631553640 1 --2- 172.21.15.82:0/751947411 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc62c10a180 0x7fc62c1a8590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:20:54.199 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:54.198+0000 7fc631553640 1 -- 172.21.15.82:0/751947411 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc62c1afb90 con 0x7fc62c109300 2024-05-20T16:20:54.205 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:54.205+0000 7fc631553640 1 --2- 172.21.15.82:0/751947411 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc62c109300 0x7fc62c1a8050 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fc61c00b450 tx=0x7fc61c00b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:20:54.210 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:54.210+0000 7fc6227fc640 1 -- 172.21.15.82:0/751947411 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc61c015020 con 0x7fc62c109300 2024-05-20T16:20:54.213 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:54.210+0000 7fc6337b7640 1 -- 172.21.15.82:0/751947411 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc62c1afe80 con 0x7fc62c109300 2024-05-20T16:20:54.213 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:54.210+0000 7fc6337b7640 1 -- 172.21.15.82:0/751947411 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc62c1b0360 con 0x7fc62c109300 2024-05-20T16:20:54.217 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:54.216+0000 7fc63a65c740 1 Processor -- start 2024-05-20T16:20:54.217 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:54.214+0000 7fc6227fc640 1 -- 172.21.15.82:0/751947411 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc61c011d70 con 0x7fc62c109300 2024-05-20T16:20:54.217 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:54.216+0000 7fc63a65c740 1 -- start start 2024-05-20T16:20:54.220 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:54.220+0000 7fc63a65c740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55732c2221b0 0x55732c5c2050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:20:54.220 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:54.220+0000 7fc63a65c740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55732c1f92f0 0x55732c5c25d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:20:54.220 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:54.220+0000 7fc63a65c740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55732c2014d0 0x55732c5c94d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:20:54.220 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:54.220+0000 7fc63a65c740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55732c526340 con 0x55732c2014d0 2024-05-20T16:20:54.220 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:54.220+0000 7fc63a65c740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x55732c4ce290 con 0x55732c1f92f0 2024-05-20T16:20:54.220 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:54.220+0000 7fc63a65c740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55732c522300 con 0x55732c2221b0 2024-05-20T16:20:54.221 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:54.220+0000 7fc6227fc640 1 -- 172.21.15.82:0/751947411 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc61c00f340 con 0x7fc62c109300 2024-05-20T16:20:54.221 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:54.220+0000 7fc6227fc640 1 -- 172.21.15.82:0/751947411 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fc61c018020 con 0x7fc62c109300 2024-05-20T16:20:54.223 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:54.223+0000 7fc631553640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55732c1f92f0 0x55732c5c25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:20:54.223 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:54.223+0000 7fc631553640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55732c1f92f0 0x55732c5c25d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.82:57654/0 (socket says 172.21.15.82:57654) 2024-05-20T16:20:54.223 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:54.223+0000 7fc631553640 1 -- 172.21.15.82:0/3339898122 learned_addr learned my addr 172.21.15.82:0/3339898122 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:20:54.224 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:54.224+0000 7fc6227fc640 1 --2- 172.21.15.82:0/751947411 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fc6180a2d00 0x7fc6180a5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:20:54.224 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:54.224+0000 7fc631d54640 1 --2- 172.21.15.82:0/3339898122 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55732c2221b0 0x55732c5c2050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:20:54.224 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:54.224+0000 7fc6227fc640 1 -- 172.21.15.82:0/751947411 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9150+0+0 (secure 0 0 0) 0x7fc61c0cfe70 con 0x7fc62c109300 2024-05-20T16:20:54.227 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:54.226+0000 7fc631d54640 1 -- 172.21.15.82:0/3339898122 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55732c1f92f0 msgr2=0x55732c5c25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:20:54.227 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:54.226+0000 7fc631d54640 1 --2- 172.21.15.82:0/3339898122 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55732c1f92f0 0x55732c5c25d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:20:54.227 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:54.226+0000 7fc631d54640 1 -- 172.21.15.82:0/3339898122 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55732c2014d0 msgr2=0x55732c5c94d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:20:54.227 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:54.226+0000 7fc631d54640 1 --2- 172.21.15.82:0/3339898122 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55732c2014d0 0x55732c5c94d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:20:54.227 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:54.226+0000 7fc631d54640 1 -- 172.21.15.82:0/3339898122 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55732c4f28e0 con 0x55732c2221b0 2024-05-20T16:20:54.227 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:54.226+0000 7fc631d54640 1 --2- 172.21.15.82:0/751947411 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fc6180a2d00 0x7fc6180a5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:20:54.235 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:54.233+0000 7fc631d54640 1 --2- 172.21.15.82:0/3339898122 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55732c2221b0 0x55732c5c2050 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fc624009bf0 tx=0x7fc624032d60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bf3f7fe2c6cef1fb server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:20:54.241 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:54.239+0000 7fc6177fe640 1 -- 172.21.15.82:0/3339898122 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc624003a30 con 0x55732c2221b0 2024-05-20T16:20:54.241 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:54.239+0000 7fc6177fe640 1 -- 172.21.15.82:0/3339898122 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc624036040 con 0x55732c2221b0 2024-05-20T16:20:54.241 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:54.239+0000 7fc6177fe640 1 -- 172.21.15.82:0/3339898122 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc624004790 con 0x55732c2221b0 2024-05-20T16:20:54.241 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:54.241+0000 7fc63a65c740 1 -- 172.21.15.82:0/3339898122 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55732c2221b0 msgr2=0x55732c5c2050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:20:54.241 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:54.241+0000 7fc63a65c740 1 --2- 172.21.15.82:0/3339898122 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55732c2221b0 0x55732c5c2050 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fc624009bf0 tx=0x7fc624032d60 comp rx=0 tx=0).stop 2024-05-20T16:20:54.244 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:54.244+0000 7fc63a65c740 1 -- 172.21.15.82:0/3339898122 shutdown_connections 2024-05-20T16:20:54.244 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:54.244+0000 7fc63a65c740 1 --2- 172.21.15.82:0/3339898122 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55732c2014d0 0x55732c5c94d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:20:54.244 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:54.244+0000 7fc63a65c740 1 --2- 172.21.15.82:0/3339898122 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55732c1f92f0 0x55732c5c25d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:20:54.244 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:54.244+0000 7fc63a65c740 1 --2- 172.21.15.82:0/3339898122 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55732c2221b0 0x55732c5c2050 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:20:54.245 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:54.244+0000 7fc63a65c740 1 -- 172.21.15.82:0/3339898122 >> 172.21.15.82:0/3339898122 conn(0x55732c216300 msgr2=0x55732c5bd9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:20:54.247 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:54.244+0000 7fc63a65c740 1 -- 172.21.15.82:0/3339898122 shutdown_connections 2024-05-20T16:20:54.247 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:54.247+0000 7fc63a65c740 1 -- 172.21.15.82:0/3339898122 wait complete. 2024-05-20T16:20:54.248 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:54.248+0000 7fc631d54640 1 --2- 172.21.15.82:0/751947411 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fc6180a2d00 0x7fc6180a5210 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fc62404e000 tx=0x7fc62404f000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:20:54.248 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:54.248+0000 7fc63a65c740 1 -- start start 2024-05-20T16:20:54.248 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:54.248+0000 7fc63a65c740 1 Processor -- start 2024-05-20T16:20:54.249 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:54.250+0000 7fc63a65c740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55732c1f92f0 0x55732c5cbd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:20:54.249 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:54.250+0000 7fc63a65c740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55732c2014d0 0x55732c5cc2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:20:54.249 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:54.250+0000 7fc63a65c740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55732c2221b0 0x55732c5d3200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:20:54.250 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:54.250+0000 7fc63a65c740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55732c5c9f20 con 0x55732c1f92f0 2024-05-20T16:20:54.250 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:54.250+0000 7fc63a65c740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x55732c4e1790 con 0x55732c2221b0 2024-05-20T16:20:54.250 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:54.250+0000 7fc63a65c740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55732c5ca0a0 con 0x55732c2014d0 2024-05-20T16:20:54.250 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:54.250+0000 7fc63a65c740 10 client.0 _ll_register_callbacks cb 0x55732c51ca20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T16:20:54.254 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:54.254+0000 7fc631d54640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55732c2221b0 0x55732c5d3200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:20:54.255 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:54.254+0000 7fc631d54640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55732c2221b0 0x55732c5d3200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.82:57666/0 (socket says 172.21.15.82:57666) 2024-05-20T16:20:54.255 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:54.254+0000 7fc631d54640 1 -- 172.21.15.82:0/511780921 learned_addr learned my addr 172.21.15.82:0/511780921 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:20:54.255 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:54.254+0000 7fc631553640 1 --2- 172.21.15.82:0/511780921 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55732c1f92f0 0x55732c5cbd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:20:54.255 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:54.254+0000 7fc630d52640 1 --2- 172.21.15.82:0/511780921 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55732c2014d0 0x55732c5cc2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:20:54.255 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:54.256+0000 7fc631553640 1 -- 172.21.15.82:0/511780921 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55732c2014d0 msgr2=0x55732c5cc2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:20:54.255 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:54.256+0000 7fc631553640 1 --2- 172.21.15.82:0/511780921 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55732c2014d0 0x55732c5cc2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:20:54.256 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:54.256+0000 7fc631553640 1 -- 172.21.15.82:0/511780921 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55732c2221b0 msgr2=0x55732c5d3200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:20:54.256 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:54.256+0000 7fc631553640 1 --2- 172.21.15.82:0/511780921 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55732c2221b0 0x55732c5d3200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:20:54.256 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:54.256+0000 7fc631553640 1 -- 172.21.15.82:0/511780921 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55732c23a210 con 0x55732c1f92f0 2024-05-20T16:20:54.256 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:54.256+0000 7fc631553640 1 --2- 172.21.15.82:0/511780921 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55732c1f92f0 0x55732c5cbd60 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fc61c09c2f0 tx=0x7fc61c008030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39d531ad500ce346 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:20:54.258 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:54.259+0000 7fc614ff9640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-05-20T16:20:54.259 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:54.259+0000 7fc614ff9640 1 -- 172.21.15.82:0/511780921 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc61c0a0710 con 0x55732c1f92f0 2024-05-20T16:20:54.259 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:54.259+0000 7fc63a65c740 1 -- 172.21.15.82:0/511780921 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc624036040 con 0x55732c1f92f0 2024-05-20T16:20:54.259 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:54.259+0000 7fc614ff9640 1 -- 172.21.15.82:0/511780921 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9150+0+0 (secure 0 0 0) 0x7fc61c0103e0 con 0x55732c1f92f0 2024-05-20T16:20:54.271 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:54.272+0000 7fc614ff9640 1 -- 172.21.15.82:0/511780921 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fc61c0a4fd0 con 0x55732c1f92f0 2024-05-20T16:20:54.273 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:54.274+0000 7fc63a65c740 10 client.15777 fetch_fsmap learned FSMap version 27 2024-05-20T16:20:54.273 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:54.274+0000 7fc63a65c740 1 -- 172.21.15.82:0/511780921 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55732c306620 con 0x55732c1f92f0 2024-05-20T16:20:54.278 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:54.279+0000 7fc614ff9640 1 -- 172.21.15.82:0/511780921 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc624036040 con 0x55732c1f92f0 2024-05-20T16:20:54.278 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:54.279+0000 7fc614ff9640 1 -- 172.21.15.82:0/511780921 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fc61c09b7b0 con 0x55732c1f92f0 2024-05-20T16:20:54.280 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:54.280+0000 7fc63a65c740 10 client.15777 fetch_fsmap finished waiting for FSMap version 27 2024-05-20T16:20:54.280 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:54.280+0000 7fc63a65c740 10 client.15777 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T16:20:54.280 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:54.280+0000 7fc63a65c740 20 client.15777 populate_metadata read hostname 'smithi082' 2024-05-20T16:20:54.281 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:54.281+0000 7fc63a65c740 1 --2- 172.21.15.82:0/511780921 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x55732c1e0d80 0x55732c5d96c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:20:54.281 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:54.281+0000 7fc63a65c740 4 client.15777 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T16:20:54.281 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:54.281+0000 7fc63a65c740 1 -- 172.21.15.82:0/511780921 --> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55732c533a70 con 0x55732c1e0d80 2024-05-20T16:20:54.285 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:54.285+0000 7fc630d52640 1 --2- 172.21.15.82:0/511780921 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x55732c1e0d80 0x55732c5d96c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:20:54.289 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:54.289+0000 7fc630d52640 1 --2- 172.21.15.82:0/511780921 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x55732c1e0d80 0x55732c5d96c0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e1ee571cc57c7d9f server_cookie=dbe07aa1eda7cadb in_seq=0 out_seq=0 2024-05-20T16:20:54.291 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:54.292+0000 7fc614ff9640 10 client.15777 ms_handle_connect on v2:172.21.15.160:6838/2765554482 2024-05-20T16:20:54.295 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:54.296+0000 7fc614ff9640 1 -- 172.21.15.82:0/511780921 <== mds.0 v2:172.21.15.160:6838/2765554482 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55732c533a70 con 0x55732c1e0d80 2024-05-20T16:20:54.295 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:54.296+0000 7fc614ff9640 10 client.15777 handle_command_reply: tid=0 2024-05-20T16:20:54.298 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:54.298+0000 7fc63a65c740 1 client.15777 shutdown 2024-05-20T16:20:54.299 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:54.298+0000 7fc63a65c740 1 -- 172.21.15.82:0/511780921 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55732c1f92f0 msgr2=0x55732c5cbd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:20:54.299 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:54.298+0000 7fc63a65c740 1 --2- 172.21.15.82:0/511780921 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55732c1f92f0 0x55732c5cbd60 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fc61c09c2f0 tx=0x7fc61c008030 comp rx=0 tx=0).stop 2024-05-20T16:20:54.300 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:54.300+0000 7fc63a65c740 1 -- 172.21.15.82:0/511780921 shutdown_connections 2024-05-20T16:20:54.300 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:54.300+0000 7fc63a65c740 1 --2- 172.21.15.82:0/511780921 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x55732c1e0d80 0x55732c5d96c0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:20:54.300 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:54.300+0000 7fc63a65c740 1 --2- 172.21.15.82:0/511780921 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55732c2221b0 0x55732c5d3200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:20:54.300 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:54.300+0000 7fc63a65c740 1 --2- 172.21.15.82:0/511780921 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55732c2014d0 0x55732c5cc2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:20:54.300 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:54.300+0000 7fc63a65c740 1 --2- 172.21.15.82:0/511780921 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55732c1f92f0 0x55732c5cbd60 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:20:54.300 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:54.300+0000 7fc63a65c740 1 -- 172.21.15.82:0/511780921 >> 172.21.15.82:0/511780921 conn(0x55732c216300 msgr2=0x55732c5bcab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:20:54.301 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:54.301+0000 7fc614ff9640 0 client.15777 ms_handle_reset on v2:172.21.15.160:6838/2765554482 2024-05-20T16:20:54.304 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:54.304+0000 7fc63a65c740 1 -- 172.21.15.82:0/511780921 shutdown_connections 2024-05-20T16:20:54.304 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:54.305+0000 7fc63a65c740 1 -- 172.21.15.82:0/511780921 wait complete. 2024-05-20T16:20:54.305 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:54.305+0000 7fc63a65c740 20 client.15777 trim_cache size 0 max 16384 2024-05-20T16:20:54.349 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:54.349+0000 7fc63a65c740 1 Processor -- start 2024-05-20T16:20:54.350 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:54.349+0000 7fc63a65c740 1 -- start start 2024-05-20T16:20:54.350 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:54.350+0000 7fc63a65c740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55732c1f92f0 0x55732c5c6160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:20:54.350 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:54.350+0000 7fc63a65c740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55732c2014d0 0x55732c5c66a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:20:54.350 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:54.350+0000 7fc63a65c740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55732c2221b0 0x55732c5bd440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:20:54.350 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:54.350+0000 7fc63a65c740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fc6100125a0 con 0x55732c1f92f0 2024-05-20T16:20:54.350 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:54.350+0000 7fc63a65c740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fc61000fe70 con 0x55732c2014d0 2024-05-20T16:20:54.350 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:54.350+0000 7fc63a65c740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fc61000dcc0 con 0x55732c2221b0 2024-05-20T16:20:54.351 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:54.351+0000 7fc631d54640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55732c2221b0 0x55732c5bd440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:20:54.351 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:54.352+0000 7fc631553640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55732c1f92f0 0x55732c5c6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:20:54.351 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:54.352+0000 7fc631553640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55732c1f92f0 0x55732c5c6160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:37856/0 (socket says 172.21.15.82:37856) 2024-05-20T16:20:54.351 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:54.352+0000 7fc631553640 1 -- 172.21.15.82:0/4139165726 learned_addr learned my addr 172.21.15.82:0/4139165726 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:20:54.352 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:54.352+0000 7fc631553640 1 -- 172.21.15.82:0/4139165726 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55732c2221b0 msgr2=0x55732c5bd440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:20:54.352 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:54.352+0000 7fc631553640 1 --2- 172.21.15.82:0/4139165726 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55732c2221b0 0x55732c5bd440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:20:54.352 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:54.352+0000 7fc631553640 1 -- 172.21.15.82:0/4139165726 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55732c2014d0 msgr2=0x55732c5c66a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-20T16:20:54.352 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:54.352+0000 7fc631553640 1 --2- 172.21.15.82:0/4139165726 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55732c2014d0 0x55732c5c66a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:20:54.352 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:54.352+0000 7fc631553640 1 -- 172.21.15.82:0/4139165726 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55732c5c1c00 con 0x55732c1f92f0 2024-05-20T16:20:54.353 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:54.354+0000 7fc631553640 1 --2- 172.21.15.82:0/4139165726 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55732c1f92f0 0x55732c5c6160 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fc61c00be10 tx=0x7fc61c0980b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6fe74525b17445e server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:20:54.354 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:54.355+0000 7fc616ffd640 1 -- 172.21.15.82:0/4139165726 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc61c0b9020 con 0x55732c1f92f0 2024-05-20T16:20:54.354 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:54.355+0000 7fc616ffd640 1 -- 172.21.15.82:0/4139165726 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc61c0a0710 con 0x55732c1f92f0 2024-05-20T16:20:54.355 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:54.355+0000 7fc616ffd640 1 -- 172.21.15.82:0/4139165726 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc61c0b5310 con 0x55732c1f92f0 2024-05-20T16:20:54.357 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:54.357+0000 7fc63a65c740 1 -- 172.21.15.82:0/4139165726 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55732c1f92f0 msgr2=0x55732c5c6160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:20:54.357 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:54.357+0000 7fc63a65c740 1 --2- 172.21.15.82:0/4139165726 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55732c1f92f0 0x55732c5c6160 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fc61c00be10 tx=0x7fc61c0980b0 comp rx=0 tx=0).stop 2024-05-20T16:20:54.358 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:54.358+0000 7fc63a65c740 1 -- 172.21.15.82:0/4139165726 shutdown_connections 2024-05-20T16:20:54.358 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:54.358+0000 7fc63a65c740 1 --2- 172.21.15.82:0/4139165726 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55732c2221b0 0x55732c5bd440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:20:54.358 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:54.358+0000 7fc63a65c740 1 --2- 172.21.15.82:0/4139165726 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55732c2014d0 0x55732c5c66a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:20:54.358 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:54.358+0000 7fc63a65c740 1 --2- 172.21.15.82:0/4139165726 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55732c1f92f0 0x55732c5c6160 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:20:54.358 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:54.358+0000 7fc63a65c740 1 -- 172.21.15.82:0/4139165726 >> 172.21.15.82:0/4139165726 conn(0x55732c216300 msgr2=0x55732c5c1fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:20:54.371 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:54.372+0000 7fc63a65c740 1 -- 172.21.15.82:0/4139165726 shutdown_connections 2024-05-20T16:20:54.376 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:54.375+0000 7fc63a65c740 1 -- 172.21.15.82:0/4139165726 wait complete. 2024-05-20T16:20:54.376 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:54.376+0000 7fc63a65c740 1 -- start start 2024-05-20T16:20:54.376 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:54.376+0000 7fc63a65c740 1 Processor -- start 2024-05-20T16:20:54.377 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:54.377+0000 7fc63a65c740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55732c1f92f0 0x55732c5cff10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:20:54.377 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:54.377+0000 7fc63a65c740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55732c2014d0 0x55732c5d0450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:20:54.378 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:54.377+0000 7fc63a65c740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55732c2221b0 0x55732c5d12c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:20:54.378 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:54.377+0000 7fc63a65c740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fc61000e330 con 0x55732c1f92f0 2024-05-20T16:20:54.378 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:54.377+0000 7fc63a65c740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fc61000f7a0 con 0x55732c2014d0 2024-05-20T16:20:54.378 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:54.377+0000 7fc63a65c740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fc61000ea00 con 0x55732c2221b0 2024-05-20T16:20:54.378 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:54.377+0000 7fc63a65c740 10 client.0 _ll_register_callbacks cb 0x7fc61c098600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T16:20:54.381 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:54.378+0000 7fc631553640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55732c1f92f0 0x55732c5cff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:20:54.381 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:54.378+0000 7fc631553640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55732c1f92f0 0x55732c5cff10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:37864/0 (socket says 172.21.15.82:37864) 2024-05-20T16:20:54.381 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:54.378+0000 7fc631553640 1 -- 172.21.15.82:0/4178154398 learned_addr learned my addr 172.21.15.82:0/4178154398 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:20:54.381 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:54.379+0000 7fc631553640 1 -- 172.21.15.82:0/4178154398 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55732c2221b0 msgr2=0x55732c5d12c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-20T16:20:54.381 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:54.379+0000 7fc631553640 1 --2- 172.21.15.82:0/4178154398 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55732c2221b0 0x55732c5d12c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:20:54.382 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:54.379+0000 7fc631553640 1 -- 172.21.15.82:0/4178154398 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55732c2014d0 msgr2=0x55732c5d0450 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-20T16:20:54.382 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:54.379+0000 7fc631553640 1 --2- 172.21.15.82:0/4178154398 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55732c2014d0 0x55732c5d0450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:20:54.382 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:54.379+0000 7fc631553640 1 -- 172.21.15.82:0/4178154398 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55732c5d1880 con 0x55732c1f92f0 2024-05-20T16:20:54.385 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:54.382+0000 7fc631553640 1 --2- 172.21.15.82:0/4178154398 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55732c1f92f0 0x55732c5cff10 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fc61c098d30 tx=0x7fc61c011ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20fd94354c249b75 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:20:54.385 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:54.385+0000 7fc614ff9640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-05-20T16:20:54.385 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:54.385+0000 7fc614ff9640 1 -- 172.21.15.82:0/4178154398 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc61c0af060 con 0x55732c1f92f0 2024-05-20T16:20:54.385 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:54.385+0000 7fc63a65c740 1 -- 172.21.15.82:0/4178154398 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc61c0a0710 con 0x55732c1f92f0 2024-05-20T16:20:54.385 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:54.385+0000 7fc614ff9640 1 -- 172.21.15.82:0/4178154398 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9150+0+0 (secure 0 0 0) 0x7fc61c0bc110 con 0x55732c1f92f0 2024-05-20T16:20:54.389 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:54.389+0000 7fc614ff9640 1 -- 172.21.15.82:0/4178154398 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fc61c0b9070 con 0x55732c1f92f0 2024-05-20T16:20:54.389 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:54.390+0000 7fc63a65c740 10 client.15783 fetch_fsmap learned FSMap version 27 2024-05-20T16:20:54.389 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:54.390+0000 7fc63a65c740 1 -- 172.21.15.82:0/4178154398 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55732c5d2b10 con 0x55732c1f92f0 2024-05-20T16:20:54.393 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:54.393+0000 7fc614ff9640 1 -- 172.21.15.82:0/4178154398 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc61c0a0710 con 0x55732c1f92f0 2024-05-20T16:20:54.393 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:54.393+0000 7fc614ff9640 1 -- 172.21.15.82:0/4178154398 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fc61c0b0250 con 0x55732c1f92f0 2024-05-20T16:20:54.393 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:54.393+0000 7fc63a65c740 10 client.15783 fetch_fsmap finished waiting for FSMap version 27 2024-05-20T16:20:54.393 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:54.393+0000 7fc63a65c740 10 client.15783 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T16:20:54.393 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:54.393+0000 7fc63a65c740 20 client.15783 populate_metadata read hostname 'smithi082' 2024-05-20T16:20:54.393 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:54.393+0000 7fc63a65c740 1 --2- 172.21.15.82:0/4178154398 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x55732c1b61c0 0x55732c654ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:20:54.393 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:54.393+0000 7fc63a65c740 4 client.15783 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-20T16:20:54.394 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:54.393+0000 7fc63a65c740 1 -- 172.21.15.82:0/4178154398 --> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55732c52ddc0 con 0x55732c1b61c0 2024-05-20T16:20:54.397 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:54.397+0000 7fc630d52640 1 --2- 172.21.15.82:0/4178154398 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x55732c1b61c0 0x55732c654ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:20:54.401 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:54.401+0000 7fc630d52640 1 --2- 172.21.15.82:0/4178154398 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x55732c1b61c0 0x55732c654ea0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1997a9040154430f server_cookie=e7350b5ffa2b290c in_seq=0 out_seq=0 2024-05-20T16:20:54.402 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:54.403+0000 7fc614ff9640 10 client.15783 ms_handle_connect on v2:172.21.15.160:6838/2765554482 2024-05-20T16:20:57.062 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:20:56 smithi160 ceph-mon[36347]: pgmap v571: 129 pgs: 129 active+clean; 10 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 807 MiB/s wr, 1.50k op/s 2024-05-20T16:20:57.112 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:20:56 smithi082 ceph-mon[32238]: pgmap v571: 129 pgs: 129 active+clean; 10 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 807 MiB/s wr, 1.50k op/s 2024-05-20T16:20:57.135 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:20:56 smithi184 ceph-mon[36209]: pgmap v571: 129 pgs: 129 active+clean; 10 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 807 MiB/s wr, 1.50k op/s 2024-05-20T16:20:59.062 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:20:58 smithi160 ceph-mon[36347]: pgmap v572: 129 pgs: 129 active+clean; 10 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 625 MiB/s wr, 988 op/s 2024-05-20T16:20:59.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:20:58 smithi082 ceph-mon[32238]: pgmap v572: 129 pgs: 129 active+clean; 10 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 625 MiB/s wr, 988 op/s 2024-05-20T16:20:59.136 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:20:58 smithi184 ceph-mon[36209]: pgmap v572: 129 pgs: 129 active+clean; 10 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 625 MiB/s wr, 988 op/s 2024-05-20T16:20:59.720 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:59.720+0000 7fc614ff9640 1 -- 172.21.15.82:0/4178154398 <== mds.0 v2:172.21.15.160:6838/2765554482 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55732c52ddc0 con 0x55732c1b61c0 2024-05-20T16:20:59.720 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:59.720+0000 7fc614ff9640 10 client.15783 handle_command_reply: tid=0 2024-05-20T16:20:59.720 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:59.720+0000 7fc63a65c740 1 client.15783 shutdown 2024-05-20T16:20:59.720 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:59.720+0000 7fc63a65c740 1 -- 172.21.15.82:0/4178154398 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55732c1f92f0 msgr2=0x55732c5cff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:20:59.720 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:59.720+0000 7fc63a65c740 1 --2- 172.21.15.82:0/4178154398 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55732c1f92f0 0x55732c5cff10 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fc61c098d30 tx=0x7fc61c011ac0 comp rx=0 tx=0).stop 2024-05-20T16:20:59.721 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:59.721+0000 7fc63a65c740 1 -- 172.21.15.82:0/4178154398 shutdown_connections 2024-05-20T16:20:59.721 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:59.721+0000 7fc63a65c740 1 --2- 172.21.15.82:0/4178154398 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x55732c1b61c0 0x55732c654ea0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:20:59.721 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:59.721+0000 7fc63a65c740 1 --2- 172.21.15.82:0/4178154398 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55732c2221b0 0x55732c5d12c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:20:59.721 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:59.721+0000 7fc63a65c740 1 --2- 172.21.15.82:0/4178154398 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55732c2014d0 0x55732c5d0450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:20:59.721 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:59.721+0000 7fc63a65c740 1 --2- 172.21.15.82:0/4178154398 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55732c1f92f0 0x55732c5cff10 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:20:59.721 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:59.721+0000 7fc63a65c740 1 -- 172.21.15.82:0/4178154398 >> 172.21.15.82:0/4178154398 conn(0x55732c216300 msgr2=0x55732c5c6160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:20:59.721 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:59.721+0000 7fc614ff9640 0 client.15783 ms_handle_reset on v2:172.21.15.160:6838/2765554482 2024-05-20T16:20:59.722 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:59.721+0000 7fc63a65c740 1 -- 172.21.15.82:0/4178154398 shutdown_connections 2024-05-20T16:20:59.722 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:59.721+0000 7fc63a65c740 1 -- 172.21.15.82:0/4178154398 wait complete. 2024-05-20T16:20:59.722 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:59.722+0000 7fc63a65c740 20 client.15783 trim_cache size 0 max 16384 2024-05-20T16:20:59.722 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:{ 2024-05-20T16:20:59.722 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "message": "", 2024-05-20T16:20:59.722 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "return_code": 0 2024-05-20T16:20:59.722 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:} 2024-05-20T16:20:59.726 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:59.727+0000 7fc6337b7640 1 -- 172.21.15.82:0/751947411 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fc6180a2d00 msgr2=0x7fc6180a5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:20:59.726 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:59.727+0000 7fc6337b7640 1 --2- 172.21.15.82:0/751947411 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fc6180a2d00 0x7fc6180a5210 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fc62404e000 tx=0x7fc62404f000 comp rx=0 tx=0).stop 2024-05-20T16:20:59.727 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:59.727+0000 7fc6337b7640 1 -- 172.21.15.82:0/751947411 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc62c109300 msgr2=0x7fc62c1a8050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:20:59.727 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:59.727+0000 7fc6337b7640 1 --2- 172.21.15.82:0/751947411 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc62c109300 0x7fc62c1a8050 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fc61c00b450 tx=0x7fc61c00b910 comp rx=0 tx=0).stop 2024-05-20T16:20:59.727 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:59.728+0000 7fc6337b7640 1 -- 172.21.15.82:0/751947411 shutdown_connections 2024-05-20T16:20:59.728 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:59.728+0000 7fc6337b7640 1 --2- 172.21.15.82:0/751947411 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fc6180a2d00 0x7fc6180a5210 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:20:59.728 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:59.728+0000 7fc6337b7640 1 --2- 172.21.15.82:0/751947411 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc62c10eea0 0x7fc62c1af520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:20:59.728 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:59.728+0000 7fc6337b7640 1 --2- 172.21.15.82:0/751947411 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc62c10a180 0x7fc62c1a8590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:20:59.728 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:59.728+0000 7fc6337b7640 1 --2- 172.21.15.82:0/751947411 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc62c109300 0x7fc62c1a8050 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:20:59.729 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:59.728+0000 7fc6337b7640 1 -- 172.21.15.82:0/751947411 >> 172.21.15.82:0/751947411 conn(0x7fc62c06f340 msgr2=0x7fc62c10ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:20:59.731 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:59.732+0000 7fc6337b7640 1 -- 172.21.15.82:0/751947411 shutdown_connections 2024-05-20T16:20:59.731 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:20:59.732+0000 7fc6337b7640 1 -- 172.21.15.82:0/751947411 wait complete. 2024-05-20T16:21:01.062 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:21:00 smithi160 ceph-mon[36347]: pgmap v573: 129 pgs: 129 active+clean; 10 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 806 MiB/s wr, 1.33k op/s 2024-05-20T16:21:01.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:21:00 smithi082 ceph-mon[32238]: pgmap v573: 129 pgs: 129 active+clean; 10 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 806 MiB/s wr, 1.33k op/s 2024-05-20T16:21:01.135 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:21:00 smithi184 ceph-mon[36209]: pgmap v573: 129 pgs: 129 active+clean; 10 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 806 MiB/s wr, 1.33k op/s 2024-05-20T16:21:03.062 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:21:02 smithi160 ceph-mon[36347]: pgmap v574: 129 pgs: 129 active+clean; 10 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 720 MiB/s wr, 1.18k op/s 2024-05-20T16:21:03.110 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:21:02 smithi082 ceph-mon[32238]: pgmap v574: 129 pgs: 129 active+clean; 10 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 720 MiB/s wr, 1.18k op/s 2024-05-20T16:21:03.135 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:21:02 smithi184 ceph-mon[36209]: pgmap v574: 129 pgs: 129 active+clean; 10 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 720 MiB/s wr, 1.18k op/s 2024-05-20T16:21:05.063 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:21:04 smithi160 ceph-mon[36347]: pgmap v575: 129 pgs: 129 active+clean; 10 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 755 MiB/s wr, 1.29k op/s 2024-05-20T16:21:05.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:21:04 smithi082 ceph-mon[32238]: pgmap v575: 129 pgs: 129 active+clean; 10 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 755 MiB/s wr, 1.29k op/s 2024-05-20T16:21:05.136 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:21:04 smithi184 ceph-mon[36209]: pgmap v575: 129 pgs: 129 active+clean; 10 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 755 MiB/s wr, 1.29k op/s 2024-05-20T16:21:06.062 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:21:05 smithi160 ceph-mon[36347]: pgmap v576: 129 pgs: 129 active+clean; 10 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 887 MiB/s wr, 1.78k op/s 2024-05-20T16:21:06.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:21:05 smithi082 ceph-mon[32238]: pgmap v576: 129 pgs: 129 active+clean; 10 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 887 MiB/s wr, 1.78k op/s 2024-05-20T16:21:06.135 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:21:05 smithi184 ceph-mon[36209]: pgmap v576: 129 pgs: 129 active+clean; 10 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 887 MiB/s wr, 1.78k op/s 2024-05-20T16:21:09.062 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:21:08 smithi160 ceph-mon[36347]: pgmap v577: 129 pgs: 129 active+clean; 10 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 650 MiB/s wr, 1.37k op/s 2024-05-20T16:21:09.113 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:21:08 smithi082 ceph-mon[32238]: pgmap v577: 129 pgs: 129 active+clean; 10 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 650 MiB/s wr, 1.37k op/s 2024-05-20T16:21:09.135 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:21:08 smithi184 ceph-mon[36209]: pgmap v577: 129 pgs: 129 active+clean; 10 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 650 MiB/s wr, 1.37k op/s 2024-05-20T16:21:11.062 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:21:10 smithi160 ceph-mon[36347]: pgmap v578: 129 pgs: 129 active+clean; 10 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 826 MiB/s wr, 1.78k op/s 2024-05-20T16:21:11.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:21:10 smithi082 ceph-mon[32238]: pgmap v578: 129 pgs: 129 active+clean; 10 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 826 MiB/s wr, 1.78k op/s 2024-05-20T16:21:11.135 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:21:10 smithi184 ceph-mon[36209]: pgmap v578: 129 pgs: 129 active+clean; 10 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 826 MiB/s wr, 1.78k op/s 2024-05-20T16:21:13.046 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:13.042+0000 7f39dea73640 1 -- 172.21.15.82:0/3173037563 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f39d8109280 msgr2=0x7f39d8109fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:21:13.046 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:13.042+0000 7f39dea73640 1 --2- 172.21.15.82:0/3173037563 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f39d8109280 0x7f39d8109fc0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f39c4009900 tx=0x7f39c402a440 comp rx=0 tx=0).stop 2024-05-20T16:21:13.046 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:13.046+0000 7f39dea73640 1 -- 172.21.15.82:0/3173037563 shutdown_connections 2024-05-20T16:21:13.046 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:13.046+0000 7f39dea73640 1 --2- 172.21.15.82:0/3173037563 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f39d810a500 0x7f39d810a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:21:13.046 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:13.046+0000 7f39dea73640 1 --2- 172.21.15.82:0/3173037563 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f39d810f290 0x7f39d81116b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:21:13.046 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:13.046+0000 7f39dea73640 1 --2- 172.21.15.82:0/3173037563 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f39d8109280 0x7f39d8109fc0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:21:13.047 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:13.046+0000 7f39dea73640 1 -- 172.21.15.82:0/3173037563 >> 172.21.15.82:0/3173037563 conn(0x7f39d806f760 msgr2=0x7f39d8071c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:21:13.052 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:13.053+0000 7f39dea73640 1 -- 172.21.15.82:0/3173037563 shutdown_connections 2024-05-20T16:21:13.056 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:13.055+0000 7f39dea73640 1 -- 172.21.15.82:0/3173037563 wait complete. 2024-05-20T16:21:13.056 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:13.056+0000 7f39dea73640 1 Processor -- start 2024-05-20T16:21:13.056 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:13.056+0000 7f39dea73640 1 -- start start 2024-05-20T16:21:13.056 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:13.057+0000 7f39dea73640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f39d8109280 0x7f39d811aee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:21:13.057 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:13.057+0000 7f39dea73640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f39d810a500 0x7f39d8119580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:21:13.057 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:13.057+0000 7f39dea73640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f39d810f290 0x7f39d8119ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:21:13.057 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:13.057+0000 7f39dea73640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f39d81138a0 con 0x7f39d8109280 2024-05-20T16:21:13.057 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:13.058+0000 7f39dea73640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f39d8113720 con 0x7f39d810a500 2024-05-20T16:21:13.057 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:13.058+0000 7f39dea73640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f39d8113a20 con 0x7f39d810f290 2024-05-20T16:21:13.061 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:13.059+0000 7f39dc80f640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f39d8109280 0x7f39d811aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:21:13.061 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:13.059+0000 7f39dc80f640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f39d8109280 0x7f39d811aee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:40806/0 (socket says 172.21.15.82:40806) 2024-05-20T16:21:13.061 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:13.059+0000 7f39dc80f640 1 -- 172.21.15.82:0/1227297270 learned_addr learned my addr 172.21.15.82:0/1227297270 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:21:13.061 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:13.060+0000 7f39d7fff640 1 --2- 172.21.15.82:0/1227297270 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f39d810a500 0x7f39d8119580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:21:13.062 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:13.060+0000 7f39dc80f640 1 -- 172.21.15.82:0/1227297270 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f39d810f290 msgr2=0x7f39d8119ac0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-20T16:21:13.062 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:13.060+0000 7f39dc80f640 1 --2- 172.21.15.82:0/1227297270 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f39d810f290 0x7f39d8119ac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:21:13.062 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:13.060+0000 7f39dc80f640 1 -- 172.21.15.82:0/1227297270 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f39d810a500 msgr2=0x7f39d8119580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:21:13.062 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:13.060+0000 7f39dc80f640 1 --2- 172.21.15.82:0/1227297270 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f39d810a500 0x7f39d8119580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:21:13.062 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:13.060+0000 7f39dc80f640 1 -- 172.21.15.82:0/1227297270 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39d811a240 con 0x7f39d8109280 2024-05-20T16:21:13.062 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:13.061+0000 7f39dc80f640 1 --2- 172.21.15.82:0/1227297270 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f39d8109280 0x7f39d811aee0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f39c4031440 tx=0x7f39c4032e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:21:13.062 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:21:12 smithi160 ceph-mon[36347]: pgmap v579: 129 pgs: 129 active+clean; 10 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 817 KiB/s rd, 782 MiB/s wr, 1.61k op/s 2024-05-20T16:21:13.064 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:13.064+0000 7f39d5ffb640 1 -- 172.21.15.82:0/1227297270 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39c403f030 con 0x7f39d8109280 2024-05-20T16:21:13.068 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:13.065+0000 7f39dea73640 1 -- 172.21.15.82:0/1227297270 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39d81b87d0 con 0x7f39d8109280 2024-05-20T16:21:13.068 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:13.065+0000 7f39dea73640 1 -- 172.21.15.82:0/1227297270 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39d81b8c50 con 0x7f39d8109280 2024-05-20T16:21:13.068 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:13.069+0000 7f39e59f8740 1 Processor -- start 2024-05-20T16:21:13.069 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:13.069+0000 7f39e59f8740 1 -- start start 2024-05-20T16:21:13.069 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:13.069+0000 7f39e59f8740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e4686301b0 0x55e4689d0050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:21:13.069 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:13.069+0000 7f39e59f8740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e4686072f0 0x55e4689d05d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:21:13.069 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:13.069+0000 7f39e59f8740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e46860f4d0 0x55e4689d74d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:21:13.069 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:13.069+0000 7f39e59f8740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55e468934340 con 0x55e46860f4d0 2024-05-20T16:21:13.069 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:13.069+0000 7f39e59f8740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x55e4688dc290 con 0x55e4686072f0 2024-05-20T16:21:13.069 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:13.069+0000 7f39e59f8740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55e468930300 con 0x55e4686301b0 2024-05-20T16:21:13.070 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:13.070+0000 7f39d5ffb640 1 -- 172.21.15.82:0/1227297270 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f39c4003ce0 con 0x7f39d8109280 2024-05-20T16:21:13.070 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:13.070+0000 7f39dd010640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e4686301b0 0x55e4689d0050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:21:13.070 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:13.070+0000 7f39dc80f640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e4686072f0 0x55e4689d05d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:21:13.071 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:13.070+0000 7f39dc80f640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e4686072f0 0x55e4689d05d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.82:46786/0 (socket says 172.21.15.82:46786) 2024-05-20T16:21:13.071 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:13.070+0000 7f39dc80f640 1 -- 172.21.15.82:0/404007944 learned_addr learned my addr 172.21.15.82:0/404007944 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:21:13.071 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:13.070+0000 7f39d7fff640 1 --2- 172.21.15.82:0/404007944 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e46860f4d0 0x55e4689d74d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:21:13.071 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:13.070+0000 7f39d5ffb640 1 -- 172.21.15.82:0/1227297270 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39c4035070 con 0x7f39d8109280 2024-05-20T16:21:13.071 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:13.071+0000 7f39dc80f640 1 -- 172.21.15.82:0/404007944 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e4686301b0 msgr2=0x55e4689d0050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:21:13.071 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:13.071+0000 7f39dc80f640 1 --2- 172.21.15.82:0/404007944 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e4686301b0 0x55e4689d0050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:21:13.071 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:13.071+0000 7f39dc80f640 1 -- 172.21.15.82:0/404007944 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e46860f4d0 msgr2=0x55e4689d74d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:21:13.071 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:13.071+0000 7f39dc80f640 1 --2- 172.21.15.82:0/404007944 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e46860f4d0 0x55e4689d74d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:21:13.072 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:13.072+0000 7f39dc80f640 1 -- 172.21.15.82:0/404007944 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e4689008e0 con 0x55e4686072f0 2024-05-20T16:21:13.072 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:13.073+0000 7f39dc80f640 1 --2- 172.21.15.82:0/404007944 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e4686072f0 0x55e4689d05d0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f39c4005f70 tx=0x7f39c4040000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b75a79354c30333b server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:21:13.072 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:13.073+0000 7f39d7fff640 1 --2- 172.21.15.82:0/404007944 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e46860f4d0 0x55e4689d74d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T16:21:13.073 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:13.073+0000 7f39c2ffd640 1 -- 172.21.15.82:0/404007944 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39c403b460 con 0x55e4686072f0 2024-05-20T16:21:13.073 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:13.073+0000 7f39c2ffd640 1 -- 172.21.15.82:0/404007944 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f39c4044e30 con 0x55e4686072f0 2024-05-20T16:21:13.074 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:13.075+0000 7f39e59f8740 1 -- 172.21.15.82:0/404007944 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e4686072f0 msgr2=0x55e4689d05d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:21:13.074 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:13.075+0000 7f39e59f8740 1 --2- 172.21.15.82:0/404007944 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e4686072f0 0x55e4689d05d0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f39c4005f70 tx=0x7f39c4040000 comp rx=0 tx=0).stop 2024-05-20T16:21:13.075 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:13.075+0000 7f39e59f8740 1 -- 172.21.15.82:0/404007944 shutdown_connections 2024-05-20T16:21:13.075 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:13.075+0000 7f39e59f8740 1 --2- 172.21.15.82:0/404007944 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e46860f4d0 0x55e4689d74d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:21:13.075 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:13.075+0000 7f39e59f8740 1 --2- 172.21.15.82:0/404007944 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e4686072f0 0x55e4689d05d0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:21:13.075 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:13.075+0000 7f39e59f8740 1 --2- 172.21.15.82:0/404007944 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e4686301b0 0x55e4689d0050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:21:13.075 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:13.075+0000 7f39e59f8740 1 -- 172.21.15.82:0/404007944 >> 172.21.15.82:0/404007944 conn(0x55e468624300 msgr2=0x55e4689cb9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:21:13.075 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:13.076+0000 7f39e59f8740 1 -- 172.21.15.82:0/404007944 shutdown_connections 2024-05-20T16:21:13.081 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:13.081+0000 7f39e59f8740 1 -- 172.21.15.82:0/404007944 wait complete. 2024-05-20T16:21:13.081 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:13.081+0000 7f39e59f8740 1 -- start start 2024-05-20T16:21:13.082 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:13.081+0000 7f39e59f8740 1 Processor -- start 2024-05-20T16:21:13.082 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:13.082+0000 7f39d5ffb640 1 -- 172.21.15.82:0/1227297270 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f39c4042030 con 0x7f39d8109280 2024-05-20T16:21:13.082 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:13.082+0000 7f39e59f8740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e4686072f0 0x55e4689d99d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:21:13.082 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:13.082+0000 7f39e59f8740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e46860f4d0 0x55e4689d9f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:21:13.082 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:13.082+0000 7f39e59f8740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e4686301b0 0x55e4689e0e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:21:13.082 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:13.082+0000 7f39e59f8740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55e4689d7ce0 con 0x55e4686072f0 2024-05-20T16:21:13.082 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:13.082+0000 7f39e59f8740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x55e4688ef790 con 0x55e4686301b0 2024-05-20T16:21:13.083 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:13.082+0000 7f39e59f8740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55e4689d7e60 con 0x55e46860f4d0 2024-05-20T16:21:13.083 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:13.082+0000 7f39e59f8740 10 client.0 _ll_register_callbacks cb 0x55e46892aa20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T16:21:13.083 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:13.083+0000 7f39dd010640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e4686072f0 0x55e4689d99d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:21:13.083 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:13.083+0000 7f39dd010640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e4686072f0 0x55e4689d99d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:40824/0 (socket says 172.21.15.82:40824) 2024-05-20T16:21:13.083 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:13.083+0000 7f39dd010640 1 -- 172.21.15.82:0/4165833997 learned_addr learned my addr 172.21.15.82:0/4165833997 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:21:13.088 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:13.084+0000 7f39dd010640 1 -- 172.21.15.82:0/4165833997 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e46860f4d0 msgr2=0x55e4689d9f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:21:13.088 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:13.084+0000 7f39dc80f640 1 --2- 172.21.15.82:0/4165833997 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e46860f4d0 0x55e4689d9f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:21:13.088 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:13.086+0000 7f39d7fff640 1 --2- 172.21.15.82:0/4165833997 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e4686301b0 0x55e4689e0e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:21:13.089 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:13.088+0000 7f39dd010640 1 --2- 172.21.15.82:0/4165833997 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e46860f4d0 0x55e4689d9f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:21:13.089 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:13.088+0000 7f39dd010640 1 -- 172.21.15.82:0/4165833997 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e4686301b0 msgr2=0x55e4689e0e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:21:13.089 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:13.088+0000 7f39dd010640 1 --2- 172.21.15.82:0/4165833997 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e4686301b0 0x55e4689e0e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:21:13.089 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:13.088+0000 7f39dd010640 1 -- 172.21.15.82:0/4165833997 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e468648210 con 0x55e4686072f0 2024-05-20T16:21:13.089 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:13.089+0000 7f39dc80f640 1 --2- 172.21.15.82:0/4165833997 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e46860f4d0 0x55e4689d9f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T16:21:13.097 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:13.096+0000 7f39d5ffb640 1 --2- 172.21.15.82:0/1227297270 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f39c80a2c60 0x7f39c80a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:21:13.097 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:13.097+0000 7f39d5ffb640 1 -- 172.21.15.82:0/1227297270 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9150+0+0 (secure 0 0 0) 0x7f39c40f1ea0 con 0x7f39d8109280 2024-05-20T16:21:13.097 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:13.098+0000 7f39dd010640 1 --2- 172.21.15.82:0/4165833997 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e4686072f0 0x55e4689d99d0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f39d000ab50 tx=0x7f39d000e490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff4927d5ba8bdddb server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:21:13.098 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:13.098+0000 7f39dc80f640 1 --2- 172.21.15.82:0/1227297270 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f39c80a2c60 0x7f39c80a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:21:13.098 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:13.098+0000 7f39bbfff640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-05-20T16:21:13.098 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:13.099+0000 7f39bbfff640 1 -- 172.21.15.82:0/4165833997 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39d0018020 con 0x55e4686072f0 2024-05-20T16:21:13.099 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:13.099+0000 7f39bbfff640 1 -- 172.21.15.82:0/4165833997 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9150+0+0 (secure 0 0 0) 0x7f39d000eda0 con 0x55e4686072f0 2024-05-20T16:21:13.100 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:13.100+0000 7f39dc80f640 1 --2- 172.21.15.82:0/1227297270 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f39c80a2c60 0x7f39c80a5170 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x55e4689daa40 tx=0x7f39c4000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:21:13.101 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:13.101+0000 7f39e59f8740 1 -- 172.21.15.82:0/4165833997 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f39c4044e30 con 0x55e4686072f0 2024-05-20T16:21:13.102 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:13.102+0000 7f39bbfff640 1 -- 172.21.15.82:0/4165833997 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f39d0014400 con 0x55e4686072f0 2024-05-20T16:21:13.102 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:13.102+0000 7f39e59f8740 10 client.15792 fetch_fsmap learned FSMap version 27 2024-05-20T16:21:13.102 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:13.102+0000 7f39e59f8740 1 -- 172.21.15.82:0/4165833997 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55e468714620 con 0x55e4686072f0 2024-05-20T16:21:13.102 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:13.103+0000 7f39bbfff640 1 -- 172.21.15.82:0/4165833997 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f39c4044e30 con 0x55e4686072f0 2024-05-20T16:21:13.103 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:13.103+0000 7f39bbfff640 1 -- 172.21.15.82:0/4165833997 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f39d001c6b0 con 0x55e4686072f0 2024-05-20T16:21:13.103 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:13.103+0000 7f39e59f8740 10 client.15792 fetch_fsmap finished waiting for FSMap version 27 2024-05-20T16:21:13.103 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:13.103+0000 7f39e59f8740 10 client.15792 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T16:21:13.103 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:13.103+0000 7f39e59f8740 20 client.15792 populate_metadata read hostname 'smithi082' 2024-05-20T16:21:13.103 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:13.103+0000 7f39e59f8740 1 --2- 172.21.15.82:0/4165833997 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x55e4685eed80 0x55e4689e77d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:21:13.103 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:13.103+0000 7f39e59f8740 4 client.15792 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T16:21:13.103 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:13.103+0000 7f39e59f8740 1 -- 172.21.15.82:0/4165833997 --> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e468941a70 con 0x55e4685eed80 2024-05-20T16:21:13.104 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:13.104+0000 7f39d7fff640 1 --2- 172.21.15.82:0/4165833997 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x55e4685eed80 0x55e4689e77d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:21:13.104 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:13.105+0000 7f39bbfff640 10 client.15792 ms_handle_connect on v2:172.21.15.160:6838/2765554482 2024-05-20T16:21:13.104 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:13.105+0000 7f39d7fff640 1 --2- 172.21.15.82:0/4165833997 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x55e4685eed80 0x55e4689e77d0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=44eb9b67fca2715b server_cookie=c3e0e99e3692bb15 in_seq=0 out_seq=0 2024-05-20T16:21:13.108 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:13.108+0000 7f39bbfff640 1 -- 172.21.15.82:0/4165833997 <== mds.0 v2:172.21.15.160:6838/2765554482 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55e468941a70 con 0x55e4685eed80 2024-05-20T16:21:13.108 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:13.108+0000 7f39bbfff640 10 client.15792 handle_command_reply: tid=0 2024-05-20T16:21:13.108 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:13.108+0000 7f39e59f8740 1 client.15792 shutdown 2024-05-20T16:21:13.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:21:12 smithi082 ceph-mon[32238]: pgmap v579: 129 pgs: 129 active+clean; 10 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 817 KiB/s rd, 782 MiB/s wr, 1.61k op/s 2024-05-20T16:21:13.114 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:13.113+0000 7f39e59f8740 1 -- 172.21.15.82:0/4165833997 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e4686072f0 msgr2=0x55e4689d99d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:21:13.114 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:13.113+0000 7f39e59f8740 1 --2- 172.21.15.82:0/4165833997 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e4686072f0 0x55e4689d99d0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f39d000ab50 tx=0x7f39d000e490 comp rx=0 tx=0).stop 2024-05-20T16:21:13.121 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:13.120+0000 7f39e59f8740 1 -- 172.21.15.82:0/4165833997 shutdown_connections 2024-05-20T16:21:13.121 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:13.120+0000 7f39e59f8740 1 --2- 172.21.15.82:0/4165833997 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x55e4685eed80 0x55e4689e77d0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:21:13.121 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:13.120+0000 7f39e59f8740 1 --2- 172.21.15.82:0/4165833997 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e4686301b0 0x55e4689e0e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:21:13.121 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:13.120+0000 7f39e59f8740 1 --2- 172.21.15.82:0/4165833997 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e46860f4d0 0x55e4689d9f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:21:13.121 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:13.120+0000 7f39e59f8740 1 --2- 172.21.15.82:0/4165833997 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e4686072f0 0x55e4689d99d0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:21:13.121 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:13.120+0000 7f39e59f8740 1 -- 172.21.15.82:0/4165833997 >> 172.21.15.82:0/4165833997 conn(0x55e468624300 msgr2=0x55e4689ce990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:21:13.121 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:13.120+0000 7f39bbfff640 0 client.15792 ms_handle_reset on v2:172.21.15.160:6838/2765554482 2024-05-20T16:21:13.122 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:13.123+0000 7f39e59f8740 1 -- 172.21.15.82:0/4165833997 shutdown_connections 2024-05-20T16:21:13.123 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:13.124+0000 7f39e59f8740 1 -- 172.21.15.82:0/4165833997 wait complete. 2024-05-20T16:21:13.123 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:13.124+0000 7f39e59f8740 20 client.15792 trim_cache size 0 max 16384 2024-05-20T16:21:13.135 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:21:12 smithi184 ceph-mon[36209]: pgmap v579: 129 pgs: 129 active+clean; 10 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 817 KiB/s rd, 782 MiB/s wr, 1.61k op/s 2024-05-20T16:21:13.169 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:13.169+0000 7f39e59f8740 1 Processor -- start 2024-05-20T16:21:13.169 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:13.169+0000 7f39e59f8740 1 -- start start 2024-05-20T16:21:13.169 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:13.170+0000 7f39e59f8740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e4686072f0 0x55e4689ce880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:21:13.170 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:13.170+0000 7f39e59f8740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e46860f4d0 0x55e4689e0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:21:13.170 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:13.170+0000 7f39e59f8740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e4686301b0 0x55e4689e1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:21:13.170 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:13.170+0000 7f39e59f8740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f39bc0125f0 con 0x55e46860f4d0 2024-05-20T16:21:13.170 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:13.170+0000 7f39e59f8740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f39bc00fec0 con 0x55e4686301b0 2024-05-20T16:21:13.170 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:13.170+0000 7f39e59f8740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f39bc00dd10 con 0x55e4686072f0 2024-05-20T16:21:13.173 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:13.170+0000 7f39dd010640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e4686301b0 0x55e4689e1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:21:13.173 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:13.170+0000 7f39dd010640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e4686301b0 0x55e4689e1340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.82:46804/0 (socket says 172.21.15.82:46804) 2024-05-20T16:21:13.173 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:13.170+0000 7f39dd010640 1 -- 172.21.15.82:0/3155438409 learned_addr learned my addr 172.21.15.82:0/3155438409 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:21:13.174 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:13.174+0000 7f39dd010640 1 --2- 172.21.15.82:0/3155438409 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e4686072f0 0x55e4689ce880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:21:13.176 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:13.176+0000 7f39dd010640 1 -- 172.21.15.82:0/3155438409 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e4686072f0 msgr2=0x55e4689ce880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:21:13.176 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:13.176+0000 7f39dd010640 1 --2- 172.21.15.82:0/3155438409 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e4686072f0 0x55e4689ce880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:21:13.176 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:13.176+0000 7f39dd010640 1 -- 172.21.15.82:0/3155438409 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e46860f4d0 msgr2=0x55e4689e0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:21:13.176 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:13.176+0000 7f39dd010640 1 --2- 172.21.15.82:0/3155438409 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e46860f4d0 0x55e4689e0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:21:13.176 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:13.176+0000 7f39dd010640 1 -- 172.21.15.82:0/3155438409 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e4689cfc00 con 0x55e4686301b0 2024-05-20T16:21:13.178 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:13.178+0000 7f39dd010640 1 --2- 172.21.15.82:0/3155438409 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e4686301b0 0x55e4689e1340 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f39d001cb00 tx=0x7f39d0009390 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a622baf4cbb7dfc7 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:21:13.182 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:13.182+0000 7f39c27fc640 1 -- 172.21.15.82:0/3155438409 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39d0018070 con 0x55e4686301b0 2024-05-20T16:21:13.182 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:13.182+0000 7f39c27fc640 1 -- 172.21.15.82:0/3155438409 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f39d001c880 con 0x55e4686301b0 2024-05-20T16:21:13.182 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:13.182+0000 7f39c27fc640 1 -- 172.21.15.82:0/3155438409 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39d0014ac0 con 0x55e4686301b0 2024-05-20T16:21:13.188 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:13.187+0000 7f39e59f8740 1 -- 172.21.15.82:0/3155438409 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e4686301b0 msgr2=0x55e4689e1340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:21:13.188 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:13.187+0000 7f39e59f8740 1 --2- 172.21.15.82:0/3155438409 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e4686301b0 0x55e4689e1340 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f39d001cb00 tx=0x7f39d0009390 comp rx=0 tx=0).stop 2024-05-20T16:21:13.188 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:13.189+0000 7f39e59f8740 1 -- 172.21.15.82:0/3155438409 shutdown_connections 2024-05-20T16:21:13.189 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:13.189+0000 7f39e59f8740 1 --2- 172.21.15.82:0/3155438409 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e46860f4d0 0x55e4689e0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:21:13.189 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:13.189+0000 7f39e59f8740 1 --2- 172.21.15.82:0/3155438409 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e4686301b0 0x55e4689e1340 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:21:13.189 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:13.189+0000 7f39e59f8740 1 --2- 172.21.15.82:0/3155438409 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e4686072f0 0x55e4689ce880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:21:13.189 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:13.189+0000 7f39e59f8740 1 -- 172.21.15.82:0/3155438409 >> 172.21.15.82:0/3155438409 conn(0x55e468624300 msgr2=0x55e4689cb840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:21:13.192 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:13.190+0000 7f39e59f8740 1 -- 172.21.15.82:0/3155438409 shutdown_connections 2024-05-20T16:21:13.194 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:13.194+0000 7f39e59f8740 1 -- 172.21.15.82:0/3155438409 wait complete. 2024-05-20T16:21:13.194 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:13.194+0000 7f39e59f8740 1 -- start start 2024-05-20T16:21:13.194 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:13.194+0000 7f39e59f8740 1 Processor -- start 2024-05-20T16:21:13.195 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:13.195+0000 7f39e59f8740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e4686072f0 0x55e4689dee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:21:13.195 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:13.195+0000 7f39e59f8740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e46860f4d0 0x55e4689df340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:21:13.195 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:13.195+0000 7f39e59f8740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e4686301b0 0x55e4689e0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:21:13.195 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:13.195+0000 7f39e59f8740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f39bc00e380 con 0x55e46860f4d0 2024-05-20T16:21:13.195 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:13.195+0000 7f39e59f8740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f39bc00f7f0 con 0x55e4686301b0 2024-05-20T16:21:13.195 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:13.195+0000 7f39e59f8740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f39bc00ea50 con 0x55e4686072f0 2024-05-20T16:21:13.196 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:13.195+0000 7f39e59f8740 10 client.0 _ll_register_callbacks cb 0x7f39d00099b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T16:21:13.196 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:13.195+0000 7f39d7fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e46860f4d0 0x55e4689df340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:21:13.196 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:13.196+0000 7f39d7fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e46860f4d0 0x55e4689df340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:40840/0 (socket says 172.21.15.82:40840) 2024-05-20T16:21:13.196 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:13.196+0000 7f39d7fff640 1 -- 172.21.15.82:0/1308553874 learned_addr learned my addr 172.21.15.82:0/1308553874 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:21:13.197 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:13.196+0000 7f39dd010640 1 --2- 172.21.15.82:0/1308553874 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e4686072f0 0x55e4689dee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:21:13.197 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:13.196+0000 7f39dd010640 1 --2- 172.21.15.82:0/1308553874 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e4686301b0 0x55e4689e0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:21:13.197 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:13.196+0000 7f39dd010640 1 -- 172.21.15.82:0/1308553874 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e4686301b0 msgr2=0x55e4689e0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:21:13.197 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:13.196+0000 7f39dd010640 1 --2- 172.21.15.82:0/1308553874 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e4686301b0 0x55e4689e0200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:21:13.197 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:13.196+0000 7f39dd010640 1 -- 172.21.15.82:0/1308553874 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e46860f4d0 msgr2=0x55e4689df340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:21:13.197 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:13.196+0000 7f39dd010640 1 --2- 172.21.15.82:0/1308553874 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e46860f4d0 0x55e4689df340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:21:13.197 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:13.196+0000 7f39dd010640 1 -- 172.21.15.82:0/1308553874 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e4689e07c0 con 0x55e4686072f0 2024-05-20T16:21:13.199 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:13.198+0000 7f39dd010640 1 --2- 172.21.15.82:0/1308553874 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e4686072f0 0x55e4689dee00 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f39d001cba0 tx=0x7f39d00096e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e5342f1ef5363934 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:21:13.199 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:13.198+0000 7f39bbfff640 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-05-20T16:21:13.199 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:13.198+0000 7f39bbfff640 1 -- 172.21.15.82:0/1308553874 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39d0018040 con 0x55e4686072f0 2024-05-20T16:21:13.199 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:13.198+0000 7f39bbfff640 1 -- 172.21.15.82:0/1308553874 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9150+0+0 (secure 0 0 0) 0x7f39d001c5a0 con 0x55e4686072f0 2024-05-20T16:21:13.199 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:13.200+0000 7f39e59f8740 1 -- 172.21.15.82:0/1308553874 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f39d001c880 con 0x55e4686072f0 2024-05-20T16:21:13.201 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:13.202+0000 7f39bbfff640 1 -- 172.21.15.82:0/1308553874 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f39d0003a10 con 0x55e4686072f0 2024-05-20T16:21:13.209 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:13.210+0000 7f39e59f8740 10 client.25333 fetch_fsmap learned FSMap version 27 2024-05-20T16:21:13.210 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:13.210+0000 7f39e59f8740 1 -- 172.21.15.82:0/1308553874 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55e468a4fd50 con 0x55e4686072f0 2024-05-20T16:21:13.212 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:13.212+0000 7f39bbfff640 1 -- 172.21.15.82:0/1308553874 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f39d001c880 con 0x55e4686072f0 2024-05-20T16:21:13.212 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:13.212+0000 7f39bbfff640 1 -- 172.21.15.82:0/1308553874 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f39d00046a0 con 0x55e4686072f0 2024-05-20T16:21:13.212 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:13.212+0000 7f39e59f8740 10 client.25333 fetch_fsmap finished waiting for FSMap version 27 2024-05-20T16:21:13.212 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:13.212+0000 7f39e59f8740 10 client.25333 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T16:21:13.212 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:13.212+0000 7f39e59f8740 20 client.25333 populate_metadata read hostname 'smithi082' 2024-05-20T16:21:13.213 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:13.212+0000 7f39e59f8740 1 --2- 172.21.15.82:0/1308553874 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x55e4685c41c0 0x55e468a52660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:21:13.213 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:13.212+0000 7f39e59f8740 4 client.25333 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-20T16:21:13.213 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:13.212+0000 7f39e59f8740 1 -- 172.21.15.82:0/1308553874 --> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e46893bdc0 con 0x55e4685c41c0 2024-05-20T16:21:13.213 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:13.213+0000 7f39d7fff640 1 --2- 172.21.15.82:0/1308553874 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x55e4685c41c0 0x55e468a52660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:21:13.216 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:13.217+0000 7f39d7fff640 1 --2- 172.21.15.82:0/1308553874 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x55e4685c41c0 0x55e468a52660 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bad68c774be88de2 server_cookie=abc8bacbdf9f205e in_seq=0 out_seq=0 2024-05-20T16:21:13.217 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:13.217+0000 7f39bbfff640 10 client.25333 ms_handle_connect on v2:172.21.15.160:6838/2765554482 2024-05-20T16:21:15.062 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:21:14 smithi160 ceph-mon[36347]: pgmap v580: 129 pgs: 129 active+clean; 10 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 729 MiB/s wr, 1.50k op/s 2024-05-20T16:21:15.114 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:21:14 smithi082 ceph-mon[32238]: pgmap v580: 129 pgs: 129 active+clean; 10 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 729 MiB/s wr, 1.50k op/s 2024-05-20T16:21:15.135 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:21:14 smithi184 ceph-mon[36209]: pgmap v580: 129 pgs: 129 active+clean; 10 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 729 MiB/s wr, 1.50k op/s 2024-05-20T16:21:15.700 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:15.699+0000 7f39bbfff640 1 -- 172.21.15.82:0/1308553874 <== mds.0 v2:172.21.15.160:6838/2765554482 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e46893bdc0 con 0x55e4685c41c0 2024-05-20T16:21:15.701 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:15.699+0000 7f39bbfff640 10 client.25333 handle_command_reply: tid=0 2024-05-20T16:21:15.701 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:15.699+0000 7f39e59f8740 1 client.25333 shutdown 2024-05-20T16:21:15.701 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:15.700+0000 7f39e59f8740 1 -- 172.21.15.82:0/1308553874 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e4686072f0 msgr2=0x55e4689dee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:21:15.701 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:15.700+0000 7f39e59f8740 1 --2- 172.21.15.82:0/1308553874 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e4686072f0 0x55e4689dee00 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f39d001cba0 tx=0x7f39d00096e0 comp rx=0 tx=0).stop 2024-05-20T16:21:15.701 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:15.700+0000 7f39e59f8740 1 -- 172.21.15.82:0/1308553874 shutdown_connections 2024-05-20T16:21:15.701 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:15.700+0000 7f39e59f8740 1 --2- 172.21.15.82:0/1308553874 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x55e4685c41c0 0x55e468a52660 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:21:15.701 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:15.700+0000 7f39e59f8740 1 --2- 172.21.15.82:0/1308553874 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e46860f4d0 0x55e4689df340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:21:15.701 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:15.700+0000 7f39e59f8740 1 --2- 172.21.15.82:0/1308553874 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e4686301b0 0x55e4689e0200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:21:15.701 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:15.700+0000 7f39e59f8740 1 --2- 172.21.15.82:0/1308553874 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e4686072f0 0x55e4689dee00 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:21:15.701 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:15.700+0000 7f39e59f8740 1 -- 172.21.15.82:0/1308553874 >> 172.21.15.82:0/1308553874 conn(0x55e468624300 msgr2=0x55e4689ca080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:21:15.701 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:15.700+0000 7f39bbfff640 0 client.25333 ms_handle_reset on v2:172.21.15.160:6838/2765554482 2024-05-20T16:21:15.702 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:15.700+0000 7f39e59f8740 1 -- 172.21.15.82:0/1308553874 shutdown_connections 2024-05-20T16:21:15.702 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:15.700+0000 7f39e59f8740 1 -- 172.21.15.82:0/1308553874 wait complete. 2024-05-20T16:21:15.702 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:15.700+0000 7f39e59f8740 20 client.25333 trim_cache size 0 max 16384 2024-05-20T16:21:15.702 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:{ 2024-05-20T16:21:15.702 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "message": "", 2024-05-20T16:21:15.702 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "return_code": 0 2024-05-20T16:21:15.702 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:} 2024-05-20T16:21:15.715 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:15.715+0000 7f39dea73640 1 -- 172.21.15.82:0/1227297270 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f39c80a2c60 msgr2=0x7f39c80a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:21:15.715 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:15.715+0000 7f39dea73640 1 --2- 172.21.15.82:0/1227297270 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f39c80a2c60 0x7f39c80a5170 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x55e4689daa40 tx=0x7f39c4000bc0 comp rx=0 tx=0).stop 2024-05-20T16:21:15.715 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:15.715+0000 7f39dea73640 1 -- 172.21.15.82:0/1227297270 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f39d8109280 msgr2=0x7f39d811aee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:21:15.715 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:15.715+0000 7f39dea73640 1 --2- 172.21.15.82:0/1227297270 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f39d8109280 0x7f39d811aee0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f39c4031440 tx=0x7f39c4032e30 comp rx=0 tx=0).stop 2024-05-20T16:21:15.715 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:15.715+0000 7f39dea73640 1 -- 172.21.15.82:0/1227297270 shutdown_connections 2024-05-20T16:21:15.715 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:15.715+0000 7f39dea73640 1 --2- 172.21.15.82:0/1227297270 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f39c80a2c60 0x7f39c80a5170 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:21:15.715 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:15.715+0000 7f39dea73640 1 --2- 172.21.15.82:0/1227297270 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f39d810f290 0x7f39d8119ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:21:15.715 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:15.715+0000 7f39dea73640 1 --2- 172.21.15.82:0/1227297270 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f39d810a500 0x7f39d8119580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:21:15.716 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:15.715+0000 7f39dea73640 1 --2- 172.21.15.82:0/1227297270 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f39d8109280 0x7f39d811aee0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:21:15.716 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:15.715+0000 7f39dea73640 1 -- 172.21.15.82:0/1227297270 >> 172.21.15.82:0/1227297270 conn(0x7f39d806f760 msgr2=0x7f39d810b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:21:15.716 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:15.716+0000 7f39dea73640 1 -- 172.21.15.82:0/1227297270 shutdown_connections 2024-05-20T16:21:15.716 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:15.717+0000 7f39dea73640 1 -- 172.21.15.82:0/1227297270 wait complete. 2024-05-20T16:21:17.062 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:21:16 smithi160 ceph-mon[36347]: pgmap v581: 129 pgs: 129 active+clean; 10 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 794 MiB/s wr, 1.58k op/s 2024-05-20T16:21:17.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:21:16 smithi082 ceph-mon[32238]: pgmap v581: 129 pgs: 129 active+clean; 10 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 794 MiB/s wr, 1.58k op/s 2024-05-20T16:21:17.138 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:21:16 smithi184 ceph-mon[36209]: pgmap v581: 129 pgs: 129 active+clean; 10 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 794 MiB/s wr, 1.58k op/s 2024-05-20T16:21:19.062 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:21:18 smithi160 ceph-mon[36347]: pgmap v582: 129 pgs: 129 active+clean; 10 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 653 MiB/s wr, 1.15k op/s 2024-05-20T16:21:19.114 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:21:18 smithi082 ceph-mon[32238]: pgmap v582: 129 pgs: 129 active+clean; 10 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 653 MiB/s wr, 1.15k op/s 2024-05-20T16:21:19.135 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:21:18 smithi184 ceph-mon[36209]: pgmap v582: 129 pgs: 129 active+clean; 10 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 653 MiB/s wr, 1.15k op/s 2024-05-20T16:21:20.062 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:21:19 smithi160 ceph-mon[36347]: pgmap v583: 129 pgs: 129 active+clean; 10 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 831 MiB/s wr, 1.55k op/s 2024-05-20T16:21:20.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:21:19 smithi082 ceph-mon[32238]: pgmap v583: 129 pgs: 129 active+clean; 10 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 831 MiB/s wr, 1.55k op/s 2024-05-20T16:21:20.136 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:21:19 smithi184 ceph-mon[36209]: pgmap v583: 129 pgs: 129 active+clean; 10 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 831 MiB/s wr, 1.55k op/s 2024-05-20T16:21:23.062 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:21:22 smithi160 ceph-mon[36347]: pgmap v584: 129 pgs: 129 active+clean; 10 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 747 MiB/s wr, 1.32k op/s 2024-05-20T16:21:23.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:21:22 smithi082 ceph-mon[32238]: pgmap v584: 129 pgs: 129 active+clean; 10 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 747 MiB/s wr, 1.32k op/s 2024-05-20T16:21:23.135 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:21:22 smithi184 ceph-mon[36209]: pgmap v584: 129 pgs: 129 active+clean; 10 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 747 MiB/s wr, 1.32k op/s 2024-05-20T16:21:25.062 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:21:24 smithi160 ceph-mon[36347]: pgmap v585: 129 pgs: 129 active+clean; 10 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 727 MiB/s wr, 1.32k op/s 2024-05-20T16:21:25.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:21:24 smithi082 ceph-mon[32238]: pgmap v585: 129 pgs: 129 active+clean; 10 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 727 MiB/s wr, 1.32k op/s 2024-05-20T16:21:25.135 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:21:24 smithi184 ceph-mon[36209]: pgmap v585: 129 pgs: 129 active+clean; 10 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 727 MiB/s wr, 1.32k op/s 2024-05-20T16:21:27.062 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:21:26 smithi160 ceph-mon[36347]: pgmap v586: 129 pgs: 129 active+clean; 10 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 794 MiB/s wr, 1.45k op/s 2024-05-20T16:21:27.110 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:21:26 smithi082 ceph-mon[32238]: pgmap v586: 129 pgs: 129 active+clean; 10 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 794 MiB/s wr, 1.45k op/s 2024-05-20T16:21:27.135 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:21:26 smithi184 ceph-mon[36209]: pgmap v586: 129 pgs: 129 active+clean; 10 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 794 MiB/s wr, 1.45k op/s 2024-05-20T16:21:29.025 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:29.025+0000 7fa9e34ae640 1 -- 172.21.15.82:0/3506576717 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa9dc106a50 msgr2=0x7fa9dc108e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:21:29.025 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:29.025+0000 7fa9e34ae640 1 --2- 172.21.15.82:0/3506576717 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa9dc106a50 0x7fa9dc108e70 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fa9d8009860 tx=0x7fa9d802a340 comp rx=0 tx=0).stop 2024-05-20T16:21:29.026 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:29.026+0000 7fa9e34ae640 1 -- 172.21.15.82:0/3506576717 shutdown_connections 2024-05-20T16:21:29.026 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:29.026+0000 7fa9e34ae640 1 --2- 172.21.15.82:0/3506576717 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa9dc106a50 0x7fa9dc108e70 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:21:29.026 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:29.026+0000 7fa9e34ae640 1 --2- 172.21.15.82:0/3506576717 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa9dc101d50 0x7fa9dc102160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:21:29.026 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:29.026+0000 7fa9e34ae640 1 --2- 172.21.15.82:0/3506576717 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa9dc100b10 0x7fa9dc101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:21:29.026 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:29.026+0000 7fa9e34ae640 1 -- 172.21.15.82:0/3506576717 >> 172.21.15.82:0/3506576717 conn(0x7fa9dc0fbd20 msgr2=0x7fa9dc0fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:21:29.030 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:29.028+0000 7fa9e34ae640 1 -- 172.21.15.82:0/3506576717 shutdown_connections 2024-05-20T16:21:29.034 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:29.031+0000 7fa9e34ae640 1 -- 172.21.15.82:0/3506576717 wait complete. 2024-05-20T16:21:29.034 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:29.033+0000 7fa9e34ae640 1 Processor -- start 2024-05-20T16:21:29.034 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:29.033+0000 7fa9e34ae640 1 -- start start 2024-05-20T16:21:29.034 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:29.033+0000 7fa9e34ae640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa9dc100b10 0x7fa9dc112190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:21:29.034 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:29.033+0000 7fa9e34ae640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa9dc101d50 0x7fa9dc112710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:21:29.034 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:29.033+0000 7fa9e34ae640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa9dc106a50 0x7fa9dc117680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:21:29.034 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:29.033+0000 7fa9e34ae640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fa9dc10be30 con 0x7fa9dc101d50 2024-05-20T16:21:29.034 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:29.033+0000 7fa9e34ae640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fa9dc10bcb0 con 0x7fa9dc106a50 2024-05-20T16:21:29.034 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:29.033+0000 7fa9e34ae640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa9dc10bfb0 con 0x7fa9dc100b10 2024-05-20T16:21:29.037 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:29.037+0000 7fa9e24ac640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa9dc100b10 0x7fa9dc112190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:21:29.037 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:29.037+0000 7fa9e2cad640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa9dc106a50 0x7fa9dc117680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:21:29.038 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:29.038+0000 7fa9e24ac640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa9dc100b10 0x7fa9dc112190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.82:39032/0 (socket says 172.21.15.82:39032) 2024-05-20T16:21:29.038 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:29.038+0000 7fa9e24ac640 1 -- 172.21.15.82:0/3782039960 learned_addr learned my addr 172.21.15.82:0/3782039960 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:21:29.041 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:29.041+0000 7fa9e1cab640 1 --2- 172.21.15.82:0/3782039960 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa9dc101d50 0x7fa9dc112710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:21:29.043 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:29.042+0000 7fa9e24ac640 1 -- 172.21.15.82:0/3782039960 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa9dc106a50 msgr2=0x7fa9dc117680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:21:29.043 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:29.042+0000 7fa9e24ac640 1 --2- 172.21.15.82:0/3782039960 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa9dc106a50 0x7fa9dc117680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:21:29.043 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:29.043+0000 7fa9e24ac640 1 -- 172.21.15.82:0/3782039960 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa9dc101d50 msgr2=0x7fa9dc112710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:21:29.043 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:29.043+0000 7fa9e24ac640 1 --2- 172.21.15.82:0/3782039960 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa9dc101d50 0x7fa9dc112710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:21:29.043 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:29.043+0000 7fa9e24ac640 1 -- 172.21.15.82:0/3782039960 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9dc117c70 con 0x7fa9dc100b10 2024-05-20T16:21:29.046 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:29.047+0000 7fa9e24ac640 1 --2- 172.21.15.82:0/3782039960 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa9dc100b10 0x7fa9dc112190 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fa9d4007bc0 tx=0x7fa9d400d490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:21:29.050 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:29.050+0000 7fa9d37fe640 1 -- 172.21.15.82:0/3782039960 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9d400dda0 con 0x7fa9dc100b10 2024-05-20T16:21:29.050 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:29.050+0000 7fa9d37fe640 1 -- 172.21.15.82:0/3782039960 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9d4018070 con 0x7fa9dc100b10 2024-05-20T16:21:29.050 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:29.051+0000 7fa9d37fe640 1 -- 172.21.15.82:0/3782039960 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9d4013660 con 0x7fa9dc100b10 2024-05-20T16:21:29.050 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:29.051+0000 7fa9e34ae640 1 -- 172.21.15.82:0/3782039960 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9dc117f60 con 0x7fa9dc100b10 2024-05-20T16:21:29.050 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:29.051+0000 7fa9e34ae640 1 -- 172.21.15.82:0/3782039960 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9dc1184b0 con 0x7fa9dc100b10 2024-05-20T16:21:29.058 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:29.058+0000 7fa9d37fe640 1 -- 172.21.15.82:0/3782039960 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fa9d4012070 con 0x7fa9dc100b10 2024-05-20T16:21:29.059 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:29.060+0000 7fa9d37fe640 1 --2- 172.21.15.82:0/3782039960 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fa9c80a2d90 0x7fa9c80a52a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:21:29.060 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:29.060+0000 7fa9d37fe640 1 -- 172.21.15.82:0/3782039960 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9150+0+0 (secure 0 0 0) 0x7fa9d40cf130 con 0x7fa9dc100b10 2024-05-20T16:21:29.061 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:29.061+0000 7fa9eb5ba740 1 Processor -- start 2024-05-20T16:21:29.062 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:21:28 smithi160 ceph-mon[36347]: pgmap v587: 129 pgs: 129 active+clean; 10 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 734 MiB/s wr, 1.37k op/s 2024-05-20T16:21:29.062 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:29.062+0000 7fa9eb5ba740 1 -- start start 2024-05-20T16:21:29.062 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:29.062+0000 7fa9eb5ba740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5639085291b0 0x5639088c9050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:21:29.062 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:29.062+0000 7fa9eb5ba740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5639085002f0 0x5639088c95d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:21:29.063 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:29.062+0000 7fa9eb5ba740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5639085084d0 0x5639088d04d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:21:29.063 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:29.062+0000 7fa9eb5ba740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x56390882d340 con 0x5639085002f0 2024-05-20T16:21:29.063 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:29.063+0000 7fa9eb5ba740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x5639087d5290 con 0x5639085291b0 2024-05-20T16:21:29.063 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:29.063+0000 7fa9eb5ba740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x563908829300 con 0x5639085084d0 2024-05-20T16:21:29.064 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:29.064+0000 7fa9e2cad640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5639085084d0 0x5639088d04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:21:29.064 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:29.064+0000 7fa9e2cad640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5639085084d0 0x5639088d04d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.82:39036/0 (socket says 172.21.15.82:39036) 2024-05-20T16:21:29.064 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:29.064+0000 7fa9e2cad640 1 -- 172.21.15.82:0/1734894019 learned_addr learned my addr 172.21.15.82:0/1734894019 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:21:29.065 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:29.064+0000 7fa9e1cab640 1 --2- 172.21.15.82:0/3782039960 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fa9c80a2d90 0x7fa9c80a52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:21:29.065 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:29.065+0000 7fa9e24ac640 1 --2- 172.21.15.82:0/1734894019 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5639085291b0 0x5639088c9050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:21:29.066 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:29.066+0000 7fa9e2cad640 1 -- 172.21.15.82:0/1734894019 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5639085291b0 msgr2=0x5639088c9050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:21:29.066 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:29.066+0000 7fa9e2cad640 1 --2- 172.21.15.82:0/1734894019 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5639085291b0 0x5639088c9050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:21:29.066 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:29.066+0000 7fa9e2cad640 1 -- 172.21.15.82:0/1734894019 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5639085002f0 msgr2=0x5639088c95d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:21:29.066 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:29.066+0000 7fa9e2cad640 1 --2- 172.21.15.82:0/1734894019 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5639085002f0 0x5639088c95d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:21:29.066 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:29.066+0000 7fa9e2cad640 1 -- 172.21.15.82:0/1734894019 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5639087f98e0 con 0x5639085084d0 2024-05-20T16:21:29.073 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:29.072+0000 7fa9e2cad640 1 --2- 172.21.15.82:0/1734894019 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5639085084d0 0x5639088d04d0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fa9d8009860 tx=0x7fa9d8030d90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=496b2f7e924ac7f9 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:21:29.081 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:29.078+0000 7fa9d0ff9640 1 -- 172.21.15.82:0/1734894019 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9d8037040 con 0x5639085084d0 2024-05-20T16:21:29.081 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:29.078+0000 7fa9d0ff9640 1 -- 172.21.15.82:0/1734894019 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9d8042070 con 0x5639085084d0 2024-05-20T16:21:29.082 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:29.078+0000 7fa9d0ff9640 1 -- 172.21.15.82:0/1734894019 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9d80046c0 con 0x5639085084d0 2024-05-20T16:21:29.082 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:29.078+0000 7fa9e1cab640 1 --2- 172.21.15.82:0/3782039960 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fa9c80a2d90 0x7fa9c80a52a0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x5639088ca430 tx=0x7fa9cc00a590 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:21:29.084 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:29.083+0000 7fa9eb5ba740 1 -- 172.21.15.82:0/1734894019 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5639085084d0 msgr2=0x5639088d04d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:21:29.084 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:29.083+0000 7fa9eb5ba740 1 --2- 172.21.15.82:0/1734894019 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5639085084d0 0x5639088d04d0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fa9d8009860 tx=0x7fa9d8030d90 comp rx=0 tx=0).stop 2024-05-20T16:21:29.087 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:29.086+0000 7fa9eb5ba740 1 -- 172.21.15.82:0/1734894019 shutdown_connections 2024-05-20T16:21:29.087 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:29.086+0000 7fa9eb5ba740 1 --2- 172.21.15.82:0/1734894019 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5639085002f0 0x5639088c95d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:21:29.087 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:29.086+0000 7fa9eb5ba740 1 --2- 172.21.15.82:0/1734894019 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5639085084d0 0x5639088d04d0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:21:29.087 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:29.086+0000 7fa9eb5ba740 1 --2- 172.21.15.82:0/1734894019 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5639085291b0 0x5639088c9050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:21:29.087 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:29.086+0000 7fa9eb5ba740 1 -- 172.21.15.82:0/1734894019 >> 172.21.15.82:0/1734894019 conn(0x56390851d300 msgr2=0x5639088c49e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:21:29.087 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:29.087+0000 7fa9eb5ba740 1 -- 172.21.15.82:0/1734894019 shutdown_connections 2024-05-20T16:21:29.092 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:29.090+0000 7fa9eb5ba740 1 -- 172.21.15.82:0/1734894019 wait complete. 2024-05-20T16:21:29.092 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:29.090+0000 7fa9eb5ba740 1 -- start start 2024-05-20T16:21:29.092 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:29.090+0000 7fa9eb5ba740 1 Processor -- start 2024-05-20T16:21:29.092 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:29.091+0000 7fa9eb5ba740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5639085002f0 0x5639088d2d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:21:29.092 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:29.091+0000 7fa9eb5ba740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5639085084d0 0x5639088d3290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:21:29.093 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:29.091+0000 7fa9eb5ba740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5639085291b0 0x5639088da1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:21:29.093 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:29.091+0000 7fa9eb5ba740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x5639088d0ec0 con 0x5639085084d0 2024-05-20T16:21:29.093 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:29.091+0000 7fa9eb5ba740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x5639087e8790 con 0x5639085291b0 2024-05-20T16:21:29.093 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:29.091+0000 7fa9eb5ba740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5639088d1040 con 0x5639085002f0 2024-05-20T16:21:29.093 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:29.091+0000 7fa9eb5ba740 10 client.0 _ll_register_callbacks cb 0x563908823a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T16:21:29.093 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:29.091+0000 7fa9e1cab640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5639085084d0 0x5639088d3290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:21:29.093 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:29.091+0000 7fa9e1cab640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5639085084d0 0x5639088d3290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:56390/0 (socket says 172.21.15.82:56390) 2024-05-20T16:21:29.093 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:29.091+0000 7fa9e1cab640 1 -- 172.21.15.82:0/4008603077 learned_addr learned my addr 172.21.15.82:0/4008603077 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:21:29.094 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:29.093+0000 7fa9e1cab640 1 -- 172.21.15.82:0/4008603077 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5639085002f0 msgr2=0x5639088d2d50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-20T16:21:29.094 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:29.093+0000 7fa9e1cab640 1 --2- 172.21.15.82:0/4008603077 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5639085002f0 0x5639088d2d50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:21:29.094 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:29.093+0000 7fa9e1cab640 1 -- 172.21.15.82:0/4008603077 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5639085291b0 msgr2=0x5639088da1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:21:29.094 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:29.093+0000 7fa9e1cab640 1 --2- 172.21.15.82:0/4008603077 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5639085291b0 0x5639088da1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:21:29.096 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:29.093+0000 7fa9e1cab640 1 -- 172.21.15.82:0/4008603077 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563908541210 con 0x5639085084d0 2024-05-20T16:21:29.097 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:29.093+0000 7fa9e1cab640 1 --2- 172.21.15.82:0/4008603077 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5639085084d0 0x5639088d3290 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fa9cc0129e0 tx=0x7fa9cc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d504408ec086db server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:21:29.097 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:29.094+0000 7fa9c5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-05-20T16:21:29.097 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:29.094+0000 7fa9c5ffb640 1 -- 172.21.15.82:0/4008603077 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9cc023020 con 0x5639085084d0 2024-05-20T16:21:29.097 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:29.094+0000 7fa9c5ffb640 1 -- 172.21.15.82:0/4008603077 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9150+0+0 (secure 0 0 0) 0x7fa9cc011b60 con 0x5639085084d0 2024-05-20T16:21:29.097 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:29.095+0000 7fa9eb5ba740 1 -- 172.21.15.82:0/4008603077 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa9d8042070 con 0x5639085084d0 2024-05-20T16:21:29.097 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:29.095+0000 7fa9c5ffb640 1 -- 172.21.15.82:0/4008603077 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa9cc011da0 con 0x5639085084d0 2024-05-20T16:21:29.097 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:29.095+0000 7fa9eb5ba740 10 client.15804 fetch_fsmap learned FSMap version 27 2024-05-20T16:21:29.097 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:29.095+0000 7fa9eb5ba740 1 -- 172.21.15.82:0/4008603077 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56390860d620 con 0x5639085084d0 2024-05-20T16:21:29.097 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:29.096+0000 7fa9c5ffb640 1 -- 172.21.15.82:0/4008603077 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9d8042070 con 0x5639085084d0 2024-05-20T16:21:29.097 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:29.096+0000 7fa9c5ffb640 1 -- 172.21.15.82:0/4008603077 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fa9cc027520 con 0x5639085084d0 2024-05-20T16:21:29.097 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:29.096+0000 7fa9eb5ba740 10 client.15804 fetch_fsmap finished waiting for FSMap version 27 2024-05-20T16:21:29.097 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:29.096+0000 7fa9eb5ba740 10 client.15804 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T16:21:29.098 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:29.096+0000 7fa9eb5ba740 20 client.15804 populate_metadata read hostname 'smithi082' 2024-05-20T16:21:29.098 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:29.096+0000 7fa9eb5ba740 1 --2- 172.21.15.82:0/4008603077 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x5639084e7d80 0x5639088e0c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:21:29.098 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:29.096+0000 7fa9eb5ba740 4 client.15804 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T16:21:29.098 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:29.096+0000 7fa9eb5ba740 1 -- 172.21.15.82:0/4008603077 --> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56390883aa70 con 0x5639084e7d80 2024-05-20T16:21:29.098 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:29.097+0000 7fa9e24ac640 1 --2- 172.21.15.82:0/4008603077 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x5639084e7d80 0x5639088e0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:21:29.099 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:29.100+0000 7fa9e24ac640 1 --2- 172.21.15.82:0/4008603077 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x5639084e7d80 0x5639088e0c90 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea4cbddcff5abb5a server_cookie=c727275a9a780c6e in_seq=0 out_seq=0 2024-05-20T16:21:29.099 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:29.100+0000 7fa9c5ffb640 10 client.15804 ms_handle_connect on v2:172.21.15.160:6838/2765554482 2024-05-20T16:21:29.103 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:29.103+0000 7fa9c5ffb640 1 -- 172.21.15.82:0/4008603077 <== mds.0 v2:172.21.15.160:6838/2765554482 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56390883aa70 con 0x5639084e7d80 2024-05-20T16:21:29.103 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:29.103+0000 7fa9c5ffb640 10 client.15804 handle_command_reply: tid=0 2024-05-20T16:21:29.103 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:29.103+0000 7fa9eb5ba740 1 client.15804 shutdown 2024-05-20T16:21:29.107 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:29.107+0000 7fa9eb5ba740 1 -- 172.21.15.82:0/4008603077 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5639085084d0 msgr2=0x5639088d3290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:21:29.107 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:29.107+0000 7fa9eb5ba740 1 --2- 172.21.15.82:0/4008603077 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5639085084d0 0x5639088d3290 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fa9cc0129e0 tx=0x7fa9cc0046c0 comp rx=0 tx=0).stop 2024-05-20T16:21:29.107 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:29.108+0000 7fa9eb5ba740 1 -- 172.21.15.82:0/4008603077 shutdown_connections 2024-05-20T16:21:29.107 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:29.108+0000 7fa9eb5ba740 1 --2- 172.21.15.82:0/4008603077 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x5639084e7d80 0x5639088e0c90 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:21:29.107 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:29.108+0000 7fa9eb5ba740 1 --2- 172.21.15.82:0/4008603077 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5639085084d0 0x5639088d3290 secure :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fa9cc0129e0 tx=0x7fa9cc0046c0 comp rx=0 tx=0).stop 2024-05-20T16:21:29.108 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:29.108+0000 7fa9eb5ba740 1 --2- 172.21.15.82:0/4008603077 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5639085291b0 0x5639088da1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:21:29.108 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:29.108+0000 7fa9eb5ba740 1 --2- 172.21.15.82:0/4008603077 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5639085002f0 0x5639088d2d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:21:29.108 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:29.108+0000 7fa9eb5ba740 1 -- 172.21.15.82:0/4008603077 >> 172.21.15.82:0/4008603077 conn(0x56390851d300 msgr2=0x5639088c3ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:21:29.112 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:21:28 smithi082 ceph-mon[32238]: pgmap v587: 129 pgs: 129 active+clean; 10 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 734 MiB/s wr, 1.37k op/s 2024-05-20T16:21:29.112 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:29.112+0000 7fa9c5ffb640 0 client.15804 ms_handle_reset on v2:172.21.15.160:6838/2765554482 2024-05-20T16:21:29.115 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:29.115+0000 7fa9eb5ba740 1 -- 172.21.15.82:0/4008603077 shutdown_connections 2024-05-20T16:21:29.117 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:29.116+0000 7fa9eb5ba740 1 -- 172.21.15.82:0/4008603077 wait complete. 2024-05-20T16:21:29.119 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:29.118+0000 7fa9eb5ba740 20 client.15804 trim_cache size 0 max 16384 2024-05-20T16:21:29.135 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:21:28 smithi184 ceph-mon[36209]: pgmap v587: 129 pgs: 129 active+clean; 10 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 734 MiB/s wr, 1.37k op/s 2024-05-20T16:21:29.172 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:29.172+0000 7fa9eb5ba740 1 Processor -- start 2024-05-20T16:21:29.172 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:29.172+0000 7fa9eb5ba740 1 -- start start 2024-05-20T16:21:29.173 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:29.173+0000 7fa9eb5ba740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5639085002f0 0x5639088c8fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:21:29.173 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:29.173+0000 7fa9eb5ba740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5639085291b0 0x5639088c9510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:21:29.173 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:29.174+0000 7fa9e24ac640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5639085002f0 0x5639088c8fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:21:29.173 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:29.174+0000 7fa9e24ac640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5639085002f0 0x5639088c8fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.82:49112/0 (socket says 172.21.15.82:49112) 2024-05-20T16:21:29.174 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:29.174+0000 7fa9e24ac640 1 -- 172.21.15.82:0/1033632685 learned_addr learned my addr 172.21.15.82:0/1033632685 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:21:29.174 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:29.174+0000 7fa9eb5ba740 1 --2- 172.21.15.82:0/1033632685 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5639084bd1c0 0x5639088cd1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:21:29.174 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:29.174+0000 7fa9e1cab640 1 --2- 172.21.15.82:0/1033632685 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5639085291b0 0x5639088c9510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:21:29.175 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:29.174+0000 7fa9e2cad640 1 --2- 172.21.15.82:0/1033632685 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5639084bd1c0 0x5639088cd1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:21:29.175 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:29.174+0000 7fa9eb5ba740 1 -- 172.21.15.82:0/1033632685 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fa9dc1ac000 con 0x5639084bd1c0 2024-05-20T16:21:29.175 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:29.174+0000 7fa9eb5ba740 1 -- 172.21.15.82:0/1033632685 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fa9dc1a98d0 con 0x5639085002f0 2024-05-20T16:21:29.175 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:29.174+0000 7fa9eb5ba740 1 -- 172.21.15.82:0/1033632685 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa9dc1a7720 con 0x5639085291b0 2024-05-20T16:21:29.176 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:29.176+0000 7fa9e1cab640 1 -- 172.21.15.82:0/1033632685 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5639085002f0 msgr2=0x5639088c8fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:21:29.176 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:29.176+0000 7fa9e1cab640 1 --2- 172.21.15.82:0/1033632685 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5639085002f0 0x5639088c8fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:21:29.176 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:29.176+0000 7fa9e1cab640 1 -- 172.21.15.82:0/1033632685 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5639084bd1c0 msgr2=0x5639088cd1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:21:29.176 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:29.176+0000 7fa9e1cab640 1 --2- 172.21.15.82:0/1033632685 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5639084bd1c0 0x5639088cd1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:21:29.176 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:29.176+0000 7fa9e1cab640 1 -- 172.21.15.82:0/1033632685 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5639088c8c00 con 0x5639085291b0 2024-05-20T16:21:29.176 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:29.177+0000 7fa9e1cab640 1 --2- 172.21.15.82:0/1033632685 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5639085291b0 0x5639088c9510 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fa9cc018000 tx=0x7fa9cc027e80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=837fbec6e18f0b4e server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:21:29.182 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:29.180+0000 7fa9c7fff640 1 -- 172.21.15.82:0/1033632685 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9cc036070 con 0x5639085291b0 2024-05-20T16:21:29.182 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:29.180+0000 7fa9c7fff640 1 -- 172.21.15.82:0/1033632685 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9cc018030 con 0x5639085291b0 2024-05-20T16:21:29.182 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:29.180+0000 7fa9c7fff640 1 -- 172.21.15.82:0/1033632685 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9cc023040 con 0x5639085291b0 2024-05-20T16:21:29.184 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:29.184+0000 7fa9eb5ba740 1 -- 172.21.15.82:0/1033632685 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5639085291b0 msgr2=0x5639088c9510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:21:29.184 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:29.185+0000 7fa9eb5ba740 1 --2- 172.21.15.82:0/1033632685 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5639085291b0 0x5639088c9510 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fa9cc018000 tx=0x7fa9cc027e80 comp rx=0 tx=0).stop 2024-05-20T16:21:29.190 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:29.190+0000 7fa9eb5ba740 1 -- 172.21.15.82:0/1033632685 shutdown_connections 2024-05-20T16:21:29.190 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:29.190+0000 7fa9eb5ba740 1 --2- 172.21.15.82:0/1033632685 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5639084bd1c0 0x5639088cd1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:21:29.190 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:29.190+0000 7fa9eb5ba740 1 --2- 172.21.15.82:0/1033632685 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5639085291b0 0x5639088c9510 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:21:29.190 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:29.190+0000 7fa9eb5ba740 1 --2- 172.21.15.82:0/1033632685 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5639085002f0 0x5639088c8fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:21:29.190 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:29.190+0000 7fa9eb5ba740 1 -- 172.21.15.82:0/1033632685 >> 172.21.15.82:0/1033632685 conn(0x56390851d300 msgr2=0x5639088d1410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:21:29.192 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:29.192+0000 7fa9eb5ba740 1 -- 172.21.15.82:0/1033632685 shutdown_connections 2024-05-20T16:21:29.192 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:29.192+0000 7fa9eb5ba740 1 -- 172.21.15.82:0/1033632685 wait complete. 2024-05-20T16:21:29.192 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:29.193+0000 7fa9eb5ba740 1 -- start start 2024-05-20T16:21:29.193 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:29.193+0000 7fa9eb5ba740 1 Processor -- start 2024-05-20T16:21:29.193 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:29.194+0000 7fa9eb5ba740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5639084bd1c0 0x5639088d42c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:21:29.194 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:29.194+0000 7fa9eb5ba740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5639085002f0 0x5639088d9750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:21:29.195 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:29.194+0000 7fa9e24ac640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5639084bd1c0 0x5639088d42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:21:29.195 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:29.194+0000 7fa9e24ac640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5639084bd1c0 0x5639088d42c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.82:49114/0 (socket says 172.21.15.82:49114) 2024-05-20T16:21:29.195 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:29.195+0000 7fa9e24ac640 1 -- 172.21.15.82:0/912012830 learned_addr learned my addr 172.21.15.82:0/912012830 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:21:29.195 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:29.195+0000 7fa9eb5ba740 1 --2- 172.21.15.82:0/912012830 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5639085291b0 0x5639088d4800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:21:29.196 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:29.195+0000 7fa9eb5ba740 1 -- 172.21.15.82:0/912012830 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fa9dc1a7d90 con 0x5639085002f0 2024-05-20T16:21:29.196 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:29.195+0000 7fa9eb5ba740 1 -- 172.21.15.82:0/912012830 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fa9dc1a9200 con 0x5639084bd1c0 2024-05-20T16:21:29.196 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:29.195+0000 7fa9e2cad640 1 --2- 172.21.15.82:0/912012830 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5639085291b0 0x5639088d4800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:21:29.196 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:29.195+0000 7fa9eb5ba740 1 -- 172.21.15.82:0/912012830 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa9dc1a8460 con 0x5639085291b0 2024-05-20T16:21:29.198 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:29.195+0000 7fa9eb5ba740 10 client.0 _ll_register_callbacks cb 0x7fa9cc00a7e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T16:21:29.198 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:29.196+0000 7fa9e2cad640 1 -- 172.21.15.82:0/912012830 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5639084bd1c0 msgr2=0x5639088d42c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:21:29.198 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:29.196+0000 7fa9e2cad640 1 --2- 172.21.15.82:0/912012830 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5639084bd1c0 0x5639088d42c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:21:29.198 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:29.196+0000 7fa9e2cad640 1 -- 172.21.15.82:0/912012830 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5639085002f0 msgr2=0x5639088d9750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:21:29.198 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:29.196+0000 7fa9e2cad640 1 --2- 172.21.15.82:0/912012830 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5639085002f0 0x5639088d9750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:21:29.198 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:29.196+0000 7fa9e2cad640 1 -- 172.21.15.82:0/912012830 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563908947030 con 0x5639085291b0 2024-05-20T16:21:29.198 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:29.196+0000 7fa9e2cad640 1 --2- 172.21.15.82:0/912012830 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5639085291b0 0x5639088d4800 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fa9d8009830 tx=0x7fa9d8030ae0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1d46ea4ac0647a5d server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:21:29.198 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:29.196+0000 7fa9eb5ba740 1 -- 172.21.15.82:0/912012830 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa9cc018030 con 0x5639085291b0 2024-05-20T16:21:29.198 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:29.196+0000 7fa9c5ffb640 10 client.25345 ms_handle_connect on v2:172.21.15.184:3300/0 2024-05-20T16:21:29.199 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:29.196+0000 7fa9c5ffb640 1 -- 172.21.15.82:0/912012830 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9d804a070 con 0x5639085291b0 2024-05-20T16:21:29.199 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:29.197+0000 7fa9c5ffb640 1 -- 172.21.15.82:0/912012830 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9150+0+0 (secure 0 0 0) 0x7fa9d8005040 con 0x5639085291b0 2024-05-20T16:21:29.199 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:29.197+0000 7fa9c5ffb640 1 -- 172.21.15.82:0/912012830 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa9d8056050 con 0x5639085291b0 2024-05-20T16:21:29.201 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:29.201+0000 7fa9eb5ba740 10 client.25345 fetch_fsmap learned FSMap version 27 2024-05-20T16:21:29.201 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:29.201+0000 7fa9eb5ba740 1 -- 172.21.15.82:0/912012830 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({fsmap=27}) -- 0x5639089480b0 con 0x5639085291b0 2024-05-20T16:21:29.202 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:29.203+0000 7fa9c5ffb640 1 -- 172.21.15.82:0/912012830 <== mon.1 v2:172.21.15.184:3300/0 4 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fa9d8005240 con 0x5639085291b0 2024-05-20T16:21:29.203 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:29.204+0000 7fa9eb5ba740 10 client.25345 fetch_fsmap finished waiting for FSMap version 27 2024-05-20T16:21:29.203 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:29.204+0000 7fa9eb5ba740 10 client.25345 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T16:21:29.203 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:29.204+0000 7fa9eb5ba740 20 client.25345 populate_metadata read hostname 'smithi082' 2024-05-20T16:21:29.204 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:29.204+0000 7fa9eb5ba740 1 --2- 172.21.15.82:0/912012830 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x563908948440 0x56390894a860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:21:29.204 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:29.204+0000 7fa9eb5ba740 4 client.25345 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-20T16:21:29.204 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:29.204+0000 7fa9eb5ba740 1 -- 172.21.15.82:0/912012830 --> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7fa9cc018030 con 0x563908948440 2024-05-20T16:21:29.205 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:29.205+0000 7fa9e24ac640 1 --2- 172.21.15.82:0/912012830 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x563908948440 0x56390894a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:21:29.209 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:29.210+0000 7fa9e24ac640 1 --2- 172.21.15.82:0/912012830 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x563908948440 0x56390894a860 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8e4c2fcf9e7f5dfc server_cookie=6a642e6184b3020f in_seq=0 out_seq=0 2024-05-20T16:21:29.210 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:29.211+0000 7fa9c5ffb640 10 client.25345 ms_handle_connect on v2:172.21.15.160:6838/2765554482 2024-05-20T16:21:31.062 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:21:30 smithi160 ceph-mon[36347]: pgmap v588: 129 pgs: 129 active+clean; 10 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 846 MiB/s wr, 1.60k op/s 2024-05-20T16:21:31.116 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:21:30 smithi082 ceph-mon[32238]: pgmap v588: 129 pgs: 129 active+clean; 10 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 846 MiB/s wr, 1.60k op/s 2024-05-20T16:21:31.135 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:21:30 smithi184 ceph-mon[36209]: pgmap v588: 129 pgs: 129 active+clean; 10 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 846 MiB/s wr, 1.60k op/s 2024-05-20T16:21:33.062 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:21:32 smithi160 ceph-mon[36347]: pgmap v589: 129 pgs: 129 active+clean; 10 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 769 MiB/s wr, 1.38k op/s 2024-05-20T16:21:33.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:21:32 smithi082 ceph-mon[32238]: pgmap v589: 129 pgs: 129 active+clean; 10 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 769 MiB/s wr, 1.38k op/s 2024-05-20T16:21:33.135 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:21:32 smithi184 ceph-mon[36209]: pgmap v589: 129 pgs: 129 active+clean; 10 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 769 MiB/s wr, 1.38k op/s 2024-05-20T16:21:34.062 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:21:33 smithi160 ceph-mon[36347]: pgmap v590: 129 pgs: 129 active+clean; 10 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 791 MiB/s wr, 1.43k op/s 2024-05-20T16:21:34.110 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:21:33 smithi082 ceph-mon[32238]: pgmap v590: 129 pgs: 129 active+clean; 10 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 791 MiB/s wr, 1.43k op/s 2024-05-20T16:21:34.135 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:21:33 smithi184 ceph-mon[36209]: pgmap v590: 129 pgs: 129 active+clean; 10 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 791 MiB/s wr, 1.43k op/s 2024-05-20T16:21:36.407 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:36.406+0000 7fa9c5ffb640 1 -- 172.21.15.82:0/912012830 <== mds.0 v2:172.21.15.160:6838/2765554482 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7fa9cc018030 con 0x563908948440 2024-05-20T16:21:36.407 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:36.406+0000 7fa9c5ffb640 10 client.25345 handle_command_reply: tid=0 2024-05-20T16:21:36.407 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:36.406+0000 7fa9eb5ba740 1 client.25345 shutdown 2024-05-20T16:21:36.408 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:36.407+0000 7fa9eb5ba740 1 -- 172.21.15.82:0/912012830 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5639085291b0 msgr2=0x5639088d4800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:21:36.408 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:36.407+0000 7fa9eb5ba740 1 --2- 172.21.15.82:0/912012830 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5639085291b0 0x5639088d4800 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fa9d8009830 tx=0x7fa9d8030ae0 comp rx=0 tx=0).stop 2024-05-20T16:21:36.408 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:36.407+0000 7fa9eb5ba740 1 -- 172.21.15.82:0/912012830 shutdown_connections 2024-05-20T16:21:36.408 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:36.407+0000 7fa9eb5ba740 1 --2- 172.21.15.82:0/912012830 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x563908948440 0x56390894a860 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:21:36.408 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:36.407+0000 7fa9eb5ba740 1 --2- 172.21.15.82:0/912012830 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5639085002f0 0x5639088d9750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:21:36.408 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:36.407+0000 7fa9eb5ba740 1 --2- 172.21.15.82:0/912012830 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5639085291b0 0x5639088d4800 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:21:36.408 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:36.407+0000 7fa9eb5ba740 1 --2- 172.21.15.82:0/912012830 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5639084bd1c0 0x5639088d42c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:21:36.408 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:36.407+0000 7fa9eb5ba740 1 -- 172.21.15.82:0/912012830 >> 172.21.15.82:0/912012830 conn(0x56390851d300 msgr2=0x5639088d1410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:21:36.408 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:36.407+0000 7fa9c5ffb640 0 client.25345 ms_handle_reset on v2:172.21.15.160:6838/2765554482 2024-05-20T16:21:36.408 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:36.407+0000 7fa9eb5ba740 1 -- 172.21.15.82:0/912012830 shutdown_connections 2024-05-20T16:21:36.408 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:36.407+0000 7fa9eb5ba740 1 -- 172.21.15.82:0/912012830 wait complete. 2024-05-20T16:21:36.408 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:36.407+0000 7fa9eb5ba740 20 client.25345 trim_cache size 0 max 16384 2024-05-20T16:21:36.409 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:{ 2024-05-20T16:21:36.409 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "message": "", 2024-05-20T16:21:36.409 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "return_code": 0 2024-05-20T16:21:36.409 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:} 2024-05-20T16:21:36.410 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:36.410+0000 7fa9e34ae640 1 -- 172.21.15.82:0/3782039960 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fa9c80a2d90 msgr2=0x7fa9c80a52a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:21:36.411 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:36.411+0000 7fa9e34ae640 1 --2- 172.21.15.82:0/3782039960 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fa9c80a2d90 0x7fa9c80a52a0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x5639088ca430 tx=0x7fa9cc00a590 comp rx=0 tx=0).stop 2024-05-20T16:21:36.411 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:36.411+0000 7fa9e34ae640 1 -- 172.21.15.82:0/3782039960 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa9dc100b10 msgr2=0x7fa9dc112190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:21:36.411 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:36.411+0000 7fa9e34ae640 1 --2- 172.21.15.82:0/3782039960 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa9dc100b10 0x7fa9dc112190 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fa9d4007bc0 tx=0x7fa9d400d490 comp rx=0 tx=0).stop 2024-05-20T16:21:36.418 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:36.418+0000 7fa9e34ae640 1 -- 172.21.15.82:0/3782039960 shutdown_connections 2024-05-20T16:21:36.418 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:36.418+0000 7fa9e34ae640 1 --2- 172.21.15.82:0/3782039960 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fa9c80a2d90 0x7fa9c80a52a0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:21:36.418 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:36.418+0000 7fa9e34ae640 1 --2- 172.21.15.82:0/3782039960 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa9dc101d50 0x7fa9dc112710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:21:36.418 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:36.418+0000 7fa9e34ae640 1 --2- 172.21.15.82:0/3782039960 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa9dc106a50 0x7fa9dc117680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:21:36.418 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:36.418+0000 7fa9e34ae640 1 --2- 172.21.15.82:0/3782039960 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa9dc100b10 0x7fa9dc112190 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:21:36.418 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:36.418+0000 7fa9e34ae640 1 -- 172.21.15.82:0/3782039960 >> 172.21.15.82:0/3782039960 conn(0x7fa9dc0fbd20 msgr2=0x7fa9dc102a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:21:36.419 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:36.420+0000 7fa9e34ae640 1 -- 172.21.15.82:0/3782039960 shutdown_connections 2024-05-20T16:21:36.420 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:36.420+0000 7fa9e34ae640 1 -- 172.21.15.82:0/3782039960 wait complete. 2024-05-20T16:21:37.062 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:21:36 smithi160 ceph-mon[36347]: pgmap v591: 129 pgs: 129 active+clean; 10 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 825 MiB/s wr, 1.52k op/s 2024-05-20T16:21:37.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:21:36 smithi082 ceph-mon[32238]: pgmap v591: 129 pgs: 129 active+clean; 10 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 825 MiB/s wr, 1.52k op/s 2024-05-20T16:21:37.135 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:21:36 smithi184 ceph-mon[36209]: pgmap v591: 129 pgs: 129 active+clean; 10 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 825 MiB/s wr, 1.52k op/s 2024-05-20T16:21:39.062 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:21:38 smithi160 ceph-mon[36347]: pgmap v592: 129 pgs: 129 active+clean; 10 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 738 MiB/s wr, 1.41k op/s 2024-05-20T16:21:39.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:21:38 smithi082 ceph-mon[32238]: pgmap v592: 129 pgs: 129 active+clean; 10 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 738 MiB/s wr, 1.41k op/s 2024-05-20T16:21:39.135 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:21:38 smithi184 ceph-mon[36209]: pgmap v592: 129 pgs: 129 active+clean; 10 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 738 MiB/s wr, 1.41k op/s 2024-05-20T16:21:41.062 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:21:40 smithi160 ceph-mon[36347]: pgmap v593: 129 pgs: 129 active+clean; 10 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 865 MiB/s wr, 1.66k op/s 2024-05-20T16:21:41.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:21:40 smithi082 ceph-mon[32238]: pgmap v593: 129 pgs: 129 active+clean; 10 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 865 MiB/s wr, 1.66k op/s 2024-05-20T16:21:41.135 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:21:40 smithi184 ceph-mon[36209]: pgmap v593: 129 pgs: 129 active+clean; 10 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 865 MiB/s wr, 1.66k op/s 2024-05-20T16:21:43.062 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:21:42 smithi160 ceph-mon[36347]: pgmap v594: 129 pgs: 129 active+clean; 10 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 761 MiB/s wr, 1.41k op/s 2024-05-20T16:21:43.111 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:21:42 smithi082 ceph-mon[32238]: pgmap v594: 129 pgs: 129 active+clean; 10 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 761 MiB/s wr, 1.41k op/s 2024-05-20T16:21:43.135 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:21:42 smithi184 ceph-mon[36209]: pgmap v594: 129 pgs: 129 active+clean; 10 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 761 MiB/s wr, 1.41k op/s 2024-05-20T16:21:45.062 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:21:44 smithi160 ceph-mon[36347]: pgmap v595: 129 pgs: 129 active+clean; 10 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 733 MiB/s wr, 1.34k op/s 2024-05-20T16:21:45.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:21:44 smithi082 ceph-mon[32238]: pgmap v595: 129 pgs: 129 active+clean; 10 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 733 MiB/s wr, 1.34k op/s 2024-05-20T16:21:45.135 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:21:44 smithi184 ceph-mon[36209]: pgmap v595: 129 pgs: 129 active+clean; 10 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 733 MiB/s wr, 1.34k op/s 2024-05-20T16:21:46.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:21:45 smithi082 ceph-mon[32238]: pgmap v596: 129 pgs: 129 active+clean; 10 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 840 MiB/s wr, 1.54k op/s 2024-05-20T16:21:46.135 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:21:45 smithi184 ceph-mon[36209]: pgmap v596: 129 pgs: 129 active+clean; 10 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 840 MiB/s wr, 1.54k op/s 2024-05-20T16:21:46.312 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:21:45 smithi160 ceph-mon[36347]: pgmap v596: 129 pgs: 129 active+clean; 10 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 840 MiB/s wr, 1.54k op/s 2024-05-20T16:21:49.062 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:21:48 smithi160 ceph-mon[36347]: pgmap v597: 129 pgs: 129 active+clean; 10 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 726 MiB/s wr, 1.37k op/s 2024-05-20T16:21:49.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:21:48 smithi082 ceph-mon[32238]: pgmap v597: 129 pgs: 129 active+clean; 10 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 726 MiB/s wr, 1.37k op/s 2024-05-20T16:21:49.135 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:21:48 smithi184 ceph-mon[36209]: pgmap v597: 129 pgs: 129 active+clean; 10 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 726 MiB/s wr, 1.37k op/s 2024-05-20T16:21:49.663 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:49.651+0000 7fc90b206640 1 -- 172.21.15.82:0/20614982 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc90410a300 msgr2=0x7fc90410a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:21:49.666 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:49.651+0000 7fc90b206640 1 --2- 172.21.15.82:0/20614982 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc90410a300 0x7fc90410a710 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fc8ec009900 tx=0x7fc8ec02a440 comp rx=0 tx=0).stop 2024-05-20T16:21:49.666 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:49.666+0000 7fc90b206640 1 -- 172.21.15.82:0/20614982 shutdown_connections 2024-05-20T16:21:49.666 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:49.666+0000 7fc90b206640 1 --2- 172.21.15.82:0/20614982 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc90410f020 0x7fc90410f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:21:49.666 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:49.666+0000 7fc90b206640 1 --2- 172.21.15.82:0/20614982 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc90410a300 0x7fc90410a710 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:21:49.666 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:49.666+0000 7fc90b206640 1 --2- 172.21.15.82:0/20614982 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc9041093d0 0x7fc904109dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:21:49.666 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:49.666+0000 7fc90b206640 1 -- 172.21.15.82:0/20614982 >> 172.21.15.82:0/20614982 conn(0x7fc90406f4f0 msgr2=0x7fc904071980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:21:49.670 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:49.668+0000 7fc90b206640 1 -- 172.21.15.82:0/20614982 shutdown_connections 2024-05-20T16:21:49.671 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:49.672+0000 7fc90b206640 1 -- 172.21.15.82:0/20614982 wait complete. 2024-05-20T16:21:49.672 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:49.673+0000 7fc90b206640 1 Processor -- start 2024-05-20T16:21:49.677 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:49.675+0000 7fc90b206640 1 -- start start 2024-05-20T16:21:49.677 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:49.676+0000 7fc90b206640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc9041093d0 0x7fc90411a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:21:49.677 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:49.676+0000 7fc90b206640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc90410a300 0x7fc904119090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:21:49.677 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:49.676+0000 7fc90b206640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc90410f020 0x7fc9041195d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:21:49.678 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:49.676+0000 7fc90b206640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fc9041114c0 con 0x7fc90410f020 2024-05-20T16:21:49.678 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:49.676+0000 7fc90b206640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fc904111340 con 0x7fc9041093d0 2024-05-20T16:21:49.678 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:49.676+0000 7fc90b206640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fc904111640 con 0x7fc90410a300 2024-05-20T16:21:49.678 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:49.676+0000 7fc903fff640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc90410a300 0x7fc904119090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:21:49.678 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:49.676+0000 7fc903fff640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc90410a300 0x7fc904119090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.82:37930/0 (socket says 172.21.15.82:37930) 2024-05-20T16:21:49.678 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:49.676+0000 7fc903fff640 1 -- 172.21.15.82:0/3330103489 learned_addr learned my addr 172.21.15.82:0/3330103489 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:21:49.679 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:49.679+0000 7fc9097a3640 1 --2- 172.21.15.82:0/3330103489 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc90410f020 0x7fc9041195d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:21:49.681 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:49.681+0000 7fc908fa2640 1 --2- 172.21.15.82:0/3330103489 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc9041093d0 0x7fc90411a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:21:49.681 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:49.681+0000 7fc903fff640 1 -- 172.21.15.82:0/3330103489 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc9041093d0 msgr2=0x7fc90411a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:21:49.681 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:49.681+0000 7fc903fff640 1 --2- 172.21.15.82:0/3330103489 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc9041093d0 0x7fc90411a9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:21:49.682 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:49.681+0000 7fc903fff640 1 -- 172.21.15.82:0/3330103489 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc90410f020 msgr2=0x7fc9041195d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:21:49.682 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:49.681+0000 7fc903fff640 1 --2- 172.21.15.82:0/3330103489 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc90410f020 0x7fc9041195d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:21:49.682 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:49.681+0000 7fc903fff640 1 -- 172.21.15.82:0/3330103489 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc904119b90 con 0x7fc90410a300 2024-05-20T16:21:49.682 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:49.683+0000 7fc903fff640 1 --2- 172.21.15.82:0/3330103489 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc90410a300 0x7fc904119090 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fc8ec033040 tx=0x7fc8ec005230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:21:49.685 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:49.685+0000 7fc901ffb640 1 -- 172.21.15.82:0/3330103489 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8ec040030 con 0x7fc90410a300 2024-05-20T16:21:49.686 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:49.685+0000 7fc90b206640 1 -- 172.21.15.82:0/3330103489 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc904119e20 con 0x7fc90410a300 2024-05-20T16:21:49.686 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:49.686+0000 7fc90b206640 1 -- 172.21.15.82:0/3330103489 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9041b8cb0 con 0x7fc90410a300 2024-05-20T16:21:49.686 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:49.686+0000 7fc901ffb640 1 -- 172.21.15.82:0/3330103489 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc8ec003e30 con 0x7fc90410a300 2024-05-20T16:21:49.686 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:49.686+0000 7fc901ffb640 1 -- 172.21.15.82:0/3330103489 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8ec03c940 con 0x7fc90410a300 2024-05-20T16:21:49.693 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:49.693+0000 7fc901ffb640 1 -- 172.21.15.82:0/3330103489 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fc8ec043030 con 0x7fc90410a300 2024-05-20T16:21:49.694 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:49.694+0000 7fc901ffb640 1 --2- 172.21.15.82:0/3330103489 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fc8f00a2d90 0x7fc8f00a52a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:21:49.695 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:49.695+0000 7fc901ffb640 1 -- 172.21.15.82:0/3330103489 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9150+0+0 (secure 0 0 0) 0x7fc8ec0efdc0 con 0x7fc90410a300 2024-05-20T16:21:49.698 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:49.699+0000 7fc912187740 1 Processor -- start 2024-05-20T16:21:49.699 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:49.699+0000 7fc912187740 1 -- start start 2024-05-20T16:21:49.701 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:49.701+0000 7fc912187740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f9f8c111b0 0x55f9f8fb0e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:21:49.701 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:49.701+0000 7fc912187740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f9f8be82f0 0x55f9f8fb13e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:21:49.701 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:49.701+0000 7fc912187740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f9f8bf04d0 0x55f9f8fb82e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:21:49.701 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:49.701+0000 7fc912187740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55f9f8f15340 con 0x55f9f8be82f0 2024-05-20T16:21:49.701 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:49.701+0000 7fc912187740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x55f9f8ebd290 con 0x55f9f8c111b0 2024-05-20T16:21:49.701 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:49.702+0000 7fc912187740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55f9f8f11300 con 0x55f9f8bf04d0 2024-05-20T16:21:49.702 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:49.703+0000 7fc9097a3640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f9f8bf04d0 0x55f9f8fb82e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:21:49.703 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:49.703+0000 7fc9097a3640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f9f8bf04d0 0x55f9f8fb82e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.82:37940/0 (socket says 172.21.15.82:37940) 2024-05-20T16:21:49.703 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:49.703+0000 7fc9097a3640 1 -- 172.21.15.82:0/1444012485 learned_addr learned my addr 172.21.15.82:0/1444012485 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:21:49.703 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:49.704+0000 7fc903fff640 1 --2- 172.21.15.82:0/1444012485 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f9f8be82f0 0x55f9f8fb13e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:21:49.704 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:49.704+0000 7fc903fff640 1 -- 172.21.15.82:0/1444012485 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f9f8bf04d0 msgr2=0x55f9f8fb82e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:21:49.704 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:49.704+0000 7fc903fff640 1 --2- 172.21.15.82:0/1444012485 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f9f8bf04d0 0x55f9f8fb82e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:21:49.705 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:49.704+0000 7fc903fff640 1 -- 172.21.15.82:0/1444012485 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f9f8c111b0 msgr2=0x55f9f8fb0e60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-20T16:21:49.705 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:49.704+0000 7fc903fff640 1 --2- 172.21.15.82:0/1444012485 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f9f8c111b0 0x55f9f8fb0e60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:21:49.705 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:49.704+0000 7fc903fff640 1 -- 172.21.15.82:0/1444012485 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f9f8ee18e0 con 0x55f9f8be82f0 2024-05-20T16:21:49.705 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:49.705+0000 7fc908fa2640 1 --2- 172.21.15.82:0/3330103489 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fc8f00a2d90 0x7fc8f00a52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:21:49.713 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:49.714+0000 7fc903fff640 1 --2- 172.21.15.82:0/1444012485 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f9f8be82f0 0x55f9f8fb13e0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fc8ec009850 tx=0x7fc8ec0bbd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20e008f1a556cf54 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:21:49.723 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:49.723+0000 7fc8faffd640 1 -- 172.21.15.82:0/1444012485 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8ec04a040 con 0x55f9f8be82f0 2024-05-20T16:21:49.723 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:49.724+0000 7fc8faffd640 1 -- 172.21.15.82:0/1444012485 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc8ec0043d0 con 0x55f9f8be82f0 2024-05-20T16:21:49.725 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:49.725+0000 7fc912187740 1 -- 172.21.15.82:0/1444012485 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f9f8be82f0 msgr2=0x55f9f8fb13e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:21:49.725 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:49.725+0000 7fc912187740 1 --2- 172.21.15.82:0/1444012485 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f9f8be82f0 0x55f9f8fb13e0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fc8ec009850 tx=0x7fc8ec0bbd70 comp rx=0 tx=0).stop 2024-05-20T16:21:49.725 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:49.725+0000 7fc912187740 1 -- 172.21.15.82:0/1444012485 shutdown_connections 2024-05-20T16:21:49.725 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:49.725+0000 7fc912187740 1 --2- 172.21.15.82:0/1444012485 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f9f8be82f0 0x55f9f8fb13e0 secure :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fc8ec009850 tx=0x7fc8ec0bbd70 comp rx=0 tx=0).stop 2024-05-20T16:21:49.725 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:49.725+0000 7fc912187740 1 --2- 172.21.15.82:0/1444012485 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f9f8bf04d0 0x55f9f8fb82e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:21:49.725 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:49.725+0000 7fc912187740 1 --2- 172.21.15.82:0/1444012485 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f9f8c111b0 0x55f9f8fb0e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:21:49.725 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:49.725+0000 7fc912187740 1 -- 172.21.15.82:0/1444012485 >> 172.21.15.82:0/1444012485 conn(0x55f9f8c05300 msgr2=0x55f9f8fac7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:21:49.728 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:49.729+0000 7fc908fa2640 1 --2- 172.21.15.82:0/3330103489 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fc8f00a2d90 0x7fc8f00a52a0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x55f9f8fb2030 tx=0x7fc8fc008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:21:49.729 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:49.729+0000 7fc912187740 1 -- 172.21.15.82:0/1444012485 shutdown_connections 2024-05-20T16:21:49.730 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:49.729+0000 7fc912187740 1 -- 172.21.15.82:0/1444012485 wait complete. 2024-05-20T16:21:49.730 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:49.730+0000 7fc912187740 1 -- start start 2024-05-20T16:21:49.730 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:49.730+0000 7fc912187740 1 Processor -- start 2024-05-20T16:21:49.731 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:49.731+0000 7fc912187740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f9f8bf04d0 0x55f9f8fbaab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:21:49.731 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:49.731+0000 7fc912187740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f9f8c111b0 0x55f9f8fbaff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:21:49.731 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:49.731+0000 7fc912187740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f9f8bcfd80 0x55f9f8fc1f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:21:49.731 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:49.731+0000 7fc912187740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55f9f8fb8af0 con 0x55f9f8bcfd80 2024-05-20T16:21:49.731 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:49.731+0000 7fc912187740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x55f9f8ed0790 con 0x55f9f8bf04d0 2024-05-20T16:21:49.731 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:49.731+0000 7fc912187740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55f9f8fb8c70 con 0x55f9f8c111b0 2024-05-20T16:21:49.732 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:49.732+0000 7fc908fa2640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f9f8bf04d0 0x55f9f8fbaab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:21:49.732 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:49.731+0000 7fc912187740 10 client.0 _ll_register_callbacks cb 0x55f9f8f0ba20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T16:21:49.734 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:49.734+0000 7fc903fff640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f9f8c111b0 0x55f9f8fbaff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:21:49.734 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:49.734+0000 7fc908fa2640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f9f8bf04d0 0x55f9f8fbaab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.82:53274/0 (socket says 172.21.15.82:53274) 2024-05-20T16:21:49.734 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:49.734+0000 7fc908fa2640 1 -- 172.21.15.82:0/202849317 learned_addr learned my addr 172.21.15.82:0/202849317 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:21:49.734 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:49.735+0000 7fc9097a3640 1 --2- 172.21.15.82:0/202849317 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f9f8bcfd80 0x55f9f8fc1f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:21:49.737 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:49.737+0000 7fc9097a3640 1 -- 172.21.15.82:0/202849317 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f9f8c111b0 msgr2=0x55f9f8fbaff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:21:49.737 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:49.738+0000 7fc9097a3640 1 --2- 172.21.15.82:0/202849317 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f9f8c111b0 0x55f9f8fbaff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:21:49.738 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:49.738+0000 7fc9097a3640 1 -- 172.21.15.82:0/202849317 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f9f8bf04d0 msgr2=0x55f9f8fbaab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:21:49.738 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:49.738+0000 7fc9097a3640 1 --2- 172.21.15.82:0/202849317 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f9f8bf04d0 0x55f9f8fbaab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:21:49.738 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:49.738+0000 7fc9097a3640 1 -- 172.21.15.82:0/202849317 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f9f8c29210 con 0x55f9f8bcfd80 2024-05-20T16:21:49.747 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:49.747+0000 7fc9097a3640 1 --2- 172.21.15.82:0/202849317 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f9f8bcfd80 0x55f9f8fc1f50 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fc8f4009fc0 tx=0x7fc8f4003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd22a7e127f05f01 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:21:49.747 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:49.748+0000 7fc8e3fff640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-05-20T16:21:49.750 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:49.750+0000 7fc8e3fff640 1 -- 172.21.15.82:0/202849317 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8f40118b0 con 0x55f9f8bcfd80 2024-05-20T16:21:49.750 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:49.751+0000 7fc912187740 1 -- 172.21.15.82:0/202849317 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc8ec0043d0 con 0x55f9f8bcfd80 2024-05-20T16:21:49.751 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:49.751+0000 7fc8e3fff640 1 -- 172.21.15.82:0/202849317 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9150+0+0 (secure 0 0 0) 0x7fc8f4011a50 con 0x55f9f8bcfd80 2024-05-20T16:21:49.753 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:49.752+0000 7fc8e3fff640 1 -- 172.21.15.82:0/202849317 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fc8f4011c90 con 0x55f9f8bcfd80 2024-05-20T16:21:49.753 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:49.753+0000 7fc912187740 10 client.15816 fetch_fsmap learned FSMap version 27 2024-05-20T16:21:49.753 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:49.753+0000 7fc912187740 1 -- 172.21.15.82:0/202849317 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55f9f8cf5620 con 0x55f9f8bcfd80 2024-05-20T16:21:49.759 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:49.759+0000 7fc8e3fff640 1 -- 172.21.15.82:0/202849317 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc8ec0043d0 con 0x55f9f8bcfd80 2024-05-20T16:21:49.760 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:49.759+0000 7fc8e3fff640 1 -- 172.21.15.82:0/202849317 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fc8f40105d0 con 0x55f9f8bcfd80 2024-05-20T16:21:49.762 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:49.762+0000 7fc912187740 10 client.15816 fetch_fsmap finished waiting for FSMap version 27 2024-05-20T16:21:49.762 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:49.762+0000 7fc912187740 10 client.15816 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T16:21:49.762 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:49.762+0000 7fc912187740 20 client.15816 populate_metadata read hostname 'smithi082' 2024-05-20T16:21:49.762 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:49.762+0000 7fc912187740 1 --2- 172.21.15.82:0/202849317 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x55f9f8ba51c0 0x55f9f8fc8420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:21:49.763 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:49.762+0000 7fc912187740 4 client.15816 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T16:21:49.763 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:49.762+0000 7fc912187740 1 -- 172.21.15.82:0/202849317 --> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f9f8f22a70 con 0x55f9f8ba51c0 2024-05-20T16:21:49.764 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:49.764+0000 7fc908fa2640 1 --2- 172.21.15.82:0/202849317 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x55f9f8ba51c0 0x55f9f8fc8420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:21:49.767 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:49.768+0000 7fc908fa2640 1 --2- 172.21.15.82:0/202849317 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x55f9f8ba51c0 0x55f9f8fc8420 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=37f73274ac473187 server_cookie=f7f6506d385f18a3 in_seq=0 out_seq=0 2024-05-20T16:21:49.769 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:49.769+0000 7fc8e3fff640 10 client.15816 ms_handle_connect on v2:172.21.15.160:6838/2765554482 2024-05-20T16:21:49.776 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:49.776+0000 7fc8e3fff640 1 -- 172.21.15.82:0/202849317 <== mds.0 v2:172.21.15.160:6838/2765554482 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55f9f8f22a70 con 0x55f9f8ba51c0 2024-05-20T16:21:49.776 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:49.776+0000 7fc8e3fff640 10 client.15816 handle_command_reply: tid=0 2024-05-20T16:21:49.777 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:49.776+0000 7fc912187740 1 client.15816 shutdown 2024-05-20T16:21:49.778 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:49.779+0000 7fc912187740 1 -- 172.21.15.82:0/202849317 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f9f8bcfd80 msgr2=0x55f9f8fc1f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:21:49.779 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:49.779+0000 7fc912187740 1 --2- 172.21.15.82:0/202849317 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f9f8bcfd80 0x55f9f8fc1f50 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fc8f4009fc0 tx=0x7fc8f4003f90 comp rx=0 tx=0).stop 2024-05-20T16:21:49.782 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:49.783+0000 7fc912187740 1 -- 172.21.15.82:0/202849317 shutdown_connections 2024-05-20T16:21:49.783 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:49.783+0000 7fc912187740 1 --2- 172.21.15.82:0/202849317 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x55f9f8ba51c0 0x55f9f8fc8420 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:21:49.783 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:49.783+0000 7fc912187740 1 --2- 172.21.15.82:0/202849317 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f9f8bcfd80 0x55f9f8fc1f50 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:21:49.783 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:49.783+0000 7fc912187740 1 --2- 172.21.15.82:0/202849317 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f9f8c111b0 0x55f9f8fbaff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:21:49.783 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:49.783+0000 7fc912187740 1 --2- 172.21.15.82:0/202849317 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f9f8bf04d0 0x55f9f8fbaab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:21:49.783 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:49.783+0000 7fc912187740 1 -- 172.21.15.82:0/202849317 >> 172.21.15.82:0/202849317 conn(0x55f9f8c05300 msgr2=0x55f9f8faf7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:21:49.786 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:49.786+0000 7fc8e3fff640 0 client.15816 ms_handle_reset on v2:172.21.15.160:6838/2765554482 2024-05-20T16:21:49.788 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:49.788+0000 7fc912187740 1 -- 172.21.15.82:0/202849317 shutdown_connections 2024-05-20T16:21:49.788 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:49.788+0000 7fc912187740 1 -- 172.21.15.82:0/202849317 wait complete. 2024-05-20T16:21:49.791 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:49.790+0000 7fc912187740 20 client.15816 trim_cache size 0 max 16384 2024-05-20T16:21:49.837 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:49.837+0000 7fc912187740 1 Processor -- start 2024-05-20T16:21:49.838 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:49.838+0000 7fc912187740 1 -- start start 2024-05-20T16:21:49.838 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:49.838+0000 7fc912187740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f9f8bcfd80 0x55f9f8fb9780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:21:49.838 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:49.838+0000 7fc912187740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f9f8bf04d0 0x55f9f8fb5760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:21:49.838 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:49.838+0000 7fc912187740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f9f8c111b0 0x55f9f8fc1f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:21:49.838 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:49.838+0000 7fc912187740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fc8e40125f0 con 0x55f9f8c111b0 2024-05-20T16:21:49.838 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:49.838+0000 7fc912187740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fc8e400fec0 con 0x55f9f8bcfd80 2024-05-20T16:21:49.838 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:49.838+0000 7fc912187740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fc8e400dd10 con 0x55f9f8bf04d0 2024-05-20T16:21:49.839 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:49.839+0000 7fc9097a3640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f9f8c111b0 0x55f9f8fc1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:21:49.839 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:49.839+0000 7fc9097a3640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f9f8c111b0 0x55f9f8fc1f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:50142/0 (socket says 172.21.15.82:50142) 2024-05-20T16:21:49.839 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:49.839+0000 7fc9097a3640 1 -- 172.21.15.82:0/4284649711 learned_addr learned my addr 172.21.15.82:0/4284649711 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:21:49.839 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:49.839+0000 7fc903fff640 1 --2- 172.21.15.82:0/4284649711 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f9f8bf04d0 0x55f9f8fb5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:21:49.840 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:49.841+0000 7fc908fa2640 1 --2- 172.21.15.82:0/4284649711 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f9f8bcfd80 0x55f9f8fb9780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:21:49.841 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:49.842+0000 7fc9097a3640 1 -- 172.21.15.82:0/4284649711 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f9f8bf04d0 msgr2=0x55f9f8fb5760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:21:49.842 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:49.842+0000 7fc9097a3640 1 --2- 172.21.15.82:0/4284649711 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f9f8bf04d0 0x55f9f8fb5760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:21:49.842 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:49.842+0000 7fc9097a3640 1 -- 172.21.15.82:0/4284649711 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f9f8bcfd80 msgr2=0x55f9f8fb9780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:21:49.842 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:49.842+0000 7fc9097a3640 1 --2- 172.21.15.82:0/4284649711 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f9f8bcfd80 0x55f9f8fb9780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:21:49.842 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:49.842+0000 7fc9097a3640 1 -- 172.21.15.82:0/4284649711 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f9f8fad420 con 0x55f9f8c111b0 2024-05-20T16:21:49.844 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:49.844+0000 7fc9097a3640 1 --2- 172.21.15.82:0/4284649711 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f9f8c111b0 0x55f9f8fc1f10 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fc8f4003e30 tx=0x7fc8f4003e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bdfe1621f0ef968 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:21:49.846 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:49.846+0000 7fc8fa7fc640 1 -- 172.21.15.82:0/4284649711 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8f402e030 con 0x55f9f8c111b0 2024-05-20T16:21:49.846 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:49.846+0000 7fc8fa7fc640 1 -- 172.21.15.82:0/4284649711 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc8f4010cd0 con 0x55f9f8c111b0 2024-05-20T16:21:49.847 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:49.848+0000 7fc912187740 1 -- 172.21.15.82:0/4284649711 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f9f8c111b0 msgr2=0x55f9f8fc1f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:21:49.847 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:49.848+0000 7fc912187740 1 --2- 172.21.15.82:0/4284649711 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f9f8c111b0 0x55f9f8fc1f10 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fc8f4003e30 tx=0x7fc8f4003e60 comp rx=0 tx=0).stop 2024-05-20T16:21:49.850 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:49.850+0000 7fc912187740 1 -- 172.21.15.82:0/4284649711 shutdown_connections 2024-05-20T16:21:49.850 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:49.850+0000 7fc912187740 1 --2- 172.21.15.82:0/4284649711 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f9f8c111b0 0x55f9f8fc1f10 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:21:49.850 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:49.850+0000 7fc912187740 1 --2- 172.21.15.82:0/4284649711 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f9f8bf04d0 0x55f9f8fb5760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:21:49.850 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:49.850+0000 7fc912187740 1 --2- 172.21.15.82:0/4284649711 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f9f8bcfd80 0x55f9f8fb9780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:21:49.850 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:49.850+0000 7fc912187740 1 -- 172.21.15.82:0/4284649711 >> 172.21.15.82:0/4284649711 conn(0x55f9f8c05300 msgr2=0x55f9f8fb9070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:21:49.851 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:49.851+0000 7fc912187740 1 -- 172.21.15.82:0/4284649711 shutdown_connections 2024-05-20T16:21:49.851 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:49.851+0000 7fc912187740 1 -- 172.21.15.82:0/4284649711 wait complete. 2024-05-20T16:21:49.851 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:49.851+0000 7fc912187740 1 -- start start 2024-05-20T16:21:49.851 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:49.851+0000 7fc912187740 1 Processor -- start 2024-05-20T16:21:49.852 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:49.852+0000 7fc912187740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f9f8bcfd80 0x55f9f8fc25c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:21:49.852 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:49.852+0000 7fc912187740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f9f8bf04d0 0x55f9f8fc0c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:21:49.852 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:49.852+0000 7fc912187740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f9f8c111b0 0x55f9f8fc1170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:21:49.852 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:49.852+0000 7fc912187740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fc8e400e380 con 0x55f9f8bcfd80 2024-05-20T16:21:49.852 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:49.852+0000 7fc912187740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fc8e400f7f0 con 0x55f9f8c111b0 2024-05-20T16:21:49.852 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:49.852+0000 7fc912187740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fc8e400ea50 con 0x55f9f8bf04d0 2024-05-20T16:21:49.852 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:49.852+0000 7fc912187740 10 client.0 _ll_register_callbacks cb 0x7fc8f4009c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T16:21:49.852 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:49.852+0000 7fc908fa2640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f9f8bcfd80 0x55f9f8fc25c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:21:49.852 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:49.852+0000 7fc908fa2640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f9f8bcfd80 0x55f9f8fc25c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:50152/0 (socket says 172.21.15.82:50152) 2024-05-20T16:21:49.852 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:49.852+0000 7fc908fa2640 1 -- 172.21.15.82:0/179902753 learned_addr learned my addr 172.21.15.82:0/179902753 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:21:49.853 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:49.853+0000 7fc903fff640 1 --2- 172.21.15.82:0/179902753 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f9f8bf04d0 0x55f9f8fc0c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:21:49.853 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:49.853+0000 7fc908fa2640 1 -- 172.21.15.82:0/179902753 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f9f8bf04d0 msgr2=0x55f9f8fc0c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:21:49.853 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:49.853+0000 7fc908fa2640 1 --2- 172.21.15.82:0/179902753 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f9f8bf04d0 0x55f9f8fc0c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:21:49.853 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:49.853+0000 7fc908fa2640 1 -- 172.21.15.82:0/179902753 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f9f8c111b0 msgr2=0x55f9f8fc1170 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-20T16:21:49.853 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:49.853+0000 7fc908fa2640 1 --2- 172.21.15.82:0/179902753 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f9f8c111b0 0x55f9f8fc1170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:21:49.853 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:49.853+0000 7fc908fa2640 1 -- 172.21.15.82:0/179902753 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f9f8fc18d0 con 0x55f9f8bcfd80 2024-05-20T16:21:49.854 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:49.854+0000 7fc908fa2640 1 --2- 172.21.15.82:0/179902753 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f9f8bcfd80 0x55f9f8fc25c0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fc8fc00cea0 tx=0x7fc8fc0047d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65967197599ecb18 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:21:49.854 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:49.854+0000 7fc8e3fff640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-05-20T16:21:49.856 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:49.854+0000 7fc8e3fff640 1 -- 172.21.15.82:0/179902753 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8fc013e30 con 0x55f9f8bcfd80 2024-05-20T16:21:49.856 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:49.854+0000 7fc912187740 1 -- 172.21.15.82:0/179902753 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc8f4010cd0 con 0x55f9f8bcfd80 2024-05-20T16:21:49.856 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:49.854+0000 7fc8e3fff640 1 -- 172.21.15.82:0/179902753 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9150+0+0 (secure 0 0 0) 0x7fc8fc005bd0 con 0x55f9f8bcfd80 2024-05-20T16:21:49.856 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:49.855+0000 7fc8e3fff640 1 -- 172.21.15.82:0/179902753 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fc8fc034050 con 0x55f9f8bcfd80 2024-05-20T16:21:49.856 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:49.855+0000 7fc912187740 10 client.15822 fetch_fsmap learned FSMap version 27 2024-05-20T16:21:49.856 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:49.855+0000 7fc912187740 1 -- 172.21.15.82:0/179902753 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55f9f902f800 con 0x55f9f8bcfd80 2024-05-20T16:21:49.856 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:49.855+0000 7fc8e3fff640 1 -- 172.21.15.82:0/179902753 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc8f4010cd0 con 0x55f9f8bcfd80 2024-05-20T16:21:49.856 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:49.856+0000 7fc8e3fff640 1 -- 172.21.15.82:0/179902753 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fc8fc0251f0 con 0x55f9f8bcfd80 2024-05-20T16:21:49.856 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:49.856+0000 7fc912187740 10 client.15822 fetch_fsmap finished waiting for FSMap version 27 2024-05-20T16:21:49.856 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:49.856+0000 7fc912187740 10 client.15822 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T16:21:49.856 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:49.856+0000 7fc912187740 20 client.15822 populate_metadata read hostname 'smithi082' 2024-05-20T16:21:49.856 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:49.856+0000 7fc912187740 1 --2- 172.21.15.82:0/179902753 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x55f9f90301c0 0x55f9f90326b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:21:49.856 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:49.856+0000 7fc912187740 4 client.15822 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-20T16:21:49.857 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:49.856+0000 7fc912187740 1 -- 172.21.15.82:0/179902753 --> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f9f8f1cdc0 con 0x55f9f90301c0 2024-05-20T16:21:49.857 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:49.857+0000 7fc903fff640 1 --2- 172.21.15.82:0/179902753 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x55f9f90301c0 0x55f9f90326b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:21:49.858 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:49.858+0000 7fc8e3fff640 10 client.15822 ms_handle_connect on v2:172.21.15.160:6838/2765554482 2024-05-20T16:21:49.859 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:49.858+0000 7fc903fff640 1 --2- 172.21.15.82:0/179902753 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x55f9f90301c0 0x55f9f90326b0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=47cb586ccf08651a server_cookie=4b6902ce88cbcf24 in_seq=0 out_seq=0 2024-05-20T16:21:51.062 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:21:50 smithi160 ceph-mon[36347]: pgmap v598: 129 pgs: 129 active+clean; 10 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 817 MiB/s wr, 1.74k op/s 2024-05-20T16:21:51.110 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:21:50 smithi082 ceph-mon[32238]: pgmap v598: 129 pgs: 129 active+clean; 10 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 817 MiB/s wr, 1.74k op/s 2024-05-20T16:21:51.135 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:21:50 smithi184 ceph-mon[36209]: pgmap v598: 129 pgs: 129 active+clean; 10 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 817 MiB/s wr, 1.74k op/s 2024-05-20T16:21:51.300 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:51.300+0000 7fc8e3fff640 1 -- 172.21.15.82:0/179902753 <== mds.0 v2:172.21.15.160:6838/2765554482 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f9f8f1cdc0 con 0x55f9f90301c0 2024-05-20T16:21:51.300 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:51.300+0000 7fc8e3fff640 10 client.15822 handle_command_reply: tid=0 2024-05-20T16:21:51.300 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:51.300+0000 7fc912187740 1 client.15822 shutdown 2024-05-20T16:21:51.301 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:51.301+0000 7fc912187740 1 -- 172.21.15.82:0/179902753 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f9f8bcfd80 msgr2=0x55f9f8fc25c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:21:51.301 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:51.301+0000 7fc912187740 1 --2- 172.21.15.82:0/179902753 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f9f8bcfd80 0x55f9f8fc25c0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fc8fc00cea0 tx=0x7fc8fc0047d0 comp rx=0 tx=0).stop 2024-05-20T16:21:51.302 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:51.302+0000 7fc912187740 1 -- 172.21.15.82:0/179902753 shutdown_connections 2024-05-20T16:21:51.302 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:51.302+0000 7fc912187740 1 --2- 172.21.15.82:0/179902753 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x55f9f90301c0 0x55f9f90326b0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:21:51.302 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:51.302+0000 7fc912187740 1 --2- 172.21.15.82:0/179902753 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f9f8c111b0 0x55f9f8fc1170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:21:51.302 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:51.302+0000 7fc912187740 1 --2- 172.21.15.82:0/179902753 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f9f8bf04d0 0x55f9f8fc0c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:21:51.302 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:51.302+0000 7fc912187740 1 --2- 172.21.15.82:0/179902753 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f9f8bcfd80 0x55f9f8fc25c0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:21:51.302 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:51.302+0000 7fc912187740 1 -- 172.21.15.82:0/179902753 >> 172.21.15.82:0/179902753 conn(0x55f9f8c05300 msgr2=0x55f9f8faf690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:21:51.302 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:51.302+0000 7fc8e3fff640 0 client.15822 ms_handle_reset on v2:172.21.15.160:6838/2765554482 2024-05-20T16:21:51.302 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:51.302+0000 7fc912187740 1 -- 172.21.15.82:0/179902753 shutdown_connections 2024-05-20T16:21:51.302 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:51.302+0000 7fc912187740 1 -- 172.21.15.82:0/179902753 wait complete. 2024-05-20T16:21:51.303 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:51.303+0000 7fc912187740 20 client.15822 trim_cache size 0 max 16384 2024-05-20T16:21:51.303 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:{ 2024-05-20T16:21:51.303 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "message": "", 2024-05-20T16:21:51.304 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "return_code": 0 2024-05-20T16:21:51.304 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:} 2024-05-20T16:21:51.304 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:51.304+0000 7fc90b206640 1 -- 172.21.15.82:0/3330103489 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fc8f00a2d90 msgr2=0x7fc8f00a52a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:21:51.304 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:51.305+0000 7fc90b206640 1 --2- 172.21.15.82:0/3330103489 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fc8f00a2d90 0x7fc8f00a52a0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x55f9f8fb2030 tx=0x7fc8fc008040 comp rx=0 tx=0).stop 2024-05-20T16:21:51.304 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:51.305+0000 7fc90b206640 1 -- 172.21.15.82:0/3330103489 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc90410a300 msgr2=0x7fc904119090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:21:51.305 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:51.305+0000 7fc90b206640 1 --2- 172.21.15.82:0/3330103489 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc90410a300 0x7fc904119090 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fc8ec033040 tx=0x7fc8ec005230 comp rx=0 tx=0).stop 2024-05-20T16:21:51.306 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:51.306+0000 7fc90b206640 1 -- 172.21.15.82:0/3330103489 shutdown_connections 2024-05-20T16:21:51.306 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:51.306+0000 7fc90b206640 1 --2- 172.21.15.82:0/3330103489 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fc8f00a2d90 0x7fc8f00a52a0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:21:51.306 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:51.306+0000 7fc90b206640 1 --2- 172.21.15.82:0/3330103489 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc90410f020 0x7fc9041195d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:21:51.306 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:51.306+0000 7fc90b206640 1 --2- 172.21.15.82:0/3330103489 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc90410a300 0x7fc904119090 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:21:51.306 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:51.306+0000 7fc90b206640 1 --2- 172.21.15.82:0/3330103489 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc9041093d0 0x7fc90411a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:21:51.306 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:51.306+0000 7fc90b206640 1 -- 172.21.15.82:0/3330103489 >> 172.21.15.82:0/3330103489 conn(0x7fc90406f4f0 msgr2=0x7fc90410d010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:21:51.306 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:51.306+0000 7fc90b206640 1 -- 172.21.15.82:0/3330103489 shutdown_connections 2024-05-20T16:21:51.306 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:21:51.306+0000 7fc90b206640 1 -- 172.21.15.82:0/3330103489 wait complete. 2024-05-20T16:21:52.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:21:51 smithi082 ceph-mon[32238]: pgmap v599: 129 pgs: 129 active+clean; 10 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 706 MiB/s wr, 1.63k op/s 2024-05-20T16:21:52.135 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:21:51 smithi184 ceph-mon[36209]: pgmap v599: 129 pgs: 129 active+clean; 10 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 706 MiB/s wr, 1.63k op/s 2024-05-20T16:21:52.312 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:21:51 smithi160 ceph-mon[36347]: pgmap v599: 129 pgs: 129 active+clean; 10 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 706 MiB/s wr, 1.63k op/s 2024-05-20T16:21:55.062 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:21:54 smithi160 ceph-mon[36347]: pgmap v600: 129 pgs: 129 active+clean; 10 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 685 MiB/s wr, 1.59k op/s 2024-05-20T16:21:55.062 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:21:54 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T16:21:55.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:21:54 smithi082 ceph-mon[32238]: pgmap v600: 129 pgs: 129 active+clean; 10 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 685 MiB/s wr, 1.59k op/s 2024-05-20T16:21:55.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:21:54 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T16:21:55.135 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:21:54 smithi184 ceph-mon[36209]: pgmap v600: 129 pgs: 129 active+clean; 10 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 685 MiB/s wr, 1.59k op/s 2024-05-20T16:21:55.135 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:21:54 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T16:21:56.036 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:21:55 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-20T16:21:56.036 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:21:55 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-20T16:21:56.036 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:21:55 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-20T16:21:56.036 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:21:55 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-20T16:21:56.036 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:21:55 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:21:56.036 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:21:55 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T16:21:56.036 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:21:55 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:21:56.062 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:21:55 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-20T16:21:56.062 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:21:55 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-20T16:21:56.062 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:21:55 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-20T16:21:56.062 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:21:55 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-20T16:21:56.062 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:21:55 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:21:56.062 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:21:55 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T16:21:56.063 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:21:55 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:21:56.118 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:21:55 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-20T16:21:56.118 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:21:55 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-20T16:21:56.118 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:21:55 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-20T16:21:56.118 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:21:55 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-20T16:21:56.118 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:21:55 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:21:56.118 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:21:55 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T16:21:56.118 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:21:55 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:21:57.062 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:21:56 smithi160 ceph-mon[36347]: Adjusting osd_memory_target on smithi184 to 176.2M 2024-05-20T16:21:57.062 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:21:56 smithi160 ceph-mon[36347]: Unable to set osd_memory_target on smithi184 to 184791756: error parsing value: Value '184791756' is below minimum 939524096 2024-05-20T16:21:57.062 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:21:56 smithi160 ceph-mon[36347]: pgmap v601: 129 pgs: 129 active+clean; 10 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 818 MiB/s wr, 1.68k op/s 2024-05-20T16:21:57.110 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:21:56 smithi082 ceph-mon[32238]: Adjusting osd_memory_target on smithi184 to 176.2M 2024-05-20T16:21:57.110 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:21:56 smithi082 ceph-mon[32238]: Unable to set osd_memory_target on smithi184 to 184791756: error parsing value: Value '184791756' is below minimum 939524096 2024-05-20T16:21:57.111 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:21:56 smithi082 ceph-mon[32238]: pgmap v601: 129 pgs: 129 active+clean; 10 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 818 MiB/s wr, 1.68k op/s 2024-05-20T16:21:57.135 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:21:56 smithi184 ceph-mon[36209]: Adjusting osd_memory_target on smithi184 to 176.2M 2024-05-20T16:21:57.135 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:21:56 smithi184 ceph-mon[36209]: Unable to set osd_memory_target on smithi184 to 184791756: error parsing value: Value '184791756' is below minimum 939524096 2024-05-20T16:21:57.135 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:21:56 smithi184 ceph-mon[36209]: pgmap v601: 129 pgs: 129 active+clean; 10 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 818 MiB/s wr, 1.68k op/s 2024-05-20T16:21:58.062 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:21:57 smithi160 ceph-mon[36347]: pgmap v602: 129 pgs: 129 active+clean; 10 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 628 MiB/s wr, 1.29k op/s 2024-05-20T16:21:58.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:21:57 smithi082 ceph-mon[32238]: pgmap v602: 129 pgs: 129 active+clean; 10 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 628 MiB/s wr, 1.29k op/s 2024-05-20T16:21:58.135 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:21:57 smithi184 ceph-mon[36209]: pgmap v602: 129 pgs: 129 active+clean; 10 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 628 MiB/s wr, 1.29k op/s 2024-05-20T16:22:01.062 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:22:00 smithi160 ceph-mon[36347]: pgmap v603: 129 pgs: 129 active+clean; 10 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 762 MiB/s wr, 1.41k op/s 2024-05-20T16:22:01.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:22:00 smithi082 ceph-mon[32238]: pgmap v603: 129 pgs: 129 active+clean; 10 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 762 MiB/s wr, 1.41k op/s 2024-05-20T16:22:01.135 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:22:00 smithi184 ceph-mon[36209]: pgmap v603: 129 pgs: 129 active+clean; 10 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 762 MiB/s wr, 1.41k op/s 2024-05-20T16:22:03.062 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:22:02 smithi160 ceph-mon[36347]: pgmap v604: 129 pgs: 129 active+clean; 10 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 731 MiB/s wr, 1.13k op/s 2024-05-20T16:22:03.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:22:02 smithi082 ceph-mon[32238]: pgmap v604: 129 pgs: 129 active+clean; 10 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 731 MiB/s wr, 1.13k op/s 2024-05-20T16:22:03.135 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:22:02 smithi184 ceph-mon[36209]: pgmap v604: 129 pgs: 129 active+clean; 10 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 731 MiB/s wr, 1.13k op/s 2024-05-20T16:22:04.062 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:22:03 smithi160 ceph-mon[36347]: pgmap v605: 129 pgs: 129 active+clean; 10 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 706 MiB/s wr, 970 op/s 2024-05-20T16:22:04.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:22:03 smithi082 ceph-mon[32238]: pgmap v605: 129 pgs: 129 active+clean; 10 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 706 MiB/s wr, 970 op/s 2024-05-20T16:22:04.135 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:22:03 smithi184 ceph-mon[36209]: pgmap v605: 129 pgs: 129 active+clean; 10 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 706 MiB/s wr, 970 op/s 2024-05-20T16:22:04.568 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:04.568+0000 7efd244e2640 1 -- 172.21.15.82:0/2800634894 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efd1c1093d0 msgr2=0x7efd1c109dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:22:04.568 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:04.568+0000 7efd244e2640 1 --2- 172.21.15.82:0/2800634894 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efd1c1093d0 0x7efd1c109dc0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7efd18009900 tx=0x7efd1802a440 comp rx=0 tx=0).stop 2024-05-20T16:22:04.584 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:04.583+0000 7efd244e2640 1 -- 172.21.15.82:0/2800634894 shutdown_connections 2024-05-20T16:22:04.584 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:04.583+0000 7efd244e2640 1 --2- 172.21.15.82:0/2800634894 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efd1c10f020 0x7efd1c10f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:22:04.584 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:04.583+0000 7efd244e2640 1 --2- 172.21.15.82:0/2800634894 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efd1c10a300 0x7efd1c10a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:22:04.585 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:04.583+0000 7efd244e2640 1 --2- 172.21.15.82:0/2800634894 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efd1c1093d0 0x7efd1c109dc0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:22:04.585 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:04.583+0000 7efd244e2640 1 -- 172.21.15.82:0/2800634894 >> 172.21.15.82:0/2800634894 conn(0x7efd1c06f4f0 msgr2=0x7efd1c071980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:22:04.585 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:04.584+0000 7efd244e2640 1 -- 172.21.15.82:0/2800634894 shutdown_connections 2024-05-20T16:22:04.588 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:04.584+0000 7efd244e2640 1 -- 172.21.15.82:0/2800634894 wait complete. 2024-05-20T16:22:04.588 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:04.585+0000 7efd244e2640 1 Processor -- start 2024-05-20T16:22:04.588 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:04.585+0000 7efd244e2640 1 -- start start 2024-05-20T16:22:04.588 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:04.586+0000 7efd244e2640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efd1c1093d0 0x7efd1c11a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:22:04.589 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:04.586+0000 7efd244e2640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efd1c10a300 0x7efd1c119050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:22:04.589 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:04.586+0000 7efd244e2640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efd1c10f020 0x7efd1c119590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:22:04.589 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:04.586+0000 7efd244e2640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7efd1c111480 con 0x7efd1c1093d0 2024-05-20T16:22:04.589 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:04.586+0000 7efd244e2640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7efd1c111300 con 0x7efd1c10f020 2024-05-20T16:22:04.589 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:04.586+0000 7efd244e2640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7efd1c111600 con 0x7efd1c10a300 2024-05-20T16:22:04.589 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:04.586+0000 7efd2227e640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efd1c1093d0 0x7efd1c11a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:22:04.590 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:04.586+0000 7efd2227e640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efd1c1093d0 0x7efd1c11a9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:52578/0 (socket says 172.21.15.82:52578) 2024-05-20T16:22:04.590 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:04.586+0000 7efd2227e640 1 -- 172.21.15.82:0/460346369 learned_addr learned my addr 172.21.15.82:0/460346369 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:22:04.590 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:04.587+0000 7efd21a7d640 1 --2- 172.21.15.82:0/460346369 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efd1c10a300 0x7efd1c119050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:22:04.590 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:04.587+0000 7efd22a7f640 1 --2- 172.21.15.82:0/460346369 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efd1c10f020 0x7efd1c119590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:22:04.591 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:04.588+0000 7efd2227e640 1 -- 172.21.15.82:0/460346369 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efd1c10a300 msgr2=0x7efd1c119050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:22:04.591 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:04.588+0000 7efd2227e640 1 --2- 172.21.15.82:0/460346369 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efd1c10a300 0x7efd1c119050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:22:04.591 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:04.588+0000 7efd2227e640 1 -- 172.21.15.82:0/460346369 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efd1c10f020 msgr2=0x7efd1c119590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:22:04.591 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:04.588+0000 7efd2227e640 1 --2- 172.21.15.82:0/460346369 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efd1c10f020 0x7efd1c119590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:22:04.591 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:04.588+0000 7efd2227e640 1 -- 172.21.15.82:0/460346369 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd1c119ad0 con 0x7efd1c1093d0 2024-05-20T16:22:04.592 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:04.590+0000 7efd2227e640 1 --2- 172.21.15.82:0/460346369 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efd1c1093d0 0x7efd1c11a9b0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7efd18032bf0 tx=0x7efd18032c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:22:04.592 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:04.591+0000 7efd137fe640 1 -- 172.21.15.82:0/460346369 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd1803f030 con 0x7efd1c1093d0 2024-05-20T16:22:04.592 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:04.591+0000 7efd244e2640 1 -- 172.21.15.82:0/460346369 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd1c119d60 con 0x7efd1c1093d0 2024-05-20T16:22:04.592 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:04.591+0000 7efd244e2640 1 -- 172.21.15.82:0/460346369 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd1c1b8c30 con 0x7efd1c1093d0 2024-05-20T16:22:04.592 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:04.592+0000 7efd137fe640 1 -- 172.21.15.82:0/460346369 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efd18032e40 con 0x7efd1c1093d0 2024-05-20T16:22:04.592 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:04.592+0000 7efd137fe640 1 -- 172.21.15.82:0/460346369 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd18035070 con 0x7efd1c1093d0 2024-05-20T16:22:04.601 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:04.600+0000 7efd137fe640 1 -- 172.21.15.82:0/460346369 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7efd18042030 con 0x7efd1c1093d0 2024-05-20T16:22:04.602 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:04.602+0000 7efd137fe640 1 --2- 172.21.15.82:0/460346369 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7efd080a2d90 0x7efd080a52a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:22:04.602 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:04.603+0000 7efd137fe640 1 -- 172.21.15.82:0/460346369 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9150+0+0 (secure 0 0 0) 0x7efd180ef860 con 0x7efd1c1093d0 2024-05-20T16:22:04.603 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:04.603+0000 7efd2b386740 1 Processor -- start 2024-05-20T16:22:04.603 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:04.604+0000 7efd22a7f640 1 --2- 172.21.15.82:0/460346369 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7efd080a2d90 0x7efd080a52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:22:04.604 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:04.604+0000 7efd2b386740 1 -- start start 2024-05-20T16:22:04.604 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:04.605+0000 7efd2b386740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fee99cf1b0 0x55fee9d6ee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:22:04.605 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:04.605+0000 7efd2b386740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fee99a62f0 0x55fee9d6f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:22:04.605 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:04.605+0000 7efd2b386740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fee99ae4d0 0x55fee9d762e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:22:04.605 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:04.606+0000 7efd2b386740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55fee9cd3340 con 0x55fee99a62f0 2024-05-20T16:22:04.605 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:04.606+0000 7efd2b386740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x55fee9c7b290 con 0x55fee99ae4d0 2024-05-20T16:22:04.606 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:04.606+0000 7efd2b386740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55fee9ccf300 con 0x55fee99cf1b0 2024-05-20T16:22:04.607 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:04.606+0000 7efd21a7d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fee99a62f0 0x55fee9d6f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:22:04.607 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:04.606+0000 7efd21a7d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fee99a62f0 0x55fee9d6f3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:52584/0 (socket says 172.21.15.82:52584) 2024-05-20T16:22:04.607 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:04.606+0000 7efd21a7d640 1 -- 172.21.15.82:0/1657538587 learned_addr learned my addr 172.21.15.82:0/1657538587 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:22:04.608 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:04.608+0000 7efd22a7f640 1 --2- 172.21.15.82:0/460346369 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7efd080a2d90 0x7efd080a52a0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7efd14005fd0 tx=0x7efd140097f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:22:04.608 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:04.608+0000 7efd22a7f640 1 --2- 172.21.15.82:0/1657538587 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fee99ae4d0 0x55fee9d762e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:22:04.608 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:04.608+0000 7efd21a7d640 1 -- 172.21.15.82:0/1657538587 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fee99cf1b0 msgr2=0x55fee9d6ee60 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-20T16:22:04.608 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:04.608+0000 7efd21a7d640 1 --2- 172.21.15.82:0/1657538587 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fee99cf1b0 0x55fee9d6ee60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:22:04.609 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:04.608+0000 7efd21a7d640 1 -- 172.21.15.82:0/1657538587 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fee99ae4d0 msgr2=0x55fee9d762e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:22:04.609 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:04.608+0000 7efd21a7d640 1 --2- 172.21.15.82:0/1657538587 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fee99ae4d0 0x55fee9d762e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:22:04.609 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:04.608+0000 7efd21a7d640 1 -- 172.21.15.82:0/1657538587 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fee9c9f8e0 con 0x55fee99a62f0 2024-05-20T16:22:04.609 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:04.609+0000 7efd21a7d640 1 --2- 172.21.15.82:0/1657538587 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fee99a62f0 0x55fee9d6f3e0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7efd0c009c90 tx=0x7efd0c011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c74f96af572285c server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:22:04.609 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:04.609+0000 7efd10bfe640 1 -- 172.21.15.82:0/1657538587 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd0c005220 con 0x55fee99a62f0 2024-05-20T16:22:04.610 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:04.610+0000 7efd10bfe640 1 -- 172.21.15.82:0/1657538587 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efd0c004c80 con 0x55fee99a62f0 2024-05-20T16:22:04.610 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:04.610+0000 7efd10bfe640 1 -- 172.21.15.82:0/1657538587 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd0c0198a0 con 0x55fee99a62f0 2024-05-20T16:22:04.614 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:04.614+0000 7efd2b386740 1 -- 172.21.15.82:0/1657538587 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fee99a62f0 msgr2=0x55fee9d6f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:22:04.614 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:04.614+0000 7efd2b386740 1 --2- 172.21.15.82:0/1657538587 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fee99a62f0 0x55fee9d6f3e0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7efd0c009c90 tx=0x7efd0c011810 comp rx=0 tx=0).stop 2024-05-20T16:22:04.619 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:04.619+0000 7efd2b386740 1 -- 172.21.15.82:0/1657538587 shutdown_connections 2024-05-20T16:22:04.619 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:04.619+0000 7efd2b386740 1 --2- 172.21.15.82:0/1657538587 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fee99a62f0 0x55fee9d6f3e0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:22:04.619 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:04.619+0000 7efd2b386740 1 --2- 172.21.15.82:0/1657538587 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fee99ae4d0 0x55fee9d762e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:22:04.619 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:04.619+0000 7efd2b386740 1 --2- 172.21.15.82:0/1657538587 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fee99cf1b0 0x55fee9d6ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:22:04.619 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:04.619+0000 7efd2b386740 1 -- 172.21.15.82:0/1657538587 >> 172.21.15.82:0/1657538587 conn(0x55fee99c3300 msgr2=0x55fee9d6a7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:22:04.623 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:04.623+0000 7efd2b386740 1 -- 172.21.15.82:0/1657538587 shutdown_connections 2024-05-20T16:22:04.623 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:04.624+0000 7efd2b386740 1 -- 172.21.15.82:0/1657538587 wait complete. 2024-05-20T16:22:04.624 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:04.624+0000 7efd2b386740 1 -- start start 2024-05-20T16:22:04.624 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:04.625+0000 7efd2b386740 1 Processor -- start 2024-05-20T16:22:04.625 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:04.625+0000 7efd2b386740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fee99a62f0 0x55fee9d78cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:22:04.625 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:04.625+0000 7efd2b386740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fee99ae4d0 0x55fee9d79200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:22:04.625 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:04.626+0000 7efd2b386740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fee99cf1b0 0x55fee9d80160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:22:04.626 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:04.626+0000 7efd2b386740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55fee9d76d30 con 0x55fee99a62f0 2024-05-20T16:22:04.626 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:04.626+0000 7efd2b386740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x55fee9c8e790 con 0x55fee99ae4d0 2024-05-20T16:22:04.626 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:04.626+0000 7efd22a7f640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fee99cf1b0 0x55fee9d80160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:22:04.626 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:04.626+0000 7efd22a7f640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fee99cf1b0 0x55fee9d80160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.82:59550/0 (socket says 172.21.15.82:59550) 2024-05-20T16:22:04.626 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:04.626+0000 7efd22a7f640 1 -- 172.21.15.82:0/1175109888 learned_addr learned my addr 172.21.15.82:0/1175109888 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:22:04.626 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:04.626+0000 7efd2b386740 1 -- 172.21.15.82:0/1175109888 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55fee9d76eb0 con 0x55fee99cf1b0 2024-05-20T16:22:04.627 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:04.626+0000 7efd2227e640 1 --2- 172.21.15.82:0/1175109888 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fee99a62f0 0x55fee9d78cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:22:04.627 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:04.627+0000 7efd21a7d640 1 --2- 172.21.15.82:0/1175109888 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fee99ae4d0 0x55fee9d79200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:22:04.627 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:04.627+0000 7efd2b386740 10 client.0 _ll_register_callbacks cb 0x55fee9cc9a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T16:22:04.627 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:04.627+0000 7efd22a7f640 1 -- 172.21.15.82:0/1175109888 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fee99ae4d0 msgr2=0x55fee9d79200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:22:04.627 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:04.627+0000 7efd22a7f640 1 --2- 172.21.15.82:0/1175109888 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fee99ae4d0 0x55fee9d79200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:22:04.628 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:04.627+0000 7efd22a7f640 1 -- 172.21.15.82:0/1175109888 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fee99a62f0 msgr2=0x55fee9d78cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:22:04.628 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:04.627+0000 7efd22a7f640 1 --2- 172.21.15.82:0/1175109888 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fee99a62f0 0x55fee9d78cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:22:04.628 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:04.627+0000 7efd22a7f640 1 -- 172.21.15.82:0/1175109888 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fee99e7210 con 0x55fee99cf1b0 2024-05-20T16:22:04.628 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:04.627+0000 7efd2b386740 1 -- 172.21.15.82:0/1175109888 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efd0c004c80 con 0x55fee99cf1b0 2024-05-20T16:22:04.628 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:04.627+0000 7efd22a7f640 1 --2- 172.21.15.82:0/1175109888 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fee99cf1b0 0x55fee9d80160 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7efd14012760 tx=0x7efd14004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=89ed392742d3d34b server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:22:04.628 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:04.629+0000 7efd05ffb640 10 client.25372 ms_handle_connect on v2:172.21.15.184:3300/0 2024-05-20T16:22:04.629 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:04.629+0000 7efd05ffb640 1 -- 172.21.15.82:0/1175109888 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd14023070 con 0x55fee99cf1b0 2024-05-20T16:22:04.629 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:04.629+0000 7efd05ffb640 1 -- 172.21.15.82:0/1175109888 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9150+0+0 (secure 0 0 0) 0x7efd14011a60 con 0x55fee99cf1b0 2024-05-20T16:22:04.629 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:04.629+0000 7efd05ffb640 1 -- 172.21.15.82:0/1175109888 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7efd14011ca0 con 0x55fee99cf1b0 2024-05-20T16:22:04.629 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:04.630+0000 7efd2b386740 10 client.25372 fetch_fsmap learned FSMap version 27 2024-05-20T16:22:04.629 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:04.630+0000 7efd2b386740 1 -- 172.21.15.82:0/1175109888 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({fsmap=27}) -- 0x55fee9ab3620 con 0x55fee99cf1b0 2024-05-20T16:22:04.635 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:04.635+0000 7efd05ffb640 1 -- 172.21.15.82:0/1175109888 <== mon.1 v2:172.21.15.184:3300/0 4 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7efd1400f040 con 0x55fee99cf1b0 2024-05-20T16:22:04.635 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:04.636+0000 7efd2b386740 10 client.25372 fetch_fsmap finished waiting for FSMap version 27 2024-05-20T16:22:04.636 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:04.636+0000 7efd2b386740 10 client.25372 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T16:22:04.636 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:04.636+0000 7efd2b386740 20 client.25372 populate_metadata read hostname 'smithi082' 2024-05-20T16:22:04.636 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:04.636+0000 7efd2b386740 1 --2- 172.21.15.82:0/1175109888 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x55fee998dd80 0x55fee9d864e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:22:04.636 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:04.636+0000 7efd2b386740 4 client.25372 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T16:22:04.636 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:04.636+0000 7efd2b386740 1 -- 172.21.15.82:0/1175109888 --> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7efd0c004c80 con 0x55fee998dd80 2024-05-20T16:22:04.642 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:04.642+0000 7efd2227e640 1 --2- 172.21.15.82:0/1175109888 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x55fee998dd80 0x55fee9d864e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:22:04.649 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:04.650+0000 7efd2227e640 1 --2- 172.21.15.82:0/1175109888 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x55fee998dd80 0x55fee9d864e0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=454fc03058676b91 server_cookie=f06c0d2849fb4ff5 in_seq=0 out_seq=0 2024-05-20T16:22:04.649 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:04.650+0000 7efd05ffb640 10 client.25372 ms_handle_connect on v2:172.21.15.160:6838/2765554482 2024-05-20T16:22:04.653 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:04.653+0000 7efd05ffb640 1 -- 172.21.15.82:0/1175109888 <== mds.0 v2:172.21.15.160:6838/2765554482 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7efd0c004c80 con 0x55fee998dd80 2024-05-20T16:22:04.653 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:04.653+0000 7efd05ffb640 10 client.25372 handle_command_reply: tid=0 2024-05-20T16:22:04.653 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:04.653+0000 7efd2b386740 1 client.25372 shutdown 2024-05-20T16:22:04.655 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:04.656+0000 7efd2b386740 1 -- 172.21.15.82:0/1175109888 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fee99cf1b0 msgr2=0x55fee9d80160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:22:04.655 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:04.656+0000 7efd2b386740 1 --2- 172.21.15.82:0/1175109888 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fee99cf1b0 0x55fee9d80160 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7efd14012760 tx=0x7efd14004a40 comp rx=0 tx=0).stop 2024-05-20T16:22:04.656 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:04.656+0000 7efd2b386740 1 -- 172.21.15.82:0/1175109888 shutdown_connections 2024-05-20T16:22:04.656 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:04.656+0000 7efd2b386740 1 --2- 172.21.15.82:0/1175109888 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x55fee998dd80 0x55fee9d864e0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:22:04.656 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:04.656+0000 7efd2b386740 1 --2- 172.21.15.82:0/1175109888 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fee99cf1b0 0x55fee9d80160 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:22:04.656 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:04.656+0000 7efd2b386740 1 --2- 172.21.15.82:0/1175109888 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fee99ae4d0 0x55fee9d79200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:22:04.656 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:04.656+0000 7efd2b386740 1 --2- 172.21.15.82:0/1175109888 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fee99a62f0 0x55fee9d78cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:22:04.656 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:04.656+0000 7efd2b386740 1 -- 172.21.15.82:0/1175109888 >> 172.21.15.82:0/1175109888 conn(0x55fee99c3300 msgr2=0x55fee9d6d7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:22:04.657 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:04.657+0000 7efd05ffb640 0 client.25372 ms_handle_reset on v2:172.21.15.160:6838/2765554482 2024-05-20T16:22:04.657 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:04.657+0000 7efd2b386740 1 -- 172.21.15.82:0/1175109888 shutdown_connections 2024-05-20T16:22:04.657 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:04.657+0000 7efd2b386740 1 -- 172.21.15.82:0/1175109888 wait complete. 2024-05-20T16:22:04.657 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:04.657+0000 7efd2b386740 20 client.25372 trim_cache size 0 max 16384 2024-05-20T16:22:04.690 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:04.691+0000 7efd2b386740 1 Processor -- start 2024-05-20T16:22:04.691 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:04.691+0000 7efd2b386740 1 -- start start 2024-05-20T16:22:04.691 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:04.691+0000 7efd2b386740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fee99a62f0 0x55fee9d772e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:22:04.691 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:04.691+0000 7efd2b386740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fee99ae4d0 0x55fee9d77820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:22:04.691 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:04.692+0000 7efd2b386740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fee99cf1b0 0x55fee9d6a250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:22:04.692 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:04.692+0000 7efd21a7d640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fee99ae4d0 0x55fee9d77820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:22:04.692 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:04.692+0000 7efd21a7d640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fee99ae4d0 0x55fee9d77820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.82:59564/0 (socket says 172.21.15.82:59564) 2024-05-20T16:22:04.692 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:04.692+0000 7efd21a7d640 1 -- 172.21.15.82:0/2154307240 learned_addr learned my addr 172.21.15.82:0/2154307240 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:22:04.692 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:04.692+0000 7efd2b386740 1 -- 172.21.15.82:0/2154307240 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7efd00012550 con 0x55fee99cf1b0 2024-05-20T16:22:04.692 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:04.692+0000 7efd2b386740 1 -- 172.21.15.82:0/2154307240 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7efd0000fe20 con 0x55fee99a62f0 2024-05-20T16:22:04.692 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:04.692+0000 7efd2b386740 1 -- 172.21.15.82:0/2154307240 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7efd0000dc70 con 0x55fee99ae4d0 2024-05-20T16:22:04.693 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:04.693+0000 7efd2227e640 1 --2- 172.21.15.82:0/2154307240 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fee99a62f0 0x55fee9d772e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:22:04.693 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:04.693+0000 7efd22a7f640 1 --2- 172.21.15.82:0/2154307240 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fee99cf1b0 0x55fee9d6a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:22:04.693 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:04.693+0000 7efd2227e640 1 -- 172.21.15.82:0/2154307240 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fee99ae4d0 msgr2=0x55fee9d77820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:22:04.693 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:04.693+0000 7efd2227e640 1 --2- 172.21.15.82:0/2154307240 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fee99ae4d0 0x55fee9d77820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:22:04.693 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:04.693+0000 7efd2227e640 1 -- 172.21.15.82:0/2154307240 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fee99cf1b0 msgr2=0x55fee9d6a250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:22:04.694 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:04.693+0000 7efd2227e640 1 --2- 172.21.15.82:0/2154307240 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fee99cf1b0 0x55fee9d6a250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:22:04.694 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:04.693+0000 7efd2227e640 1 -- 172.21.15.82:0/2154307240 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fee9d77d60 con 0x55fee99a62f0 2024-05-20T16:22:04.694 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:04.693+0000 7efd21a7d640 1 --2- 172.21.15.82:0/2154307240 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fee99ae4d0 0x55fee9d77820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T16:22:04.694 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:04.694+0000 7efd2227e640 1 --2- 172.21.15.82:0/2154307240 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fee99a62f0 0x55fee9d772e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x55fee9d70030 tx=0x7efd180bbb30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d851e3496efdbe20 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:22:04.696 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:04.696+0000 7efd07fff640 1 -- 172.21.15.82:0/2154307240 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd180c4940 con 0x55fee99a62f0 2024-05-20T16:22:04.696 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:04.696+0000 7efd07fff640 1 -- 172.21.15.82:0/2154307240 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efd180b9fd0 con 0x55fee99a62f0 2024-05-20T16:22:04.698 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:04.698+0000 7efd2b386740 1 -- 172.21.15.82:0/2154307240 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fee99a62f0 msgr2=0x55fee9d772e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:22:04.698 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:04.698+0000 7efd2b386740 1 --2- 172.21.15.82:0/2154307240 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fee99a62f0 0x55fee9d772e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x55fee9d70030 tx=0x7efd180bbb30 comp rx=0 tx=0).stop 2024-05-20T16:22:04.698 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:04.698+0000 7efd2b386740 1 -- 172.21.15.82:0/2154307240 shutdown_connections 2024-05-20T16:22:04.698 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:04.698+0000 7efd2b386740 1 --2- 172.21.15.82:0/2154307240 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fee99cf1b0 0x55fee9d6a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:22:04.698 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:04.698+0000 7efd2b386740 1 --2- 172.21.15.82:0/2154307240 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fee99ae4d0 0x55fee9d77820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:22:04.698 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:04.698+0000 7efd2b386740 1 --2- 172.21.15.82:0/2154307240 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fee99a62f0 0x55fee9d772e0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:22:04.698 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:04.698+0000 7efd2b386740 1 -- 172.21.15.82:0/2154307240 >> 172.21.15.82:0/2154307240 conn(0x55fee99c3300 msgr2=0x55fee9d73060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:22:04.700 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:04.699+0000 7efd2b386740 1 -- 172.21.15.82:0/2154307240 shutdown_connections 2024-05-20T16:22:04.702 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:04.702+0000 7efd2b386740 1 -- 172.21.15.82:0/2154307240 wait complete. 2024-05-20T16:22:04.702 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:04.702+0000 7efd2b386740 1 -- start start 2024-05-20T16:22:04.703 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:04.702+0000 7efd2b386740 1 Processor -- start 2024-05-20T16:22:04.703 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:04.703+0000 7efd2b386740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fee99a62f0 0x55fee9d7c660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:22:04.703 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:04.703+0000 7efd2b386740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fee99ae4d0 0x55fee9d7cba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:22:04.703 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:04.703+0000 7efd2b386740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fee99cf1b0 0x55fee9dec880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:22:04.703 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:04.703+0000 7efd2b386740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7efd0000e2e0 con 0x55fee99cf1b0 2024-05-20T16:22:04.703 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:04.703+0000 7efd2b386740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7efd0000f750 con 0x55fee99ae4d0 2024-05-20T16:22:04.703 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:04.703+0000 7efd2b386740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7efd0000e9b0 con 0x55fee99a62f0 2024-05-20T16:22:04.703 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:04.703+0000 7efd2b386740 10 client.0 _ll_register_callbacks cb 0x7efd14016070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T16:22:04.703 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:04.703+0000 7efd2227e640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fee99a62f0 0x55fee9d7c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:22:04.703 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:04.703+0000 7efd2227e640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fee99a62f0 0x55fee9d7c660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.82:59568/0 (socket says 172.21.15.82:59568) 2024-05-20T16:22:04.703 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:04.703+0000 7efd2227e640 1 -- 172.21.15.82:0/3454910913 learned_addr learned my addr 172.21.15.82:0/3454910913 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:22:04.703 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:04.703+0000 7efd21a7d640 1 --2- 172.21.15.82:0/3454910913 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fee99ae4d0 0x55fee9d7cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:22:04.703 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:04.703+0000 7efd22a7f640 1 --2- 172.21.15.82:0/3454910913 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fee99cf1b0 0x55fee9dec880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:22:04.704 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:04.704+0000 7efd21a7d640 1 -- 172.21.15.82:0/3454910913 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fee99a62f0 msgr2=0x55fee9d7c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:22:04.704 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:04.704+0000 7efd21a7d640 1 --2- 172.21.15.82:0/3454910913 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fee99a62f0 0x55fee9d7c660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:22:04.704 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:04.704+0000 7efd21a7d640 1 -- 172.21.15.82:0/3454910913 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fee99cf1b0 msgr2=0x55fee9dec880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:22:04.704 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:04.704+0000 7efd21a7d640 1 --2- 172.21.15.82:0/3454910913 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fee99cf1b0 0x55fee9dec880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:22:04.704 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:04.704+0000 7efd21a7d640 1 -- 172.21.15.82:0/3454910913 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fee9decdc0 con 0x55fee99ae4d0 2024-05-20T16:22:04.704 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:04.704+0000 7efd2227e640 1 --2- 172.21.15.82:0/3454910913 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fee99a62f0 0x55fee9d7c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T16:22:04.705 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:04.705+0000 7efd21a7d640 1 --2- 172.21.15.82:0/3454910913 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fee99ae4d0 0x55fee9d7cba0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7efd0c011d10 tx=0x7efd0c0099d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a10fe63eb16d3bf7 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:22:04.705 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:04.705+0000 7efd05ffb640 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-20T16:22:04.705 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:04.705+0000 7efd05ffb640 1 -- 172.21.15.82:0/3454910913 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd0c004a50 con 0x55fee99ae4d0 2024-05-20T16:22:04.705 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:04.705+0000 7efd2b386740 1 -- 172.21.15.82:0/3454910913 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efd180b9fd0 con 0x55fee99ae4d0 2024-05-20T16:22:04.705 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:04.706+0000 7efd05ffb640 1 -- 172.21.15.82:0/3454910913 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9150+0+0 (secure 0 0 0) 0x7efd0c004bf0 con 0x55fee99ae4d0 2024-05-20T16:22:04.706 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:04.706+0000 7efd05ffb640 1 -- 172.21.15.82:0/3454910913 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7efd0c038050 con 0x55fee99ae4d0 2024-05-20T16:22:04.707 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:04.707+0000 7efd2b386740 10 client.25346 fetch_fsmap learned FSMap version 27 2024-05-20T16:22:04.707 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:04.707+0000 7efd2b386740 1 -- 172.21.15.82:0/3454910913 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55fee9e0ff50 con 0x55fee99ae4d0 2024-05-20T16:22:04.708 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:04.709+0000 7efd05ffb640 1 -- 172.21.15.82:0/3454910913 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efd180b9fd0 con 0x55fee99ae4d0 2024-05-20T16:22:04.708 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:04.709+0000 7efd05ffb640 1 -- 172.21.15.82:0/3454910913 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7efd0c019430 con 0x55fee99ae4d0 2024-05-20T16:22:04.709 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:04.709+0000 7efd2b386740 10 client.25346 fetch_fsmap finished waiting for FSMap version 27 2024-05-20T16:22:04.709 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:04.709+0000 7efd2b386740 10 client.25346 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T16:22:04.709 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:04.709+0000 7efd2b386740 20 client.25346 populate_metadata read hostname 'smithi082' 2024-05-20T16:22:04.709 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:04.709+0000 7efd2b386740 1 --2- 172.21.15.82:0/3454910913 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x55fee99631c0 0x55fee9e126b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:22:04.709 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:04.709+0000 7efd2b386740 4 client.25346 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-20T16:22:04.709 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:04.709+0000 7efd2b386740 1 -- 172.21.15.82:0/3454910913 --> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55fee9ce0a70 con 0x55fee99631c0 2024-05-20T16:22:04.709 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:04.709+0000 7efd2227e640 1 --2- 172.21.15.82:0/3454910913 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x55fee99631c0 0x55fee9e126b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:22:04.711 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:04.711+0000 7efd05ffb640 10 client.25346 ms_handle_connect on v2:172.21.15.160:6838/2765554482 2024-05-20T16:22:04.711 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:04.711+0000 7efd2227e640 1 --2- 172.21.15.82:0/3454910913 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x55fee99631c0 0x55fee9e126b0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=be29ea1b6aec8eb3 server_cookie=45fe1d03444e3a42 in_seq=0 out_seq=0 2024-05-20T16:22:05.698 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:05.698+0000 7efd05ffb640 1 -- 172.21.15.82:0/3454910913 <== mds.0 v2:172.21.15.160:6838/2765554482 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55fee9ce0a70 con 0x55fee99631c0 2024-05-20T16:22:05.698 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:05.698+0000 7efd05ffb640 10 client.25346 handle_command_reply: tid=0 2024-05-20T16:22:05.698 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:05.698+0000 7efd2b386740 1 client.25346 shutdown 2024-05-20T16:22:05.698 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:05.698+0000 7efd2b386740 1 -- 172.21.15.82:0/3454910913 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fee99ae4d0 msgr2=0x55fee9d7cba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:22:05.698 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:05.698+0000 7efd2b386740 1 --2- 172.21.15.82:0/3454910913 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fee99ae4d0 0x55fee9d7cba0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7efd0c011d10 tx=0x7efd0c0099d0 comp rx=0 tx=0).stop 2024-05-20T16:22:05.699 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:05.698+0000 7efd2b386740 1 -- 172.21.15.82:0/3454910913 shutdown_connections 2024-05-20T16:22:05.699 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:05.698+0000 7efd2b386740 1 --2- 172.21.15.82:0/3454910913 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x55fee99631c0 0x55fee9e126b0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:22:05.699 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:05.698+0000 7efd2b386740 1 --2- 172.21.15.82:0/3454910913 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fee99cf1b0 0x55fee9dec880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:22:05.699 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:05.698+0000 7efd2b386740 1 --2- 172.21.15.82:0/3454910913 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fee99ae4d0 0x55fee9d7cba0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:22:05.699 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:05.698+0000 7efd2b386740 1 --2- 172.21.15.82:0/3454910913 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fee99a62f0 0x55fee9d7c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:22:05.699 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:05.698+0000 7efd2b386740 1 -- 172.21.15.82:0/3454910913 >> 172.21.15.82:0/3454910913 conn(0x55fee99c3300 msgr2=0x55fee9d78c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:22:05.699 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:05.698+0000 7efd05ffb640 0 client.25346 ms_handle_reset on v2:172.21.15.160:6838/2765554482 2024-05-20T16:22:05.699 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:05.699+0000 7efd2b386740 1 -- 172.21.15.82:0/3454910913 shutdown_connections 2024-05-20T16:22:05.700 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:05.699+0000 7efd2b386740 1 -- 172.21.15.82:0/3454910913 wait complete. 2024-05-20T16:22:05.700 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:05.699+0000 7efd2b386740 20 client.25346 trim_cache size 0 max 16384 2024-05-20T16:22:05.700 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:{ 2024-05-20T16:22:05.700 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "message": "", 2024-05-20T16:22:05.700 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "return_code": 0 2024-05-20T16:22:05.700 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:} 2024-05-20T16:22:05.701 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:05.700+0000 7efd244e2640 1 -- 172.21.15.82:0/460346369 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7efd080a2d90 msgr2=0x7efd080a52a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:22:05.701 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:05.700+0000 7efd244e2640 1 --2- 172.21.15.82:0/460346369 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7efd080a2d90 0x7efd080a52a0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7efd14005fd0 tx=0x7efd140097f0 comp rx=0 tx=0).stop 2024-05-20T16:22:05.701 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:05.700+0000 7efd244e2640 1 -- 172.21.15.82:0/460346369 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efd1c1093d0 msgr2=0x7efd1c11a9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:22:05.701 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:05.700+0000 7efd244e2640 1 --2- 172.21.15.82:0/460346369 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efd1c1093d0 0x7efd1c11a9b0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7efd18032bf0 tx=0x7efd18032c20 comp rx=0 tx=0).stop 2024-05-20T16:22:05.701 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:05.700+0000 7efd244e2640 1 -- 172.21.15.82:0/460346369 shutdown_connections 2024-05-20T16:22:05.701 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:05.700+0000 7efd244e2640 1 --2- 172.21.15.82:0/460346369 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7efd080a2d90 0x7efd080a52a0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:22:05.701 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:05.700+0000 7efd244e2640 1 --2- 172.21.15.82:0/460346369 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efd1c10f020 0x7efd1c119590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:22:05.701 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:05.700+0000 7efd244e2640 1 --2- 172.21.15.82:0/460346369 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efd1c10a300 0x7efd1c119050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:22:05.701 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:05.700+0000 7efd244e2640 1 --2- 172.21.15.82:0/460346369 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efd1c1093d0 0x7efd1c11a9b0 secure :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7efd18032bf0 tx=0x7efd18032c20 comp rx=0 tx=0).stop 2024-05-20T16:22:05.701 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:05.700+0000 7efd244e2640 1 -- 172.21.15.82:0/460346369 >> 172.21.15.82:0/460346369 conn(0x7efd1c06f4f0 msgr2=0x7efd1c10d010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:22:05.702 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:05.702+0000 7efd244e2640 1 -- 172.21.15.82:0/460346369 shutdown_connections 2024-05-20T16:22:05.702 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:05.702+0000 7efd244e2640 1 -- 172.21.15.82:0/460346369 wait complete. 2024-05-20T16:22:06.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:22:05 smithi082 ceph-mon[32238]: pgmap v606: 129 pgs: 129 active+clean; 10 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 791 MiB/s wr, 1.11k op/s 2024-05-20T16:22:06.135 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:22:05 smithi184 ceph-mon[36209]: pgmap v606: 129 pgs: 129 active+clean; 10 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 791 MiB/s wr, 1.11k op/s 2024-05-20T16:22:06.312 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:22:05 smithi160 ceph-mon[36347]: pgmap v606: 129 pgs: 129 active+clean; 10 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 791 MiB/s wr, 1.11k op/s 2024-05-20T16:22:08.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:22:07 smithi082 ceph-mon[32238]: pgmap v607: 129 pgs: 129 active+clean; 10 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 682 MiB/s wr, 1.06k op/s 2024-05-20T16:22:08.135 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:22:07 smithi184 ceph-mon[36209]: pgmap v607: 129 pgs: 129 active+clean; 10 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 682 MiB/s wr, 1.06k op/s 2024-05-20T16:22:08.312 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:22:07 smithi160 ceph-mon[36347]: pgmap v607: 129 pgs: 129 active+clean; 10 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 682 MiB/s wr, 1.06k op/s 2024-05-20T16:22:10.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:22:09 smithi082 ceph-mon[32238]: pgmap v608: 129 pgs: 129 active+clean; 10 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 798 MiB/s wr, 1.36k op/s 2024-05-20T16:22:10.135 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:22:09 smithi184 ceph-mon[36209]: pgmap v608: 129 pgs: 129 active+clean; 10 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 798 MiB/s wr, 1.36k op/s 2024-05-20T16:22:10.312 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:22:09 smithi160 ceph-mon[36347]: pgmap v608: 129 pgs: 129 active+clean; 10 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 798 MiB/s wr, 1.36k op/s 2024-05-20T16:22:12.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:22:11 smithi082 ceph-mon[32238]: pgmap v609: 129 pgs: 129 active+clean; 10 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 730 MiB/s wr, 1.35k op/s 2024-05-20T16:22:12.135 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:22:11 smithi184 ceph-mon[36209]: pgmap v609: 129 pgs: 129 active+clean; 10 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 730 MiB/s wr, 1.35k op/s 2024-05-20T16:22:12.311 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:22:11 smithi160 ceph-mon[36347]: pgmap v609: 129 pgs: 129 active+clean; 10 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 730 MiB/s wr, 1.35k op/s 2024-05-20T16:22:14.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:22:13 smithi082 ceph-mon[32238]: pgmap v610: 129 pgs: 129 active+clean; 10 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 710 MiB/s wr, 1.26k op/s 2024-05-20T16:22:14.135 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:22:13 smithi184 ceph-mon[36209]: pgmap v610: 129 pgs: 129 active+clean; 10 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 710 MiB/s wr, 1.26k op/s 2024-05-20T16:22:14.311 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:22:13 smithi160 ceph-mon[36347]: pgmap v610: 129 pgs: 129 active+clean; 10 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 710 MiB/s wr, 1.26k op/s 2024-05-20T16:22:16.110 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:22:15 smithi082 ceph-mon[32238]: pgmap v611: 129 pgs: 129 active+clean; 10 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 722 MiB/s wr, 1.20k op/s 2024-05-20T16:22:16.135 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:22:15 smithi184 ceph-mon[36209]: pgmap v611: 129 pgs: 129 active+clean; 10 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 722 MiB/s wr, 1.20k op/s 2024-05-20T16:22:16.312 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:22:15 smithi160 ceph-mon[36347]: pgmap v611: 129 pgs: 129 active+clean; 10 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 722 MiB/s wr, 1.20k op/s 2024-05-20T16:22:18.131 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:22:17 smithi082 ceph-mon[32238]: pgmap v612: 129 pgs: 129 active+clean; 10 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 685 MiB/s wr, 1.15k op/s 2024-05-20T16:22:18.135 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:22:17 smithi184 ceph-mon[36209]: pgmap v612: 129 pgs: 129 active+clean; 10 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 685 MiB/s wr, 1.15k op/s 2024-05-20T16:22:18.312 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:22:17 smithi160 ceph-mon[36347]: pgmap v612: 129 pgs: 129 active+clean; 10 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 685 MiB/s wr, 1.15k op/s 2024-05-20T16:22:18.977 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:18.977+0000 7fdeafa37640 1 -- 172.21.15.82:0/861035009 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdea81017c0 msgr2=0x7fdea8101c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:22:18.977 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:18.977+0000 7fdeafa37640 1 --2- 172.21.15.82:0/861035009 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdea81017c0 0x7fdea8101c90 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fdea001acb0 tx=0x7fdea003b920 comp rx=0 tx=0).stop 2024-05-20T16:22:18.981 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:18.982+0000 7fdeafa37640 1 -- 172.21.15.82:0/861035009 shutdown_connections 2024-05-20T16:22:18.981 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:18.982+0000 7fdeafa37640 1 --2- 172.21.15.82:0/861035009 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdea8106930 0x7fdea8106d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:22:18.982 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:18.982+0000 7fdeafa37640 1 --2- 172.21.15.82:0/861035009 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdea81017c0 0x7fdea8101c90 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:22:18.982 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:18.982+0000 7fdeafa37640 1 --2- 172.21.15.82:0/861035009 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdea8100d80 0x7fdea81011d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:22:18.982 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:18.982+0000 7fdeafa37640 1 -- 172.21.15.82:0/861035009 >> 172.21.15.82:0/861035009 conn(0x7fdea80fc370 msgr2=0x7fdea80fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:22:18.982 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:18.982+0000 7fdeafa37640 1 -- 172.21.15.82:0/861035009 shutdown_connections 2024-05-20T16:22:18.987 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:18.985+0000 7fdeafa37640 1 -- 172.21.15.82:0/861035009 wait complete. 2024-05-20T16:22:18.987 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:18.986+0000 7fdeafa37640 1 Processor -- start 2024-05-20T16:22:18.988 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:18.988+0000 7fdeafa37640 1 -- start start 2024-05-20T16:22:18.988 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:18.988+0000 7fdeafa37640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdea8100d80 0x7fdea80728a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:22:18.988 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:18.988+0000 7fdeafa37640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdea81017c0 0x7fdea8072de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:22:18.989 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:18.989+0000 7fdead7d3640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdea8100d80 0x7fdea80728a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:22:18.989 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:18.989+0000 7fdead7d3640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdea8100d80 0x7fdea80728a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.82:47340/0 (socket says 172.21.15.82:47340) 2024-05-20T16:22:18.989 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:18.989+0000 7fdead7d3640 1 -- 172.21.15.82:0/4000797259 learned_addr learned my addr 172.21.15.82:0/4000797259 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:22:18.989 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:18.989+0000 7fdeafa37640 1 --2- 172.21.15.82:0/4000797259 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdea8106930 0x7fdea81a6d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:22:18.989 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:18.989+0000 7fdeafa37640 1 -- 172.21.15.82:0/4000797259 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fdea810a6a0 con 0x7fdea81017c0 2024-05-20T16:22:18.989 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:18.989+0000 7fdeadfd4640 1 --2- 172.21.15.82:0/4000797259 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdea8106930 0x7fdea81a6d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:22:18.989 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:18.989+0000 7fdeafa37640 1 -- 172.21.15.82:0/4000797259 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fdea810a520 con 0x7fdea8106930 2024-05-20T16:22:18.990 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:18.989+0000 7fdeafa37640 1 -- 172.21.15.82:0/4000797259 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fdea810a820 con 0x7fdea8100d80 2024-05-20T16:22:18.990 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:18.990+0000 7fdeacfd2640 1 --2- 172.21.15.82:0/4000797259 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdea81017c0 0x7fdea8072de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:22:18.990 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:18.990+0000 7fdead7d3640 1 -- 172.21.15.82:0/4000797259 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdea8106930 msgr2=0x7fdea81a6d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:22:18.990 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:18.990+0000 7fdead7d3640 1 --2- 172.21.15.82:0/4000797259 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdea8106930 0x7fdea81a6d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:22:18.990 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:18.990+0000 7fdead7d3640 1 -- 172.21.15.82:0/4000797259 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdea81017c0 msgr2=0x7fdea8072de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:22:18.990 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:18.991+0000 7fdead7d3640 1 --2- 172.21.15.82:0/4000797259 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdea81017c0 0x7fdea8072de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:22:18.990 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:18.991+0000 7fdead7d3640 1 -- 172.21.15.82:0/4000797259 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdea81a72f0 con 0x7fdea8100d80 2024-05-20T16:22:18.991 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:18.991+0000 7fdead7d3640 1 --2- 172.21.15.82:0/4000797259 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdea8100d80 0x7fdea80728a0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fde9800c920 tx=0x7fde9800cde0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:22:18.996 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:18.996+0000 7fde9e7fc640 1 -- 172.21.15.82:0/4000797259 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde98007c70 con 0x7fdea8100d80 2024-05-20T16:22:18.996 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:18.996+0000 7fdeafa37640 1 -- 172.21.15.82:0/4000797259 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdea81a7580 con 0x7fdea8100d80 2024-05-20T16:22:18.996 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:18.997+0000 7fdeafa37640 1 -- 172.21.15.82:0/4000797259 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdea81a7a70 con 0x7fdea8100d80 2024-05-20T16:22:19.003 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:19.001+0000 7fdeb69b5740 1 Processor -- start 2024-05-20T16:22:19.004 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:19.004+0000 7fdeb69b5740 1 -- start start 2024-05-20T16:22:19.004 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:19.004+0000 7fdeb69b5740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563155f441b0 0x5631562e4170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:22:19.004 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:19.004+0000 7fdeb69b5740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563155f1b2f0 0x5631562e46f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:22:19.004 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:19.004+0000 7fdeb69b5740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563155f234d0 0x5631562eb5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:22:19.004 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:19.004+0000 7fdeb69b5740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x563156248340 con 0x563155f441b0 2024-05-20T16:22:19.004 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:19.004+0000 7fdeb69b5740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x5631561f0290 con 0x563155f1b2f0 2024-05-20T16:22:19.004 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:19.004+0000 7fdeb69b5740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x563156244300 con 0x563155f234d0 2024-05-20T16:22:19.005 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:19.005+0000 7fdeadfd4640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563155f441b0 0x5631562e4170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:22:19.005 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:19.005+0000 7fdeadfd4640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563155f441b0 0x5631562e4170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:33242/0 (socket says 172.21.15.82:33242) 2024-05-20T16:22:19.005 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:19.005+0000 7fdeadfd4640 1 -- 172.21.15.82:0/145693873 learned_addr learned my addr 172.21.15.82:0/145693873 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:22:19.005 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:19.005+0000 7fdead7d3640 1 --2- 172.21.15.82:0/145693873 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563155f1b2f0 0x5631562e46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:22:19.005 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:19.005+0000 7fde9e7fc640 1 -- 172.21.15.82:0/4000797259 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fde98007e10 con 0x7fdea8100d80 2024-05-20T16:22:19.006 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:19.006+0000 7fde9e7fc640 1 -- 172.21.15.82:0/4000797259 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde980188c0 con 0x7fdea8100d80 2024-05-20T16:22:19.006 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:19.006+0000 7fde9e7fc640 1 -- 172.21.15.82:0/4000797259 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fde98018b40 con 0x7fdea8100d80 2024-05-20T16:22:19.006 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:19.006+0000 7fdead7d3640 1 -- 172.21.15.82:0/145693873 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563155f234d0 msgr2=0x5631562eb5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:22:19.006 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:19.006+0000 7fdead7d3640 1 --2- 172.21.15.82:0/145693873 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563155f234d0 0x5631562eb5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:22:19.006 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:19.006+0000 7fdead7d3640 1 -- 172.21.15.82:0/145693873 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563155f441b0 msgr2=0x5631562e4170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:22:19.006 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:19.006+0000 7fdead7d3640 1 --2- 172.21.15.82:0/145693873 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563155f441b0 0x5631562e4170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:22:19.006 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:19.006+0000 7fdead7d3640 1 -- 172.21.15.82:0/145693873 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5631562148e0 con 0x563155f1b2f0 2024-05-20T16:22:19.007 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:19.007+0000 7fdead7d3640 1 --2- 172.21.15.82:0/145693873 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563155f1b2f0 0x5631562e46f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fde980cdc20 tx=0x7fde980d8950 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1b3ab389e8d7e179 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:22:19.008 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:19.008+0000 7fde9e7fc640 1 --2- 172.21.15.82:0/4000797259 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fde940a2dc0 0x7fde940a52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:22:19.009 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:19.009+0000 7fde9e7fc640 1 -- 172.21.15.82:0/4000797259 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9150+0+0 (secure 0 0 0) 0x7fde980d04f0 con 0x7fdea8100d80 2024-05-20T16:22:19.009 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:19.010+0000 7fde92ffd640 1 -- 172.21.15.82:0/145693873 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde980e3070 con 0x563155f1b2f0 2024-05-20T16:22:19.010 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:19.010+0000 7fde92ffd640 1 -- 172.21.15.82:0/145693873 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fde980d0ce0 con 0x563155f1b2f0 2024-05-20T16:22:19.011 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:19.012+0000 7fdeb69b5740 1 -- 172.21.15.82:0/145693873 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563155f1b2f0 msgr2=0x5631562e46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:22:19.012 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:19.012+0000 7fdeb69b5740 1 --2- 172.21.15.82:0/145693873 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563155f1b2f0 0x5631562e46f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fde980cdc20 tx=0x7fde980d8950 comp rx=0 tx=0).stop 2024-05-20T16:22:19.016 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:19.016+0000 7fdeadfd4640 1 --2- 172.21.15.82:0/4000797259 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fde940a2dc0 0x7fde940a52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:22:19.016 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:19.016+0000 7fdeb69b5740 1 -- 172.21.15.82:0/145693873 shutdown_connections 2024-05-20T16:22:19.016 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:19.016+0000 7fdeb69b5740 1 --2- 172.21.15.82:0/145693873 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563155f234d0 0x5631562eb5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:22:19.016 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:19.016+0000 7fdeb69b5740 1 --2- 172.21.15.82:0/145693873 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563155f1b2f0 0x5631562e46f0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:22:19.016 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:19.016+0000 7fdeb69b5740 1 --2- 172.21.15.82:0/145693873 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563155f441b0 0x5631562e4170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:22:19.016 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:19.016+0000 7fdeb69b5740 1 -- 172.21.15.82:0/145693873 >> 172.21.15.82:0/145693873 conn(0x563155f38300 msgr2=0x5631562dfb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:22:19.017 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:19.016+0000 7fdeb69b5740 1 -- 172.21.15.82:0/145693873 shutdown_connections 2024-05-20T16:22:19.017 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:19.016+0000 7fdeb69b5740 1 -- 172.21.15.82:0/145693873 wait complete. 2024-05-20T16:22:19.017 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:19.017+0000 7fdeb69b5740 1 -- start start 2024-05-20T16:22:19.017 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:19.017+0000 7fdeb69b5740 1 Processor -- start 2024-05-20T16:22:19.019 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:19.018+0000 7fdeb69b5740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563155f1b2f0 0x5631562ede90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:22:19.019 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:19.018+0000 7fdeb69b5740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563155f234d0 0x5631562ee3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:22:19.019 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:19.018+0000 7fdeb69b5740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563155f441b0 0x5631562f5330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:22:19.019 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:19.018+0000 7fdeb69b5740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x5631562ebfb0 con 0x563155f1b2f0 2024-05-20T16:22:19.020 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:19.018+0000 7fdeb69b5740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x563156203790 con 0x563155f234d0 2024-05-20T16:22:19.020 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:19.018+0000 7fdeb69b5740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5631562ec130 con 0x563155f441b0 2024-05-20T16:22:19.020 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:19.018+0000 7fdeb69b5740 10 client.0 _ll_register_callbacks cb 0x56315623ea20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T16:22:19.023 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:19.022+0000 7fdeacfd2640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563155f234d0 0x5631562ee3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:22:19.023 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:19.022+0000 7fdead7d3640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563155f1b2f0 0x5631562ede90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:22:19.023 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:19.022+0000 7fdead7d3640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563155f1b2f0 0x5631562ede90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:33250/0 (socket says 172.21.15.82:33250) 2024-05-20T16:22:19.023 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:19.022+0000 7fdead7d3640 1 -- 172.21.15.82:0/202726611 learned_addr learned my addr 172.21.15.82:0/202726611 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:22:19.023 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:19.023+0000 7fdeadfd4640 1 --2- 172.21.15.82:0/202726611 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563155f441b0 0x5631562f5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:22:19.028 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:19.028+0000 7fdead7d3640 1 -- 172.21.15.82:0/202726611 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563155f441b0 msgr2=0x5631562f5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:22:19.028 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:19.028+0000 7fdead7d3640 1 --2- 172.21.15.82:0/202726611 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563155f441b0 0x5631562f5330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:22:19.028 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:19.028+0000 7fdead7d3640 1 -- 172.21.15.82:0/202726611 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563155f234d0 msgr2=0x5631562ee3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:22:19.028 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:19.028+0000 7fdead7d3640 1 --2- 172.21.15.82:0/202726611 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563155f234d0 0x5631562ee3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:22:19.028 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:19.028+0000 7fdead7d3640 1 -- 172.21.15.82:0/202726611 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563155f5c210 con 0x563155f1b2f0 2024-05-20T16:22:19.028 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:19.028+0000 7fdeadfd4640 1 --2- 172.21.15.82:0/4000797259 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fde940a2dc0 0x7fde940a52d0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fdea4004650 tx=0x7fdea400bb80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:22:19.030 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:19.031+0000 7fdead7d3640 1 --2- 172.21.15.82:0/202726611 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563155f1b2f0 0x5631562ede90 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fde9800c8f0 tx=0x7fde980046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23f581c4065f76d7 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:22:19.030 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:19.031+0000 7fde90ff9640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-05-20T16:22:19.037 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:19.036+0000 7fde90ff9640 1 -- 172.21.15.82:0/202726611 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde98098c80 con 0x563155f1b2f0 2024-05-20T16:22:19.037 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:19.036+0000 7fde90ff9640 1 -- 172.21.15.82:0/202726611 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9150+0+0 (secure 0 0 0) 0x7fde980de400 con 0x563155f1b2f0 2024-05-20T16:22:19.038 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:19.038+0000 7fdeb69b5740 1 -- 172.21.15.82:0/202726611 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fde980d0ce0 con 0x563155f1b2f0 2024-05-20T16:22:19.044 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:19.040+0000 7fde90ff9640 1 -- 172.21.15.82:0/202726611 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fde9809c340 con 0x563155f1b2f0 2024-05-20T16:22:19.044 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:19.044+0000 7fdeb69b5740 10 client.15852 fetch_fsmap learned FSMap version 27 2024-05-20T16:22:19.044 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:19.044+0000 7fdeb69b5740 1 -- 172.21.15.82:0/202726611 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x563156028620 con 0x563155f1b2f0 2024-05-20T16:22:19.051 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:19.052+0000 7fde90ff9640 1 -- 172.21.15.82:0/202726611 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fde980d0ce0 con 0x563155f1b2f0 2024-05-20T16:22:19.052 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:19.052+0000 7fde90ff9640 1 -- 172.21.15.82:0/202726611 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fde98020030 con 0x563155f1b2f0 2024-05-20T16:22:19.052 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:19.052+0000 7fdeb69b5740 10 client.15852 fetch_fsmap finished waiting for FSMap version 27 2024-05-20T16:22:19.052 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:19.052+0000 7fdeb69b5740 10 client.15852 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T16:22:19.052 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:19.052+0000 7fdeb69b5740 20 client.15852 populate_metadata read hostname 'smithi082' 2024-05-20T16:22:19.055 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:19.052+0000 7fdeb69b5740 1 --2- 172.21.15.82:0/202726611 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x563155f02d80 0x5631562fb3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:22:19.055 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:19.052+0000 7fdeb69b5740 4 client.15852 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T16:22:19.055 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:19.052+0000 7fdeb69b5740 1 -- 172.21.15.82:0/202726611 --> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563156255a70 con 0x563155f02d80 2024-05-20T16:22:19.059 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:19.060+0000 7fdeacfd2640 1 --2- 172.21.15.82:0/202726611 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x563155f02d80 0x5631562fb3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:22:19.074 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:19.075+0000 7fde90ff9640 10 client.15852 ms_handle_connect on v2:172.21.15.160:6838/2765554482 2024-05-20T16:22:19.074 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:19.075+0000 7fdeacfd2640 1 --2- 172.21.15.82:0/202726611 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x563155f02d80 0x5631562fb3f0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f08de736267c626 server_cookie=635bfacfc9f05ca4 in_seq=0 out_seq=0 2024-05-20T16:22:19.076 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:19.077+0000 7fde90ff9640 1 -- 172.21.15.82:0/202726611 <== mds.0 v2:172.21.15.160:6838/2765554482 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x563156255a70 con 0x563155f02d80 2024-05-20T16:22:19.077 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:19.077+0000 7fde90ff9640 10 client.15852 handle_command_reply: tid=0 2024-05-20T16:22:19.077 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:19.077+0000 7fdeb69b5740 1 client.15852 shutdown 2024-05-20T16:22:19.080 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:19.080+0000 7fdeb69b5740 1 -- 172.21.15.82:0/202726611 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563155f1b2f0 msgr2=0x5631562ede90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:22:19.080 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:19.080+0000 7fdeb69b5740 1 --2- 172.21.15.82:0/202726611 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563155f1b2f0 0x5631562ede90 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fde9800c8f0 tx=0x7fde980046c0 comp rx=0 tx=0).stop 2024-05-20T16:22:19.083 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:19.084+0000 7fdeb69b5740 1 -- 172.21.15.82:0/202726611 shutdown_connections 2024-05-20T16:22:19.084 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:19.084+0000 7fdeb69b5740 1 --2- 172.21.15.82:0/202726611 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x563155f02d80 0x5631562fb3f0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:22:19.084 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:19.084+0000 7fdeb69b5740 1 --2- 172.21.15.82:0/202726611 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563155f441b0 0x5631562f5330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:22:19.084 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:19.084+0000 7fdeb69b5740 1 --2- 172.21.15.82:0/202726611 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563155f234d0 0x5631562ee3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:22:19.084 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:19.084+0000 7fdeb69b5740 1 --2- 172.21.15.82:0/202726611 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563155f1b2f0 0x5631562ede90 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:22:19.084 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:19.084+0000 7fde90ff9640 0 client.15852 ms_handle_reset on v2:172.21.15.160:6838/2765554482 2024-05-20T16:22:19.084 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:19.084+0000 7fdeb69b5740 1 -- 172.21.15.82:0/202726611 >> 172.21.15.82:0/202726611 conn(0x563155f38300 msgr2=0x5631562deb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:22:19.086 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:19.086+0000 7fdeb69b5740 1 -- 172.21.15.82:0/202726611 shutdown_connections 2024-05-20T16:22:19.086 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:19.086+0000 7fdeb69b5740 1 -- 172.21.15.82:0/202726611 wait complete. 2024-05-20T16:22:19.090 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:19.090+0000 7fdeb69b5740 20 client.15852 trim_cache size 0 max 16384 2024-05-20T16:22:19.135 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:19.135+0000 7fdeb69b5740 1 Processor -- start 2024-05-20T16:22:19.135 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:19.135+0000 7fdeb69b5740 1 -- start start 2024-05-20T16:22:19.136 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:19.136+0000 7fdeb69b5740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563155f1b2f0 0x5631562de830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:22:19.136 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:19.136+0000 7fdeb69b5740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563155f234d0 0x5631562ed470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:22:19.136 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:19.136+0000 7fdeb69b5740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563155f441b0 0x5631562ed9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:22:19.136 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:19.136+0000 7fdeb69b5740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fdea81b4910 con 0x563155f1b2f0 2024-05-20T16:22:19.136 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:19.136+0000 7fdeb69b5740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fdea81b21e0 con 0x563155f234d0 2024-05-20T16:22:19.137 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:19.136+0000 7fdeb69b5740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fdea81b0030 con 0x563155f441b0 2024-05-20T16:22:19.137 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:19.137+0000 7fdeacfd2640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563155f234d0 0x5631562ed470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:22:19.137 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:19.137+0000 7fdeadfd4640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563155f441b0 0x5631562ed9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:22:19.137 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:19.137+0000 7fdeacfd2640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563155f234d0 0x5631562ed470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.82:57806/0 (socket says 172.21.15.82:57806) 2024-05-20T16:22:19.137 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:19.137+0000 7fdeacfd2640 1 -- 172.21.15.82:0/353753062 learned_addr learned my addr 172.21.15.82:0/353753062 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:22:19.138 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:19.138+0000 7fdeacfd2640 1 -- 172.21.15.82:0/353753062 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563155f441b0 msgr2=0x5631562ed9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:22:19.138 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:19.138+0000 7fdeacfd2640 1 --2- 172.21.15.82:0/353753062 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563155f441b0 0x5631562ed9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:22:19.138 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:19.138+0000 7fdeacfd2640 1 -- 172.21.15.82:0/353753062 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563155f1b2f0 msgr2=0x5631562de830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:22:19.138 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:19.138+0000 7fdeacfd2640 1 --2- 172.21.15.82:0/353753062 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563155f1b2f0 0x5631562de830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:22:19.138 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:19.138+0000 7fdeacfd2640 1 -- 172.21.15.82:0/353753062 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5631562e3d20 con 0x563155f234d0 2024-05-20T16:22:19.138 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:19.139+0000 7fdeacfd2640 1 --2- 172.21.15.82:0/353753062 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563155f234d0 0x5631562ed470 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fdea0040890 tx=0x7fdea003be20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=162bcf9253e2503e server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:22:19.139 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:19.139+0000 7fde927fc640 1 -- 172.21.15.82:0/353753062 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdea0045040 con 0x563155f234d0 2024-05-20T16:22:19.139 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:19.140+0000 7fde927fc640 1 -- 172.21.15.82:0/353753062 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdea005c030 con 0x563155f234d0 2024-05-20T16:22:19.141 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:19.141+0000 7fde927fc640 1 -- 172.21.15.82:0/353753062 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdea00588e0 con 0x563155f234d0 2024-05-20T16:22:19.141 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:19.141+0000 7fdeb69b5740 1 -- 172.21.15.82:0/353753062 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563155f234d0 msgr2=0x5631562ed470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:22:19.141 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:19.141+0000 7fdeb69b5740 1 --2- 172.21.15.82:0/353753062 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563155f234d0 0x5631562ed470 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fdea0040890 tx=0x7fdea003be20 comp rx=0 tx=0).stop 2024-05-20T16:22:19.150 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:19.150+0000 7fdeb69b5740 1 -- 172.21.15.82:0/353753062 shutdown_connections 2024-05-20T16:22:19.150 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:19.150+0000 7fdeb69b5740 1 --2- 172.21.15.82:0/353753062 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563155f441b0 0x5631562ed9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:22:19.150 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:19.150+0000 7fdeb69b5740 1 --2- 172.21.15.82:0/353753062 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563155f234d0 0x5631562ed470 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:22:19.150 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:19.150+0000 7fdeb69b5740 1 --2- 172.21.15.82:0/353753062 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563155f1b2f0 0x5631562de830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:22:19.150 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:19.150+0000 7fdeb69b5740 1 -- 172.21.15.82:0/353753062 >> 172.21.15.82:0/353753062 conn(0x563155f38300 msgr2=0x5631562de120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:22:19.150 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:19.150+0000 7fdeb69b5740 1 -- 172.21.15.82:0/353753062 shutdown_connections 2024-05-20T16:22:19.151 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:19.150+0000 7fdeb69b5740 1 -- 172.21.15.82:0/353753062 wait complete. 2024-05-20T16:22:19.151 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:19.151+0000 7fdeb69b5740 1 -- start start 2024-05-20T16:22:19.151 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:19.151+0000 7fdeb69b5740 1 Processor -- start 2024-05-20T16:22:19.152 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:19.152+0000 7fdeb69b5740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563155f1b2f0 0x5631562f3e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:22:19.152 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:19.152+0000 7fdeb69b5740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563155f234d0 0x5631562f43d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:22:19.152 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:19.152+0000 7fdeb69b5740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563155f441b0 0x5631563618d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:22:19.152 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:19.152+0000 7fdeb69b5740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fdea81b06a0 con 0x563155f234d0 2024-05-20T16:22:19.152 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:19.152+0000 7fdeb69b5740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fdea81b1b10 con 0x563155f1b2f0 2024-05-20T16:22:19.152 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:19.152+0000 7fdeb69b5740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fdea81b0d70 con 0x563155f441b0 2024-05-20T16:22:19.152 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:19.152+0000 7fdeb69b5740 10 client.0 _ll_register_callbacks cb 0x7fde980e1070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T16:22:19.154 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:19.155+0000 7fdeadfd4640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563155f441b0 0x5631563618d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:22:19.155 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:19.155+0000 7fdead7d3640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563155f1b2f0 0x5631562f3e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:22:19.157 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:19.158+0000 7fdeacfd2640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563155f234d0 0x5631562f43d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:22:19.157 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:19.158+0000 7fdeacfd2640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563155f234d0 0x5631562f43d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:33260/0 (socket says 172.21.15.82:33260) 2024-05-20T16:22:19.157 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:19.158+0000 7fdeacfd2640 1 -- 172.21.15.82:0/1266031100 learned_addr learned my addr 172.21.15.82:0/1266031100 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:22:19.168 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:19.168+0000 7fdeacfd2640 1 -- 172.21.15.82:0/1266031100 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563155f441b0 msgr2=0x5631563618d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:22:19.168 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:19.168+0000 7fdeacfd2640 1 --2- 172.21.15.82:0/1266031100 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563155f441b0 0x5631563618d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:22:19.168 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:19.168+0000 7fdeacfd2640 1 -- 172.21.15.82:0/1266031100 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563155f1b2f0 msgr2=0x5631562f3e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:22:19.168 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:19.168+0000 7fdeacfd2640 1 --2- 172.21.15.82:0/1266031100 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563155f1b2f0 0x5631562f3e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:22:19.168 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:19.168+0000 7fdeacfd2640 1 -- 172.21.15.82:0/1266031100 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563156361e90 con 0x563155f234d0 2024-05-20T16:22:19.187 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:19.188+0000 7fdeacfd2640 1 --2- 172.21.15.82:0/1266031100 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563155f234d0 0x5631562f43d0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fdea003bf10 tx=0x7fdea001a960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d74a1e2c28e0ebb0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:22:19.193 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:19.194+0000 7fde8bfff640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-05-20T16:22:19.197 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:19.198+0000 7fde8bfff640 1 -- 172.21.15.82:0/1266031100 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdea0069020 con 0x563155f234d0 2024-05-20T16:22:19.197 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:19.198+0000 7fde8bfff640 1 -- 172.21.15.82:0/1266031100 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9150+0+0 (secure 0 0 0) 0x7fdea0040370 con 0x563155f234d0 2024-05-20T16:22:19.200 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:19.201+0000 7fdeb69b5740 1 -- 172.21.15.82:0/1266031100 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdea005c030 con 0x563155f234d0 2024-05-20T16:22:19.223 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:19.223+0000 7fde8bfff640 1 -- 172.21.15.82:0/1266031100 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fdea0045040 con 0x563155f234d0 2024-05-20T16:22:19.223 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:19.223+0000 7fdeb69b5740 10 client.15855 fetch_fsmap learned FSMap version 27 2024-05-20T16:22:19.224 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:19.224+0000 7fdeb69b5740 1 -- 172.21.15.82:0/1266031100 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x563156363150 con 0x563155f234d0 2024-05-20T16:22:19.224 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:19.225+0000 7fde8bfff640 1 -- 172.21.15.82:0/1266031100 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdea005c030 con 0x563155f234d0 2024-05-20T16:22:19.226 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:19.225+0000 7fde8bfff640 1 -- 172.21.15.82:0/1266031100 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fdea0076020 con 0x563155f234d0 2024-05-20T16:22:19.226 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:19.225+0000 7fdeb69b5740 10 client.15855 fetch_fsmap finished waiting for FSMap version 27 2024-05-20T16:22:19.226 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:19.225+0000 7fdeb69b5740 10 client.15855 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T16:22:19.226 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:19.225+0000 7fdeb69b5740 20 client.15855 populate_metadata read hostname 'smithi082' 2024-05-20T16:22:19.226 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:19.225+0000 7fdeb69b5740 1 --2- 172.21.15.82:0/1266031100 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x563155ed81c0 0x563156365b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:22:19.226 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:19.225+0000 7fdeb69b5740 4 client.15855 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-20T16:22:19.227 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:19.225+0000 7fdeb69b5740 1 -- 172.21.15.82:0/1266031100 --> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56315624fdc0 con 0x563155ed81c0 2024-05-20T16:22:19.231 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:19.232+0000 7fdead7d3640 1 --2- 172.21.15.82:0/1266031100 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x563155ed81c0 0x563156365b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:22:19.250 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:19.250+0000 7fdead7d3640 1 --2- 172.21.15.82:0/1266031100 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x563155ed81c0 0x563156365b50 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6ccc543402296fe server_cookie=b182bd6636b8b4c1 in_seq=0 out_seq=0 2024-05-20T16:22:19.250 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:19.250+0000 7fde8bfff640 10 client.15855 ms_handle_connect on v2:172.21.15.160:6838/2765554482 2024-05-20T16:22:20.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:22:19 smithi082 ceph-mon[32238]: pgmap v613: 129 pgs: 129 active+clean; 10 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 721 MiB/s wr, 1.32k op/s 2024-05-20T16:22:20.135 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:22:19 smithi184 ceph-mon[36209]: pgmap v613: 129 pgs: 129 active+clean; 10 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 721 MiB/s wr, 1.32k op/s 2024-05-20T16:22:20.312 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:22:19 smithi160 ceph-mon[36347]: pgmap v613: 129 pgs: 129 active+clean; 10 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 721 MiB/s wr, 1.32k op/s 2024-05-20T16:22:22.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:22:21 smithi082 ceph-mon[32238]: pgmap v614: 129 pgs: 129 active+clean; 10 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 635 MiB/s wr, 1.19k op/s 2024-05-20T16:22:22.135 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:22:21 smithi184 ceph-mon[36209]: pgmap v614: 129 pgs: 129 active+clean; 10 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 635 MiB/s wr, 1.19k op/s 2024-05-20T16:22:22.311 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:22:21 smithi160 ceph-mon[36347]: pgmap v614: 129 pgs: 129 active+clean; 10 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 635 MiB/s wr, 1.19k op/s 2024-05-20T16:22:24.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:22:23 smithi082 ceph-mon[32238]: pgmap v615: 129 pgs: 129 active+clean; 10 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 674 MiB/s wr, 1.26k op/s 2024-05-20T16:22:24.135 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:22:23 smithi184 ceph-mon[36209]: pgmap v615: 129 pgs: 129 active+clean; 10 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 674 MiB/s wr, 1.26k op/s 2024-05-20T16:22:24.312 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:22:23 smithi160 ceph-mon[36347]: pgmap v615: 129 pgs: 129 active+clean; 10 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 674 MiB/s wr, 1.26k op/s 2024-05-20T16:22:25.001 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:25.001+0000 7fde8bfff640 1 -- 172.21.15.82:0/1266031100 <== mds.0 v2:172.21.15.160:6838/2765554482 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56315624fdc0 con 0x563155ed81c0 2024-05-20T16:22:25.001 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:25.001+0000 7fde8bfff640 10 client.15855 handle_command_reply: tid=0 2024-05-20T16:22:25.001 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:25.001+0000 7fdeb69b5740 1 client.15855 shutdown 2024-05-20T16:22:25.002 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:25.003+0000 7fdeb69b5740 1 -- 172.21.15.82:0/1266031100 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563155f234d0 msgr2=0x5631562f43d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:22:25.003 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:25.003+0000 7fdeb69b5740 1 --2- 172.21.15.82:0/1266031100 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563155f234d0 0x5631562f43d0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fdea003bf10 tx=0x7fdea001a960 comp rx=0 tx=0).stop 2024-05-20T16:22:25.007 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:25.007+0000 7fdeb69b5740 1 -- 172.21.15.82:0/1266031100 shutdown_connections 2024-05-20T16:22:25.007 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:25.007+0000 7fdeb69b5740 1 --2- 172.21.15.82:0/1266031100 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x563155ed81c0 0x563156365b50 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:22:25.007 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:25.007+0000 7fdeb69b5740 1 --2- 172.21.15.82:0/1266031100 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563155f234d0 0x5631562f43d0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:22:25.007 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:25.007+0000 7fdeb69b5740 1 --2- 172.21.15.82:0/1266031100 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563155f441b0 0x5631563618d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:22:25.007 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:25.007+0000 7fdeb69b5740 1 --2- 172.21.15.82:0/1266031100 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563155f1b2f0 0x5631562f3e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:22:25.007 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:25.007+0000 7fdeb69b5740 1 -- 172.21.15.82:0/1266031100 >> 172.21.15.82:0/1266031100 conn(0x563155f38300 msgr2=0x5631562deb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:22:25.007 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:25.007+0000 7fde8bfff640 0 client.15855 ms_handle_reset on v2:172.21.15.160:6838/2765554482 2024-05-20T16:22:25.007 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:25.007+0000 7fdeb69b5740 1 -- 172.21.15.82:0/1266031100 shutdown_connections 2024-05-20T16:22:25.007 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:25.007+0000 7fdeb69b5740 1 -- 172.21.15.82:0/1266031100 wait complete. 2024-05-20T16:22:25.007 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:25.007+0000 7fdeb69b5740 20 client.15855 trim_cache size 0 max 16384 2024-05-20T16:22:25.008 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:{ 2024-05-20T16:22:25.008 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "message": "", 2024-05-20T16:22:25.008 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "return_code": 0 2024-05-20T16:22:25.008 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:} 2024-05-20T16:22:25.032 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:25.032+0000 7fde93fff640 1 -- 172.21.15.82:0/4000797259 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fde940a2dc0 msgr2=0x7fde940a52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:22:25.032 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:25.032+0000 7fde93fff640 1 --2- 172.21.15.82:0/4000797259 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fde940a2dc0 0x7fde940a52d0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fdea4004650 tx=0x7fdea400bb80 comp rx=0 tx=0).stop 2024-05-20T16:22:25.032 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:25.032+0000 7fde93fff640 1 -- 172.21.15.82:0/4000797259 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdea8100d80 msgr2=0x7fdea80728a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:22:25.032 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:25.032+0000 7fde93fff640 1 --2- 172.21.15.82:0/4000797259 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdea8100d80 0x7fdea80728a0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fde9800c920 tx=0x7fde9800cde0 comp rx=0 tx=0).stop 2024-05-20T16:22:25.032 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:25.032+0000 7fde93fff640 1 -- 172.21.15.82:0/4000797259 shutdown_connections 2024-05-20T16:22:25.032 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:25.032+0000 7fde93fff640 1 --2- 172.21.15.82:0/4000797259 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fde940a2dc0 0x7fde940a52d0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:22:25.033 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:25.032+0000 7fde93fff640 1 --2- 172.21.15.82:0/4000797259 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdea81017c0 0x7fdea8072de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:22:25.033 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:25.032+0000 7fde93fff640 1 --2- 172.21.15.82:0/4000797259 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdea8106930 0x7fdea81a6d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:22:25.033 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:25.032+0000 7fde93fff640 1 --2- 172.21.15.82:0/4000797259 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdea8100d80 0x7fdea80728a0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:22:25.033 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:25.033+0000 7fde93fff640 1 -- 172.21.15.82:0/4000797259 >> 172.21.15.82:0/4000797259 conn(0x7fdea80fc370 msgr2=0x7fdea80fe6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:22:25.034 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:25.035+0000 7fde93fff640 1 -- 172.21.15.82:0/4000797259 shutdown_connections 2024-05-20T16:22:25.034 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:25.035+0000 7fde93fff640 1 -- 172.21.15.82:0/4000797259 wait complete. 2024-05-20T16:22:26.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:22:25 smithi082 ceph-mon[32238]: pgmap v616: 129 pgs: 129 active+clean; 10 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 677 MiB/s wr, 1.29k op/s 2024-05-20T16:22:26.135 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:22:25 smithi184 ceph-mon[36209]: pgmap v616: 129 pgs: 129 active+clean; 10 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 677 MiB/s wr, 1.29k op/s 2024-05-20T16:22:26.311 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:22:25 smithi160 ceph-mon[36347]: pgmap v616: 129 pgs: 129 active+clean; 10 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 677 MiB/s wr, 1.29k op/s 2024-05-20T16:22:29.062 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:22:28 smithi160 ceph-mon[36347]: pgmap v617: 129 pgs: 129 active+clean; 10 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 673 MiB/s wr, 1.37k op/s 2024-05-20T16:22:29.120 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:22:28 smithi082 ceph-mon[32238]: pgmap v617: 129 pgs: 129 active+clean; 10 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 673 MiB/s wr, 1.37k op/s 2024-05-20T16:22:29.135 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:22:28 smithi184 ceph-mon[36209]: pgmap v617: 129 pgs: 129 active+clean; 10 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 673 MiB/s wr, 1.37k op/s 2024-05-20T16:22:30.110 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:22:29 smithi082 ceph-mon[32238]: pgmap v618: 129 pgs: 129 active+clean; 10 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 714 MiB/s wr, 1.45k op/s 2024-05-20T16:22:30.135 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:22:29 smithi184 ceph-mon[36209]: pgmap v618: 129 pgs: 129 active+clean; 10 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 714 MiB/s wr, 1.45k op/s 2024-05-20T16:22:30.312 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:22:29 smithi160 ceph-mon[36347]: pgmap v618: 129 pgs: 129 active+clean; 10 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 714 MiB/s wr, 1.45k op/s 2024-05-20T16:22:33.062 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:22:32 smithi160 ceph-mon[36347]: pgmap v619: 129 pgs: 129 active+clean; 10 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 653 MiB/s wr, 1.22k op/s 2024-05-20T16:22:33.114 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:22:32 smithi082 ceph-mon[32238]: pgmap v619: 129 pgs: 129 active+clean; 10 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 653 MiB/s wr, 1.22k op/s 2024-05-20T16:22:33.135 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:22:32 smithi184 ceph-mon[36209]: pgmap v619: 129 pgs: 129 active+clean; 10 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 653 MiB/s wr, 1.22k op/s 2024-05-20T16:22:34.062 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:22:33 smithi160 ceph-mon[36347]: pgmap v620: 129 pgs: 129 active+clean; 10 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 751 MiB/s wr, 1.16k op/s 2024-05-20T16:22:34.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:22:33 smithi082 ceph-mon[32238]: pgmap v620: 129 pgs: 129 active+clean; 10 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 751 MiB/s wr, 1.16k op/s 2024-05-20T16:22:34.136 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:22:33 smithi184 ceph-mon[36209]: pgmap v620: 129 pgs: 129 active+clean; 10 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 751 MiB/s wr, 1.16k op/s 2024-05-20T16:22:36.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:22:35 smithi082 ceph-mon[32238]: pgmap v621: 129 pgs: 129 active+clean; 10 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 709 MiB/s wr, 962 op/s 2024-05-20T16:22:36.135 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:22:35 smithi184 ceph-mon[36209]: pgmap v621: 129 pgs: 129 active+clean; 10 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 709 MiB/s wr, 962 op/s 2024-05-20T16:22:36.312 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:22:35 smithi160 ceph-mon[36347]: pgmap v621: 129 pgs: 129 active+clean; 10 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 709 MiB/s wr, 962 op/s 2024-05-20T16:22:38.110 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:22:37 smithi082 ceph-mon[32238]: pgmap v622: 129 pgs: 129 active+clean; 10 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 705 MiB/s wr, 913 op/s 2024-05-20T16:22:38.135 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:22:37 smithi184 ceph-mon[36209]: pgmap v622: 129 pgs: 129 active+clean; 10 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 705 MiB/s wr, 913 op/s 2024-05-20T16:22:38.285 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:38.285+0000 7f0619861640 1 -- 172.21.15.82:0/546528851 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f06141093d0 msgr2=0x7f0614109dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:22:38.285 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:38.285+0000 7f0619861640 1 --2- 172.21.15.82:0/546528851 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f06141093d0 0x7f0614109dc0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f0608009860 tx=0x7f060802a360 comp rx=0 tx=0).stop 2024-05-20T16:22:38.291 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:38.291+0000 7f0619861640 1 -- 172.21.15.82:0/546528851 shutdown_connections 2024-05-20T16:22:38.291 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:38.291+0000 7f0619861640 1 --2- 172.21.15.82:0/546528851 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f061410a300 0x7f061410a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:22:38.291 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:38.291+0000 7f0619861640 1 --2- 172.21.15.82:0/546528851 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f061410f020 0x7f061410f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:22:38.291 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:38.291+0000 7f0619861640 1 --2- 172.21.15.82:0/546528851 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f06141093d0 0x7f0614109dc0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:22:38.291 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:38.291+0000 7f0619861640 1 -- 172.21.15.82:0/546528851 >> 172.21.15.82:0/546528851 conn(0x7f061406f4f0 msgr2=0x7f0614071980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:22:38.301 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:38.298+0000 7f0619861640 1 -- 172.21.15.82:0/546528851 shutdown_connections 2024-05-20T16:22:38.301 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:38.298+0000 7f0619861640 1 -- 172.21.15.82:0/546528851 wait complete. 2024-05-20T16:22:38.302 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:38.299+0000 7f0619861640 1 Processor -- start 2024-05-20T16:22:38.302 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:38.299+0000 7f0619861640 1 -- start start 2024-05-20T16:22:38.302 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:38.300+0000 7f0619861640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f06141093d0 0x7f0614115fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:22:38.302 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:38.300+0000 7f0619861640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f061410a300 0x7f0614116500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:22:38.302 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:38.300+0000 7f0619861640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f061410f020 0x7f0614117460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:22:38.302 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:38.300+0000 7f0619861640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f061411d680 con 0x7f061410a300 2024-05-20T16:22:38.302 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:38.300+0000 7f0619861640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f061411d500 con 0x7f061410f020 2024-05-20T16:22:38.302 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:38.300+0000 7f0619861640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f061411d800 con 0x7f06141093d0 2024-05-20T16:22:38.302 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:38.301+0000 7f06127fc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f061410a300 0x7f0614116500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:22:38.302 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:38.301+0000 7f06127fc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f061410a300 0x7f0614116500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:52568/0 (socket says 172.21.15.82:52568) 2024-05-20T16:22:38.302 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:38.301+0000 7f06127fc640 1 -- 172.21.15.82:0/1703235830 learned_addr learned my addr 172.21.15.82:0/1703235830 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:22:38.302 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:38.301+0000 7f0612ffd640 1 --2- 172.21.15.82:0/1703235830 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f06141093d0 0x7f0614115fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:22:38.304 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:38.302+0000 7f0612ffd640 1 -- 172.21.15.82:0/1703235830 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f061410f020 msgr2=0x7f0614117460 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-20T16:22:38.304 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:38.302+0000 7f0612ffd640 1 --2- 172.21.15.82:0/1703235830 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f061410f020 0x7f0614117460 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:22:38.304 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:38.302+0000 7f0612ffd640 1 -- 172.21.15.82:0/1703235830 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f061410a300 msgr2=0x7f0614116500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:22:38.304 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:38.302+0000 7f0612ffd640 1 --2- 172.21.15.82:0/1703235830 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f061410a300 0x7f0614116500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:22:38.304 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:38.302+0000 7f0612ffd640 1 -- 172.21.15.82:0/1703235830 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0614117ad0 con 0x7f06141093d0 2024-05-20T16:22:38.304 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:38.303+0000 7f0612ffd640 1 --2- 172.21.15.82:0/1703235830 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f06141093d0 0x7f0614115fc0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f0608009780 tx=0x7f0608030ee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:22:38.304 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:38.304+0000 7f061885f640 1 -- 172.21.15.82:0/1703235830 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f060803f020 con 0x7f06141093d0 2024-05-20T16:22:38.305 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:38.304+0000 7f0619861640 1 -- 172.21.15.82:0/1703235830 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0614117d60 con 0x7f06141093d0 2024-05-20T16:22:38.305 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:38.305+0000 7f0619861640 1 -- 172.21.15.82:0/1703235830 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06141182a0 con 0x7f06141093d0 2024-05-20T16:22:38.305 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:38.305+0000 7f061885f640 1 -- 172.21.15.82:0/1703235830 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0608033d60 con 0x7f06141093d0 2024-05-20T16:22:38.305 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:38.305+0000 7f061885f640 1 -- 172.21.15.82:0/1703235830 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0608037040 con 0x7f06141093d0 2024-05-20T16:22:38.311 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:22:37 smithi160 ceph-mon[36347]: pgmap v622: 129 pgs: 129 active+clean; 10 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 705 MiB/s wr, 913 op/s 2024-05-20T16:22:38.328 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:38.325+0000 7f061885f640 1 -- 172.21.15.82:0/1703235830 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f0608042070 con 0x7f06141093d0 2024-05-20T16:22:38.328 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:38.327+0000 7f061885f640 1 --2- 172.21.15.82:0/1703235830 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f05fc0a2cc0 0x7f05fc0a51d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:22:38.328 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:38.327+0000 7f06127fc640 1 --2- 172.21.15.82:0/1703235830 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f05fc0a2cc0 0x7f05fc0a51d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:22:38.328 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:38.328+0000 7f061885f640 1 -- 172.21.15.82:0/1703235830 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9150+0+0 (secure 0 0 0) 0x7f06080ef790 con 0x7f06141093d0 2024-05-20T16:22:38.350 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:38.345+0000 7f06207e6740 1 Processor -- start 2024-05-20T16:22:38.350 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:38.345+0000 7f06127fc640 1 --2- 172.21.15.82:0/1703235830 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f05fc0a2cc0 0x7f05fc0a51d0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f0614117060 tx=0x7f0604008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:22:38.350 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:38.345+0000 7f06207e6740 1 -- start start 2024-05-20T16:22:38.350 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:38.345+0000 7f06207e6740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5651e8bb01b0 0x5651e8f4fe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:22:38.350 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:38.345+0000 7f06207e6740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5651e8b872f0 0x5651e8f503e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:22:38.350 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:38.346+0000 7f06207e6740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5651e8b8f4d0 0x5651e8f572e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:22:38.350 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:38.346+0000 7f06207e6740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x5651e8eb4340 con 0x5651e8bb01b0 2024-05-20T16:22:38.350 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:38.346+0000 7f06207e6740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x5651e8e5c290 con 0x5651e8b872f0 2024-05-20T16:22:38.350 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:38.346+0000 7f06207e6740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5651e8eb0300 con 0x5651e8b8f4d0 2024-05-20T16:22:38.352 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:38.350+0000 7f06137fe640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5651e8b8f4d0 0x5651e8f572e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:22:38.353 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:38.350+0000 7f06127fc640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5651e8b872f0 0x5651e8f503e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:22:38.353 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:38.350+0000 7f0612ffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5651e8bb01b0 0x5651e8f4fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:22:38.353 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:38.350+0000 7f0612ffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5651e8bb01b0 0x5651e8f4fe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:52586/0 (socket says 172.21.15.82:52586) 2024-05-20T16:22:38.353 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:38.350+0000 7f0612ffd640 1 -- 172.21.15.82:0/1040683874 learned_addr learned my addr 172.21.15.82:0/1040683874 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:22:38.353 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:38.352+0000 7f0612ffd640 1 -- 172.21.15.82:0/1040683874 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5651e8b8f4d0 msgr2=0x5651e8f572e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:22:38.354 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:38.352+0000 7f0612ffd640 1 --2- 172.21.15.82:0/1040683874 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5651e8b8f4d0 0x5651e8f572e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:22:38.354 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:38.353+0000 7f0612ffd640 1 -- 172.21.15.82:0/1040683874 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5651e8b872f0 msgr2=0x5651e8f503e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:22:38.354 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:38.353+0000 7f0612ffd640 1 --2- 172.21.15.82:0/1040683874 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5651e8b872f0 0x5651e8f503e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:22:38.354 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:38.353+0000 7f0612ffd640 1 -- 172.21.15.82:0/1040683874 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5651e8e808e0 con 0x5651e8bb01b0 2024-05-20T16:22:38.354 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:38.353+0000 7f0612ffd640 1 --2- 172.21.15.82:0/1040683874 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5651e8bb01b0 0x5651e8f4fe60 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f060802a950 tx=0x7f060802a360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbcbe0d19cfd4087 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:22:38.354 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:38.354+0000 7f05f9ffb640 1 -- 172.21.15.82:0/1040683874 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06080bbc00 con 0x5651e8bb01b0 2024-05-20T16:22:38.355 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:38.354+0000 7f05f9ffb640 1 -- 172.21.15.82:0/1040683874 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f06080d2070 con 0x5651e8bb01b0 2024-05-20T16:22:38.356 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:38.356+0000 7f06207e6740 1 -- 172.21.15.82:0/1040683874 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5651e8bb01b0 msgr2=0x5651e8f4fe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:22:38.357 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:38.356+0000 7f06207e6740 1 --2- 172.21.15.82:0/1040683874 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5651e8bb01b0 0x5651e8f4fe60 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f060802a950 tx=0x7f060802a360 comp rx=0 tx=0).stop 2024-05-20T16:22:38.357 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:38.356+0000 7f06207e6740 1 -- 172.21.15.82:0/1040683874 shutdown_connections 2024-05-20T16:22:38.357 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:38.356+0000 7f06207e6740 1 --2- 172.21.15.82:0/1040683874 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5651e8b8f4d0 0x5651e8f572e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:22:38.357 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:38.356+0000 7f06207e6740 1 --2- 172.21.15.82:0/1040683874 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5651e8b872f0 0x5651e8f503e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:22:38.357 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:38.356+0000 7f06207e6740 1 --2- 172.21.15.82:0/1040683874 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5651e8bb01b0 0x5651e8f4fe60 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:22:38.357 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:38.356+0000 7f06207e6740 1 -- 172.21.15.82:0/1040683874 >> 172.21.15.82:0/1040683874 conn(0x5651e8ba4300 msgr2=0x5651e8f4b7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:22:38.357 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:38.356+0000 7f06207e6740 1 -- 172.21.15.82:0/1040683874 shutdown_connections 2024-05-20T16:22:38.358 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:38.357+0000 7f06207e6740 1 -- 172.21.15.82:0/1040683874 wait complete. 2024-05-20T16:22:38.358 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:38.357+0000 7f06207e6740 1 -- start start 2024-05-20T16:22:38.358 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:38.357+0000 7f06207e6740 1 Processor -- start 2024-05-20T16:22:38.360 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:38.358+0000 7f06207e6740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5651e8b872f0 0x5651e8f59800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:22:38.360 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:38.358+0000 7f06207e6740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5651e8b8f4d0 0x5651e8f59d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:22:38.360 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:38.358+0000 7f06207e6740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5651e8b6ed80 0x5651e8f60ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:22:38.360 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:38.358+0000 7f06207e6740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x5651e8f57af0 con 0x5651e8b872f0 2024-05-20T16:22:38.360 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:38.358+0000 7f06207e6740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x5651e8e6f790 con 0x5651e8b8f4d0 2024-05-20T16:22:38.360 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:38.358+0000 7f06207e6740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5651e8f57c70 con 0x5651e8b6ed80 2024-05-20T16:22:38.360 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:38.358+0000 7f06207e6740 10 client.0 _ll_register_callbacks cb 0x5651e8eaaa20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T16:22:38.360 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:38.359+0000 7f0612ffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5651e8b872f0 0x5651e8f59800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:22:38.360 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:38.359+0000 7f0612ffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5651e8b872f0 0x5651e8f59800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:52592/0 (socket says 172.21.15.82:52592) 2024-05-20T16:22:38.360 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:38.359+0000 7f0612ffd640 1 -- 172.21.15.82:0/3018792323 learned_addr learned my addr 172.21.15.82:0/3018792323 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:22:38.361 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:38.360+0000 7f0612ffd640 1 -- 172.21.15.82:0/3018792323 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5651e8b6ed80 msgr2=0x5651e8f60ca0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-20T16:22:38.361 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:38.360+0000 7f0612ffd640 1 --2- 172.21.15.82:0/3018792323 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5651e8b6ed80 0x5651e8f60ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:22:38.361 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:38.360+0000 7f0612ffd640 1 -- 172.21.15.82:0/3018792323 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5651e8b8f4d0 msgr2=0x5651e8f59d40 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-20T16:22:38.361 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:38.360+0000 7f0612ffd640 1 --2- 172.21.15.82:0/3018792323 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5651e8b8f4d0 0x5651e8f59d40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:22:38.362 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:38.360+0000 7f0612ffd640 1 -- 172.21.15.82:0/3018792323 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5651e8bc8210 con 0x5651e8b872f0 2024-05-20T16:22:38.362 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:38.360+0000 7f0612ffd640 1 --2- 172.21.15.82:0/3018792323 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5651e8b872f0 0x5651e8f59800 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f06080bb900 tx=0x7f0608005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aae08ae54bfe11a9 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:22:38.362 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:38.361+0000 7f05f2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-05-20T16:22:38.362 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:38.362+0000 7f05f2ffd640 1 -- 172.21.15.82:0/3018792323 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06080bbc00 con 0x5651e8b872f0 2024-05-20T16:22:38.363 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:38.363+0000 7f06207e6740 1 -- 172.21.15.82:0/3018792323 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5651e8ec1a70 con 0x5651e8b872f0 2024-05-20T16:22:38.363 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:38.363+0000 7f05f2ffd640 1 -- 172.21.15.82:0/3018792323 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9150+0+0 (secure 0 0 0) 0x7f06080c43e0 con 0x5651e8b872f0 2024-05-20T16:22:38.364 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:38.364+0000 7f05f2ffd640 1 -- 172.21.15.82:0/3018792323 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f06080a2050 con 0x5651e8b872f0 2024-05-20T16:22:38.373 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:38.373+0000 7f06207e6740 10 client.15864 fetch_fsmap learned FSMap version 27 2024-05-20T16:22:38.373 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:38.373+0000 7f06207e6740 1 -- 172.21.15.82:0/3018792323 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5651e8c94620 con 0x5651e8b872f0 2024-05-20T16:22:38.374 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:38.375+0000 7f05f2ffd640 1 -- 172.21.15.82:0/3018792323 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5651e8ec1a70 con 0x5651e8b872f0 2024-05-20T16:22:38.374 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:38.375+0000 7f05f2ffd640 1 -- 172.21.15.82:0/3018792323 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f060803a070 con 0x5651e8b872f0 2024-05-20T16:22:38.375 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:38.375+0000 7f06207e6740 10 client.15864 fetch_fsmap finished waiting for FSMap version 27 2024-05-20T16:22:38.375 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:38.375+0000 7f06207e6740 10 client.15864 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T16:22:38.376 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:38.375+0000 7f06207e6740 20 client.15864 populate_metadata read hostname 'smithi082' 2024-05-20T16:22:38.376 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:38.375+0000 7f06207e6740 1 --2- 172.21.15.82:0/3018792323 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x5651e8b441c0 0x5651e8f67090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:22:38.376 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:38.375+0000 7f06207e6740 4 client.15864 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T16:22:38.376 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:38.375+0000 7f06207e6740 1 -- 172.21.15.82:0/3018792323 --> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5651e8ebbdc0 con 0x5651e8b441c0 2024-05-20T16:22:38.379 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:38.380+0000 7f06127fc640 1 --2- 172.21.15.82:0/3018792323 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x5651e8b441c0 0x5651e8f67090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:22:38.404 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:38.404+0000 7f06127fc640 1 --2- 172.21.15.82:0/3018792323 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x5651e8b441c0 0x5651e8f67090 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5d6f4e3ff738c4f0 server_cookie=3bfc8d88136e83ca in_seq=0 out_seq=0 2024-05-20T16:22:38.405 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:38.405+0000 7f05f2ffd640 10 client.15864 ms_handle_connect on v2:172.21.15.160:6838/2765554482 2024-05-20T16:22:38.409 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:38.409+0000 7f05f2ffd640 1 -- 172.21.15.82:0/3018792323 <== mds.0 v2:172.21.15.160:6838/2765554482 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5651e8ebbdc0 con 0x5651e8b441c0 2024-05-20T16:22:38.409 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:38.409+0000 7f05f2ffd640 10 client.15864 handle_command_reply: tid=0 2024-05-20T16:22:38.409 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:38.409+0000 7f06207e6740 1 client.15864 shutdown 2024-05-20T16:22:38.412 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:38.410+0000 7f06207e6740 1 -- 172.21.15.82:0/3018792323 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5651e8b872f0 msgr2=0x5651e8f59800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:22:38.412 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:38.410+0000 7f06207e6740 1 --2- 172.21.15.82:0/3018792323 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5651e8b872f0 0x5651e8f59800 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f06080bb900 tx=0x7f0608005bd0 comp rx=0 tx=0).stop 2024-05-20T16:22:38.417 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:38.418+0000 7f06207e6740 1 -- 172.21.15.82:0/3018792323 shutdown_connections 2024-05-20T16:22:38.418 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:38.418+0000 7f06207e6740 1 --2- 172.21.15.82:0/3018792323 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x5651e8b441c0 0x5651e8f67090 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:22:38.418 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:38.418+0000 7f06207e6740 1 --2- 172.21.15.82:0/3018792323 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5651e8b6ed80 0x5651e8f60ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:22:38.418 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:38.418+0000 7f06207e6740 1 --2- 172.21.15.82:0/3018792323 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5651e8b8f4d0 0x5651e8f59d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:22:38.418 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:38.418+0000 7f06207e6740 1 --2- 172.21.15.82:0/3018792323 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5651e8b872f0 0x5651e8f59800 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:22:38.418 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:38.418+0000 7f06207e6740 1 -- 172.21.15.82:0/3018792323 >> 172.21.15.82:0/3018792323 conn(0x5651e8ba4300 msgr2=0x5651e8f4e810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:22:38.418 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:38.418+0000 7f05f2ffd640 0 client.15864 ms_handle_reset on v2:172.21.15.160:6838/2765554482 2024-05-20T16:22:38.424 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:38.422+0000 7f06207e6740 1 -- 172.21.15.82:0/3018792323 shutdown_connections 2024-05-20T16:22:38.428 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:38.426+0000 7f06207e6740 1 -- 172.21.15.82:0/3018792323 wait complete. 2024-05-20T16:22:38.429 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:38.428+0000 7f06207e6740 20 client.15864 trim_cache size 0 max 16384 2024-05-20T16:22:38.482 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:38.480+0000 7f06207e6740 1 Processor -- start 2024-05-20T16:22:38.482 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:38.481+0000 7f06207e6740 1 -- start start 2024-05-20T16:22:38.482 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:38.481+0000 7f06207e6740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5651e8b6ed80 0x5651e8f59df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:22:38.482 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:38.481+0000 7f06207e6740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5651e8b872f0 0x5651e8f58780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:22:38.483 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:38.481+0000 7f06207e6740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5651e8b8f4d0 0x5651e8f4e690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:22:38.483 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:38.481+0000 7f06207e6740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f0600012550 con 0x5651e8b6ed80 2024-05-20T16:22:38.483 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:38.481+0000 7f06207e6740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f060000fe20 con 0x5651e8b8f4d0 2024-05-20T16:22:38.484 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:38.481+0000 7f06207e6740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f060000dc70 con 0x5651e8b872f0 2024-05-20T16:22:38.487 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:38.488+0000 7f0612ffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5651e8b6ed80 0x5651e8f59df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:22:38.488 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:38.488+0000 7f0612ffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5651e8b6ed80 0x5651e8f59df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:52602/0 (socket says 172.21.15.82:52602) 2024-05-20T16:22:38.488 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:38.488+0000 7f0612ffd640 1 -- 172.21.15.82:0/1799347619 learned_addr learned my addr 172.21.15.82:0/1799347619 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:22:38.493 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:38.493+0000 7f0612ffd640 1 -- 172.21.15.82:0/1799347619 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5651e8b872f0 msgr2=0x5651e8f58780 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-20T16:22:38.493 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:38.493+0000 7f0612ffd640 1 --2- 172.21.15.82:0/1799347619 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5651e8b872f0 0x5651e8f58780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:22:38.493 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:38.493+0000 7f0612ffd640 1 -- 172.21.15.82:0/1799347619 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5651e8b8f4d0 msgr2=0x5651e8f4e690 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-20T16:22:38.493 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:38.493+0000 7f0612ffd640 1 --2- 172.21.15.82:0/1799347619 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5651e8b8f4d0 0x5651e8f4e690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:22:38.493 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:38.493+0000 7f0612ffd640 1 -- 172.21.15.82:0/1799347619 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5651e8f5a870 con 0x5651e8b6ed80 2024-05-20T16:22:38.496 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:38.494+0000 7f0612ffd640 1 --2- 172.21.15.82:0/1799347619 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5651e8b6ed80 0x5651e8f59df0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f06080c4340 tx=0x7f060802a360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=989734fe7bd5f521 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:22:38.497 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:38.495+0000 7f05f97fa640 1 -- 172.21.15.82:0/1799347619 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06080b8380 con 0x5651e8b6ed80 2024-05-20T16:22:38.497 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:38.495+0000 7f05f97fa640 1 -- 172.21.15.82:0/1799347619 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f06080d9070 con 0x5651e8b6ed80 2024-05-20T16:22:38.497 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:38.495+0000 7f05f97fa640 1 -- 172.21.15.82:0/1799347619 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06080b9d40 con 0x5651e8b6ed80 2024-05-20T16:22:38.497 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:38.496+0000 7f06207e6740 1 -- 172.21.15.82:0/1799347619 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5651e8b6ed80 msgr2=0x5651e8f59df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:22:38.497 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:38.496+0000 7f06207e6740 1 --2- 172.21.15.82:0/1799347619 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5651e8b6ed80 0x5651e8f59df0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f06080c4340 tx=0x7f060802a360 comp rx=0 tx=0).stop 2024-05-20T16:22:38.505 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:38.505+0000 7f06207e6740 1 -- 172.21.15.82:0/1799347619 shutdown_connections 2024-05-20T16:22:38.505 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:38.505+0000 7f06207e6740 1 --2- 172.21.15.82:0/1799347619 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5651e8b8f4d0 0x5651e8f4e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:22:38.505 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:38.505+0000 7f06207e6740 1 --2- 172.21.15.82:0/1799347619 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5651e8b872f0 0x5651e8f58780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:22:38.505 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:38.505+0000 7f06207e6740 1 --2- 172.21.15.82:0/1799347619 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5651e8b6ed80 0x5651e8f59df0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:22:38.505 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:38.505+0000 7f06207e6740 1 -- 172.21.15.82:0/1799347619 >> 172.21.15.82:0/1799347619 conn(0x5651e8ba4300 msgr2=0x5651e8f58070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:22:38.513 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:38.514+0000 7f06207e6740 1 -- 172.21.15.82:0/1799347619 shutdown_connections 2024-05-20T16:22:38.519 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:38.520+0000 7f06207e6740 1 -- 172.21.15.82:0/1799347619 wait complete. 2024-05-20T16:22:38.520 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:38.520+0000 7f06207e6740 1 -- start start 2024-05-20T16:22:38.520 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:38.521+0000 7f06207e6740 1 Processor -- start 2024-05-20T16:22:38.523 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:38.522+0000 7f06207e6740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5651e8b6ed80 0x5651e8f5fc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:22:38.523 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:38.522+0000 7f06207e6740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5651e8b872f0 0x5651e8f601a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:22:38.523 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:38.523+0000 7f06207e6740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5651e8b8f4d0 0x5651e8f5c830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:22:38.523 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:38.523+0000 7f0612ffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5651e8b6ed80 0x5651e8f5fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:22:38.523 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:38.523+0000 7f06207e6740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f060000e2e0 con 0x5651e8b6ed80 2024-05-20T16:22:38.523 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:38.523+0000 7f0612ffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5651e8b6ed80 0x5651e8f5fc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:52612/0 (socket says 172.21.15.82:52612) 2024-05-20T16:22:38.523 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:38.523+0000 7f06207e6740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f060000f750 con 0x5651e8b8f4d0 2024-05-20T16:22:38.523 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:38.523+0000 7f0612ffd640 1 -- 172.21.15.82:0/1980390060 learned_addr learned my addr 172.21.15.82:0/1980390060 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:22:38.523 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:38.523+0000 7f06207e6740 1 -- 172.21.15.82:0/1980390060 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f060000e9b0 con 0x5651e8b872f0 2024-05-20T16:22:38.524 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:38.523+0000 7f06207e6740 10 client.0 _ll_register_callbacks cb 0x7f06080bba60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T16:22:38.524 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:38.523+0000 7f06137fe640 1 --2- 172.21.15.82:0/1980390060 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5651e8b8f4d0 0x5651e8f5c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:22:38.524 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:38.523+0000 7f06127fc640 1 --2- 172.21.15.82:0/1980390060 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5651e8b872f0 0x5651e8f601a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:22:38.524 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:38.524+0000 7f06127fc640 1 -- 172.21.15.82:0/1980390060 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5651e8b8f4d0 msgr2=0x5651e8f5c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:22:38.524 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:38.524+0000 7f06127fc640 1 --2- 172.21.15.82:0/1980390060 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5651e8b8f4d0 0x5651e8f5c830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:22:38.524 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:38.524+0000 7f06127fc640 1 -- 172.21.15.82:0/1980390060 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5651e8b6ed80 msgr2=0x5651e8f5fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:22:38.524 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:38.524+0000 7f06127fc640 1 --2- 172.21.15.82:0/1980390060 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5651e8b6ed80 0x5651e8f5fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:22:38.524 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:38.524+0000 7f06127fc640 1 -- 172.21.15.82:0/1980390060 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5651e8f5cf20 con 0x5651e8b872f0 2024-05-20T16:22:38.530 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:38.530+0000 7f06127fc640 1 --2- 172.21.15.82:0/1980390060 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5651e8b872f0 0x5651e8f601a0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x5651e8f677d0 tx=0x7f06040046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=caa7f077849c0968 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:22:38.530 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:38.531+0000 7f05f2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-05-20T16:22:38.531 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:38.531+0000 7f05f2ffd640 1 -- 172.21.15.82:0/1980390060 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06040059c0 con 0x5651e8b872f0 2024-05-20T16:22:38.531 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:38.531+0000 7f05f2ffd640 1 -- 172.21.15.82:0/1980390060 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9150+0+0 (secure 0 0 0) 0x7f060400f040 con 0x5651e8b872f0 2024-05-20T16:22:38.531 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:38.531+0000 7f06207e6740 1 -- 172.21.15.82:0/1980390060 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f06080d9070 con 0x5651e8b872f0 2024-05-20T16:22:38.532 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:38.533+0000 7f05f2ffd640 1 -- 172.21.15.82:0/1980390060 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f0604034050 con 0x5651e8b872f0 2024-05-20T16:22:38.532 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:38.533+0000 7f06207e6740 10 client.25399 fetch_fsmap learned FSMap version 27 2024-05-20T16:22:38.533 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:38.533+0000 7f06207e6740 1 -- 172.21.15.82:0/1980390060 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5651e8fce0d0 con 0x5651e8b872f0 2024-05-20T16:22:38.538 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:38.537+0000 7f05f2ffd640 1 -- 172.21.15.82:0/1980390060 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f06080d9070 con 0x5651e8b872f0 2024-05-20T16:22:38.538 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:38.538+0000 7f05f2ffd640 1 -- 172.21.15.82:0/1980390060 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f0604013070 con 0x5651e8b872f0 2024-05-20T16:22:38.538 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:38.538+0000 7f06207e6740 10 client.25399 fetch_fsmap finished waiting for FSMap version 27 2024-05-20T16:22:38.538 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:38.538+0000 7f06207e6740 10 client.25399 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T16:22:38.538 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:38.538+0000 7f06207e6740 20 client.25399 populate_metadata read hostname 'smithi082' 2024-05-20T16:22:38.538 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:38.538+0000 7f06207e6740 1 --2- 172.21.15.82:0/1980390060 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x5651e8fcea80 0x5651e8fd0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:22:38.538 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:38.538+0000 7f06207e6740 4 client.25399 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-20T16:22:38.538 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:38.538+0000 7f06207e6740 1 -- 172.21.15.82:0/1980390060 --> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5651e8e398e0 con 0x5651e8fcea80 2024-05-20T16:22:38.539 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:38.539+0000 7f0612ffd640 1 --2- 172.21.15.82:0/1980390060 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x5651e8fcea80 0x5651e8fd0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:22:38.542 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:38.542+0000 7f0612ffd640 1 --2- 172.21.15.82:0/1980390060 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x5651e8fcea80 0x5651e8fd0ea0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c783a4a98953b2a3 server_cookie=b2ca59c2ba6f7ff4 in_seq=0 out_seq=0 2024-05-20T16:22:38.542 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:38.542+0000 7f05f2ffd640 10 client.25399 ms_handle_connect on v2:172.21.15.160:6838/2765554482 2024-05-20T16:22:40.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:22:39 smithi082 ceph-mon[32238]: pgmap v623: 129 pgs: 129 active+clean; 10 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 811 MiB/s wr, 1.01k op/s 2024-05-20T16:22:40.135 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:22:39 smithi184 ceph-mon[36209]: pgmap v623: 129 pgs: 129 active+clean; 10 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 811 MiB/s wr, 1.01k op/s 2024-05-20T16:22:40.199 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:40.198+0000 7f05f2ffd640 1 -- 172.21.15.82:0/1980390060 <== mds.0 v2:172.21.15.160:6838/2765554482 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5651e8e398e0 con 0x5651e8fcea80 2024-05-20T16:22:40.200 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:40.198+0000 7f05f2ffd640 10 client.25399 handle_command_reply: tid=0 2024-05-20T16:22:40.200 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:40.198+0000 7f06207e6740 1 client.25399 shutdown 2024-05-20T16:22:40.200 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:40.198+0000 7f06207e6740 1 -- 172.21.15.82:0/1980390060 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5651e8b872f0 msgr2=0x5651e8f601a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:22:40.200 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:40.198+0000 7f06207e6740 1 --2- 172.21.15.82:0/1980390060 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5651e8b872f0 0x5651e8f601a0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x5651e8f677d0 tx=0x7f06040046c0 comp rx=0 tx=0).stop 2024-05-20T16:22:40.200 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:40.198+0000 7f06207e6740 1 -- 172.21.15.82:0/1980390060 shutdown_connections 2024-05-20T16:22:40.200 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:40.198+0000 7f06207e6740 1 --2- 172.21.15.82:0/1980390060 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x5651e8fcea80 0x5651e8fd0ea0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:22:40.200 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:40.198+0000 7f06207e6740 1 --2- 172.21.15.82:0/1980390060 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5651e8b8f4d0 0x5651e8f5c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:22:40.200 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:40.198+0000 7f06207e6740 1 --2- 172.21.15.82:0/1980390060 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5651e8b872f0 0x5651e8f601a0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:22:40.200 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:40.198+0000 7f06207e6740 1 --2- 172.21.15.82:0/1980390060 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5651e8b6ed80 0x5651e8f5fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:22:40.201 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:40.198+0000 7f06207e6740 1 -- 172.21.15.82:0/1980390060 >> 172.21.15.82:0/1980390060 conn(0x5651e8ba4300 msgr2=0x5651e8f4e690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:22:40.201 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:40.198+0000 7f05f2ffd640 0 client.25399 ms_handle_reset on v2:172.21.15.160:6838/2765554482 2024-05-20T16:22:40.201 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:40.199+0000 7f06207e6740 1 -- 172.21.15.82:0/1980390060 shutdown_connections 2024-05-20T16:22:40.201 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:40.199+0000 7f06207e6740 1 -- 172.21.15.82:0/1980390060 wait complete. 2024-05-20T16:22:40.201 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:40.199+0000 7f06207e6740 20 client.25399 trim_cache size 0 max 16384 2024-05-20T16:22:40.201 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:{ 2024-05-20T16:22:40.201 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "message": "", 2024-05-20T16:22:40.201 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "return_code": 0 2024-05-20T16:22:40.201 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:} 2024-05-20T16:22:40.202 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:40.200+0000 7f0619861640 1 -- 172.21.15.82:0/1703235830 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f05fc0a2cc0 msgr2=0x7f05fc0a51d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:22:40.202 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:40.200+0000 7f0619861640 1 --2- 172.21.15.82:0/1703235830 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f05fc0a2cc0 0x7f05fc0a51d0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f0614117060 tx=0x7f0604008040 comp rx=0 tx=0).stop 2024-05-20T16:22:40.202 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:40.200+0000 7f0619861640 1 -- 172.21.15.82:0/1703235830 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f06141093d0 msgr2=0x7f0614115fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:22:40.202 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:40.200+0000 7f0619861640 1 --2- 172.21.15.82:0/1703235830 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f06141093d0 0x7f0614115fc0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f0608009780 tx=0x7f0608030ee0 comp rx=0 tx=0).stop 2024-05-20T16:22:40.202 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:40.200+0000 7f0619861640 1 -- 172.21.15.82:0/1703235830 shutdown_connections 2024-05-20T16:22:40.202 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:40.200+0000 7f0619861640 1 --2- 172.21.15.82:0/1703235830 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f05fc0a2cc0 0x7f05fc0a51d0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:22:40.202 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:40.200+0000 7f0619861640 1 --2- 172.21.15.82:0/1703235830 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f061410a300 0x7f0614116500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:22:40.203 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:40.200+0000 7f0619861640 1 --2- 172.21.15.82:0/1703235830 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f061410f020 0x7f0614117460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:22:40.203 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:40.200+0000 7f0619861640 1 --2- 172.21.15.82:0/1703235830 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f06141093d0 0x7f0614115fc0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:22:40.203 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:40.200+0000 7f0619861640 1 -- 172.21.15.82:0/1703235830 >> 172.21.15.82:0/1703235830 conn(0x7f061406f4f0 msgr2=0x7f061410b000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:22:40.203 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:40.200+0000 7f0619861640 1 -- 172.21.15.82:0/1703235830 shutdown_connections 2024-05-20T16:22:40.203 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:40.200+0000 7f0619861640 1 -- 172.21.15.82:0/1703235830 wait complete. 2024-05-20T16:22:40.312 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:22:39 smithi160 ceph-mon[36347]: pgmap v623: 129 pgs: 129 active+clean; 10 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 811 MiB/s wr, 1.01k op/s 2024-05-20T16:22:42.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:22:41 smithi082 ceph-mon[32238]: pgmap v624: 129 pgs: 129 active+clean; 10 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 704 MiB/s wr, 803 op/s 2024-05-20T16:22:42.135 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:22:41 smithi184 ceph-mon[36209]: pgmap v624: 129 pgs: 129 active+clean; 10 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 704 MiB/s wr, 803 op/s 2024-05-20T16:22:42.312 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:22:41 smithi160 ceph-mon[36347]: pgmap v624: 129 pgs: 129 active+clean; 10 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 704 MiB/s wr, 803 op/s 2024-05-20T16:22:44.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:22:43 smithi082 ceph-mon[32238]: pgmap v625: 129 pgs: 129 active+clean; 10 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 744 MiB/s wr, 936 op/s 2024-05-20T16:22:44.136 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:22:43 smithi184 ceph-mon[36209]: pgmap v625: 129 pgs: 129 active+clean; 10 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 744 MiB/s wr, 936 op/s 2024-05-20T16:22:44.312 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:22:43 smithi160 ceph-mon[36347]: pgmap v625: 129 pgs: 129 active+clean; 10 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 744 MiB/s wr, 936 op/s 2024-05-20T16:22:46.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:22:45 smithi082 ceph-mon[32238]: pgmap v626: 129 pgs: 129 active+clean; 10 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 720 MiB/s wr, 1.06k op/s 2024-05-20T16:22:46.135 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:22:45 smithi184 ceph-mon[36209]: pgmap v626: 129 pgs: 129 active+clean; 10 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 720 MiB/s wr, 1.06k op/s 2024-05-20T16:22:46.312 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:22:45 smithi160 ceph-mon[36347]: pgmap v626: 129 pgs: 129 active+clean; 10 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 720 MiB/s wr, 1.06k op/s 2024-05-20T16:22:48.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:22:47 smithi082 ceph-mon[32238]: pgmap v627: 129 pgs: 129 active+clean; 10 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 689 MiB/s wr, 1.08k op/s 2024-05-20T16:22:48.135 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:22:47 smithi184 ceph-mon[36209]: pgmap v627: 129 pgs: 129 active+clean; 10 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 689 MiB/s wr, 1.08k op/s 2024-05-20T16:22:48.312 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:22:47 smithi160 ceph-mon[36347]: pgmap v627: 129 pgs: 129 active+clean; 10 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 689 MiB/s wr, 1.08k op/s 2024-05-20T16:22:50.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:22:49 smithi082 ceph-mon[32238]: pgmap v628: 129 pgs: 129 active+clean; 10 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 822 MiB/s wr, 1.42k op/s 2024-05-20T16:22:50.135 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:22:49 smithi184 ceph-mon[36209]: pgmap v628: 129 pgs: 129 active+clean; 10 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 822 MiB/s wr, 1.42k op/s 2024-05-20T16:22:50.311 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:22:49 smithi160 ceph-mon[36347]: pgmap v628: 129 pgs: 129 active+clean; 10 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 822 MiB/s wr, 1.42k op/s 2024-05-20T16:22:52.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:22:51 smithi082 ceph-mon[32238]: pgmap v629: 129 pgs: 129 active+clean; 10 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 657 MiB/s wr, 1.22k op/s 2024-05-20T16:22:52.135 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:22:51 smithi184 ceph-mon[36209]: pgmap v629: 129 pgs: 129 active+clean; 10 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 657 MiB/s wr, 1.22k op/s 2024-05-20T16:22:52.312 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:22:51 smithi160 ceph-mon[36347]: pgmap v629: 129 pgs: 129 active+clean; 10 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 657 MiB/s wr, 1.22k op/s 2024-05-20T16:22:53.479 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:53.479+0000 7febfab48640 1 -- 172.21.15.82:0/3520717050 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7febf410ee30 msgr2=0x7febf4111250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:22:53.479 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:53.479+0000 7febfab48640 1 --2- 172.21.15.82:0/3520717050 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7febf410ee30 0x7febf4111250 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7febec009ca0 tx=0x7febec02a8f0 comp rx=0 tx=0).stop 2024-05-20T16:22:53.480 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:53.480+0000 7febfab48640 1 -- 172.21.15.82:0/3520717050 shutdown_connections 2024-05-20T16:22:53.480 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:53.480+0000 7febfab48640 1 --2- 172.21.15.82:0/3520717050 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7febf410ee30 0x7febf4111250 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:22:53.480 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:53.480+0000 7febfab48640 1 --2- 172.21.15.82:0/3520717050 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7febf410a130 0x7febf410a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:22:53.480 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:53.480+0000 7febfab48640 1 --2- 172.21.15.82:0/3520717050 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7febf41092d0 0x7febf4109bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:22:53.480 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:53.480+0000 7febfab48640 1 -- 172.21.15.82:0/3520717050 >> 172.21.15.82:0/3520717050 conn(0x7febf4077b60 msgr2=0x7febf4079ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:22:53.483 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:53.484+0000 7febfab48640 1 -- 172.21.15.82:0/3520717050 shutdown_connections 2024-05-20T16:22:53.485 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:53.485+0000 7febfab48640 1 -- 172.21.15.82:0/3520717050 wait complete. 2024-05-20T16:22:53.487 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:53.486+0000 7febfab48640 1 Processor -- start 2024-05-20T16:22:53.487 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:53.486+0000 7febfab48640 1 -- start start 2024-05-20T16:22:53.487 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:53.487+0000 7febfab48640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7febf41092d0 0x7febf40709c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:22:53.487 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:53.487+0000 7febfab48640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7febf410a130 0x7febf4070f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:22:53.487 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:53.487+0000 7febfab48640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7febf410ee30 0x7febf4071e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:22:53.487 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:53.487+0000 7febfab48640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7febf4114b90 con 0x7febf41092d0 2024-05-20T16:22:53.487 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:53.487+0000 7febfab48640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7febf4114a10 con 0x7febf410ee30 2024-05-20T16:22:53.487 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:53.487+0000 7febfab48640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7febf4114d10 con 0x7febf410a130 2024-05-20T16:22:53.487 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:53.487+0000 7febfa347640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7febf410ee30 0x7febf4071e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:22:53.488 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:53.487+0000 7febfa347640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7febf410ee30 0x7febf4071e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.82:43508/0 (socket says 172.21.15.82:43508) 2024-05-20T16:22:53.488 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:53.487+0000 7febfa347640 1 -- 172.21.15.82:0/660035768 learned_addr learned my addr 172.21.15.82:0/660035768 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:22:53.489 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:53.487+0000 7febfa347640 1 -- 172.21.15.82:0/660035768 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7febf410a130 msgr2=0x7febf4070f00 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-20T16:22:53.489 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:53.487+0000 7febfa347640 1 --2- 172.21.15.82:0/660035768 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7febf410a130 0x7febf4070f00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:22:53.489 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:53.487+0000 7febfa347640 1 -- 172.21.15.82:0/660035768 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7febf41092d0 msgr2=0x7febf40709c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:22:53.489 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:53.487+0000 7febfa347640 1 --2- 172.21.15.82:0/660035768 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7febf41092d0 0x7febf40709c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:22:53.490 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:53.487+0000 7febfa347640 1 -- 172.21.15.82:0/660035768 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febf40724d0 con 0x7febf410ee30 2024-05-20T16:22:53.497 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:53.497+0000 7febfa347640 1 --2- 172.21.15.82:0/660035768 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7febf410ee30 0x7febf4071e60 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7febec0317f0 tx=0x7febec031820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:22:53.505 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:53.505+0000 7febeaffd640 1 -- 172.21.15.82:0/660035768 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febec02fa50 con 0x7febf410ee30 2024-05-20T16:22:53.505 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:53.506+0000 7febeaffd640 1 -- 172.21.15.82:0/660035768 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7febec02d070 con 0x7febf410ee30 2024-05-20T16:22:53.505 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:53.506+0000 7febfab48640 1 -- 172.21.15.82:0/660035768 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7febf4072760 con 0x7febf410ee30 2024-05-20T16:22:53.505 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:53.506+0000 7febeaffd640 1 -- 172.21.15.82:0/660035768 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febec0466e0 con 0x7febf410ee30 2024-05-20T16:22:53.506 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:53.506+0000 7febfab48640 1 -- 172.21.15.82:0/660035768 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7febf4072cb0 con 0x7febf410ee30 2024-05-20T16:22:53.513 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:53.513+0000 7fec02c53740 1 Processor -- start 2024-05-20T16:22:53.515 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:53.514+0000 7fec02c53740 1 -- start start 2024-05-20T16:22:53.515 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:53.514+0000 7fec02c53740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5560798f31b0 0x556079c93050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:22:53.515 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:53.514+0000 7fec02c53740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5560798ca2f0 0x556079c935d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:22:53.515 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:53.514+0000 7fec02c53740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5560798d24d0 0x556079c9a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:22:53.515 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:53.514+0000 7fec02c53740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x556079bf7340 con 0x5560798ca2f0 2024-05-20T16:22:53.515 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:53.515+0000 7fec02c53740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x556079b9f290 con 0x5560798f31b0 2024-05-20T16:22:53.515 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:53.515+0000 7fec02c53740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x556079bf3300 con 0x5560798d24d0 2024-05-20T16:22:53.515 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:53.515+0000 7febeaffd640 1 -- 172.21.15.82:0/660035768 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7febec02fbf0 con 0x7febf410ee30 2024-05-20T16:22:53.516 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:53.516+0000 7febf9345640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5560798d24d0 0x556079c9a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:22:53.516 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:53.516+0000 7febf9345640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5560798d24d0 0x556079c9a4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.82:50300/0 (socket says 172.21.15.82:50300) 2024-05-20T16:22:53.516 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:53.516+0000 7febf9345640 1 -- 172.21.15.82:0/2636464531 learned_addr learned my addr 172.21.15.82:0/2636464531 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:22:53.516 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:53.516+0000 7febf9345640 1 --2- 172.21.15.82:0/2636464531 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5560798f31b0 0x556079c93050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:22:53.517 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:53.516+0000 7febf9345640 1 -- 172.21.15.82:0/2636464531 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5560798f31b0 msgr2=0x556079c93050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:22:53.517 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:53.516+0000 7febf9345640 1 --2- 172.21.15.82:0/2636464531 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5560798f31b0 0x556079c93050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:22:53.517 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:53.516+0000 7febf9345640 1 -- 172.21.15.82:0/2636464531 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5560798ca2f0 msgr2=0x556079c935d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:22:53.517 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:53.516+0000 7febf9345640 1 --2- 172.21.15.82:0/2636464531 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5560798ca2f0 0x556079c935d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:22:53.517 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:53.516+0000 7febf9345640 1 -- 172.21.15.82:0/2636464531 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556079bc38e0 con 0x5560798d24d0 2024-05-20T16:22:53.517 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:53.517+0000 7febf9345640 1 --2- 172.21.15.82:0/2636464531 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5560798d24d0 0x556079c9a4d0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7febe4013720 tx=0x7febe4009660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4fdee57f25dfac3c server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:22:53.517 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:53.517+0000 7febdffff640 1 -- 172.21.15.82:0/2636464531 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febe4005020 con 0x5560798d24d0 2024-05-20T16:22:53.518 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:53.518+0000 7febdffff640 1 -- 172.21.15.82:0/2636464531 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7febe4009b30 con 0x5560798d24d0 2024-05-20T16:22:53.519 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:53.519+0000 7fec02c53740 1 -- 172.21.15.82:0/2636464531 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5560798d24d0 msgr2=0x556079c9a4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:22:53.519 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:53.519+0000 7fec02c53740 1 --2- 172.21.15.82:0/2636464531 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5560798d24d0 0x556079c9a4d0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7febe4013720 tx=0x7febe4009660 comp rx=0 tx=0).stop 2024-05-20T16:22:53.519 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:53.520+0000 7fec02c53740 1 -- 172.21.15.82:0/2636464531 shutdown_connections 2024-05-20T16:22:53.519 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:53.520+0000 7fec02c53740 1 --2- 172.21.15.82:0/2636464531 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5560798ca2f0 0x556079c935d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:22:53.520 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:53.520+0000 7fec02c53740 1 --2- 172.21.15.82:0/2636464531 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5560798d24d0 0x556079c9a4d0 secure :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7febe4013720 tx=0x7febe4009660 comp rx=0 tx=0).stop 2024-05-20T16:22:53.520 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:53.520+0000 7fec02c53740 1 --2- 172.21.15.82:0/2636464531 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5560798f31b0 0x556079c93050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:22:53.520 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:53.520+0000 7fec02c53740 1 -- 172.21.15.82:0/2636464531 >> 172.21.15.82:0/2636464531 conn(0x5560798e7300 msgr2=0x556079c8e9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:22:53.524 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:53.525+0000 7febeaffd640 1 --2- 172.21.15.82:0/660035768 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7febe00a2d90 0x7febe00a52a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:22:53.525 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:53.525+0000 7febeaffd640 1 -- 172.21.15.82:0/660035768 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9150+0+0 (secure 0 0 0) 0x7febec0f09e0 con 0x7febf410ee30 2024-05-20T16:22:53.526 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:53.526+0000 7fec02c53740 1 -- 172.21.15.82:0/2636464531 shutdown_connections 2024-05-20T16:22:53.530 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:53.528+0000 7fec02c53740 1 -- 172.21.15.82:0/2636464531 wait complete. 2024-05-20T16:22:53.530 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:53.529+0000 7fec02c53740 1 -- start start 2024-05-20T16:22:53.530 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:53.529+0000 7fec02c53740 1 Processor -- start 2024-05-20T16:22:53.531 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:53.530+0000 7fec02c53740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5560798ca2f0 0x556079c9cba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:22:53.531 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:53.530+0000 7fec02c53740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5560798f31b0 0x556079c9d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:22:53.531 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:53.530+0000 7fec02c53740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5560798b1d80 0x556079ca4040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:22:53.531 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:53.530+0000 7fec02c53740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x556079c9afc0 con 0x5560798b1d80 2024-05-20T16:22:53.531 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:53.530+0000 7fec02c53740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x556079bb2790 con 0x5560798f31b0 2024-05-20T16:22:53.531 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:53.530+0000 7fec02c53740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x556079c9b140 con 0x5560798ca2f0 2024-05-20T16:22:53.532 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:53.530+0000 7fec02c53740 10 client.0 _ll_register_callbacks cb 0x556079beda20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T16:22:53.532 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:53.530+0000 7febf9345640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5560798f31b0 0x556079c9d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:22:53.532 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:53.530+0000 7febf9345640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5560798f31b0 0x556079c9d0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.82:43524/0 (socket says 172.21.15.82:43524) 2024-05-20T16:22:53.532 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:53.530+0000 7febf9345640 1 -- 172.21.15.82:0/3703403887 learned_addr learned my addr 172.21.15.82:0/3703403887 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:22:53.532 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:53.530+0000 7febfa347640 1 --2- 172.21.15.82:0/3703403887 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5560798b1d80 0x556079ca4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:22:53.533 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:53.530+0000 7febf9345640 1 -- 172.21.15.82:0/3703403887 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5560798ca2f0 msgr2=0x556079c9cba0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-20T16:22:53.533 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:53.530+0000 7febf9345640 1 --2- 172.21.15.82:0/3703403887 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5560798ca2f0 0x556079c9cba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:22:53.533 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:53.530+0000 7febf9345640 1 -- 172.21.15.82:0/3703403887 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5560798b1d80 msgr2=0x556079ca4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:22:53.533 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:53.530+0000 7febf9345640 1 --2- 172.21.15.82:0/3703403887 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5560798b1d80 0x556079ca4040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:22:53.533 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:53.530+0000 7febf9345640 1 -- 172.21.15.82:0/3703403887 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55607990b210 con 0x5560798f31b0 2024-05-20T16:22:53.534 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:53.531+0000 7febf9345640 1 --2- 172.21.15.82:0/3703403887 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5560798f31b0 0x556079c9d0e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7febe4009900 tx=0x7febe4009b10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4162ff308f37b16 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:22:53.535 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:53.536+0000 7febf9b46640 1 --2- 172.21.15.82:0/660035768 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7febe00a2d90 0x7febe00a52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:22:53.539 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:53.537+0000 7febdd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-20T16:22:53.539 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:53.537+0000 7febdd7fa640 1 -- 172.21.15.82:0/3703403887 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febe40050d0 con 0x5560798f31b0 2024-05-20T16:22:53.539 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:53.538+0000 7fec02c53740 1 -- 172.21.15.82:0/3703403887 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x556079c04a70 con 0x5560798f31b0 2024-05-20T16:22:53.539 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:53.538+0000 7febdd7fa640 1 -- 172.21.15.82:0/3703403887 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9150+0+0 (secure 0 0 0) 0x7febe4005270 con 0x5560798f31b0 2024-05-20T16:22:53.539 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:53.538+0000 7febdd7fa640 1 -- 172.21.15.82:0/3703403887 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7febe402f050 con 0x5560798f31b0 2024-05-20T16:22:53.540 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:53.538+0000 7fec02c53740 10 client.25382 fetch_fsmap learned FSMap version 27 2024-05-20T16:22:53.540 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:53.538+0000 7fec02c53740 1 -- 172.21.15.82:0/3703403887 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5560799d7620 con 0x5560798f31b0 2024-05-20T16:22:53.540 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:53.539+0000 7febdd7fa640 1 -- 172.21.15.82:0/3703403887 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x556079c04a70 con 0x5560798f31b0 2024-05-20T16:22:53.540 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:53.539+0000 7febdd7fa640 1 -- 172.21.15.82:0/3703403887 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7febe400f5a0 con 0x5560798f31b0 2024-05-20T16:22:53.540 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:53.539+0000 7fec02c53740 10 client.25382 fetch_fsmap finished waiting for FSMap version 27 2024-05-20T16:22:53.540 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:53.539+0000 7fec02c53740 10 client.25382 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T16:22:53.540 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:53.539+0000 7fec02c53740 20 client.25382 populate_metadata read hostname 'smithi082' 2024-05-20T16:22:53.541 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:53.539+0000 7fec02c53740 1 --2- 172.21.15.82:0/3703403887 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x5560798871c0 0x556079caa870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:22:53.541 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:53.539+0000 7fec02c53740 4 client.25382 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T16:22:53.541 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:53.539+0000 7fec02c53740 1 -- 172.21.15.82:0/3703403887 --> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556079bfedc0 con 0x5560798871c0 2024-05-20T16:22:53.542 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:53.540+0000 7febf9b46640 1 --2- 172.21.15.82:0/3703403887 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x5560798871c0 0x556079caa870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:22:53.549 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:53.549+0000 7febf9b46640 1 --2- 172.21.15.82:0/660035768 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7febe00a2d90 0x7febe00a52a0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x556079c8f810 tx=0x7febf000a9f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:22:53.568 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:53.563+0000 7febf9b46640 1 --2- 172.21.15.82:0/3703403887 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x5560798871c0 0x556079caa870 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc5ea6d2e2bd83c server_cookie=db46c9829b97ba48 in_seq=0 out_seq=0 2024-05-20T16:22:53.569 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:53.563+0000 7febdd7fa640 10 client.25382 ms_handle_connect on v2:172.21.15.160:6838/2765554482 2024-05-20T16:22:53.569 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:53.566+0000 7febdd7fa640 1 -- 172.21.15.82:0/3703403887 <== mds.0 v2:172.21.15.160:6838/2765554482 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x556079bfedc0 con 0x5560798871c0 2024-05-20T16:22:53.569 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:53.566+0000 7febdd7fa640 10 client.25382 handle_command_reply: tid=0 2024-05-20T16:22:53.569 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:53.566+0000 7fec02c53740 1 client.25382 shutdown 2024-05-20T16:22:53.570 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:53.567+0000 7fec02c53740 1 -- 172.21.15.82:0/3703403887 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5560798f31b0 msgr2=0x556079c9d0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:22:53.570 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:53.567+0000 7fec02c53740 1 --2- 172.21.15.82:0/3703403887 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5560798f31b0 0x556079c9d0e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7febe4009900 tx=0x7febe4009b10 comp rx=0 tx=0).stop 2024-05-20T16:22:53.570 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:53.568+0000 7fec02c53740 1 -- 172.21.15.82:0/3703403887 shutdown_connections 2024-05-20T16:22:53.570 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:53.568+0000 7fec02c53740 1 --2- 172.21.15.82:0/3703403887 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x5560798871c0 0x556079caa870 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:22:53.570 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:53.568+0000 7fec02c53740 1 --2- 172.21.15.82:0/3703403887 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5560798b1d80 0x556079ca4040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:22:53.570 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:53.568+0000 7fec02c53740 1 --2- 172.21.15.82:0/3703403887 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5560798f31b0 0x556079c9d0e0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:22:53.571 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:53.568+0000 7fec02c53740 1 --2- 172.21.15.82:0/3703403887 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5560798ca2f0 0x556079c9cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:22:53.571 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:53.568+0000 7fec02c53740 1 -- 172.21.15.82:0/3703403887 >> 172.21.15.82:0/3703403887 conn(0x5560798e7300 msgr2=0x556079c8dab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:22:53.572 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:53.570+0000 7febdd7fa640 0 client.25382 ms_handle_reset on v2:172.21.15.160:6838/2765554482 2024-05-20T16:22:53.572 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:53.570+0000 7fec02c53740 1 -- 172.21.15.82:0/3703403887 shutdown_connections 2024-05-20T16:22:53.572 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:53.570+0000 7fec02c53740 1 -- 172.21.15.82:0/3703403887 wait complete. 2024-05-20T16:22:53.573 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:53.570+0000 7fec02c53740 20 client.25382 trim_cache size 0 max 16384 2024-05-20T16:22:53.613 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:53.613+0000 7fec02c53740 1 Processor -- start 2024-05-20T16:22:53.616 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:53.616+0000 7fec02c53740 1 -- start start 2024-05-20T16:22:53.617 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:53.616+0000 7fec02c53740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5560798b1d80 0x556079c8e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:22:53.617 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:53.616+0000 7fec02c53740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5560798ca2f0 0x556079c8e980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:22:53.617 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:53.617+0000 7fec02c53740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5560798f31b0 0x556079c9ae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:22:53.617 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:53.617+0000 7fec02c53740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7febf41bd010 con 0x5560798b1d80 2024-05-20T16:22:53.617 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:53.617+0000 7fec02c53740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7febf41ba8e0 con 0x5560798f31b0 2024-05-20T16:22:53.618 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:53.617+0000 7fec02c53740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7febf41b8730 con 0x5560798ca2f0 2024-05-20T16:22:53.620 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:53.620+0000 7febfa347640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5560798f31b0 0x556079c9ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:22:53.620 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:53.620+0000 7febf9345640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5560798ca2f0 0x556079c8e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:22:53.620 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:53.620+0000 7febfa347640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5560798f31b0 0x556079c9ae00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.82:43528/0 (socket says 172.21.15.82:43528) 2024-05-20T16:22:53.620 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:53.620+0000 7febfa347640 1 -- 172.21.15.82:0/2550586862 learned_addr learned my addr 172.21.15.82:0/2550586862 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:22:53.624 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:53.622+0000 7febfa347640 1 -- 172.21.15.82:0/2550586862 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5560798ca2f0 msgr2=0x556079c8e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:22:53.624 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:53.622+0000 7febfa347640 1 --2- 172.21.15.82:0/2550586862 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5560798ca2f0 0x556079c8e980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:22:53.624 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:53.622+0000 7febfa347640 1 -- 172.21.15.82:0/2550586862 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5560798b1d80 msgr2=0x556079c8e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:22:53.624 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:53.622+0000 7febfa347640 1 --2- 172.21.15.82:0/2550586862 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5560798b1d80 0x556079c8e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:22:53.624 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:53.622+0000 7febfa347640 1 -- 172.21.15.82:0/2550586862 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556079c92c00 con 0x5560798f31b0 2024-05-20T16:22:53.633 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:53.631+0000 7febfa347640 1 --2- 172.21.15.82:0/2550586862 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5560798f31b0 0x556079c9ae00 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7febec0b9220 tx=0x7febec0dd040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=12f481fa1930d55c server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:22:53.642 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:53.638+0000 7febdf7fe640 1 -- 172.21.15.82:0/2550586862 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febec0098a0 con 0x5560798f31b0 2024-05-20T16:22:53.642 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:53.638+0000 7febdf7fe640 1 -- 172.21.15.82:0/2550586862 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7febec0bf030 con 0x5560798f31b0 2024-05-20T16:22:53.643 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:53.639+0000 7fec02c53740 1 -- 172.21.15.82:0/2550586862 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5560798f31b0 msgr2=0x556079c9ae00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:22:53.643 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:53.639+0000 7fec02c53740 1 --2- 172.21.15.82:0/2550586862 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5560798f31b0 0x556079c9ae00 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7febec0b9220 tx=0x7febec0dd040 comp rx=0 tx=0).stop 2024-05-20T16:22:53.643 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:53.639+0000 7fec02c53740 1 -- 172.21.15.82:0/2550586862 shutdown_connections 2024-05-20T16:22:53.643 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:53.639+0000 7fec02c53740 1 --2- 172.21.15.82:0/2550586862 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5560798f31b0 0x556079c9ae00 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:22:53.643 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:53.639+0000 7fec02c53740 1 --2- 172.21.15.82:0/2550586862 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5560798ca2f0 0x556079c8e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:22:53.643 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:53.639+0000 7fec02c53740 1 --2- 172.21.15.82:0/2550586862 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5560798b1d80 0x556079c8e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:22:53.643 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:53.639+0000 7fec02c53740 1 -- 172.21.15.82:0/2550586862 >> 172.21.15.82:0/2550586862 conn(0x5560798e7300 msgr2=0x556079c96eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:22:53.643 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:53.641+0000 7fec02c53740 1 -- 172.21.15.82:0/2550586862 shutdown_connections 2024-05-20T16:22:53.644 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:53.644+0000 7fec02c53740 1 -- 172.21.15.82:0/2550586862 wait complete. 2024-05-20T16:22:53.644 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:53.644+0000 7fec02c53740 1 -- start start 2024-05-20T16:22:53.645 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:53.644+0000 7fec02c53740 1 Processor -- start 2024-05-20T16:22:53.646 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:53.646+0000 7fec02c53740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5560798b1d80 0x556079ca05f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:22:53.646 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:53.647+0000 7fec02c53740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5560798ca2f0 0x556079ca0b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:22:53.646 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:53.647+0000 7fec02c53740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5560798f31b0 0x556079d10c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:22:53.646 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:53.647+0000 7fec02c53740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7febf41b8da0 con 0x5560798f31b0 2024-05-20T16:22:53.647 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:53.647+0000 7fec02c53740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7febf41ba210 con 0x5560798b1d80 2024-05-20T16:22:53.647 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:53.647+0000 7fec02c53740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7febf41b9470 con 0x5560798ca2f0 2024-05-20T16:22:53.647 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:53.647+0000 7fec02c53740 10 client.0 _ll_register_callbacks cb 0x7febe4009d00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T16:22:53.647 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:53.647+0000 7febf9345640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5560798ca2f0 0x556079ca0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:22:53.647 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:53.647+0000 7febf9345640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5560798ca2f0 0x556079ca0b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.82:50332/0 (socket says 172.21.15.82:50332) 2024-05-20T16:22:53.647 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:53.647+0000 7febf9345640 1 -- 172.21.15.82:0/2392934204 learned_addr learned my addr 172.21.15.82:0/2392934204 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:22:53.647 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:53.647+0000 7febf9b46640 1 --2- 172.21.15.82:0/2392934204 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5560798b1d80 0x556079ca05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:22:53.647 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:53.647+0000 7febfa347640 1 --2- 172.21.15.82:0/2392934204 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5560798f31b0 0x556079d10c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:22:53.648 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:53.648+0000 7febfa347640 1 -- 172.21.15.82:0/2392934204 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5560798ca2f0 msgr2=0x556079ca0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:22:53.648 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:53.648+0000 7febfa347640 1 --2- 172.21.15.82:0/2392934204 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5560798ca2f0 0x556079ca0b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:22:53.648 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:53.648+0000 7febfa347640 1 -- 172.21.15.82:0/2392934204 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5560798b1d80 msgr2=0x556079ca05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:22:53.648 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:53.648+0000 7febfa347640 1 --2- 172.21.15.82:0/2392934204 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5560798b1d80 0x556079ca05f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:22:53.648 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:53.648+0000 7febfa347640 1 -- 172.21.15.82:0/2392934204 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556079d11150 con 0x5560798f31b0 2024-05-20T16:22:53.649 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:53.649+0000 7febfa347640 1 --2- 172.21.15.82:0/2392934204 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5560798f31b0 0x556079d10c10 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7febec0b9220 tx=0x7febec02a8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbba5cf921755a60 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:22:53.649 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:53.649+0000 7febdd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-05-20T16:22:53.651 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:53.650+0000 7febdd7fa640 1 -- 172.21.15.82:0/2392934204 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febec0cb070 con 0x5560798f31b0 2024-05-20T16:22:53.651 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:53.650+0000 7febdd7fa640 1 -- 172.21.15.82:0/2392934204 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9150+0+0 (secure 0 0 0) 0x7febec0b9430 con 0x5560798f31b0 2024-05-20T16:22:53.654 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:53.653+0000 7fec02c53740 1 -- 172.21.15.82:0/2392934204 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x556079b7c8e0 con 0x5560798f31b0 2024-05-20T16:22:53.657 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:53.657+0000 7febdd7fa640 1 -- 172.21.15.82:0/2392934204 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7febec0d1080 con 0x5560798f31b0 2024-05-20T16:22:53.661 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:53.662+0000 7fec02c53740 10 client.15879 fetch_fsmap learned FSMap version 27 2024-05-20T16:22:53.662 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:53.662+0000 7fec02c53740 1 -- 172.21.15.82:0/2392934204 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x556079d12360 con 0x5560798f31b0 2024-05-20T16:22:53.678 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:53.678+0000 7febdd7fa640 1 -- 172.21.15.82:0/2392934204 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x556079b7c8e0 con 0x5560798f31b0 2024-05-20T16:22:53.678 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:53.678+0000 7febdd7fa640 1 -- 172.21.15.82:0/2392934204 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7febec04e020 con 0x5560798f31b0 2024-05-20T16:22:53.679 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:53.678+0000 7fec02c53740 10 client.15879 fetch_fsmap finished waiting for FSMap version 27 2024-05-20T16:22:53.679 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:53.678+0000 7fec02c53740 10 client.15879 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T16:22:53.679 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:53.678+0000 7fec02c53740 20 client.15879 populate_metadata read hostname 'smithi082' 2024-05-20T16:22:53.679 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:53.678+0000 7fec02c53740 1 --2- 172.21.15.82:0/2392934204 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x556079d12ba0 0x556079d14fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:22:53.679 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:53.678+0000 7fec02c53740 4 client.15879 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-20T16:22:53.679 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:53.678+0000 7fec02c53740 1 -- 172.21.15.82:0/2392934204 --> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556079bc8870 con 0x556079d12ba0 2024-05-20T16:22:53.679 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:53.679+0000 7febf9b46640 1 --2- 172.21.15.82:0/2392934204 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x556079d12ba0 0x556079d14fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:22:53.680 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:53.680+0000 7febf9b46640 1 --2- 172.21.15.82:0/2392934204 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x556079d12ba0 0x556079d14fc0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=30508948ad887d50 server_cookie=244ff2bd68538a2d in_seq=0 out_seq=0 2024-05-20T16:22:53.680 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:53.680+0000 7febdd7fa640 10 client.15879 ms_handle_connect on v2:172.21.15.160:6838/2765554482 2024-05-20T16:22:54.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:22:53 smithi082 ceph-mon[32238]: pgmap v630: 129 pgs: 129 active+clean; 10 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 728 MiB/s wr, 1.31k op/s 2024-05-20T16:22:54.135 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:22:53 smithi184 ceph-mon[36209]: pgmap v630: 129 pgs: 129 active+clean; 10 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 728 MiB/s wr, 1.31k op/s 2024-05-20T16:22:54.312 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:22:53 smithi160 ceph-mon[36347]: pgmap v630: 129 pgs: 129 active+clean; 10 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 728 MiB/s wr, 1.31k op/s 2024-05-20T16:22:56.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:22:55 smithi082 ceph-mon[32238]: pgmap v631: 129 pgs: 129 active+clean; 10 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 761 MiB/s wr, 1.24k op/s 2024-05-20T16:22:56.136 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:22:55 smithi184 ceph-mon[36209]: pgmap v631: 129 pgs: 129 active+clean; 10 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 761 MiB/s wr, 1.24k op/s 2024-05-20T16:22:56.312 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:22:55 smithi160 ceph-mon[36347]: pgmap v631: 129 pgs: 129 active+clean; 10 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 761 MiB/s wr, 1.24k op/s 2024-05-20T16:22:57.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:22:56 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T16:22:57.135 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:22:56 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T16:22:57.285 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:57.283+0000 7febdd7fa640 1 -- 172.21.15.82:0/2392934204 <== mds.0 v2:172.21.15.160:6838/2765554482 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556079bc8870 con 0x556079d12ba0 2024-05-20T16:22:57.285 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:57.283+0000 7febdd7fa640 10 client.15879 handle_command_reply: tid=0 2024-05-20T16:22:57.285 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:57.285+0000 7fec02c53740 1 client.15879 shutdown 2024-05-20T16:22:57.285 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:57.285+0000 7fec02c53740 1 -- 172.21.15.82:0/2392934204 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5560798f31b0 msgr2=0x556079d10c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:22:57.285 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:57.285+0000 7fec02c53740 1 --2- 172.21.15.82:0/2392934204 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5560798f31b0 0x556079d10c10 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7febec0b9220 tx=0x7febec02a8f0 comp rx=0 tx=0).stop 2024-05-20T16:22:57.285 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:57.285+0000 7fec02c53740 1 -- 172.21.15.82:0/2392934204 shutdown_connections 2024-05-20T16:22:57.285 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:57.285+0000 7fec02c53740 1 --2- 172.21.15.82:0/2392934204 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x556079d12ba0 0x556079d14fc0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:22:57.285 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:57.285+0000 7fec02c53740 1 --2- 172.21.15.82:0/2392934204 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5560798f31b0 0x556079d10c10 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:22:57.286 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:57.285+0000 7fec02c53740 1 --2- 172.21.15.82:0/2392934204 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5560798ca2f0 0x556079ca0b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:22:57.286 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:57.285+0000 7fec02c53740 1 --2- 172.21.15.82:0/2392934204 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5560798b1d80 0x556079ca05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:22:57.286 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:57.285+0000 7fec02c53740 1 -- 172.21.15.82:0/2392934204 >> 172.21.15.82:0/2392934204 conn(0x5560798e7300 msgr2=0x556079c8e440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:22:57.286 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:57.285+0000 7febdd7fa640 0 client.15879 ms_handle_reset on v2:172.21.15.160:6838/2765554482 2024-05-20T16:22:57.286 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:57.286+0000 7fec02c53740 1 -- 172.21.15.82:0/2392934204 shutdown_connections 2024-05-20T16:22:57.286 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:57.286+0000 7fec02c53740 1 -- 172.21.15.82:0/2392934204 wait complete. 2024-05-20T16:22:57.287 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:57.287+0000 7fec02c53740 20 client.15879 trim_cache size 0 max 16384 2024-05-20T16:22:57.288 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:{ 2024-05-20T16:22:57.288 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "message": "", 2024-05-20T16:22:57.289 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "return_code": 0 2024-05-20T16:22:57.289 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:} 2024-05-20T16:22:57.293 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:57.293+0000 7febfab48640 1 -- 172.21.15.82:0/660035768 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7febe00a2d90 msgr2=0x7febe00a52a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:22:57.294 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:57.293+0000 7febfab48640 1 --2- 172.21.15.82:0/660035768 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7febe00a2d90 0x7febe00a52a0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x556079c8f810 tx=0x7febf000a9f0 comp rx=0 tx=0).stop 2024-05-20T16:22:57.294 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:57.293+0000 7febfab48640 1 -- 172.21.15.82:0/660035768 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7febf410ee30 msgr2=0x7febf4071e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:22:57.294 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:57.293+0000 7febfab48640 1 --2- 172.21.15.82:0/660035768 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7febf410ee30 0x7febf4071e60 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7febec0317f0 tx=0x7febec031820 comp rx=0 tx=0).stop 2024-05-20T16:22:57.294 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:57.293+0000 7febfab48640 1 -- 172.21.15.82:0/660035768 shutdown_connections 2024-05-20T16:22:57.294 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:57.293+0000 7febfab48640 1 --2- 172.21.15.82:0/660035768 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7febe00a2d90 0x7febe00a52a0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:22:57.294 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:57.293+0000 7febfab48640 1 --2- 172.21.15.82:0/660035768 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7febf410ee30 0x7febf4071e60 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:22:57.294 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:57.293+0000 7febfab48640 1 --2- 172.21.15.82:0/660035768 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7febf410a130 0x7febf4070f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:22:57.294 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:57.293+0000 7febfab48640 1 --2- 172.21.15.82:0/660035768 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7febf41092d0 0x7febf40709c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:22:57.294 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:57.294+0000 7febfab48640 1 -- 172.21.15.82:0/660035768 >> 172.21.15.82:0/660035768 conn(0x7febf4077b60 msgr2=0x7febf410ae10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:22:57.294 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:57.294+0000 7febfab48640 1 -- 172.21.15.82:0/660035768 shutdown_connections 2024-05-20T16:22:57.294 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:22:57.294+0000 7febfab48640 1 -- 172.21.15.82:0/660035768 wait complete. 2024-05-20T16:22:57.312 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:22:56 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T16:22:58.136 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:22:57 smithi184 ceph-mon[36209]: pgmap v632: 129 pgs: 129 active+clean; 10 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 695 MiB/s wr, 1.06k op/s 2024-05-20T16:22:58.136 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:22:57 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:22:58.136 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:22:57 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T16:22:58.136 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:22:57 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:22:58.312 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:22:57 smithi160 ceph-mon[36347]: pgmap v632: 129 pgs: 129 active+clean; 10 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 695 MiB/s wr, 1.06k op/s 2024-05-20T16:22:58.312 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:22:57 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:22:58.312 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:22:57 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T16:22:58.312 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:22:57 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:22:58.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:22:57 smithi082 ceph-mon[32238]: pgmap v632: 129 pgs: 129 active+clean; 10 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 695 MiB/s wr, 1.06k op/s 2024-05-20T16:22:58.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:22:57 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:22:58.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:22:57 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T16:22:58.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:22:57 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:23:00.135 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:22:59 smithi184 ceph-mon[36209]: pgmap v633: 129 pgs: 129 active+clean; 10 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 802 MiB/s wr, 1.18k op/s 2024-05-20T16:23:00.312 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:22:59 smithi160 ceph-mon[36347]: pgmap v633: 129 pgs: 129 active+clean; 10 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 802 MiB/s wr, 1.18k op/s 2024-05-20T16:23:00.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:22:59 smithi082 ceph-mon[32238]: pgmap v633: 129 pgs: 129 active+clean; 10 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 802 MiB/s wr, 1.18k op/s 2024-05-20T16:23:03.062 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:23:02 smithi160 ceph-mon[36347]: pgmap v634: 129 pgs: 129 active+clean; 10 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 644 MiB/s wr, 887 op/s 2024-05-20T16:23:03.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:23:02 smithi082 ceph-mon[32238]: pgmap v634: 129 pgs: 129 active+clean; 10 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 644 MiB/s wr, 887 op/s 2024-05-20T16:23:03.135 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:23:02 smithi184 ceph-mon[36209]: pgmap v634: 129 pgs: 129 active+clean; 10 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 644 MiB/s wr, 887 op/s 2024-05-20T16:23:04.062 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:23:03 smithi160 ceph-mon[36347]: pgmap v635: 129 pgs: 129 active+clean; 10 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 754 MiB/s wr, 1.13k op/s 2024-05-20T16:23:04.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:23:03 smithi082 ceph-mon[32238]: pgmap v635: 129 pgs: 129 active+clean; 10 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 754 MiB/s wr, 1.13k op/s 2024-05-20T16:23:04.135 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:23:03 smithi184 ceph-mon[36209]: pgmap v635: 129 pgs: 129 active+clean; 10 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 754 MiB/s wr, 1.13k op/s 2024-05-20T16:23:07.062 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:23:06 smithi160 ceph-mon[36347]: pgmap v636: 129 pgs: 129 active+clean; 10 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 694 MiB/s wr, 1.05k op/s 2024-05-20T16:23:07.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:23:06 smithi082 ceph-mon[32238]: pgmap v636: 129 pgs: 129 active+clean; 10 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 694 MiB/s wr, 1.05k op/s 2024-05-20T16:23:07.135 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:23:06 smithi184 ceph-mon[36209]: pgmap v636: 129 pgs: 129 active+clean; 10 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 694 MiB/s wr, 1.05k op/s 2024-05-20T16:23:08.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:23:07 smithi082 ceph-mon[32238]: pgmap v637: 129 pgs: 129 active+clean; 10 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 641 MiB/s wr, 991 op/s 2024-05-20T16:23:08.135 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:23:07 smithi184 ceph-mon[36209]: pgmap v637: 129 pgs: 129 active+clean; 10 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 641 MiB/s wr, 991 op/s 2024-05-20T16:23:08.313 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:23:07 smithi160 ceph-mon[36347]: pgmap v637: 129 pgs: 129 active+clean; 10 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 641 MiB/s wr, 991 op/s 2024-05-20T16:23:10.135 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:23:09 smithi184 ceph-mon[36209]: pgmap v638: 129 pgs: 129 active+clean; 10 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 796 MiB/s wr, 1.12k op/s 2024-05-20T16:23:10.312 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:23:09 smithi160 ceph-mon[36347]: pgmap v638: 129 pgs: 129 active+clean; 10 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 796 MiB/s wr, 1.12k op/s 2024-05-20T16:23:10.362 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:23:09 smithi082 ceph-mon[32238]: pgmap v638: 129 pgs: 129 active+clean; 10 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 796 MiB/s wr, 1.12k op/s 2024-05-20T16:23:10.560 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:10.560+0000 7f8e3df03640 1 -- 172.21.15.82:0/1315134493 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8e3810ee60 msgr2=0x7f8e3810f270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:23:10.560 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:10.560+0000 7f8e3df03640 1 --2- 172.21.15.82:0/1315134493 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8e3810ee60 0x7f8e3810f270 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f8e30009be0 tx=0x7f8e3002a710 comp rx=0 tx=0).stop 2024-05-20T16:23:10.561 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:10.562+0000 7f8e3df03640 1 -- 172.21.15.82:0/1315134493 shutdown_connections 2024-05-20T16:23:10.561 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:10.562+0000 7f8e3df03640 1 --2- 172.21.15.82:0/1315134493 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8e3810a140 0x7f8e3810a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:23:10.561 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:10.562+0000 7f8e3df03640 1 --2- 172.21.15.82:0/1315134493 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8e3810ee60 0x7f8e3810f270 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:23:10.562 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:10.562+0000 7f8e3df03640 1 --2- 172.21.15.82:0/1315134493 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8e381092f0 0x7f8e38109c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:23:10.562 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:10.562+0000 7f8e3df03640 1 -- 172.21.15.82:0/1315134493 >> 172.21.15.82:0/1315134493 conn(0x7f8e3806f2e0 msgr2=0x7f8e38071790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:23:10.570 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:10.571+0000 7f8e3df03640 1 -- 172.21.15.82:0/1315134493 shutdown_connections 2024-05-20T16:23:10.572 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:10.572+0000 7f8e3df03640 1 -- 172.21.15.82:0/1315134493 wait complete. 2024-05-20T16:23:10.573 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:10.573+0000 7f8e3df03640 1 Processor -- start 2024-05-20T16:23:10.573 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:10.574+0000 7f8e3df03640 1 -- start start 2024-05-20T16:23:10.575 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:10.575+0000 7f8e3df03640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8e381092f0 0x7f8e381a7e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:23:10.575 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:10.575+0000 7f8e3df03640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8e3810a140 0x7f8e381a8360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:23:10.575 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:10.575+0000 7f8e3df03640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8e3810ee60 0x7f8e381af2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:23:10.575 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:10.575+0000 7f8e3df03640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f8e3811d390 con 0x7f8e381092f0 2024-05-20T16:23:10.575 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:10.575+0000 7f8e3df03640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f8e3811d210 con 0x7f8e3810a140 2024-05-20T16:23:10.575 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:10.575+0000 7f8e3df03640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f8e3811d510 con 0x7f8e3810ee60 2024-05-20T16:23:10.575 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:10.575+0000 7f8e3cf01640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8e381092f0 0x7f8e381a7e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:23:10.575 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:10.575+0000 7f8e3cf01640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8e381092f0 0x7f8e381a7e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:46918/0 (socket says 172.21.15.82:46918) 2024-05-20T16:23:10.576 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:10.575+0000 7f8e3cf01640 1 -- 172.21.15.82:0/1625674003 learned_addr learned my addr 172.21.15.82:0/1625674003 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:23:10.578 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:10.579+0000 7f8e3d702640 1 --2- 172.21.15.82:0/1625674003 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8e3810ee60 0x7f8e381af2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:23:10.586 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:10.582+0000 7f8e3cf01640 1 -- 172.21.15.82:0/1625674003 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8e3810ee60 msgr2=0x7f8e381af2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:23:10.586 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:10.583+0000 7f8e3cf01640 1 --2- 172.21.15.82:0/1625674003 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8e3810ee60 0x7f8e381af2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:23:10.586 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:10.583+0000 7f8e3cf01640 1 -- 172.21.15.82:0/1625674003 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8e3810a140 msgr2=0x7f8e381a8360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:23:10.586 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:10.583+0000 7f8e3cf01640 1 --2- 172.21.15.82:0/1625674003 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8e3810a140 0x7f8e381a8360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:23:10.586 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:10.583+0000 7f8e3cf01640 1 -- 172.21.15.82:0/1625674003 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e381af960 con 0x7f8e381092f0 2024-05-20T16:23:10.586 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:10.585+0000 7f8e3cf01640 1 --2- 172.21.15.82:0/1625674003 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8e381092f0 0x7f8e381a7e20 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f8e2400e9c0 tx=0x7f8e2400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:23:10.593 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:10.591+0000 7f8e35ffb640 1 -- 172.21.15.82:0/1625674003 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e2400cd80 con 0x7f8e381092f0 2024-05-20T16:23:10.593 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:10.591+0000 7f8e3df03640 1 -- 172.21.15.82:0/1625674003 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e381afc50 con 0x7f8e381092f0 2024-05-20T16:23:10.594 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:10.591+0000 7f8e3df03640 1 -- 172.21.15.82:0/1625674003 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e381b01a0 con 0x7f8e381092f0 2024-05-20T16:23:10.594 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:10.591+0000 7f8e35ffb640 1 -- 172.21.15.82:0/1625674003 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8e24005500 con 0x7f8e381092f0 2024-05-20T16:23:10.594 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:10.591+0000 7f8e35ffb640 1 -- 172.21.15.82:0/1625674003 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e24010690 con 0x7f8e381092f0 2024-05-20T16:23:10.596 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:10.596+0000 7f8e4600d740 1 Processor -- start 2024-05-20T16:23:10.596 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:10.596+0000 7f8e4600d740 1 -- start start 2024-05-20T16:23:10.597 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:10.596+0000 7f8e4600d740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ca271671b0 0x55ca27507060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:23:10.597 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:10.597+0000 7f8e4600d740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ca2713e2f0 0x55ca275075e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:23:10.597 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:10.597+0000 7f8e4600d740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ca271464d0 0x55ca2750e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:23:10.597 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:10.597+0000 7f8e4600d740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55ca2746b340 con 0x55ca2713e2f0 2024-05-20T16:23:10.597 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:10.597+0000 7f8e4600d740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x55ca27413290 con 0x55ca271671b0 2024-05-20T16:23:10.597 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:10.597+0000 7f8e4600d740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55ca27467300 con 0x55ca271464d0 2024-05-20T16:23:10.598 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:10.599+0000 7f8e37fff640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ca271464d0 0x55ca2750e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:23:10.599 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:10.599+0000 7f8e3cf01640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ca2713e2f0 0x55ca275075e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:23:10.599 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:10.599+0000 7f8e3cf01640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ca2713e2f0 0x55ca275075e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:46932/0 (socket says 172.21.15.82:46932) 2024-05-20T16:23:10.599 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:10.599+0000 7f8e35ffb640 1 -- 172.21.15.82:0/1625674003 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f8e24005020 con 0x7f8e381092f0 2024-05-20T16:23:10.599 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:10.599+0000 7f8e3cf01640 1 -- 172.21.15.82:0/1283859145 learned_addr learned my addr 172.21.15.82:0/1283859145 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:23:10.601 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:10.601+0000 7f8e35ffb640 1 --2- 172.21.15.82:0/1625674003 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f8e280a2d90 0x7f8e280a52a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:23:10.601 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:10.601+0000 7f8e3d702640 1 --2- 172.21.15.82:0/1283859145 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ca271671b0 0x55ca27507060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:23:10.602 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:10.601+0000 7f8e35ffb640 1 -- 172.21.15.82:0/1625674003 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9150+0+0 (secure 0 0 0) 0x7f8e240cf840 con 0x7f8e381092f0 2024-05-20T16:23:10.603 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:10.604+0000 7f8e3cf01640 1 -- 172.21.15.82:0/1283859145 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ca271464d0 msgr2=0x55ca2750e4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:23:10.604 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:10.604+0000 7f8e3cf01640 1 --2- 172.21.15.82:0/1283859145 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ca271464d0 0x55ca2750e4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:23:10.604 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:10.604+0000 7f8e3cf01640 1 -- 172.21.15.82:0/1283859145 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ca271671b0 msgr2=0x55ca27507060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:23:10.604 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:10.604+0000 7f8e3cf01640 1 --2- 172.21.15.82:0/1283859145 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ca271671b0 0x55ca27507060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:23:10.604 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:10.604+0000 7f8e3cf01640 1 -- 172.21.15.82:0/1283859145 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ca274378e0 con 0x55ca2713e2f0 2024-05-20T16:23:10.605 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:10.606+0000 7f8e3d702640 1 --2- 172.21.15.82:0/1625674003 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f8e280a2d90 0x7f8e280a52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:23:10.606 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:10.606+0000 7f8e3cf01640 1 --2- 172.21.15.82:0/1283859145 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ca2713e2f0 0x55ca275075e0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f8e24098680 tx=0x7f8e24004960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dae4ef2dec951e51 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:23:10.607 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:10.608+0000 7f8e22ffd640 1 -- 172.21.15.82:0/1283859145 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e2409b7f0 con 0x55ca2713e2f0 2024-05-20T16:23:10.607 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:10.608+0000 7f8e22ffd640 1 -- 172.21.15.82:0/1283859145 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8e2409c3b0 con 0x55ca2713e2f0 2024-05-20T16:23:10.609 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:10.609+0000 7f8e4600d740 1 -- 172.21.15.82:0/1283859145 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ca2713e2f0 msgr2=0x55ca275075e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:23:10.609 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:10.609+0000 7f8e4600d740 1 --2- 172.21.15.82:0/1283859145 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ca2713e2f0 0x55ca275075e0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f8e24098680 tx=0x7f8e24004960 comp rx=0 tx=0).stop 2024-05-20T16:23:10.617 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:10.618+0000 7f8e4600d740 1 -- 172.21.15.82:0/1283859145 shutdown_connections 2024-05-20T16:23:10.617 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:10.618+0000 7f8e4600d740 1 --2- 172.21.15.82:0/1283859145 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ca2713e2f0 0x55ca275075e0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:23:10.617 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:10.618+0000 7f8e4600d740 1 --2- 172.21.15.82:0/1283859145 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ca271464d0 0x55ca2750e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:23:10.617 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:10.618+0000 7f8e4600d740 1 --2- 172.21.15.82:0/1283859145 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ca271671b0 0x55ca27507060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:23:10.618 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:10.618+0000 7f8e4600d740 1 -- 172.21.15.82:0/1283859145 >> 172.21.15.82:0/1283859145 conn(0x55ca2715b300 msgr2=0x55ca275029f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:23:10.620 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:10.621+0000 7f8e4600d740 1 -- 172.21.15.82:0/1283859145 shutdown_connections 2024-05-20T16:23:10.625 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:10.624+0000 7f8e3d702640 1 --2- 172.21.15.82:0/1625674003 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f8e280a2d90 0x7f8e280a52a0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f8e30030ad0 tx=0x7f8e3002a710 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:23:10.626 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:10.626+0000 7f8e4600d740 1 -- 172.21.15.82:0/1283859145 wait complete. 2024-05-20T16:23:10.626 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:10.627+0000 7f8e4600d740 1 -- start start 2024-05-20T16:23:10.626 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:10.627+0000 7f8e4600d740 1 Processor -- start 2024-05-20T16:23:10.629 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:10.629+0000 7f8e4600d740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ca2713e2f0 0x55ca27510d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:23:10.630 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:10.629+0000 7f8e4600d740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ca271464d0 0x55ca27511270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:23:10.630 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:10.629+0000 7f8e4600d740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ca271671b0 0x55ca275181d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:23:10.630 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:10.629+0000 7f8e4600d740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55ca2750ef30 con 0x55ca2713e2f0 2024-05-20T16:23:10.630 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:10.629+0000 7f8e4600d740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x55ca27426790 con 0x55ca271464d0 2024-05-20T16:23:10.630 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:10.629+0000 7f8e4600d740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55ca2750f0b0 con 0x55ca271671b0 2024-05-20T16:23:10.630 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:10.629+0000 7f8e4600d740 10 client.0 _ll_register_callbacks cb 0x55ca27461a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T16:23:10.631 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:10.630+0000 7f8e3cf01640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ca2713e2f0 0x55ca27510d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:23:10.631 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:10.630+0000 7f8e3cf01640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ca2713e2f0 0x55ca27510d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:46940/0 (socket says 172.21.15.82:46940) 2024-05-20T16:23:10.631 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:10.630+0000 7f8e3cf01640 1 -- 172.21.15.82:0/2531086352 learned_addr learned my addr 172.21.15.82:0/2531086352 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:23:10.631 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:10.631+0000 7f8e3d702640 1 --2- 172.21.15.82:0/2531086352 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ca271671b0 0x55ca275181d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:23:10.631 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:10.631+0000 7f8e37fff640 1 --2- 172.21.15.82:0/2531086352 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ca271464d0 0x55ca27511270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:23:10.633 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:10.634+0000 7f8e3cf01640 1 -- 172.21.15.82:0/2531086352 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ca271671b0 msgr2=0x55ca275181d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:23:10.634 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:10.634+0000 7f8e3cf01640 1 --2- 172.21.15.82:0/2531086352 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ca271671b0 0x55ca275181d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:23:10.634 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:10.634+0000 7f8e3cf01640 1 -- 172.21.15.82:0/2531086352 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ca271464d0 msgr2=0x55ca27511270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:23:10.634 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:10.634+0000 7f8e3cf01640 1 --2- 172.21.15.82:0/2531086352 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ca271464d0 0x55ca27511270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:23:10.634 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:10.634+0000 7f8e3cf01640 1 -- 172.21.15.82:0/2531086352 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ca2717f210 con 0x55ca2713e2f0 2024-05-20T16:23:10.635 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:10.635+0000 7f8e3cf01640 1 --2- 172.21.15.82:0/2531086352 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ca2713e2f0 0x55ca27510d30 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f8e240a43f0 tx=0x7f8e2409bcb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55ac1b07d59646ca server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:23:10.635 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:10.636+0000 7f8e17fff640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-05-20T16:23:10.636 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:10.636+0000 7f8e17fff640 1 -- 172.21.15.82:0/2531086352 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e240a0060 con 0x55ca2713e2f0 2024-05-20T16:23:10.636 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:10.636+0000 7f8e17fff640 1 -- 172.21.15.82:0/2531086352 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9150+0+0 (secure 0 0 0) 0x7f8e240cd060 con 0x55ca2713e2f0 2024-05-20T16:23:10.644 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:10.645+0000 7f8e4600d740 1 -- 172.21.15.82:0/2531086352 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8e2409c3b0 con 0x55ca2713e2f0 2024-05-20T16:23:10.646 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:10.647+0000 7f8e17fff640 1 -- 172.21.15.82:0/2531086352 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f8e24099fb0 con 0x55ca2713e2f0 2024-05-20T16:23:10.658 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:10.659+0000 7f8e4600d740 10 client.15891 fetch_fsmap learned FSMap version 27 2024-05-20T16:23:10.658 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:10.659+0000 7f8e4600d740 1 -- 172.21.15.82:0/2531086352 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ca2724b620 con 0x55ca2713e2f0 2024-05-20T16:23:10.663 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:10.664+0000 7f8e17fff640 1 -- 172.21.15.82:0/2531086352 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8e2409c3b0 con 0x55ca2713e2f0 2024-05-20T16:23:10.663 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:10.664+0000 7f8e17fff640 1 -- 172.21.15.82:0/2531086352 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f8e2409b2a0 con 0x55ca2713e2f0 2024-05-20T16:23:10.664 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:10.664+0000 7f8e4600d740 10 client.15891 fetch_fsmap finished waiting for FSMap version 27 2024-05-20T16:23:10.664 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:10.664+0000 7f8e4600d740 10 client.15891 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T16:23:10.664 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:10.664+0000 7f8e4600d740 20 client.15891 populate_metadata read hostname 'smithi082' 2024-05-20T16:23:10.664 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:10.664+0000 7f8e4600d740 1 --2- 172.21.15.82:0/2531086352 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x55ca27125d80 0x55ca2752f850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:23:10.664 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:10.664+0000 7f8e4600d740 4 client.15891 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T16:23:10.664 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:10.664+0000 7f8e4600d740 1 -- 172.21.15.82:0/2531086352 --> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ca27478a70 con 0x55ca27125d80 2024-05-20T16:23:10.667 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:10.667+0000 7f8e37fff640 1 --2- 172.21.15.82:0/2531086352 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x55ca27125d80 0x55ca2752f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:23:10.677 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:10.678+0000 7f8e37fff640 1 --2- 172.21.15.82:0/2531086352 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x55ca27125d80 0x55ca2752f850 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3055d837e675dd11 server_cookie=a01e7c6de27eb195 in_seq=0 out_seq=0 2024-05-20T16:23:10.678 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:10.678+0000 7f8e17fff640 10 client.15891 ms_handle_connect on v2:172.21.15.160:6838/2765554482 2024-05-20T16:23:10.681 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:10.681+0000 7f8e17fff640 1 -- 172.21.15.82:0/2531086352 <== mds.0 v2:172.21.15.160:6838/2765554482 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55ca27478a70 con 0x55ca27125d80 2024-05-20T16:23:10.681 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:10.681+0000 7f8e17fff640 10 client.15891 handle_command_reply: tid=0 2024-05-20T16:23:10.682 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:10.682+0000 7f8e4600d740 1 client.15891 shutdown 2024-05-20T16:23:10.689 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:10.690+0000 7f8e4600d740 1 -- 172.21.15.82:0/2531086352 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ca2713e2f0 msgr2=0x55ca27510d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:23:10.689 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:10.690+0000 7f8e4600d740 1 --2- 172.21.15.82:0/2531086352 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ca2713e2f0 0x55ca27510d30 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f8e240a43f0 tx=0x7f8e2409bcb0 comp rx=0 tx=0).stop 2024-05-20T16:23:10.696 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:10.697+0000 7f8e4600d740 1 -- 172.21.15.82:0/2531086352 shutdown_connections 2024-05-20T16:23:10.697 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:10.697+0000 7f8e4600d740 1 --2- 172.21.15.82:0/2531086352 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x55ca27125d80 0x55ca2752f850 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:23:10.697 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:10.697+0000 7f8e4600d740 1 --2- 172.21.15.82:0/2531086352 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ca271671b0 0x55ca275181d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:23:10.697 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:10.697+0000 7f8e4600d740 1 --2- 172.21.15.82:0/2531086352 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ca271464d0 0x55ca27511270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:23:10.697 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:10.697+0000 7f8e4600d740 1 --2- 172.21.15.82:0/2531086352 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ca2713e2f0 0x55ca27510d30 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:23:10.697 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:10.697+0000 7f8e4600d740 1 -- 172.21.15.82:0/2531086352 >> 172.21.15.82:0/2531086352 conn(0x55ca2715b300 msgr2=0x55ca275018d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:23:10.697 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:10.697+0000 7f8e17fff640 0 client.15891 ms_handle_reset on v2:172.21.15.160:6838/2765554482 2024-05-20T16:23:10.697 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:10.698+0000 7f8e4600d740 1 -- 172.21.15.82:0/2531086352 shutdown_connections 2024-05-20T16:23:10.701 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:10.702+0000 7f8e4600d740 1 -- 172.21.15.82:0/2531086352 wait complete. 2024-05-20T16:23:10.703 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:10.704+0000 7f8e4600d740 20 client.15891 trim_cache size 0 max 16384 2024-05-20T16:23:10.743 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:10.743+0000 7f8e4600d740 1 Processor -- start 2024-05-20T16:23:10.743 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:10.744+0000 7f8e4600d740 1 -- start start 2024-05-20T16:23:10.744 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:10.744+0000 7f8e4600d740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ca2713e2f0 0x55ca2750a450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:23:10.744 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:10.744+0000 7f8e4600d740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ca271464d0 0x55ca2750f500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:23:10.744 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:10.744+0000 7f8e4600d740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ca271671b0 0x55ca2750fa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:23:10.744 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:10.745+0000 7f8e4600d740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f8e1c0125f0 con 0x55ca271464d0 2024-05-20T16:23:10.744 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:10.745+0000 7f8e4600d740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f8e1c00fec0 con 0x55ca2713e2f0 2024-05-20T16:23:10.745 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:10.745+0000 7f8e4600d740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f8e1c00dd10 con 0x55ca271671b0 2024-05-20T16:23:10.746 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:10.747+0000 7f8e37fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ca271464d0 0x55ca2750f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:23:10.747 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:10.747+0000 7f8e37fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ca271464d0 0x55ca2750f500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:46956/0 (socket says 172.21.15.82:46956) 2024-05-20T16:23:10.747 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:10.747+0000 7f8e37fff640 1 -- 172.21.15.82:0/2613861329 learned_addr learned my addr 172.21.15.82:0/2613861329 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:23:10.747 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:10.748+0000 7f8e37fff640 1 -- 172.21.15.82:0/2613861329 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ca271671b0 msgr2=0x55ca2750fa40 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-20T16:23:10.747 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:10.748+0000 7f8e37fff640 1 --2- 172.21.15.82:0/2613861329 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ca271671b0 0x55ca2750fa40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:23:10.747 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:10.748+0000 7f8e37fff640 1 -- 172.21.15.82:0/2613861329 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ca2713e2f0 msgr2=0x55ca2750a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:23:10.747 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:10.748+0000 7f8e37fff640 1 --2- 172.21.15.82:0/2613861329 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ca2713e2f0 0x55ca2750a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:23:10.747 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:10.748+0000 7f8e37fff640 1 -- 172.21.15.82:0/2613861329 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ca2752fd80 con 0x55ca271464d0 2024-05-20T16:23:10.752 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:10.752+0000 7f8e37fff640 1 --2- 172.21.15.82:0/2613861329 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ca271464d0 0x55ca2750f500 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x55ca27503820 tx=0x7f8e2c004410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=351cd4771b0f092d server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:23:10.757 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:10.758+0000 7f8e227fc640 1 -- 172.21.15.82:0/2613861329 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e2c01e020 con 0x55ca271464d0 2024-05-20T16:23:10.757 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:10.758+0000 7f8e227fc640 1 -- 172.21.15.82:0/2613861329 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8e2c003de0 con 0x55ca271464d0 2024-05-20T16:23:10.757 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:10.758+0000 7f8e227fc640 1 -- 172.21.15.82:0/2613861329 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e2c0119a0 con 0x55ca271464d0 2024-05-20T16:23:10.760 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:10.760+0000 7f8e4600d740 1 -- 172.21.15.82:0/2613861329 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ca271464d0 msgr2=0x55ca2750f500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:23:10.760 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:10.760+0000 7f8e4600d740 1 --2- 172.21.15.82:0/2613861329 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ca271464d0 0x55ca2750f500 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x55ca27503820 tx=0x7f8e2c004410 comp rx=0 tx=0).stop 2024-05-20T16:23:10.761 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:10.761+0000 7f8e4600d740 1 -- 172.21.15.82:0/2613861329 shutdown_connections 2024-05-20T16:23:10.761 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:10.761+0000 7f8e4600d740 1 --2- 172.21.15.82:0/2613861329 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ca271464d0 0x55ca2750f500 secure :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0x55ca27503820 tx=0x7f8e2c004410 comp rx=0 tx=0).stop 2024-05-20T16:23:10.761 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:10.761+0000 7f8e4600d740 1 --2- 172.21.15.82:0/2613861329 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ca271671b0 0x55ca2750fa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:23:10.761 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:10.761+0000 7f8e4600d740 1 --2- 172.21.15.82:0/2613861329 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ca2713e2f0 0x55ca2750a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:23:10.761 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:10.761+0000 7f8e4600d740 1 -- 172.21.15.82:0/2613861329 >> 172.21.15.82:0/2613861329 conn(0x55ca2715b300 msgr2=0x55ca2750b110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:23:10.766 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:10.766+0000 7f8e4600d740 1 -- 172.21.15.82:0/2613861329 shutdown_connections 2024-05-20T16:23:10.766 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:10.766+0000 7f8e4600d740 1 -- 172.21.15.82:0/2613861329 wait complete. 2024-05-20T16:23:10.767 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:10.767+0000 7f8e4600d740 1 -- start start 2024-05-20T16:23:10.767 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:10.767+0000 7f8e4600d740 1 Processor -- start 2024-05-20T16:23:10.767 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:10.768+0000 7f8e4600d740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ca2713e2f0 0x55ca27514650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:23:10.768 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:10.768+0000 7f8e4600d740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ca271671b0 0x55ca27514b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:23:10.768 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:10.768+0000 7f8e4600d740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ca270fb1c0 0x55ca27595bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:23:10.768 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:10.769+0000 7f8e4600d740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f8e1c00e380 con 0x55ca2713e2f0 2024-05-20T16:23:10.769 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:10.769+0000 7f8e3cf01640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ca2713e2f0 0x55ca27514650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:23:10.769 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:10.769+0000 7f8e3cf01640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ca2713e2f0 0x55ca27514650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:46972/0 (socket says 172.21.15.82:46972) 2024-05-20T16:23:10.769 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:10.769+0000 7f8e3cf01640 1 -- 172.21.15.82:0/1347202764 learned_addr learned my addr 172.21.15.82:0/1347202764 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:23:10.769 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:10.769+0000 7f8e4600d740 1 -- 172.21.15.82:0/1347202764 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f8e1c00f7f0 con 0x55ca271671b0 2024-05-20T16:23:10.769 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:10.769+0000 7f8e4600d740 1 -- 172.21.15.82:0/1347202764 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f8e1c00ea50 con 0x55ca270fb1c0 2024-05-20T16:23:10.769 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:10.769+0000 7f8e4600d740 10 client.0 _ll_register_callbacks cb 0x7f8e240056b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T16:23:10.771 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:10.772+0000 7f8e37fff640 1 --2- 172.21.15.82:0/1347202764 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ca271671b0 0x55ca27514b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:23:10.772 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:10.772+0000 7f8e3d702640 1 --2- 172.21.15.82:0/1347202764 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ca270fb1c0 0x55ca27595bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:23:10.774 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:10.775+0000 7f8e3cf01640 1 -- 172.21.15.82:0/1347202764 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ca270fb1c0 msgr2=0x55ca27595bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:23:10.774 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:10.775+0000 7f8e3cf01640 1 --2- 172.21.15.82:0/1347202764 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ca270fb1c0 0x55ca27595bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:23:10.774 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:10.775+0000 7f8e3cf01640 1 -- 172.21.15.82:0/1347202764 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ca271671b0 msgr2=0x55ca27514b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:23:10.775 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:10.775+0000 7f8e3cf01640 1 --2- 172.21.15.82:0/1347202764 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ca271671b0 0x55ca27514b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:23:10.775 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:10.775+0000 7f8e3cf01640 1 -- 172.21.15.82:0/1347202764 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ca27515150 con 0x55ca2713e2f0 2024-05-20T16:23:10.784 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:10.779+0000 7f8e3cf01640 1 --2- 172.21.15.82:0/1347202764 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ca2713e2f0 0x55ca27514650 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f8e2409bc40 tx=0x7f8e240cfbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a79e039e4f8cb2a server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:23:10.784 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:10.783+0000 7f8e17fff640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-05-20T16:23:10.784 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:10.783+0000 7f8e17fff640 1 -- 172.21.15.82:0/1347202764 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e2409b7f0 con 0x55ca2713e2f0 2024-05-20T16:23:10.784 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:10.783+0000 7f8e17fff640 1 -- 172.21.15.82:0/1347202764 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9150+0+0 (secure 0 0 0) 0x7f8e2401fda0 con 0x55ca2713e2f0 2024-05-20T16:23:10.784 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:10.784+0000 7f8e4600d740 1 -- 172.21.15.82:0/1347202764 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8e2c003de0 con 0x55ca2713e2f0 2024-05-20T16:23:10.786 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:10.786+0000 7f8e17fff640 1 -- 172.21.15.82:0/1347202764 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f8e240a0060 con 0x55ca2713e2f0 2024-05-20T16:23:10.786 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:10.786+0000 7f8e4600d740 10 client.15897 fetch_fsmap learned FSMap version 27 2024-05-20T16:23:10.786 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:10.787+0000 7f8e4600d740 1 -- 172.21.15.82:0/1347202764 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ca27515150 con 0x55ca2713e2f0 2024-05-20T16:23:10.787 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:10.788+0000 7f8e17fff640 1 -- 172.21.15.82:0/1347202764 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8e2409c3b0 con 0x55ca2713e2f0 2024-05-20T16:23:10.787 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:10.788+0000 7f8e17fff640 1 -- 172.21.15.82:0/1347202764 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f8e2409b100 con 0x55ca2713e2f0 2024-05-20T16:23:10.789 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:10.789+0000 7f8e4600d740 10 client.15897 fetch_fsmap finished waiting for FSMap version 27 2024-05-20T16:23:10.789 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:10.789+0000 7f8e4600d740 10 client.15897 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T16:23:10.789 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:10.789+0000 7f8e4600d740 20 client.15897 populate_metadata read hostname 'smithi082' 2024-05-20T16:23:10.789 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:10.789+0000 7f8e4600d740 1 --2- 172.21.15.82:0/1347202764 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x55ca27597730 0x55ca27599b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:23:10.789 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:10.790+0000 7f8e4600d740 4 client.15897 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-20T16:23:10.789 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:10.790+0000 7f8e4600d740 1 -- 172.21.15.82:0/1347202764 --> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f8e2c003de0 con 0x55ca27597730 2024-05-20T16:23:10.790 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:10.791+0000 7f8e37fff640 1 --2- 172.21.15.82:0/1347202764 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x55ca27597730 0x55ca27599b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:23:10.791 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:10.791+0000 7f8e37fff640 1 --2- 172.21.15.82:0/1347202764 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x55ca27597730 0x55ca27599b50 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ee1a8ffb6b5f1792 server_cookie=7392df5690bf914c in_seq=0 out_seq=0 2024-05-20T16:23:10.791 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:10.792+0000 7f8e17fff640 10 client.15897 ms_handle_connect on v2:172.21.15.160:6838/2765554482 2024-05-20T16:23:13.062 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:23:12 smithi160 ceph-mon[36347]: pgmap v639: 129 pgs: 129 active+clean; 10 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 704 MiB/s wr, 1.02k op/s 2024-05-20T16:23:13.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:23:12 smithi082 ceph-mon[32238]: pgmap v639: 129 pgs: 129 active+clean; 10 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 704 MiB/s wr, 1.02k op/s 2024-05-20T16:23:13.135 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:23:12 smithi184 ceph-mon[36209]: pgmap v639: 129 pgs: 129 active+clean; 10 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 704 MiB/s wr, 1.02k op/s 2024-05-20T16:23:15.110 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:23:14 smithi082 ceph-mon[32238]: pgmap v640: 129 pgs: 129 active+clean; 10 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 809 MiB/s wr, 1.22k op/s 2024-05-20T16:23:15.135 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:23:14 smithi184 ceph-mon[36209]: pgmap v640: 129 pgs: 129 active+clean; 10 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 809 MiB/s wr, 1.22k op/s 2024-05-20T16:23:15.312 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:23:14 smithi160 ceph-mon[36347]: pgmap v640: 129 pgs: 129 active+clean; 10 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 809 MiB/s wr, 1.22k op/s 2024-05-20T16:23:16.506 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:16.505+0000 7f8e17fff640 1 -- 172.21.15.82:0/1347202764 <== mds.0 v2:172.21.15.160:6838/2765554482 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f8e2c003de0 con 0x55ca27597730 2024-05-20T16:23:16.506 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:16.505+0000 7f8e17fff640 10 client.15897 handle_command_reply: tid=0 2024-05-20T16:23:16.506 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:16.505+0000 7f8e4600d740 1 client.15897 shutdown 2024-05-20T16:23:16.506 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:16.505+0000 7f8e4600d740 1 -- 172.21.15.82:0/1347202764 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ca2713e2f0 msgr2=0x55ca27514650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:23:16.506 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:16.505+0000 7f8e4600d740 1 --2- 172.21.15.82:0/1347202764 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ca2713e2f0 0x55ca27514650 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f8e2409bc40 tx=0x7f8e240cfbe0 comp rx=0 tx=0).stop 2024-05-20T16:23:16.506 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:16.506+0000 7f8e4600d740 1 -- 172.21.15.82:0/1347202764 shutdown_connections 2024-05-20T16:23:16.507 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:16.506+0000 7f8e4600d740 1 --2- 172.21.15.82:0/1347202764 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x55ca27597730 0x55ca27599b50 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:23:16.507 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:16.506+0000 7f8e4600d740 1 --2- 172.21.15.82:0/1347202764 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ca270fb1c0 0x55ca27595bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:23:16.507 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:16.506+0000 7f8e17fff640 0 client.15897 ms_handle_reset on v2:172.21.15.160:6838/2765554482 2024-05-20T16:23:16.507 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:16.506+0000 7f8e4600d740 1 --2- 172.21.15.82:0/1347202764 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ca271671b0 0x55ca27514b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:23:16.507 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:16.506+0000 7f8e4600d740 1 --2- 172.21.15.82:0/1347202764 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ca2713e2f0 0x55ca27514650 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:23:16.507 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:16.506+0000 7f8e4600d740 1 -- 172.21.15.82:0/1347202764 >> 172.21.15.82:0/1347202764 conn(0x55ca2715b300 msgr2=0x55ca2750f500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:23:16.507 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:16.506+0000 7f8e4600d740 1 -- 172.21.15.82:0/1347202764 shutdown_connections 2024-05-20T16:23:16.507 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:16.506+0000 7f8e4600d740 1 -- 172.21.15.82:0/1347202764 wait complete. 2024-05-20T16:23:16.507 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:16.506+0000 7f8e4600d740 20 client.15897 trim_cache size 0 max 16384 2024-05-20T16:23:16.507 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:{ 2024-05-20T16:23:16.508 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "message": "", 2024-05-20T16:23:16.508 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "return_code": 0 2024-05-20T16:23:16.508 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:} 2024-05-20T16:23:16.508 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:16.507+0000 7f8e3df03640 1 -- 172.21.15.82:0/1625674003 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f8e280a2d90 msgr2=0x7f8e280a52a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:23:16.508 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:16.507+0000 7f8e3df03640 1 --2- 172.21.15.82:0/1625674003 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f8e280a2d90 0x7f8e280a52a0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f8e30030ad0 tx=0x7f8e3002a710 comp rx=0 tx=0).stop 2024-05-20T16:23:16.509 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:16.507+0000 7f8e3df03640 1 -- 172.21.15.82:0/1625674003 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8e381092f0 msgr2=0x7f8e381a7e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:23:16.509 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:16.507+0000 7f8e3df03640 1 --2- 172.21.15.82:0/1625674003 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8e381092f0 0x7f8e381a7e20 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f8e2400e9c0 tx=0x7f8e2400ee80 comp rx=0 tx=0).stop 2024-05-20T16:23:16.509 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:16.507+0000 7f8e3df03640 1 -- 172.21.15.82:0/1625674003 shutdown_connections 2024-05-20T16:23:16.509 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:16.507+0000 7f8e3df03640 1 --2- 172.21.15.82:0/1625674003 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f8e280a2d90 0x7f8e280a52a0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:23:16.509 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:16.507+0000 7f8e3df03640 1 --2- 172.21.15.82:0/1625674003 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8e3810ee60 0x7f8e381af2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:23:16.509 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:16.507+0000 7f8e3df03640 1 --2- 172.21.15.82:0/1625674003 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8e3810a140 0x7f8e381a8360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:23:16.509 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:16.507+0000 7f8e3df03640 1 --2- 172.21.15.82:0/1625674003 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8e381092f0 0x7f8e381a7e20 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:23:16.509 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:16.507+0000 7f8e3df03640 1 -- 172.21.15.82:0/1625674003 >> 172.21.15.82:0/1625674003 conn(0x7f8e3806f2e0 msgr2=0x7f8e38071770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:23:16.509 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:16.507+0000 7f8e3df03640 1 -- 172.21.15.82:0/1625674003 shutdown_connections 2024-05-20T16:23:16.509 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:16.507+0000 7f8e3df03640 1 -- 172.21.15.82:0/1625674003 wait complete. 2024-05-20T16:23:17.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:23:16 smithi082 ceph-mon[32238]: pgmap v641: 129 pgs: 129 active+clean; 10 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 703 MiB/s wr, 956 op/s 2024-05-20T16:23:17.136 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:23:16 smithi184 ceph-mon[36209]: pgmap v641: 129 pgs: 129 active+clean; 10 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 703 MiB/s wr, 956 op/s 2024-05-20T16:23:17.312 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:23:16 smithi160 ceph-mon[36347]: pgmap v641: 129 pgs: 129 active+clean; 10 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 703 MiB/s wr, 956 op/s 2024-05-20T16:23:18.135 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:23:17 smithi184 ceph-mon[36209]: pgmap v642: 129 pgs: 129 active+clean; 10 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 728 MiB/s wr, 1.07k op/s 2024-05-20T16:23:18.312 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:23:17 smithi160 ceph-mon[36347]: pgmap v642: 129 pgs: 129 active+clean; 10 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 728 MiB/s wr, 1.07k op/s 2024-05-20T16:23:18.362 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:23:17 smithi082 ceph-mon[32238]: pgmap v642: 129 pgs: 129 active+clean; 10 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 728 MiB/s wr, 1.07k op/s 2024-05-20T16:23:21.062 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:23:20 smithi160 ceph-mon[36347]: pgmap v643: 129 pgs: 129 active+clean; 10 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 830 MiB/s wr, 1.41k op/s 2024-05-20T16:23:21.112 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:23:20 smithi082 ceph-mon[32238]: pgmap v643: 129 pgs: 129 active+clean; 10 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 830 MiB/s wr, 1.41k op/s 2024-05-20T16:23:21.135 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:23:20 smithi184 ceph-mon[36209]: pgmap v643: 129 pgs: 129 active+clean; 10 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 830 MiB/s wr, 1.41k op/s 2024-05-20T16:23:23.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:23:22 smithi082 ceph-mon[32238]: pgmap v644: 129 pgs: 129 active+clean; 10 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 636 MiB/s wr, 1.23k op/s 2024-05-20T16:23:23.135 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:23:22 smithi184 ceph-mon[36209]: pgmap v644: 129 pgs: 129 active+clean; 10 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 636 MiB/s wr, 1.23k op/s 2024-05-20T16:23:23.312 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:23:22 smithi160 ceph-mon[36347]: pgmap v644: 129 pgs: 129 active+clean; 10 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 636 MiB/s wr, 1.23k op/s 2024-05-20T16:23:25.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:23:24 smithi082 ceph-mon[32238]: pgmap v645: 129 pgs: 129 active+clean; 10 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 769 MiB/s wr, 1.31k op/s 2024-05-20T16:23:25.135 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:23:24 smithi184 ceph-mon[36209]: pgmap v645: 129 pgs: 129 active+clean; 10 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 769 MiB/s wr, 1.31k op/s 2024-05-20T16:23:25.312 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:23:24 smithi160 ceph-mon[36347]: pgmap v645: 129 pgs: 129 active+clean; 10 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 769 MiB/s wr, 1.31k op/s 2024-05-20T16:23:27.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:23:26 smithi082 ceph-mon[32238]: pgmap v646: 129 pgs: 129 active+clean; 10 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 655 MiB/s wr, 1.02k op/s 2024-05-20T16:23:27.135 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:23:26 smithi184 ceph-mon[36209]: pgmap v646: 129 pgs: 129 active+clean; 10 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 655 MiB/s wr, 1.02k op/s 2024-05-20T16:23:27.312 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:23:26 smithi160 ceph-mon[36347]: pgmap v646: 129 pgs: 129 active+clean; 10 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 655 MiB/s wr, 1.02k op/s 2024-05-20T16:23:29.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:23:28 smithi082 ceph-mon[32238]: pgmap v647: 129 pgs: 129 active+clean; 10 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 690 MiB/s wr, 1.08k op/s 2024-05-20T16:23:29.135 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:23:28 smithi184 ceph-mon[36209]: pgmap v647: 129 pgs: 129 active+clean; 10 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 690 MiB/s wr, 1.08k op/s 2024-05-20T16:23:29.311 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:23:28 smithi160 ceph-mon[36347]: pgmap v647: 129 pgs: 129 active+clean; 10 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 690 MiB/s wr, 1.08k op/s 2024-05-20T16:23:29.786 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:29.786+0000 7fd32ad03640 1 -- 172.21.15.82:0/3718934997 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd32410ee60 msgr2=0x7fd32410f270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:23:29.787 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:29.786+0000 7fd32ad03640 1 --2- 172.21.15.82:0/3718934997 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd32410ee60 0x7fd32410f270 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fd318009900 tx=0x7fd31802a440 comp rx=0 tx=0).stop 2024-05-20T16:23:29.792 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:29.792+0000 7fd32ad03640 1 -- 172.21.15.82:0/3718934997 shutdown_connections 2024-05-20T16:23:29.792 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:29.792+0000 7fd32ad03640 1 --2- 172.21.15.82:0/3718934997 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd32410ee60 0x7fd32410f270 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:23:29.792 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:29.792+0000 7fd32ad03640 1 --2- 172.21.15.82:0/3718934997 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd32410a140 0x7fd32410a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:23:29.792 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:29.792+0000 7fd32ad03640 1 --2- 172.21.15.82:0/3718934997 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd3241092f0 0x7fd324109c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:23:29.792 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:29.792+0000 7fd32ad03640 1 -- 172.21.15.82:0/3718934997 >> 172.21.15.82:0/3718934997 conn(0x7fd32406f2e0 msgr2=0x7fd324071790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:23:29.799 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:29.799+0000 7fd32ad03640 1 -- 172.21.15.82:0/3718934997 shutdown_connections 2024-05-20T16:23:29.802 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:29.802+0000 7fd32ad03640 1 -- 172.21.15.82:0/3718934997 wait complete. 2024-05-20T16:23:29.805 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:29.803+0000 7fd32ad03640 1 Processor -- start 2024-05-20T16:23:29.805 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:29.806+0000 7fd32ad03640 1 -- start start 2024-05-20T16:23:29.808 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:29.806+0000 7fd32ad03640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd3241092f0 0x7fd32411a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:23:29.809 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:29.806+0000 7fd32ad03640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd32410a140 0x7fd32411abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:23:29.809 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:29.806+0000 7fd32ad03640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd32410ee60 0x7fd324118d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:23:29.809 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:29.806+0000 7fd32ad03640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fd324111080 con 0x7fd3241092f0 2024-05-20T16:23:29.809 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:29.806+0000 7fd32ad03640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fd324110f00 con 0x7fd32410a140 2024-05-20T16:23:29.809 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:29.806+0000 7fd32ad03640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd324111200 con 0x7fd32410ee60 2024-05-20T16:23:29.809 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:29.806+0000 7fd329d01640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd3241092f0 0x7fd32411a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:23:29.809 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:29.806+0000 7fd329d01640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd3241092f0 0x7fd32411a6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:36210/0 (socket says 172.21.15.82:36210) 2024-05-20T16:23:29.809 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:29.806+0000 7fd329d01640 1 -- 172.21.15.82:0/2880489991 learned_addr learned my addr 172.21.15.82:0/2880489991 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:23:29.809 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:29.807+0000 7fd329d01640 1 -- 172.21.15.82:0/2880489991 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd32410ee60 msgr2=0x7fd324118d50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-20T16:23:29.809 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:29.807+0000 7fd329d01640 1 --2- 172.21.15.82:0/2880489991 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd32410ee60 0x7fd324118d50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:23:29.810 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:29.807+0000 7fd329d01640 1 -- 172.21.15.82:0/2880489991 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd32410a140 msgr2=0x7fd32411abf0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-20T16:23:29.810 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:29.807+0000 7fd329d01640 1 --2- 172.21.15.82:0/2880489991 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd32410a140 0x7fd32411abf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:23:29.810 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:29.807+0000 7fd329d01640 1 -- 172.21.15.82:0/2880489991 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd324119570 con 0x7fd3241092f0 2024-05-20T16:23:29.810 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:29.807+0000 7fd329d01640 1 --2- 172.21.15.82:0/2880489991 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd3241092f0 0x7fd32411a6b0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fd32000b9c0 tx=0x7fd32000be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:23:29.811 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:29.811+0000 7fd316ffd640 1 -- 172.21.15.82:0/2880489991 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd320019070 con 0x7fd3241092f0 2024-05-20T16:23:29.811 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:29.811+0000 7fd316ffd640 1 -- 172.21.15.82:0/2880489991 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd32000cba0 con 0x7fd3241092f0 2024-05-20T16:23:29.812 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:29.811+0000 7fd316ffd640 1 -- 172.21.15.82:0/2880489991 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd320012670 con 0x7fd3241092f0 2024-05-20T16:23:29.812 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:29.811+0000 7fd32ad03640 1 -- 172.21.15.82:0/2880489991 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd324119860 con 0x7fd3241092f0 2024-05-20T16:23:29.812 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:29.812+0000 7fd32ad03640 1 -- 172.21.15.82:0/2880489991 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3241b8cf0 con 0x7fd3241092f0 2024-05-20T16:23:29.817 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:29.817+0000 7fd316ffd640 1 -- 172.21.15.82:0/2880489991 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fd32001c070 con 0x7fd3241092f0 2024-05-20T16:23:29.819 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:29.819+0000 7fd316ffd640 1 --2- 172.21.15.82:0/2880489991 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fd3100a2c40 0x7fd3100a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:23:29.819 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:29.819+0000 7fd316ffd640 1 -- 172.21.15.82:0/2880489991 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9150+0+0 (secure 0 0 0) 0x7fd3200cf030 con 0x7fd3241092f0 2024-05-20T16:23:29.822 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:29.822+0000 7fd332e12740 1 Processor -- start 2024-05-20T16:23:29.822 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:29.820+0000 7fd329500640 1 --2- 172.21.15.82:0/2880489991 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fd3100a2c40 0x7fd3100a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:23:29.823 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:29.823+0000 7fd329500640 1 --2- 172.21.15.82:0/2880489991 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fd3100a2c40 0x7fd3100a5150 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fd32411a050 tx=0x7fd31c006bd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:23:29.825 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:29.825+0000 7fd332e12740 1 -- start start 2024-05-20T16:23:29.825 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:29.826+0000 7fd332e12740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56164c1ba1b0 0x56164c55a060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:23:29.826 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:29.826+0000 7fd332e12740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56164c1912f0 0x56164c55a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:23:29.826 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:29.826+0000 7fd332e12740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56164c1994d0 0x56164c5614e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:23:29.826 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:29.826+0000 7fd332e12740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x56164c4be340 con 0x56164c1ba1b0 2024-05-20T16:23:29.826 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:29.826+0000 7fd332e12740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x56164c466290 con 0x56164c1994d0 2024-05-20T16:23:29.826 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:29.826+0000 7fd332e12740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x56164c4ba300 con 0x56164c1912f0 2024-05-20T16:23:29.827 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:29.828+0000 7fd329d01640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56164c1ba1b0 0x56164c55a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:23:29.827 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:29.828+0000 7fd329d01640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56164c1ba1b0 0x56164c55a060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:36226/0 (socket says 172.21.15.82:36226) 2024-05-20T16:23:29.827 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:29.828+0000 7fd329d01640 1 -- 172.21.15.82:0/3967696120 learned_addr learned my addr 172.21.15.82:0/3967696120 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:23:29.828 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:29.828+0000 7fd329d01640 1 -- 172.21.15.82:0/3967696120 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56164c1912f0 msgr2=0x56164c55a5e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-20T16:23:29.828 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:29.828+0000 7fd329500640 1 --2- 172.21.15.82:0/3967696120 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56164c1912f0 0x56164c55a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:23:29.828 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:29.828+0000 7fd32a502640 1 --2- 172.21.15.82:0/3967696120 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56164c1994d0 0x56164c5614e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:23:29.829 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:29.829+0000 7fd329d01640 1 --2- 172.21.15.82:0/3967696120 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56164c1912f0 0x56164c55a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:23:29.829 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:29.829+0000 7fd329d01640 1 -- 172.21.15.82:0/3967696120 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56164c1994d0 msgr2=0x56164c5614e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:23:29.829 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:29.829+0000 7fd329d01640 1 --2- 172.21.15.82:0/3967696120 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56164c1994d0 0x56164c5614e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:23:29.829 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:29.829+0000 7fd329d01640 1 -- 172.21.15.82:0/3967696120 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56164c48a8e0 con 0x56164c1ba1b0 2024-05-20T16:23:29.829 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:29.829+0000 7fd329500640 1 --2- 172.21.15.82:0/3967696120 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56164c1912f0 0x56164c55a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T16:23:29.829 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:29.829+0000 7fd329d01640 1 --2- 172.21.15.82:0/3967696120 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56164c1ba1b0 0x56164c55a060 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fd32009b490 tx=0x7fd32009ae60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abb4dcf52f5bfb23 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:23:29.830 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:29.830+0000 7fd30ffff640 1 -- 172.21.15.82:0/3967696120 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd32009b500 con 0x56164c1ba1b0 2024-05-20T16:23:29.830 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:29.830+0000 7fd30ffff640 1 -- 172.21.15.82:0/3967696120 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd32009f8b0 con 0x56164c1ba1b0 2024-05-20T16:23:29.830 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:29.830+0000 7fd30ffff640 1 -- 172.21.15.82:0/3967696120 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd32009a950 con 0x56164c1ba1b0 2024-05-20T16:23:29.831 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:29.831+0000 7fd332e12740 1 -- 172.21.15.82:0/3967696120 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56164c1ba1b0 msgr2=0x56164c55a060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:23:29.831 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:29.831+0000 7fd332e12740 1 --2- 172.21.15.82:0/3967696120 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56164c1ba1b0 0x56164c55a060 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fd32009b490 tx=0x7fd32009ae60 comp rx=0 tx=0).stop 2024-05-20T16:23:29.831 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:29.831+0000 7fd332e12740 1 -- 172.21.15.82:0/3967696120 shutdown_connections 2024-05-20T16:23:29.831 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:29.831+0000 7fd332e12740 1 --2- 172.21.15.82:0/3967696120 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56164c1994d0 0x56164c5614e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:23:29.831 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:29.831+0000 7fd332e12740 1 --2- 172.21.15.82:0/3967696120 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56164c1912f0 0x56164c55a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:23:29.831 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:29.831+0000 7fd332e12740 1 --2- 172.21.15.82:0/3967696120 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56164c1ba1b0 0x56164c55a060 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:23:29.832 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:29.831+0000 7fd332e12740 1 -- 172.21.15.82:0/3967696120 >> 172.21.15.82:0/3967696120 conn(0x56164c1ae300 msgr2=0x56164c5559f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:23:29.834 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:29.833+0000 7fd332e12740 1 -- 172.21.15.82:0/3967696120 shutdown_connections 2024-05-20T16:23:29.837 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:29.837+0000 7fd332e12740 1 -- 172.21.15.82:0/3967696120 wait complete. 2024-05-20T16:23:29.837 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:29.837+0000 7fd332e12740 1 -- start start 2024-05-20T16:23:29.837 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:29.838+0000 7fd332e12740 1 Processor -- start 2024-05-20T16:23:29.838 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:29.838+0000 7fd332e12740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56164c1912f0 0x56164c563e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:23:29.838 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:29.838+0000 7fd332e12740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56164c1994d0 0x56164c564340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:23:29.838 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:29.838+0000 7fd332e12740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56164c1ba1b0 0x56164c56b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:23:29.838 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:29.838+0000 7fd332e12740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x56164c561f30 con 0x56164c1912f0 2024-05-20T16:23:29.838 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:29.838+0000 7fd332e12740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x56164c479790 con 0x56164c1ba1b0 2024-05-20T16:23:29.838 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:29.838+0000 7fd332e12740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x56164c5620b0 con 0x56164c1994d0 2024-05-20T16:23:29.838 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:29.838+0000 7fd332e12740 10 client.0 _ll_register_callbacks cb 0x56164c4b4a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T16:23:29.839 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:29.839+0000 7fd329500640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56164c1994d0 0x56164c564340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:23:29.839 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:29.839+0000 7fd329500640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56164c1994d0 0x56164c564340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.82:47204/0 (socket says 172.21.15.82:47204) 2024-05-20T16:23:29.839 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:29.839+0000 7fd329500640 1 -- 172.21.15.82:0/4043589411 learned_addr learned my addr 172.21.15.82:0/4043589411 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:23:29.840 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:29.839+0000 7fd32a502640 1 --2- 172.21.15.82:0/4043589411 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56164c1ba1b0 0x56164c56b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:23:29.840 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:29.839+0000 7fd329500640 1 -- 172.21.15.82:0/4043589411 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56164c1ba1b0 msgr2=0x56164c56b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:23:29.840 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:29.839+0000 7fd329500640 1 --2- 172.21.15.82:0/4043589411 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56164c1ba1b0 0x56164c56b2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:23:29.840 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:29.839+0000 7fd329500640 1 -- 172.21.15.82:0/4043589411 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56164c1912f0 msgr2=0x56164c563e00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-20T16:23:29.840 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:29.839+0000 7fd329500640 1 --2- 172.21.15.82:0/4043589411 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56164c1912f0 0x56164c563e00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:23:29.840 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:29.839+0000 7fd329500640 1 -- 172.21.15.82:0/4043589411 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56164c1d2210 con 0x56164c1994d0 2024-05-20T16:23:29.840 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:29.840+0000 7fd329500640 1 --2- 172.21.15.82:0/4043589411 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56164c1994d0 0x56164c564340 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fd31c012390 tx=0x7fd31c004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c625cb441df3e965 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:23:29.840 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:29.840+0000 7fd30d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-05-20T16:23:29.841 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:29.840+0000 7fd30d7fa640 1 -- 172.21.15.82:0/4043589411 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd31c01d7d0 con 0x56164c1994d0 2024-05-20T16:23:29.841 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:29.840+0000 7fd332e12740 1 -- 172.21.15.82:0/4043589411 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd32009f8b0 con 0x56164c1994d0 2024-05-20T16:23:29.841 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:29.840+0000 7fd30d7fa640 1 -- 172.21.15.82:0/4043589411 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9150+0+0 (secure 0 0 0) 0x7fd31c01d970 con 0x56164c1994d0 2024-05-20T16:23:29.845 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:29.842+0000 7fd30d7fa640 1 -- 172.21.15.82:0/4043589411 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fd31c02c050 con 0x56164c1994d0 2024-05-20T16:23:29.845 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:29.846+0000 7fd332e12740 10 client.25429 fetch_fsmap learned FSMap version 27 2024-05-20T16:23:29.845 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:29.846+0000 7fd332e12740 1 -- 172.21.15.82:0/4043589411 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56164c29e620 con 0x56164c1994d0 2024-05-20T16:23:29.849 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:29.846+0000 7fd30d7fa640 1 -- 172.21.15.82:0/4043589411 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd32009f8b0 con 0x56164c1994d0 2024-05-20T16:23:29.849 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:29.846+0000 7fd30d7fa640 1 -- 172.21.15.82:0/4043589411 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fd31c012070 con 0x56164c1994d0 2024-05-20T16:23:29.849 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:29.850+0000 7fd332e12740 10 client.25429 fetch_fsmap finished waiting for FSMap version 27 2024-05-20T16:23:29.849 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:29.850+0000 7fd332e12740 10 client.25429 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T16:23:29.849 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:29.850+0000 7fd332e12740 20 client.25429 populate_metadata read hostname 'smithi082' 2024-05-20T16:23:29.850 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:29.850+0000 7fd332e12740 1 --2- 172.21.15.82:0/4043589411 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x56164c178d80 0x56164c5717f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:23:29.850 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:29.850+0000 7fd332e12740 4 client.25429 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T16:23:29.850 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:29.850+0000 7fd332e12740 1 -- 172.21.15.82:0/4043589411 --> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56164c4cba70 con 0x56164c178d80 2024-05-20T16:23:29.851 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:29.851+0000 7fd329d01640 1 --2- 172.21.15.82:0/4043589411 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x56164c178d80 0x56164c5717f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:23:29.854 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:29.854+0000 7fd329d01640 1 --2- 172.21.15.82:0/4043589411 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x56164c178d80 0x56164c5717f0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=86f1fc3a5606ecba server_cookie=c5821dfaacf734fe in_seq=0 out_seq=0 2024-05-20T16:23:29.854 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:29.855+0000 7fd30d7fa640 10 client.25429 ms_handle_connect on v2:172.21.15.160:6838/2765554482 2024-05-20T16:23:29.860 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:29.860+0000 7fd30d7fa640 1 -- 172.21.15.82:0/4043589411 <== mds.0 v2:172.21.15.160:6838/2765554482 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56164c4cba70 con 0x56164c178d80 2024-05-20T16:23:29.860 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:29.860+0000 7fd30d7fa640 10 client.25429 handle_command_reply: tid=0 2024-05-20T16:23:29.860 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:29.860+0000 7fd332e12740 1 client.25429 shutdown 2024-05-20T16:23:29.861 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:29.862+0000 7fd332e12740 1 -- 172.21.15.82:0/4043589411 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56164c1994d0 msgr2=0x56164c564340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:23:29.862 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:29.862+0000 7fd332e12740 1 --2- 172.21.15.82:0/4043589411 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56164c1994d0 0x56164c564340 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fd31c012390 tx=0x7fd31c004750 comp rx=0 tx=0).stop 2024-05-20T16:23:29.863 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:29.864+0000 7fd332e12740 1 -- 172.21.15.82:0/4043589411 shutdown_connections 2024-05-20T16:23:29.863 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:29.864+0000 7fd332e12740 1 --2- 172.21.15.82:0/4043589411 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x56164c178d80 0x56164c5717f0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:23:29.864 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:29.864+0000 7fd332e12740 1 --2- 172.21.15.82:0/4043589411 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56164c1ba1b0 0x56164c56b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:23:29.864 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:29.864+0000 7fd332e12740 1 --2- 172.21.15.82:0/4043589411 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56164c1994d0 0x56164c564340 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:23:29.864 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:29.864+0000 7fd332e12740 1 --2- 172.21.15.82:0/4043589411 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56164c1912f0 0x56164c563e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:23:29.864 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:29.864+0000 7fd332e12740 1 -- 172.21.15.82:0/4043589411 >> 172.21.15.82:0/4043589411 conn(0x56164c1ae300 msgr2=0x56164c5548d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:23:29.864 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:29.864+0000 7fd30d7fa640 0 client.25429 ms_handle_reset on v2:172.21.15.160:6838/2765554482 2024-05-20T16:23:29.865 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:29.865+0000 7fd332e12740 1 -- 172.21.15.82:0/4043589411 shutdown_connections 2024-05-20T16:23:29.865 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:29.866+0000 7fd332e12740 1 -- 172.21.15.82:0/4043589411 wait complete. 2024-05-20T16:23:29.867 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:29.868+0000 7fd332e12740 20 client.25429 trim_cache size 0 max 16384 2024-05-20T16:23:29.939 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:29.939+0000 7fd332e12740 1 Processor -- start 2024-05-20T16:23:29.943 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:29.944+0000 7fd332e12740 1 -- start start 2024-05-20T16:23:29.944 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:29.944+0000 7fd332e12740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56164c1912f0 0x56164c55d980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:23:29.945 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:29.945+0000 7fd332e12740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56164c1994d0 0x56164c56b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:23:29.945 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:29.945+0000 7fd332e12740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56164c1ba1b0 0x56164c56b700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:23:29.945 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:29.945+0000 7fd332e12740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fd3241c5b40 con 0x56164c1912f0 2024-05-20T16:23:29.945 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:29.945+0000 7fd332e12740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fd3241c3410 con 0x56164c1ba1b0 2024-05-20T16:23:29.945 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:29.945+0000 7fd332e12740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd3241c1260 con 0x56164c1994d0 2024-05-20T16:23:29.948 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:29.946+0000 7fd329500640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56164c1994d0 0x56164c56b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:23:29.948 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:29.946+0000 7fd329500640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56164c1994d0 0x56164c56b1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.82:47216/0 (socket says 172.21.15.82:47216) 2024-05-20T16:23:29.949 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:29.946+0000 7fd329500640 1 -- 172.21.15.82:0/1408353652 learned_addr learned my addr 172.21.15.82:0/1408353652 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:23:29.949 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:29.946+0000 7fd32a502640 1 --2- 172.21.15.82:0/1408353652 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56164c1ba1b0 0x56164c56b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:23:29.949 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:29.947+0000 7fd329500640 1 -- 172.21.15.82:0/1408353652 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56164c1ba1b0 msgr2=0x56164c56b700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:23:29.949 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:29.947+0000 7fd329500640 1 --2- 172.21.15.82:0/1408353652 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56164c1ba1b0 0x56164c56b700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:23:29.949 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:29.947+0000 7fd329500640 1 -- 172.21.15.82:0/1408353652 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56164c1912f0 msgr2=0x56164c55d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:23:29.949 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:29.947+0000 7fd329500640 1 --2- 172.21.15.82:0/1408353652 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56164c1912f0 0x56164c55d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:23:29.949 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:29.947+0000 7fd329500640 1 -- 172.21.15.82:0/1408353652 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56164c562b30 con 0x56164c1994d0 2024-05-20T16:23:29.953 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:29.949+0000 7fd329500640 1 --2- 172.21.15.82:0/1408353652 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56164c1994d0 0x56164c56b1c0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fd31c009ed0 tx=0x7fd31c0046a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a7bbdbee9ddb57af server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:23:29.955 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:29.954+0000 7fd30f7fe640 1 -- 172.21.15.82:0/1408353652 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd31c012070 con 0x56164c1994d0 2024-05-20T16:23:29.955 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:29.954+0000 7fd30f7fe640 1 -- 172.21.15.82:0/1408353652 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd31c023070 con 0x56164c1994d0 2024-05-20T16:23:29.955 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:29.954+0000 7fd30f7fe640 1 -- 172.21.15.82:0/1408353652 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd31c01e7c0 con 0x56164c1994d0 2024-05-20T16:23:29.960 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:29.959+0000 7fd332e12740 1 -- 172.21.15.82:0/1408353652 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56164c1994d0 msgr2=0x56164c56b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:23:29.960 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:29.960+0000 7fd332e12740 1 --2- 172.21.15.82:0/1408353652 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56164c1994d0 0x56164c56b1c0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fd31c009ed0 tx=0x7fd31c0046a0 comp rx=0 tx=0).stop 2024-05-20T16:23:29.965 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:29.964+0000 7fd332e12740 1 -- 172.21.15.82:0/1408353652 shutdown_connections 2024-05-20T16:23:29.965 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:29.964+0000 7fd332e12740 1 --2- 172.21.15.82:0/1408353652 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56164c1ba1b0 0x56164c56b700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:23:29.965 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:29.965+0000 7fd332e12740 1 --2- 172.21.15.82:0/1408353652 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56164c1994d0 0x56164c56b1c0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:23:29.965 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:29.965+0000 7fd332e12740 1 --2- 172.21.15.82:0/1408353652 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56164c1912f0 0x56164c55d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:23:29.965 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:29.965+0000 7fd332e12740 1 -- 172.21.15.82:0/1408353652 >> 172.21.15.82:0/1408353652 conn(0x56164c1ae300 msgr2=0x56164c55d270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:23:29.975 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:29.975+0000 7fd332e12740 1 -- 172.21.15.82:0/1408353652 shutdown_connections 2024-05-20T16:23:29.975 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:29.975+0000 7fd332e12740 1 -- 172.21.15.82:0/1408353652 wait complete. 2024-05-20T16:23:29.976 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:29.976+0000 7fd332e12740 1 -- start start 2024-05-20T16:23:29.976 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:29.977+0000 7fd332e12740 1 Processor -- start 2024-05-20T16:23:29.978 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:29.977+0000 7fd332e12740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56164c1912f0 0x56164c56b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:23:29.979 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:29.978+0000 7fd332e12740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56164c1994d0 0x56164c56ba90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:23:29.979 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:29.979+0000 7fd332e12740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56164c1ba1b0 0x56164c568170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:23:29.979 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:29.979+0000 7fd332e12740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fd3241c18d0 con 0x56164c1994d0 2024-05-20T16:23:29.979 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:29.979+0000 7fd332e12740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fd3241c2d40 con 0x56164c1ba1b0 2024-05-20T16:23:29.979 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:29.979+0000 7fd332e12740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd3241c1fa0 con 0x56164c1912f0 2024-05-20T16:23:29.979 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:29.979+0000 7fd332e12740 10 client.0 _ll_register_callbacks cb 0x7fd31c009380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T16:23:29.980 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:29.979+0000 7fd32a502640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56164c1ba1b0 0x56164c568170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:23:29.980 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:29.979+0000 7fd32a502640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56164c1ba1b0 0x56164c568170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.82:39548/0 (socket says 172.21.15.82:39548) 2024-05-20T16:23:29.980 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:29.979+0000 7fd32a502640 1 -- 172.21.15.82:0/3543201717 learned_addr learned my addr 172.21.15.82:0/3543201717 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:23:29.980 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:29.980+0000 7fd329d01640 1 --2- 172.21.15.82:0/3543201717 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56164c1912f0 0x56164c56b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:23:29.981 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:29.981+0000 7fd32a502640 1 -- 172.21.15.82:0/3543201717 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56164c1912f0 msgr2=0x56164c56b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:23:29.981 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:29.981+0000 7fd32a502640 1 --2- 172.21.15.82:0/3543201717 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56164c1912f0 0x56164c56b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:23:29.981 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:29.981+0000 7fd32a502640 1 -- 172.21.15.82:0/3543201717 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56164c1994d0 msgr2=0x56164c56ba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:23:29.981 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:29.981+0000 7fd32a502640 1 --2- 172.21.15.82:0/3543201717 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56164c1994d0 0x56164c56ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:23:29.981 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:29.981+0000 7fd32a502640 1 -- 172.21.15.82:0/3543201717 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56164c5688d0 con 0x56164c1ba1b0 2024-05-20T16:23:29.983 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:29.983+0000 7fd32a502640 1 --2- 172.21.15.82:0/3543201717 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56164c1ba1b0 0x56164c568170 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fd318009900 tx=0x7fd318004580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9abefd1176b785d5 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:23:29.985 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:29.986+0000 7fd30d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-20T16:23:29.985 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:29.986+0000 7fd30d7fa640 1 -- 172.21.15.82:0/3543201717 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd318035040 con 0x56164c1ba1b0 2024-05-20T16:23:29.986 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:29.986+0000 7fd332e12740 1 -- 172.21.15.82:0/3543201717 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd31c023070 con 0x56164c1ba1b0 2024-05-20T16:23:29.986 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:29.986+0000 7fd30d7fa640 1 -- 172.21.15.82:0/3543201717 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9150+0+0 (secure 0 0 0) 0x7fd3180054d0 con 0x56164c1ba1b0 2024-05-20T16:23:29.991 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:29.991+0000 7fd30d7fa640 1 -- 172.21.15.82:0/3543201717 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fd31804f050 con 0x56164c1ba1b0 2024-05-20T16:23:29.991 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:29.991+0000 7fd332e12740 10 client.25406 fetch_fsmap learned FSMap version 27 2024-05-20T16:23:29.991 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:29.991+0000 7fd332e12740 1 -- 172.21.15.82:0/3543201717 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56164c5d8950 con 0x56164c1ba1b0 2024-05-20T16:23:29.994 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:29.994+0000 7fd30d7fa640 1 -- 172.21.15.82:0/3543201717 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd31c023070 con 0x56164c1ba1b0 2024-05-20T16:23:29.994 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:29.994+0000 7fd30d7fa640 1 -- 172.21.15.82:0/3543201717 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fd318003b90 con 0x56164c1ba1b0 2024-05-20T16:23:29.996 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:29.996+0000 7fd332e12740 10 client.25406 fetch_fsmap finished waiting for FSMap version 27 2024-05-20T16:23:29.996 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:29.996+0000 7fd332e12740 10 client.25406 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T16:23:29.996 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:29.996+0000 7fd332e12740 20 client.25406 populate_metadata read hostname 'smithi082' 2024-05-20T16:23:29.996 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:29.996+0000 7fd332e12740 1 --2- 172.21.15.82:0/3543201717 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x56164c14e1c0 0x56164c5db390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:23:29.997 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:29.996+0000 7fd332e12740 4 client.25406 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-20T16:23:29.997 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:29.996+0000 7fd332e12740 1 -- 172.21.15.82:0/3543201717 --> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56164c4c5dc0 con 0x56164c14e1c0 2024-05-20T16:23:29.998 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:29.998+0000 7fd329d01640 1 --2- 172.21.15.82:0/3543201717 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x56164c14e1c0 0x56164c5db390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:23:30.006 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:30.006+0000 7fd329d01640 1 --2- 172.21.15.82:0/3543201717 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x56164c14e1c0 0x56164c5db390 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fed69830f9ece8b3 server_cookie=6e5498cdaea70d51 in_seq=0 out_seq=0 2024-05-20T16:23:30.008 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:30.009+0000 7fd30d7fa640 10 client.25406 ms_handle_connect on v2:172.21.15.160:6838/2765554482 2024-05-20T16:23:30.135 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:23:29 smithi184 ceph-mon[36209]: pgmap v648: 129 pgs: 129 active+clean; 10 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 793 MiB/s wr, 1.23k op/s 2024-05-20T16:23:30.311 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:23:29 smithi160 ceph-mon[36347]: pgmap v648: 129 pgs: 129 active+clean; 10 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 793 MiB/s wr, 1.23k op/s 2024-05-20T16:23:30.361 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:23:29 smithi082 ceph-mon[32238]: pgmap v648: 129 pgs: 129 active+clean; 10 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 793 MiB/s wr, 1.23k op/s 2024-05-20T16:23:32.544 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:32.544+0000 7fd30d7fa640 1 -- 172.21.15.82:0/3543201717 <== mds.0 v2:172.21.15.160:6838/2765554482 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56164c4c5dc0 con 0x56164c14e1c0 2024-05-20T16:23:32.544 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:32.544+0000 7fd30d7fa640 10 client.25406 handle_command_reply: tid=0 2024-05-20T16:23:32.544 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:32.544+0000 7fd332e12740 1 client.25406 shutdown 2024-05-20T16:23:32.548 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:32.548+0000 7fd332e12740 1 -- 172.21.15.82:0/3543201717 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56164c1ba1b0 msgr2=0x56164c568170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:23:32.548 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:32.548+0000 7fd332e12740 1 --2- 172.21.15.82:0/3543201717 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56164c1ba1b0 0x56164c568170 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fd318009900 tx=0x7fd318004580 comp rx=0 tx=0).stop 2024-05-20T16:23:32.548 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:32.548+0000 7fd332e12740 1 -- 172.21.15.82:0/3543201717 shutdown_connections 2024-05-20T16:23:32.548 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:32.548+0000 7fd332e12740 1 --2- 172.21.15.82:0/3543201717 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x56164c14e1c0 0x56164c5db390 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:23:32.548 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:32.548+0000 7fd332e12740 1 --2- 172.21.15.82:0/3543201717 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56164c1994d0 0x56164c56ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:23:32.548 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:32.548+0000 7fd332e12740 1 --2- 172.21.15.82:0/3543201717 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56164c1ba1b0 0x56164c568170 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:23:32.548 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:32.548+0000 7fd332e12740 1 --2- 172.21.15.82:0/3543201717 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56164c1912f0 0x56164c56b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:23:32.549 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:32.548+0000 7fd332e12740 1 -- 172.21.15.82:0/3543201717 >> 172.21.15.82:0/3543201717 conn(0x56164c1ae300 msgr2=0x56164c5630b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:23:32.549 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:32.548+0000 7fd30d7fa640 0 client.25406 ms_handle_reset on v2:172.21.15.160:6838/2765554482 2024-05-20T16:23:32.549 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:32.549+0000 7fd332e12740 1 -- 172.21.15.82:0/3543201717 shutdown_connections 2024-05-20T16:23:32.549 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:32.549+0000 7fd332e12740 1 -- 172.21.15.82:0/3543201717 wait complete. 2024-05-20T16:23:32.549 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:32.549+0000 7fd332e12740 20 client.25406 trim_cache size 0 max 16384 2024-05-20T16:23:32.549 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:{ 2024-05-20T16:23:32.549 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "message": "", 2024-05-20T16:23:32.549 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "return_code": 0 2024-05-20T16:23:32.549 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:} 2024-05-20T16:23:32.551 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:32.551+0000 7fd32ad03640 1 -- 172.21.15.82:0/2880489991 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fd3100a2c40 msgr2=0x7fd3100a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:23:32.551 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:32.551+0000 7fd32ad03640 1 --2- 172.21.15.82:0/2880489991 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fd3100a2c40 0x7fd3100a5150 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fd32411a050 tx=0x7fd31c006bd0 comp rx=0 tx=0).stop 2024-05-20T16:23:32.551 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:32.551+0000 7fd32ad03640 1 -- 172.21.15.82:0/2880489991 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd3241092f0 msgr2=0x7fd32411a6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:23:32.551 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:32.551+0000 7fd32ad03640 1 --2- 172.21.15.82:0/2880489991 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd3241092f0 0x7fd32411a6b0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fd32000b9c0 tx=0x7fd32000be80 comp rx=0 tx=0).stop 2024-05-20T16:23:32.552 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:32.553+0000 7fd32ad03640 1 -- 172.21.15.82:0/2880489991 shutdown_connections 2024-05-20T16:23:32.553 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:32.553+0000 7fd32ad03640 1 --2- 172.21.15.82:0/2880489991 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fd3100a2c40 0x7fd3100a5150 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:23:32.553 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:32.553+0000 7fd32ad03640 1 --2- 172.21.15.82:0/2880489991 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd32410ee60 0x7fd324118d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:23:32.553 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:32.553+0000 7fd32ad03640 1 --2- 172.21.15.82:0/2880489991 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd32410a140 0x7fd32411abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:23:32.553 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:32.553+0000 7fd32ad03640 1 --2- 172.21.15.82:0/2880489991 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd3241092f0 0x7fd32411a6b0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:23:32.553 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:32.553+0000 7fd32ad03640 1 -- 172.21.15.82:0/2880489991 >> 172.21.15.82:0/2880489991 conn(0x7fd32406f2e0 msgr2=0x7fd32410ce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:23:32.553 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:32.553+0000 7fd32ad03640 1 -- 172.21.15.82:0/2880489991 shutdown_connections 2024-05-20T16:23:32.553 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:32.553+0000 7fd32ad03640 1 -- 172.21.15.82:0/2880489991 wait complete. 2024-05-20T16:23:33.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:23:32 smithi082 ceph-mon[32238]: pgmap v649: 129 pgs: 129 active+clean; 10 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 663 MiB/s wr, 903 op/s 2024-05-20T16:23:33.135 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:23:32 smithi184 ceph-mon[36209]: pgmap v649: 129 pgs: 129 active+clean; 10 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 663 MiB/s wr, 903 op/s 2024-05-20T16:23:33.312 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:23:32 smithi160 ceph-mon[36347]: pgmap v649: 129 pgs: 129 active+clean; 10 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 663 MiB/s wr, 903 op/s 2024-05-20T16:23:35.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:23:34 smithi082 ceph-mon[32238]: pgmap v650: 129 pgs: 129 active+clean; 10 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 846 MiB/s wr, 1.18k op/s 2024-05-20T16:23:35.135 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:23:34 smithi184 ceph-mon[36209]: pgmap v650: 129 pgs: 129 active+clean; 10 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 846 MiB/s wr, 1.18k op/s 2024-05-20T16:23:35.311 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:23:34 smithi160 ceph-mon[36347]: pgmap v650: 129 pgs: 129 active+clean; 10 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 846 MiB/s wr, 1.18k op/s 2024-05-20T16:23:37.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:23:36 smithi082 ceph-mon[32238]: pgmap v651: 129 pgs: 129 active+clean; 10 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 703 MiB/s wr, 1.07k op/s 2024-05-20T16:23:37.135 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:23:36 smithi184 ceph-mon[36209]: pgmap v651: 129 pgs: 129 active+clean; 10 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 703 MiB/s wr, 1.07k op/s 2024-05-20T16:23:37.312 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:23:36 smithi160 ceph-mon[36347]: pgmap v651: 129 pgs: 129 active+clean; 10 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 703 MiB/s wr, 1.07k op/s 2024-05-20T16:23:39.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:23:38 smithi082 ceph-mon[32238]: pgmap v652: 129 pgs: 129 active+clean; 10 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 719 MiB/s wr, 1.19k op/s 2024-05-20T16:23:39.135 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:23:38 smithi184 ceph-mon[36209]: pgmap v652: 129 pgs: 129 active+clean; 10 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 719 MiB/s wr, 1.19k op/s 2024-05-20T16:23:39.312 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:23:38 smithi160 ceph-mon[36347]: pgmap v652: 129 pgs: 129 active+clean; 10 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 719 MiB/s wr, 1.19k op/s 2024-05-20T16:23:41.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:23:40 smithi082 ceph-mon[32238]: pgmap v653: 129 pgs: 129 active+clean; 10 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 795 MiB/s wr, 1.48k op/s 2024-05-20T16:23:41.135 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:23:40 smithi184 ceph-mon[36209]: pgmap v653: 129 pgs: 129 active+clean; 10 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 795 MiB/s wr, 1.48k op/s 2024-05-20T16:23:41.312 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:23:40 smithi160 ceph-mon[36347]: pgmap v653: 129 pgs: 129 active+clean; 10 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 795 MiB/s wr, 1.48k op/s 2024-05-20T16:23:43.112 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:23:42 smithi082 ceph-mon[32238]: pgmap v654: 129 pgs: 129 active+clean; 10 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 664 MiB/s wr, 1.30k op/s 2024-05-20T16:23:43.135 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:23:42 smithi184 ceph-mon[36209]: pgmap v654: 129 pgs: 129 active+clean; 10 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 664 MiB/s wr, 1.30k op/s 2024-05-20T16:23:43.312 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:23:42 smithi160 ceph-mon[36347]: pgmap v654: 129 pgs: 129 active+clean; 10 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 664 MiB/s wr, 1.30k op/s 2024-05-20T16:23:44.312 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:23:43 smithi160 ceph-mon[36347]: pgmap v655: 129 pgs: 129 active+clean; 10 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 843 MiB/s wr, 1.42k op/s 2024-05-20T16:23:44.361 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:23:43 smithi082 ceph-mon[32238]: pgmap v655: 129 pgs: 129 active+clean; 10 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 843 MiB/s wr, 1.42k op/s 2024-05-20T16:23:44.385 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:23:43 smithi184 ceph-mon[36209]: pgmap v655: 129 pgs: 129 active+clean; 10 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 843 MiB/s wr, 1.42k op/s 2024-05-20T16:23:45.829 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:45.829+0000 7fd1e86a2640 1 -- 172.21.15.82:0/413197052 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd1e0104b60 msgr2=0x7fd1e0105860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:23:45.830 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:45.829+0000 7fd1e86a2640 1 --2- 172.21.15.82:0/413197052 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd1e0104b60 0x7fd1e0105860 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fd1d8009ca0 tx=0x7fd1d802a8d0 comp rx=0 tx=0).stop 2024-05-20T16:23:45.840 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:45.840+0000 7fd1e86a2640 1 -- 172.21.15.82:0/413197052 shutdown_connections 2024-05-20T16:23:45.840 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:45.840+0000 7fd1e86a2640 1 --2- 172.21.15.82:0/413197052 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd1e0105da0 0x7fd1e01061b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:23:45.841 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:45.840+0000 7fd1e86a2640 1 --2- 172.21.15.82:0/413197052 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd1e010aaa0 0x7fd1e010cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:23:45.841 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:45.840+0000 7fd1e86a2640 1 --2- 172.21.15.82:0/413197052 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd1e0104b60 0x7fd1e0105860 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:23:45.841 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:45.840+0000 7fd1e86a2640 1 -- 172.21.15.82:0/413197052 >> 172.21.15.82:0/413197052 conn(0x7fd1e00ffd50 msgr2=0x7fd1e0102200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:23:45.853 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:45.850+0000 7fd1e86a2640 1 -- 172.21.15.82:0/413197052 shutdown_connections 2024-05-20T16:23:45.867 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:45.859+0000 7fd1e86a2640 1 -- 172.21.15.82:0/413197052 wait complete. 2024-05-20T16:23:45.867 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:45.859+0000 7fd1e86a2640 1 Processor -- start 2024-05-20T16:23:45.867 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:45.860+0000 7fd1e86a2640 1 -- start start 2024-05-20T16:23:45.867 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:45.864+0000 7fd1e86a2640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd1e0104b60 0x7fd1e006d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:23:45.867 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:45.864+0000 7fd1e86a2640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd1e0105da0 0x7fd1e006d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:23:45.868 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:45.864+0000 7fd1e86a2640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd1e010aaa0 0x7fd1e0072970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:23:45.868 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:45.864+0000 7fd1e86a2640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fd1e006bae0 con 0x7fd1e010aaa0 2024-05-20T16:23:45.868 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:45.864+0000 7fd1e86a2640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fd1e006b960 con 0x7fd1e0104b60 2024-05-20T16:23:45.868 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:45.864+0000 7fd1e86a2640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd1e006bc60 con 0x7fd1e0105da0 2024-05-20T16:23:45.868 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:45.864+0000 7fd1e6c3f640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd1e010aaa0 0x7fd1e0072970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:23:45.868 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:45.864+0000 7fd1e6c3f640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd1e010aaa0 0x7fd1e0072970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:33328/0 (socket says 172.21.15.82:33328) 2024-05-20T16:23:45.868 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:45.864+0000 7fd1e6c3f640 1 -- 172.21.15.82:0/1682829846 learned_addr learned my addr 172.21.15.82:0/1682829846 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:23:45.868 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:45.865+0000 7fd1e643e640 1 --2- 172.21.15.82:0/1682829846 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd1e0104b60 0x7fd1e006d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:23:45.868 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:45.865+0000 7fd1e5c3d640 1 --2- 172.21.15.82:0/1682829846 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd1e0105da0 0x7fd1e006d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:23:45.868 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:45.865+0000 7fd1e6c3f640 1 -- 172.21.15.82:0/1682829846 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd1e0105da0 msgr2=0x7fd1e006d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:23:45.868 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:45.865+0000 7fd1e6c3f640 1 --2- 172.21.15.82:0/1682829846 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd1e0105da0 0x7fd1e006d9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:23:45.868 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:45.865+0000 7fd1e6c3f640 1 -- 172.21.15.82:0/1682829846 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd1e0104b60 msgr2=0x7fd1e006d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:23:45.868 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:45.865+0000 7fd1e6c3f640 1 --2- 172.21.15.82:0/1682829846 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd1e0104b60 0x7fd1e006d430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:23:45.868 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:45.865+0000 7fd1e6c3f640 1 -- 172.21.15.82:0/1682829846 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1e0072fe0 con 0x7fd1e010aaa0 2024-05-20T16:23:45.874 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:45.874+0000 7fd1e6c3f640 1 --2- 172.21.15.82:0/1682829846 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd1e010aaa0 0x7fd1e0072970 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fd1d000c970 tx=0x7fd1d000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:23:45.887 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:45.884+0000 7fd1d77fe640 1 -- 172.21.15.82:0/1682829846 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1d0007bb0 con 0x7fd1e010aaa0 2024-05-20T16:23:45.888 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:45.884+0000 7fd1e86a2640 1 -- 172.21.15.82:0/1682829846 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1e00732d0 con 0x7fd1e010aaa0 2024-05-20T16:23:45.888 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:45.884+0000 7fd1e86a2640 1 -- 172.21.15.82:0/1682829846 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1e00737f0 con 0x7fd1e010aaa0 2024-05-20T16:23:45.888 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:45.884+0000 7fd1d77fe640 1 -- 172.21.15.82:0/1682829846 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd1d0007d50 con 0x7fd1e010aaa0 2024-05-20T16:23:45.888 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:45.884+0000 7fd1d77fe640 1 -- 172.21.15.82:0/1682829846 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1d000f450 con 0x7fd1e010aaa0 2024-05-20T16:23:45.888 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:45.888+0000 7fd1ef623740 1 Processor -- start 2024-05-20T16:23:45.888 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:45.889+0000 7fd1ef623740 1 -- start start 2024-05-20T16:23:45.889 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:45.889+0000 7fd1ef623740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c506d8b1b0 0x55c50712b050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:23:45.889 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:45.889+0000 7fd1ef623740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c506d622f0 0x55c50712b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:23:45.889 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:45.889+0000 7fd1ef623740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c506d6a4d0 0x55c5071324d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:23:45.889 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:45.889+0000 7fd1ef623740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55c50708f340 con 0x55c506d622f0 2024-05-20T16:23:45.889 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:45.889+0000 7fd1ef623740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x55c507037290 con 0x55c506d6a4d0 2024-05-20T16:23:45.889 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:45.889+0000 7fd1ef623740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55c50708b300 con 0x55c506d8b1b0 2024-05-20T16:23:45.890 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:45.890+0000 7fd1e5c3d640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c506d6a4d0 0x55c5071324d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:23:45.890 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:45.890+0000 7fd1e5c3d640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c506d8b1b0 0x55c50712b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:23:45.890 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:45.890+0000 7fd1e5c3d640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c506d8b1b0 0x55c50712b050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.82:39082/0 (socket says 172.21.15.82:39082) 2024-05-20T16:23:45.890 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:45.890+0000 7fd1e5c3d640 1 -- 172.21.15.82:0/235502186 learned_addr learned my addr 172.21.15.82:0/235502186 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:23:45.890 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:45.890+0000 7fd1e5c3d640 1 -- 172.21.15.82:0/235502186 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c506d6a4d0 msgr2=0x55c5071324d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:23:45.891 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:45.890+0000 7fd1e5c3d640 1 --2- 172.21.15.82:0/235502186 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c506d6a4d0 0x55c5071324d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:23:45.891 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:45.890+0000 7fd1e5c3d640 1 -- 172.21.15.82:0/235502186 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c506d622f0 msgr2=0x55c50712b5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:23:45.891 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:45.890+0000 7fd1e5c3d640 1 --2- 172.21.15.82:0/235502186 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c506d622f0 0x55c50712b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:23:45.891 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:45.890+0000 7fd1e5c3d640 1 -- 172.21.15.82:0/235502186 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c50705b8e0 con 0x55c506d8b1b0 2024-05-20T16:23:45.891 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:45.891+0000 7fd1e5c3d640 1 --2- 172.21.15.82:0/235502186 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c506d8b1b0 0x55c50712b050 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fd1dc012770 tx=0x7fd1dc00b390 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a67361fba880bd20 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:23:45.892 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:45.893+0000 7fd1d4bfe640 1 -- 172.21.15.82:0/235502186 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1dc004030 con 0x55c506d8b1b0 2024-05-20T16:23:45.892 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:45.893+0000 7fd1d4bfe640 1 -- 172.21.15.82:0/235502186 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd1dc005500 con 0x55c506d8b1b0 2024-05-20T16:23:45.893 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:45.894+0000 7fd1d77fe640 1 -- 172.21.15.82:0/1682829846 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fd1d000f5f0 con 0x7fd1e010aaa0 2024-05-20T16:23:45.894 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:45.894+0000 7fd1ef623740 1 -- 172.21.15.82:0/235502186 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c506d8b1b0 msgr2=0x55c50712b050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:23:45.894 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:45.894+0000 7fd1ef623740 1 --2- 172.21.15.82:0/235502186 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c506d8b1b0 0x55c50712b050 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fd1dc012770 tx=0x7fd1dc00b390 comp rx=0 tx=0).stop 2024-05-20T16:23:45.894 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:45.895+0000 7fd1d77fe640 1 --2- 172.21.15.82:0/1682829846 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fd1cc0a2dc0 0x7fd1cc0a52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:23:45.894 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:45.895+0000 7fd1d77fe640 1 -- 172.21.15.82:0/1682829846 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9150+0+0 (secure 0 0 0) 0x7fd1d00cef20 con 0x7fd1e010aaa0 2024-05-20T16:23:45.895 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:45.896+0000 7fd1ef623740 1 -- 172.21.15.82:0/235502186 shutdown_connections 2024-05-20T16:23:45.895 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:45.896+0000 7fd1ef623740 1 --2- 172.21.15.82:0/235502186 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c506d622f0 0x55c50712b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:23:45.895 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:45.896+0000 7fd1ef623740 1 --2- 172.21.15.82:0/235502186 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c506d6a4d0 0x55c5071324d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:23:45.895 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:45.896+0000 7fd1ef623740 1 --2- 172.21.15.82:0/235502186 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c506d8b1b0 0x55c50712b050 secure :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fd1dc012770 tx=0x7fd1dc00b390 comp rx=0 tx=0).stop 2024-05-20T16:23:45.895 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:45.896+0000 7fd1ef623740 1 -- 172.21.15.82:0/235502186 >> 172.21.15.82:0/235502186 conn(0x55c506d7f300 msgr2=0x55c5071269e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:23:45.896 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:45.896+0000 7fd1e5c3d640 1 --2- 172.21.15.82:0/1682829846 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fd1cc0a2dc0 0x7fd1cc0a52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:23:45.896 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:45.897+0000 7fd1ef623740 1 -- 172.21.15.82:0/235502186 shutdown_connections 2024-05-20T16:23:45.897 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:45.897+0000 7fd1ef623740 1 -- 172.21.15.82:0/235502186 wait complete. 2024-05-20T16:23:45.897 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:45.898+0000 7fd1e5c3d640 1 --2- 172.21.15.82:0/1682829846 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fd1cc0a2dc0 0x7fd1cc0a52d0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fd1dc011000 tx=0x7fd1dc01b040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:23:45.898 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:45.898+0000 7fd1ef623740 1 -- start start 2024-05-20T16:23:45.898 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:45.899+0000 7fd1ef623740 1 Processor -- start 2024-05-20T16:23:45.902 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:45.902+0000 7fd1ef623740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c506d622f0 0x55c507134e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:23:45.902 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:45.902+0000 7fd1ef623740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c506d6a4d0 0x55c5071353a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:23:45.902 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:45.902+0000 7fd1ef623740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c506d49d80 0x55c50713c300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:23:45.902 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:45.902+0000 7fd1ef623740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55c507132f70 con 0x55c506d6a4d0 2024-05-20T16:23:45.902 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:45.902+0000 7fd1ef623740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x55c50704a790 con 0x55c506d622f0 2024-05-20T16:23:45.902 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:45.902+0000 7fd1ef623740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55c5071330f0 con 0x55c506d49d80 2024-05-20T16:23:45.903 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:45.902+0000 7fd1ef623740 10 client.0 _ll_register_callbacks cb 0x55c507085a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T16:23:45.903 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:45.903+0000 7fd1e6c3f640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c506d49d80 0x55c50713c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:23:45.903 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:45.903+0000 7fd1e6c3f640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c506d49d80 0x55c50713c300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.82:39084/0 (socket says 172.21.15.82:39084) 2024-05-20T16:23:45.903 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:45.903+0000 7fd1e6c3f640 1 -- 172.21.15.82:0/2951843667 learned_addr learned my addr 172.21.15.82:0/2951843667 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:23:45.903 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:45.903+0000 7fd1e643e640 1 --2- 172.21.15.82:0/2951843667 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c506d622f0 0x55c507134e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:23:45.903 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:45.903+0000 7fd1e5c3d640 1 --2- 172.21.15.82:0/2951843667 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c506d6a4d0 0x55c5071353a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:23:45.907 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:45.908+0000 7fd1e643e640 1 -- 172.21.15.82:0/2951843667 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c506d49d80 msgr2=0x55c50713c300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:23:45.907 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:45.908+0000 7fd1e643e640 1 --2- 172.21.15.82:0/2951843667 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c506d49d80 0x55c50713c300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:23:45.908 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:45.908+0000 7fd1e643e640 1 -- 172.21.15.82:0/2951843667 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c506d6a4d0 msgr2=0x55c5071353a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:23:45.908 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:45.908+0000 7fd1e643e640 1 --2- 172.21.15.82:0/2951843667 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c506d6a4d0 0x55c5071353a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:23:45.908 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:45.908+0000 7fd1e643e640 1 -- 172.21.15.82:0/2951843667 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c506da3210 con 0x55c506d622f0 2024-05-20T16:23:45.909 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:45.909+0000 7fd1e643e640 1 --2- 172.21.15.82:0/2951843667 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c506d622f0 0x55c507134e60 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fd1d8009bf0 tx=0x7fd1d8031690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=79eac69ce18185af server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:23:45.909 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:45.909+0000 7fd1c9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-20T16:23:45.909 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:45.909+0000 7fd1c9ffb640 1 -- 172.21.15.82:0/2951843667 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1d8003b70 con 0x55c506d622f0 2024-05-20T16:23:45.909 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:45.909+0000 7fd1ef623740 1 -- 172.21.15.82:0/2951843667 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd1dc005500 con 0x55c506d622f0 2024-05-20T16:23:45.909 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:45.910+0000 7fd1c9ffb640 1 -- 172.21.15.82:0/2951843667 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9150+0+0 (secure 0 0 0) 0x7fd1d8003d10 con 0x55c506d622f0 2024-05-20T16:23:45.910 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:45.910+0000 7fd1c9ffb640 1 -- 172.21.15.82:0/2951843667 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fd1d8031800 con 0x55c506d622f0 2024-05-20T16:23:45.910 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:45.910+0000 7fd1ef623740 10 client.25415 fetch_fsmap learned FSMap version 27 2024-05-20T16:23:45.910 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:45.910+0000 7fd1ef623740 1 -- 172.21.15.82:0/2951843667 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c506e6f620 con 0x55c506d622f0 2024-05-20T16:23:45.910 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:45.911+0000 7fd1c9ffb640 1 -- 172.21.15.82:0/2951843667 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd1dc005500 con 0x55c506d622f0 2024-05-20T16:23:45.911 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:45.911+0000 7fd1c9ffb640 1 -- 172.21.15.82:0/2951843667 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fd1d80308f0 con 0x55c506d622f0 2024-05-20T16:23:45.911 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:45.911+0000 7fd1ef623740 10 client.25415 fetch_fsmap finished waiting for FSMap version 27 2024-05-20T16:23:45.911 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:45.911+0000 7fd1ef623740 10 client.25415 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T16:23:45.911 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:45.911+0000 7fd1ef623740 20 client.25415 populate_metadata read hostname 'smithi082' 2024-05-20T16:23:45.911 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:45.911+0000 7fd1ef623740 1 --2- 172.21.15.82:0/2951843667 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x55c506d1f1c0 0x55c507142c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:23:45.911 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:45.911+0000 7fd1ef623740 4 client.25415 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T16:23:45.911 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:45.911+0000 7fd1ef623740 1 -- 172.21.15.82:0/2951843667 --> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c50709ca70 con 0x55c506d1f1c0 2024-05-20T16:23:45.912 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:45.911+0000 7fd1e5c3d640 1 --2- 172.21.15.82:0/2951843667 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x55c506d1f1c0 0x55c507142c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:23:45.922 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:45.922+0000 7fd1e5c3d640 1 --2- 172.21.15.82:0/2951843667 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x55c506d1f1c0 0x55c507142c90 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=790f9ecbfcc80f64 server_cookie=6c6f0cd8abae378d in_seq=0 out_seq=0 2024-05-20T16:23:45.922 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:45.922+0000 7fd1c9ffb640 10 client.25415 ms_handle_connect on v2:172.21.15.160:6838/2765554482 2024-05-20T16:23:45.925 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:45.925+0000 7fd1c9ffb640 1 -- 172.21.15.82:0/2951843667 <== mds.0 v2:172.21.15.160:6838/2765554482 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55c50709ca70 con 0x55c506d1f1c0 2024-05-20T16:23:45.925 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:45.925+0000 7fd1c9ffb640 10 client.25415 handle_command_reply: tid=0 2024-05-20T16:23:45.925 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:45.925+0000 7fd1ef623740 1 client.25415 shutdown 2024-05-20T16:23:45.927 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:45.927+0000 7fd1ef623740 1 -- 172.21.15.82:0/2951843667 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c506d622f0 msgr2=0x55c507134e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:23:45.927 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:45.927+0000 7fd1ef623740 1 --2- 172.21.15.82:0/2951843667 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c506d622f0 0x55c507134e60 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fd1d8009bf0 tx=0x7fd1d8031690 comp rx=0 tx=0).stop 2024-05-20T16:23:45.933 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:45.933+0000 7fd1ef623740 1 -- 172.21.15.82:0/2951843667 shutdown_connections 2024-05-20T16:23:45.933 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:45.933+0000 7fd1ef623740 1 --2- 172.21.15.82:0/2951843667 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x55c506d1f1c0 0x55c507142c90 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:23:45.933 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:45.933+0000 7fd1ef623740 1 --2- 172.21.15.82:0/2951843667 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c506d6a4d0 0x55c5071353a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:23:45.933 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:45.933+0000 7fd1ef623740 1 --2- 172.21.15.82:0/2951843667 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c506d49d80 0x55c50713c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:23:45.933 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:45.933+0000 7fd1ef623740 1 --2- 172.21.15.82:0/2951843667 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c506d622f0 0x55c507134e60 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:23:45.933 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:45.933+0000 7fd1ef623740 1 -- 172.21.15.82:0/2951843667 >> 172.21.15.82:0/2951843667 conn(0x55c506d7f300 msgr2=0x55c507125ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:23:45.933 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:45.933+0000 7fd1c9ffb640 0 client.25415 ms_handle_reset on v2:172.21.15.160:6838/2765554482 2024-05-20T16:23:45.935 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:45.935+0000 7fd1ef623740 1 -- 172.21.15.82:0/2951843667 shutdown_connections 2024-05-20T16:23:45.935 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:45.935+0000 7fd1ef623740 1 -- 172.21.15.82:0/2951843667 wait complete. 2024-05-20T16:23:45.935 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:45.936+0000 7fd1ef623740 20 client.25415 trim_cache size 0 max 16384 2024-05-20T16:23:45.971 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:45.972+0000 7fd1ef623740 1 Processor -- start 2024-05-20T16:23:45.971 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:45.972+0000 7fd1ef623740 1 -- start start 2024-05-20T16:23:45.972 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:45.972+0000 7fd1ef623740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c506d49d80 0x55c507126400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:23:45.972 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:45.973+0000 7fd1ef623740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c506d622f0 0x55c50713c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:23:45.973 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:45.973+0000 7fd1e643e640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c506d49d80 0x55c507126400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:23:45.973 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:45.973+0000 7fd1e643e640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c506d49d80 0x55c507126400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.82:39092/0 (socket says 172.21.15.82:39092) 2024-05-20T16:23:45.973 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:45.973+0000 7fd1e5c3d640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c506d622f0 0x55c50713c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:23:45.973 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:45.973+0000 7fd1e643e640 1 -- 172.21.15.82:0/3878543176 learned_addr learned my addr 172.21.15.82:0/3878543176 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:23:45.973 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:45.973+0000 7fd1ef623740 1 --2- 172.21.15.82:0/3878543176 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c506d6a4d0 0x55c507126940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:23:45.974 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:45.974+0000 7fd1ef623740 1 -- 172.21.15.82:0/3878543176 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fd1c40125f0 con 0x55c506d6a4d0 2024-05-20T16:23:45.974 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:45.974+0000 7fd1ef623740 1 -- 172.21.15.82:0/3878543176 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fd1c400fec0 con 0x55c506d622f0 2024-05-20T16:23:45.974 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:45.974+0000 7fd1ef623740 1 -- 172.21.15.82:0/3878543176 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd1c400dd10 con 0x55c506d49d80 2024-05-20T16:23:45.974 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:45.975+0000 7fd1e643e640 1 -- 172.21.15.82:0/3878543176 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c506d622f0 msgr2=0x55c50713c2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:23:45.975 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:45.975+0000 7fd1e643e640 1 --2- 172.21.15.82:0/3878543176 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c506d622f0 0x55c50713c2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:23:45.975 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:45.975+0000 7fd1e643e640 1 -- 172.21.15.82:0/3878543176 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c506d6a4d0 msgr2=0x55c507126940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:23:45.975 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:45.975+0000 7fd1e643e640 1 --2- 172.21.15.82:0/3878543176 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c506d6a4d0 0x55c507126940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:23:45.975 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:45.975+0000 7fd1e643e640 1 -- 172.21.15.82:0/3878543176 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c507135a60 con 0x55c506d49d80 2024-05-20T16:23:45.976 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:45.976+0000 7fd1e643e640 1 --2- 172.21.15.82:0/3878543176 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c506d49d80 0x55c507126400 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fd1d80319a0 tx=0x7fd1d802a8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a0fde26df4442e7a server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:23:45.977 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:45.977+0000 7fd1cbfff640 1 -- 172.21.15.82:0/3878543176 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1d8057070 con 0x55c506d49d80 2024-05-20T16:23:45.977 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:45.977+0000 7fd1cbfff640 1 -- 172.21.15.82:0/3878543176 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd1d8036040 con 0x55c506d49d80 2024-05-20T16:23:45.977 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:45.977+0000 7fd1cbfff640 1 -- 172.21.15.82:0/3878543176 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1d8004260 con 0x55c506d49d80 2024-05-20T16:23:45.979 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:45.979+0000 7fd1ef623740 1 -- 172.21.15.82:0/3878543176 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c506d49d80 msgr2=0x55c507126400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:23:45.979 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:45.979+0000 7fd1ef623740 1 --2- 172.21.15.82:0/3878543176 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c506d49d80 0x55c507126400 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fd1d80319a0 tx=0x7fd1d802a8d0 comp rx=0 tx=0).stop 2024-05-20T16:23:45.979 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:45.979+0000 7fd1ef623740 1 -- 172.21.15.82:0/3878543176 shutdown_connections 2024-05-20T16:23:45.979 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:45.979+0000 7fd1ef623740 1 --2- 172.21.15.82:0/3878543176 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c506d6a4d0 0x55c507126940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:23:45.979 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:45.979+0000 7fd1ef623740 1 --2- 172.21.15.82:0/3878543176 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c506d622f0 0x55c50713c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:23:45.979 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:45.979+0000 7fd1ef623740 1 --2- 172.21.15.82:0/3878543176 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c506d49d80 0x55c507126400 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:23:45.979 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:45.979+0000 7fd1ef623740 1 -- 172.21.15.82:0/3878543176 >> 172.21.15.82:0/3878543176 conn(0x55c506d7f300 msgr2=0x55c50712f1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:23:45.979 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:45.979+0000 7fd1ef623740 1 -- 172.21.15.82:0/3878543176 shutdown_connections 2024-05-20T16:23:45.980 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:45.979+0000 7fd1ef623740 1 -- 172.21.15.82:0/3878543176 wait complete. 2024-05-20T16:23:45.980 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:45.980+0000 7fd1ef623740 1 -- start start 2024-05-20T16:23:45.980 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:45.980+0000 7fd1ef623740 1 Processor -- start 2024-05-20T16:23:45.981 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:45.981+0000 7fd1ef623740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c506d49d80 0x55c50713be00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:23:45.981 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:45.981+0000 7fd1ef623740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c506d622f0 0x55c507138a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:23:45.981 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:45.981+0000 7fd1ef623740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c506d6a4d0 0x55c507138f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:23:45.981 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:45.981+0000 7fd1ef623740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fd1c400e380 con 0x55c506d49d80 2024-05-20T16:23:45.981 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:45.981+0000 7fd1ef623740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fd1c400f7f0 con 0x55c506d6a4d0 2024-05-20T16:23:45.981 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:45.981+0000 7fd1ef623740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd1c400ea50 con 0x55c506d622f0 2024-05-20T16:23:45.981 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:45.981+0000 7fd1ef623740 10 client.0 _ll_register_callbacks cb 0x7fd1d8009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T16:23:45.981 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:45.981+0000 7fd1e5c3d640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c506d622f0 0x55c507138a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:23:45.981 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:45.981+0000 7fd1e5c3d640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c506d622f0 0x55c507138a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.82:39094/0 (socket says 172.21.15.82:39094) 2024-05-20T16:23:45.981 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:45.981+0000 7fd1e5c3d640 1 -- 172.21.15.82:0/1797806568 learned_addr learned my addr 172.21.15.82:0/1797806568 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:23:45.981 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:45.981+0000 7fd1e6c3f640 1 --2- 172.21.15.82:0/1797806568 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c506d6a4d0 0x55c507138f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:23:45.983 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:45.982+0000 7fd1e6c3f640 1 -- 172.21.15.82:0/1797806568 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c506d622f0 msgr2=0x55c507138a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:23:45.983 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:45.982+0000 7fd1e6c3f640 1 --2- 172.21.15.82:0/1797806568 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c506d622f0 0x55c507138a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:23:45.983 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:45.982+0000 7fd1e6c3f640 1 -- 172.21.15.82:0/1797806568 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c506d49d80 msgr2=0x55c50713be00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:23:45.983 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:45.982+0000 7fd1e6c3f640 1 --2- 172.21.15.82:0/1797806568 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c506d49d80 0x55c50713be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:23:45.983 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:45.982+0000 7fd1e6c3f640 1 -- 172.21.15.82:0/1797806568 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c50713c4a0 con 0x55c506d6a4d0 2024-05-20T16:23:45.983 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:45.982+0000 7fd1e6c3f640 1 --2- 172.21.15.82:0/1797806568 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c506d6a4d0 0x55c507138f60 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fd1d009b820 tx=0x7fd1d009c0d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c3550073f5d0d662 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:23:45.983 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:45.982+0000 7fd1c9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-20T16:23:45.983 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:45.983+0000 7fd1c9ffb640 1 -- 172.21.15.82:0/1797806568 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1d001d040 con 0x55c506d6a4d0 2024-05-20T16:23:45.983 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:45.983+0000 7fd1ef623740 1 -- 172.21.15.82:0/1797806568 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd1d8036040 con 0x55c506d6a4d0 2024-05-20T16:23:45.983 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:45.983+0000 7fd1c9ffb640 1 -- 172.21.15.82:0/1797806568 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9150+0+0 (secure 0 0 0) 0x7fd1d00b4260 con 0x55c506d6a4d0 2024-05-20T16:23:45.983 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:45.983+0000 7fd1c9ffb640 1 -- 172.21.15.82:0/1797806568 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fd1d0097f00 con 0x55c506d6a4d0 2024-05-20T16:23:45.983 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:45.983+0000 7fd1ef623740 10 client.25421 fetch_fsmap learned FSMap version 27 2024-05-20T16:23:45.984 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:45.983+0000 7fd1ef623740 1 -- 172.21.15.82:0/1797806568 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c5071a9ed0 con 0x55c506d6a4d0 2024-05-20T16:23:45.984 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:45.984+0000 7fd1c9ffb640 1 -- 172.21.15.82:0/1797806568 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd1d8036040 con 0x55c506d6a4d0 2024-05-20T16:23:45.984 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:45.984+0000 7fd1c9ffb640 1 -- 172.21.15.82:0/1797806568 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fd1d009cbc0 con 0x55c506d6a4d0 2024-05-20T16:23:45.984 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:45.984+0000 7fd1ef623740 10 client.25421 fetch_fsmap finished waiting for FSMap version 27 2024-05-20T16:23:45.984 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:45.984+0000 7fd1ef623740 10 client.25421 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T16:23:45.984 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:45.984+0000 7fd1ef623740 20 client.25421 populate_metadata read hostname 'smithi082' 2024-05-20T16:23:45.985 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:45.984+0000 7fd1ef623740 1 --2- 172.21.15.82:0/1797806568 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x55c5071aa860 0x55c5071acca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:23:45.985 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:45.984+0000 7fd1ef623740 4 client.25421 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-20T16:23:45.985 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:45.984+0000 7fd1ef623740 1 -- 172.21.15.82:0/1797806568 --> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c507096dc0 con 0x55c5071aa860 2024-05-20T16:23:45.985 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:45.985+0000 7fd1e643e640 1 --2- 172.21.15.82:0/1797806568 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x55c5071aa860 0x55c5071acca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:23:45.985 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:45.985+0000 7fd1e643e640 1 --2- 172.21.15.82:0/1797806568 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x55c5071aa860 0x55c5071acca0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3421226a0630e969 server_cookie=8e4530fe081480e6 in_seq=0 out_seq=0 2024-05-20T16:23:45.985 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:45.985+0000 7fd1c9ffb640 10 client.25421 ms_handle_connect on v2:172.21.15.160:6838/2765554482 2024-05-20T16:23:47.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:23:46 smithi082 ceph-mon[32238]: pgmap v656: 129 pgs: 129 active+clean; 10 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 694 MiB/s wr, 1.16k op/s 2024-05-20T16:23:47.135 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:23:46 smithi184 ceph-mon[36209]: pgmap v656: 129 pgs: 129 active+clean; 10 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 694 MiB/s wr, 1.16k op/s 2024-05-20T16:23:47.311 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:23:46 smithi160 ceph-mon[36347]: pgmap v656: 129 pgs: 129 active+clean; 10 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 694 MiB/s wr, 1.16k op/s 2024-05-20T16:23:49.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:23:48 smithi082 ceph-mon[32238]: pgmap v657: 129 pgs: 129 active+clean; 10 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 691 MiB/s wr, 1.16k op/s 2024-05-20T16:23:49.135 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:23:48 smithi184 ceph-mon[36209]: pgmap v657: 129 pgs: 129 active+clean; 10 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 691 MiB/s wr, 1.16k op/s 2024-05-20T16:23:49.313 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:23:48 smithi160 ceph-mon[36347]: pgmap v657: 129 pgs: 129 active+clean; 10 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 691 MiB/s wr, 1.16k op/s 2024-05-20T16:23:51.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:23:50 smithi082 ceph-mon[32238]: pgmap v658: 129 pgs: 129 active+clean; 10 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 842 MiB/s wr, 1.44k op/s 2024-05-20T16:23:51.135 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:23:50 smithi184 ceph-mon[36209]: pgmap v658: 129 pgs: 129 active+clean; 10 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 842 MiB/s wr, 1.44k op/s 2024-05-20T16:23:51.184 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:51.184+0000 7fd1c9ffb640 1 -- 172.21.15.82:0/1797806568 <== mds.0 v2:172.21.15.160:6838/2765554482 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c507096dc0 con 0x55c5071aa860 2024-05-20T16:23:51.184 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:51.184+0000 7fd1c9ffb640 10 client.25421 handle_command_reply: tid=0 2024-05-20T16:23:51.184 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:51.184+0000 7fd1ef623740 1 client.25421 shutdown 2024-05-20T16:23:51.184 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:51.184+0000 7fd1ef623740 1 -- 172.21.15.82:0/1797806568 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c506d6a4d0 msgr2=0x55c507138f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:23:51.185 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:51.184+0000 7fd1ef623740 1 --2- 172.21.15.82:0/1797806568 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c506d6a4d0 0x55c507138f60 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fd1d009b820 tx=0x7fd1d009c0d0 comp rx=0 tx=0).stop 2024-05-20T16:23:51.185 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:51.184+0000 7fd1ef623740 1 -- 172.21.15.82:0/1797806568 shutdown_connections 2024-05-20T16:23:51.185 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:51.184+0000 7fd1ef623740 1 --2- 172.21.15.82:0/1797806568 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x55c5071aa860 0x55c5071acca0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:23:51.185 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:51.184+0000 7fd1ef623740 1 --2- 172.21.15.82:0/1797806568 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c506d6a4d0 0x55c507138f60 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:23:51.185 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:51.184+0000 7fd1ef623740 1 --2- 172.21.15.82:0/1797806568 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c506d622f0 0x55c507138a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:23:51.185 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:51.184+0000 7fd1ef623740 1 --2- 172.21.15.82:0/1797806568 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c506d49d80 0x55c50713be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:23:51.185 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:51.184+0000 7fd1ef623740 1 -- 172.21.15.82:0/1797806568 >> 172.21.15.82:0/1797806568 conn(0x55c506d7f300 msgr2=0x55c507126400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:23:51.185 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:51.185+0000 7fd1c9ffb640 0 client.25421 ms_handle_reset on v2:172.21.15.160:6838/2765554482 2024-05-20T16:23:51.185 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:51.185+0000 7fd1ef623740 1 -- 172.21.15.82:0/1797806568 shutdown_connections 2024-05-20T16:23:51.185 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:51.185+0000 7fd1ef623740 1 -- 172.21.15.82:0/1797806568 wait complete. 2024-05-20T16:23:51.186 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:51.185+0000 7fd1ef623740 20 client.25421 trim_cache size 0 max 16384 2024-05-20T16:23:51.186 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:{ 2024-05-20T16:23:51.186 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "message": "", 2024-05-20T16:23:51.186 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "return_code": 0 2024-05-20T16:23:51.186 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:} 2024-05-20T16:23:51.187 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:51.187+0000 7fd1e86a2640 1 -- 172.21.15.82:0/1682829846 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fd1cc0a2dc0 msgr2=0x7fd1cc0a52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:23:51.187 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:51.187+0000 7fd1e86a2640 1 --2- 172.21.15.82:0/1682829846 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fd1cc0a2dc0 0x7fd1cc0a52d0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fd1dc011000 tx=0x7fd1dc01b040 comp rx=0 tx=0).stop 2024-05-20T16:23:51.187 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:51.187+0000 7fd1e86a2640 1 -- 172.21.15.82:0/1682829846 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd1e010aaa0 msgr2=0x7fd1e0072970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:23:51.187 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:51.187+0000 7fd1e86a2640 1 --2- 172.21.15.82:0/1682829846 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd1e010aaa0 0x7fd1e0072970 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fd1d000c970 tx=0x7fd1d000ce30 comp rx=0 tx=0).stop 2024-05-20T16:23:51.187 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:51.187+0000 7fd1e86a2640 1 -- 172.21.15.82:0/1682829846 shutdown_connections 2024-05-20T16:23:51.187 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:51.187+0000 7fd1e86a2640 1 --2- 172.21.15.82:0/1682829846 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fd1cc0a2dc0 0x7fd1cc0a52d0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:23:51.187 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:51.187+0000 7fd1e86a2640 1 --2- 172.21.15.82:0/1682829846 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd1e010aaa0 0x7fd1e0072970 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:23:51.187 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:51.187+0000 7fd1e86a2640 1 --2- 172.21.15.82:0/1682829846 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd1e0105da0 0x7fd1e006d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:23:51.187 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:51.187+0000 7fd1e86a2640 1 --2- 172.21.15.82:0/1682829846 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd1e0104b60 0x7fd1e006d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:23:51.187 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:51.187+0000 7fd1e86a2640 1 -- 172.21.15.82:0/1682829846 >> 172.21.15.82:0/1682829846 conn(0x7fd1e00ffd50 msgr2=0x7fd1e01021e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:23:51.188 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:51.188+0000 7fd1e86a2640 1 -- 172.21.15.82:0/1682829846 shutdown_connections 2024-05-20T16:23:51.188 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:23:51.188+0000 7fd1e86a2640 1 -- 172.21.15.82:0/1682829846 wait complete. 2024-05-20T16:23:51.312 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:23:50 smithi160 ceph-mon[36347]: pgmap v658: 129 pgs: 129 active+clean; 10 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 842 MiB/s wr, 1.44k op/s 2024-05-20T16:23:53.312 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:23:52 smithi160 ceph-mon[36347]: pgmap v659: 129 pgs: 129 active+clean; 10 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 715 MiB/s wr, 1.09k op/s 2024-05-20T16:23:53.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:23:52 smithi082 ceph-mon[32238]: pgmap v659: 129 pgs: 129 active+clean; 10 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 715 MiB/s wr, 1.09k op/s 2024-05-20T16:23:53.385 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:23:52 smithi184 ceph-mon[36209]: pgmap v659: 129 pgs: 129 active+clean; 10 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 715 MiB/s wr, 1.09k op/s 2024-05-20T16:23:54.312 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:23:53 smithi160 ceph-mon[36347]: pgmap v660: 129 pgs: 129 active+clean; 10 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 830 MiB/s wr, 1.23k op/s 2024-05-20T16:23:54.363 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:23:53 smithi082 ceph-mon[32238]: pgmap v660: 129 pgs: 129 active+clean; 10 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 830 MiB/s wr, 1.23k op/s 2024-05-20T16:23:54.385 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:23:53 smithi184 ceph-mon[36209]: pgmap v660: 129 pgs: 129 active+clean; 10 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 830 MiB/s wr, 1.23k op/s 2024-05-20T16:23:57.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:23:56 smithi082 ceph-mon[32238]: pgmap v661: 129 pgs: 129 active+clean; 10 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 697 MiB/s wr, 1.18k op/s 2024-05-20T16:23:57.135 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:23:56 smithi184 ceph-mon[36209]: pgmap v661: 129 pgs: 129 active+clean; 10 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 697 MiB/s wr, 1.18k op/s 2024-05-20T16:23:57.312 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:23:56 smithi160 ceph-mon[36347]: pgmap v661: 129 pgs: 129 active+clean; 10 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 697 MiB/s wr, 1.18k op/s 2024-05-20T16:23:59.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:23:58 smithi082 ceph-mon[32238]: pgmap v662: 129 pgs: 129 active+clean; 10 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 702 MiB/s wr, 1.24k op/s 2024-05-20T16:23:59.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:23:58 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T16:23:59.135 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:23:58 smithi184 ceph-mon[36209]: pgmap v662: 129 pgs: 129 active+clean; 10 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 702 MiB/s wr, 1.24k op/s 2024-05-20T16:23:59.135 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:23:58 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T16:23:59.312 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:23:58 smithi160 ceph-mon[36347]: pgmap v662: 129 pgs: 129 active+clean; 10 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 702 MiB/s wr, 1.24k op/s 2024-05-20T16:23:59.312 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:23:58 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T16:24:01.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:24:00 smithi082 ceph-mon[32238]: pgmap v663: 129 pgs: 129 active+clean; 10 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 830 MiB/s wr, 1.46k op/s 2024-05-20T16:24:01.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:24:00 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:24:01.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:24:00 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T16:24:01.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:24:00 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:24:01.135 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:24:00 smithi184 ceph-mon[36209]: pgmap v663: 129 pgs: 129 active+clean; 10 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 830 MiB/s wr, 1.46k op/s 2024-05-20T16:24:01.136 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:24:00 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:24:01.136 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:24:00 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T16:24:01.136 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:24:00 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:24:01.312 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:24:00 smithi160 ceph-mon[36347]: pgmap v663: 129 pgs: 129 active+clean; 10 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 830 MiB/s wr, 1.46k op/s 2024-05-20T16:24:01.312 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:24:00 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:24:01.312 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:24:00 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T16:24:01.312 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:24:00 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:24:03.135 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:24:02 smithi184 ceph-mon[36209]: pgmap v664: 129 pgs: 129 active+clean; 10 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 680 MiB/s wr, 1.14k op/s 2024-05-20T16:24:03.312 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:24:02 smithi160 ceph-mon[36347]: pgmap v664: 129 pgs: 129 active+clean; 10 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 680 MiB/s wr, 1.14k op/s 2024-05-20T16:24:03.360 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:24:02 smithi082 ceph-mon[32238]: pgmap v664: 129 pgs: 129 active+clean; 10 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 680 MiB/s wr, 1.14k op/s 2024-05-20T16:24:04.474 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:04.473+0000 7fda30c54640 1 -- 172.21.15.82:0/581264297 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fda2c10a140 msgr2=0x7fda2c10a550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:24:04.474 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:04.473+0000 7fda30c54640 1 --2- 172.21.15.82:0/581264297 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fda2c10a140 0x7fda2c10a550 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fda14009860 tx=0x7fda1402a380 comp rx=0 tx=0).stop 2024-05-20T16:24:04.480 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:04.481+0000 7fda30c54640 1 -- 172.21.15.82:0/581264297 shutdown_connections 2024-05-20T16:24:04.481 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:04.481+0000 7fda30c54640 1 --2- 172.21.15.82:0/581264297 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fda2c10ee60 0x7fda2c10f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:24:04.481 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:04.481+0000 7fda30c54640 1 --2- 172.21.15.82:0/581264297 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fda2c10a140 0x7fda2c10a550 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:24:04.481 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:04.481+0000 7fda30c54640 1 --2- 172.21.15.82:0/581264297 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fda2c1092f0 0x7fda2c109c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:24:04.481 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:04.481+0000 7fda30c54640 1 -- 172.21.15.82:0/581264297 >> 172.21.15.82:0/581264297 conn(0x7fda2c06f2e0 msgr2=0x7fda2c071790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:24:04.484 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:04.485+0000 7fda30c54640 1 -- 172.21.15.82:0/581264297 shutdown_connections 2024-05-20T16:24:04.485 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:04.485+0000 7fda30c54640 1 -- 172.21.15.82:0/581264297 wait complete. 2024-05-20T16:24:04.486 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:04.486+0000 7fda30c54640 1 Processor -- start 2024-05-20T16:24:04.486 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:04.487+0000 7fda30c54640 1 -- start start 2024-05-20T16:24:04.487 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:04.487+0000 7fda30c54640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fda2c1092f0 0x7fda2c19f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:24:04.487 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:04.487+0000 7fda30c54640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fda2c10a140 0x7fda2c19fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:24:04.487 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:04.487+0000 7fda30c54640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fda2c10ee60 0x7fda2c1a6bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:24:04.487 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:04.487+0000 7fda30c54640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fda2c112be0 con 0x7fda2c10ee60 2024-05-20T16:24:04.487 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:04.487+0000 7fda30c54640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fda2c112a60 con 0x7fda2c10a140 2024-05-20T16:24:04.487 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:04.487+0000 7fda30c54640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fda2c112d60 con 0x7fda2c1092f0 2024-05-20T16:24:04.488 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:04.488+0000 7fda2b7fe640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fda2c1092f0 0x7fda2c19f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:24:04.488 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:04.488+0000 7fda2affd640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fda2c10a140 0x7fda2c19fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:24:04.488 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:04.488+0000 7fda2b7fe640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fda2c1092f0 0x7fda2c19f720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.82:59628/0 (socket says 172.21.15.82:59628) 2024-05-20T16:24:04.488 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:04.488+0000 7fda2b7fe640 1 -- 172.21.15.82:0/1271922996 learned_addr learned my addr 172.21.15.82:0/1271922996 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:24:04.488 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:04.488+0000 7fda2bfff640 1 --2- 172.21.15.82:0/1271922996 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fda2c10ee60 0x7fda2c1a6bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:24:04.488 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:04.489+0000 7fda2b7fe640 1 -- 172.21.15.82:0/1271922996 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fda2c10a140 msgr2=0x7fda2c19fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:24:04.489 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:04.489+0000 7fda2b7fe640 1 --2- 172.21.15.82:0/1271922996 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fda2c10a140 0x7fda2c19fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:24:04.489 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:04.489+0000 7fda2b7fe640 1 -- 172.21.15.82:0/1271922996 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fda2c10ee60 msgr2=0x7fda2c1a6bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:24:04.489 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:04.489+0000 7fda2b7fe640 1 --2- 172.21.15.82:0/1271922996 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fda2c10ee60 0x7fda2c1a6bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:24:04.489 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:04.489+0000 7fda2b7fe640 1 -- 172.21.15.82:0/1271922996 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda2c1a7260 con 0x7fda2c1092f0 2024-05-20T16:24:04.489 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:04.489+0000 7fda2b7fe640 1 --2- 172.21.15.82:0/1271922996 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fda2c1092f0 0x7fda2c19f720 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fda2400e9f0 tx=0x7fda2400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:24:04.492 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:04.492+0000 7fda28ff9640 1 -- 172.21.15.82:0/1271922996 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda2400ccc0 con 0x7fda2c1092f0 2024-05-20T16:24:04.492 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:04.492+0000 7fda28ff9640 1 -- 172.21.15.82:0/1271922996 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fda24010910 con 0x7fda2c1092f0 2024-05-20T16:24:04.492 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:04.492+0000 7fda28ff9640 1 -- 172.21.15.82:0/1271922996 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda240188e0 con 0x7fda2c1092f0 2024-05-20T16:24:04.492 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:04.492+0000 7fda30c54640 1 -- 172.21.15.82:0/1271922996 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda2c1a7550 con 0x7fda2c1092f0 2024-05-20T16:24:04.492 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:04.492+0000 7fda30c54640 1 -- 172.21.15.82:0/1271922996 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda2c1a7a50 con 0x7fda2c1092f0 2024-05-20T16:24:04.502 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:04.502+0000 7fda38e39740 1 Processor -- start 2024-05-20T16:24:04.502 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:04.502+0000 7fda28ff9640 1 -- 172.21.15.82:0/1271922996 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fda24018a80 con 0x7fda2c1092f0 2024-05-20T16:24:04.504 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:04.504+0000 7fda38e39740 1 -- start start 2024-05-20T16:24:04.504 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:04.504+0000 7fda38e39740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55817deb21b0 0x55817e252050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:24:04.504 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:04.504+0000 7fda38e39740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55817de892f0 0x55817e2525d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:24:04.505 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:04.505+0000 7fda38e39740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55817de914d0 0x55817e2594d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:24:04.505 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:04.506+0000 7fda38e39740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55817e1b6340 con 0x55817de914d0 2024-05-20T16:24:04.505 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:04.506+0000 7fda38e39740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x55817e15e290 con 0x55817deb21b0 2024-05-20T16:24:04.505 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:04.506+0000 7fda38e39740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55817e1b2300 con 0x55817de892f0 2024-05-20T16:24:04.506 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:04.506+0000 7fda2bfff640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55817deb21b0 0x55817e252050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:24:04.506 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:04.506+0000 7fda2bfff640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55817deb21b0 0x55817e252050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.82:50500/0 (socket says 172.21.15.82:50500) 2024-05-20T16:24:04.506 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:04.506+0000 7fda2bfff640 1 -- 172.21.15.82:0/3071145055 learned_addr learned my addr 172.21.15.82:0/3071145055 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:24:04.506 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:04.506+0000 7fda2b7fe640 1 --2- 172.21.15.82:0/3071145055 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55817de892f0 0x55817e2525d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:24:04.507 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:04.506+0000 7fda2affd640 1 --2- 172.21.15.82:0/3071145055 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55817de914d0 0x55817e2594d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:24:04.507 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:04.507+0000 7fda28ff9640 1 --2- 172.21.15.82:0/1271922996 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fda180a2d90 0x7fda180a52a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:24:04.507 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:04.507+0000 7fda28ff9640 1 -- 172.21.15.82:0/1271922996 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9150+0+0 (secure 0 0 0) 0x7fda240d08b0 con 0x7fda2c1092f0 2024-05-20T16:24:04.507 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:04.508+0000 7fda2b7fe640 1 -- 172.21.15.82:0/3071145055 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55817deb21b0 msgr2=0x55817e252050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:24:04.507 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:04.508+0000 7fda2b7fe640 1 --2- 172.21.15.82:0/3071145055 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55817deb21b0 0x55817e252050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:24:04.508 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:04.508+0000 7fda2b7fe640 1 -- 172.21.15.82:0/3071145055 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55817de914d0 msgr2=0x55817e2594d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:24:04.508 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:04.508+0000 7fda2b7fe640 1 --2- 172.21.15.82:0/3071145055 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55817de914d0 0x55817e2594d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:24:04.508 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:04.508+0000 7fda2b7fe640 1 -- 172.21.15.82:0/3071145055 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55817e1828e0 con 0x55817de892f0 2024-05-20T16:24:04.508 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:04.509+0000 7fda2b7fe640 1 --2- 172.21.15.82:0/3071145055 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55817de892f0 0x55817e2525d0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fda2400c6a0 tx=0x7fda24013000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2430dcd5f7220048 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:24:04.512 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:04.512+0000 7fda21ffb640 1 -- 172.21.15.82:0/3071145055 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda2409cd20 con 0x55817de892f0 2024-05-20T16:24:04.512 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:04.512+0000 7fda21ffb640 1 -- 172.21.15.82:0/3071145055 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fda240a10d0 con 0x55817de892f0 2024-05-20T16:24:04.513 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:04.514+0000 7fda38e39740 1 -- 172.21.15.82:0/3071145055 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55817de892f0 msgr2=0x55817e2525d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:24:04.513 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:04.514+0000 7fda38e39740 1 --2- 172.21.15.82:0/3071145055 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55817de892f0 0x55817e2525d0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fda2400c6a0 tx=0x7fda24013000 comp rx=0 tx=0).stop 2024-05-20T16:24:04.515 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:04.515+0000 7fda2bfff640 1 --2- 172.21.15.82:0/1271922996 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fda180a2d90 0x7fda180a52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:24:04.515 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:04.515+0000 7fda38e39740 1 -- 172.21.15.82:0/3071145055 shutdown_connections 2024-05-20T16:24:04.515 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:04.515+0000 7fda38e39740 1 --2- 172.21.15.82:0/3071145055 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55817de914d0 0x55817e2594d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:24:04.515 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:04.515+0000 7fda38e39740 1 --2- 172.21.15.82:0/3071145055 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55817de892f0 0x55817e2525d0 secure :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fda2400c6a0 tx=0x7fda24013000 comp rx=0 tx=0).stop 2024-05-20T16:24:04.515 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:04.515+0000 7fda38e39740 1 --2- 172.21.15.82:0/3071145055 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55817deb21b0 0x55817e252050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:24:04.515 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:04.515+0000 7fda38e39740 1 -- 172.21.15.82:0/3071145055 >> 172.21.15.82:0/3071145055 conn(0x55817dea6300 msgr2=0x55817e24d9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:24:04.517 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:04.518+0000 7fda38e39740 1 -- 172.21.15.82:0/3071145055 shutdown_connections 2024-05-20T16:24:04.517 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:04.518+0000 7fda2bfff640 1 --2- 172.21.15.82:0/1271922996 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fda180a2d90 0x7fda180a52a0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x55817e253220 tx=0x7fda1c004a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:24:04.519 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:04.518+0000 7fda38e39740 1 -- 172.21.15.82:0/3071145055 wait complete. 2024-05-20T16:24:04.519 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:04.519+0000 7fda38e39740 1 -- start start 2024-05-20T16:24:04.519 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:04.519+0000 7fda38e39740 1 Processor -- start 2024-05-20T16:24:04.522 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:04.522+0000 7fda38e39740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55817de914d0 0x55817e25bf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:24:04.522 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:04.522+0000 7fda38e39740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55817deb21b0 0x55817e25c480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:24:04.522 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:04.522+0000 7fda38e39740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55817de70d80 0x55817e263370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:24:04.522 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:04.522+0000 7fda38e39740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55817e259f10 con 0x55817de70d80 2024-05-20T16:24:04.522 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:04.522+0000 7fda38e39740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x55817e171790 con 0x55817deb21b0 2024-05-20T16:24:04.522 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:04.522+0000 7fda38e39740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55817e25a090 con 0x55817de914d0 2024-05-20T16:24:04.522 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:04.522+0000 7fda38e39740 10 client.0 _ll_register_callbacks cb 0x55817e1aca20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T16:24:04.522 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:04.522+0000 7fda2bfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55817de70d80 0x55817e263370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:24:04.523 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:04.522+0000 7fda2bfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55817de70d80 0x55817e263370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:57804/0 (socket says 172.21.15.82:57804) 2024-05-20T16:24:04.523 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:04.522+0000 7fda2bfff640 1 -- 172.21.15.82:0/98983519 learned_addr learned my addr 172.21.15.82:0/98983519 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:24:04.523 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:04.523+0000 7fda2affd640 1 --2- 172.21.15.82:0/98983519 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55817deb21b0 0x55817e25c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:24:04.523 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:04.523+0000 7fda2b7fe640 1 --2- 172.21.15.82:0/98983519 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55817de914d0 0x55817e25bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:24:04.524 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:04.524+0000 7fda2b7fe640 1 -- 172.21.15.82:0/98983519 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55817deb21b0 msgr2=0x55817e25c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:24:04.524 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:04.524+0000 7fda2b7fe640 1 --2- 172.21.15.82:0/98983519 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55817deb21b0 0x55817e25c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:24:04.524 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:04.524+0000 7fda2b7fe640 1 -- 172.21.15.82:0/98983519 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55817de70d80 msgr2=0x55817e263370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:24:04.524 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:04.524+0000 7fda2b7fe640 1 --2- 172.21.15.82:0/98983519 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55817de70d80 0x55817e263370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:24:04.524 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:04.524+0000 7fda2b7fe640 1 -- 172.21.15.82:0/98983519 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55817deca210 con 0x55817de914d0 2024-05-20T16:24:04.525 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:04.525+0000 7fda2b7fe640 1 --2- 172.21.15.82:0/98983519 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55817de914d0 0x55817e25bf40 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x55817e252e50 tx=0x7fda24009660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b2b595c814256850 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:24:04.525 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:04.525+0000 7fda0affd640 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-05-20T16:24:04.529 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:04.530+0000 7fda0affd640 1 -- 172.21.15.82:0/98983519 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda240d0d80 con 0x55817de914d0 2024-05-20T16:24:04.529 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:04.530+0000 7fda0affd640 1 -- 172.21.15.82:0/98983519 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9150+0+0 (secure 0 0 0) 0x7fda24098a70 con 0x55817de914d0 2024-05-20T16:24:04.531 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:04.531+0000 7fda38e39740 1 -- 172.21.15.82:0/98983519 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fda240a10d0 con 0x55817de914d0 2024-05-20T16:24:04.533 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:04.534+0000 7fda0affd640 1 -- 172.21.15.82:0/98983519 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fda24099390 con 0x55817de914d0 2024-05-20T16:24:04.534 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:04.534+0000 7fda38e39740 10 client.25465 fetch_fsmap learned FSMap version 27 2024-05-20T16:24:04.534 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:04.534+0000 7fda38e39740 1 -- 172.21.15.82:0/98983519 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55817df96620 con 0x55817de914d0 2024-05-20T16:24:04.540 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:04.539+0000 7fda0affd640 1 -- 172.21.15.82:0/98983519 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fda240a10d0 con 0x55817de914d0 2024-05-20T16:24:04.540 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:04.539+0000 7fda0affd640 1 -- 172.21.15.82:0/98983519 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fda2409bbe0 con 0x55817de914d0 2024-05-20T16:24:04.540 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:04.541+0000 7fda38e39740 10 client.25465 fetch_fsmap finished waiting for FSMap version 27 2024-05-20T16:24:04.540 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:04.541+0000 7fda38e39740 10 client.25465 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T16:24:04.541 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:04.541+0000 7fda38e39740 20 client.25465 populate_metadata read hostname 'smithi082' 2024-05-20T16:24:04.541 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:04.541+0000 7fda38e39740 1 --2- 172.21.15.82:0/98983519 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x55817de461c0 0x55817e269430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:24:04.541 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:04.541+0000 7fda38e39740 4 client.25465 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T16:24:04.541 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:04.541+0000 7fda38e39740 1 -- 172.21.15.82:0/98983519 --> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55817e1c3a70 con 0x55817de461c0 2024-05-20T16:24:04.545 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:04.546+0000 7fda2affd640 1 --2- 172.21.15.82:0/98983519 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x55817de461c0 0x55817e269430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:24:04.548 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:04.548+0000 7fda2affd640 1 --2- 172.21.15.82:0/98983519 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x55817de461c0 0x55817e269430 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=634016267be15fd server_cookie=484079bde0f0bba2 in_seq=0 out_seq=0 2024-05-20T16:24:04.548 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:04.548+0000 7fda0affd640 10 client.25465 ms_handle_connect on v2:172.21.15.160:6838/2765554482 2024-05-20T16:24:04.553 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:04.554+0000 7fda0affd640 1 -- 172.21.15.82:0/98983519 <== mds.0 v2:172.21.15.160:6838/2765554482 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55817e1c3a70 con 0x55817de461c0 2024-05-20T16:24:04.553 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:04.554+0000 7fda0affd640 10 client.25465 handle_command_reply: tid=0 2024-05-20T16:24:04.557 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:04.554+0000 7fda38e39740 1 client.25465 shutdown 2024-05-20T16:24:04.557 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:04.557+0000 7fda38e39740 1 -- 172.21.15.82:0/98983519 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55817de914d0 msgr2=0x55817e25bf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:24:04.557 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:04.557+0000 7fda38e39740 1 --2- 172.21.15.82:0/98983519 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55817de914d0 0x55817e25bf40 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x55817e252e50 tx=0x7fda24009660 comp rx=0 tx=0).stop 2024-05-20T16:24:04.561 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:04.558+0000 7fda38e39740 1 -- 172.21.15.82:0/98983519 shutdown_connections 2024-05-20T16:24:04.561 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:04.558+0000 7fda38e39740 1 --2- 172.21.15.82:0/98983519 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x55817de461c0 0x55817e269430 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:24:04.561 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:04.558+0000 7fda38e39740 1 --2- 172.21.15.82:0/98983519 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55817de70d80 0x55817e263370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:24:04.561 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:04.558+0000 7fda38e39740 1 --2- 172.21.15.82:0/98983519 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55817deb21b0 0x55817e25c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:24:04.561 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:04.558+0000 7fda38e39740 1 --2- 172.21.15.82:0/98983519 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55817de914d0 0x55817e25bf40 secure :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0x55817e252e50 tx=0x7fda24009660 comp rx=0 tx=0).stop 2024-05-20T16:24:04.562 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:04.558+0000 7fda38e39740 1 -- 172.21.15.82:0/98983519 >> 172.21.15.82:0/98983519 conn(0x55817dea6300 msgr2=0x55817e250990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:24:04.562 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:04.562+0000 7fda0affd640 0 client.25465 ms_handle_reset on v2:172.21.15.160:6838/2765554482 2024-05-20T16:24:04.566 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:04.564+0000 7fda38e39740 1 -- 172.21.15.82:0/98983519 shutdown_connections 2024-05-20T16:24:04.569 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:04.568+0000 7fda38e39740 1 -- 172.21.15.82:0/98983519 wait complete. 2024-05-20T16:24:04.569 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:04.568+0000 7fda38e39740 20 client.25465 trim_cache size 0 max 16384 2024-05-20T16:24:04.606 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:04.607+0000 7fda38e39740 1 Processor -- start 2024-05-20T16:24:04.607 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:04.607+0000 7fda38e39740 1 -- start start 2024-05-20T16:24:04.607 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:04.608+0000 7fda38e39740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55817de70d80 0x55817e25bc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:24:04.607 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:04.608+0000 7fda38e39740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55817deb21b0 0x55817e25c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:24:04.607 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:04.608+0000 7fda38e39740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55817e2512b0 0x55817e263300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:24:04.608 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:04.608+0000 7fda38e39740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fda0c0125f0 con 0x55817e2512b0 2024-05-20T16:24:04.608 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:04.608+0000 7fda38e39740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fda0c00fec0 con 0x55817de70d80 2024-05-20T16:24:04.608 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:04.608+0000 7fda38e39740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fda0c00dd10 con 0x55817deb21b0 2024-05-20T16:24:04.611 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:04.608+0000 7fda2bfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55817e2512b0 0x55817e263300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:24:04.611 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:04.608+0000 7fda2bfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55817e2512b0 0x55817e263300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:57814/0 (socket says 172.21.15.82:57814) 2024-05-20T16:24:04.611 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:04.608+0000 7fda2bfff640 1 -- 172.21.15.82:0/1091401960 learned_addr learned my addr 172.21.15.82:0/1091401960 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:24:04.612 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:04.609+0000 7fda2b7fe640 1 --2- 172.21.15.82:0/1091401960 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55817de70d80 0x55817e25bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:24:04.612 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:04.609+0000 7fda2bfff640 1 -- 172.21.15.82:0/1091401960 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55817deb21b0 msgr2=0x55817e25c1a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-20T16:24:04.612 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:04.609+0000 7fda2bfff640 1 --2- 172.21.15.82:0/1091401960 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55817deb21b0 0x55817e25c1a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:24:04.612 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:04.609+0000 7fda2bfff640 1 -- 172.21.15.82:0/1091401960 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55817de70d80 msgr2=0x55817e25bc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:24:04.612 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:04.609+0000 7fda2bfff640 1 --2- 172.21.15.82:0/1091401960 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55817de70d80 0x55817e25bc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:24:04.612 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:04.609+0000 7fda2bfff640 1 -- 172.21.15.82:0/1091401960 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55817e251c00 con 0x55817e2512b0 2024-05-20T16:24:04.612 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:04.609+0000 7fda2bfff640 1 --2- 172.21.15.82:0/1091401960 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55817e2512b0 0x55817e263300 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fda1c0244d0 tx=0x7fda1c026000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce595be1dfdc997b server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:24:04.612 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:04.610+0000 7fda217fa640 1 -- 172.21.15.82:0/1091401960 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda1c031640 con 0x55817e2512b0 2024-05-20T16:24:04.612 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:04.610+0000 7fda217fa640 1 -- 172.21.15.82:0/1091401960 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fda1c035030 con 0x55817e2512b0 2024-05-20T16:24:04.612 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:04.610+0000 7fda217fa640 1 -- 172.21.15.82:0/1091401960 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda1c030cf0 con 0x55817e2512b0 2024-05-20T16:24:04.612 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:04.611+0000 7fda38e39740 1 -- 172.21.15.82:0/1091401960 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55817e2512b0 msgr2=0x55817e263300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:24:04.612 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:04.611+0000 7fda38e39740 1 --2- 172.21.15.82:0/1091401960 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55817e2512b0 0x55817e263300 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fda1c0244d0 tx=0x7fda1c026000 comp rx=0 tx=0).stop 2024-05-20T16:24:04.612 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:04.611+0000 7fda38e39740 1 -- 172.21.15.82:0/1091401960 shutdown_connections 2024-05-20T16:24:04.613 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:04.611+0000 7fda38e39740 1 --2- 172.21.15.82:0/1091401960 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55817e2512b0 0x55817e263300 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:24:04.613 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:04.611+0000 7fda38e39740 1 --2- 172.21.15.82:0/1091401960 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55817deb21b0 0x55817e25c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:24:04.613 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:04.611+0000 7fda38e39740 1 --2- 172.21.15.82:0/1091401960 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55817de70d80 0x55817e25bc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:24:04.613 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:04.611+0000 7fda38e39740 1 -- 172.21.15.82:0/1091401960 >> 172.21.15.82:0/1091401960 conn(0x55817dea6300 msgr2=0x55817e25b550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:24:04.622 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:04.622+0000 7fda38e39740 1 -- 172.21.15.82:0/1091401960 shutdown_connections 2024-05-20T16:24:04.622 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:04.622+0000 7fda38e39740 1 -- 172.21.15.82:0/1091401960 wait complete. 2024-05-20T16:24:04.622 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:04.623+0000 7fda38e39740 1 -- start start 2024-05-20T16:24:04.623 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:04.623+0000 7fda38e39740 1 Processor -- start 2024-05-20T16:24:04.623 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:04.624+0000 7fda38e39740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55817de70d80 0x55817e2c7e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:24:04.623 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:04.624+0000 7fda38e39740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55817deb21b0 0x55817e2c8350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:24:04.624 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:04.624+0000 7fda38e39740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55817e2cd2a0 0x55817e2cf6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:24:04.624 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:04.625+0000 7fda38e39740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fda0c00e380 con 0x55817deb21b0 2024-05-20T16:24:04.624 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:04.625+0000 7fda38e39740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fda0c00f7f0 con 0x55817de70d80 2024-05-20T16:24:04.625 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:04.625+0000 7fda2bfff640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55817e2cd2a0 0x55817e2cf6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:24:04.625 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:04.625+0000 7fda2bfff640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55817e2cd2a0 0x55817e2cf6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.82:59664/0 (socket says 172.21.15.82:59664) 2024-05-20T16:24:04.625 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:04.625+0000 7fda2bfff640 1 -- 172.21.15.82:0/2354627425 learned_addr learned my addr 172.21.15.82:0/2354627425 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:24:04.625 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:04.625+0000 7fda2affd640 1 --2- 172.21.15.82:0/2354627425 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55817deb21b0 0x55817e2c8350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:24:04.625 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:04.625+0000 7fda38e39740 1 -- 172.21.15.82:0/2354627425 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fda0c00ea50 con 0x55817e2cd2a0 2024-05-20T16:24:04.625 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:04.626+0000 7fda38e39740 10 client.0 _ll_register_callbacks cb 0x7fda14035070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T16:24:04.626 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:04.626+0000 7fda2b7fe640 1 --2- 172.21.15.82:0/2354627425 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55817de70d80 0x55817e2c7e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:24:04.626 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:04.626+0000 7fda2bfff640 1 -- 172.21.15.82:0/2354627425 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55817de70d80 msgr2=0x55817e2c7e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:24:04.626 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:04.626+0000 7fda2bfff640 1 --2- 172.21.15.82:0/2354627425 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55817de70d80 0x55817e2c7e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:24:04.626 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:04.626+0000 7fda2bfff640 1 -- 172.21.15.82:0/2354627425 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55817deb21b0 msgr2=0x55817e2c8350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:24:04.626 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:04.626+0000 7fda2bfff640 1 --2- 172.21.15.82:0/2354627425 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55817deb21b0 0x55817e2c8350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:24:04.626 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:04.626+0000 7fda2bfff640 1 -- 172.21.15.82:0/2354627425 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55817e2cfc80 con 0x55817e2cd2a0 2024-05-20T16:24:04.627 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:04.628+0000 7fda2bfff640 1 --2- 172.21.15.82:0/2354627425 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55817e2cd2a0 0x55817e2cf6c0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fda1c01a830 tx=0x7fda1c004cc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=26ab1b74bc833823 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:24:04.627 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:04.628+0000 7fda0affd640 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-05-20T16:24:04.633 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:04.633+0000 7fda0affd640 1 -- 172.21.15.82:0/2354627425 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda1c048060 con 0x55817e2cd2a0 2024-05-20T16:24:04.633 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:04.633+0000 7fda0affd640 1 -- 172.21.15.82:0/2354627425 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9150+0+0 (secure 0 0 0) 0x7fda1c030c80 con 0x55817e2cd2a0 2024-05-20T16:24:04.636 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:04.636+0000 7fda38e39740 1 -- 172.21.15.82:0/2354627425 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fda1c035030 con 0x55817e2cd2a0 2024-05-20T16:24:04.636 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:04.636+0000 7fda0affd640 1 -- 172.21.15.82:0/2354627425 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fda1c030e80 con 0x55817e2cd2a0 2024-05-20T16:24:04.636 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:04.637+0000 7fda38e39740 10 client.25468 fetch_fsmap learned FSMap version 27 2024-05-20T16:24:04.636 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:04.637+0000 7fda38e39740 1 -- 172.21.15.82:0/2354627425 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55817e2d0ed0 con 0x55817e2cd2a0 2024-05-20T16:24:04.645 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:04.646+0000 7fda0affd640 1 -- 172.21.15.82:0/2354627425 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fda1c035030 con 0x55817e2cd2a0 2024-05-20T16:24:04.645 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:04.646+0000 7fda0affd640 1 -- 172.21.15.82:0/2354627425 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fda1c01f040 con 0x55817e2cd2a0 2024-05-20T16:24:04.647 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:04.647+0000 7fda38e39740 10 client.25468 fetch_fsmap finished waiting for FSMap version 27 2024-05-20T16:24:04.647 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:04.647+0000 7fda38e39740 10 client.25468 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T16:24:04.647 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:04.647+0000 7fda38e39740 20 client.25468 populate_metadata read hostname 'smithi082' 2024-05-20T16:24:04.647 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:04.647+0000 7fda38e39740 1 --2- 172.21.15.82:0/2354627425 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x55817e2d1770 0x55817e2d3b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:24:04.647 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:04.648+0000 7fda38e39740 4 client.25468 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-20T16:24:04.647 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:04.648+0000 7fda38e39740 1 -- 172.21.15.82:0/2354627425 --> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55817e1bddc0 con 0x55817e2d1770 2024-05-20T16:24:04.656 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:04.655+0000 7fda2b7fe640 1 --2- 172.21.15.82:0/2354627425 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x55817e2d1770 0x55817e2d3b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:24:04.666 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:04.666+0000 7fda2b7fe640 1 --2- 172.21.15.82:0/2354627425 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x55817e2d1770 0x55817e2d3b90 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=39560a1b1a0b7aea server_cookie=cf4c3cc8ef27d23a in_seq=0 out_seq=0 2024-05-20T16:24:04.666 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:04.666+0000 7fda0affd640 10 client.25468 ms_handle_connect on v2:172.21.15.160:6838/2765554482 2024-05-20T16:24:05.135 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:24:04 smithi184 ceph-mon[36209]: pgmap v665: 129 pgs: 129 active+clean; 10 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 799 MiB/s wr, 1.29k op/s 2024-05-20T16:24:05.312 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:24:04 smithi160 ceph-mon[36347]: pgmap v665: 129 pgs: 129 active+clean; 10 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 799 MiB/s wr, 1.29k op/s 2024-05-20T16:24:05.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:24:04 smithi082 ceph-mon[32238]: pgmap v665: 129 pgs: 129 active+clean; 10 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 799 MiB/s wr, 1.29k op/s 2024-05-20T16:24:05.905 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:05.905+0000 7fda0affd640 1 -- 172.21.15.82:0/2354627425 <== mds.0 v2:172.21.15.160:6838/2765554482 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55817e1bddc0 con 0x55817e2d1770 2024-05-20T16:24:05.905 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:05.905+0000 7fda0affd640 10 client.25468 handle_command_reply: tid=0 2024-05-20T16:24:05.905 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:05.905+0000 7fda38e39740 1 client.25468 shutdown 2024-05-20T16:24:05.906 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:05.906+0000 7fda38e39740 1 -- 172.21.15.82:0/2354627425 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55817e2cd2a0 msgr2=0x55817e2cf6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:24:05.906 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:05.906+0000 7fda38e39740 1 --2- 172.21.15.82:0/2354627425 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55817e2cd2a0 0x55817e2cf6c0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fda1c01a830 tx=0x7fda1c004cc0 comp rx=0 tx=0).stop 2024-05-20T16:24:05.906 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:05.906+0000 7fda38e39740 1 -- 172.21.15.82:0/2354627425 shutdown_connections 2024-05-20T16:24:05.906 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:05.906+0000 7fda38e39740 1 --2- 172.21.15.82:0/2354627425 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x55817e2d1770 0x55817e2d3b90 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:24:05.906 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:05.906+0000 7fda38e39740 1 --2- 172.21.15.82:0/2354627425 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55817deb21b0 0x55817e2c8350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:24:05.907 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:05.906+0000 7fda38e39740 1 --2- 172.21.15.82:0/2354627425 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55817e2cd2a0 0x55817e2cf6c0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:24:05.907 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:05.906+0000 7fda38e39740 1 --2- 172.21.15.82:0/2354627425 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55817de70d80 0x55817e2c7e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:24:05.907 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:05.906+0000 7fda38e39740 1 -- 172.21.15.82:0/2354627425 >> 172.21.15.82:0/2354627425 conn(0x55817dea6300 msgr2=0x55817e25bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:24:05.907 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:05.906+0000 7fda0affd640 0 client.25468 ms_handle_reset on v2:172.21.15.160:6838/2765554482 2024-05-20T16:24:05.907 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:05.906+0000 7fda38e39740 1 -- 172.21.15.82:0/2354627425 shutdown_connections 2024-05-20T16:24:05.907 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:05.906+0000 7fda38e39740 1 -- 172.21.15.82:0/2354627425 wait complete. 2024-05-20T16:24:05.907 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:05.907+0000 7fda38e39740 20 client.25468 trim_cache size 0 max 16384 2024-05-20T16:24:05.907 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:{ 2024-05-20T16:24:05.907 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "message": "", 2024-05-20T16:24:05.907 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "return_code": 0 2024-05-20T16:24:05.908 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:} 2024-05-20T16:24:05.908 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:05.908+0000 7fda30c54640 1 -- 172.21.15.82:0/1271922996 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fda180a2d90 msgr2=0x7fda180a52a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:24:05.908 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:05.908+0000 7fda30c54640 1 --2- 172.21.15.82:0/1271922996 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fda180a2d90 0x7fda180a52a0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x55817e253220 tx=0x7fda1c004a40 comp rx=0 tx=0).stop 2024-05-20T16:24:05.908 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:05.908+0000 7fda30c54640 1 -- 172.21.15.82:0/1271922996 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fda2c1092f0 msgr2=0x7fda2c19f720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:24:05.908 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:05.908+0000 7fda30c54640 1 --2- 172.21.15.82:0/1271922996 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fda2c1092f0 0x7fda2c19f720 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fda2400e9f0 tx=0x7fda2400eeb0 comp rx=0 tx=0).stop 2024-05-20T16:24:05.909 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:05.908+0000 7fda30c54640 1 -- 172.21.15.82:0/1271922996 shutdown_connections 2024-05-20T16:24:05.909 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:05.908+0000 7fda30c54640 1 --2- 172.21.15.82:0/1271922996 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fda180a2d90 0x7fda180a52a0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:24:05.909 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:05.908+0000 7fda30c54640 1 --2- 172.21.15.82:0/1271922996 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fda2c10ee60 0x7fda2c1a6bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:24:05.909 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:05.908+0000 7fda30c54640 1 --2- 172.21.15.82:0/1271922996 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fda2c10a140 0x7fda2c19fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:24:05.909 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:05.908+0000 7fda30c54640 1 --2- 172.21.15.82:0/1271922996 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fda2c1092f0 0x7fda2c19f720 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:24:05.909 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:05.908+0000 7fda30c54640 1 -- 172.21.15.82:0/1271922996 >> 172.21.15.82:0/1271922996 conn(0x7fda2c06f2e0 msgr2=0x7fda2c071790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:24:05.910 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:05.909+0000 7fda30c54640 1 -- 172.21.15.82:0/1271922996 shutdown_connections 2024-05-20T16:24:05.910 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:05.909+0000 7fda30c54640 1 -- 172.21.15.82:0/1271922996 wait complete. 2024-05-20T16:24:06.636 INFO:journalctl@ceph.mds.c.smithi184.stdout:May 20 16:24:06 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-c[67402]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2744-gd2b507c7/rpm/el8/BUILD/ceph-19.0.0-2744-gd2b507c7/src/mds/MDLog.cc: In function 'void MDLog::_replay_thread()' thread 7f1cb7ccc700 time 2024-05-20T16:24:06.207324+0000 2024-05-20T16:24:06.636 INFO:journalctl@ceph.mds.c.smithi184.stdout:May 20 16:24:06 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-c[67402]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2744-gd2b507c7/rpm/el8/BUILD/ceph-19.0.0-2744-gd2b507c7/src/mds/MDLog.cc: 1417: FAILED ceph_assert(r) 2024-05-20T16:24:06.636 INFO:journalctl@ceph.mds.c.smithi184.stdout:May 20 16:24:06 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-c[67402]: ceph version 19.0.0-2744-gd2b507c7 (d2b507c724ef6c8295ee0a35d185cdab167fd61d) squid (dev) 2024-05-20T16:24:06.636 INFO:journalctl@ceph.mds.c.smithi184.stdout:May 20 16:24:06 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-c[67402]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7f1cca0f2afe] 2024-05-20T16:24:06.636 INFO:journalctl@ceph.mds.c.smithi184.stdout:May 20 16:24:06 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-c[67402]: 2: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f1cca0f2d29] 2024-05-20T16:24:06.636 INFO:journalctl@ceph.mds.c.smithi184.stdout:May 20 16:24:06 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-c[67402]: 3: (MDLog::_replay_thread()+0x97a) [0x5602b4a040de] 2024-05-20T16:24:06.636 INFO:journalctl@ceph.mds.c.smithi184.stdout:May 20 16:24:06 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-c[67402]: 4: (MDLog::ReplayThread::entry()+0x11) [0x5602b4a0eae7] 2024-05-20T16:24:06.636 INFO:journalctl@ceph.mds.c.smithi184.stdout:May 20 16:24:06 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-c[67402]: 5: (Thread::entry_wrapper()+0x43) [0x7f1cca0c16f1] 2024-05-20T16:24:06.637 INFO:journalctl@ceph.mds.c.smithi184.stdout:May 20 16:24:06 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-c[67402]: 6: (Thread::_entry_func(void*)+0xd) [0x7f1cca0c170d] 2024-05-20T16:24:06.637 INFO:journalctl@ceph.mds.c.smithi184.stdout:May 20 16:24:06 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-c[67402]: 7: /lib64/libpthread.so.0(+0x81ca) [0x7f1cc8c661ca] 2024-05-20T16:24:06.637 INFO:journalctl@ceph.mds.c.smithi184.stdout:May 20 16:24:06 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-c[67402]: 8: clone() 2024-05-20T16:24:06.637 INFO:journalctl@ceph.mds.c.smithi184.stdout:May 20 16:24:06 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-c[67402]: 2024-05-20T16:24:06.220+0000 7f1cb7ccc700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2744-gd2b507c7/rpm/el8/BUILD/ceph-19.0.0-2744-gd2b507c7/src/mds/MDLog.cc: In function 'void MDLog::_replay_thread()' thread 7f1cb7ccc700 time 2024-05-20T16:24:06.207324+0000 2024-05-20T16:24:06.637 INFO:journalctl@ceph.mds.c.smithi184.stdout:May 20 16:24:06 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-c[67402]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2744-gd2b507c7/rpm/el8/BUILD/ceph-19.0.0-2744-gd2b507c7/src/mds/MDLog.cc: 1417: FAILED ceph_assert(r) 2024-05-20T16:24:06.637 INFO:journalctl@ceph.mds.c.smithi184.stdout:May 20 16:24:06 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-c[67402]: 2024-05-20T16:24:06.637 INFO:journalctl@ceph.mds.c.smithi184.stdout:May 20 16:24:06 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-c[67402]: ceph version 19.0.0-2744-gd2b507c7 (d2b507c724ef6c8295ee0a35d185cdab167fd61d) squid (dev) 2024-05-20T16:24:06.637 INFO:journalctl@ceph.mds.c.smithi184.stdout:May 20 16:24:06 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-c[67402]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7f1cca0f2afe] 2024-05-20T16:24:06.637 INFO:journalctl@ceph.mds.c.smithi184.stdout:May 20 16:24:06 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-c[67402]: 2: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f1cca0f2d29] 2024-05-20T16:24:06.637 INFO:journalctl@ceph.mds.c.smithi184.stdout:May 20 16:24:06 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-c[67402]: 3: (MDLog::_replay_thread()+0x97a) [0x5602b4a040de] 2024-05-20T16:24:06.637 INFO:journalctl@ceph.mds.c.smithi184.stdout:May 20 16:24:06 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-c[67402]: 4: (MDLog::ReplayThread::entry()+0x11) [0x5602b4a0eae7] 2024-05-20T16:24:06.637 INFO:journalctl@ceph.mds.c.smithi184.stdout:May 20 16:24:06 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-c[67402]: 5: (Thread::entry_wrapper()+0x43) [0x7f1cca0c16f1] 2024-05-20T16:24:06.637 INFO:journalctl@ceph.mds.c.smithi184.stdout:May 20 16:24:06 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-c[67402]: 6: (Thread::_entry_func(void*)+0xd) [0x7f1cca0c170d] 2024-05-20T16:24:06.638 INFO:journalctl@ceph.mds.c.smithi184.stdout:May 20 16:24:06 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-c[67402]: 7: /lib64/libpthread.so.0(+0x81ca) [0x7f1cc8c661ca] 2024-05-20T16:24:06.638 INFO:journalctl@ceph.mds.c.smithi184.stdout:May 20 16:24:06 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-c[67402]: 8: clone() 2024-05-20T16:24:06.638 INFO:journalctl@ceph.mds.c.smithi184.stdout:May 20 16:24:06 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-c[67402]: 2024-05-20T16:24:06.639 INFO:journalctl@ceph.mds.c.smithi184.stdout:May 20 16:24:06 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-c[67402]: *** Caught signal (Aborted) ** 2024-05-20T16:24:06.639 INFO:journalctl@ceph.mds.c.smithi184.stdout:May 20 16:24:06 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-c[67402]: in thread 7f1cb7ccc700 thread_name:md_log_replay 2024-05-20T16:24:06.639 INFO:journalctl@ceph.mds.c.smithi184.stdout:May 20 16:24:06 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-c[67402]: ceph version 19.0.0-2744-gd2b507c7 (d2b507c724ef6c8295ee0a35d185cdab167fd61d) squid (dev) 2024-05-20T16:24:06.639 INFO:journalctl@ceph.mds.c.smithi184.stdout:May 20 16:24:06 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-c[67402]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f1cc8c70d20] 2024-05-20T16:24:06.639 INFO:journalctl@ceph.mds.c.smithi184.stdout:May 20 16:24:06 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-c[67402]: 2: gsignal() 2024-05-20T16:24:06.639 INFO:journalctl@ceph.mds.c.smithi184.stdout:May 20 16:24:06 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-c[67402]: 3: abort() 2024-05-20T16:24:06.639 INFO:journalctl@ceph.mds.c.smithi184.stdout:May 20 16:24:06 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-c[67402]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x26e) [0x7f1cca0f2c45] 2024-05-20T16:24:06.639 INFO:journalctl@ceph.mds.c.smithi184.stdout:May 20 16:24:06 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-c[67402]: 5: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f1cca0f2d29] 2024-05-20T16:24:06.639 INFO:journalctl@ceph.mds.c.smithi184.stdout:May 20 16:24:06 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-c[67402]: 6: (MDLog::_replay_thread()+0x97a) [0x5602b4a040de] 2024-05-20T16:24:06.639 INFO:journalctl@ceph.mds.c.smithi184.stdout:May 20 16:24:06 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-c[67402]: 7: (MDLog::ReplayThread::entry()+0x11) [0x5602b4a0eae7] 2024-05-20T16:24:06.639 INFO:journalctl@ceph.mds.c.smithi184.stdout:May 20 16:24:06 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-c[67402]: 8: (Thread::entry_wrapper()+0x43) [0x7f1cca0c16f1] 2024-05-20T16:24:06.639 INFO:journalctl@ceph.mds.c.smithi184.stdout:May 20 16:24:06 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-c[67402]: 9: (Thread::_entry_func(void*)+0xd) [0x7f1cca0c170d] 2024-05-20T16:24:06.640 INFO:journalctl@ceph.mds.c.smithi184.stdout:May 20 16:24:06 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-c[67402]: 10: /lib64/libpthread.so.0(+0x81ca) [0x7f1cc8c661ca] 2024-05-20T16:24:06.640 INFO:journalctl@ceph.mds.c.smithi184.stdout:May 20 16:24:06 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-c[67402]: 11: clone() 2024-05-20T16:24:06.640 INFO:journalctl@ceph.mds.c.smithi184.stdout:May 20 16:24:06 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-c[67402]: 2024-05-20T16:24:06.221+0000 7f1cb7ccc700 -1 *** Caught signal (Aborted) ** 2024-05-20T16:24:06.640 INFO:journalctl@ceph.mds.c.smithi184.stdout:May 20 16:24:06 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-c[67402]: in thread 7f1cb7ccc700 thread_name:md_log_replay 2024-05-20T16:24:06.640 INFO:journalctl@ceph.mds.c.smithi184.stdout:May 20 16:24:06 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-c[67402]: 2024-05-20T16:24:06.640 INFO:journalctl@ceph.mds.c.smithi184.stdout:May 20 16:24:06 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-c[67402]: ceph version 19.0.0-2744-gd2b507c7 (d2b507c724ef6c8295ee0a35d185cdab167fd61d) squid (dev) 2024-05-20T16:24:06.640 INFO:journalctl@ceph.mds.c.smithi184.stdout:May 20 16:24:06 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-c[67402]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f1cc8c70d20] 2024-05-20T16:24:06.640 INFO:journalctl@ceph.mds.c.smithi184.stdout:May 20 16:24:06 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-c[67402]: 2: gsignal() 2024-05-20T16:24:06.640 INFO:journalctl@ceph.mds.c.smithi184.stdout:May 20 16:24:06 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-c[67402]: 3: abort() 2024-05-20T16:24:06.640 INFO:journalctl@ceph.mds.c.smithi184.stdout:May 20 16:24:06 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-c[67402]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x26e) [0x7f1cca0f2c45] 2024-05-20T16:24:06.640 INFO:journalctl@ceph.mds.c.smithi184.stdout:May 20 16:24:06 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-c[67402]: 5: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f1cca0f2d29] 2024-05-20T16:24:06.640 INFO:journalctl@ceph.mds.c.smithi184.stdout:May 20 16:24:06 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-c[67402]: 6: (MDLog::_replay_thread()+0x97a) [0x5602b4a040de] 2024-05-20T16:24:06.641 INFO:journalctl@ceph.mds.c.smithi184.stdout:May 20 16:24:06 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-c[67402]: 7: (MDLog::ReplayThread::entry()+0x11) [0x5602b4a0eae7] 2024-05-20T16:24:06.641 INFO:journalctl@ceph.mds.c.smithi184.stdout:May 20 16:24:06 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-c[67402]: 8: (Thread::entry_wrapper()+0x43) [0x7f1cca0c16f1] 2024-05-20T16:24:06.641 INFO:journalctl@ceph.mds.c.smithi184.stdout:May 20 16:24:06 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-c[67402]: 9: (Thread::_entry_func(void*)+0xd) [0x7f1cca0c170d] 2024-05-20T16:24:06.641 INFO:journalctl@ceph.mds.c.smithi184.stdout:May 20 16:24:06 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-c[67402]: 10: /lib64/libpthread.so.0(+0x81ca) [0x7f1cc8c661ca] 2024-05-20T16:24:06.641 INFO:journalctl@ceph.mds.c.smithi184.stdout:May 20 16:24:06 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-c[67402]: 11: clone() 2024-05-20T16:24:06.641 INFO:journalctl@ceph.mds.c.smithi184.stdout:May 20 16:24:06 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-c[67402]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-05-20T16:24:06.641 INFO:journalctl@ceph.mds.c.smithi184.stdout:May 20 16:24:06 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-c[67402]: 2024-05-20T16:24:06.641 INFO:journalctl@ceph.mds.c.smithi184.stdout:May 20 16:24:06 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-c[67402]: -1> 2024-05-20T16:24:06.220+0000 7f1cb7ccc700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2744-gd2b507c7/rpm/el8/BUILD/ceph-19.0.0-2744-gd2b507c7/src/mds/MDLog.cc: In function 'void MDLog::_replay_thread()' thread 7f1cb7ccc700 time 2024-05-20T16:24:06.207324+0000 2024-05-20T16:24:06.641 INFO:journalctl@ceph.mds.c.smithi184.stdout:May 20 16:24:06 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-c[67402]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2744-gd2b507c7/rpm/el8/BUILD/ceph-19.0.0-2744-gd2b507c7/src/mds/MDLog.cc: 1417: FAILED ceph_assert(r) 2024-05-20T16:24:06.641 INFO:journalctl@ceph.mds.c.smithi184.stdout:May 20 16:24:06 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-c[67402]: 2024-05-20T16:24:06.641 INFO:journalctl@ceph.mds.c.smithi184.stdout:May 20 16:24:06 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-c[67402]: ceph version 19.0.0-2744-gd2b507c7 (d2b507c724ef6c8295ee0a35d185cdab167fd61d) squid (dev) 2024-05-20T16:24:06.641 INFO:journalctl@ceph.mds.c.smithi184.stdout:May 20 16:24:06 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-c[67402]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7f1cca0f2afe] 2024-05-20T16:24:06.641 INFO:journalctl@ceph.mds.c.smithi184.stdout:May 20 16:24:06 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-c[67402]: 2: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f1cca0f2d29] 2024-05-20T16:24:06.641 INFO:journalctl@ceph.mds.c.smithi184.stdout:May 20 16:24:06 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-c[67402]: 3: (MDLog::_replay_thread()+0x97a) [0x5602b4a040de] 2024-05-20T16:24:06.641 INFO:journalctl@ceph.mds.c.smithi184.stdout:May 20 16:24:06 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-c[67402]: 4: (MDLog::ReplayThread::entry()+0x11) [0x5602b4a0eae7] 2024-05-20T16:24:06.642 INFO:journalctl@ceph.mds.c.smithi184.stdout:May 20 16:24:06 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-c[67402]: 5: (Thread::entry_wrapper()+0x43) [0x7f1cca0c16f1] 2024-05-20T16:24:06.642 INFO:journalctl@ceph.mds.c.smithi184.stdout:May 20 16:24:06 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-c[67402]: 6: (Thread::_entry_func(void*)+0xd) [0x7f1cca0c170d] 2024-05-20T16:24:06.642 INFO:journalctl@ceph.mds.c.smithi184.stdout:May 20 16:24:06 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-c[67402]: 7: /lib64/libpthread.so.0(+0x81ca) [0x7f1cc8c661ca] 2024-05-20T16:24:06.642 INFO:journalctl@ceph.mds.c.smithi184.stdout:May 20 16:24:06 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-c[67402]: 8: clone() 2024-05-20T16:24:06.642 INFO:journalctl@ceph.mds.c.smithi184.stdout:May 20 16:24:06 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-c[67402]: 2024-05-20T16:24:06.642 INFO:journalctl@ceph.mds.c.smithi184.stdout:May 20 16:24:06 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-c[67402]: 0> 2024-05-20T16:24:06.221+0000 7f1cb7ccc700 -1 *** Caught signal (Aborted) ** 2024-05-20T16:24:06.642 INFO:journalctl@ceph.mds.c.smithi184.stdout:May 20 16:24:06 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-c[67402]: in thread 7f1cb7ccc700 thread_name:md_log_replay 2024-05-20T16:24:06.642 INFO:journalctl@ceph.mds.c.smithi184.stdout:May 20 16:24:06 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-c[67402]: 2024-05-20T16:24:06.642 INFO:journalctl@ceph.mds.c.smithi184.stdout:May 20 16:24:06 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-c[67402]: ceph version 19.0.0-2744-gd2b507c7 (d2b507c724ef6c8295ee0a35d185cdab167fd61d) squid (dev) 2024-05-20T16:24:06.642 INFO:journalctl@ceph.mds.c.smithi184.stdout:May 20 16:24:06 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-c[67402]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f1cc8c70d20] 2024-05-20T16:24:06.642 INFO:journalctl@ceph.mds.c.smithi184.stdout:May 20 16:24:06 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-c[67402]: 2: gsignal() 2024-05-20T16:24:06.642 INFO:journalctl@ceph.mds.c.smithi184.stdout:May 20 16:24:06 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-c[67402]: 3: abort() 2024-05-20T16:24:06.642 INFO:journalctl@ceph.mds.c.smithi184.stdout:May 20 16:24:06 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-c[67402]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x26e) [0x7f1cca0f2c45] 2024-05-20T16:24:06.642 INFO:journalctl@ceph.mds.c.smithi184.stdout:May 20 16:24:06 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-c[67402]: 5: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f1cca0f2d29] 2024-05-20T16:24:06.643 INFO:journalctl@ceph.mds.c.smithi184.stdout:May 20 16:24:06 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-c[67402]: 6: (MDLog::_replay_thread()+0x97a) [0x5602b4a040de] 2024-05-20T16:24:06.643 INFO:journalctl@ceph.mds.c.smithi184.stdout:May 20 16:24:06 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-c[67402]: 7: (MDLog::ReplayThread::entry()+0x11) [0x5602b4a0eae7] 2024-05-20T16:24:06.643 INFO:journalctl@ceph.mds.c.smithi184.stdout:May 20 16:24:06 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-c[67402]: 8: (Thread::entry_wrapper()+0x43) [0x7f1cca0c16f1] 2024-05-20T16:24:06.643 INFO:journalctl@ceph.mds.c.smithi184.stdout:May 20 16:24:06 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-c[67402]: 9: (Thread::_entry_func(void*)+0xd) [0x7f1cca0c170d] 2024-05-20T16:24:06.643 INFO:journalctl@ceph.mds.c.smithi184.stdout:May 20 16:24:06 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-c[67402]: 10: /lib64/libpthread.so.0(+0x81ca) [0x7f1cc8c661ca] 2024-05-20T16:24:06.643 INFO:journalctl@ceph.mds.c.smithi184.stdout:May 20 16:24:06 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-c[67402]: 11: clone() 2024-05-20T16:24:06.643 INFO:journalctl@ceph.mds.c.smithi184.stdout:May 20 16:24:06 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-c[67402]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-05-20T16:24:06.643 INFO:journalctl@ceph.mds.c.smithi184.stdout:May 20 16:24:06 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-c[67402]: 2024-05-20T16:24:06.643 INFO:journalctl@ceph.mds.c.smithi184.stdout:May 20 16:24:06 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-c[67402]: -1> 2024-05-20T16:24:06.220+0000 7f1cb7ccc700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2744-gd2b507c7/rpm/el8/BUILD/ceph-19.0.0-2744-gd2b507c7/src/mds/MDLog.cc: In function 'void MDLog::_replay_thread()' thread 7f1cb7ccc700 time 2024-05-20T16:24:06.207324+0000 2024-05-20T16:24:06.643 INFO:journalctl@ceph.mds.c.smithi184.stdout:May 20 16:24:06 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-c[67402]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2744-gd2b507c7/rpm/el8/BUILD/ceph-19.0.0-2744-gd2b507c7/src/mds/MDLog.cc: 1417: FAILED ceph_assert(r) 2024-05-20T16:24:06.643 INFO:journalctl@ceph.mds.c.smithi184.stdout:May 20 16:24:06 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-c[67402]: 2024-05-20T16:24:06.643 INFO:journalctl@ceph.mds.c.smithi184.stdout:May 20 16:24:06 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-c[67402]: ceph version 19.0.0-2744-gd2b507c7 (d2b507c724ef6c8295ee0a35d185cdab167fd61d) squid (dev) 2024-05-20T16:24:06.644 INFO:journalctl@ceph.mds.c.smithi184.stdout:May 20 16:24:06 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-c[67402]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7f1cca0f2afe] 2024-05-20T16:24:06.644 INFO:journalctl@ceph.mds.c.smithi184.stdout:May 20 16:24:06 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-c[67402]: 2: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f1cca0f2d29] 2024-05-20T16:24:06.644 INFO:journalctl@ceph.mds.c.smithi184.stdout:May 20 16:24:06 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-c[67402]: 3: (MDLog::_replay_thread()+0x97a) [0x5602b4a040de] 2024-05-20T16:24:06.644 INFO:journalctl@ceph.mds.c.smithi184.stdout:May 20 16:24:06 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-c[67402]: 4: (MDLog::ReplayThread::entry()+0x11) [0x5602b4a0eae7] 2024-05-20T16:24:06.644 INFO:journalctl@ceph.mds.c.smithi184.stdout:May 20 16:24:06 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-c[67402]: 5: (Thread::entry_wrapper()+0x43) [0x7f1cca0c16f1] 2024-05-20T16:24:06.644 INFO:journalctl@ceph.mds.c.smithi184.stdout:May 20 16:24:06 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-c[67402]: 6: (Thread::_entry_func(void*)+0xd) [0x7f1cca0c170d] 2024-05-20T16:24:06.644 INFO:journalctl@ceph.mds.c.smithi184.stdout:May 20 16:24:06 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-c[67402]: 7: /lib64/libpthread.so.0(+0x81ca) [0x7f1cc8c661ca] 2024-05-20T16:24:06.644 INFO:journalctl@ceph.mds.c.smithi184.stdout:May 20 16:24:06 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-c[67402]: 8: clone() 2024-05-20T16:24:06.644 INFO:journalctl@ceph.mds.c.smithi184.stdout:May 20 16:24:06 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-c[67402]: 2024-05-20T16:24:06.644 INFO:journalctl@ceph.mds.c.smithi184.stdout:May 20 16:24:06 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-c[67402]: 0> 2024-05-20T16:24:06.221+0000 7f1cb7ccc700 -1 *** Caught signal (Aborted) ** 2024-05-20T16:24:06.644 INFO:journalctl@ceph.mds.c.smithi184.stdout:May 20 16:24:06 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-c[67402]: in thread 7f1cb7ccc700 thread_name:md_log_replay 2024-05-20T16:24:06.644 INFO:journalctl@ceph.mds.c.smithi184.stdout:May 20 16:24:06 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-c[67402]: 2024-05-20T16:24:06.644 INFO:journalctl@ceph.mds.c.smithi184.stdout:May 20 16:24:06 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-c[67402]: ceph version 19.0.0-2744-gd2b507c7 (d2b507c724ef6c8295ee0a35d185cdab167fd61d) squid (dev) 2024-05-20T16:24:06.644 INFO:journalctl@ceph.mds.c.smithi184.stdout:May 20 16:24:06 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-c[67402]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f1cc8c70d20] 2024-05-20T16:24:06.645 INFO:journalctl@ceph.mds.c.smithi184.stdout:May 20 16:24:06 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-c[67402]: 2: gsignal() 2024-05-20T16:24:06.645 INFO:journalctl@ceph.mds.c.smithi184.stdout:May 20 16:24:06 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-c[67402]: 3: abort() 2024-05-20T16:24:06.645 INFO:journalctl@ceph.mds.c.smithi184.stdout:May 20 16:24:06 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-c[67402]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x26e) [0x7f1cca0f2c45] 2024-05-20T16:24:06.645 INFO:journalctl@ceph.mds.c.smithi184.stdout:May 20 16:24:06 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-c[67402]: 5: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f1cca0f2d29] 2024-05-20T16:24:06.645 INFO:journalctl@ceph.mds.c.smithi184.stdout:May 20 16:24:06 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-c[67402]: 6: (MDLog::_replay_thread()+0x97a) [0x5602b4a040de] 2024-05-20T16:24:06.645 INFO:journalctl@ceph.mds.c.smithi184.stdout:May 20 16:24:06 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-c[67402]: 7: (MDLog::ReplayThread::entry()+0x11) [0x5602b4a0eae7] 2024-05-20T16:24:06.645 INFO:journalctl@ceph.mds.c.smithi184.stdout:May 20 16:24:06 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-c[67402]: 8: (Thread::entry_wrapper()+0x43) [0x7f1cca0c16f1] 2024-05-20T16:24:06.645 INFO:journalctl@ceph.mds.c.smithi184.stdout:May 20 16:24:06 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-c[67402]: 9: (Thread::_entry_func(void*)+0xd) [0x7f1cca0c170d] 2024-05-20T16:24:06.645 INFO:journalctl@ceph.mds.c.smithi184.stdout:May 20 16:24:06 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-c[67402]: 10: /lib64/libpthread.so.0(+0x81ca) [0x7f1cc8c661ca] 2024-05-20T16:24:06.645 INFO:journalctl@ceph.mds.c.smithi184.stdout:May 20 16:24:06 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-c[67402]: 11: clone() 2024-05-20T16:24:06.645 INFO:journalctl@ceph.mds.c.smithi184.stdout:May 20 16:24:06 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-c[67402]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-05-20T16:24:06.645 INFO:journalctl@ceph.mds.c.smithi184.stdout:May 20 16:24:06 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-c[67402]: 2024-05-20T16:24:06.645 INFO:journalctl@ceph.mds.c.smithi184.stdout:May 20 16:24:06 smithi184 podman[71146]: 2024-05-20 16:24:06.300594555 +0000 UTC m=+0.022617984 container died aa07be6b721540c695bab239d736767cd47bec50b4734f652b22c78a9c9c9d29 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-c, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240514, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, io.buildah.version=1.33.5) 2024-05-20T16:24:07.103 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:24:06 smithi184 ceph-mon[36209]: pgmap v666: 129 pgs: 129 active+clean; 10 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 721 MiB/s wr, 1.17k op/s 2024-05-20T16:24:07.311 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:24:06 smithi160 ceph-mon[36347]: pgmap v666: 129 pgs: 129 active+clean; 10 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 721 MiB/s wr, 1.17k op/s 2024-05-20T16:24:07.360 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:24:06 smithi082 ceph-mon[32238]: pgmap v666: 129 pgs: 129 active+clean; 10 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 721 MiB/s wr, 1.17k op/s 2024-05-20T16:24:07.386 INFO:journalctl@ceph.mds.c.smithi184.stdout:May 20 16:24:07 smithi184 podman[71146]: 2024-05-20 16:24:07.104518041 +0000 UTC m=+0.826541473 container remove aa07be6b721540c695bab239d736767cd47bec50b4734f652b22c78a9c9c9d29 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-c, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240514, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-05-20T16:24:07.386 INFO:journalctl@ceph.mds.c.smithi184.stdout:May 20 16:24:07 smithi184 systemd[1]: ceph-59d1600a-16c2-11ef-bc9a-c7b262605968@mds.c.service: Main process exited, code=exited, status=134/n/a 2024-05-20T16:24:07.885 INFO:journalctl@ceph.mds.c.smithi184.stdout:May 20 16:24:07 smithi184 systemd[1]: ceph-59d1600a-16c2-11ef-bc9a-c7b262605968@mds.c.service: Failed with result 'exit-code'. 2024-05-20T16:24:07.885 INFO:journalctl@ceph.mds.c.smithi184.stdout:May 20 16:24:07 smithi184 systemd[1]: ceph-59d1600a-16c2-11ef-bc9a-c7b262605968@mds.c.service: Consumed 1min 10.259s CPU time. 2024-05-20T16:24:09.312 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:24:08 smithi160 ceph-mon[36347]: pgmap v667: 129 pgs: 129 active+clean; 10 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 714 MiB/s wr, 1.26k op/s 2024-05-20T16:24:09.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:24:08 smithi082 ceph-mon[32238]: pgmap v667: 129 pgs: 129 active+clean; 10 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 714 MiB/s wr, 1.26k op/s 2024-05-20T16:24:09.386 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:24:08 smithi184 ceph-mon[36209]: pgmap v667: 129 pgs: 129 active+clean; 10 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 714 MiB/s wr, 1.26k op/s 2024-05-20T16:24:10.311 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:24:09 smithi160 ceph-mon[36347]: pgmap v668: 129 pgs: 129 active+clean; 10 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 781 MiB/s wr, 1.53k op/s 2024-05-20T16:24:10.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:24:09 smithi082 ceph-mon[32238]: pgmap v668: 129 pgs: 129 active+clean; 10 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 781 MiB/s wr, 1.53k op/s 2024-05-20T16:24:10.385 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:24:09 smithi184 ceph-mon[36209]: pgmap v668: 129 pgs: 129 active+clean; 10 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 781 MiB/s wr, 1.53k op/s 2024-05-20T16:24:13.136 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:24:12 smithi184 ceph-mon[36209]: pgmap v669: 129 pgs: 129 active+clean; 10 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 918 KiB/s rd, 650 MiB/s wr, 1.31k op/s 2024-05-20T16:24:13.311 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:24:12 smithi160 ceph-mon[36347]: pgmap v669: 129 pgs: 129 active+clean; 10 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 918 KiB/s rd, 650 MiB/s wr, 1.31k op/s 2024-05-20T16:24:13.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:24:12 smithi082 ceph-mon[32238]: pgmap v669: 129 pgs: 129 active+clean; 10 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 918 KiB/s rd, 650 MiB/s wr, 1.31k op/s 2024-05-20T16:24:15.135 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:24:14 smithi184 ceph-mon[36209]: pgmap v670: 129 pgs: 129 active+clean; 10 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 918 KiB/s rd, 758 MiB/s wr, 1.57k op/s 2024-05-20T16:24:15.312 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:24:14 smithi160 ceph-mon[36347]: pgmap v670: 129 pgs: 129 active+clean; 10 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 918 KiB/s rd, 758 MiB/s wr, 1.57k op/s 2024-05-20T16:24:15.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:24:14 smithi082 ceph-mon[32238]: pgmap v670: 129 pgs: 129 active+clean; 10 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 918 KiB/s rd, 758 MiB/s wr, 1.57k op/s 2024-05-20T16:24:17.162 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:24:16 smithi082 ceph-mon[32238]: pgmap v671: 129 pgs: 129 active+clean; 10 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 683 KiB/s rd, 645 MiB/s wr, 1.43k op/s 2024-05-20T16:24:17.312 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:24:16 smithi160 ceph-mon[36347]: pgmap v671: 129 pgs: 129 active+clean; 10 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 683 KiB/s rd, 645 MiB/s wr, 1.43k op/s 2024-05-20T16:24:17.385 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:24:16 smithi184 ceph-mon[36209]: pgmap v671: 129 pgs: 129 active+clean; 10 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 683 KiB/s rd, 645 MiB/s wr, 1.43k op/s 2024-05-20T16:24:18.012 INFO:journalctl@ceph.mds.c.smithi184.stdout:May 20 16:24:17 smithi184 systemd[1]: ceph-59d1600a-16c2-11ef-bc9a-c7b262605968@mds.c.service: Scheduled restart job, restart counter is at 1. 2024-05-20T16:24:18.012 INFO:journalctl@ceph.mds.c.smithi184.stdout:May 20 16:24:17 smithi184 systemd[1]: Stopped Ceph mds.c for 59d1600a-16c2-11ef-bc9a-c7b262605968. 2024-05-20T16:24:18.012 INFO:journalctl@ceph.mds.c.smithi184.stdout:May 20 16:24:17 smithi184 systemd[1]: ceph-59d1600a-16c2-11ef-bc9a-c7b262605968@mds.c.service: Consumed 1min 10.259s CPU time. 2024-05-20T16:24:18.012 INFO:journalctl@ceph.mds.c.smithi184.stdout:May 20 16:24:17 smithi184 systemd[1]: Starting Ceph mds.c for 59d1600a-16c2-11ef-bc9a-c7b262605968... 2024-05-20T16:24:18.340 INFO:journalctl@ceph.mds.c.smithi184.stdout:May 20 16:24:18 smithi184 podman[71230]: 2024-05-20 16:24:17.919141898 +0000 UTC m=+0.033015496 image pull cfcc61b2d476f611479310eb042aa05f11dd5f60dbb65e3b0d7e0e11d024cdb3 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708 2024-05-20T16:24:18.341 INFO:journalctl@ceph.mds.c.smithi184.stdout:May 20 16:24:18 smithi184 podman[71230]: 2024-05-20 16:24:18.063712587 +0000 UTC m=+0.177586148 container create 9f97ccd0ffc6c1a5a8a965ef498af031b2f17bcf10c3df5f194fb140c6eb8a14 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-c, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240514, ceph=True, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-05-20T16:24:18.635 INFO:journalctl@ceph.mds.c.smithi184.stdout:May 20 16:24:18 smithi184 podman[71230]: 2024-05-20 16:24:18.341993465 +0000 UTC m=+0.455867036 container init 9f97ccd0ffc6c1a5a8a965ef498af031b2f17bcf10c3df5f194fb140c6eb8a14 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-c, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, org.label-schema.build-date=20240514, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-20T16:24:18.636 INFO:journalctl@ceph.mds.c.smithi184.stdout:May 20 16:24:18 smithi184 podman[71230]: 2024-05-20 16:24:18.345718022 +0000 UTC m=+0.459591593 container start 9f97ccd0ffc6c1a5a8a965ef498af031b2f17bcf10c3df5f194fb140c6eb8a14 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-c, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240514, maintainer=Guillaume Abrioux , ceph=True) 2024-05-20T16:24:19.135 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:24:18 smithi184 ceph-mon[36209]: pgmap v672: 129 pgs: 129 active+clean; 10 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 343 KiB/s rd, 688 MiB/s wr, 1.54k op/s 2024-05-20T16:24:19.136 INFO:journalctl@ceph.mds.c.smithi184.stdout:May 20 16:24:18 smithi184 bash[71230]: 9f97ccd0ffc6c1a5a8a965ef498af031b2f17bcf10c3df5f194fb140c6eb8a14 2024-05-20T16:24:19.136 INFO:journalctl@ceph.mds.c.smithi184.stdout:May 20 16:24:18 smithi184 systemd[1]: Started Ceph mds.c for 59d1600a-16c2-11ef-bc9a-c7b262605968. 2024-05-20T16:24:19.288 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:19.288+0000 7fd5a420d640 1 -- 172.21.15.82:0/3753293405 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd59c1093d0 msgr2=0x7fd59c109dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:24:19.288 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:19.288+0000 7fd5a420d640 1 --2- 172.21.15.82:0/3753293405 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd59c1093d0 0x7fd59c109dc0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fd598009900 tx=0x7fd59802a420 comp rx=0 tx=0).stop 2024-05-20T16:24:19.289 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:19.288+0000 7fd5a420d640 1 -- 172.21.15.82:0/3753293405 shutdown_connections 2024-05-20T16:24:19.289 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:19.288+0000 7fd5a420d640 1 --2- 172.21.15.82:0/3753293405 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd59c10f020 0x7fd59c10f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:24:19.289 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:19.288+0000 7fd5a420d640 1 --2- 172.21.15.82:0/3753293405 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd59c10a300 0x7fd59c10a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:24:19.289 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:19.288+0000 7fd5a420d640 1 --2- 172.21.15.82:0/3753293405 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd59c1093d0 0x7fd59c109dc0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:24:19.289 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:19.288+0000 7fd5a420d640 1 -- 172.21.15.82:0/3753293405 >> 172.21.15.82:0/3753293405 conn(0x7fd59c06f4f0 msgr2=0x7fd59c071980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:24:19.294 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:19.291+0000 7fd5a420d640 1 -- 172.21.15.82:0/3753293405 shutdown_connections 2024-05-20T16:24:19.294 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:19.291+0000 7fd5a420d640 1 -- 172.21.15.82:0/3753293405 wait complete. 2024-05-20T16:24:19.294 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:19.293+0000 7fd5a420d640 1 Processor -- start 2024-05-20T16:24:19.294 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:19.293+0000 7fd5a420d640 1 -- start start 2024-05-20T16:24:19.294 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:19.293+0000 7fd5a420d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd59c1093d0 0x7fd59c1a8300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:24:19.294 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:19.293+0000 7fd5a420d640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd59c10a300 0x7fd59c1a8840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:24:19.294 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:19.293+0000 7fd5a420d640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd59c10f020 0x7fd59c1af7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:24:19.294 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:19.293+0000 7fd5a17a8640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd59c10a300 0x7fd59c1a8840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:24:19.294 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:19.294+0000 7fd5a17a8640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd59c10a300 0x7fd59c1a8840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.82:37950/0 (socket says 172.21.15.82:37950) 2024-05-20T16:24:19.295 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:19.294+0000 7fd5a17a8640 1 -- 172.21.15.82:0/2289878335 learned_addr learned my addr 172.21.15.82:0/2289878335 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:24:19.295 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:19.294+0000 7fd5a420d640 1 -- 172.21.15.82:0/2289878335 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fd59c11d700 con 0x7fd59c1093d0 2024-05-20T16:24:19.295 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:19.294+0000 7fd5a420d640 1 -- 172.21.15.82:0/2289878335 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fd59c11d580 con 0x7fd59c10f020 2024-05-20T16:24:19.295 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:19.294+0000 7fd5a420d640 1 -- 172.21.15.82:0/2289878335 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd59c11d880 con 0x7fd59c10a300 2024-05-20T16:24:19.295 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:19.294+0000 7fd5a1fa9640 1 --2- 172.21.15.82:0/2289878335 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd59c1093d0 0x7fd59c1a8300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:24:19.298 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:19.294+0000 7fd5a27aa640 1 --2- 172.21.15.82:0/2289878335 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd59c10f020 0x7fd59c1af7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:24:19.298 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:19.295+0000 7fd5a17a8640 1 -- 172.21.15.82:0/2289878335 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd59c10f020 msgr2=0x7fd59c1af7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:24:19.298 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:19.295+0000 7fd5a17a8640 1 --2- 172.21.15.82:0/2289878335 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd59c10f020 0x7fd59c1af7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:24:19.298 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:19.295+0000 7fd5a17a8640 1 -- 172.21.15.82:0/2289878335 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd59c1093d0 msgr2=0x7fd59c1a8300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:24:19.298 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:19.295+0000 7fd5a17a8640 1 --2- 172.21.15.82:0/2289878335 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd59c1093d0 0x7fd59c1a8300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:24:19.298 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:19.295+0000 7fd5a17a8640 1 -- 172.21.15.82:0/2289878335 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd59c1afe40 con 0x7fd59c10a300 2024-05-20T16:24:19.298 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:19.295+0000 7fd5a17a8640 1 --2- 172.21.15.82:0/2289878335 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd59c10a300 0x7fd59c1a8840 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fd59400ba50 tx=0x7fd59400bf10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:24:19.301 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:19.300+0000 7fd592ffd640 1 -- 172.21.15.82:0/2289878335 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd594019070 con 0x7fd59c10a300 2024-05-20T16:24:19.302 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:19.300+0000 7fd592ffd640 1 -- 172.21.15.82:0/2289878335 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd59400cc10 con 0x7fd59c10a300 2024-05-20T16:24:19.302 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:19.300+0000 7fd592ffd640 1 -- 172.21.15.82:0/2289878335 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd594007730 con 0x7fd59c10a300 2024-05-20T16:24:19.302 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:19.302+0000 7fd5a420d640 1 -- 172.21.15.82:0/2289878335 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd59c1b0130 con 0x7fd59c10a300 2024-05-20T16:24:19.302 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:19.302+0000 7fd5a420d640 1 -- 172.21.15.82:0/2289878335 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd59c1b0650 con 0x7fd59c10a300 2024-05-20T16:24:19.307 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:19.308+0000 7fd592ffd640 1 -- 172.21.15.82:0/2289878335 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fd59401c070 con 0x7fd59c10a300 2024-05-20T16:24:19.308 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:19.308+0000 7fd5ab192740 1 Processor -- start 2024-05-20T16:24:19.310 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:19.310+0000 7fd592ffd640 1 --2- 172.21.15.82:0/2289878335 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fd5880a2d90 0x7fd5880a52a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:24:19.310 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:19.310+0000 7fd592ffd640 1 -- 172.21.15.82:0/2289878335 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9150+0+0 (secure 0 0 0) 0x7fd5940cebe0 con 0x7fd59c10a300 2024-05-20T16:24:19.312 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:24:18 smithi160 ceph-mon[36347]: pgmap v672: 129 pgs: 129 active+clean; 10 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 343 KiB/s rd, 688 MiB/s wr, 1.54k op/s 2024-05-20T16:24:19.312 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:19.312+0000 7fd5ab192740 1 -- start start 2024-05-20T16:24:19.313 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:19.313+0000 7fd5ab192740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556b34b9c1b0 0x556b34f3be60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:24:19.313 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:19.313+0000 7fd5ab192740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556b34b732f0 0x556b34f3c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:24:19.313 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:19.313+0000 7fd5ab192740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556b34b7b4d0 0x556b34f432e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:24:19.313 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:19.313+0000 7fd5a17a8640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556b34b732f0 0x556b34f3c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:24:19.313 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:19.314+0000 7fd5a17a8640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556b34b732f0 0x556b34f3c3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:54536/0 (socket says 172.21.15.82:54536) 2024-05-20T16:24:19.313 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:19.314+0000 7fd5a27aa640 1 --2- 172.21.15.82:0/2289878335 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fd5880a2d90 0x7fd5880a52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:24:19.314 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:19.314+0000 7fd5a17a8640 1 -- 172.21.15.82:0/3412994508 learned_addr learned my addr 172.21.15.82:0/3412994508 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:24:19.314 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:19.314+0000 7fd5ab192740 1 -- 172.21.15.82:0/3412994508 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x556b34ea0340 con 0x556b34b732f0 2024-05-20T16:24:19.314 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:19.315+0000 7fd5a27aa640 1 --2- 172.21.15.82:0/3412994508 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556b34b7b4d0 0x556b34f432e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:24:19.315 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:19.315+0000 7fd5ab192740 1 -- 172.21.15.82:0/3412994508 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x556b34e48290 con 0x556b34b9c1b0 2024-05-20T16:24:19.315 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:19.315+0000 7fd5ab192740 1 -- 172.21.15.82:0/3412994508 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x556b34e9c300 con 0x556b34b7b4d0 2024-05-20T16:24:19.315 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:19.315+0000 7fd5a27aa640 1 --2- 172.21.15.82:0/2289878335 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fd5880a2d90 0x7fd5880a52a0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fd58c00b880 tx=0x7fd58c008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:24:19.318 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:19.318+0000 7fd5a27aa640 1 -- 172.21.15.82:0/3412994508 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556b34b9c1b0 msgr2=0x556b34f3be60 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-20T16:24:19.318 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:19.318+0000 7fd5a27aa640 1 --2- 172.21.15.82:0/3412994508 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556b34b9c1b0 0x556b34f3be60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:24:19.318 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:19.318+0000 7fd5a27aa640 1 -- 172.21.15.82:0/3412994508 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556b34b732f0 msgr2=0x556b34f3c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:24:19.318 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:19.318+0000 7fd5a27aa640 1 --2- 172.21.15.82:0/3412994508 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556b34b732f0 0x556b34f3c3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:24:19.318 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:19.318+0000 7fd5a27aa640 1 -- 172.21.15.82:0/3412994508 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556b34e6c8e0 con 0x556b34b7b4d0 2024-05-20T16:24:19.318 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:19.318+0000 7fd5a17a8640 1 --2- 172.21.15.82:0/3412994508 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556b34b732f0 0x556b34f3c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T16:24:19.323 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:19.323+0000 7fd5a27aa640 1 --2- 172.21.15.82:0/3412994508 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556b34b7b4d0 0x556b34f432e0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fd58c00b610 tx=0x7fd58c010000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e2acfd400b9caf26 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:24:19.323 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:19.323+0000 7fd587fff640 1 -- 172.21.15.82:0/3412994508 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd58c024070 con 0x556b34b7b4d0 2024-05-20T16:24:19.323 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:19.323+0000 7fd587fff640 1 -- 172.21.15.82:0/3412994508 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd58c014d90 con 0x556b34b7b4d0 2024-05-20T16:24:19.323 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:19.323+0000 7fd587fff640 1 -- 172.21.15.82:0/3412994508 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd58c0123e0 con 0x556b34b7b4d0 2024-05-20T16:24:19.329 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:19.329+0000 7fd5ab192740 1 -- 172.21.15.82:0/3412994508 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556b34b7b4d0 msgr2=0x556b34f432e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:24:19.330 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:19.329+0000 7fd5ab192740 1 --2- 172.21.15.82:0/3412994508 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556b34b7b4d0 0x556b34f432e0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fd58c00b610 tx=0x7fd58c010000 comp rx=0 tx=0).stop 2024-05-20T16:24:19.330 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:19.330+0000 7fd5ab192740 1 -- 172.21.15.82:0/3412994508 shutdown_connections 2024-05-20T16:24:19.330 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:19.330+0000 7fd5ab192740 1 --2- 172.21.15.82:0/3412994508 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556b34b732f0 0x556b34f3c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:24:19.330 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:19.330+0000 7fd5ab192740 1 --2- 172.21.15.82:0/3412994508 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556b34b7b4d0 0x556b34f432e0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:24:19.330 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:19.330+0000 7fd5ab192740 1 --2- 172.21.15.82:0/3412994508 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556b34b9c1b0 0x556b34f3be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:24:19.330 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:19.330+0000 7fd5ab192740 1 -- 172.21.15.82:0/3412994508 >> 172.21.15.82:0/3412994508 conn(0x556b34b90300 msgr2=0x556b34f377f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:24:19.341 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:19.341+0000 7fd5ab192740 1 -- 172.21.15.82:0/3412994508 shutdown_connections 2024-05-20T16:24:19.341 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:19.341+0000 7fd5ab192740 1 -- 172.21.15.82:0/3412994508 wait complete. 2024-05-20T16:24:19.341 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:19.342+0000 7fd5ab192740 1 -- start start 2024-05-20T16:24:19.341 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:19.342+0000 7fd5ab192740 1 Processor -- start 2024-05-20T16:24:19.342 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:19.343+0000 7fd5ab192740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556b34b732f0 0x556b34f45bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:24:19.342 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:19.343+0000 7fd5ab192740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556b34b7b4d0 0x556b34f46100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:24:19.343 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:19.343+0000 7fd5ab192740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556b34b9c1b0 0x556b34f4d060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:24:19.343 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:19.343+0000 7fd5ab192740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x556b34f43d30 con 0x556b34b9c1b0 2024-05-20T16:24:19.343 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:19.343+0000 7fd5ab192740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x556b34e5b790 con 0x556b34b7b4d0 2024-05-20T16:24:19.343 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:19.343+0000 7fd5ab192740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x556b34f43eb0 con 0x556b34b732f0 2024-05-20T16:24:19.343 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:19.343+0000 7fd5ab192740 10 client.0 _ll_register_callbacks cb 0x556b34e96a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T16:24:19.343 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:19.343+0000 7fd5a1fa9640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556b34b732f0 0x556b34f45bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:24:19.344 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:19.343+0000 7fd5a1fa9640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556b34b732f0 0x556b34f45bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.82:37958/0 (socket says 172.21.15.82:37958) 2024-05-20T16:24:19.344 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:19.343+0000 7fd5a1fa9640 1 -- 172.21.15.82:0/2912005585 learned_addr learned my addr 172.21.15.82:0/2912005585 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:24:19.344 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:19.343+0000 7fd5a27aa640 1 --2- 172.21.15.82:0/2912005585 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556b34b9c1b0 0x556b34f4d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:24:19.344 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:19.344+0000 7fd5a27aa640 1 -- 172.21.15.82:0/2912005585 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556b34b732f0 msgr2=0x556b34f45bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:24:19.344 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:19.344+0000 7fd5a27aa640 1 --2- 172.21.15.82:0/2912005585 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556b34b732f0 0x556b34f45bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:24:19.344 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:19.344+0000 7fd5a27aa640 1 -- 172.21.15.82:0/2912005585 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556b34b7b4d0 msgr2=0x556b34f46100 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-20T16:24:19.344 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:19.344+0000 7fd5a27aa640 1 --2- 172.21.15.82:0/2912005585 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556b34b7b4d0 0x556b34f46100 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:24:19.344 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:19.344+0000 7fd5a27aa640 1 -- 172.21.15.82:0/2912005585 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556b34bb4210 con 0x556b34b9c1b0 2024-05-20T16:24:19.345 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:19.345+0000 7fd5a27aa640 1 --2- 172.21.15.82:0/2912005585 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556b34b9c1b0 0x556b34f4d060 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fd58c004700 tx=0x7fd58c01fdc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e216083920974585 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:24:19.345 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:19.345+0000 7fd5857fa640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-05-20T16:24:19.345 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:19.346+0000 7fd5857fa640 1 -- 172.21.15.82:0/2912005585 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd58c02c070 con 0x556b34b9c1b0 2024-05-20T16:24:19.346 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:19.346+0000 7fd5ab192740 1 -- 172.21.15.82:0/2912005585 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd58c014d90 con 0x556b34b9c1b0 2024-05-20T16:24:19.346 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:19.346+0000 7fd5857fa640 1 -- 172.21.15.82:0/2912005585 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9150+0+0 (secure 0 0 0) 0x7fd58c014ab0 con 0x556b34b9c1b0 2024-05-20T16:24:19.346 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:19.347+0000 7fd5857fa640 1 -- 172.21.15.82:0/2912005585 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fd58c038050 con 0x556b34b9c1b0 2024-05-20T16:24:19.348 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:19.349+0000 7fd5ab192740 10 client.15942 fetch_fsmap learned FSMap version 27 2024-05-20T16:24:19.349 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:19.349+0000 7fd5ab192740 1 -- 172.21.15.82:0/2912005585 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x556b34c80620 con 0x556b34b9c1b0 2024-05-20T16:24:19.350 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:19.350+0000 7fd5857fa640 1 -- 172.21.15.82:0/2912005585 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd58c014d90 con 0x556b34b9c1b0 2024-05-20T16:24:19.350 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:19.350+0000 7fd5857fa640 1 -- 172.21.15.82:0/2912005585 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fd58c013070 con 0x556b34b9c1b0 2024-05-20T16:24:19.350 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:19.351+0000 7fd5ab192740 10 client.15942 fetch_fsmap finished waiting for FSMap version 27 2024-05-20T16:24:19.351 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:19.351+0000 7fd5ab192740 10 client.15942 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T16:24:19.351 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:19.351+0000 7fd5ab192740 20 client.15942 populate_metadata read hostname 'smithi082' 2024-05-20T16:24:19.351 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:19.351+0000 7fd5ab192740 1 --2- 172.21.15.82:0/2912005585 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x556b34b5ad80 0x556b34f53800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:24:19.351 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:19.351+0000 7fd5ab192740 4 client.15942 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T16:24:19.351 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:19.351+0000 7fd5ab192740 1 -- 172.21.15.82:0/2912005585 --> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556b34eada70 con 0x556b34b5ad80 2024-05-20T16:24:19.353 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:19.354+0000 7fd5a1fa9640 1 --2- 172.21.15.82:0/2912005585 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x556b34b5ad80 0x556b34f53800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:24:19.358 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:19.357+0000 7fd5a1fa9640 1 --2- 172.21.15.82:0/2912005585 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x556b34b5ad80 0x556b34f53800 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3f325c5c09d2cc5d server_cookie=8e0d27464ae8cb22 in_seq=0 out_seq=0 2024-05-20T16:24:19.358 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:19.358+0000 7fd5857fa640 10 client.15942 ms_handle_connect on v2:172.21.15.160:6838/2765554482 2024-05-20T16:24:19.360 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:24:18 smithi082 ceph-mon[32238]: pgmap v672: 129 pgs: 129 active+clean; 10 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 343 KiB/s rd, 688 MiB/s wr, 1.54k op/s 2024-05-20T16:24:19.363 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:19.361+0000 7fd5857fa640 1 -- 172.21.15.82:0/2912005585 <== mds.0 v2:172.21.15.160:6838/2765554482 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x556b34eada70 con 0x556b34b5ad80 2024-05-20T16:24:19.363 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:19.361+0000 7fd5857fa640 10 client.15942 handle_command_reply: tid=0 2024-05-20T16:24:19.363 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:19.361+0000 7fd5ab192740 1 client.15942 shutdown 2024-05-20T16:24:19.363 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:19.364+0000 7fd5ab192740 1 -- 172.21.15.82:0/2912005585 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556b34b9c1b0 msgr2=0x556b34f4d060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:24:19.364 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:19.364+0000 7fd5ab192740 1 --2- 172.21.15.82:0/2912005585 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556b34b9c1b0 0x556b34f4d060 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fd58c004700 tx=0x7fd58c01fdc0 comp rx=0 tx=0).stop 2024-05-20T16:24:19.365 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:19.366+0000 7fd5ab192740 1 -- 172.21.15.82:0/2912005585 shutdown_connections 2024-05-20T16:24:19.365 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:19.366+0000 7fd5ab192740 1 --2- 172.21.15.82:0/2912005585 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x556b34b5ad80 0x556b34f53800 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:24:19.366 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:19.366+0000 7fd5ab192740 1 --2- 172.21.15.82:0/2912005585 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556b34b9c1b0 0x556b34f4d060 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:24:19.366 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:19.366+0000 7fd5ab192740 1 --2- 172.21.15.82:0/2912005585 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556b34b7b4d0 0x556b34f46100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:24:19.366 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:19.366+0000 7fd5ab192740 1 --2- 172.21.15.82:0/2912005585 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556b34b732f0 0x556b34f45bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:24:19.366 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:19.366+0000 7fd5ab192740 1 -- 172.21.15.82:0/2912005585 >> 172.21.15.82:0/2912005585 conn(0x556b34b90300 msgr2=0x556b34f36810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:24:19.368 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:19.366+0000 7fd5857fa640 0 client.15942 ms_handle_reset on v2:172.21.15.160:6838/2765554482 2024-05-20T16:24:19.375 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:19.375+0000 7fd5ab192740 1 -- 172.21.15.82:0/2912005585 shutdown_connections 2024-05-20T16:24:19.375 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:19.375+0000 7fd5ab192740 1 -- 172.21.15.82:0/2912005585 wait complete. 2024-05-20T16:24:19.375 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:19.376+0000 7fd5ab192740 20 client.15942 trim_cache size 0 max 16384 2024-05-20T16:24:19.406 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:19.407+0000 7fd5ab192740 1 Processor -- start 2024-05-20T16:24:19.407 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:19.407+0000 7fd5ab192740 1 -- start start 2024-05-20T16:24:19.408 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:19.408+0000 7fd5ab192740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556b34b732f0 0x556b34f37250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:24:19.408 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:19.408+0000 7fd5ab192740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556b34b7b4d0 0x556b34f37790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:24:19.408 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:19.408+0000 7fd5a1fa9640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556b34b732f0 0x556b34f37250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:24:19.408 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:19.408+0000 7fd5a1fa9640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556b34b732f0 0x556b34f37250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.82:37960/0 (socket says 172.21.15.82:37960) 2024-05-20T16:24:19.408 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:19.408+0000 7fd5a1fa9640 1 -- 172.21.15.82:0/4243433437 learned_addr learned my addr 172.21.15.82:0/4243433437 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:24:19.408 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:19.408+0000 7fd5ab192740 1 --2- 172.21.15.82:0/4243433437 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556b34b9c1b0 0x556b34f4cf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:24:19.408 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:19.408+0000 7fd5ab192740 1 -- 172.21.15.82:0/4243433437 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fd59c1bd400 con 0x556b34b7b4d0 2024-05-20T16:24:19.408 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:19.408+0000 7fd5ab192740 1 -- 172.21.15.82:0/4243433437 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fd59c1bacd0 con 0x556b34b9c1b0 2024-05-20T16:24:19.409 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:19.409+0000 7fd5a27aa640 1 --2- 172.21.15.82:0/4243433437 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556b34b9c1b0 0x556b34f4cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:24:19.409 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:19.409+0000 7fd5ab192740 1 -- 172.21.15.82:0/4243433437 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd59c1b8b20 con 0x556b34b732f0 2024-05-20T16:24:19.409 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:19.409+0000 7fd5a17a8640 1 --2- 172.21.15.82:0/4243433437 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556b34b7b4d0 0x556b34f37790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:24:19.410 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:19.409+0000 7fd5a27aa640 1 -- 172.21.15.82:0/4243433437 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556b34b732f0 msgr2=0x556b34f37250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:24:19.410 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:19.409+0000 7fd5a27aa640 1 --2- 172.21.15.82:0/4243433437 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556b34b732f0 0x556b34f37250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:24:19.410 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:19.409+0000 7fd5a27aa640 1 -- 172.21.15.82:0/4243433437 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556b34b7b4d0 msgr2=0x556b34f37790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:24:19.410 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:19.409+0000 7fd5a27aa640 1 --2- 172.21.15.82:0/4243433437 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556b34b7b4d0 0x556b34f37790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:24:19.410 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:19.409+0000 7fd5a27aa640 1 -- 172.21.15.82:0/4243433437 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556b34f46c40 con 0x556b34b9c1b0 2024-05-20T16:24:19.410 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:19.410+0000 7fd5a27aa640 1 --2- 172.21.15.82:0/4243433437 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556b34b9c1b0 0x556b34f4cf80 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fd58c00d5d0 tx=0x7fd58c0144e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=908a3b1a843604f7 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:24:19.410 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:19.410+0000 7fd5877fe640 1 -- 172.21.15.82:0/4243433437 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd58c02c040 con 0x556b34b9c1b0 2024-05-20T16:24:19.410 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:19.410+0000 7fd5877fe640 1 -- 172.21.15.82:0/4243433437 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd58c01f990 con 0x556b34b9c1b0 2024-05-20T16:24:19.410 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:19.410+0000 7fd5877fe640 1 -- 172.21.15.82:0/4243433437 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd58c024040 con 0x556b34b9c1b0 2024-05-20T16:24:19.412 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:19.412+0000 7fd5ab192740 1 -- 172.21.15.82:0/4243433437 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556b34b9c1b0 msgr2=0x556b34f4cf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:24:19.412 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:19.413+0000 7fd5ab192740 1 --2- 172.21.15.82:0/4243433437 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556b34b9c1b0 0x556b34f4cf80 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fd58c00d5d0 tx=0x7fd58c0144e0 comp rx=0 tx=0).stop 2024-05-20T16:24:19.413 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:19.413+0000 7fd5ab192740 1 -- 172.21.15.82:0/4243433437 shutdown_connections 2024-05-20T16:24:19.413 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:19.414+0000 7fd5ab192740 1 --2- 172.21.15.82:0/4243433437 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556b34b7b4d0 0x556b34f37790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:24:19.413 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:19.414+0000 7fd5ab192740 1 --2- 172.21.15.82:0/4243433437 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556b34b9c1b0 0x556b34f4cf80 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:24:19.413 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:19.414+0000 7fd5ab192740 1 --2- 172.21.15.82:0/4243433437 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556b34b732f0 0x556b34f37250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:24:19.413 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:19.414+0000 7fd5ab192740 1 -- 172.21.15.82:0/4243433437 >> 172.21.15.82:0/4243433437 conn(0x556b34b90300 msgr2=0x556b34f44250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:24:19.415 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:19.415+0000 7fd5ab192740 1 -- 172.21.15.82:0/4243433437 shutdown_connections 2024-05-20T16:24:19.415 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:19.415+0000 7fd5ab192740 1 -- 172.21.15.82:0/4243433437 wait complete. 2024-05-20T16:24:19.415 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:19.416+0000 7fd5ab192740 1 -- start start 2024-05-20T16:24:19.416 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:19.416+0000 7fd5ab192740 1 Processor -- start 2024-05-20T16:24:19.417 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:19.417+0000 7fd5ab192740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556b34b732f0 0x556b34f4d280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:24:19.417 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:19.417+0000 7fd5ab192740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556b34b7b4d0 0x556b34f4d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:24:19.417 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:19.417+0000 7fd5ab192740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556b34b9c1b0 0x556b34f49ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:24:19.417 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:19.418+0000 7fd5ab192740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fd59c1b9190 con 0x556b34b9c1b0 2024-05-20T16:24:19.417 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:19.418+0000 7fd5ab192740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fd59c1ba600 con 0x556b34b7b4d0 2024-05-20T16:24:19.417 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:19.418+0000 7fd5ab192740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd59c1b9860 con 0x556b34b732f0 2024-05-20T16:24:19.418 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:19.418+0000 7fd5ab192740 10 client.0 _ll_register_callbacks cb 0x7fd58c022070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T16:24:19.418 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:19.419+0000 7fd5a1fa9640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556b34b732f0 0x556b34f4d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:24:19.418 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:19.419+0000 7fd5a17a8640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556b34b7b4d0 0x556b34f4d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:24:19.418 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:19.419+0000 7fd5a1fa9640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556b34b732f0 0x556b34f4d280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.82:37970/0 (socket says 172.21.15.82:37970) 2024-05-20T16:24:19.418 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:19.419+0000 7fd5a1fa9640 1 -- 172.21.15.82:0/4204541087 learned_addr learned my addr 172.21.15.82:0/4204541087 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:24:19.419 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:19.419+0000 7fd5a1fa9640 1 -- 172.21.15.82:0/4204541087 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556b34b7b4d0 msgr2=0x556b34f4d7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:24:19.419 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:19.419+0000 7fd5a1fa9640 1 --2- 172.21.15.82:0/4204541087 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556b34b7b4d0 0x556b34f4d7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:24:19.419 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:19.419+0000 7fd5a1fa9640 1 -- 172.21.15.82:0/4204541087 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556b34b9c1b0 msgr2=0x556b34f49ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:24:19.419 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:19.419+0000 7fd5a1fa9640 1 --2- 172.21.15.82:0/4204541087 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556b34b9c1b0 0x556b34f49ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:24:19.419 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:19.419+0000 7fd5a1fa9640 1 -- 172.21.15.82:0/4204541087 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556b34f53da0 con 0x556b34b732f0 2024-05-20T16:24:19.420 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:19.420+0000 7fd5a1fa9640 1 --2- 172.21.15.82:0/4204541087 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556b34b732f0 0x556b34f4d280 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fd598006010 tx=0x7fd5980045b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2cc1595ae09e1683 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:24:19.420 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:19.420+0000 7fd5857fa640 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-05-20T16:24:19.421 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:19.421+0000 7fd5857fa640 1 -- 172.21.15.82:0/4204541087 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd598035040 con 0x556b34b732f0 2024-05-20T16:24:19.421 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:19.421+0000 7fd5857fa640 1 -- 172.21.15.82:0/4204541087 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9150+0+0 (secure 0 0 0) 0x7fd598031c00 con 0x556b34b732f0 2024-05-20T16:24:19.421 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:19.421+0000 7fd5ab192740 1 -- 172.21.15.82:0/4204541087 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd58c01f990 con 0x556b34b732f0 2024-05-20T16:24:19.421 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:19.422+0000 7fd5857fa640 1 -- 172.21.15.82:0/4204541087 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fd598053050 con 0x556b34b732f0 2024-05-20T16:24:19.421 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:19.422+0000 7fd5ab192740 10 client.25489 fetch_fsmap learned FSMap version 27 2024-05-20T16:24:19.421 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:19.422+0000 7fd5ab192740 1 -- 172.21.15.82:0/4204541087 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x556b34fba740 con 0x556b34b732f0 2024-05-20T16:24:19.421 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:19.422+0000 7fd5857fa640 1 -- 172.21.15.82:0/4204541087 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd58c01f990 con 0x556b34b732f0 2024-05-20T16:24:19.424 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:19.422+0000 7fd5857fa640 1 -- 172.21.15.82:0/4204541087 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fd598003850 con 0x556b34b732f0 2024-05-20T16:24:19.424 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:19.422+0000 7fd5ab192740 10 client.25489 fetch_fsmap finished waiting for FSMap version 27 2024-05-20T16:24:19.424 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:19.423+0000 7fd5ab192740 10 client.25489 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T16:24:19.424 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:19.423+0000 7fd5ab192740 20 client.25489 populate_metadata read hostname 'smithi082' 2024-05-20T16:24:19.424 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:19.423+0000 7fd5ab192740 1 --2- 172.21.15.82:0/4204541087 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x556b34b301c0 0x556b34fbd130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:24:19.424 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:19.423+0000 7fd5ab192740 4 client.25489 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-20T16:24:19.424 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:19.423+0000 7fd5ab192740 1 -- 172.21.15.82:0/4204541087 --> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556b34ea7dc0 con 0x556b34b301c0 2024-05-20T16:24:19.424 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:19.424+0000 7fd5a17a8640 1 --2- 172.21.15.82:0/4204541087 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x556b34b301c0 0x556b34fbd130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:24:19.430 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:19.427+0000 7fd5857fa640 10 client.25489 ms_handle_connect on v2:172.21.15.160:6838/2765554482 2024-05-20T16:24:19.430 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:19.430+0000 7fd5a17a8640 1 --2- 172.21.15.82:0/4204541087 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x556b34b301c0 0x556b34fbd130 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=28c586bf5d0d9efa server_cookie=6e04fb4dda0b47a5 in_seq=0 out_seq=0 2024-05-20T16:24:19.635 INFO:journalctl@ceph.mds.c.smithi184.stdout:May 20 16:24:19 smithi184 ceph-mds[71247]: -- 172.21.15.184:0/796972349 <== mon.1 v2:172.21.15.184:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c603a3da40 con 0x55c603ab5180 2024-05-20T16:24:19.635 INFO:journalctl@ceph.mds.c.smithi184.stdout:May 20 16:24:19 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-c[71243]: starting mds.c at 2024-05-20T16:24:20.312 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:24:19 smithi160 ceph-mon[36347]: daemon mds.c restarted 2024-05-20T16:24:20.358 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:24:19 smithi082 ceph-mon[32238]: daemon mds.c restarted 2024-05-20T16:24:20.385 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:24:19 smithi184 ceph-mon[36209]: daemon mds.c restarted 2024-05-20T16:24:21.206 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:24:20 smithi184 ceph-mon[36209]: pgmap v673: 129 pgs: 129 active+clean; 10 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 343 KiB/s rd, 754 MiB/s wr, 1.52k op/s 2024-05-20T16:24:21.206 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:24:20 smithi184 ceph-mon[36209]: osdmap e97: 12 total, 12 up, 12 in 2024-05-20T16:24:21.206 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:24:20 smithi184 ceph-mon[36209]: mds.? [v2:172.21.15.184:6836/552848659,v1:172.21.15.184:6837/552848659] up:boot 2024-05-20T16:24:21.206 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:24:20 smithi184 ceph-mon[36209]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 2 up:standby-replay 7 up:standby 2024-05-20T16:24:21.206 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:24:20 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-20T16:24:21.206 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:24:20 smithi184 ceph-mon[36209]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-05-20T16:24:21.312 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:24:20 smithi160 ceph-mon[36347]: pgmap v673: 129 pgs: 129 active+clean; 10 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 343 KiB/s rd, 754 MiB/s wr, 1.52k op/s 2024-05-20T16:24:21.312 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:24:20 smithi160 ceph-mon[36347]: osdmap e97: 12 total, 12 up, 12 in 2024-05-20T16:24:21.312 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:24:20 smithi160 ceph-mon[36347]: mds.? [v2:172.21.15.184:6836/552848659,v1:172.21.15.184:6837/552848659] up:boot 2024-05-20T16:24:21.312 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:24:20 smithi160 ceph-mon[36347]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 2 up:standby-replay 7 up:standby 2024-05-20T16:24:21.312 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:24:20 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-20T16:24:21.312 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:24:20 smithi160 ceph-mon[36347]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-05-20T16:24:21.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:24:20 smithi082 ceph-mon[32238]: pgmap v673: 129 pgs: 129 active+clean; 10 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 343 KiB/s rd, 754 MiB/s wr, 1.52k op/s 2024-05-20T16:24:21.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:24:20 smithi082 ceph-mon[32238]: osdmap e97: 12 total, 12 up, 12 in 2024-05-20T16:24:21.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:24:20 smithi082 ceph-mon[32238]: mds.? [v2:172.21.15.184:6836/552848659,v1:172.21.15.184:6837/552848659] up:boot 2024-05-20T16:24:21.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:24:20 smithi082 ceph-mon[32238]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 2 up:standby-replay 7 up:standby 2024-05-20T16:24:21.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:24:20 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-20T16:24:21.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:24:20 smithi082 ceph-mon[32238]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-05-20T16:24:22.312 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:24:21 smithi160 ceph-mon[36347]: pgmap v675: 129 pgs: 129 active+clean; 10 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 735 MiB/s wr, 1.40k op/s 2024-05-20T16:24:22.361 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:24:21 smithi082 ceph-mon[32238]: pgmap v675: 129 pgs: 129 active+clean; 10 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 735 MiB/s wr, 1.40k op/s 2024-05-20T16:24:22.386 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:24:21 smithi184 ceph-mon[36209]: pgmap v675: 129 pgs: 129 active+clean; 10 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 735 MiB/s wr, 1.40k op/s 2024-05-20T16:24:23.555 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:23.554+0000 7fd5857fa640 1 -- 172.21.15.82:0/4204541087 <== mds.0 v2:172.21.15.160:6838/2765554482 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556b34ea7dc0 con 0x556b34b301c0 2024-05-20T16:24:23.555 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:23.554+0000 7fd5857fa640 10 client.25489 handle_command_reply: tid=0 2024-05-20T16:24:23.555 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:23.554+0000 7fd5ab192740 1 client.25489 shutdown 2024-05-20T16:24:23.555 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:23.555+0000 7fd5ab192740 1 -- 172.21.15.82:0/4204541087 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556b34b732f0 msgr2=0x556b34f4d280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:24:23.555 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:23.555+0000 7fd5ab192740 1 --2- 172.21.15.82:0/4204541087 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556b34b732f0 0x556b34f4d280 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fd598006010 tx=0x7fd5980045b0 comp rx=0 tx=0).stop 2024-05-20T16:24:23.555 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:23.555+0000 7fd5ab192740 1 -- 172.21.15.82:0/4204541087 shutdown_connections 2024-05-20T16:24:23.556 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:23.555+0000 7fd5ab192740 1 --2- 172.21.15.82:0/4204541087 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x556b34b301c0 0x556b34fbd130 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:24:23.556 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:23.555+0000 7fd5ab192740 1 --2- 172.21.15.82:0/4204541087 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556b34b9c1b0 0x556b34f49ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:24:23.556 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:23.555+0000 7fd5ab192740 1 --2- 172.21.15.82:0/4204541087 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556b34b7b4d0 0x556b34f4d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:24:23.556 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:23.555+0000 7fd5ab192740 1 --2- 172.21.15.82:0/4204541087 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556b34b732f0 0x556b34f4d280 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:24:23.556 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:23.555+0000 7fd5ab192740 1 -- 172.21.15.82:0/4204541087 >> 172.21.15.82:0/4204541087 conn(0x556b34b90300 msgr2=0x556b34f37250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:24:23.556 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:23.555+0000 7fd5857fa640 0 client.25489 ms_handle_reset on v2:172.21.15.160:6838/2765554482 2024-05-20T16:24:23.556 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:23.555+0000 7fd5ab192740 1 -- 172.21.15.82:0/4204541087 shutdown_connections 2024-05-20T16:24:23.556 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:23.555+0000 7fd5ab192740 1 -- 172.21.15.82:0/4204541087 wait complete. 2024-05-20T16:24:23.557 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:23.555+0000 7fd5ab192740 20 client.25489 trim_cache size 0 max 16384 2024-05-20T16:24:23.557 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:{ 2024-05-20T16:24:23.557 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "message": "", 2024-05-20T16:24:23.557 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "return_code": 0 2024-05-20T16:24:23.557 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:} 2024-05-20T16:24:23.557 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:23.556+0000 7fd5a420d640 1 -- 172.21.15.82:0/2289878335 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fd5880a2d90 msgr2=0x7fd5880a52a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:24:23.557 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:23.556+0000 7fd5a420d640 1 --2- 172.21.15.82:0/2289878335 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fd5880a2d90 0x7fd5880a52a0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fd58c00b880 tx=0x7fd58c008040 comp rx=0 tx=0).stop 2024-05-20T16:24:23.558 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:23.556+0000 7fd5a420d640 1 -- 172.21.15.82:0/2289878335 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd59c10a300 msgr2=0x7fd59c1a8840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:24:23.558 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:23.556+0000 7fd5a420d640 1 --2- 172.21.15.82:0/2289878335 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd59c10a300 0x7fd59c1a8840 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fd59400ba50 tx=0x7fd59400bf10 comp rx=0 tx=0).stop 2024-05-20T16:24:23.558 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:23.557+0000 7fd5a420d640 1 -- 172.21.15.82:0/2289878335 shutdown_connections 2024-05-20T16:24:23.558 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:23.557+0000 7fd5a420d640 1 --2- 172.21.15.82:0/2289878335 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fd5880a2d90 0x7fd5880a52a0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:24:23.558 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:23.557+0000 7fd5a420d640 1 --2- 172.21.15.82:0/2289878335 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd59c10f020 0x7fd59c1af7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:24:23.558 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:23.557+0000 7fd5a420d640 1 --2- 172.21.15.82:0/2289878335 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd59c10a300 0x7fd59c1a8840 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:24:23.558 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:23.557+0000 7fd5a420d640 1 --2- 172.21.15.82:0/2289878335 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd59c1093d0 0x7fd59c1a8300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:24:23.558 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:23.557+0000 7fd5a420d640 1 -- 172.21.15.82:0/2289878335 >> 172.21.15.82:0/2289878335 conn(0x7fd59c06f4f0 msgr2=0x7fd59c10b000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:24:23.558 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:23.558+0000 7fd5a420d640 1 -- 172.21.15.82:0/2289878335 shutdown_connections 2024-05-20T16:24:23.558 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:23.558+0000 7fd5a420d640 1 -- 172.21.15.82:0/2289878335 wait complete. 2024-05-20T16:24:25.138 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:24:24 smithi184 ceph-mon[36209]: pgmap v676: 129 pgs: 129 active+clean; 10 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 699 MiB/s wr, 1.67k op/s 2024-05-20T16:24:25.312 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:24:24 smithi160 ceph-mon[36347]: pgmap v676: 129 pgs: 129 active+clean; 10 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 699 MiB/s wr, 1.67k op/s 2024-05-20T16:24:25.361 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:24:24 smithi082 ceph-mon[32238]: pgmap v676: 129 pgs: 129 active+clean; 10 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 699 MiB/s wr, 1.67k op/s 2024-05-20T16:24:27.135 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:24:26 smithi184 ceph-mon[36209]: pgmap v677: 129 pgs: 129 active+clean; 10 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 690 MiB/s wr, 1.75k op/s 2024-05-20T16:24:27.312 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:24:26 smithi160 ceph-mon[36347]: pgmap v677: 129 pgs: 129 active+clean; 10 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 690 MiB/s wr, 1.75k op/s 2024-05-20T16:24:27.364 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:24:26 smithi082 ceph-mon[32238]: pgmap v677: 129 pgs: 129 active+clean; 10 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 690 MiB/s wr, 1.75k op/s 2024-05-20T16:24:29.312 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:24:28 smithi160 ceph-mon[36347]: pgmap v678: 129 pgs: 129 active+clean; 10 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 631 MiB/s wr, 1.57k op/s 2024-05-20T16:24:29.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:24:28 smithi082 ceph-mon[32238]: pgmap v678: 129 pgs: 129 active+clean; 10 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 631 MiB/s wr, 1.57k op/s 2024-05-20T16:24:29.385 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:24:28 smithi184 ceph-mon[36209]: pgmap v678: 129 pgs: 129 active+clean; 10 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 631 MiB/s wr, 1.57k op/s 2024-05-20T16:24:31.312 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:24:30 smithi160 ceph-mon[36347]: pgmap v679: 129 pgs: 129 active+clean; 10 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 711 MiB/s wr, 1.51k op/s 2024-05-20T16:24:31.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:24:30 smithi082 ceph-mon[32238]: pgmap v679: 129 pgs: 129 active+clean; 10 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 711 MiB/s wr, 1.51k op/s 2024-05-20T16:24:31.385 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:24:30 smithi184 ceph-mon[36209]: pgmap v679: 129 pgs: 129 active+clean; 10 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 711 MiB/s wr, 1.51k op/s 2024-05-20T16:24:33.312 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:24:32 smithi160 ceph-mon[36347]: pgmap v680: 129 pgs: 129 active+clean; 10 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 665 MiB/s wr, 1.40k op/s 2024-05-20T16:24:33.358 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:24:32 smithi082 ceph-mon[32238]: pgmap v680: 129 pgs: 129 active+clean; 10 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 665 MiB/s wr, 1.40k op/s 2024-05-20T16:24:33.385 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:24:32 smithi184 ceph-mon[36209]: pgmap v680: 129 pgs: 129 active+clean; 10 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 665 MiB/s wr, 1.40k op/s 2024-05-20T16:24:34.312 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:24:33 smithi160 ceph-mon[36347]: pgmap v681: 129 pgs: 129 active+clean; 10 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 730 MiB/s wr, 1.54k op/s 2024-05-20T16:24:34.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:24:33 smithi082 ceph-mon[32238]: pgmap v681: 129 pgs: 129 active+clean; 10 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 730 MiB/s wr, 1.54k op/s 2024-05-20T16:24:34.385 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:24:33 smithi184 ceph-mon[36209]: pgmap v681: 129 pgs: 129 active+clean; 10 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 730 MiB/s wr, 1.54k op/s 2024-05-20T16:24:36.876 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:36.876+0000 7fc97bdea640 1 -- 172.21.15.82:0/2861189629 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc97410a300 msgr2=0x7fc97410a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:24:36.876 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:36.876+0000 7fc97bdea640 1 --2- 172.21.15.82:0/2861189629 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc97410a300 0x7fc97410a710 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fc964009900 tx=0x7fc96402a440 comp rx=0 tx=0).stop 2024-05-20T16:24:36.887 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:36.883+0000 7fc97bdea640 1 -- 172.21.15.82:0/2861189629 shutdown_connections 2024-05-20T16:24:36.887 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:36.883+0000 7fc97bdea640 1 --2- 172.21.15.82:0/2861189629 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc97410a300 0x7fc97410a710 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:24:36.887 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:36.883+0000 7fc97bdea640 1 --2- 172.21.15.82:0/2861189629 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc97410f020 0x7fc97410f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:24:36.887 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:36.883+0000 7fc97bdea640 1 --2- 172.21.15.82:0/2861189629 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc9741093d0 0x7fc974109dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:24:36.887 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:36.883+0000 7fc97bdea640 1 -- 172.21.15.82:0/2861189629 >> 172.21.15.82:0/2861189629 conn(0x7fc97406f4f0 msgr2=0x7fc974071980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:24:36.887 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:36.887+0000 7fc97bdea640 1 -- 172.21.15.82:0/2861189629 shutdown_connections 2024-05-20T16:24:36.887 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:36.887+0000 7fc97bdea640 1 -- 172.21.15.82:0/2861189629 wait complete. 2024-05-20T16:24:36.898 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:36.888+0000 7fc97bdea640 1 Processor -- start 2024-05-20T16:24:36.898 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:36.888+0000 7fc97bdea640 1 -- start start 2024-05-20T16:24:36.898 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:36.889+0000 7fc97bdea640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc9741093d0 0x7fc974117f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:24:36.898 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:36.889+0000 7fc97bdea640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc97410a300 0x7fc974118490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:24:36.898 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:36.889+0000 7fc97bdea640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc97410f020 0x7fc974119430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:24:36.899 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:36.889+0000 7fc97bdea640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fc97411d620 con 0x7fc9741093d0 2024-05-20T16:24:36.899 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:36.889+0000 7fc97bdea640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fc97411d4a0 con 0x7fc97410a300 2024-05-20T16:24:36.899 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:36.889+0000 7fc97bdea640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fc97411d7a0 con 0x7fc97410f020 2024-05-20T16:24:36.899 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:36.889+0000 7fc979385640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc97410a300 0x7fc974118490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:24:36.899 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:36.890+0000 7fc979385640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc97410a300 0x7fc974118490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.82:58428/0 (socket says 172.21.15.82:58428) 2024-05-20T16:24:36.899 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:36.890+0000 7fc979385640 1 -- 172.21.15.82:0/1987666557 learned_addr learned my addr 172.21.15.82:0/1987666557 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:24:36.899 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:36.890+0000 7fc97a387640 1 --2- 172.21.15.82:0/1987666557 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc97410f020 0x7fc974119430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:24:36.899 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:36.894+0000 7fc979b86640 1 --2- 172.21.15.82:0/1987666557 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc9741093d0 0x7fc974117f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:24:36.902 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:36.899+0000 7fc979385640 1 -- 172.21.15.82:0/1987666557 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc97410f020 msgr2=0x7fc974119430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:24:36.902 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:36.899+0000 7fc979385640 1 --2- 172.21.15.82:0/1987666557 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc97410f020 0x7fc974119430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:24:36.902 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:36.899+0000 7fc979385640 1 -- 172.21.15.82:0/1987666557 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc9741093d0 msgr2=0x7fc974117f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:24:36.902 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:36.899+0000 7fc979385640 1 --2- 172.21.15.82:0/1987666557 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc9741093d0 0x7fc974117f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:24:36.902 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:36.899+0000 7fc979385640 1 -- 172.21.15.82:0/1987666557 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc974119aa0 con 0x7fc97410a300 2024-05-20T16:24:36.903 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:36.900+0000 7fc979b86640 1 --2- 172.21.15.82:0/1987666557 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc9741093d0 0x7fc974117f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T16:24:36.903 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:36.900+0000 7fc97a387640 1 --2- 172.21.15.82:0/1987666557 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc97410f020 0x7fc974119430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T16:24:36.903 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:36.900+0000 7fc979385640 1 --2- 172.21.15.82:0/1987666557 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc97410a300 0x7fc974118490 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fc964033040 tx=0x7fc964005230 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:24:36.903 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:36.900+0000 7fc96affd640 1 -- 172.21.15.82:0/1987666557 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc964040030 con 0x7fc97410a300 2024-05-20T16:24:36.903 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:36.901+0000 7fc96affd640 1 -- 172.21.15.82:0/1987666557 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc964003df0 con 0x7fc97410a300 2024-05-20T16:24:36.903 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:36.901+0000 7fc96affd640 1 -- 172.21.15.82:0/1987666557 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc96403ca70 con 0x7fc97410a300 2024-05-20T16:24:36.903 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:36.902+0000 7fc97bdea640 1 -- 172.21.15.82:0/1987666557 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc974119d30 con 0x7fc97410a300 2024-05-20T16:24:36.903 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:36.902+0000 7fc97bdea640 1 -- 172.21.15.82:0/1987666557 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc97411a1d0 con 0x7fc97410a300 2024-05-20T16:24:36.911 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:36.910+0000 7fc982d6b740 1 Processor -- start 2024-05-20T16:24:36.911 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:36.910+0000 7fc96affd640 1 -- 172.21.15.82:0/1987666557 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fc964043030 con 0x7fc97410a300 2024-05-20T16:24:36.914 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:36.912+0000 7fc96affd640 1 --2- 172.21.15.82:0/1987666557 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fc9600a2c80 0x7fc9600a5190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:24:36.914 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:36.912+0000 7fc96affd640 1 -- 172.21.15.82:0/1987666557 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9150+0+0 (secure 0 0 0) 0x7fc9640efe00 con 0x7fc97410a300 2024-05-20T16:24:36.914 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:36.912+0000 7fc982d6b740 1 -- start start 2024-05-20T16:24:36.914 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:36.913+0000 7fc982d6b740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56480e4451b0 0x56480e7e4e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:24:36.914 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:36.913+0000 7fc982d6b740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56480e41c2f0 0x56480e7e53e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:24:36.914 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:36.913+0000 7fc982d6b740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56480e4244d0 0x56480e7ec2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:24:36.914 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:36.913+0000 7fc982d6b740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x56480e749340 con 0x56480e4244d0 2024-05-20T16:24:36.914 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:36.913+0000 7fc982d6b740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x56480e6f1290 con 0x56480e41c2f0 2024-05-20T16:24:36.914 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:36.913+0000 7fc982d6b740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x56480e745300 con 0x56480e4451b0 2024-05-20T16:24:36.914 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:36.914+0000 7fc97a387640 1 --2- 172.21.15.82:0/1987666557 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fc9600a2c80 0x7fc9600a5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:24:36.915 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:36.914+0000 7fc979b86640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56480e4451b0 0x56480e7e4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:24:36.915 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:36.914+0000 7fc979b86640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56480e4451b0 0x56480e7e4e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.82:54286/0 (socket says 172.21.15.82:54286) 2024-05-20T16:24:36.915 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:36.915+0000 7fc979b86640 1 -- 172.21.15.82:0/2432761347 learned_addr learned my addr 172.21.15.82:0/2432761347 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:24:36.918 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:36.915+0000 7fc979385640 1 --2- 172.21.15.82:0/2432761347 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56480e41c2f0 0x56480e7e53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:24:36.919 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:36.916+0000 7fc97a387640 1 --2- 172.21.15.82:0/2432761347 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56480e4244d0 0x56480e7ec2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:24:36.919 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:36.919+0000 7fc979b86640 1 -- 172.21.15.82:0/2432761347 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56480e41c2f0 msgr2=0x56480e7e53e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:24:36.919 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:36.919+0000 7fc979b86640 1 --2- 172.21.15.82:0/2432761347 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56480e41c2f0 0x56480e7e53e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:24:36.919 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:36.919+0000 7fc979b86640 1 -- 172.21.15.82:0/2432761347 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56480e4244d0 msgr2=0x56480e7ec2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:24:36.919 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:36.919+0000 7fc979b86640 1 --2- 172.21.15.82:0/2432761347 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56480e4244d0 0x56480e7ec2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:24:36.919 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:36.919+0000 7fc979b86640 1 -- 172.21.15.82:0/2432761347 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56480e7158e0 con 0x56480e4451b0 2024-05-20T16:24:36.919 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:36.919+0000 7fc97a387640 1 --2- 172.21.15.82:0/1987666557 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fc9600a2c80 0x7fc9600a5190 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fc96c005fd0 tx=0x7fc96c014790 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:24:36.921 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:36.921+0000 7fc979b86640 1 --2- 172.21.15.82:0/2432761347 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56480e4451b0 0x56480e7e4e60 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fc97000bb10 tx=0x7fc97000d340 comp rx=0 tx=0).ready entity=mon.1 client_cookie=12166810b1bada79 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:24:36.921 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:36.922+0000 7fc95ffff640 1 -- 172.21.15.82:0/2432761347 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc970017420 con 0x56480e4451b0 2024-05-20T16:24:36.921 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:36.922+0000 7fc95ffff640 1 -- 172.21.15.82:0/2432761347 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc970017aa0 con 0x56480e4451b0 2024-05-20T16:24:36.926 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:36.926+0000 7fc95ffff640 1 -- 172.21.15.82:0/2432761347 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc97000fb00 con 0x56480e4451b0 2024-05-20T16:24:36.926 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:36.927+0000 7fc982d6b740 1 -- 172.21.15.82:0/2432761347 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56480e4451b0 msgr2=0x56480e7e4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:24:36.926 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:36.927+0000 7fc982d6b740 1 --2- 172.21.15.82:0/2432761347 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56480e4451b0 0x56480e7e4e60 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fc97000bb10 tx=0x7fc97000d340 comp rx=0 tx=0).stop 2024-05-20T16:24:36.931 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:36.932+0000 7fc982d6b740 1 -- 172.21.15.82:0/2432761347 shutdown_connections 2024-05-20T16:24:36.931 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:36.932+0000 7fc982d6b740 1 --2- 172.21.15.82:0/2432761347 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56480e4244d0 0x56480e7ec2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:24:36.931 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:36.932+0000 7fc982d6b740 1 --2- 172.21.15.82:0/2432761347 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56480e41c2f0 0x56480e7e53e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:24:36.931 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:36.932+0000 7fc982d6b740 1 --2- 172.21.15.82:0/2432761347 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56480e4451b0 0x56480e7e4e60 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:24:36.932 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:36.932+0000 7fc982d6b740 1 -- 172.21.15.82:0/2432761347 >> 172.21.15.82:0/2432761347 conn(0x56480e439300 msgr2=0x56480e7e07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:24:36.936 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:36.937+0000 7fc982d6b740 1 -- 172.21.15.82:0/2432761347 shutdown_connections 2024-05-20T16:24:36.939 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:36.939+0000 7fc982d6b740 1 -- 172.21.15.82:0/2432761347 wait complete. 2024-05-20T16:24:36.939 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:36.940+0000 7fc982d6b740 1 -- start start 2024-05-20T16:24:36.939 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:36.940+0000 7fc982d6b740 1 Processor -- start 2024-05-20T16:24:36.941 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:36.941+0000 7fc982d6b740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56480e41c2f0 0x56480e7eec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:24:36.941 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:36.941+0000 7fc982d6b740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56480e4244d0 0x56480e7ef140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:24:36.941 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:36.941+0000 7fc982d6b740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56480e4451b0 0x56480e7f60a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:24:36.941 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:36.941+0000 7fc982d6b740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x56480e7ecd30 con 0x56480e4244d0 2024-05-20T16:24:36.941 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:36.941+0000 7fc982d6b740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x56480e704790 con 0x56480e4451b0 2024-05-20T16:24:36.941 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:36.941+0000 7fc982d6b740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x56480e7eceb0 con 0x56480e41c2f0 2024-05-20T16:24:36.941 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:36.941+0000 7fc982d6b740 10 client.0 _ll_register_callbacks cb 0x56480e73fa20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T16:24:36.941 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:36.941+0000 7fc979b86640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56480e41c2f0 0x56480e7eec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:24:36.942 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:36.941+0000 7fc979b86640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56480e41c2f0 0x56480e7eec00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.82:54288/0 (socket says 172.21.15.82:54288) 2024-05-20T16:24:36.942 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:36.941+0000 7fc979b86640 1 -- 172.21.15.82:0/288451681 learned_addr learned my addr 172.21.15.82:0/288451681 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:24:36.942 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:36.941+0000 7fc979385640 1 --2- 172.21.15.82:0/288451681 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56480e4244d0 0x56480e7ef140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:24:36.942 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:36.941+0000 7fc97a387640 1 --2- 172.21.15.82:0/288451681 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56480e4451b0 0x56480e7f60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:24:36.943 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:36.942+0000 7fc979b86640 1 -- 172.21.15.82:0/288451681 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56480e4451b0 msgr2=0x56480e7f60a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:24:36.943 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:36.942+0000 7fc979b86640 1 --2- 172.21.15.82:0/288451681 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56480e4451b0 0x56480e7f60a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:24:36.943 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:36.942+0000 7fc979b86640 1 -- 172.21.15.82:0/288451681 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56480e4244d0 msgr2=0x56480e7ef140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:24:36.943 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:36.942+0000 7fc979b86640 1 --2- 172.21.15.82:0/288451681 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56480e4244d0 0x56480e7ef140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:24:36.943 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:36.942+0000 7fc979b86640 1 -- 172.21.15.82:0/288451681 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56480e45d210 con 0x56480e41c2f0 2024-05-20T16:24:36.943 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:36.942+0000 7fc979b86640 1 --2- 172.21.15.82:0/288451681 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56480e41c2f0 0x56480e7eec00 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fc97000bfd0 tx=0x7fc97000df80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c67ba5fca5b18c92 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:24:36.943 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:36.942+0000 7fc95d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-05-20T16:24:36.943 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:36.942+0000 7fc95d7fa640 1 -- 172.21.15.82:0/288451681 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc970027020 con 0x56480e41c2f0 2024-05-20T16:24:36.944 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:36.943+0000 7fc982d6b740 1 -- 172.21.15.82:0/288451681 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc970017aa0 con 0x56480e41c2f0 2024-05-20T16:24:36.944 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:36.943+0000 7fc95d7fa640 1 -- 172.21.15.82:0/288451681 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9150+0+0 (secure 0 0 0) 0x7fc970017630 con 0x56480e41c2f0 2024-05-20T16:24:36.944 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:36.943+0000 7fc95d7fa640 1 -- 172.21.15.82:0/288451681 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fc970032050 con 0x56480e41c2f0 2024-05-20T16:24:36.944 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:36.943+0000 7fc982d6b740 10 client.25498 fetch_fsmap learned FSMap version 29 2024-05-20T16:24:36.944 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:36.943+0000 7fc982d6b740 1 -- 172.21.15.82:0/288451681 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56480e529620 con 0x56480e41c2f0 2024-05-20T16:24:36.944 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:36.945+0000 7fc95d7fa640 1 -- 172.21.15.82:0/288451681 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc970017aa0 con 0x56480e41c2f0 2024-05-20T16:24:36.944 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:36.945+0000 7fc95d7fa640 1 -- 172.21.15.82:0/288451681 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7fc970023a20 con 0x56480e41c2f0 2024-05-20T16:24:36.945 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:36.945+0000 7fc982d6b740 10 client.25498 fetch_fsmap finished waiting for FSMap version 29 2024-05-20T16:24:36.945 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:36.945+0000 7fc982d6b740 10 client.25498 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T16:24:36.945 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:36.945+0000 7fc982d6b740 20 client.25498 populate_metadata read hostname 'smithi082' 2024-05-20T16:24:36.945 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:36.945+0000 7fc982d6b740 1 --2- 172.21.15.82:0/288451681 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x56480e403d80 0x56480e7fca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:24:36.945 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:36.945+0000 7fc982d6b740 4 client.25498 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T16:24:36.945 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:36.945+0000 7fc982d6b740 1 -- 172.21.15.82:0/288451681 --> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56480e756a70 con 0x56480e403d80 2024-05-20T16:24:36.946 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:36.947+0000 7fc979385640 1 --2- 172.21.15.82:0/288451681 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x56480e403d80 0x56480e7fca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:24:36.951 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:36.951+0000 7fc979385640 1 --2- 172.21.15.82:0/288451681 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x56480e403d80 0x56480e7fca30 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=70caaa9ecfe66ae7 server_cookie=84c03965bcf1db8d in_seq=0 out_seq=0 2024-05-20T16:24:36.951 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:36.951+0000 7fc95d7fa640 10 client.25498 ms_handle_connect on v2:172.21.15.160:6838/2765554482 2024-05-20T16:24:36.952 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:36.953+0000 7fc95d7fa640 1 -- 172.21.15.82:0/288451681 <== mds.0 v2:172.21.15.160:6838/2765554482 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56480e756a70 con 0x56480e403d80 2024-05-20T16:24:36.953 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:36.953+0000 7fc95d7fa640 10 client.25498 handle_command_reply: tid=0 2024-05-20T16:24:36.953 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:36.953+0000 7fc982d6b740 1 client.25498 shutdown 2024-05-20T16:24:36.962 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:36.963+0000 7fc982d6b740 1 -- 172.21.15.82:0/288451681 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56480e41c2f0 msgr2=0x56480e7eec00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:24:36.962 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:36.963+0000 7fc982d6b740 1 --2- 172.21.15.82:0/288451681 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56480e41c2f0 0x56480e7eec00 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fc97000bfd0 tx=0x7fc97000df80 comp rx=0 tx=0).stop 2024-05-20T16:24:36.966 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:36.964+0000 7fc982d6b740 1 -- 172.21.15.82:0/288451681 shutdown_connections 2024-05-20T16:24:36.966 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:36.964+0000 7fc982d6b740 1 --2- 172.21.15.82:0/288451681 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x56480e403d80 0x56480e7fca30 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:24:36.966 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:36.964+0000 7fc982d6b740 1 --2- 172.21.15.82:0/288451681 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56480e4244d0 0x56480e7ef140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:24:36.966 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:36.964+0000 7fc982d6b740 1 --2- 172.21.15.82:0/288451681 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56480e4451b0 0x56480e7f60a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:24:36.967 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:36.964+0000 7fc982d6b740 1 --2- 172.21.15.82:0/288451681 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56480e41c2f0 0x56480e7eec00 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:24:36.967 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:36.964+0000 7fc982d6b740 1 -- 172.21.15.82:0/288451681 >> 172.21.15.82:0/288451681 conn(0x56480e439300 msgr2=0x56480e7df810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:24:36.967 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:36.964+0000 7fc95d7fa640 0 client.25498 ms_handle_reset on v2:172.21.15.160:6838/2765554482 2024-05-20T16:24:36.967 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:36.964+0000 7fc982d6b740 1 -- 172.21.15.82:0/288451681 shutdown_connections 2024-05-20T16:24:36.967 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:36.964+0000 7fc982d6b740 1 -- 172.21.15.82:0/288451681 wait complete. 2024-05-20T16:24:36.969 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:36.964+0000 7fc982d6b740 20 client.25498 trim_cache size 0 max 16384 2024-05-20T16:24:37.002 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:37.002+0000 7fc982d6b740 1 Processor -- start 2024-05-20T16:24:37.002 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:37.002+0000 7fc982d6b740 1 -- start start 2024-05-20T16:24:37.003 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:37.003+0000 7fc982d6b740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56480e41c2f0 0x56480e7e08b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:24:37.003 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:37.003+0000 7fc982d6b740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56480e4244d0 0x56480e7f5fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:24:37.003 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:37.003+0000 7fc982d6b740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56480e4451b0 0x56480e7f6500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:24:37.003 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:37.003+0000 7fc982d6b740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fc958012550 con 0x56480e4244d0 2024-05-20T16:24:37.003 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:37.003+0000 7fc979b86640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56480e41c2f0 0x56480e7e08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:24:37.003 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:37.003+0000 7fc979b86640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56480e41c2f0 0x56480e7e08b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.82:58472/0 (socket says 172.21.15.82:58472) 2024-05-20T16:24:37.004 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:37.003+0000 7fc979b86640 1 -- 172.21.15.82:0/1152843864 learned_addr learned my addr 172.21.15.82:0/1152843864 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:24:37.004 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:37.003+0000 7fc97a387640 1 --2- 172.21.15.82:0/1152843864 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56480e4451b0 0x56480e7f6500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:24:37.004 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:37.004+0000 7fc982d6b740 1 -- 172.21.15.82:0/1152843864 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fc95800fe20 con 0x56480e41c2f0 2024-05-20T16:24:37.004 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:37.004+0000 7fc982d6b740 1 -- 172.21.15.82:0/1152843864 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fc95800dc70 con 0x56480e4451b0 2024-05-20T16:24:37.005 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:37.006+0000 7fc979b86640 1 -- 172.21.15.82:0/1152843864 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56480e4451b0 msgr2=0x56480e7f6500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:24:37.005 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:37.006+0000 7fc979b86640 1 --2- 172.21.15.82:0/1152843864 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56480e4451b0 0x56480e7f6500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:24:37.005 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:37.006+0000 7fc979b86640 1 -- 172.21.15.82:0/1152843864 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56480e4244d0 msgr2=0x56480e7f5fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:24:37.006 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:37.006+0000 7fc979b86640 1 --2- 172.21.15.82:0/1152843864 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56480e4244d0 0x56480e7f5fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:24:37.006 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:37.006+0000 7fc979b86640 1 -- 172.21.15.82:0/1152843864 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56480e7ef800 con 0x56480e41c2f0 2024-05-20T16:24:37.009 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:37.009+0000 7fc979b86640 1 --2- 172.21.15.82:0/1152843864 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56480e41c2f0 0x56480e7e08b0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fc97000e010 tx=0x7fc97000b400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1575a58867b70a9f server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:24:37.010 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:37.011+0000 7fc95f7fe640 1 -- 172.21.15.82:0/1152843864 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc970027040 con 0x56480e41c2f0 2024-05-20T16:24:37.011 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:37.011+0000 7fc95f7fe640 1 -- 172.21.15.82:0/1152843864 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc970021d70 con 0x56480e41c2f0 2024-05-20T16:24:37.011 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:37.011+0000 7fc95f7fe640 1 -- 172.21.15.82:0/1152843864 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9700237e0 con 0x56480e41c2f0 2024-05-20T16:24:37.014 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:37.013+0000 7fc982d6b740 1 -- 172.21.15.82:0/1152843864 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56480e41c2f0 msgr2=0x56480e7e08b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:24:37.014 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:37.013+0000 7fc982d6b740 1 --2- 172.21.15.82:0/1152843864 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56480e41c2f0 0x56480e7e08b0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fc97000e010 tx=0x7fc97000b400 comp rx=0 tx=0).stop 2024-05-20T16:24:37.014 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:37.013+0000 7fc982d6b740 1 -- 172.21.15.82:0/1152843864 shutdown_connections 2024-05-20T16:24:37.014 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:37.013+0000 7fc982d6b740 1 --2- 172.21.15.82:0/1152843864 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56480e4244d0 0x56480e7f5fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:24:37.014 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:37.013+0000 7fc982d6b740 1 --2- 172.21.15.82:0/1152843864 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56480e4451b0 0x56480e7f6500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:24:37.014 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:37.013+0000 7fc982d6b740 1 --2- 172.21.15.82:0/1152843864 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56480e41c2f0 0x56480e7e08b0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:24:37.014 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:37.013+0000 7fc982d6b740 1 -- 172.21.15.82:0/1152843864 >> 172.21.15.82:0/1152843864 conn(0x56480e439300 msgr2=0x56480e7e01a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:24:37.021 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:37.020+0000 7fc982d6b740 1 -- 172.21.15.82:0/1152843864 shutdown_connections 2024-05-20T16:24:37.022 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:37.022+0000 7fc982d6b740 1 -- 172.21.15.82:0/1152843864 wait complete. 2024-05-20T16:24:37.022 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:37.022+0000 7fc982d6b740 1 -- start start 2024-05-20T16:24:37.022 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:37.023+0000 7fc982d6b740 1 Processor -- start 2024-05-20T16:24:37.023 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:37.023+0000 7fc982d6b740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56480e41c2f0 0x56480e7f0f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:24:37.023 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:37.023+0000 7fc982d6b740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56480e4244d0 0x56480e7f14a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:24:37.023 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:37.023+0000 7fc982d6b740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56480e4451b0 0x56480e7f6380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:24:37.023 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:37.023+0000 7fc982d6b740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fc95800e2e0 con 0x56480e4451b0 2024-05-20T16:24:37.023 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:37.023+0000 7fc982d6b740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fc95800f750 con 0x56480e4244d0 2024-05-20T16:24:37.023 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:37.023+0000 7fc982d6b740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fc95800e9b0 con 0x56480e41c2f0 2024-05-20T16:24:37.024 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:37.023+0000 7fc982d6b740 10 client.0 _ll_register_callbacks cb 0x7fc96c014630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T16:24:37.024 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:37.024+0000 7fc979b86640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56480e41c2f0 0x56480e7f0f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:24:37.024 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:37.024+0000 7fc979b86640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56480e41c2f0 0x56480e7f0f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.82:54310/0 (socket says 172.21.15.82:54310) 2024-05-20T16:24:37.024 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:37.024+0000 7fc979b86640 1 -- 172.21.15.82:0/3264734477 learned_addr learned my addr 172.21.15.82:0/3264734477 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:24:37.025 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:37.025+0000 7fc979b86640 1 -- 172.21.15.82:0/3264734477 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56480e4244d0 msgr2=0x56480e7f14a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-20T16:24:37.025 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:37.025+0000 7fc979b86640 1 --2- 172.21.15.82:0/3264734477 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56480e4244d0 0x56480e7f14a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:24:37.025 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:37.025+0000 7fc979b86640 1 -- 172.21.15.82:0/3264734477 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56480e4451b0 msgr2=0x56480e7f6380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:24:37.025 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:37.025+0000 7fc979b86640 1 --2- 172.21.15.82:0/3264734477 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56480e4451b0 0x56480e7f6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:24:37.025 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:37.025+0000 7fc979b86640 1 -- 172.21.15.82:0/3264734477 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56480e7f6940 con 0x56480e41c2f0 2024-05-20T16:24:37.029 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:37.030+0000 7fc979b86640 1 --2- 172.21.15.82:0/3264734477 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56480e41c2f0 0x56480e7f0f60 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fc97000bfd0 tx=0x7fc970017840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=94c8df6acc66ba17 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:24:37.037 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:37.034+0000 7fc95d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-05-20T16:24:37.037 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:37.035+0000 7fc95d7fa640 1 -- 172.21.15.82:0/3264734477 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc970044060 con 0x56480e41c2f0 2024-05-20T16:24:37.037 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:37.035+0000 7fc95d7fa640 1 -- 172.21.15.82:0/3264734477 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9150+0+0 (secure 0 0 0) 0x7fc970021a50 con 0x56480e41c2f0 2024-05-20T16:24:37.037 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:37.036+0000 7fc982d6b740 1 -- 172.21.15.82:0/3264734477 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc970021d70 con 0x56480e41c2f0 2024-05-20T16:24:37.045 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:37.043+0000 7fc95d7fa640 1 -- 172.21.15.82:0/3264734477 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fc970027040 con 0x56480e41c2f0 2024-05-20T16:24:37.045 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:37.045+0000 7fc982d6b740 10 client.25504 fetch_fsmap learned FSMap version 29 2024-05-20T16:24:37.045 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:37.045+0000 7fc982d6b740 1 -- 172.21.15.82:0/3264734477 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56480e863bf0 con 0x56480e41c2f0 2024-05-20T16:24:37.052 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:37.053+0000 7fc95d7fa640 1 -- 172.21.15.82:0/3264734477 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc970021d70 con 0x56480e41c2f0 2024-05-20T16:24:37.052 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:37.053+0000 7fc95d7fa640 1 -- 172.21.15.82:0/3264734477 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7fc970004ae0 con 0x56480e41c2f0 2024-05-20T16:24:37.055 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:37.056+0000 7fc982d6b740 10 client.25504 fetch_fsmap finished waiting for FSMap version 29 2024-05-20T16:24:37.056 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:37.056+0000 7fc982d6b740 10 client.25504 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T16:24:37.056 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:37.056+0000 7fc982d6b740 20 client.25504 populate_metadata read hostname 'smithi082' 2024-05-20T16:24:37.056 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:37.056+0000 7fc982d6b740 1 --2- 172.21.15.82:0/3264734477 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x56480e3d91c0 0x56480e866420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:24:37.056 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:37.056+0000 7fc982d6b740 4 client.25504 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-20T16:24:37.056 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:37.056+0000 7fc982d6b740 1 -- 172.21.15.82:0/3264734477 --> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56480e750dc0 con 0x56480e3d91c0 2024-05-20T16:24:37.058 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:37.058+0000 7fc979385640 1 --2- 172.21.15.82:0/3264734477 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x56480e3d91c0 0x56480e866420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:24:37.072 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:37.072+0000 7fc979385640 1 --2- 172.21.15.82:0/3264734477 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x56480e3d91c0 0x56480e866420 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a51b7ee9cbe6e28b server_cookie=bbe0df206130eecd in_seq=0 out_seq=0 2024-05-20T16:24:37.072 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:37.073+0000 7fc95d7fa640 10 client.25504 ms_handle_connect on v2:172.21.15.160:6838/2765554482 2024-05-20T16:24:37.314 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:24:36 smithi160 ceph-mon[36347]: pgmap v682: 129 pgs: 129 active+clean; 10 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 667 MiB/s wr, 1.20k op/s 2024-05-20T16:24:37.362 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:24:36 smithi082 ceph-mon[32238]: pgmap v682: 129 pgs: 129 active+clean; 10 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 667 MiB/s wr, 1.20k op/s 2024-05-20T16:24:37.385 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:24:36 smithi184 ceph-mon[36209]: pgmap v682: 129 pgs: 129 active+clean; 10 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 667 MiB/s wr, 1.20k op/s 2024-05-20T16:24:39.312 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:24:38 smithi160 ceph-mon[36347]: pgmap v683: 129 pgs: 129 active+clean; 10 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 715 MiB/s wr, 1.29k op/s 2024-05-20T16:24:39.358 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:24:38 smithi082 ceph-mon[32238]: pgmap v683: 129 pgs: 129 active+clean; 10 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 715 MiB/s wr, 1.29k op/s 2024-05-20T16:24:39.385 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:24:38 smithi184 ceph-mon[36209]: pgmap v683: 129 pgs: 129 active+clean; 10 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 715 MiB/s wr, 1.29k op/s 2024-05-20T16:24:41.023 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:41.022+0000 7fc95d7fa640 1 -- 172.21.15.82:0/3264734477 <== mds.0 v2:172.21.15.160:6838/2765554482 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56480e750dc0 con 0x56480e3d91c0 2024-05-20T16:24:41.024 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:41.022+0000 7fc95d7fa640 10 client.25504 handle_command_reply: tid=0 2024-05-20T16:24:41.024 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:41.022+0000 7fc982d6b740 1 client.25504 shutdown 2024-05-20T16:24:41.024 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:41.022+0000 7fc982d6b740 1 -- 172.21.15.82:0/3264734477 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56480e41c2f0 msgr2=0x56480e7f0f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:24:41.024 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:41.022+0000 7fc982d6b740 1 --2- 172.21.15.82:0/3264734477 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56480e41c2f0 0x56480e7f0f60 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fc97000bfd0 tx=0x7fc970017840 comp rx=0 tx=0).stop 2024-05-20T16:24:41.024 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:41.022+0000 7fc982d6b740 1 -- 172.21.15.82:0/3264734477 shutdown_connections 2024-05-20T16:24:41.025 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:41.022+0000 7fc982d6b740 1 --2- 172.21.15.82:0/3264734477 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x56480e3d91c0 0x56480e866420 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:24:41.025 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:41.022+0000 7fc982d6b740 1 --2- 172.21.15.82:0/3264734477 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56480e4451b0 0x56480e7f6380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:24:41.025 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:41.022+0000 7fc982d6b740 1 --2- 172.21.15.82:0/3264734477 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56480e4244d0 0x56480e7f14a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:24:41.025 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:41.022+0000 7fc982d6b740 1 --2- 172.21.15.82:0/3264734477 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56480e41c2f0 0x56480e7f0f60 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:24:41.025 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:41.022+0000 7fc982d6b740 1 -- 172.21.15.82:0/3264734477 >> 172.21.15.82:0/3264734477 conn(0x56480e439300 msgr2=0x56480e7ee7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:24:41.026 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:41.023+0000 7fc95d7fa640 0 client.25504 ms_handle_reset on v2:172.21.15.160:6838/2765554482 2024-05-20T16:24:41.026 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:41.023+0000 7fc982d6b740 1 -- 172.21.15.82:0/3264734477 shutdown_connections 2024-05-20T16:24:41.026 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:41.023+0000 7fc982d6b740 1 -- 172.21.15.82:0/3264734477 wait complete. 2024-05-20T16:24:41.026 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:41.023+0000 7fc982d6b740 20 client.25504 trim_cache size 0 max 16384 2024-05-20T16:24:41.027 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:{ 2024-05-20T16:24:41.027 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "message": "", 2024-05-20T16:24:41.027 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "return_code": 0 2024-05-20T16:24:41.027 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:} 2024-05-20T16:24:41.029 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:41.025+0000 7fc97bdea640 1 -- 172.21.15.82:0/1987666557 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fc9600a2c80 msgr2=0x7fc9600a5190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:24:41.029 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:41.025+0000 7fc97bdea640 1 --2- 172.21.15.82:0/1987666557 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fc9600a2c80 0x7fc9600a5190 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fc96c005fd0 tx=0x7fc96c014790 comp rx=0 tx=0).stop 2024-05-20T16:24:41.029 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:41.025+0000 7fc97bdea640 1 -- 172.21.15.82:0/1987666557 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc97410a300 msgr2=0x7fc974118490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:24:41.029 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:41.025+0000 7fc97bdea640 1 --2- 172.21.15.82:0/1987666557 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc97410a300 0x7fc974118490 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fc964033040 tx=0x7fc964005230 comp rx=0 tx=0).stop 2024-05-20T16:24:41.029 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:41.025+0000 7fc97bdea640 1 -- 172.21.15.82:0/1987666557 shutdown_connections 2024-05-20T16:24:41.030 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:41.025+0000 7fc97bdea640 1 --2- 172.21.15.82:0/1987666557 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fc9600a2c80 0x7fc9600a5190 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:24:41.030 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:41.025+0000 7fc97bdea640 1 --2- 172.21.15.82:0/1987666557 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc97410f020 0x7fc974119430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:24:41.030 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:41.025+0000 7fc97bdea640 1 --2- 172.21.15.82:0/1987666557 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc97410a300 0x7fc974118490 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:24:41.030 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:41.025+0000 7fc97bdea640 1 --2- 172.21.15.82:0/1987666557 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc9741093d0 0x7fc974117f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:24:41.030 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:41.025+0000 7fc97bdea640 1 -- 172.21.15.82:0/1987666557 >> 172.21.15.82:0/1987666557 conn(0x7fc97406f4f0 msgr2=0x7fc97410d010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:24:41.030 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:41.025+0000 7fc97bdea640 1 -- 172.21.15.82:0/1987666557 shutdown_connections 2024-05-20T16:24:41.031 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:41.025+0000 7fc97bdea640 1 -- 172.21.15.82:0/1987666557 wait complete. 2024-05-20T16:24:41.312 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:24:40 smithi160 ceph-mon[36347]: pgmap v684: 129 pgs: 129 active+clean; 10 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 790 MiB/s wr, 1.71k op/s 2024-05-20T16:24:41.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:24:40 smithi082 ceph-mon[32238]: pgmap v684: 129 pgs: 129 active+clean; 10 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 790 MiB/s wr, 1.71k op/s 2024-05-20T16:24:41.385 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:24:40 smithi184 ceph-mon[36209]: pgmap v684: 129 pgs: 129 active+clean; 10 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 790 MiB/s wr, 1.71k op/s 2024-05-20T16:24:43.312 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:24:42 smithi160 ceph-mon[36347]: pgmap v685: 129 pgs: 129 active+clean; 10 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 576 MiB/s wr, 1.55k op/s 2024-05-20T16:24:43.360 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:24:42 smithi082 ceph-mon[32238]: pgmap v685: 129 pgs: 129 active+clean; 10 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 576 MiB/s wr, 1.55k op/s 2024-05-20T16:24:43.385 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:24:42 smithi184 ceph-mon[36209]: pgmap v685: 129 pgs: 129 active+clean; 10 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 576 MiB/s wr, 1.55k op/s 2024-05-20T16:24:45.312 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:24:44 smithi160 ceph-mon[36347]: pgmap v686: 129 pgs: 129 active+clean; 10 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 683 MiB/s wr, 1.60k op/s 2024-05-20T16:24:45.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:24:44 smithi082 ceph-mon[32238]: pgmap v686: 129 pgs: 129 active+clean; 10 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 683 MiB/s wr, 1.60k op/s 2024-05-20T16:24:45.385 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:24:44 smithi184 ceph-mon[36209]: pgmap v686: 129 pgs: 129 active+clean; 10 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 683 MiB/s wr, 1.60k op/s 2024-05-20T16:24:47.311 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:24:46 smithi160 ceph-mon[36347]: pgmap v687: 129 pgs: 129 active+clean; 10 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 656 MiB/s wr, 1.45k op/s 2024-05-20T16:24:47.360 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:24:46 smithi082 ceph-mon[32238]: pgmap v687: 129 pgs: 129 active+clean; 10 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 656 MiB/s wr, 1.45k op/s 2024-05-20T16:24:47.385 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:24:46 smithi184 ceph-mon[36209]: pgmap v687: 129 pgs: 129 active+clean; 10 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 656 MiB/s wr, 1.45k op/s 2024-05-20T16:24:48.312 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:24:47 smithi160 ceph-mon[36347]: pgmap v688: 129 pgs: 129 active+clean; 10 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 620 MiB/s wr, 1.32k op/s 2024-05-20T16:24:48.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:24:47 smithi082 ceph-mon[32238]: pgmap v688: 129 pgs: 129 active+clean; 10 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 620 MiB/s wr, 1.32k op/s 2024-05-20T16:24:48.386 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:24:47 smithi184 ceph-mon[36209]: pgmap v688: 129 pgs: 129 active+clean; 10 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 620 MiB/s wr, 1.32k op/s 2024-05-20T16:24:51.312 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:24:50 smithi160 ceph-mon[36347]: pgmap v689: 129 pgs: 129 active+clean; 10 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 744 MiB/s wr, 1.74k op/s 2024-05-20T16:24:51.363 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:24:50 smithi082 ceph-mon[32238]: pgmap v689: 129 pgs: 129 active+clean; 10 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 744 MiB/s wr, 1.74k op/s 2024-05-20T16:24:51.385 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:24:50 smithi184 ceph-mon[36209]: pgmap v689: 129 pgs: 129 active+clean; 10 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 744 MiB/s wr, 1.74k op/s 2024-05-20T16:24:53.312 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:24:52 smithi160 ceph-mon[36347]: pgmap v690: 129 pgs: 129 active+clean; 10 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 600 MiB/s wr, 1.26k op/s 2024-05-20T16:24:53.361 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:24:52 smithi082 ceph-mon[32238]: pgmap v690: 129 pgs: 129 active+clean; 10 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 600 MiB/s wr, 1.26k op/s 2024-05-20T16:24:53.385 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:24:52 smithi184 ceph-mon[36209]: pgmap v690: 129 pgs: 129 active+clean; 10 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 600 MiB/s wr, 1.26k op/s 2024-05-20T16:24:54.262 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:54.262+0000 7fdc3cdaa640 1 -- 172.21.15.82:0/3336133975 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdc3810f020 msgr2=0x7fdc3810f430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:24:54.262 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:54.262+0000 7fdc3cdaa640 1 --2- 172.21.15.82:0/3336133975 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdc3810f020 0x7fdc3810f430 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fdc30009ca0 tx=0x7fdc3002a8f0 comp rx=0 tx=0).stop 2024-05-20T16:24:54.262 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:54.263+0000 7fdc3cdaa640 1 -- 172.21.15.82:0/3336133975 shutdown_connections 2024-05-20T16:24:54.263 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:54.263+0000 7fdc3cdaa640 1 --2- 172.21.15.82:0/3336133975 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdc3810f020 0x7fdc3810f430 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:24:54.263 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:54.263+0000 7fdc3cdaa640 1 --2- 172.21.15.82:0/3336133975 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdc3810a300 0x7fdc3810a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:24:54.263 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:54.263+0000 7fdc3cdaa640 1 --2- 172.21.15.82:0/3336133975 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdc381093d0 0x7fdc38109dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:24:54.263 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:54.263+0000 7fdc3cdaa640 1 -- 172.21.15.82:0/3336133975 >> 172.21.15.82:0/3336133975 conn(0x7fdc3806f4f0 msgr2=0x7fdc38071980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:24:54.275 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:54.276+0000 7fdc3cdaa640 1 -- 172.21.15.82:0/3336133975 shutdown_connections 2024-05-20T16:24:54.279 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:54.278+0000 7fdc3cdaa640 1 -- 172.21.15.82:0/3336133975 wait complete. 2024-05-20T16:24:54.279 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:54.279+0000 7fdc3cdaa640 1 Processor -- start 2024-05-20T16:24:54.279 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:54.279+0000 7fdc3cdaa640 1 -- start start 2024-05-20T16:24:54.279 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:54.280+0000 7fdc3cdaa640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdc381093d0 0x7fdc381a81a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:24:54.279 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:54.280+0000 7fdc3cdaa640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdc3810a300 0x7fdc381a86e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:24:54.279 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:54.280+0000 7fdc3cdaa640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdc3810f020 0x7fdc381af670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:24:54.279 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:54.280+0000 7fdc3cdaa640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fdc3811d720 con 0x7fdc3810f020 2024-05-20T16:24:54.280 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:54.280+0000 7fdc3cdaa640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fdc3811d5a0 con 0x7fdc3810a300 2024-05-20T16:24:54.280 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:54.280+0000 7fdc3cdaa640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fdc3811d8a0 con 0x7fdc381093d0 2024-05-20T16:24:54.282 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:54.282+0000 7fdc36d9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdc3810f020 0x7fdc381af670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:24:54.282 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:54.282+0000 7fdc36d9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdc3810f020 0x7fdc381af670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58538/0 (socket says 172.21.15.82:58538) 2024-05-20T16:24:54.282 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:54.282+0000 7fdc36d9d640 1 -- 172.21.15.82:0/1931290262 learned_addr learned my addr 172.21.15.82:0/1931290262 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:24:54.282 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:54.282+0000 7fdc3659c640 1 --2- 172.21.15.82:0/1931290262 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdc381093d0 0x7fdc381a81a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:24:54.285 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:54.285+0000 7fdc36d9d640 1 -- 172.21.15.82:0/1931290262 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdc381093d0 msgr2=0x7fdc381a81a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:24:54.285 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:54.285+0000 7fdc36d9d640 1 --2- 172.21.15.82:0/1931290262 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdc381093d0 0x7fdc381a81a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:24:54.285 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:54.285+0000 7fdc36d9d640 1 -- 172.21.15.82:0/1931290262 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdc3810a300 msgr2=0x7fdc381a86e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-20T16:24:54.285 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:54.285+0000 7fdc36d9d640 1 --2- 172.21.15.82:0/1931290262 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdc3810a300 0x7fdc381a86e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:24:54.285 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:54.285+0000 7fdc36d9d640 1 -- 172.21.15.82:0/1931290262 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc38110c40 con 0x7fdc3810f020 2024-05-20T16:24:54.290 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:54.289+0000 7fdc36d9d640 1 --2- 172.21.15.82:0/1931290262 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdc3810f020 0x7fdc381af670 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fdc30009430 tx=0x7fdc30003e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:24:54.290 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:54.290+0000 7fdc277fe640 1 -- 172.21.15.82:0/1931290262 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc30030c50 con 0x7fdc3810f020 2024-05-20T16:24:54.290 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:54.290+0000 7fdc3cdaa640 1 -- 172.21.15.82:0/1931290262 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc38110ed0 con 0x7fdc3810f020 2024-05-20T16:24:54.290 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:54.290+0000 7fdc3cdaa640 1 -- 172.21.15.82:0/1931290262 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc381114a0 con 0x7fdc3810f020 2024-05-20T16:24:54.290 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:54.291+0000 7fdc277fe640 1 -- 172.21.15.82:0/1931290262 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc3002d070 con 0x7fdc3810f020 2024-05-20T16:24:54.291 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:54.291+0000 7fdc277fe640 1 -- 172.21.15.82:0/1931290262 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc3003b790 con 0x7fdc3810f020 2024-05-20T16:24:54.293 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:54.294+0000 7fdc43c4f740 1 Processor -- start 2024-05-20T16:24:54.294 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:54.294+0000 7fdc43c4f740 1 -- start start 2024-05-20T16:24:54.295 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:54.296+0000 7fdc43c4f740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x565125c7d1b0 0x56512601ce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:24:54.295 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:54.296+0000 7fdc43c4f740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x565125c542f0 0x56512601d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:24:54.295 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:54.296+0000 7fdc43c4f740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x565125c5c4d0 0x5651260242e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:24:54.295 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:54.296+0000 7fdc43c4f740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x565125f81340 con 0x565125c542f0 2024-05-20T16:24:54.296 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:54.296+0000 7fdc43c4f740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x565125f29290 con 0x565125c5c4d0 2024-05-20T16:24:54.296 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:54.296+0000 7fdc43c4f740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x565125f7d300 con 0x565125c7d1b0 2024-05-20T16:24:54.299 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:54.298+0000 7fdc35d9b640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x565125c5c4d0 0x5651260242e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:24:54.299 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:54.298+0000 7fdc35d9b640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x565125c5c4d0 0x5651260242e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.82:33306/0 (socket says 172.21.15.82:33306) 2024-05-20T16:24:54.304 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:54.298+0000 7fdc35d9b640 1 -- 172.21.15.82:0/440091797 learned_addr learned my addr 172.21.15.82:0/440091797 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:24:54.304 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:54.298+0000 7fdc35d9b640 1 --2- 172.21.15.82:0/440091797 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x565125c7d1b0 0x56512601ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:24:54.304 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:54.298+0000 7fdc3659c640 1 --2- 172.21.15.82:0/440091797 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x565125c542f0 0x56512601d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:24:54.304 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:54.303+0000 7fdc35d9b640 1 -- 172.21.15.82:0/440091797 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x565125c7d1b0 msgr2=0x56512601ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:24:54.304 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:54.303+0000 7fdc35d9b640 1 --2- 172.21.15.82:0/440091797 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x565125c7d1b0 0x56512601ce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:24:54.304 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:54.303+0000 7fdc35d9b640 1 -- 172.21.15.82:0/440091797 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x565125c542f0 msgr2=0x56512601d3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:24:54.304 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:54.303+0000 7fdc35d9b640 1 --2- 172.21.15.82:0/440091797 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x565125c542f0 0x56512601d3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:24:54.304 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:54.303+0000 7fdc35d9b640 1 -- 172.21.15.82:0/440091797 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565125f4d8e0 con 0x565125c5c4d0 2024-05-20T16:24:54.304 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:54.304+0000 7fdc35d9b640 1 --2- 172.21.15.82:0/440091797 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x565125c5c4d0 0x5651260242e0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fdc28014370 tx=0x7fdc2800b0a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fd7e13304a4733ad server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:24:54.305 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:54.305+0000 7fdc24ff9640 1 -- 172.21.15.82:0/440091797 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc28011040 con 0x565125c5c4d0 2024-05-20T16:24:54.305 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:54.305+0000 7fdc24ff9640 1 -- 172.21.15.82:0/440091797 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc280065d0 con 0x565125c5c4d0 2024-05-20T16:24:54.305 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:54.305+0000 7fdc24ff9640 1 -- 172.21.15.82:0/440091797 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc28018690 con 0x565125c5c4d0 2024-05-20T16:24:54.306 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:54.307+0000 7fdc43c4f740 1 -- 172.21.15.82:0/440091797 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x565125c5c4d0 msgr2=0x5651260242e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:24:54.306 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:54.307+0000 7fdc43c4f740 1 --2- 172.21.15.82:0/440091797 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x565125c5c4d0 0x5651260242e0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fdc28014370 tx=0x7fdc2800b0a0 comp rx=0 tx=0).stop 2024-05-20T16:24:54.307 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:54.308+0000 7fdc277fe640 1 -- 172.21.15.82:0/1931290262 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fdc300305f0 con 0x7fdc3810f020 2024-05-20T16:24:54.310 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:54.309+0000 7fdc277fe640 1 --2- 172.21.15.82:0/1931290262 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fdc200a2d20 0x7fdc200a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:24:54.310 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:54.310+0000 7fdc277fe640 1 -- 172.21.15.82:0/1931290262 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9150+0+0 (secure 0 0 0) 0x7fdc300f0c90 con 0x7fdc3810f020 2024-05-20T16:24:54.310 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:54.310+0000 7fdc3659c640 1 --2- 172.21.15.82:0/440091797 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x565125c542f0 0x56512601d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T16:24:54.310 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:54.310+0000 7fdc43c4f740 1 -- 172.21.15.82:0/440091797 shutdown_connections 2024-05-20T16:24:54.310 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:54.310+0000 7fdc43c4f740 1 --2- 172.21.15.82:0/440091797 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x565125c542f0 0x56512601d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:24:54.310 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:54.310+0000 7fdc43c4f740 1 --2- 172.21.15.82:0/440091797 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x565125c5c4d0 0x5651260242e0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:24:54.310 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:54.310+0000 7fdc43c4f740 1 --2- 172.21.15.82:0/440091797 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x565125c7d1b0 0x56512601ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:24:54.310 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:54.310+0000 7fdc43c4f740 1 -- 172.21.15.82:0/440091797 >> 172.21.15.82:0/440091797 conn(0x565125c71300 msgr2=0x5651260187f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:24:54.310 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:54.310+0000 7fdc43c4f740 1 -- 172.21.15.82:0/440091797 shutdown_connections 2024-05-20T16:24:54.310 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:54.310+0000 7fdc43c4f740 1 -- 172.21.15.82:0/440091797 wait complete. 2024-05-20T16:24:54.311 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:54.311+0000 7fdc43c4f740 1 -- start start 2024-05-20T16:24:54.311 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:54.311+0000 7fdc35d9b640 1 --2- 172.21.15.82:0/1931290262 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fdc200a2d20 0x7fdc200a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:24:54.311 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:54.311+0000 7fdc43c4f740 1 Processor -- start 2024-05-20T16:24:54.312 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:54.311+0000 7fdc43c4f740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x565125c542f0 0x565126026c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:24:54.312 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:54.311+0000 7fdc43c4f740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x565125c5c4d0 0x565126027140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:24:54.312 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:54.311+0000 7fdc43c4f740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x565125c7d1b0 0x56512602e030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:24:54.312 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:54.311+0000 7fdc43c4f740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x565126024cd0 con 0x565125c5c4d0 2024-05-20T16:24:54.312 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:54.311+0000 7fdc43c4f740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x565125f3c790 con 0x565125c7d1b0 2024-05-20T16:24:54.312 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:54.311+0000 7fdc43c4f740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x565126024e50 con 0x565125c542f0 2024-05-20T16:24:54.312 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:54.311+0000 7fdc43c4f740 10 client.0 _ll_register_callbacks cb 0x565125f77a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T16:24:54.312 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:54.312+0000 7fdc3659c640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x565125c542f0 0x565126026c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:24:54.312 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:54.312+0000 7fdc3659c640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x565125c542f0 0x565126026c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.82:45550/0 (socket says 172.21.15.82:45550) 2024-05-20T16:24:54.313 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:54.312+0000 7fdc3659c640 1 -- 172.21.15.82:0/3197821446 learned_addr learned my addr 172.21.15.82:0/3197821446 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:24:54.313 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:54.312+0000 7fdc35d9b640 1 --2- 172.21.15.82:0/3197821446 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x565125c5c4d0 0x565126027140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:24:54.313 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:54.312+0000 7fdc36d9d640 1 --2- 172.21.15.82:0/3197821446 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x565125c7d1b0 0x56512602e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:24:54.313 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:54.312+0000 7fdc3659c640 1 -- 172.21.15.82:0/3197821446 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x565125c7d1b0 msgr2=0x56512602e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:24:54.314 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:54.312+0000 7fdc3659c640 1 --2- 172.21.15.82:0/3197821446 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x565125c7d1b0 0x56512602e030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:24:54.314 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:54.312+0000 7fdc3659c640 1 -- 172.21.15.82:0/3197821446 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x565125c5c4d0 msgr2=0x565126027140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:24:54.314 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:54.312+0000 7fdc3659c640 1 --2- 172.21.15.82:0/3197821446 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x565125c5c4d0 0x565126027140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:24:54.314 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:54.312+0000 7fdc3659c640 1 -- 172.21.15.82:0/3197821446 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565125c95210 con 0x565125c542f0 2024-05-20T16:24:54.314 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:54.312+0000 7fdc3659c640 1 --2- 172.21.15.82:0/3197821446 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x565125c542f0 0x565126026c00 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fdc2c00be90 tx=0x7fdc2c00f5b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=49f033f5ca06f7e7 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:24:54.316 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:54.316+0000 7fdc1dffb640 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-05-20T16:24:54.318 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:54.318+0000 7fdc35d9b640 1 --2- 172.21.15.82:0/1931290262 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fdc200a2d20 0x7fdc200a5230 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fdc2801a040 tx=0x7fdc2800be10 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:24:54.318 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:54.318+0000 7fdc1dffb640 1 -- 172.21.15.82:0/3197821446 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc2c019020 con 0x565125c542f0 2024-05-20T16:24:54.319 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:54.319+0000 7fdc43c4f740 1 -- 172.21.15.82:0/3197821446 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdc280065d0 con 0x565125c542f0 2024-05-20T16:24:54.319 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:54.319+0000 7fdc1dffb640 1 -- 172.21.15.82:0/3197821446 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9150+0+0 (secure 0 0 0) 0x7fdc2c014660 con 0x565125c542f0 2024-05-20T16:24:54.319 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:54.319+0000 7fdc1dffb640 1 -- 172.21.15.82:0/3197821446 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fdc2c024050 con 0x565125c542f0 2024-05-20T16:24:54.322 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:54.322+0000 7fdc43c4f740 10 client.25516 fetch_fsmap learned FSMap version 29 2024-05-20T16:24:54.322 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:54.322+0000 7fdc43c4f740 1 -- 172.21.15.82:0/3197821446 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x565125d61620 con 0x565125c542f0 2024-05-20T16:24:54.322 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:54.323+0000 7fdc1dffb640 1 -- 172.21.15.82:0/3197821446 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc280065d0 con 0x565125c542f0 2024-05-20T16:24:54.323 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:54.323+0000 7fdc1dffb640 1 -- 172.21.15.82:0/3197821446 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7fdc2c015ac0 con 0x565125c542f0 2024-05-20T16:24:54.323 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:54.323+0000 7fdc43c4f740 10 client.25516 fetch_fsmap finished waiting for FSMap version 29 2024-05-20T16:24:54.323 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:54.323+0000 7fdc43c4f740 10 client.25516 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T16:24:54.323 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:54.323+0000 7fdc43c4f740 20 client.25516 populate_metadata read hostname 'smithi082' 2024-05-20T16:24:54.323 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:54.323+0000 7fdc43c4f740 1 --2- 172.21.15.82:0/3197821446 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x565125c3bd80 0x5651260345a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:24:54.323 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:54.323+0000 7fdc43c4f740 4 client.25516 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T16:24:54.323 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:54.323+0000 7fdc43c4f740 1 -- 172.21.15.82:0/3197821446 --> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x565125f8ea70 con 0x565125c3bd80 2024-05-20T16:24:54.324 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:54.324+0000 7fdc35d9b640 1 --2- 172.21.15.82:0/3197821446 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x565125c3bd80 0x5651260345a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:24:54.333 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:54.334+0000 7fdc35d9b640 1 --2- 172.21.15.82:0/3197821446 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x565125c3bd80 0x5651260345a0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=feadfbdca6338e7 server_cookie=c10fb8aeda100476 in_seq=0 out_seq=0 2024-05-20T16:24:54.333 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:54.334+0000 7fdc1dffb640 10 client.25516 ms_handle_connect on v2:172.21.15.160:6838/2765554482 2024-05-20T16:24:54.337 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:54.338+0000 7fdc1dffb640 1 -- 172.21.15.82:0/3197821446 <== mds.0 v2:172.21.15.160:6838/2765554482 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x565125f8ea70 con 0x565125c3bd80 2024-05-20T16:24:54.337 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:54.338+0000 7fdc1dffb640 10 client.25516 handle_command_reply: tid=0 2024-05-20T16:24:54.340 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:54.338+0000 7fdc43c4f740 1 client.25516 shutdown 2024-05-20T16:24:54.342 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:54.343+0000 7fdc43c4f740 1 -- 172.21.15.82:0/3197821446 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x565125c542f0 msgr2=0x565126026c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:24:54.342 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:54.343+0000 7fdc43c4f740 1 --2- 172.21.15.82:0/3197821446 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x565125c542f0 0x565126026c00 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fdc2c00be90 tx=0x7fdc2c00f5b0 comp rx=0 tx=0).stop 2024-05-20T16:24:54.352 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:54.344+0000 7fdc43c4f740 1 -- 172.21.15.82:0/3197821446 shutdown_connections 2024-05-20T16:24:54.352 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:54.344+0000 7fdc43c4f740 1 --2- 172.21.15.82:0/3197821446 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x565125c3bd80 0x5651260345a0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:24:54.352 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:54.344+0000 7fdc43c4f740 1 --2- 172.21.15.82:0/3197821446 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x565125c5c4d0 0x565126027140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:24:54.352 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:54.344+0000 7fdc43c4f740 1 --2- 172.21.15.82:0/3197821446 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x565125c7d1b0 0x56512602e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:24:54.353 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:54.344+0000 7fdc43c4f740 1 --2- 172.21.15.82:0/3197821446 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x565125c542f0 0x565126026c00 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:24:54.353 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:54.344+0000 7fdc43c4f740 1 -- 172.21.15.82:0/3197821446 >> 172.21.15.82:0/3197821446 conn(0x565125c71300 msgr2=0x56512601b7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:24:54.353 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:54.345+0000 7fdc1dffb640 0 client.25516 ms_handle_reset on v2:172.21.15.160:6838/2765554482 2024-05-20T16:24:54.353 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:54.349+0000 7fdc43c4f740 1 -- 172.21.15.82:0/3197821446 shutdown_connections 2024-05-20T16:24:54.358 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:54.354+0000 7fdc43c4f740 1 -- 172.21.15.82:0/3197821446 wait complete. 2024-05-20T16:24:54.358 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:54.354+0000 7fdc43c4f740 20 client.25516 trim_cache size 0 max 16384 2024-05-20T16:24:54.417 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:54.417+0000 7fdc43c4f740 1 Processor -- start 2024-05-20T16:24:54.417 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:54.417+0000 7fdc43c4f740 1 -- start start 2024-05-20T16:24:54.418 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:54.418+0000 7fdc43c4f740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x565125c542f0 0x56512601bda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:24:54.418 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:54.418+0000 7fdc43c4f740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x565125c5c4d0 0x56512601c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:24:54.418 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:54.418+0000 7fdc43c4f740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x565125c7d1b0 0x56512602df50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:24:54.419 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:54.418+0000 7fdc43c4f740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fdc18012550 con 0x565125c5c4d0 2024-05-20T16:24:54.419 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:54.418+0000 7fdc43c4f740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fdc1800fe20 con 0x565125c542f0 2024-05-20T16:24:54.419 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:54.419+0000 7fdc43c4f740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fdc1800dc70 con 0x565125c7d1b0 2024-05-20T16:24:54.421 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:54.421+0000 7fdc35d9b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x565125c5c4d0 0x56512601c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:24:54.421 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:54.421+0000 7fdc35d9b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x565125c5c4d0 0x56512601c2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58568/0 (socket says 172.21.15.82:58568) 2024-05-20T16:24:54.421 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:54.421+0000 7fdc35d9b640 1 -- 172.21.15.82:0/1422425880 learned_addr learned my addr 172.21.15.82:0/1422425880 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:24:54.421 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:54.421+0000 7fdc36d9d640 1 --2- 172.21.15.82:0/1422425880 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x565125c7d1b0 0x56512602df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:24:54.425 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:54.423+0000 7fdc3659c640 1 --2- 172.21.15.82:0/1422425880 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x565125c542f0 0x56512601bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:24:54.429 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:54.430+0000 7fdc36d9d640 1 -- 172.21.15.82:0/1422425880 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x565125c542f0 msgr2=0x56512601bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:24:54.430 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:54.430+0000 7fdc36d9d640 1 --2- 172.21.15.82:0/1422425880 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x565125c542f0 0x56512601bda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:24:54.430 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:54.430+0000 7fdc36d9d640 1 -- 172.21.15.82:0/1422425880 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x565125c5c4d0 msgr2=0x56512601c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:24:54.430 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:54.430+0000 7fdc36d9d640 1 --2- 172.21.15.82:0/1422425880 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x565125c5c4d0 0x56512601c2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:24:54.430 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:54.430+0000 7fdc36d9d640 1 -- 172.21.15.82:0/1422425880 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56512601db50 con 0x565125c7d1b0 2024-05-20T16:24:54.438 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:54.438+0000 7fdc36d9d640 1 --2- 172.21.15.82:0/1422425880 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x565125c7d1b0 0x56512602df50 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fdc300ee450 tx=0x7fdc300dd040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d01588022ad5bc82 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:24:54.449 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:54.448+0000 7fdc1ffff640 1 -- 172.21.15.82:0/1422425880 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc300bd160 con 0x565125c7d1b0 2024-05-20T16:24:54.449 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:54.448+0000 7fdc1ffff640 1 -- 172.21.15.82:0/1422425880 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc30036030 con 0x565125c7d1b0 2024-05-20T16:24:54.449 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:54.448+0000 7fdc1ffff640 1 -- 172.21.15.82:0/1422425880 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc300dd0b0 con 0x565125c7d1b0 2024-05-20T16:24:54.451 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:54.451+0000 7fdc43c4f740 1 -- 172.21.15.82:0/1422425880 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x565125c7d1b0 msgr2=0x56512602df50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:24:54.451 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:54.451+0000 7fdc43c4f740 1 --2- 172.21.15.82:0/1422425880 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x565125c7d1b0 0x56512602df50 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fdc300ee450 tx=0x7fdc300dd040 comp rx=0 tx=0).stop 2024-05-20T16:24:54.453 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:54.453+0000 7fdc43c4f740 1 -- 172.21.15.82:0/1422425880 shutdown_connections 2024-05-20T16:24:54.453 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:54.453+0000 7fdc43c4f740 1 --2- 172.21.15.82:0/1422425880 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x565125c5c4d0 0x56512601c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:24:54.453 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:54.453+0000 7fdc43c4f740 1 --2- 172.21.15.82:0/1422425880 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x565125c7d1b0 0x56512602df50 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:24:54.454 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:54.453+0000 7fdc43c4f740 1 --2- 172.21.15.82:0/1422425880 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x565125c542f0 0x56512601bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:24:54.454 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:54.453+0000 7fdc43c4f740 1 -- 172.21.15.82:0/1422425880 >> 172.21.15.82:0/1422425880 conn(0x565125c71300 msgr2=0x56512601b690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:24:54.454 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:54.453+0000 7fdc43c4f740 1 -- 172.21.15.82:0/1422425880 shutdown_connections 2024-05-20T16:24:54.454 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:54.453+0000 7fdc43c4f740 1 -- 172.21.15.82:0/1422425880 wait complete. 2024-05-20T16:24:54.454 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:54.454+0000 7fdc43c4f740 1 -- start start 2024-05-20T16:24:54.454 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:54.455+0000 7fdc43c4f740 1 Processor -- start 2024-05-20T16:24:54.455 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:54.455+0000 7fdc43c4f740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x565125c542f0 0x56512602b520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:24:54.455 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:54.456+0000 7fdc43c4f740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x565125c5c4d0 0x56512602ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:24:54.455 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:54.456+0000 7fdc43c4f740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x565125c7d1b0 0x56512609a900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:24:54.456 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:54.456+0000 7fdc43c4f740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fdc1800e2e0 con 0x565125c7d1b0 2024-05-20T16:24:54.456 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:54.456+0000 7fdc3659c640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x565125c542f0 0x56512602b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:24:54.456 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:54.456+0000 7fdc3659c640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x565125c542f0 0x56512602b520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.82:33340/0 (socket says 172.21.15.82:33340) 2024-05-20T16:24:54.456 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:54.456+0000 7fdc3659c640 1 -- 172.21.15.82:0/2290756071 learned_addr learned my addr 172.21.15.82:0/2290756071 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:24:54.457 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:54.456+0000 7fdc43c4f740 1 -- 172.21.15.82:0/2290756071 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fdc1800f750 con 0x565125c542f0 2024-05-20T16:24:54.457 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:54.456+0000 7fdc43c4f740 1 -- 172.21.15.82:0/2290756071 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fdc1800e9b0 con 0x565125c5c4d0 2024-05-20T16:24:54.457 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:54.456+0000 7fdc43c4f740 10 client.0 _ll_register_callbacks cb 0x7fdc2c00b180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T16:24:54.459 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:54.460+0000 7fdc35d9b640 1 --2- 172.21.15.82:0/2290756071 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x565125c5c4d0 0x56512602ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:24:54.459 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:54.460+0000 7fdc36d9d640 1 --2- 172.21.15.82:0/2290756071 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x565125c7d1b0 0x56512609a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:24:54.465 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:54.466+0000 7fdc36d9d640 1 -- 172.21.15.82:0/2290756071 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x565125c5c4d0 msgr2=0x56512602ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:24:54.466 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:54.466+0000 7fdc36d9d640 1 --2- 172.21.15.82:0/2290756071 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x565125c5c4d0 0x56512602ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:24:54.466 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:54.466+0000 7fdc36d9d640 1 -- 172.21.15.82:0/2290756071 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x565125c542f0 msgr2=0x56512602b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:24:54.466 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:54.466+0000 7fdc36d9d640 1 --2- 172.21.15.82:0/2290756071 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x565125c542f0 0x56512602b520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:24:54.466 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:54.466+0000 7fdc36d9d640 1 -- 172.21.15.82:0/2290756071 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56512602e9b0 con 0x565125c7d1b0 2024-05-20T16:24:54.475 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:54.476+0000 7fdc36d9d640 1 --2- 172.21.15.82:0/2290756071 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x565125c7d1b0 0x56512609a900 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fdc300ee770 tx=0x7fdc300b9a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=886215e19d8f0f1f server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:24:54.475 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:54.476+0000 7fdc1dffb640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-05-20T16:24:54.479 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:54.480+0000 7fdc1dffb640 1 -- 172.21.15.82:0/2290756071 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc300d5070 con 0x565125c7d1b0 2024-05-20T16:24:54.480 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:54.480+0000 7fdc43c4f740 1 -- 172.21.15.82:0/2290756071 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdc30036030 con 0x565125c7d1b0 2024-05-20T16:24:54.480 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:54.480+0000 7fdc1dffb640 1 -- 172.21.15.82:0/2290756071 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9150+0+0 (secure 0 0 0) 0x7fdc300dd0b0 con 0x565125c7d1b0 2024-05-20T16:24:54.484 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:54.485+0000 7fdc1dffb640 1 -- 172.21.15.82:0/2290756071 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fdc300cf050 con 0x565125c7d1b0 2024-05-20T16:24:54.489 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:54.488+0000 7fdc43c4f740 10 client.15966 fetch_fsmap learned FSMap version 29 2024-05-20T16:24:54.490 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:54.488+0000 7fdc43c4f740 1 -- 172.21.15.82:0/2290756071 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56512609bd40 con 0x565125c7d1b0 2024-05-20T16:24:54.491 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:54.491+0000 7fdc1dffb640 1 -- 172.21.15.82:0/2290756071 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc30036030 con 0x565125c7d1b0 2024-05-20T16:24:54.492 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:54.491+0000 7fdc1dffb640 1 -- 172.21.15.82:0/2290756071 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7fdc300c5940 con 0x565125c7d1b0 2024-05-20T16:24:54.494 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:54.492+0000 7fdc43c4f740 10 client.15966 fetch_fsmap finished waiting for FSMap version 29 2024-05-20T16:24:54.494 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:54.492+0000 7fdc43c4f740 10 client.15966 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T16:24:54.494 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:54.492+0000 7fdc43c4f740 20 client.15966 populate_metadata read hostname 'smithi082' 2024-05-20T16:24:54.494 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:54.492+0000 7fdc43c4f740 1 --2- 172.21.15.82:0/2290756071 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x565125c111c0 0x56512609e690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:24:54.494 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:54.492+0000 7fdc43c4f740 4 client.15966 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-20T16:24:54.494 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:54.492+0000 7fdc43c4f740 1 -- 172.21.15.82:0/2290756071 --> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] -- command(tid 0: {"prefix": "flush journal"}) -- 0x565125f88dc0 con 0x565125c111c0 2024-05-20T16:24:54.501 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:54.501+0000 7fdc3659c640 1 --2- 172.21.15.82:0/2290756071 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x565125c111c0 0x56512609e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:24:54.510 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:54.508+0000 7fdc3659c640 1 --2- 172.21.15.82:0/2290756071 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x565125c111c0 0x56512609e690 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9a398dfc084d6895 server_cookie=209d7ddc84eedbfe in_seq=0 out_seq=0 2024-05-20T16:24:54.510 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:54.508+0000 7fdc1dffb640 10 client.15966 ms_handle_connect on v2:172.21.15.160:6838/2765554482 2024-05-20T16:24:55.312 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:24:54 smithi160 ceph-mon[36347]: pgmap v691: 129 pgs: 129 active+clean; 10 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 743 MiB/s wr, 1.38k op/s 2024-05-20T16:24:55.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:24:54 smithi082 ceph-mon[32238]: pgmap v691: 129 pgs: 129 active+clean; 10 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 743 MiB/s wr, 1.38k op/s 2024-05-20T16:24:55.385 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:24:54 smithi184 ceph-mon[36209]: pgmap v691: 129 pgs: 129 active+clean; 10 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 743 MiB/s wr, 1.38k op/s 2024-05-20T16:24:56.312 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:24:55 smithi160 ceph-mon[36347]: pgmap v692: 129 pgs: 129 active+clean; 10 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 680 MiB/s wr, 1.28k op/s 2024-05-20T16:24:56.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:24:55 smithi082 ceph-mon[32238]: pgmap v692: 129 pgs: 129 active+clean; 10 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 680 MiB/s wr, 1.28k op/s 2024-05-20T16:24:56.385 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:24:55 smithi184 ceph-mon[36209]: pgmap v692: 129 pgs: 129 active+clean; 10 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 680 MiB/s wr, 1.28k op/s 2024-05-20T16:24:56.766 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:56.766+0000 7fdc1dffb640 1 -- 172.21.15.82:0/2290756071 <== mds.0 v2:172.21.15.160:6838/2765554482 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x565125f88dc0 con 0x565125c111c0 2024-05-20T16:24:56.767 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:56.766+0000 7fdc1dffb640 10 client.15966 handle_command_reply: tid=0 2024-05-20T16:24:56.767 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:56.767+0000 7fdc43c4f740 1 client.15966 shutdown 2024-05-20T16:24:56.767 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:56.767+0000 7fdc43c4f740 1 -- 172.21.15.82:0/2290756071 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x565125c7d1b0 msgr2=0x56512609a900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:24:56.767 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:56.767+0000 7fdc43c4f740 1 --2- 172.21.15.82:0/2290756071 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x565125c7d1b0 0x56512609a900 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fdc300ee770 tx=0x7fdc300b9a00 comp rx=0 tx=0).stop 2024-05-20T16:24:56.768 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:56.767+0000 7fdc43c4f740 1 -- 172.21.15.82:0/2290756071 shutdown_connections 2024-05-20T16:24:56.768 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:56.767+0000 7fdc43c4f740 1 --2- 172.21.15.82:0/2290756071 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x565125c111c0 0x56512609e690 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:24:56.768 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:56.767+0000 7fdc43c4f740 1 --2- 172.21.15.82:0/2290756071 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x565125c7d1b0 0x56512609a900 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:24:56.768 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:56.767+0000 7fdc43c4f740 1 --2- 172.21.15.82:0/2290756071 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x565125c5c4d0 0x56512602ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:24:56.768 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:56.767+0000 7fdc43c4f740 1 --2- 172.21.15.82:0/2290756071 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x565125c542f0 0x56512602b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:24:56.768 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:56.767+0000 7fdc43c4f740 1 -- 172.21.15.82:0/2290756071 >> 172.21.15.82:0/2290756071 conn(0x565125c71300 msgr2=0x56512601be20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:24:56.768 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:56.767+0000 7fdc1dffb640 0 client.15966 ms_handle_reset on v2:172.21.15.160:6838/2765554482 2024-05-20T16:24:56.768 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:56.768+0000 7fdc43c4f740 1 -- 172.21.15.82:0/2290756071 shutdown_connections 2024-05-20T16:24:56.768 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:56.768+0000 7fdc43c4f740 1 -- 172.21.15.82:0/2290756071 wait complete. 2024-05-20T16:24:56.769 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:56.768+0000 7fdc43c4f740 20 client.15966 trim_cache size 0 max 16384 2024-05-20T16:24:56.769 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:{ 2024-05-20T16:24:56.769 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "message": "", 2024-05-20T16:24:56.769 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "return_code": 0 2024-05-20T16:24:56.769 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:} 2024-05-20T16:24:56.774 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:56.773+0000 7fdc3cdaa640 1 -- 172.21.15.82:0/1931290262 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fdc200a2d20 msgr2=0x7fdc200a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:24:56.774 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:56.773+0000 7fdc3cdaa640 1 --2- 172.21.15.82:0/1931290262 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fdc200a2d20 0x7fdc200a5230 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fdc2801a040 tx=0x7fdc2800be10 comp rx=0 tx=0).stop 2024-05-20T16:24:56.774 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:56.773+0000 7fdc3cdaa640 1 -- 172.21.15.82:0/1931290262 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdc3810f020 msgr2=0x7fdc381af670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:24:56.774 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:56.773+0000 7fdc3cdaa640 1 --2- 172.21.15.82:0/1931290262 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdc3810f020 0x7fdc381af670 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fdc30009430 tx=0x7fdc30003e60 comp rx=0 tx=0).stop 2024-05-20T16:24:56.775 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:56.774+0000 7fdc3cdaa640 1 -- 172.21.15.82:0/1931290262 shutdown_connections 2024-05-20T16:24:56.775 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:56.774+0000 7fdc3cdaa640 1 --2- 172.21.15.82:0/1931290262 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fdc200a2d20 0x7fdc200a5230 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:24:56.775 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:56.774+0000 7fdc3cdaa640 1 --2- 172.21.15.82:0/1931290262 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdc3810f020 0x7fdc381af670 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:24:56.775 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:56.774+0000 7fdc3cdaa640 1 --2- 172.21.15.82:0/1931290262 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdc3810a300 0x7fdc381a86e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:24:56.775 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:56.774+0000 7fdc3cdaa640 1 --2- 172.21.15.82:0/1931290262 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdc381093d0 0x7fdc381a81a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:24:56.775 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:56.774+0000 7fdc3cdaa640 1 -- 172.21.15.82:0/1931290262 >> 172.21.15.82:0/1931290262 conn(0x7fdc3806f4f0 msgr2=0x7fdc38071980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:24:56.782 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:56.781+0000 7fdc3cdaa640 1 -- 172.21.15.82:0/1931290262 shutdown_connections 2024-05-20T16:24:56.782 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:24:56.781+0000 7fdc3cdaa640 1 -- 172.21.15.82:0/1931290262 wait complete. 2024-05-20T16:24:59.313 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:24:58 smithi160 ceph-mon[36347]: pgmap v693: 129 pgs: 129 active+clean; 10 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 638 MiB/s wr, 1.22k op/s 2024-05-20T16:24:59.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:24:58 smithi082 ceph-mon[32238]: pgmap v693: 129 pgs: 129 active+clean; 10 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 638 MiB/s wr, 1.22k op/s 2024-05-20T16:24:59.385 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:24:58 smithi184 ceph-mon[36209]: pgmap v693: 129 pgs: 129 active+clean; 10 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 638 MiB/s wr, 1.22k op/s 2024-05-20T16:25:01.313 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:25:00 smithi160 ceph-mon[36347]: pgmap v694: 129 pgs: 129 active+clean; 10 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 761 MiB/s wr, 1.46k op/s 2024-05-20T16:25:01.313 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:25:00 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T16:25:01.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:25:00 smithi082 ceph-mon[32238]: pgmap v694: 129 pgs: 129 active+clean; 10 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 761 MiB/s wr, 1.46k op/s 2024-05-20T16:25:01.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:25:00 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T16:25:01.385 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:25:00 smithi184 ceph-mon[36209]: pgmap v694: 129 pgs: 129 active+clean; 10 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 761 MiB/s wr, 1.46k op/s 2024-05-20T16:25:01.385 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:25:00 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T16:25:03.312 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:25:02 smithi160 ceph-mon[36347]: pgmap v695: 129 pgs: 129 active+clean; 10 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 642 MiB/s wr, 962 op/s 2024-05-20T16:25:03.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:25:02 smithi082 ceph-mon[32238]: pgmap v695: 129 pgs: 129 active+clean; 10 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 642 MiB/s wr, 962 op/s 2024-05-20T16:25:03.385 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:25:02 smithi184 ceph-mon[36209]: pgmap v695: 129 pgs: 129 active+clean; 10 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 642 MiB/s wr, 962 op/s 2024-05-20T16:25:03.886 INFO:journalctl@ceph.mds.c.smithi184.stdout:May 20 16:25:03 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-c[71243]: ignoring --setuser ceph since I am not root 2024-05-20T16:25:03.886 INFO:journalctl@ceph.mds.c.smithi184.stdout:May 20 16:25:03 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-c[71243]: ignoring --setgroup ceph since I am not root 2024-05-20T16:25:03.886 INFO:journalctl@ceph.mds.c.smithi184.stdout:May 20 16:25:03 smithi184 ceph-mds[71247]: -- 172.21.15.184:0/995467309 <== mon.1 v2:172.21.15.184:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f4c45c1a40 con 0x55f4c4639180 2024-05-20T16:25:03.886 INFO:journalctl@ceph.mds.c.smithi184.stdout:May 20 16:25:03 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-c[71243]: starting mds.c at 2024-05-20T16:25:04.312 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:25:03 smithi160 ceph-mon[36347]: daemon mds.c restarted 2024-05-20T16:25:04.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:25:03 smithi082 ceph-mon[32238]: daemon mds.c restarted 2024-05-20T16:25:04.385 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:25:03 smithi184 ceph-mon[36209]: daemon mds.c restarted 2024-05-20T16:25:05.312 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:25:04 smithi160 ceph-mon[36347]: pgmap v696: 129 pgs: 129 active+clean; 10 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 755 MiB/s wr, 1.27k op/s 2024-05-20T16:25:05.312 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:25:04 smithi160 ceph-mon[36347]: osdmap e98: 12 total, 12 up, 12 in 2024-05-20T16:25:05.312 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:25:04 smithi160 ceph-mon[36347]: mds.? [v2:172.21.15.184:6836/371714807,v1:172.21.15.184:6837/371714807] up:boot 2024-05-20T16:25:05.313 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:25:04 smithi160 ceph-mon[36347]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 2 up:standby-replay 7 up:standby 2024-05-20T16:25:05.313 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:25:04 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-20T16:25:05.313 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:25:04 smithi160 ceph-mon[36347]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-05-20T16:25:05.363 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:25:04 smithi082 ceph-mon[32238]: pgmap v696: 129 pgs: 129 active+clean; 10 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 755 MiB/s wr, 1.27k op/s 2024-05-20T16:25:05.363 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:25:04 smithi082 ceph-mon[32238]: osdmap e98: 12 total, 12 up, 12 in 2024-05-20T16:25:05.363 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:25:04 smithi082 ceph-mon[32238]: mds.? [v2:172.21.15.184:6836/371714807,v1:172.21.15.184:6837/371714807] up:boot 2024-05-20T16:25:05.363 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:25:04 smithi082 ceph-mon[32238]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 2 up:standby-replay 7 up:standby 2024-05-20T16:25:05.363 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:25:04 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-20T16:25:05.363 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:25:04 smithi082 ceph-mon[32238]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-05-20T16:25:05.385 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:25:04 smithi184 ceph-mon[36209]: pgmap v696: 129 pgs: 129 active+clean; 10 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 755 MiB/s wr, 1.27k op/s 2024-05-20T16:25:05.385 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:25:04 smithi184 ceph-mon[36209]: osdmap e98: 12 total, 12 up, 12 in 2024-05-20T16:25:05.386 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:25:04 smithi184 ceph-mon[36209]: mds.? [v2:172.21.15.184:6836/371714807,v1:172.21.15.184:6837/371714807] up:boot 2024-05-20T16:25:05.386 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:25:04 smithi184 ceph-mon[36209]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 2 up:standby-replay 7 up:standby 2024-05-20T16:25:05.386 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:25:04 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-20T16:25:05.386 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:25:04 smithi184 ceph-mon[36209]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-05-20T16:25:06.135 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:25:05 smithi184 ceph-mon[36209]: pgmap v698: 129 pgs: 129 active+clean; 10 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 412 KiB/s rd, 636 MiB/s wr, 1.40k op/s 2024-05-20T16:25:06.311 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:25:05 smithi160 ceph-mon[36347]: pgmap v698: 129 pgs: 129 active+clean; 10 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 412 KiB/s rd, 636 MiB/s wr, 1.40k op/s 2024-05-20T16:25:06.358 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:25:05 smithi082 ceph-mon[32238]: pgmap v698: 129 pgs: 129 active+clean; 10 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 412 KiB/s rd, 636 MiB/s wr, 1.40k op/s 2024-05-20T16:25:09.312 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:25:08 smithi160 ceph-mon[36347]: pgmap v699: 129 pgs: 129 active+clean; 10 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 904 KiB/s rd, 677 MiB/s wr, 1.49k op/s 2024-05-20T16:25:09.361 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:25:08 smithi082 ceph-mon[32238]: pgmap v699: 129 pgs: 129 active+clean; 10 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 904 KiB/s rd, 677 MiB/s wr, 1.49k op/s 2024-05-20T16:25:09.385 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:25:08 smithi184 ceph-mon[36209]: pgmap v699: 129 pgs: 129 active+clean; 10 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 904 KiB/s rd, 677 MiB/s wr, 1.49k op/s 2024-05-20T16:25:10.100 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:10.099+0000 7f1289ae9640 1 -- 172.21.15.82:0/2762452762 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1284103d40 msgr2=0x7f1284106160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:25:10.100 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:10.099+0000 7f1289ae9640 1 --2- 172.21.15.82:0/2762452762 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1284103d40 0x7f1284106160 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f1278009900 tx=0x7f127802a420 comp rx=0 tx=0).stop 2024-05-20T16:25:10.100 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:10.099+0000 7f1289ae9640 1 -- 172.21.15.82:0/2762452762 shutdown_connections 2024-05-20T16:25:10.100 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:10.099+0000 7f1289ae9640 1 --2- 172.21.15.82:0/2762452762 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1284103d40 0x7f1284106160 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:25:10.100 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:10.099+0000 7f1289ae9640 1 --2- 172.21.15.82:0/2762452762 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f12841066a0 0x7f1284106ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:25:10.100 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:10.099+0000 7f1289ae9640 1 --2- 172.21.15.82:0/2762452762 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f12841013e0 0x7f1284103800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:25:10.100 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:10.099+0000 7f1289ae9640 1 -- 172.21.15.82:0/2762452762 >> 172.21.15.82:0/2762452762 conn(0x7f12840fcc10 msgr2=0x7f12840ff0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:25:10.100 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:10.100+0000 7f1289ae9640 1 -- 172.21.15.82:0/2762452762 shutdown_connections 2024-05-20T16:25:10.104 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:10.102+0000 7f1289ae9640 1 -- 172.21.15.82:0/2762452762 wait complete. 2024-05-20T16:25:10.104 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:10.103+0000 7f1289ae9640 1 Processor -- start 2024-05-20T16:25:10.104 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:10.105+0000 7f1289ae9640 1 -- start start 2024-05-20T16:25:10.105 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:10.105+0000 7f1289ae9640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f12841013e0 0x7f12840729d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:25:10.105 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:10.105+0000 7f1289ae9640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1284103d40 0x7f1284072f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:25:10.105 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:10.105+0000 7f1289ae9640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f12841066a0 0x7f128410e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:25:10.105 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:10.105+0000 7f1289ae9640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f1284107e40 con 0x7f12841013e0 2024-05-20T16:25:10.105 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:10.105+0000 7f1289ae9640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f1284107cc0 con 0x7f12841066a0 2024-05-20T16:25:10.105 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:10.105+0000 7f1289ae9640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f1284107fc0 con 0x7f1284103d40 2024-05-20T16:25:10.106 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:10.107+0000 7f1283fff640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f12841066a0 0x7f128410e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:25:10.107 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:10.107+0000 7f1282ffd640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1284103d40 0x7f1284072f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:25:10.107 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:10.107+0000 7f1282ffd640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1284103d40 0x7f1284072f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.82:47134/0 (socket says 172.21.15.82:47134) 2024-05-20T16:25:10.107 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:10.107+0000 7f1282ffd640 1 -- 172.21.15.82:0/1553739263 learned_addr learned my addr 172.21.15.82:0/1553739263 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:25:10.110 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:10.110+0000 7f1283fff640 1 -- 172.21.15.82:0/1553739263 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1284103d40 msgr2=0x7f1284072f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:25:10.110 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:10.110+0000 7f1283fff640 1 --2- 172.21.15.82:0/1553739263 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1284103d40 0x7f1284072f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:25:10.110 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:10.110+0000 7f1283fff640 1 -- 172.21.15.82:0/1553739263 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f12841013e0 msgr2=0x7f12840729d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:25:10.110 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:10.110+0000 7f1283fff640 1 --2- 172.21.15.82:0/1553739263 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f12841013e0 0x7f12840729d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:25:10.110 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:10.110+0000 7f1283fff640 1 -- 172.21.15.82:0/1553739263 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f128410efe0 con 0x7f12841066a0 2024-05-20T16:25:10.116 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:10.115+0000 7f1283fff640 1 --2- 172.21.15.82:0/1553739263 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f12841066a0 0x7f128410e940 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f127400c8c0 tx=0x7f127400cd80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:25:10.116 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:10.116+0000 7f1280ff9640 1 -- 172.21.15.82:0/1553739263 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1274004230 con 0x7f12841066a0 2024-05-20T16:25:10.116 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:10.116+0000 7f1289ae9640 1 -- 172.21.15.82:0/1553739263 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f128410f270 con 0x7f12841066a0 2024-05-20T16:25:10.116 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:10.116+0000 7f1289ae9640 1 -- 172.21.15.82:0/1553739263 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f128410f7c0 con 0x7f12841066a0 2024-05-20T16:25:10.116 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:10.116+0000 7f1280ff9640 1 -- 172.21.15.82:0/1553739263 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f127400ae40 con 0x7f12841066a0 2024-05-20T16:25:10.116 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:10.117+0000 7f1280ff9640 1 -- 172.21.15.82:0/1553739263 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f127400f450 con 0x7f12841066a0 2024-05-20T16:25:10.124 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:10.125+0000 7f129098c740 1 Processor -- start 2024-05-20T16:25:10.127 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:10.126+0000 7f1280ff9640 1 -- 172.21.15.82:0/1553739263 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f127400f5f0 con 0x7f12841066a0 2024-05-20T16:25:10.128 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:10.128+0000 7f1280ff9640 1 --2- 172.21.15.82:0/1553739263 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f126c0a2d50 0x7f126c0a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:25:10.128 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:10.128+0000 7f1280ff9640 1 -- 172.21.15.82:0/1553739263 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9150+0+0 (secure 0 0 0) 0x7f12740cf470 con 0x7f12841066a0 2024-05-20T16:25:10.128 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:10.129+0000 7f129098c740 1 -- start start 2024-05-20T16:25:10.129 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:10.129+0000 7f129098c740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562a17e661b0 0x562a18205e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:25:10.130 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:10.129+0000 7f129098c740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562a17e3d2f0 0x562a182063e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:25:10.130 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:10.129+0000 7f129098c740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562a17e454d0 0x562a1820d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:25:10.130 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:10.129+0000 7f129098c740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x562a1816a340 con 0x562a17e3d2f0 2024-05-20T16:25:10.130 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:10.129+0000 7f129098c740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x562a18112290 con 0x562a17e454d0 2024-05-20T16:25:10.130 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:10.129+0000 7f129098c740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x562a18166300 con 0x562a17e661b0 2024-05-20T16:25:10.130 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:10.130+0000 7f1282ffd640 1 --2- 172.21.15.82:0/1553739263 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f126c0a2d50 0x7f126c0a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:25:10.132 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:10.133+0000 7f1282ffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562a17e3d2f0 0x562a182063e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:25:10.132 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:10.133+0000 7f1282ffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562a17e3d2f0 0x562a182063e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:32776/0 (socket says 172.21.15.82:32776) 2024-05-20T16:25:10.133 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:10.133+0000 7f1282ffd640 1 -- 172.21.15.82:0/653489316 learned_addr learned my addr 172.21.15.82:0/653489316 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:25:10.133 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:10.134+0000 7f1283fff640 1 --2- 172.21.15.82:0/653489316 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562a17e454d0 0x562a1820d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:25:10.137 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:10.135+0000 7f1283fff640 1 -- 172.21.15.82:0/653489316 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562a17e661b0 msgr2=0x562a18205e60 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-20T16:25:10.137 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:10.135+0000 7f1283fff640 1 --2- 172.21.15.82:0/653489316 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562a17e661b0 0x562a18205e60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:25:10.137 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:10.135+0000 7f1283fff640 1 -- 172.21.15.82:0/653489316 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562a17e3d2f0 msgr2=0x562a182063e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:25:10.137 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:10.135+0000 7f1283fff640 1 --2- 172.21.15.82:0/653489316 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562a17e3d2f0 0x562a182063e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:25:10.137 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:10.136+0000 7f1283fff640 1 -- 172.21.15.82:0/653489316 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562a181368e0 con 0x562a17e454d0 2024-05-20T16:25:10.139 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:10.138+0000 7f1283fff640 1 --2- 172.21.15.82:0/653489316 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562a17e454d0 0x562a1820d2e0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f127400cfd0 tx=0x7f127409b300 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c12362816fa5d2e3 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:25:10.139 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:10.138+0000 7f1282ffd640 1 --2- 172.21.15.82:0/1553739263 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f126c0a2d50 0x7f126c0a5260 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f12780315a0 tx=0x7f1278004510 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:25:10.145 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:10.145+0000 7f1269ffb640 1 -- 172.21.15.82:0/653489316 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f127401d040 con 0x562a17e454d0 2024-05-20T16:25:10.145 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:10.145+0000 7f1269ffb640 1 -- 172.21.15.82:0/653489316 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f127409fc90 con 0x562a17e454d0 2024-05-20T16:25:10.145 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:10.145+0000 7f1269ffb640 1 -- 172.21.15.82:0/653489316 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1274097ec0 con 0x562a17e454d0 2024-05-20T16:25:10.146 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:10.147+0000 7f129098c740 1 -- 172.21.15.82:0/653489316 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562a17e454d0 msgr2=0x562a1820d2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:25:10.146 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:10.147+0000 7f129098c740 1 --2- 172.21.15.82:0/653489316 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562a17e454d0 0x562a1820d2e0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f127400cfd0 tx=0x7f127409b300 comp rx=0 tx=0).stop 2024-05-20T16:25:10.150 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:10.150+0000 7f129098c740 1 -- 172.21.15.82:0/653489316 shutdown_connections 2024-05-20T16:25:10.150 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:10.150+0000 7f129098c740 1 --2- 172.21.15.82:0/653489316 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562a17e3d2f0 0x562a182063e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:25:10.150 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:10.150+0000 7f129098c740 1 --2- 172.21.15.82:0/653489316 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562a17e454d0 0x562a1820d2e0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:25:10.150 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:10.150+0000 7f129098c740 1 --2- 172.21.15.82:0/653489316 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562a17e661b0 0x562a18205e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:25:10.150 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:10.150+0000 7f129098c740 1 -- 172.21.15.82:0/653489316 >> 172.21.15.82:0/653489316 conn(0x562a17e5a300 msgr2=0x562a182017f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:25:10.153 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:10.154+0000 7f129098c740 1 -- 172.21.15.82:0/653489316 shutdown_connections 2024-05-20T16:25:10.155 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:10.154+0000 7f129098c740 1 -- 172.21.15.82:0/653489316 wait complete. 2024-05-20T16:25:10.155 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:10.155+0000 7f129098c740 1 -- start start 2024-05-20T16:25:10.155 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:10.155+0000 7f129098c740 1 Processor -- start 2024-05-20T16:25:10.156 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:10.156+0000 7f129098c740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562a17e3d2f0 0x562a1820fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:25:10.156 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:10.156+0000 7f129098c740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562a17e454d0 0x562a18210100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:25:10.156 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:10.156+0000 7f129098c740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562a17e661b0 0x562a18217060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:25:10.156 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:10.156+0000 7f129098c740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x562a1820dc70 con 0x562a17e454d0 2024-05-20T16:25:10.156 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:10.156+0000 7f129098c740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x562a18125790 con 0x562a17e661b0 2024-05-20T16:25:10.156 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:10.156+0000 7f129098c740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x562a1820ddf0 con 0x562a17e3d2f0 2024-05-20T16:25:10.156 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:10.156+0000 7f129098c740 10 client.0 _ll_register_callbacks cb 0x562a18160a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T16:25:10.157 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:10.157+0000 7f12837fe640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562a17e3d2f0 0x562a1820fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:25:10.157 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:10.157+0000 7f12837fe640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562a17e3d2f0 0x562a1820fbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.82:47152/0 (socket says 172.21.15.82:47152) 2024-05-20T16:25:10.157 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:10.157+0000 7f12837fe640 1 -- 172.21.15.82:0/1408708982 learned_addr learned my addr 172.21.15.82:0/1408708982 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:25:10.157 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:10.157+0000 7f1283fff640 1 --2- 172.21.15.82:0/1408708982 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562a17e661b0 0x562a18217060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:25:10.157 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:10.158+0000 7f1283fff640 1 -- 172.21.15.82:0/1408708982 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562a17e3d2f0 msgr2=0x562a1820fbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:25:10.157 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:10.158+0000 7f1283fff640 1 --2- 172.21.15.82:0/1408708982 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562a17e3d2f0 0x562a1820fbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:25:10.158 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:10.158+0000 7f1283fff640 1 -- 172.21.15.82:0/1408708982 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562a17e454d0 msgr2=0x562a18210100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:25:10.158 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:10.158+0000 7f1283fff640 1 --2- 172.21.15.82:0/1408708982 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562a17e454d0 0x562a18210100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:25:10.158 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:10.158+0000 7f1283fff640 1 -- 172.21.15.82:0/1408708982 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562a17e7e210 con 0x562a17e661b0 2024-05-20T16:25:10.158 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:10.158+0000 7f1283fff640 1 --2- 172.21.15.82:0/1408708982 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562a17e661b0 0x562a18217060 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f1274003880 tx=0x7f127409b370 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c88dd5828d44dd6e server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:25:10.159 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:10.158+0000 7f1262ffd640 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-20T16:25:10.159 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:10.159+0000 7f1262ffd640 1 -- 172.21.15.82:0/1408708982 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12740b3020 con 0x562a17e661b0 2024-05-20T16:25:10.159 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:10.159+0000 7f129098c740 1 -- 172.21.15.82:0/1408708982 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f127409fc90 con 0x562a17e661b0 2024-05-20T16:25:10.159 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:10.159+0000 7f1262ffd640 1 -- 172.21.15.82:0/1408708982 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9150+0+0 (secure 0 0 0) 0x7f1274003680 con 0x562a17e661b0 2024-05-20T16:25:10.159 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:10.159+0000 7f1262ffd640 1 -- 172.21.15.82:0/1408708982 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f127401d040 con 0x562a17e661b0 2024-05-20T16:25:10.160 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:10.161+0000 7f129098c740 10 client.25484 fetch_fsmap learned FSMap version 31 2024-05-20T16:25:10.160 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:10.161+0000 7f129098c740 1 -- 172.21.15.82:0/1408708982 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x562a17f4a620 con 0x562a17e661b0 2024-05-20T16:25:10.163 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:10.163+0000 7f1262ffd640 1 -- 172.21.15.82:0/1408708982 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f127409fc90 con 0x562a17e661b0 2024-05-20T16:25:10.163 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:10.163+0000 7f1262ffd640 1 -- 172.21.15.82:0/1408708982 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7f127409ad30 con 0x562a17e661b0 2024-05-20T16:25:10.163 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:10.163+0000 7f129098c740 10 client.25484 fetch_fsmap finished waiting for FSMap version 31 2024-05-20T16:25:10.163 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:10.163+0000 7f129098c740 10 client.25484 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T16:25:10.163 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:10.163+0000 7f129098c740 20 client.25484 populate_metadata read hostname 'smithi082' 2024-05-20T16:25:10.163 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:10.163+0000 7f129098c740 1 --2- 172.21.15.82:0/1408708982 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x562a17e24d80 0x562a1821d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:25:10.163 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:10.163+0000 7f129098c740 4 client.25484 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T16:25:10.163 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:10.163+0000 7f129098c740 1 -- 172.21.15.82:0/1408708982 --> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562a18177a70 con 0x562a17e24d80 2024-05-20T16:25:10.164 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:10.164+0000 7f12837fe640 1 --2- 172.21.15.82:0/1408708982 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x562a17e24d80 0x562a1821d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:25:10.166 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:10.166+0000 7f1262ffd640 10 client.25484 ms_handle_connect on v2:172.21.15.160:6838/2765554482 2024-05-20T16:25:10.167 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:10.167+0000 7f12837fe640 1 --2- 172.21.15.82:0/1408708982 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x562a17e24d80 0x562a1821d120 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c74d5b0c31f6d264 server_cookie=88f5c56d5b13eef in_seq=0 out_seq=0 2024-05-20T16:25:10.179 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:10.179+0000 7f1262ffd640 1 -- 172.21.15.82:0/1408708982 <== mds.0 v2:172.21.15.160:6838/2765554482 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x562a18177a70 con 0x562a17e24d80 2024-05-20T16:25:10.179 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:10.179+0000 7f1262ffd640 10 client.25484 handle_command_reply: tid=0 2024-05-20T16:25:10.179 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:10.179+0000 7f129098c740 1 client.25484 shutdown 2024-05-20T16:25:10.185 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:10.186+0000 7f129098c740 1 -- 172.21.15.82:0/1408708982 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562a17e661b0 msgr2=0x562a18217060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:25:10.185 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:10.186+0000 7f129098c740 1 --2- 172.21.15.82:0/1408708982 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562a17e661b0 0x562a18217060 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f1274003880 tx=0x7f127409b370 comp rx=0 tx=0).stop 2024-05-20T16:25:10.187 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:10.188+0000 7f129098c740 1 -- 172.21.15.82:0/1408708982 shutdown_connections 2024-05-20T16:25:10.188 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:10.188+0000 7f129098c740 1 --2- 172.21.15.82:0/1408708982 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x562a17e24d80 0x562a1821d120 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:25:10.188 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:10.188+0000 7f129098c740 1 --2- 172.21.15.82:0/1408708982 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562a17e454d0 0x562a18210100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:25:10.188 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:10.188+0000 7f129098c740 1 --2- 172.21.15.82:0/1408708982 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562a17e661b0 0x562a18217060 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:25:10.188 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:10.188+0000 7f129098c740 1 --2- 172.21.15.82:0/1408708982 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562a17e3d2f0 0x562a1820fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:25:10.188 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:10.188+0000 7f129098c740 1 -- 172.21.15.82:0/1408708982 >> 172.21.15.82:0/1408708982 conn(0x562a17e5a300 msgr2=0x562a18200810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:25:10.190 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:10.190+0000 7f1262ffd640 0 client.25484 ms_handle_reset on v2:172.21.15.160:6838/2765554482 2024-05-20T16:25:10.192 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:10.193+0000 7f129098c740 1 -- 172.21.15.82:0/1408708982 shutdown_connections 2024-05-20T16:25:10.199 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:10.197+0000 7f129098c740 1 -- 172.21.15.82:0/1408708982 wait complete. 2024-05-20T16:25:10.199 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:10.197+0000 7f129098c740 20 client.25484 trim_cache size 0 max 16384 2024-05-20T16:25:10.290 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:10.290+0000 7f129098c740 1 Processor -- start 2024-05-20T16:25:10.290 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:10.291+0000 7f129098c740 1 -- start start 2024-05-20T16:25:10.291 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:10.291+0000 7f129098c740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562a17e3d2f0 0x562a182096e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:25:10.291 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:10.291+0000 7f129098c740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562a17e454d0 0x562a1820ee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:25:10.291 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:10.292+0000 7f129098c740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562a17e661b0 0x562a1820f390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:25:10.291 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:10.292+0000 7f129098c740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f12700123e0 con 0x562a17e3d2f0 2024-05-20T16:25:10.292 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:10.292+0000 7f129098c740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f127000fcb0 con 0x562a17e454d0 2024-05-20T16:25:10.292 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:10.292+0000 7f129098c740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f127000db00 con 0x562a17e661b0 2024-05-20T16:25:10.293 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:10.293+0000 7f12837fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562a17e3d2f0 0x562a182096e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:25:10.293 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:10.293+0000 7f12837fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562a17e3d2f0 0x562a182096e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:32798/0 (socket says 172.21.15.82:32798) 2024-05-20T16:25:10.293 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:10.293+0000 7f12837fe640 1 -- 172.21.15.82:0/268611512 learned_addr learned my addr 172.21.15.82:0/268611512 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:25:10.293 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:10.294+0000 7f12837fe640 1 -- 172.21.15.82:0/268611512 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562a17e661b0 msgr2=0x562a1820f390 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-20T16:25:10.294 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:10.294+0000 7f12837fe640 1 --2- 172.21.15.82:0/268611512 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562a17e661b0 0x562a1820f390 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:25:10.294 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:10.294+0000 7f12837fe640 1 -- 172.21.15.82:0/268611512 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562a17e454d0 msgr2=0x562a1820ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:25:10.294 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:10.294+0000 7f1282ffd640 1 --2- 172.21.15.82:0/268611512 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562a17e454d0 0x562a1820ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:25:10.294 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:10.294+0000 7f12837fe640 1 --2- 172.21.15.82:0/268611512 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562a17e454d0 0x562a1820ee50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:25:10.294 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:10.294+0000 7f12837fe640 1 -- 172.21.15.82:0/268611512 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562a18205a10 con 0x562a17e3d2f0 2024-05-20T16:25:10.294 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:10.295+0000 7f12837fe640 1 --2- 172.21.15.82:0/268611512 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562a17e3d2f0 0x562a182096e0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f127c0071b0 tx=0x7f127c009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be05377598e63a37 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:25:10.295 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:10.296+0000 7f12697fa640 1 -- 172.21.15.82:0/268611512 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f127c0192d0 con 0x562a17e3d2f0 2024-05-20T16:25:10.295 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:10.296+0000 7f12697fa640 1 -- 172.21.15.82:0/268611512 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f127c019950 con 0x562a17e3d2f0 2024-05-20T16:25:10.295 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:10.296+0000 7f12697fa640 1 -- 172.21.15.82:0/268611512 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f127c017e00 con 0x562a17e3d2f0 2024-05-20T16:25:10.297 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:10.297+0000 7f129098c740 1 -- 172.21.15.82:0/268611512 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562a17e3d2f0 msgr2=0x562a182096e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:25:10.297 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:10.297+0000 7f129098c740 1 --2- 172.21.15.82:0/268611512 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562a17e3d2f0 0x562a182096e0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f127c0071b0 tx=0x7f127c009580 comp rx=0 tx=0).stop 2024-05-20T16:25:10.298 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:10.298+0000 7f129098c740 1 -- 172.21.15.82:0/268611512 shutdown_connections 2024-05-20T16:25:10.298 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:10.298+0000 7f129098c740 1 --2- 172.21.15.82:0/268611512 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562a17e661b0 0x562a1820f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:25:10.298 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:10.298+0000 7f129098c740 1 --2- 172.21.15.82:0/268611512 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562a17e454d0 0x562a1820ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:25:10.298 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:10.298+0000 7f129098c740 1 --2- 172.21.15.82:0/268611512 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562a17e3d2f0 0x562a182096e0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:25:10.298 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:10.298+0000 7f129098c740 1 -- 172.21.15.82:0/268611512 >> 172.21.15.82:0/268611512 conn(0x562a17e5a300 msgr2=0x562a1820a080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:25:10.309 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:10.310+0000 7f129098c740 1 -- 172.21.15.82:0/268611512 shutdown_connections 2024-05-20T16:25:10.309 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:10.310+0000 7f129098c740 1 -- 172.21.15.82:0/268611512 wait complete. 2024-05-20T16:25:10.310 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:10.310+0000 7f129098c740 1 -- start start 2024-05-20T16:25:10.310 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:10.311+0000 7f129098c740 1 Processor -- start 2024-05-20T16:25:10.311 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:10.311+0000 7f129098c740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562a17e3d2f0 0x562a18213520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:25:10.311 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:10.312+0000 7f129098c740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562a17e454d0 0x562a18213a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:25:10.311 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:10.312+0000 7f129098c740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562a17e661b0 0x562a18283450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:25:10.312 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:10.312+0000 7f12837fe640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562a17e3d2f0 0x562a18213520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:25:10.312 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:10.312+0000 7f129098c740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f127000e170 con 0x562a17e661b0 2024-05-20T16:25:10.312 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:10.312+0000 7f129098c740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f127000f5e0 con 0x562a17e3d2f0 2024-05-20T16:25:10.312 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:10.312+0000 7f129098c740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f127000e840 con 0x562a17e454d0 2024-05-20T16:25:10.312 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:10.312+0000 7f1282ffd640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562a17e454d0 0x562a18213a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:25:10.312 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:10.312+0000 7f129098c740 10 client.0 _ll_register_callbacks cb 0x7f12780317c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T16:25:10.312 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:10.312+0000 7f1282ffd640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562a17e454d0 0x562a18213a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.82:47180/0 (socket says 172.21.15.82:47180) 2024-05-20T16:25:10.312 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:10.312+0000 7f1282ffd640 1 -- 172.21.15.82:0/1151959851 learned_addr learned my addr 172.21.15.82:0/1151959851 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:25:10.312 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:10.312+0000 7f12837fe640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562a17e3d2f0 0x562a18213520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.82:36894/0 (socket says 172.21.15.82:36894) 2024-05-20T16:25:10.313 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:10.313+0000 7f1282ffd640 1 -- 172.21.15.82:0/1151959851 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562a17e3d2f0 msgr2=0x562a18213520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:25:10.313 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:10.313+0000 7f1282ffd640 1 --2- 172.21.15.82:0/1151959851 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562a17e3d2f0 0x562a18213520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:25:10.313 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:10.313+0000 7f1282ffd640 1 -- 172.21.15.82:0/1151959851 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562a17e661b0 msgr2=0x562a18283450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:25:10.313 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:10.313+0000 7f1282ffd640 1 --2- 172.21.15.82:0/1151959851 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562a17e661b0 0x562a18283450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:25:10.313 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:10.313+0000 7f1282ffd640 1 -- 172.21.15.82:0/1151959851 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562a182140a0 con 0x562a17e454d0 2024-05-20T16:25:10.313 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:10.313+0000 7f1282ffd640 1 --2- 172.21.15.82:0/1151959851 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562a17e454d0 0x562a18213a60 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f1278006010 tx=0x7f1278009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5aee7f00d3ef9d29 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:25:10.315 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:10.315+0000 7f1262ffd640 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-05-20T16:25:10.319 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:10.318+0000 7f1262ffd640 1 -- 172.21.15.82:0/1151959851 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1278035050 con 0x562a17e454d0 2024-05-20T16:25:10.319 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:10.318+0000 7f129098c740 1 -- 172.21.15.82:0/1151959851 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f127c019950 con 0x562a17e454d0 2024-05-20T16:25:10.319 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:10.318+0000 7f1262ffd640 1 -- 172.21.15.82:0/1151959851 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9150+0+0 (secure 0 0 0) 0x7f12780048c0 con 0x562a17e454d0 2024-05-20T16:25:10.325 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:10.325+0000 7f1262ffd640 1 -- 172.21.15.82:0/1151959851 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f1278004ac0 con 0x562a17e454d0 2024-05-20T16:25:10.325 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:10.325+0000 7f129098c740 10 client.25537 fetch_fsmap learned FSMap version 31 2024-05-20T16:25:10.325 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:10.325+0000 7f129098c740 1 -- 172.21.15.82:0/1151959851 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x562a18284aa0 con 0x562a17e454d0 2024-05-20T16:25:10.331 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:10.331+0000 7f1262ffd640 1 -- 172.21.15.82:0/1151959851 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f127c019950 con 0x562a17e454d0 2024-05-20T16:25:10.332 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:10.333+0000 7f1262ffd640 1 -- 172.21.15.82:0/1151959851 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7f127804a070 con 0x562a17e454d0 2024-05-20T16:25:10.336 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:10.336+0000 7f129098c740 10 client.25537 fetch_fsmap finished waiting for FSMap version 31 2024-05-20T16:25:10.336 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:10.336+0000 7f129098c740 10 client.25537 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T16:25:10.336 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:10.336+0000 7f129098c740 20 client.25537 populate_metadata read hostname 'smithi082' 2024-05-20T16:25:10.336 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:10.336+0000 7f129098c740 1 --2- 172.21.15.82:0/1151959851 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x562a17dfa1c0 0x562a18287440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:25:10.336 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:10.336+0000 7f129098c740 4 client.25537 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-20T16:25:10.337 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:10.336+0000 7f129098c740 1 -- 172.21.15.82:0/1151959851 --> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562a18171dc0 con 0x562a17dfa1c0 2024-05-20T16:25:10.337 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:10.337+0000 7f12837fe640 1 --2- 172.21.15.82:0/1151959851 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x562a17dfa1c0 0x562a18287440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:25:10.346 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:10.344+0000 7f12837fe640 1 --2- 172.21.15.82:0/1151959851 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x562a17dfa1c0 0x562a18287440 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b3c88c7617dbc1b6 server_cookie=fc38877dd5b07946 in_seq=0 out_seq=0 2024-05-20T16:25:10.346 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:10.347+0000 7f1262ffd640 10 client.25537 ms_handle_connect on v2:172.21.15.160:6838/2765554482 2024-05-20T16:25:11.312 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:25:10 smithi160 ceph-mon[36347]: pgmap v700: 129 pgs: 129 active+clean; 10 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 638 MiB/s wr, 1.23k op/s 2024-05-20T16:25:11.312 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:25:10 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:25:11.312 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:25:10 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:25:11.312 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:25:10 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:25:11.375 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:25:10 smithi082 ceph-mon[32238]: pgmap v700: 129 pgs: 129 active+clean; 10 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 638 MiB/s wr, 1.23k op/s 2024-05-20T16:25:11.375 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:25:10 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:25:11.375 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:25:10 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:25:11.375 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:25:10 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:25:11.385 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:25:10 smithi184 ceph-mon[36209]: pgmap v700: 129 pgs: 129 active+clean; 10 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 638 MiB/s wr, 1.23k op/s 2024-05-20T16:25:11.385 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:25:10 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:25:11.385 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:25:10 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:25:11.385 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:25:10 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:25:12.312 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:25:11 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:25:12.312 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:25:11 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:25:12.312 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:25:11 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:25:12.312 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:25:11 smithi160 ceph-mon[36347]: pgmap v701: 129 pgs: 129 active+clean; 10 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 586 MiB/s wr, 1.15k op/s 2024-05-20T16:25:12.360 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:25:11 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:25:12.361 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:25:11 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:25:12.361 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:25:11 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:25:12.361 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:25:11 smithi082 ceph-mon[32238]: pgmap v701: 129 pgs: 129 active+clean; 10 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 586 MiB/s wr, 1.15k op/s 2024-05-20T16:25:12.385 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:25:11 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:25:12.386 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:25:11 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:25:12.386 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:25:11 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:25:12.386 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:25:11 smithi184 ceph-mon[36209]: pgmap v701: 129 pgs: 129 active+clean; 10 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 586 MiB/s wr, 1.15k op/s 2024-05-20T16:25:13.484 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:13.484+0000 7f1262ffd640 1 -- 172.21.15.82:0/1151959851 <== mds.0 v2:172.21.15.160:6838/2765554482 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x562a18171dc0 con 0x562a17dfa1c0 2024-05-20T16:25:13.484 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:13.484+0000 7f1262ffd640 10 client.25537 handle_command_reply: tid=0 2024-05-20T16:25:13.485 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:13.484+0000 7f129098c740 1 client.25537 shutdown 2024-05-20T16:25:13.485 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:13.485+0000 7f129098c740 1 -- 172.21.15.82:0/1151959851 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562a17e454d0 msgr2=0x562a18213a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:25:13.485 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:13.485+0000 7f129098c740 1 --2- 172.21.15.82:0/1151959851 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562a17e454d0 0x562a18213a60 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f1278006010 tx=0x7f1278009260 comp rx=0 tx=0).stop 2024-05-20T16:25:13.485 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:13.485+0000 7f129098c740 1 -- 172.21.15.82:0/1151959851 shutdown_connections 2024-05-20T16:25:13.485 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:13.485+0000 7f129098c740 1 --2- 172.21.15.82:0/1151959851 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x562a17dfa1c0 0x562a18287440 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:25:13.485 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:13.485+0000 7f129098c740 1 --2- 172.21.15.82:0/1151959851 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562a17e661b0 0x562a18283450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:25:13.485 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:13.485+0000 7f129098c740 1 --2- 172.21.15.82:0/1151959851 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562a17e454d0 0x562a18213a60 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:25:13.485 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:13.485+0000 7f129098c740 1 --2- 172.21.15.82:0/1151959851 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562a17e3d2f0 0x562a18213520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:25:13.485 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:13.485+0000 7f129098c740 1 -- 172.21.15.82:0/1151959851 >> 172.21.15.82:0/1151959851 conn(0x562a17e5a300 msgr2=0x562a1820ee50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:25:13.485 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:13.485+0000 7f1262ffd640 0 client.25537 ms_handle_reset on v2:172.21.15.160:6838/2765554482 2024-05-20T16:25:13.486 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:13.485+0000 7f129098c740 1 -- 172.21.15.82:0/1151959851 shutdown_connections 2024-05-20T16:25:13.486 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:13.485+0000 7f129098c740 1 -- 172.21.15.82:0/1151959851 wait complete. 2024-05-20T16:25:13.486 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:13.485+0000 7f129098c740 20 client.25537 trim_cache size 0 max 16384 2024-05-20T16:25:13.486 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:{ 2024-05-20T16:25:13.486 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "message": "", 2024-05-20T16:25:13.486 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "return_code": 0 2024-05-20T16:25:13.487 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:} 2024-05-20T16:25:13.501 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:13.501+0000 7f1289ae9640 1 -- 172.21.15.82:0/1553739263 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f126c0a2d50 msgr2=0x7f126c0a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:25:13.501 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:13.502+0000 7f1289ae9640 1 --2- 172.21.15.82:0/1553739263 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f126c0a2d50 0x7f126c0a5260 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f12780315a0 tx=0x7f1278004510 comp rx=0 tx=0).stop 2024-05-20T16:25:13.501 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:13.502+0000 7f1289ae9640 1 -- 172.21.15.82:0/1553739263 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f12841066a0 msgr2=0x7f128410e940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:25:13.502 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:13.502+0000 7f1289ae9640 1 --2- 172.21.15.82:0/1553739263 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f12841066a0 0x7f128410e940 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f127400c8c0 tx=0x7f127400cd80 comp rx=0 tx=0).stop 2024-05-20T16:25:13.502 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:13.502+0000 7f1289ae9640 1 -- 172.21.15.82:0/1553739263 shutdown_connections 2024-05-20T16:25:13.502 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:13.502+0000 7f1289ae9640 1 --2- 172.21.15.82:0/1553739263 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f126c0a2d50 0x7f126c0a5260 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:25:13.502 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:13.502+0000 7f1289ae9640 1 --2- 172.21.15.82:0/1553739263 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f12841066a0 0x7f128410e940 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:25:13.502 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:13.502+0000 7f1289ae9640 1 --2- 172.21.15.82:0/1553739263 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1284103d40 0x7f1284072f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:25:13.503 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:13.502+0000 7f1289ae9640 1 --2- 172.21.15.82:0/1553739263 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f12841013e0 0x7f12840729d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:25:13.503 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:13.502+0000 7f1289ae9640 1 -- 172.21.15.82:0/1553739263 >> 172.21.15.82:0/1553739263 conn(0x7f12840fcc10 msgr2=0x7f12840fd020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:25:13.503 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:13.502+0000 7f1289ae9640 1 -- 172.21.15.82:0/1553739263 shutdown_connections 2024-05-20T16:25:13.503 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:13.503+0000 7f1289ae9640 1 -- 172.21.15.82:0/1553739263 wait complete. 2024-05-20T16:25:13.616 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:25:13 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:25:13.617 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:25:13 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T16:25:13.617 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:25:13 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:25:13.635 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:25:13 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:25:13.635 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:25:13 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T16:25:13.635 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:25:13 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:25:13.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:25:13 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:25:13.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:25:13 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T16:25:13.813 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:25:13 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:25:14.635 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:25:14 smithi184 ceph-mon[36209]: pgmap v702: 129 pgs: 129 active+clean; 10 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 637 MiB/s wr, 1.17k op/s 2024-05-20T16:25:14.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:25:14 smithi160 ceph-mon[36347]: pgmap v702: 129 pgs: 129 active+clean; 10 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 637 MiB/s wr, 1.17k op/s 2024-05-20T16:25:14.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:25:14 smithi082 ceph-mon[32238]: pgmap v702: 129 pgs: 129 active+clean; 10 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 637 MiB/s wr, 1.17k op/s 2024-05-20T16:25:17.312 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:25:16 smithi160 ceph-mon[36347]: pgmap v703: 129 pgs: 129 active+clean; 10 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 601 MiB/s wr, 1.07k op/s 2024-05-20T16:25:17.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:25:16 smithi082 ceph-mon[32238]: pgmap v703: 129 pgs: 129 active+clean; 10 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 601 MiB/s wr, 1.07k op/s 2024-05-20T16:25:17.385 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:25:16 smithi184 ceph-mon[36209]: pgmap v703: 129 pgs: 129 active+clean; 10 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 601 MiB/s wr, 1.07k op/s 2024-05-20T16:25:19.312 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:25:18 smithi160 ceph-mon[36347]: pgmap v704: 129 pgs: 129 active+clean; 10 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 613 MiB/s wr, 1.01k op/s 2024-05-20T16:25:19.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:25:18 smithi082 ceph-mon[32238]: pgmap v704: 129 pgs: 129 active+clean; 10 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 613 MiB/s wr, 1.01k op/s 2024-05-20T16:25:19.385 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:25:18 smithi184 ceph-mon[36209]: pgmap v704: 129 pgs: 129 active+clean; 10 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 613 MiB/s wr, 1.01k op/s 2024-05-20T16:25:21.312 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:25:20 smithi160 ceph-mon[36347]: pgmap v705: 129 pgs: 129 active+clean; 10 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 716 MiB/s wr, 1.09k op/s 2024-05-20T16:25:21.360 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:25:20 smithi082 ceph-mon[32238]: pgmap v705: 129 pgs: 129 active+clean; 10 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 716 MiB/s wr, 1.09k op/s 2024-05-20T16:25:21.385 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:25:20 smithi184 ceph-mon[36209]: pgmap v705: 129 pgs: 129 active+clean; 10 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 716 MiB/s wr, 1.09k op/s 2024-05-20T16:25:23.312 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:25:22 smithi160 ceph-mon[36347]: pgmap v706: 129 pgs: 129 active+clean; 10 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 645 MiB/s wr, 1.04k op/s 2024-05-20T16:25:23.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:25:22 smithi082 ceph-mon[32238]: pgmap v706: 129 pgs: 129 active+clean; 10 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 645 MiB/s wr, 1.04k op/s 2024-05-20T16:25:23.385 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:25:22 smithi184 ceph-mon[36209]: pgmap v706: 129 pgs: 129 active+clean; 10 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 645 MiB/s wr, 1.04k op/s 2024-05-20T16:25:25.312 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:25:24 smithi160 ceph-mon[36347]: pgmap v707: 129 pgs: 129 active+clean; 10 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 776 MiB/s wr, 1.38k op/s 2024-05-20T16:25:25.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:25:24 smithi082 ceph-mon[32238]: pgmap v707: 129 pgs: 129 active+clean; 10 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 776 MiB/s wr, 1.38k op/s 2024-05-20T16:25:25.385 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:25:24 smithi184 ceph-mon[36209]: pgmap v707: 129 pgs: 129 active+clean; 10 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 776 MiB/s wr, 1.38k op/s 2024-05-20T16:25:26.312 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:25:25 smithi160 ceph-mon[36347]: pgmap v708: 129 pgs: 129 active+clean; 10 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 677 MiB/s wr, 1.17k op/s 2024-05-20T16:25:26.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:25:25 smithi082 ceph-mon[32238]: pgmap v708: 129 pgs: 129 active+clean; 10 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 677 MiB/s wr, 1.17k op/s 2024-05-20T16:25:26.385 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:25:25 smithi184 ceph-mon[36209]: pgmap v708: 129 pgs: 129 active+clean; 10 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 677 MiB/s wr, 1.17k op/s 2024-05-20T16:25:26.722 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:26.720+0000 7efe10a2a640 1 -- 172.21.15.82:0/3546080386 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efe0c10ee60 msgr2=0x7efe0c10f270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:25:26.722 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:26.720+0000 7efe10a2a640 1 --2- 172.21.15.82:0/3546080386 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efe0c10ee60 0x7efe0c10f270 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7efe04009ca0 tx=0x7efe0402a8c0 comp rx=0 tx=0).stop 2024-05-20T16:25:26.729 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:26.728+0000 7efe10a2a640 1 -- 172.21.15.82:0/3546080386 shutdown_connections 2024-05-20T16:25:26.729 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:26.728+0000 7efe10a2a640 1 --2- 172.21.15.82:0/3546080386 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efe0c10a140 0x7efe0c10a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:25:26.731 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:26.728+0000 7efe10a2a640 1 --2- 172.21.15.82:0/3546080386 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efe0c10ee60 0x7efe0c10f270 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:25:26.731 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:26.728+0000 7efe10a2a640 1 --2- 172.21.15.82:0/3546080386 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efe0c1092f0 0x7efe0c109c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:25:26.731 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:26.728+0000 7efe10a2a640 1 -- 172.21.15.82:0/3546080386 >> 172.21.15.82:0/3546080386 conn(0x7efe0c06f2e0 msgr2=0x7efe0c071790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:25:26.731 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:26.730+0000 7efe10a2a640 1 -- 172.21.15.82:0/3546080386 shutdown_connections 2024-05-20T16:25:26.733 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:26.732+0000 7efe10a2a640 1 -- 172.21.15.82:0/3546080386 wait complete. 2024-05-20T16:25:26.734 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:26.733+0000 7efe10a2a640 1 Processor -- start 2024-05-20T16:25:26.736 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:26.735+0000 7efe10a2a640 1 -- start start 2024-05-20T16:25:26.736 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:26.735+0000 7efe10a2a640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efe0c1092f0 0x7efe0c1a7e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:25:26.736 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:26.735+0000 7efe10a2a640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efe0c10a140 0x7efe0c1a83c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:25:26.736 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:26.735+0000 7efe10a2a640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efe0c10ee60 0x7efe0c1af350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:25:26.736 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:26.735+0000 7efe10a2a640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7efe0c11d330 con 0x7efe0c10a140 2024-05-20T16:25:26.736 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:26.735+0000 7efe10a2a640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7efe0c11d1b0 con 0x7efe0c10ee60 2024-05-20T16:25:26.736 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:26.735+0000 7efe10a2a640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7efe0c11d4b0 con 0x7efe0c1092f0 2024-05-20T16:25:26.737 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:26.737+0000 7efe0a59c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efe0c10a140 0x7efe0c1a83c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:25:26.737 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:26.737+0000 7efe0a59c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efe0c10a140 0x7efe0c1a83c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:42260/0 (socket says 172.21.15.82:42260) 2024-05-20T16:25:26.737 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:26.737+0000 7efe0a59c640 1 -- 172.21.15.82:0/3324680962 learned_addr learned my addr 172.21.15.82:0/3324680962 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:25:26.739 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:26.738+0000 7efe0a59c640 1 -- 172.21.15.82:0/3324680962 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efe0c1092f0 msgr2=0x7efe0c1a7e80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-20T16:25:26.739 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:26.738+0000 7efe0a59c640 1 --2- 172.21.15.82:0/3324680962 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efe0c1092f0 0x7efe0c1a7e80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:25:26.739 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:26.738+0000 7efe0a59c640 1 -- 172.21.15.82:0/3324680962 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efe0c10ee60 msgr2=0x7efe0c1af350 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-20T16:25:26.739 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:26.738+0000 7efe0a59c640 1 --2- 172.21.15.82:0/3324680962 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efe0c10ee60 0x7efe0c1af350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:25:26.739 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:26.738+0000 7efe0a59c640 1 -- 172.21.15.82:0/3324680962 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe0c110c40 con 0x7efe0c10a140 2024-05-20T16:25:26.744 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:26.744+0000 7efe0a59c640 1 --2- 172.21.15.82:0/3324680962 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efe0c10a140 0x7efe0c1a83c0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7efe0000d950 tx=0x7efe0000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:25:26.750 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:26.747+0000 7efdf3fff640 1 -- 172.21.15.82:0/3324680962 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe00017070 con 0x7efe0c10a140 2024-05-20T16:25:26.750 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:26.747+0000 7efdf3fff640 1 -- 172.21.15.82:0/3324680962 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe00009c30 con 0x7efe0c10a140 2024-05-20T16:25:26.750 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:26.747+0000 7efdf3fff640 1 -- 172.21.15.82:0/3324680962 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe0000f600 con 0x7efe0c10a140 2024-05-20T16:25:26.750 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:26.750+0000 7efe10a2a640 1 -- 172.21.15.82:0/3324680962 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe0c110f30 con 0x7efe0c10a140 2024-05-20T16:25:26.751 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:26.750+0000 7efe10a2a640 1 -- 172.21.15.82:0/3324680962 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe0c111430 con 0x7efe0c10a140 2024-05-20T16:25:26.757 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:26.756+0000 7efe181ad740 1 Processor -- start 2024-05-20T16:25:26.758 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:26.757+0000 7efe181ad740 1 -- start start 2024-05-20T16:25:26.759 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:26.759+0000 7efe181ad740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556c0ca991b0 0x556c0ce39050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:25:26.760 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:26.759+0000 7efe181ad740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556c0ca702f0 0x556c0ce395d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:25:26.760 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:26.759+0000 7efe181ad740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556c0ca784d0 0x556c0ce404d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:25:26.760 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:26.759+0000 7efe181ad740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x556c0cd9d340 con 0x556c0ca702f0 2024-05-20T16:25:26.760 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:26.759+0000 7efe181ad740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x556c0cd45290 con 0x556c0ca784d0 2024-05-20T16:25:26.760 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:26.759+0000 7efe181ad740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x556c0cd99300 con 0x556c0ca991b0 2024-05-20T16:25:26.765 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:26.763+0000 7efe0b59e640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556c0ca991b0 0x556c0ce39050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:25:26.765 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:26.763+0000 7efe0a59c640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556c0ca784d0 0x556c0ce404d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:25:26.765 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:26.764+0000 7efe0b59e640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556c0ca991b0 0x556c0ce39050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.82:36248/0 (socket says 172.21.15.82:36248) 2024-05-20T16:25:26.765 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:26.764+0000 7efe0b59e640 1 -- 172.21.15.82:0/2446600671 learned_addr learned my addr 172.21.15.82:0/2446600671 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:25:26.767 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:26.766+0000 7efe0ad9d640 1 --2- 172.21.15.82:0/2446600671 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556c0ca702f0 0x556c0ce395d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:25:26.767 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:26.767+0000 7efdf3fff640 1 -- 172.21.15.82:0/3324680962 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7efe00005040 con 0x7efe0c10a140 2024-05-20T16:25:26.769 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:26.768+0000 7efdf3fff640 1 --2- 172.21.15.82:0/3324680962 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7efdf40a2d90 0x7efdf40a52a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:25:26.769 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:26.768+0000 7efe0b59e640 1 -- 172.21.15.82:0/2446600671 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556c0ca784d0 msgr2=0x556c0ce404d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:25:26.770 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:26.768+0000 7efe0b59e640 1 --2- 172.21.15.82:0/2446600671 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556c0ca784d0 0x556c0ce404d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:25:26.770 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:26.768+0000 7efe0b59e640 1 -- 172.21.15.82:0/2446600671 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556c0ca702f0 msgr2=0x556c0ce395d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:25:26.770 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:26.768+0000 7efe0b59e640 1 --2- 172.21.15.82:0/2446600671 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556c0ca702f0 0x556c0ce395d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:25:26.770 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:26.768+0000 7efe0b59e640 1 -- 172.21.15.82:0/2446600671 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556c0cd698e0 con 0x556c0ca991b0 2024-05-20T16:25:26.772 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:26.771+0000 7efdf3fff640 1 -- 172.21.15.82:0/3324680962 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9150+0+0 (secure 0 0 0) 0x7efe000d2360 con 0x7efe0c10a140 2024-05-20T16:25:26.775 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:26.774+0000 7efe0b59e640 1 --2- 172.21.15.82:0/3324680962 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7efdf40a2d90 0x7efdf40a52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:25:26.776 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:26.775+0000 7efe0b59e640 1 --2- 172.21.15.82:0/2446600671 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556c0ca991b0 0x556c0ce39050 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7efe04009a50 tx=0x7efe04004a60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d5774e72c1a4f1b2 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:25:26.780 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:26.779+0000 7efe0b59e640 1 --2- 172.21.15.82:0/3324680962 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7efdf40a2d90 0x7efdf40a52a0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7efe04030930 tx=0x7efe04003f50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:25:26.787 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:26.786+0000 7efe10a2a640 1 -- 172.21.15.82:0/2446600671 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe0404b070 con 0x556c0ca991b0 2024-05-20T16:25:26.787 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:26.786+0000 7efe10a2a640 1 -- 172.21.15.82:0/2446600671 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe0403f070 con 0x556c0ca991b0 2024-05-20T16:25:26.788 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:26.788+0000 7efe181ad740 1 -- 172.21.15.82:0/2446600671 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556c0ca991b0 msgr2=0x556c0ce39050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:25:26.789 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:26.788+0000 7efe181ad740 1 --2- 172.21.15.82:0/2446600671 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556c0ca991b0 0x556c0ce39050 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7efe04009a50 tx=0x7efe04004a60 comp rx=0 tx=0).stop 2024-05-20T16:25:26.792 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:26.789+0000 7efe181ad740 1 -- 172.21.15.82:0/2446600671 shutdown_connections 2024-05-20T16:25:26.792 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:26.790+0000 7efe181ad740 1 --2- 172.21.15.82:0/2446600671 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556c0ca702f0 0x556c0ce395d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:25:26.792 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:26.790+0000 7efe181ad740 1 --2- 172.21.15.82:0/2446600671 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556c0ca784d0 0x556c0ce404d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:25:26.792 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:26.790+0000 7efe181ad740 1 --2- 172.21.15.82:0/2446600671 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556c0ca991b0 0x556c0ce39050 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:25:26.792 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:26.790+0000 7efe181ad740 1 -- 172.21.15.82:0/2446600671 >> 172.21.15.82:0/2446600671 conn(0x556c0ca8d300 msgr2=0x556c0ce34ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:25:26.792 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:26.791+0000 7efe181ad740 1 -- 172.21.15.82:0/2446600671 shutdown_connections 2024-05-20T16:25:26.792 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:26.791+0000 7efe181ad740 1 -- 172.21.15.82:0/2446600671 wait complete. 2024-05-20T16:25:26.792 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:26.791+0000 7efe181ad740 1 -- start start 2024-05-20T16:25:26.793 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:26.792+0000 7efe181ad740 1 Processor -- start 2024-05-20T16:25:26.793 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:26.792+0000 7efe181ad740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556c0ca702f0 0x556c0ce42e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:25:26.793 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:26.792+0000 7efe181ad740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556c0ca784d0 0x556c0ce43360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:25:26.793 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:26.792+0000 7efe181ad740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556c0ca991b0 0x556c0ce4a250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:25:26.793 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:26.792+0000 7efe181ad740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x556c0ce40f70 con 0x556c0ca991b0 2024-05-20T16:25:26.793 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:26.792+0000 7efe181ad740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x556c0cd58790 con 0x556c0ca702f0 2024-05-20T16:25:26.793 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:26.792+0000 7efe181ad740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x556c0ce410f0 con 0x556c0ca784d0 2024-05-20T16:25:26.793 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:26.792+0000 7efe181ad740 10 client.0 _ll_register_callbacks cb 0x556c0cd93a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T16:25:26.797 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:26.797+0000 7efe0ad9d640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556c0ca702f0 0x556c0ce42e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:25:26.798 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:26.797+0000 7efe0ad9d640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556c0ca702f0 0x556c0ce42e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.82:37988/0 (socket says 172.21.15.82:37988) 2024-05-20T16:25:26.798 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:26.797+0000 7efe0ad9d640 1 -- 172.21.15.82:0/2108573270 learned_addr learned my addr 172.21.15.82:0/2108573270 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:25:26.798 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:26.797+0000 7efe0b59e640 1 --2- 172.21.15.82:0/2108573270 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556c0ca991b0 0x556c0ce4a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:25:26.798 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:26.797+0000 7efe0a59c640 1 --2- 172.21.15.82:0/2108573270 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556c0ca784d0 0x556c0ce43360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:25:26.805 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:26.805+0000 7efe0ad9d640 1 -- 172.21.15.82:0/2108573270 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556c0ca784d0 msgr2=0x556c0ce43360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:25:26.806 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:26.805+0000 7efe0ad9d640 1 --2- 172.21.15.82:0/2108573270 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556c0ca784d0 0x556c0ce43360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:25:26.806 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:26.805+0000 7efe0ad9d640 1 -- 172.21.15.82:0/2108573270 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556c0ca991b0 msgr2=0x556c0ce4a250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:25:26.806 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:26.805+0000 7efe0ad9d640 1 --2- 172.21.15.82:0/2108573270 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556c0ca991b0 0x556c0ce4a250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:25:26.806 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:26.805+0000 7efe0ad9d640 1 -- 172.21.15.82:0/2108573270 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556c0cab1210 con 0x556c0ca702f0 2024-05-20T16:25:26.810 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:26.810+0000 7efe0ad9d640 1 --2- 172.21.15.82:0/2108573270 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556c0ca702f0 0x556c0ce42e20 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7efdfc0114c0 tx=0x7efdfc011980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b4422f627963e010 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:25:26.810 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:26.810+0000 7efdeeff5640 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-20T16:25:26.811 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:26.811+0000 7efdeeff5640 1 -- 172.21.15.82:0/2108573270 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdfc004300 con 0x556c0ca702f0 2024-05-20T16:25:26.812 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:26.811+0000 7efe181ad740 1 -- 172.21.15.82:0/2108573270 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efe0403f070 con 0x556c0ca702f0 2024-05-20T16:25:26.812 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:26.811+0000 7efdeeff5640 1 -- 172.21.15.82:0/2108573270 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9150+0+0 (secure 0 0 0) 0x7efdfc005020 con 0x556c0ca702f0 2024-05-20T16:25:26.814 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:26.813+0000 7efdeeff5640 1 -- 172.21.15.82:0/2108573270 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7efdfc023050 con 0x556c0ca702f0 2024-05-20T16:25:26.814 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:26.813+0000 7efe181ad740 10 client.25496 fetch_fsmap learned FSMap version 31 2024-05-20T16:25:26.814 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:26.813+0000 7efe181ad740 1 -- 172.21.15.82:0/2108573270 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x556c0cb7d620 con 0x556c0ca702f0 2024-05-20T16:25:26.818 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:26.818+0000 7efdeeff5640 1 -- 172.21.15.82:0/2108573270 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe0403f070 con 0x556c0ca702f0 2024-05-20T16:25:26.819 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:26.818+0000 7efdeeff5640 1 -- 172.21.15.82:0/2108573270 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7efdfc0035f0 con 0x556c0ca702f0 2024-05-20T16:25:26.819 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:26.818+0000 7efe181ad740 10 client.25496 fetch_fsmap finished waiting for FSMap version 31 2024-05-20T16:25:26.819 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:26.818+0000 7efe181ad740 10 client.25496 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T16:25:26.819 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:26.818+0000 7efe181ad740 20 client.25496 populate_metadata read hostname 'smithi082' 2024-05-20T16:25:26.819 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:26.818+0000 7efe181ad740 1 --2- 172.21.15.82:0/2108573270 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x556c0ca57d80 0x556c0ce50ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:25:26.819 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:26.818+0000 7efe181ad740 4 client.25496 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T16:25:26.819 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:26.818+0000 7efe181ad740 1 -- 172.21.15.82:0/2108573270 --> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556c0cdaaa70 con 0x556c0ca57d80 2024-05-20T16:25:26.820 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:26.819+0000 7efe0a59c640 1 --2- 172.21.15.82:0/2108573270 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x556c0ca57d80 0x556c0ce50ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:25:26.836 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:26.835+0000 7efe0a59c640 1 --2- 172.21.15.82:0/2108573270 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x556c0ca57d80 0x556c0ce50ca0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4b95d599859e9f00 server_cookie=b48c0b7f5a6057be in_seq=0 out_seq=0 2024-05-20T16:25:26.836 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:26.836+0000 7efdeeff5640 10 client.25496 ms_handle_connect on v2:172.21.15.160:6838/2765554482 2024-05-20T16:25:26.841 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:26.840+0000 7efdeeff5640 1 -- 172.21.15.82:0/2108573270 <== mds.0 v2:172.21.15.160:6838/2765554482 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x556c0cdaaa70 con 0x556c0ca57d80 2024-05-20T16:25:26.841 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:26.840+0000 7efdeeff5640 10 client.25496 handle_command_reply: tid=0 2024-05-20T16:25:26.841 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:26.841+0000 7efe181ad740 1 client.25496 shutdown 2024-05-20T16:25:26.856 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:26.856+0000 7efe181ad740 1 -- 172.21.15.82:0/2108573270 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556c0ca702f0 msgr2=0x556c0ce42e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:25:26.857 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:26.856+0000 7efe181ad740 1 --2- 172.21.15.82:0/2108573270 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556c0ca702f0 0x556c0ce42e20 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7efdfc0114c0 tx=0x7efdfc011980 comp rx=0 tx=0).stop 2024-05-20T16:25:26.857 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:26.856+0000 7efe181ad740 1 -- 172.21.15.82:0/2108573270 shutdown_connections 2024-05-20T16:25:26.857 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:26.856+0000 7efe181ad740 1 --2- 172.21.15.82:0/2108573270 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x556c0ca57d80 0x556c0ce50ca0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:25:26.857 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:26.856+0000 7efe181ad740 1 --2- 172.21.15.82:0/2108573270 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556c0ca991b0 0x556c0ce4a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:25:26.857 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:26.856+0000 7efe181ad740 1 --2- 172.21.15.82:0/2108573270 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556c0ca784d0 0x556c0ce43360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:25:26.857 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:26.856+0000 7efe181ad740 1 --2- 172.21.15.82:0/2108573270 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556c0ca702f0 0x556c0ce42e20 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:25:26.857 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:26.856+0000 7efe181ad740 1 -- 172.21.15.82:0/2108573270 >> 172.21.15.82:0/2108573270 conn(0x556c0ca8d300 msgr2=0x556c0ce33e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:25:26.857 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:26.856+0000 7efdeeff5640 0 client.25496 ms_handle_reset on v2:172.21.15.160:6838/2765554482 2024-05-20T16:25:26.857 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:26.857+0000 7efe181ad740 1 -- 172.21.15.82:0/2108573270 shutdown_connections 2024-05-20T16:25:26.859 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:26.859+0000 7efe181ad740 1 -- 172.21.15.82:0/2108573270 wait complete. 2024-05-20T16:25:26.862 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:26.861+0000 7efe181ad740 20 client.25496 trim_cache size 0 max 16384 2024-05-20T16:25:26.907 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:26.906+0000 7efe181ad740 1 Processor -- start 2024-05-20T16:25:26.908 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:26.907+0000 7efe181ad740 1 -- start start 2024-05-20T16:25:26.909 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:26.907+0000 7efe181ad740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556c0ca702f0 0x556c0ce430c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:25:26.909 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:26.907+0000 7efe181ad740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556c0ca784d0 0x556c0ce38290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:25:26.909 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:26.907+0000 7efe181ad740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556c0ca991b0 0x556c0ce4a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:25:26.909 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:26.907+0000 7efe181ad740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7efdf80125f0 con 0x556c0ca991b0 2024-05-20T16:25:26.909 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:26.907+0000 7efe181ad740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7efdf800fec0 con 0x556c0ca784d0 2024-05-20T16:25:26.909 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:26.907+0000 7efe181ad740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7efdf800dd10 con 0x556c0ca702f0 2024-05-20T16:25:26.911 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:26.910+0000 7efe0a59c640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556c0ca784d0 0x556c0ce38290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:25:26.911 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:26.910+0000 7efe0b59e640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556c0ca991b0 0x556c0ce4a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:25:26.911 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:26.910+0000 7efe0b59e640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556c0ca991b0 0x556c0ce4a1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:42282/0 (socket says 172.21.15.82:42282) 2024-05-20T16:25:26.911 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:26.910+0000 7efe0b59e640 1 -- 172.21.15.82:0/2879222122 learned_addr learned my addr 172.21.15.82:0/2879222122 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:25:26.912 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:26.911+0000 7efe0ad9d640 1 --2- 172.21.15.82:0/2879222122 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556c0ca702f0 0x556c0ce430c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:25:26.915 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:26.914+0000 7efe0b59e640 1 -- 172.21.15.82:0/2879222122 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556c0ca702f0 msgr2=0x556c0ce430c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:25:26.915 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:26.914+0000 7efe0b59e640 1 --2- 172.21.15.82:0/2879222122 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556c0ca702f0 0x556c0ce430c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:25:26.915 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:26.914+0000 7efe0b59e640 1 -- 172.21.15.82:0/2879222122 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556c0ca784d0 msgr2=0x556c0ce38290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:25:26.915 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:26.914+0000 7efe0b59e640 1 --2- 172.21.15.82:0/2879222122 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556c0ca784d0 0x556c0ce38290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:25:26.915 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:26.914+0000 7efe0b59e640 1 -- 172.21.15.82:0/2879222122 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556c0ce40530 con 0x556c0ca991b0 2024-05-20T16:25:26.917 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:26.916+0000 7efe0b59e640 1 --2- 172.21.15.82:0/2879222122 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556c0ca991b0 0x556c0ce4a1e0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7efe0402a900 tx=0x7efe0404f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=628b496f42d8b469 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:25:26.917 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:26.917+0000 7efdf0ff9640 1 -- 172.21.15.82:0/2879222122 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe0404b040 con 0x556c0ca991b0 2024-05-20T16:25:26.918 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:26.917+0000 7efdf0ff9640 1 -- 172.21.15.82:0/2879222122 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe04036030 con 0x556c0ca991b0 2024-05-20T16:25:26.918 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:26.917+0000 7efdf0ff9640 1 -- 172.21.15.82:0/2879222122 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe04030a00 con 0x556c0ca991b0 2024-05-20T16:25:26.919 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:26.918+0000 7efe181ad740 1 -- 172.21.15.82:0/2879222122 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556c0ca991b0 msgr2=0x556c0ce4a1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:25:26.919 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:26.918+0000 7efe181ad740 1 --2- 172.21.15.82:0/2879222122 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556c0ca991b0 0x556c0ce4a1e0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7efe0402a900 tx=0x7efe0404f000 comp rx=0 tx=0).stop 2024-05-20T16:25:26.920 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:26.920+0000 7efe181ad740 1 -- 172.21.15.82:0/2879222122 shutdown_connections 2024-05-20T16:25:26.920 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:26.920+0000 7efe181ad740 1 --2- 172.21.15.82:0/2879222122 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556c0ca991b0 0x556c0ce4a1e0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:25:26.920 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:26.920+0000 7efe181ad740 1 --2- 172.21.15.82:0/2879222122 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556c0ca784d0 0x556c0ce38290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:25:26.921 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:26.920+0000 7efe181ad740 1 --2- 172.21.15.82:0/2879222122 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556c0ca702f0 0x556c0ce430c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:25:26.921 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:26.920+0000 7efe181ad740 1 -- 172.21.15.82:0/2879222122 >> 172.21.15.82:0/2879222122 conn(0x556c0ca8d300 msgr2=0x556c0ce37b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:25:26.921 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:26.921+0000 7efe181ad740 1 -- 172.21.15.82:0/2879222122 shutdown_connections 2024-05-20T16:25:26.922 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:26.921+0000 7efe181ad740 1 -- 172.21.15.82:0/2879222122 wait complete. 2024-05-20T16:25:26.922 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:26.921+0000 7efe181ad740 1 -- start start 2024-05-20T16:25:26.923 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:26.922+0000 7efe181ad740 1 Processor -- start 2024-05-20T16:25:26.923 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:26.922+0000 7efe181ad740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556c0ca702f0 0x556c0ce476e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:25:26.923 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:26.922+0000 7efe181ad740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556c0ca784d0 0x556c0ce47c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:25:26.923 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:26.922+0000 7efe181ad740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556c0ca991b0 0x556c0ceb7040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:25:26.923 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:26.922+0000 7efe181ad740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7efdf800e380 con 0x556c0ca702f0 2024-05-20T16:25:26.923 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:26.922+0000 7efe181ad740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7efdf800f7f0 con 0x556c0ca784d0 2024-05-20T16:25:26.923 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:26.922+0000 7efe181ad740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7efdf800ea50 con 0x556c0ca991b0 2024-05-20T16:25:26.924 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:26.922+0000 7efe181ad740 10 client.0 _ll_register_callbacks cb 0x7efdfc0096a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T16:25:26.924 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:26.923+0000 7efe0ad9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556c0ca702f0 0x556c0ce476e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:25:26.925 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:26.923+0000 7efe0ad9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556c0ca702f0 0x556c0ce476e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:42290/0 (socket says 172.21.15.82:42290) 2024-05-20T16:25:26.925 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:26.923+0000 7efe0ad9d640 1 -- 172.21.15.82:0/506037227 learned_addr learned my addr 172.21.15.82:0/506037227 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:25:26.925 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:26.923+0000 7efe0a59c640 1 --2- 172.21.15.82:0/506037227 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556c0ca784d0 0x556c0ce47c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:25:26.925 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:26.923+0000 7efe0b59e640 1 --2- 172.21.15.82:0/506037227 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556c0ca991b0 0x556c0ceb7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:25:26.925 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:26.924+0000 7efe0ad9d640 1 -- 172.21.15.82:0/506037227 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556c0ca991b0 msgr2=0x556c0ceb7040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:25:26.925 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:26.924+0000 7efe0ad9d640 1 --2- 172.21.15.82:0/506037227 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556c0ca991b0 0x556c0ceb7040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:25:26.925 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:26.924+0000 7efe0ad9d640 1 -- 172.21.15.82:0/506037227 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556c0ca784d0 msgr2=0x556c0ce47c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T16:25:26.925 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:26.924+0000 7efe0ad9d640 1 --2- 172.21.15.82:0/506037227 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556c0ca784d0 0x556c0ce47c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:25:26.925 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:26.924+0000 7efe0ad9d640 1 -- 172.21.15.82:0/506037227 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556c0ce4abe0 con 0x556c0ca702f0 2024-05-20T16:25:26.926 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:26.924+0000 7efe0ad9d640 1 --2- 172.21.15.82:0/506037227 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556c0ca702f0 0x556c0ce476e0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7efdfc0058c0 tx=0x7efdfc004530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9e1294013350c11 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:25:26.926 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:26.924+0000 7efdeeff5640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-05-20T16:25:26.926 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:26.925+0000 7efdeeff5640 1 -- 172.21.15.82:0/506037227 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdfc0037f0 con 0x556c0ca702f0 2024-05-20T16:25:26.926 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:26.925+0000 7efe181ad740 1 -- 172.21.15.82:0/506037227 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efe04036030 con 0x556c0ca702f0 2024-05-20T16:25:26.926 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:26.925+0000 7efdeeff5640 1 -- 172.21.15.82:0/506037227 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9150+0+0 (secure 0 0 0) 0x7efdfc005040 con 0x556c0ca702f0 2024-05-20T16:25:26.929 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:26.929+0000 7efdeeff5640 1 -- 172.21.15.82:0/506037227 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7efdfc038050 con 0x556c0ca702f0 2024-05-20T16:25:26.930 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:26.929+0000 7efe181ad740 10 client.15996 fetch_fsmap learned FSMap version 31 2024-05-20T16:25:26.930 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:26.929+0000 7efe181ad740 1 -- 172.21.15.82:0/506037227 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x556c0ceb8430 con 0x556c0ca702f0 2024-05-20T16:25:26.931 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:26.930+0000 7efdeeff5640 1 -- 172.21.15.82:0/506037227 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe04036030 con 0x556c0ca702f0 2024-05-20T16:25:26.931 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:26.930+0000 7efdeeff5640 1 -- 172.21.15.82:0/506037227 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7efdfc00f540 con 0x556c0ca702f0 2024-05-20T16:25:26.931 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:26.930+0000 7efe181ad740 10 client.15996 fetch_fsmap finished waiting for FSMap version 31 2024-05-20T16:25:26.931 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:26.930+0000 7efe181ad740 10 client.15996 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-20T16:25:26.931 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:26.930+0000 7efe181ad740 20 client.15996 populate_metadata read hostname 'smithi082' 2024-05-20T16:25:26.931 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:26.930+0000 7efe181ad740 1 --2- 172.21.15.82:0/506037227 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x556c0ca2d1c0 0x556c0cebae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:25:26.931 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:26.930+0000 7efe181ad740 4 client.15996 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-20T16:25:26.931 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:26.930+0000 7efe181ad740 1 -- 172.21.15.82:0/506037227 --> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556c0cda4dc0 con 0x556c0ca2d1c0 2024-05-20T16:25:26.932 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:26.931+0000 7efe0a59c640 1 --2- 172.21.15.82:0/506037227 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x556c0ca2d1c0 0x556c0cebae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:25:26.933 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:26.932+0000 7efdeeff5640 10 client.15996 ms_handle_connect on v2:172.21.15.160:6838/2765554482 2024-05-20T16:25:26.933 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:26.932+0000 7efe0a59c640 1 --2- 172.21.15.82:0/506037227 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x556c0ca2d1c0 0x556c0cebae50 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3d613a11fed2c176 server_cookie=b5deb91c8b1ddf7b in_seq=0 out_seq=0 2024-05-20T16:25:29.311 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:25:28 smithi160 ceph-mon[36347]: pgmap v709: 129 pgs: 129 active+clean; 10 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 657 MiB/s wr, 1.20k op/s 2024-05-20T16:25:29.339 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:29.339+0000 7efdeeff5640 1 -- 172.21.15.82:0/506037227 <== mds.0 v2:172.21.15.160:6838/2765554482 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556c0cda4dc0 con 0x556c0ca2d1c0 2024-05-20T16:25:29.339 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:29.339+0000 7efdeeff5640 10 client.15996 handle_command_reply: tid=0 2024-05-20T16:25:29.339 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:29.339+0000 7efe181ad740 1 client.15996 shutdown 2024-05-20T16:25:29.340 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:29.341+0000 7efe181ad740 1 -- 172.21.15.82:0/506037227 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556c0ca702f0 msgr2=0x556c0ce476e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:25:29.340 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:29.341+0000 7efe181ad740 1 --2- 172.21.15.82:0/506037227 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556c0ca702f0 0x556c0ce476e0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7efdfc0058c0 tx=0x7efdfc004530 comp rx=0 tx=0).stop 2024-05-20T16:25:29.341 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:29.341+0000 7efe181ad740 1 -- 172.21.15.82:0/506037227 shutdown_connections 2024-05-20T16:25:29.341 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:29.341+0000 7efe181ad740 1 --2- 172.21.15.82:0/506037227 >> [v2:172.21.15.160:6838/2765554482,v1:172.21.15.160:6839/2765554482] conn(0x556c0ca2d1c0 0x556c0cebae50 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:25:29.341 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:29.341+0000 7efe181ad740 1 --2- 172.21.15.82:0/506037227 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556c0ca991b0 0x556c0ceb7040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:25:29.341 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:29.341+0000 7efe181ad740 1 --2- 172.21.15.82:0/506037227 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556c0ca784d0 0x556c0ce47c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:25:29.341 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:29.341+0000 7efe181ad740 1 --2- 172.21.15.82:0/506037227 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556c0ca702f0 0x556c0ce476e0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:25:29.342 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:29.341+0000 7efe181ad740 1 -- 172.21.15.82:0/506037227 >> 172.21.15.82:0/506037227 conn(0x556c0ca8d300 msgr2=0x556c0ce385b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:25:29.342 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:29.341+0000 7efdeeff5640 0 client.15996 ms_handle_reset on v2:172.21.15.160:6838/2765554482 2024-05-20T16:25:29.342 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:29.342+0000 7efe181ad740 1 -- 172.21.15.82:0/506037227 shutdown_connections 2024-05-20T16:25:29.342 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:29.342+0000 7efe181ad740 1 -- 172.21.15.82:0/506037227 wait complete. 2024-05-20T16:25:29.342 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:29.342+0000 7efe181ad740 20 client.15996 trim_cache size 0 max 16384 2024-05-20T16:25:29.343 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:{ 2024-05-20T16:25:29.343 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "message": "", 2024-05-20T16:25:29.343 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "return_code": 0 2024-05-20T16:25:29.343 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:} 2024-05-20T16:25:29.344 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:29.343+0000 7efdf1ffb640 1 -- 172.21.15.82:0/3324680962 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7efdf40a2d90 msgr2=0x7efdf40a52a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:25:29.344 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:29.343+0000 7efdf1ffb640 1 --2- 172.21.15.82:0/3324680962 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7efdf40a2d90 0x7efdf40a52a0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7efe04030930 tx=0x7efe04003f50 comp rx=0 tx=0).stop 2024-05-20T16:25:29.344 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:29.343+0000 7efdf1ffb640 1 -- 172.21.15.82:0/3324680962 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efe0c10a140 msgr2=0x7efe0c1a83c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:25:29.344 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:29.343+0000 7efdf1ffb640 1 --2- 172.21.15.82:0/3324680962 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efe0c10a140 0x7efe0c1a83c0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7efe0000d950 tx=0x7efe0000de10 comp rx=0 tx=0).stop 2024-05-20T16:25:29.344 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:29.343+0000 7efdf1ffb640 1 -- 172.21.15.82:0/3324680962 shutdown_connections 2024-05-20T16:25:29.344 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:29.343+0000 7efdf1ffb640 1 --2- 172.21.15.82:0/3324680962 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7efdf40a2d90 0x7efdf40a52a0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:25:29.344 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:29.343+0000 7efdf1ffb640 1 --2- 172.21.15.82:0/3324680962 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efe0c10a140 0x7efe0c1a83c0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:25:29.345 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:29.343+0000 7efdf1ffb640 1 --2- 172.21.15.82:0/3324680962 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efe0c10ee60 0x7efe0c1af350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:25:29.345 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:29.343+0000 7efdf1ffb640 1 --2- 172.21.15.82:0/3324680962 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efe0c1092f0 0x7efe0c1a7e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:25:29.345 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:29.343+0000 7efdf1ffb640 1 -- 172.21.15.82:0/3324680962 >> 172.21.15.82:0/3324680962 conn(0x7efe0c06f2e0 msgr2=0x7efe0c10ae40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:25:29.345 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:29.344+0000 7efdf1ffb640 1 -- 172.21.15.82:0/3324680962 shutdown_connections 2024-05-20T16:25:29.345 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-05-20T16:25:29.344+0000 7efdf1ffb640 1 -- 172.21.15.82:0/3324680962 wait complete. 2024-05-20T16:25:29.358 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:25:28 smithi082 ceph-mon[32238]: pgmap v709: 129 pgs: 129 active+clean; 10 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 657 MiB/s wr, 1.20k op/s 2024-05-20T16:25:29.385 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:25:28 smithi184 ceph-mon[36209]: pgmap v709: 129 pgs: 129 active+clean; 10 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 657 MiB/s wr, 1.20k op/s 2024-05-20T16:25:30.312 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:25:30 smithi160 ceph-mon[36347]: pgmap v710: 129 pgs: 129 active+clean; 10 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 810 MiB/s wr, 1.45k op/s 2024-05-20T16:25:30.361 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:25:30 smithi082 ceph-mon[32238]: pgmap v710: 129 pgs: 129 active+clean; 10 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 810 MiB/s wr, 1.45k op/s 2024-05-20T16:25:30.385 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:25:30 smithi184 ceph-mon[36209]: pgmap v710: 129 pgs: 129 active+clean; 10 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 810 MiB/s wr, 1.45k op/s 2024-05-20T16:25:33.312 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:25:32 smithi160 ceph-mon[36347]: pgmap v711: 129 pgs: 129 active+clean; 10 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 692 MiB/s wr, 1.31k op/s 2024-05-20T16:25:33.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:25:32 smithi082 ceph-mon[32238]: pgmap v711: 129 pgs: 129 active+clean; 10 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 692 MiB/s wr, 1.31k op/s 2024-05-20T16:25:33.385 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:25:32 smithi184 ceph-mon[36209]: pgmap v711: 129 pgs: 129 active+clean; 10 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 692 MiB/s wr, 1.31k op/s 2024-05-20T16:25:35.312 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:25:34 smithi160 ceph-mon[36347]: pgmap v712: 129 pgs: 129 active+clean; 10 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 795 MiB/s wr, 1.52k op/s 2024-05-20T16:25:35.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:25:34 smithi082 ceph-mon[32238]: pgmap v712: 129 pgs: 129 active+clean; 10 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 795 MiB/s wr, 1.52k op/s 2024-05-20T16:25:35.360 INFO:tasks.workunit.client.0.smithi082.stdout:stopping iogen 2024-05-20T16:25:35.360 INFO:tasks.workunit.client.0.smithi082.stderr:+ echo 'stopping iogen' 2024-05-20T16:25:35.360 INFO:tasks.workunit.client.0.smithi082.stderr:+ ./iogen -k 2024-05-20T16:25:35.385 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:25:34 smithi184 ceph-mon[36209]: pgmap v712: 129 pgs: 129 active+clean; 10 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 795 MiB/s wr, 1.52k op/s 2024-05-20T16:25:35.731 INFO:tasks.workunit.client.0.smithi082.stdout:OK 2024-05-20T16:25:35.732 INFO:tasks.workunit.client.0.smithi082.stderr:+ echo OK 2024-05-20T16:25:37.312 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:25:36 smithi160 ceph-mon[36347]: pgmap v713: 129 pgs: 129 active+clean; 10 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1011 KiB/s rd, 705 MiB/s wr, 1.24k op/s 2024-05-20T16:25:37.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:25:36 smithi082 ceph-mon[32238]: pgmap v713: 129 pgs: 129 active+clean; 10 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1011 KiB/s rd, 705 MiB/s wr, 1.24k op/s 2024-05-20T16:25:37.385 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:25:36 smithi184 ceph-mon[36209]: pgmap v713: 129 pgs: 129 active+clean; 10 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1011 KiB/s rd, 705 MiB/s wr, 1.24k op/s 2024-05-20T16:25:38.043 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-05-20T16:25:38.044 DEBUG:teuthology.orchestra.run.smithi082:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-05-20T16:25:38.108 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:25:37 smithi082 ceph-mon[32238]: pgmap v714: 129 pgs: 129 active+clean; 10 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 676 MiB/s wr, 1.15k op/s 2024-05-20T16:25:38.311 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:25:37 smithi160 ceph-mon[36347]: pgmap v714: 129 pgs: 129 active+clean; 10 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 676 MiB/s wr, 1.15k op/s 2024-05-20T16:25:38.385 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:25:37 smithi184 ceph-mon[36209]: pgmap v714: 129 pgs: 129 active+clean; 10 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 676 MiB/s wr, 1.15k op/s 2024-05-20T16:25:38.575 INFO:tasks.workunit:Stopping ['suites/iogen.sh'] on client.0... 2024-05-20T16:25:38.575 DEBUG:teuthology.orchestra.run.smithi082:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-05-20T16:25:38.976 DEBUG:teuthology.parallel:result is None 2024-05-20T16:25:38.977 DEBUG:teuthology.orchestra.run.smithi082:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-05-20T16:25:39.008 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-05-20T16:25:39.009 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2024-05-20T16:25:39.018 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi082.front.sepia.ceph.com 2024-05-20T16:25:39.057 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-05-20T16:25:39.066 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-20T16:25:39.210 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:39.208+0000 7f86effcc640 1 -- 172.21.15.82:0/1888284226 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f86e8102ab0 msgr2=0x7f86e81037b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:25:39.210 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:39.208+0000 7f86effcc640 1 --2- 172.21.15.82:0/1888284226 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f86e8102ab0 0x7f86e81037b0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f86e4009900 tx=0x7f86e402a410 comp rx=0 tx=0).stop 2024-05-20T16:25:39.210 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:39.208+0000 7f86effcc640 1 -- 172.21.15.82:0/1888284226 shutdown_connections 2024-05-20T16:25:39.211 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:39.208+0000 7f86effcc640 1 --2- 172.21.15.82:0/1888284226 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f86e81089f0 0x7f86e810ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:25:39.211 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:39.208+0000 7f86effcc640 1 --2- 172.21.15.82:0/1888284226 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f86e8103cf0 0x7f86e8104100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:25:39.211 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:39.208+0000 7f86effcc640 1 --2- 172.21.15.82:0/1888284226 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f86e8102ab0 0x7f86e81037b0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:25:39.211 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:39.208+0000 7f86effcc640 1 -- 172.21.15.82:0/1888284226 >> 172.21.15.82:0/1888284226 conn(0x7f86e80fdcc0 msgr2=0x7f86e8100150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:25:39.211 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:39.209+0000 7f86effcc640 1 -- 172.21.15.82:0/1888284226 shutdown_connections 2024-05-20T16:25:39.211 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:39.209+0000 7f86effcc640 1 -- 172.21.15.82:0/1888284226 wait complete. 2024-05-20T16:25:39.211 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:39.209+0000 7f86effcc640 1 Processor -- start 2024-05-20T16:25:39.211 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:39.209+0000 7f86effcc640 1 -- start start 2024-05-20T16:25:39.211 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:39.209+0000 7f86effcc640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f86e8102ab0 0x7f86e81992e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:25:39.211 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:39.209+0000 7f86effcc640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f86e8103cf0 0x7f86e8199820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:25:39.211 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:39.209+0000 7f86effcc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f86e81089f0 0x7f86e81a07b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:25:39.211 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:39.209+0000 7f86effcc640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f86e810e750 con 0x7f86e81089f0 2024-05-20T16:25:39.211 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:39.209+0000 7f86effcc640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f86e810e5d0 con 0x7f86e8103cf0 2024-05-20T16:25:39.212 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:39.209+0000 7f86effcc640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f86e810e8d0 con 0x7f86e8102ab0 2024-05-20T16:25:39.212 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:39.210+0000 7f86ee569640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f86e81089f0 0x7f86e81a07b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:25:39.212 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:39.210+0000 7f86ee569640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f86e81089f0 0x7f86e81a07b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44120/0 (socket says 172.21.15.82:44120) 2024-05-20T16:25:39.212 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:39.210+0000 7f86ee569640 1 -- 172.21.15.82:0/2490350355 learned_addr learned my addr 172.21.15.82:0/2490350355 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:25:39.212 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:39.210+0000 7f86edd68640 1 --2- 172.21.15.82:0/2490350355 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f86e8102ab0 0x7f86e81992e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:25:39.212 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:39.210+0000 7f86ed567640 1 --2- 172.21.15.82:0/2490350355 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f86e8103cf0 0x7f86e8199820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:25:39.212 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:39.210+0000 7f86ee569640 1 -- 172.21.15.82:0/2490350355 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f86e8102ab0 msgr2=0x7f86e81992e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:25:39.212 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:39.210+0000 7f86ee569640 1 --2- 172.21.15.82:0/2490350355 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f86e8102ab0 0x7f86e81992e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:25:39.212 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:39.210+0000 7f86ee569640 1 -- 172.21.15.82:0/2490350355 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f86e8103cf0 msgr2=0x7f86e8199820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:25:39.212 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:39.210+0000 7f86ee569640 1 --2- 172.21.15.82:0/2490350355 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f86e8103cf0 0x7f86e8199820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:25:39.212 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:39.210+0000 7f86ee569640 1 -- 172.21.15.82:0/2490350355 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86e81a0e20 con 0x7f86e81089f0 2024-05-20T16:25:39.212 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:39.210+0000 7f86ee569640 1 --2- 172.21.15.82:0/2490350355 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f86e81089f0 0x7f86e81a07b0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f86e000eb10 tx=0x7f86e000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:25:39.213 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:39.210+0000 7f86deffd640 1 -- 172.21.15.82:0/2490350355 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86e0004b20 con 0x7f86e81089f0 2024-05-20T16:25:39.213 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:39.210+0000 7f86deffd640 1 -- 172.21.15.82:0/2490350355 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f86e0004cc0 con 0x7f86e81089f0 2024-05-20T16:25:39.213 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:39.210+0000 7f86effcc640 1 -- 172.21.15.82:0/2490350355 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86e81a1110 con 0x7f86e81089f0 2024-05-20T16:25:39.213 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:39.210+0000 7f86deffd640 1 -- 172.21.15.82:0/2490350355 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86e0010760 con 0x7f86e81089f0 2024-05-20T16:25:39.213 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:39.210+0000 7f86effcc640 1 -- 172.21.15.82:0/2490350355 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86e81a1660 con 0x7f86e81089f0 2024-05-20T16:25:39.214 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:39.211+0000 7f86effcc640 1 -- 172.21.15.82:0/2490350355 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86e8107800 con 0x7f86e81089f0 2024-05-20T16:25:39.214 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:39.212+0000 7f86deffd640 1 -- 172.21.15.82:0/2490350355 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f86e0005000 con 0x7f86e81089f0 2024-05-20T16:25:39.214 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:39.213+0000 7f86deffd640 1 --2- 172.21.15.82:0/2490350355 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f86d40a2c10 0x7f86d40a5120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:25:39.214 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:39.213+0000 7f86edd68640 1 --2- 172.21.15.82:0/2490350355 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f86d40a2c10 0x7f86d40a5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:25:39.214 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:39.213+0000 7f86deffd640 1 -- 172.21.15.82:0/2490350355 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9150+0+0 (secure 0 0 0) 0x7f86e00cfba0 con 0x7f86e81089f0 2024-05-20T16:25:39.214 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:39.214+0000 7f86edd68640 1 --2- 172.21.15.82:0/2490350355 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f86d40a2c10 0x7f86d40a5120 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f86e402aa00 tx=0x7f86e4005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:25:39.216 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:39.217+0000 7f86deffd640 1 -- 172.21.15.82:0/2490350355 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f86e0053850 con 0x7f86e81089f0 2024-05-20T16:25:39.399 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:39.400+0000 7f86effcc640 1 -- 172.21.15.82:0/2490350355 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f86e802d470 con 0x7f86e81089f0 2024-05-20T16:25:39.401 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:39.401+0000 7f86deffd640 1 -- 172.21.15.82:0/2490350355 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v98) ==== 74+0+24771 (secure 0 0 0) 0x7f86e009c770 con 0x7f86e81089f0 2024-05-20T16:25:39.401 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-05-20T16:25:39.401 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":98,"fsid":"59d1600a-16c2-11ef-bc9a-c7b262605968","created":"2024-05-20T16:03:46.611556+0000","modified":"2024-05-20T16:25:03.536196+0000","last_up_change":"2024-05-20T16:10:59.303479+0000","last_in_change":"2024-05-20T16:10:39.063561+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-20T16:07:09.493727+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-20T16:11:45.058035+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-20T16:11:45.943862+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-20T16:11:49.637579+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"bae842a9-03ec-490e-b56d-f7957235cdc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":1484330167},{"type":"v1","addr":"172.21.15.82:6803","nonce":1484330167}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":1484330167},{"type":"v1","addr":"172.21.15.82:6805","nonce":1484330167}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1484330167},{"type":"v1","addr":"172.21.15.82:6809","nonce":1484330167}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":1484330167},{"type":"v1","addr":"172.21.15.82:6807","nonce":1484330167}]},"public_addr":"172.21.15.82:6803/1484330167","cluster_addr":"172.21.15.82:6805/1484330167","heartbeat_back_addr":"172.21.15.82:6809/1484330167","heartbeat_front_addr":"172.21.15.82:6807/1484330167","state":["exists","up"]},{"osd":1,"uuid":"59f6a2ba-b5de-4161-81c5-92a417abacdf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":1457003568},{"type":"v1","addr":"172.21.15.160:6801","nonce":1457003568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":1457003568},{"type":"v1","addr":"172.21.15.160:6803","nonce":1457003568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6806","nonce":1457003568},{"type":"v1","addr":"172.21.15.160:6807","nonce":1457003568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":1457003568},{"type":"v1","addr":"172.21.15.160:6805","nonce":1457003568}]},"public_addr":"172.21.15.160:6801/1457003568","cluster_addr":"172.21.15.160:6803/1457003568","heartbeat_back_addr":"172.21.15.160:6807/1457003568","heartbeat_front_addr":"172.21.15.160:6805/1457003568","state":["exists","up"]},{"osd":2,"uuid":"3bbb5587-b449-41f3-bd26-67ec3d6afdfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":4051293934},{"type":"v1","addr":"172.21.15.184:6801","nonce":4051293934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":4051293934},{"type":"v1","addr":"172.21.15.184:6803","nonce":4051293934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":4051293934},{"type":"v1","addr":"172.21.15.184:6807","nonce":4051293934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":4051293934},{"type":"v1","addr":"172.21.15.184:6805","nonce":4051293934}]},"public_addr":"172.21.15.184:6801/4051293934","cluster_addr":"172.21.15.184:6803/4051293934","heartbeat_back_addr":"172.21.15.184:6807/4051293934","heartbeat_front_addr":"172.21.15.184:6805/4051293934","state":["exists","up"]},{"osd":3,"uuid":"ca651f90-09b8-4081-a6f9-40b31268a837","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":3947485599},{"type":"v1","addr":"172.21.15.82:6811","nonce":3947485599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":3947485599},{"type":"v1","addr":"172.21.15.82:6813","nonce":3947485599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":3947485599},{"type":"v1","addr":"172.21.15.82:6817","nonce":3947485599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":3947485599},{"type":"v1","addr":"172.21.15.82:6815","nonce":3947485599}]},"public_addr":"172.21.15.82:6811/3947485599","cluster_addr":"172.21.15.82:6813/3947485599","heartbeat_back_addr":"172.21.15.82:6817/3947485599","heartbeat_front_addr":"172.21.15.82:6815/3947485599","state":["exists","up"]},{"osd":4,"uuid":"cb3dfd56-f677-40cb-9c96-8319ea5ac788","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":4160830014},{"type":"v1","addr":"172.21.15.160:6809","nonce":4160830014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":4160830014},{"type":"v1","addr":"172.21.15.160:6811","nonce":4160830014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":4160830014},{"type":"v1","addr":"172.21.15.160:6815","nonce":4160830014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6812","nonce":4160830014},{"type":"v1","addr":"172.21.15.160:6813","nonce":4160830014}]},"public_addr":"172.21.15.160:6809/4160830014","cluster_addr":"172.21.15.160:6811/4160830014","heartbeat_back_addr":"172.21.15.160:6815/4160830014","heartbeat_front_addr":"172.21.15.160:6813/4160830014","state":["exists","up"]},{"osd":5,"uuid":"c70cfcd1-a4ff-461f-8c73-479b061200fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":1046282672},{"type":"v1","addr":"172.21.15.184:6809","nonce":1046282672}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":1046282672},{"type":"v1","addr":"172.21.15.184:6811","nonce":1046282672}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":1046282672},{"type":"v1","addr":"172.21.15.184:6815","nonce":1046282672}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":1046282672},{"type":"v1","addr":"172.21.15.184:6813","nonce":1046282672}]},"public_addr":"172.21.15.184:6809/1046282672","cluster_addr":"172.21.15.184:6811/1046282672","heartbeat_back_addr":"172.21.15.184:6815/1046282672","heartbeat_front_addr":"172.21.15.184:6813/1046282672","state":["exists","up"]},{"osd":6,"uuid":"d47ce001-9dd1-4830-bcc9-888601187c6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":2789736499},{"type":"v1","addr":"172.21.15.82:6819","nonce":2789736499}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":2789736499},{"type":"v1","addr":"172.21.15.82:6821","nonce":2789736499}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":2789736499},{"type":"v1","addr":"172.21.15.82:6825","nonce":2789736499}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":2789736499},{"type":"v1","addr":"172.21.15.82:6823","nonce":2789736499}]},"public_addr":"172.21.15.82:6819/2789736499","cluster_addr":"172.21.15.82:6821/2789736499","heartbeat_back_addr":"172.21.15.82:6825/2789736499","heartbeat_front_addr":"172.21.15.82:6823/2789736499","state":["exists","up"]},{"osd":7,"uuid":"25c54f43-18cb-41e4-83b4-2732b5a93889","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6816","nonce":2340683378},{"type":"v1","addr":"172.21.15.160:6817","nonce":2340683378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":2340683378},{"type":"v1","addr":"172.21.15.160:6819","nonce":2340683378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":2340683378},{"type":"v1","addr":"172.21.15.160:6823","nonce":2340683378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6820","nonce":2340683378},{"type":"v1","addr":"172.21.15.160:6821","nonce":2340683378}]},"public_addr":"172.21.15.160:6817/2340683378","cluster_addr":"172.21.15.160:6819/2340683378","heartbeat_back_addr":"172.21.15.160:6823/2340683378","heartbeat_front_addr":"172.21.15.160:6821/2340683378","state":["exists","up"]},{"osd":8,"uuid":"1f8b478d-287f-45f5-8526-d1a58c5bc07c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":3831576372},{"type":"v1","addr":"172.21.15.184:6817","nonce":3831576372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":3831576372},{"type":"v1","addr":"172.21.15.184:6819","nonce":3831576372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":3831576372},{"type":"v1","addr":"172.21.15.184:6823","nonce":3831576372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":3831576372},{"type":"v1","addr":"172.21.15.184:6821","nonce":3831576372}]},"public_addr":"172.21.15.184:6817/3831576372","cluster_addr":"172.21.15.184:6819/3831576372","heartbeat_back_addr":"172.21.15.184:6823/3831576372","heartbeat_front_addr":"172.21.15.184:6821/3831576372","state":["exists","up"]},{"osd":9,"uuid":"0d69e319-954f-4b18-95ea-31794f18dc3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":716330782},{"type":"v1","addr":"172.21.15.82:6827","nonce":716330782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":716330782},{"type":"v1","addr":"172.21.15.82:6829","nonce":716330782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":716330782},{"type":"v1","addr":"172.21.15.82:6833","nonce":716330782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":716330782},{"type":"v1","addr":"172.21.15.82:6831","nonce":716330782}]},"public_addr":"172.21.15.82:6827/716330782","cluster_addr":"172.21.15.82:6829/716330782","heartbeat_back_addr":"172.21.15.82:6833/716330782","heartbeat_front_addr":"172.21.15.82:6831/716330782","state":["exists","up"]},{"osd":10,"uuid":"39dc1b78-61cb-42ca-b5b6-1e44ac2b6500","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":2327428014},{"type":"v1","addr":"172.21.15.160:6825","nonce":2327428014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":2327428014},{"type":"v1","addr":"172.21.15.160:6827","nonce":2327428014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":2327428014},{"type":"v1","addr":"172.21.15.160:6831","nonce":2327428014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":2327428014},{"type":"v1","addr":"172.21.15.160:6829","nonce":2327428014}]},"public_addr":"172.21.15.160:6825/2327428014","cluster_addr":"172.21.15.160:6827/2327428014","heartbeat_back_addr":"172.21.15.160:6831/2327428014","heartbeat_front_addr":"172.21.15.160:6829/2327428014","state":["exists","up"]},{"osd":11,"uuid":"4306b3c1-707a-4010-987c-209ea0e948e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":3594364650},{"type":"v1","addr":"172.21.15.184:6825","nonce":3594364650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":3594364650},{"type":"v1","addr":"172.21.15.184:6827","nonce":3594364650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3594364650},{"type":"v1","addr":"172.21.15.184:6831","nonce":3594364650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3594364650},{"type":"v1","addr":"172.21.15.184:6829","nonce":3594364650}]},"public_addr":"172.21.15.184:6825/3594364650","cluster_addr":"172.21.15.184:6827/3594364650","heartbeat_back_addr":"172.21.15.184:6831/3594364650","heartbeat_front_addr":"172.21.15.184:6829/3594364650","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:06:17.855998+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:06:41.473270+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:07:05.304108+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:07:29.887472+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:07:55.387286+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:08:20.579669+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:08:46.333051+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:09:12.289668+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:09:38.180696+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:10:04.471459+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:10:31.041912+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:10:57.477266+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.1d","mappings":[{"from":5,"to":9}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.5","mappings":[{"from":9,"to":10}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.82:0/2005424509":"2024-05-21T16:04:45.161949+0000","172.21.15.82:6801/1763049308":"2024-05-21T16:04:45.161949+0000","172.21.15.82:6801/1959459348":"2024-05-21T16:04:19.208291+0000","172.21.15.82:0/3131904556":"2024-05-21T16:04:19.208291+0000","172.21.15.82:6800/1959459348":"2024-05-21T16:04:19.208291+0000","172.21.15.82:0/2456966342":"2024-05-21T16:04:45.161949+0000","172.21.15.82:0/4179978833":"2024-05-21T16:04:19.208291+0000","172.21.15.82:0/1602862225":"2024-05-21T16:04:45.161949+0000","172.21.15.82:0/106101180":"2024-05-21T16:04:19.208291+0000","172.21.15.82:6800/1763049308":"2024-05-21T16:04:45.161949+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-20T16:25:39.403 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:39.403+0000 7f86effcc640 1 -- 172.21.15.82:0/2490350355 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f86d40a2c10 msgr2=0x7f86d40a5120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:25:39.403 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:39.403+0000 7f86effcc640 1 --2- 172.21.15.82:0/2490350355 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f86d40a2c10 0x7f86d40a5120 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f86e402aa00 tx=0x7f86e4005a20 comp rx=0 tx=0).stop 2024-05-20T16:25:39.403 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:39.403+0000 7f86effcc640 1 -- 172.21.15.82:0/2490350355 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f86e81089f0 msgr2=0x7f86e81a07b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:25:39.403 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:39.403+0000 7f86effcc640 1 --2- 172.21.15.82:0/2490350355 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f86e81089f0 0x7f86e81a07b0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f86e000eb10 tx=0x7f86e000c2d0 comp rx=0 tx=0).stop 2024-05-20T16:25:39.403 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:39.403+0000 7f86effcc640 1 -- 172.21.15.82:0/2490350355 shutdown_connections 2024-05-20T16:25:39.403 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:39.403+0000 7f86effcc640 1 --2- 172.21.15.82:0/2490350355 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f86d40a2c10 0x7f86d40a5120 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:25:39.403 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:39.403+0000 7f86effcc640 1 --2- 172.21.15.82:0/2490350355 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f86e81089f0 0x7f86e81a07b0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:25:39.404 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:39.403+0000 7f86effcc640 1 --2- 172.21.15.82:0/2490350355 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f86e8103cf0 0x7f86e8199820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:25:39.404 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:39.403+0000 7f86effcc640 1 --2- 172.21.15.82:0/2490350355 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f86e8102ab0 0x7f86e81992e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:25:39.404 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:39.403+0000 7f86effcc640 1 -- 172.21.15.82:0/2490350355 >> 172.21.15.82:0/2490350355 conn(0x7f86e80fdcc0 msgr2=0x7f86e81049d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:25:39.404 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:39.403+0000 7f86effcc640 1 -- 172.21.15.82:0/2490350355 shutdown_connections 2024-05-20T16:25:39.404 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:39.403+0000 7f86effcc640 1 -- 172.21.15.82:0/2490350355 wait complete. 2024-05-20T16:25:39.415 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-20T16:07:09.493727+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-20T16:11:45.058035+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-20T16:11:45.943862+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-20T16:11:49.637579+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '91', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-20T16:25:39.416 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-20T16:25:39.532 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:39.533+0000 7fa02087a640 1 -- 172.21.15.82:0/1516188339 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa01c1019b0 msgr2=0x7fa01c1026b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:25:39.533 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:39.533+0000 7fa02087a640 1 --2- 172.21.15.82:0/1516188339 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa01c1019b0 0x7fa01c1026b0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fa00c009ca0 tx=0x7fa00c02a8c0 comp rx=0 tx=0).stop 2024-05-20T16:25:39.533 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:39.533+0000 7fa02087a640 1 -- 172.21.15.82:0/1516188339 shutdown_connections 2024-05-20T16:25:39.533 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:39.533+0000 7fa02087a640 1 --2- 172.21.15.82:0/1516188339 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa01c1078f0 0x7fa01c109d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:25:39.533 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:39.533+0000 7fa02087a640 1 --2- 172.21.15.82:0/1516188339 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa01c102bf0 0x7fa01c103000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:25:39.533 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:39.533+0000 7fa02087a640 1 --2- 172.21.15.82:0/1516188339 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa01c1019b0 0x7fa01c1026b0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:25:39.533 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:39.533+0000 7fa02087a640 1 -- 172.21.15.82:0/1516188339 >> 172.21.15.82:0/1516188339 conn(0x7fa01c0fcba0 msgr2=0x7fa01c0ff050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:25:39.533 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:39.533+0000 7fa02087a640 1 -- 172.21.15.82:0/1516188339 shutdown_connections 2024-05-20T16:25:39.533 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:39.533+0000 7fa02087a640 1 -- 172.21.15.82:0/1516188339 wait complete. 2024-05-20T16:25:39.535 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:39.533+0000 7fa02087a640 1 Processor -- start 2024-05-20T16:25:39.535 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:39.533+0000 7fa02087a640 1 -- start start 2024-05-20T16:25:39.536 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:39.534+0000 7fa02087a640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa01c1019b0 0x7fa01c198260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:25:39.536 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:39.534+0000 7fa02087a640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa01c102bf0 0x7fa01c1987a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:25:39.536 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:39.534+0000 7fa02087a640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa01c1078f0 0x7fa01c19f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:25:39.536 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:39.534+0000 7fa02087a640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fa01c10d600 con 0x7fa01c1019b0 2024-05-20T16:25:39.536 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:39.534+0000 7fa02087a640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fa01c10d480 con 0x7fa01c1078f0 2024-05-20T16:25:39.536 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:39.534+0000 7fa02087a640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa01c10d780 con 0x7fa01c102bf0 2024-05-20T16:25:39.536 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:39.534+0000 7fa01a59c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa01c1019b0 0x7fa01c198260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:25:39.536 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:39.534+0000 7fa01a59c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa01c1019b0 0x7fa01c198260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:59382/0 (socket says 172.21.15.82:59382) 2024-05-20T16:25:39.536 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:39.534+0000 7fa01a59c640 1 -- 172.21.15.82:0/2382010358 learned_addr learned my addr 172.21.15.82:0/2382010358 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:25:39.536 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:39.534+0000 7fa01ad9d640 1 --2- 172.21.15.82:0/2382010358 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa01c1078f0 0x7fa01c19f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:25:39.536 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:39.534+0000 7fa019d9b640 1 --2- 172.21.15.82:0/2382010358 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa01c102bf0 0x7fa01c1987a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:25:39.536 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:39.534+0000 7fa01a59c640 1 -- 172.21.15.82:0/2382010358 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa01c102bf0 msgr2=0x7fa01c1987a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:25:39.536 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:39.534+0000 7fa01a59c640 1 --2- 172.21.15.82:0/2382010358 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa01c102bf0 0x7fa01c1987a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:25:39.537 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:39.534+0000 7fa01a59c640 1 -- 172.21.15.82:0/2382010358 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa01c1078f0 msgr2=0x7fa01c19f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:25:39.537 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:39.534+0000 7fa01a59c640 1 --2- 172.21.15.82:0/2382010358 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa01c1078f0 0x7fa01c19f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:25:39.537 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:39.534+0000 7fa01a59c640 1 -- 172.21.15.82:0/2382010358 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa01c19fda0 con 0x7fa01c1019b0 2024-05-20T16:25:39.537 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:39.534+0000 7fa01a59c640 1 --2- 172.21.15.82:0/2382010358 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa01c1019b0 0x7fa01c198260 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fa00c000f80 tx=0x7fa00c031ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:25:39.537 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:39.534+0000 7fa0137fe640 1 -- 172.21.15.82:0/2382010358 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa00c04a070 con 0x7fa01c1019b0 2024-05-20T16:25:39.537 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:39.534+0000 7fa0137fe640 1 -- 172.21.15.82:0/2382010358 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa00c02faf0 con 0x7fa01c1019b0 2024-05-20T16:25:39.537 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:39.534+0000 7fa02087a640 1 -- 172.21.15.82:0/2382010358 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa01c073d80 con 0x7fa01c1019b0 2024-05-20T16:25:39.537 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:39.535+0000 7fa0137fe640 1 -- 172.21.15.82:0/2382010358 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa00c030dc0 con 0x7fa01c1019b0 2024-05-20T16:25:39.537 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:39.535+0000 7fa02087a640 1 -- 172.21.15.82:0/2382010358 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa01c0742d0 con 0x7fa01c1019b0 2024-05-20T16:25:39.537 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:39.536+0000 7fa02087a640 1 -- 172.21.15.82:0/2382010358 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa01c069110 con 0x7fa01c1019b0 2024-05-20T16:25:39.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:39.537+0000 7fa0137fe640 1 -- 172.21.15.82:0/2382010358 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fa00c04d070 con 0x7fa01c1019b0 2024-05-20T16:25:39.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:39.537+0000 7fa0137fe640 1 --2- 172.21.15.82:0/2382010358 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fa0080a2b90 0x7fa0080a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:25:39.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:39.538+0000 7fa0137fe640 1 -- 172.21.15.82:0/2382010358 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9150+0+0 (secure 0 0 0) 0x7fa00c0f1d00 con 0x7fa01c1019b0 2024-05-20T16:25:39.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:39.538+0000 7fa019d9b640 1 --2- 172.21.15.82:0/2382010358 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fa0080a2b90 0x7fa0080a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:25:39.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:39.538+0000 7fa019d9b640 1 --2- 172.21.15.82:0/2382010358 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fa0080a2b90 0x7fa0080a50a0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fa004005fd0 tx=0x7fa004005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:25:39.540 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:39.541+0000 7fa0137fe640 1 -- 172.21.15.82:0/2382010358 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fa00c054080 con 0x7fa01c1019b0 2024-05-20T16:25:39.723 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:39.724+0000 7fa02087a640 1 -- 172.21.15.82:0/2382010358 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fa01c106300 con 0x7fa01c1019b0 2024-05-20T16:25:39.724 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:39.724+0000 7fa0137fe640 1 -- 172.21.15.82:0/2382010358 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v98) ==== 93+0+10 (secure 0 0 0) 0x7fa00c0be8d0 con 0x7fa01c1019b0 2024-05-20T16:25:39.724 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 1 2024-05-20T16:25:39.726 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:39.726+0000 7fa02087a640 1 -- 172.21.15.82:0/2382010358 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fa0080a2b90 msgr2=0x7fa0080a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:25:39.726 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:39.726+0000 7fa02087a640 1 --2- 172.21.15.82:0/2382010358 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fa0080a2b90 0x7fa0080a50a0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fa004005fd0 tx=0x7fa004005850 comp rx=0 tx=0).stop 2024-05-20T16:25:39.726 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:39.726+0000 7fa02087a640 1 -- 172.21.15.82:0/2382010358 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa01c1019b0 msgr2=0x7fa01c198260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:25:39.726 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:39.726+0000 7fa02087a640 1 --2- 172.21.15.82:0/2382010358 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa01c1019b0 0x7fa01c198260 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fa00c000f80 tx=0x7fa00c031ec0 comp rx=0 tx=0).stop 2024-05-20T16:25:39.726 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:39.726+0000 7fa02087a640 1 -- 172.21.15.82:0/2382010358 shutdown_connections 2024-05-20T16:25:39.726 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:39.726+0000 7fa02087a640 1 --2- 172.21.15.82:0/2382010358 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fa0080a2b90 0x7fa0080a50a0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:25:39.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:39.726+0000 7fa02087a640 1 --2- 172.21.15.82:0/2382010358 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa01c1078f0 0x7fa01c19f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:25:39.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:39.726+0000 7fa02087a640 1 --2- 172.21.15.82:0/2382010358 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa01c102bf0 0x7fa01c1987a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:25:39.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:39.726+0000 7fa02087a640 1 --2- 172.21.15.82:0/2382010358 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa01c1019b0 0x7fa01c198260 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:25:39.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:39.726+0000 7fa02087a640 1 -- 172.21.15.82:0/2382010358 >> 172.21.15.82:0/2382010358 conn(0x7fa01c0fcba0 msgr2=0x7fa01c1038d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:25:39.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:39.726+0000 7fa02087a640 1 -- 172.21.15.82:0/2382010358 shutdown_connections 2024-05-20T16:25:39.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:39.726+0000 7fa02087a640 1 -- 172.21.15.82:0/2382010358 wait complete. 2024-05-20T16:25:39.737 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:25:39 smithi082 ceph-mon[32238]: from='client.? 172.21.15.82:0/2490350355' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-20T16:25:39.738 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-20T16:25:39.811 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:25:39 smithi160 ceph-mon[36347]: from='client.? 172.21.15.82:0/2490350355' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-20T16:25:39.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:39.853+0000 7fa407673640 1 -- 172.21.15.82:0/1554718248 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa4001078f0 msgr2=0x7fa400109d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:25:39.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:39.853+0000 7fa407673640 1 --2- 172.21.15.82:0/1554718248 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa4001078f0 0x7fa400109d10 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fa3f8009900 tx=0x7fa3f802a410 comp rx=0 tx=0).stop 2024-05-20T16:25:39.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:39.853+0000 7fa407673640 1 -- 172.21.15.82:0/1554718248 shutdown_connections 2024-05-20T16:25:39.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:39.853+0000 7fa407673640 1 --2- 172.21.15.82:0/1554718248 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa4001078f0 0x7fa400109d10 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:25:39.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:39.853+0000 7fa407673640 1 --2- 172.21.15.82:0/1554718248 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa400102bf0 0x7fa400103000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:25:39.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:39.853+0000 7fa407673640 1 --2- 172.21.15.82:0/1554718248 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa4001019b0 0x7fa4001026b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:25:39.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:39.853+0000 7fa407673640 1 -- 172.21.15.82:0/1554718248 >> 172.21.15.82:0/1554718248 conn(0x7fa4000fcba0 msgr2=0x7fa4000ff050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:25:39.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:39.853+0000 7fa407673640 1 -- 172.21.15.82:0/1554718248 shutdown_connections 2024-05-20T16:25:39.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:39.853+0000 7fa407673640 1 -- 172.21.15.82:0/1554718248 wait complete. 2024-05-20T16:25:39.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:39.853+0000 7fa407673640 1 Processor -- start 2024-05-20T16:25:39.854 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:39.854+0000 7fa407673640 1 -- start start 2024-05-20T16:25:39.856 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:39.854+0000 7fa407673640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa4001019b0 0x7fa400199230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:25:39.856 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:39.854+0000 7fa407673640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa400102bf0 0x7fa400199770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:25:39.856 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:39.854+0000 7fa407673640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa4001078f0 0x7fa4001a0700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:25:39.856 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:39.854+0000 7fa407673640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fa40010d610 con 0x7fa4001019b0 2024-05-20T16:25:39.856 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:39.854+0000 7fa407673640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fa40010d490 con 0x7fa4001078f0 2024-05-20T16:25:39.856 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:39.854+0000 7fa407673640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa40010d790 con 0x7fa400102bf0 2024-05-20T16:25:39.856 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:39.854+0000 7fa40540f640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa4001019b0 0x7fa400199230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:25:39.856 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:39.854+0000 7fa40540f640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa4001019b0 0x7fa400199230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:59402/0 (socket says 172.21.15.82:59402) 2024-05-20T16:25:39.856 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:39.854+0000 7fa40540f640 1 -- 172.21.15.82:0/2558355108 learned_addr learned my addr 172.21.15.82:0/2558355108 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:25:39.856 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:39.854+0000 7fa405c10640 1 --2- 172.21.15.82:0/2558355108 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa4001078f0 0x7fa4001a0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:25:39.856 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:39.854+0000 7fa404c0e640 1 --2- 172.21.15.82:0/2558355108 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa400102bf0 0x7fa400199770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:25:39.856 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:39.854+0000 7fa40540f640 1 -- 172.21.15.82:0/2558355108 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa400102bf0 msgr2=0x7fa400199770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:25:39.856 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:39.854+0000 7fa40540f640 1 --2- 172.21.15.82:0/2558355108 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa400102bf0 0x7fa400199770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:25:39.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:39.854+0000 7fa40540f640 1 -- 172.21.15.82:0/2558355108 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa4001078f0 msgr2=0x7fa4001a0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:25:39.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:39.854+0000 7fa40540f640 1 --2- 172.21.15.82:0/2558355108 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa4001078f0 0x7fa4001a0700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:25:39.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:39.854+0000 7fa40540f640 1 -- 172.21.15.82:0/2558355108 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4001a0d70 con 0x7fa4001019b0 2024-05-20T16:25:39.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:39.854+0000 7fa404c0e640 1 --2- 172.21.15.82:0/2558355108 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa400102bf0 0x7fa400199770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T16:25:39.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:39.854+0000 7fa40540f640 1 --2- 172.21.15.82:0/2558355108 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa4001019b0 0x7fa400199230 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fa3f000ead0 tx=0x7fa3f000ef90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:25:39.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:39.855+0000 7fa3fe7fc640 1 -- 172.21.15.82:0/2558355108 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3f000ce40 con 0x7fa4001019b0 2024-05-20T16:25:39.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:39.855+0000 7fa3fe7fc640 1 -- 172.21.15.82:0/2558355108 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa3f0011950 con 0x7fa4001019b0 2024-05-20T16:25:39.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:39.855+0000 7fa407673640 1 -- 172.21.15.82:0/2558355108 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4001a1060 con 0x7fa4001019b0 2024-05-20T16:25:39.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:39.855+0000 7fa3fe7fc640 1 -- 172.21.15.82:0/2558355108 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3f001ac10 con 0x7fa4001019b0 2024-05-20T16:25:39.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:39.855+0000 7fa407673640 1 -- 172.21.15.82:0/2558355108 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4001a15b0 con 0x7fa4001019b0 2024-05-20T16:25:39.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:39.856+0000 7fa407673640 1 -- 172.21.15.82:0/2558355108 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa400106ff0 con 0x7fa4001019b0 2024-05-20T16:25:39.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:39.857+0000 7fa3fe7fc640 1 -- 172.21.15.82:0/2558355108 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fa3f0011b00 con 0x7fa4001019b0 2024-05-20T16:25:39.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:39.858+0000 7fa3fe7fc640 1 --2- 172.21.15.82:0/2558355108 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fa3e80a2b90 0x7fa3e80a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:25:39.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:39.858+0000 7fa404c0e640 1 --2- 172.21.15.82:0/2558355108 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fa3e80a2b90 0x7fa3e80a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:25:39.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:39.858+0000 7fa3fe7fc640 1 -- 172.21.15.82:0/2558355108 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9150+0+0 (secure 0 0 0) 0x7fa3f00cf930 con 0x7fa4001019b0 2024-05-20T16:25:39.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:39.858+0000 7fa404c0e640 1 --2- 172.21.15.82:0/2558355108 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fa3e80a2b90 0x7fa3e80a50a0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fa3f4005d90 tx=0x7fa3f4005d20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:25:39.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:39.863+0000 7fa3fe7fc640 1 -- 172.21.15.82:0/2558355108 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fa3f0016030 con 0x7fa4001019b0 2024-05-20T16:25:39.885 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:25:39 smithi184 ceph-mon[36209]: from='client.? 172.21.15.82:0/2490350355' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-20T16:25:40.041 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:40.041+0000 7fa407673640 1 -- 172.21.15.82:0/2558355108 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fa400062750 con 0x7fa4001019b0 2024-05-20T16:25:40.041 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:40.041+0000 7fa3fe7fc640 1 -- 172.21.15.82:0/2558355108 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v98) ==== 104+0+11 (secure 0 0 0) 0x7fa3f009c500 con 0x7fa4001019b0 2024-05-20T16:25:40.041 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 32 2024-05-20T16:25:40.046 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:40.043+0000 7fa407673640 1 -- 172.21.15.82:0/2558355108 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fa3e80a2b90 msgr2=0x7fa3e80a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:25:40.046 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:40.043+0000 7fa407673640 1 --2- 172.21.15.82:0/2558355108 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fa3e80a2b90 0x7fa3e80a50a0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fa3f4005d90 tx=0x7fa3f4005d20 comp rx=0 tx=0).stop 2024-05-20T16:25:40.046 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:40.043+0000 7fa407673640 1 -- 172.21.15.82:0/2558355108 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa4001019b0 msgr2=0x7fa400199230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:25:40.046 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:40.043+0000 7fa407673640 1 --2- 172.21.15.82:0/2558355108 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa4001019b0 0x7fa400199230 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fa3f000ead0 tx=0x7fa3f000ef90 comp rx=0 tx=0).stop 2024-05-20T16:25:40.046 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:40.044+0000 7fa407673640 1 -- 172.21.15.82:0/2558355108 shutdown_connections 2024-05-20T16:25:40.046 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:40.044+0000 7fa407673640 1 --2- 172.21.15.82:0/2558355108 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fa3e80a2b90 0x7fa3e80a50a0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:25:40.046 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:40.044+0000 7fa407673640 1 --2- 172.21.15.82:0/2558355108 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa4001078f0 0x7fa4001a0700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:25:40.046 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:40.044+0000 7fa407673640 1 --2- 172.21.15.82:0/2558355108 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa400102bf0 0x7fa400199770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:25:40.046 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:40.044+0000 7fa407673640 1 --2- 172.21.15.82:0/2558355108 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa4001019b0 0x7fa400199230 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:25:40.046 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:40.044+0000 7fa407673640 1 -- 172.21.15.82:0/2558355108 >> 172.21.15.82:0/2558355108 conn(0x7fa4000fcba0 msgr2=0x7fa4000fffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:25:40.047 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:40.044+0000 7fa407673640 1 -- 172.21.15.82:0/2558355108 shutdown_connections 2024-05-20T16:25:40.047 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:40.044+0000 7fa407673640 1 -- 172.21.15.82:0/2558355108 wait complete. 2024-05-20T16:25:40.055 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-20T16:25:40.170 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:40.169+0000 7f3a52b4c640 1 -- 172.21.15.82:0/4127947709 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3a4c101990 msgr2=0x7f3a4c102690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:25:40.171 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:40.169+0000 7f3a52b4c640 1 --2- 172.21.15.82:0/4127947709 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3a4c101990 0x7f3a4c102690 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f3a44009900 tx=0x7f3a4402a410 comp rx=0 tx=0).stop 2024-05-20T16:25:40.171 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:40.169+0000 7f3a52b4c640 1 -- 172.21.15.82:0/4127947709 shutdown_connections 2024-05-20T16:25:40.171 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:40.169+0000 7f3a52b4c640 1 --2- 172.21.15.82:0/4127947709 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3a4c1078d0 0x7f3a4c109cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:25:40.171 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:40.169+0000 7f3a52b4c640 1 --2- 172.21.15.82:0/4127947709 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3a4c102bd0 0x7f3a4c102fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:25:40.171 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:40.169+0000 7f3a52b4c640 1 --2- 172.21.15.82:0/4127947709 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3a4c101990 0x7f3a4c102690 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:25:40.171 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:40.169+0000 7f3a52b4c640 1 -- 172.21.15.82:0/4127947709 >> 172.21.15.82:0/4127947709 conn(0x7f3a4c0fcb80 msgr2=0x7f3a4c0ff030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:25:40.171 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:40.169+0000 7f3a52b4c640 1 -- 172.21.15.82:0/4127947709 shutdown_connections 2024-05-20T16:25:40.171 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:40.169+0000 7f3a52b4c640 1 -- 172.21.15.82:0/4127947709 wait complete. 2024-05-20T16:25:40.171 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:40.170+0000 7f3a52b4c640 1 Processor -- start 2024-05-20T16:25:40.171 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:40.170+0000 7f3a52b4c640 1 -- start start 2024-05-20T16:25:40.171 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:40.170+0000 7f3a52b4c640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3a4c101990 0x7f3a4c1984e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:25:40.171 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:40.170+0000 7f3a52b4c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3a4c102bd0 0x7f3a4c198a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:25:40.172 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:40.170+0000 7f3a52b4c640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3a4c1078d0 0x7f3a4c19f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:25:40.172 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:40.170+0000 7f3a52b4c640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f3a4c10d5c0 con 0x7f3a4c102bd0 2024-05-20T16:25:40.172 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:40.170+0000 7f3a52b4c640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f3a4c10d440 con 0x7f3a4c1078d0 2024-05-20T16:25:40.172 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:40.170+0000 7f3a52b4c640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f3a4c10d740 con 0x7f3a4c101990 2024-05-20T16:25:40.172 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:40.170+0000 7f3a4bfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3a4c102bd0 0x7f3a4c198a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:25:40.172 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:40.170+0000 7f3a4bfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3a4c102bd0 0x7f3a4c198a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:59432/0 (socket says 172.21.15.82:59432) 2024-05-20T16:25:40.172 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:40.170+0000 7f3a4bfff640 1 -- 172.21.15.82:0/3379085089 learned_addr learned my addr 172.21.15.82:0/3379085089 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:25:40.172 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:40.170+0000 7f3a510e9640 1 --2- 172.21.15.82:0/3379085089 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3a4c1078d0 0x7f3a4c19f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:25:40.172 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:40.170+0000 7f3a508e8640 1 --2- 172.21.15.82:0/3379085089 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3a4c101990 0x7f3a4c1984e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:25:40.172 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:40.170+0000 7f3a4bfff640 1 -- 172.21.15.82:0/3379085089 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3a4c101990 msgr2=0x7f3a4c1984e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:25:40.172 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:40.170+0000 7f3a4bfff640 1 --2- 172.21.15.82:0/3379085089 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3a4c101990 0x7f3a4c1984e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:25:40.172 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:40.170+0000 7f3a4bfff640 1 -- 172.21.15.82:0/3379085089 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3a4c1078d0 msgr2=0x7f3a4c19f9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:25:40.172 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:40.170+0000 7f3a4bfff640 1 --2- 172.21.15.82:0/3379085089 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3a4c1078d0 0x7f3a4c19f9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:25:40.172 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:40.170+0000 7f3a4bfff640 1 -- 172.21.15.82:0/3379085089 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a4c1a0020 con 0x7f3a4c102bd0 2024-05-20T16:25:40.172 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:40.170+0000 7f3a4bfff640 1 --2- 172.21.15.82:0/3379085089 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3a4c102bd0 0x7f3a4c198a20 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f3a3c00ebc0 tx=0x7f3a3c007590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:25:40.172 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:40.171+0000 7f3a49ffb640 1 -- 172.21.15.82:0/3379085089 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a3c01c070 con 0x7f3a4c102bd0 2024-05-20T16:25:40.173 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:40.171+0000 7f3a49ffb640 1 -- 172.21.15.82:0/3379085089 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3a3c007ca0 con 0x7f3a4c102bd0 2024-05-20T16:25:40.173 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:40.171+0000 7f3a52b4c640 1 -- 172.21.15.82:0/3379085089 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a4c068ea0 con 0x7f3a4c102bd0 2024-05-20T16:25:40.173 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:40.171+0000 7f3a49ffb640 1 -- 172.21.15.82:0/3379085089 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a3c017670 con 0x7f3a4c102bd0 2024-05-20T16:25:40.173 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:40.171+0000 7f3a52b4c640 1 -- 172.21.15.82:0/3379085089 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a4c0693f0 con 0x7f3a4c102bd0 2024-05-20T16:25:40.173 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:40.172+0000 7f3a52b4c640 1 -- 172.21.15.82:0/3379085089 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a4c06b370 con 0x7f3a4c102bd0 2024-05-20T16:25:40.174 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:40.173+0000 7f3a49ffb640 1 -- 172.21.15.82:0/3379085089 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f3a3c01f070 con 0x7f3a4c102bd0 2024-05-20T16:25:40.174 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:40.174+0000 7f3a49ffb640 1 --2- 172.21.15.82:0/3379085089 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f3a380a2b90 0x7f3a380a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:25:40.174 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:40.174+0000 7f3a508e8640 1 --2- 172.21.15.82:0/3379085089 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f3a380a2b90 0x7f3a380a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:25:40.174 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:40.174+0000 7f3a49ffb640 1 -- 172.21.15.82:0/3379085089 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9150+0+0 (secure 0 0 0) 0x7f3a3c0d0690 con 0x7f3a4c102bd0 2024-05-20T16:25:40.174 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:40.175+0000 7f3a508e8640 1 --2- 172.21.15.82:0/3379085089 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f3a380a2b90 0x7f3a380a50a0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f3a44005ab0 tx=0x7f3a44031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:25:40.177 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:40.178+0000 7f3a49ffb640 1 -- 172.21.15.82:0/3379085089 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f3a3c00ce20 con 0x7f3a4c102bd0 2024-05-20T16:25:40.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:40.363+0000 7f3a52b4c640 1 -- 172.21.15.82:0/3379085089 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f3a4c1063f0 con 0x7f3a4c102bd0 2024-05-20T16:25:40.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:40.364+0000 7f3a49ffb640 1 -- 172.21.15.82:0/3379085089 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v98) ==== 100+0+11 (secure 0 0 0) 0x7f3a3c09cc10 con 0x7f3a4c102bd0 2024-05-20T16:25:40.364 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 64 2024-05-20T16:25:40.366 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:40.366+0000 7f3a52b4c640 1 -- 172.21.15.82:0/3379085089 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f3a380a2b90 msgr2=0x7f3a380a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:25:40.366 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:40.366+0000 7f3a52b4c640 1 --2- 172.21.15.82:0/3379085089 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f3a380a2b90 0x7f3a380a50a0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f3a44005ab0 tx=0x7f3a44031400 comp rx=0 tx=0).stop 2024-05-20T16:25:40.366 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:40.366+0000 7f3a52b4c640 1 -- 172.21.15.82:0/3379085089 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3a4c102bd0 msgr2=0x7f3a4c198a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:25:40.366 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:40.366+0000 7f3a52b4c640 1 --2- 172.21.15.82:0/3379085089 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3a4c102bd0 0x7f3a4c198a20 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f3a3c00ebc0 tx=0x7f3a3c007590 comp rx=0 tx=0).stop 2024-05-20T16:25:40.366 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:40.366+0000 7f3a52b4c640 1 -- 172.21.15.82:0/3379085089 shutdown_connections 2024-05-20T16:25:40.366 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:40.366+0000 7f3a52b4c640 1 --2- 172.21.15.82:0/3379085089 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f3a380a2b90 0x7f3a380a50a0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:25:40.366 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:40.366+0000 7f3a52b4c640 1 --2- 172.21.15.82:0/3379085089 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3a4c102bd0 0x7f3a4c198a20 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:25:40.366 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:40.366+0000 7f3a52b4c640 1 --2- 172.21.15.82:0/3379085089 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3a4c1078d0 0x7f3a4c19f9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:25:40.366 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:40.366+0000 7f3a52b4c640 1 --2- 172.21.15.82:0/3379085089 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3a4c101990 0x7f3a4c1984e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:25:40.366 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:40.366+0000 7f3a52b4c640 1 -- 172.21.15.82:0/3379085089 >> 172.21.15.82:0/3379085089 conn(0x7f3a4c0fcb80 msgr2=0x7f3a4c1038b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:25:40.366 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:40.366+0000 7f3a52b4c640 1 -- 172.21.15.82:0/3379085089 shutdown_connections 2024-05-20T16:25:40.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:40.366+0000 7f3a52b4c640 1 -- 172.21.15.82:0/3379085089 wait complete. 2024-05-20T16:25:40.377 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-20T16:25:40.493 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:40.493+0000 7f4ebea0d640 1 -- 172.21.15.82:0/4129220211 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4eb8101ef0 msgr2=0x7f4eb8102300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:25:40.493 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:40.493+0000 7f4ebea0d640 1 --2- 172.21.15.82:0/4129220211 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4eb8101ef0 0x7f4eb8102300 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f4ea8009900 tx=0x7f4ea802a410 comp rx=0 tx=0).stop 2024-05-20T16:25:40.493 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:40.493+0000 7f4ebea0d640 1 -- 172.21.15.82:0/4129220211 shutdown_connections 2024-05-20T16:25:40.493 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:40.493+0000 7f4ebea0d640 1 --2- 172.21.15.82:0/4129220211 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4eb8103410 0x7f4eb8109b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:25:40.493 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:40.493+0000 7f4ebea0d640 1 --2- 172.21.15.82:0/4129220211 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4eb81028d0 0x7f4eb8102d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:25:40.493 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:40.493+0000 7f4ebea0d640 1 --2- 172.21.15.82:0/4129220211 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4eb8101ef0 0x7f4eb8102300 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:25:40.493 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:40.493+0000 7f4ebea0d640 1 -- 172.21.15.82:0/4129220211 >> 172.21.15.82:0/4129220211 conn(0x7f4eb80fc2f0 msgr2=0x7f4eb80fe7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:25:40.494 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:40.493+0000 7f4ebea0d640 1 -- 172.21.15.82:0/4129220211 shutdown_connections 2024-05-20T16:25:40.494 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:40.493+0000 7f4ebea0d640 1 -- 172.21.15.82:0/4129220211 wait complete. 2024-05-20T16:25:40.494 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:40.494+0000 7f4ebea0d640 1 Processor -- start 2024-05-20T16:25:40.494 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:40.494+0000 7f4ebea0d640 1 -- start start 2024-05-20T16:25:40.495 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:40.494+0000 7f4ebea0d640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4eb8101ef0 0x7f4eb81992c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:25:40.496 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:40.494+0000 7f4ebea0d640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4eb81028d0 0x7f4eb8199800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:25:40.496 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:40.494+0000 7f4ebea0d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4eb8103410 0x7f4eb81a0790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:25:40.496 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:40.494+0000 7f4ebea0d640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f4eb810e720 con 0x7f4eb8103410 2024-05-20T16:25:40.496 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:40.494+0000 7f4ebea0d640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f4eb810e5a0 con 0x7f4eb8101ef0 2024-05-20T16:25:40.496 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:40.494+0000 7f4ebea0d640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f4eb810e8a0 con 0x7f4eb81028d0 2024-05-20T16:25:40.496 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:40.494+0000 7f4ebcfaa640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4eb8103410 0x7f4eb81a0790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:25:40.496 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:40.494+0000 7f4ebcfaa640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4eb8103410 0x7f4eb81a0790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:59444/0 (socket says 172.21.15.82:59444) 2024-05-20T16:25:40.496 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:40.494+0000 7f4ebcfaa640 1 -- 172.21.15.82:0/3463500826 learned_addr learned my addr 172.21.15.82:0/3463500826 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:25:40.496 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:40.494+0000 7f4eb77fe640 1 --2- 172.21.15.82:0/3463500826 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4eb81028d0 0x7f4eb8199800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:25:40.496 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:40.494+0000 7f4eb7fff640 1 --2- 172.21.15.82:0/3463500826 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4eb8101ef0 0x7f4eb81992c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:25:40.496 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:40.494+0000 7f4ebcfaa640 1 -- 172.21.15.82:0/3463500826 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4eb81028d0 msgr2=0x7f4eb8199800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:25:40.496 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:40.494+0000 7f4ebcfaa640 1 --2- 172.21.15.82:0/3463500826 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4eb81028d0 0x7f4eb8199800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:25:40.496 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:40.494+0000 7f4ebcfaa640 1 -- 172.21.15.82:0/3463500826 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4eb8101ef0 msgr2=0x7f4eb81992c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:25:40.497 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:40.494+0000 7f4ebcfaa640 1 --2- 172.21.15.82:0/3463500826 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4eb8101ef0 0x7f4eb81992c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:25:40.497 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:40.494+0000 7f4ebcfaa640 1 -- 172.21.15.82:0/3463500826 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4eb81a0e00 con 0x7f4eb8103410 2024-05-20T16:25:40.497 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:40.495+0000 7f4ebcfaa640 1 --2- 172.21.15.82:0/3463500826 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4eb8103410 0x7f4eb81a0790 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f4eb000eaa0 tx=0x7f4eb000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:25:40.497 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:40.495+0000 7f4eb57fa640 1 -- 172.21.15.82:0/3463500826 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eb000cdb0 con 0x7f4eb8103410 2024-05-20T16:25:40.497 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:40.495+0000 7f4ebea0d640 1 -- 172.21.15.82:0/3463500826 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4eb81a10f0 con 0x7f4eb8103410 2024-05-20T16:25:40.497 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:40.495+0000 7f4eb57fa640 1 -- 172.21.15.82:0/3463500826 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4eb0005020 con 0x7f4eb8103410 2024-05-20T16:25:40.497 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:40.495+0000 7f4eb57fa640 1 -- 172.21.15.82:0/3463500826 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eb00116b0 con 0x7f4eb8103410 2024-05-20T16:25:40.497 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:40.495+0000 7f4ebea0d640 1 -- 172.21.15.82:0/3463500826 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4eb81a15f0 con 0x7f4eb8103410 2024-05-20T16:25:40.497 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:40.496+0000 7f4ebea0d640 1 -- 172.21.15.82:0/3463500826 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4eb8106890 con 0x7f4eb8103410 2024-05-20T16:25:40.498 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:40.497+0000 7f4eb57fa640 1 -- 172.21.15.82:0/3463500826 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f4eb0005560 con 0x7f4eb8103410 2024-05-20T16:25:40.498 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:40.498+0000 7f4eb57fa640 1 --2- 172.21.15.82:0/3463500826 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f4ea00a2b90 0x7f4ea00a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:25:40.498 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:40.498+0000 7f4eb7fff640 1 --2- 172.21.15.82:0/3463500826 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f4ea00a2b90 0x7f4ea00a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:25:40.498 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:40.499+0000 7f4eb57fa640 1 -- 172.21.15.82:0/3463500826 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9150+0+0 (secure 0 0 0) 0x7f4eb00cfac0 con 0x7f4eb8103410 2024-05-20T16:25:40.498 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:40.499+0000 7f4eb7fff640 1 --2- 172.21.15.82:0/3463500826 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f4ea00a2b90 0x7f4ea00a50a0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f4ea8009230 tx=0x7f4ea8031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:25:40.501 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:40.502+0000 7f4eb57fa640 1 -- 172.21.15.82:0/3463500826 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f4eb0016030 con 0x7f4eb8103410 2024-05-20T16:25:40.680 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:40.680+0000 7f4ebea0d640 1 -- 172.21.15.82:0/3463500826 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f4eb8062750 con 0x7f4eb8103410 2024-05-20T16:25:40.680 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:40.681+0000 7f4eb57fa640 1 -- 172.21.15.82:0/3463500826 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v98) ==== 103+0+11 (secure 0 0 0) 0x7f4eb009c690 con 0x7f4eb8103410 2024-05-20T16:25:40.680 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 32 2024-05-20T16:25:40.683 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:40.683+0000 7f4ebea0d640 1 -- 172.21.15.82:0/3463500826 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f4ea00a2b90 msgr2=0x7f4ea00a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:25:40.683 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:40.683+0000 7f4ebea0d640 1 --2- 172.21.15.82:0/3463500826 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f4ea00a2b90 0x7f4ea00a50a0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f4ea8009230 tx=0x7f4ea8031400 comp rx=0 tx=0).stop 2024-05-20T16:25:40.683 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:40.683+0000 7f4ebea0d640 1 -- 172.21.15.82:0/3463500826 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4eb8103410 msgr2=0x7f4eb81a0790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:25:40.683 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:40.683+0000 7f4ebea0d640 1 --2- 172.21.15.82:0/3463500826 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4eb8103410 0x7f4eb81a0790 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f4eb000eaa0 tx=0x7f4eb000ef60 comp rx=0 tx=0).stop 2024-05-20T16:25:40.683 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:40.683+0000 7f4ebea0d640 1 -- 172.21.15.82:0/3463500826 shutdown_connections 2024-05-20T16:25:40.683 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:40.683+0000 7f4ebea0d640 1 --2- 172.21.15.82:0/3463500826 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f4ea00a2b90 0x7f4ea00a50a0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:25:40.683 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:40.683+0000 7f4ebea0d640 1 --2- 172.21.15.82:0/3463500826 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4eb8103410 0x7f4eb81a0790 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:25:40.683 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:40.683+0000 7f4ebea0d640 1 --2- 172.21.15.82:0/3463500826 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4eb81028d0 0x7f4eb8199800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:25:40.683 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:40.683+0000 7f4ebea0d640 1 --2- 172.21.15.82:0/3463500826 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4eb8101ef0 0x7f4eb81992c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:25:40.683 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:40.683+0000 7f4ebea0d640 1 -- 172.21.15.82:0/3463500826 >> 172.21.15.82:0/3463500826 conn(0x7f4eb80fc2f0 msgr2=0x7f4eb8103a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:25:40.684 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:40.683+0000 7f4ebea0d640 1 -- 172.21.15.82:0/3463500826 shutdown_connections 2024-05-20T16:25:40.684 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:40.683+0000 7f4ebea0d640 1 -- 172.21.15.82:0/3463500826 wait complete. 2024-05-20T16:25:40.693 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:25:40 smithi082 ceph-mon[32238]: from='client.? 172.21.15.82:0/2382010358' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-20T16:25:40.693 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:25:40 smithi082 ceph-mon[32238]: pgmap v715: 129 pgs: 129 active+clean; 10 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 822 MiB/s wr, 1.48k op/s 2024-05-20T16:25:40.693 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:25:40 smithi082 ceph-mon[32238]: from='client.? 172.21.15.82:0/2558355108' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-20T16:25:40.693 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:25:40 smithi082 ceph-mon[32238]: from='client.? 172.21.15.82:0/3379085089' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-20T16:25:40.694 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-05-20T16:25:40.808 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:40.806+0000 7f47e0eeb640 1 -- 172.21.15.82:0/3363656497 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f47dc104050 msgr2=0x7f47dc104460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:25:40.808 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:40.806+0000 7f47e0eeb640 1 --2- 172.21.15.82:0/3363656497 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f47dc104050 0x7f47dc104460 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f47cc009900 tx=0x7f47cc02a410 comp rx=0 tx=0).stop 2024-05-20T16:25:40.808 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:40.806+0000 7f47e0eeb640 1 -- 172.21.15.82:0/3363656497 shutdown_connections 2024-05-20T16:25:40.808 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:40.806+0000 7f47e0eeb640 1 --2- 172.21.15.82:0/3363656497 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f47dc101200 0x7f47dc10ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:25:40.808 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:40.806+0000 7f47e0eeb640 1 --2- 172.21.15.82:0/3363656497 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f47dc100700 0x7f47dc100b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:25:40.809 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:40.806+0000 7f47e0eeb640 1 --2- 172.21.15.82:0/3363656497 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f47dc104050 0x7f47dc104460 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:25:40.809 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:40.806+0000 7f47e0eeb640 1 -- 172.21.15.82:0/3363656497 >> 172.21.15.82:0/3363656497 conn(0x7f47dc0fc320 msgr2=0x7f47dc0fe7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:25:40.809 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:40.807+0000 7f47e0eeb640 1 -- 172.21.15.82:0/3363656497 shutdown_connections 2024-05-20T16:25:40.809 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:40.807+0000 7f47e0eeb640 1 -- 172.21.15.82:0/3363656497 wait complete. 2024-05-20T16:25:40.809 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:40.807+0000 7f47e0eeb640 1 Processor -- start 2024-05-20T16:25:40.809 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:40.807+0000 7f47e0eeb640 1 -- start start 2024-05-20T16:25:40.809 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:40.807+0000 7f47e0eeb640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f47dc100700 0x7f47dc199270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:25:40.809 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:40.807+0000 7f47e0eeb640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f47dc101200 0x7f47dc1997b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:25:40.809 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:40.808+0000 7f47e0eeb640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f47dc104050 0x7f47dc1a0740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:25:40.809 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:40.808+0000 7f47e0eeb640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f47dc10e6f0 con 0x7f47dc104050 2024-05-20T16:25:40.809 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:40.808+0000 7f47e0eeb640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f47dc10e570 con 0x7f47dc100700 2024-05-20T16:25:40.809 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:40.808+0000 7f47e0eeb640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f47dc10e870 con 0x7f47dc101200 2024-05-20T16:25:40.809 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:40.808+0000 7f47dad9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f47dc104050 0x7f47dc1a0740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:25:40.809 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:40.808+0000 7f47dad9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f47dc104050 0x7f47dc1a0740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:59470/0 (socket says 172.21.15.82:59470) 2024-05-20T16:25:40.810 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:40.808+0000 7f47dad9d640 1 -- 172.21.15.82:0/4036978409 learned_addr learned my addr 172.21.15.82:0/4036978409 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:25:40.810 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:40.808+0000 7f47da59c640 1 --2- 172.21.15.82:0/4036978409 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f47dc100700 0x7f47dc199270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:25:40.810 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:40.808+0000 7f47d3fff640 1 --2- 172.21.15.82:0/4036978409 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f47dc101200 0x7f47dc1997b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:25:40.810 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:40.808+0000 7f47dad9d640 1 -- 172.21.15.82:0/4036978409 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f47dc101200 msgr2=0x7f47dc1997b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:25:40.810 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:40.808+0000 7f47dad9d640 1 --2- 172.21.15.82:0/4036978409 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f47dc101200 0x7f47dc1997b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:25:40.810 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:40.808+0000 7f47dad9d640 1 -- 172.21.15.82:0/4036978409 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f47dc100700 msgr2=0x7f47dc199270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:25:40.810 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:40.808+0000 7f47dad9d640 1 --2- 172.21.15.82:0/4036978409 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f47dc100700 0x7f47dc199270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:25:40.810 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:40.808+0000 7f47dad9d640 1 -- 172.21.15.82:0/4036978409 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47dc1a0db0 con 0x7f47dc104050 2024-05-20T16:25:40.810 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:40.808+0000 7f47da59c640 1 --2- 172.21.15.82:0/4036978409 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f47dc100700 0x7f47dc199270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T16:25:40.810 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:40.808+0000 7f47dad9d640 1 --2- 172.21.15.82:0/4036978409 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f47dc104050 0x7f47dc1a0740 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f47d4009fd0 tx=0x7f47d400b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:25:40.810 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:40.808+0000 7f47d37fe640 1 -- 172.21.15.82:0/4036978409 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47d4017070 con 0x7f47dc104050 2024-05-20T16:25:40.810 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:40.808+0000 7f47d37fe640 1 -- 172.21.15.82:0/4036978409 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f47d4003e50 con 0x7f47dc104050 2024-05-20T16:25:40.810 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:40.808+0000 7f47e0eeb640 1 -- 172.21.15.82:0/4036978409 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47dc1a10a0 con 0x7f47dc104050 2024-05-20T16:25:40.810 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:40.808+0000 7f47d37fe640 1 -- 172.21.15.82:0/4036978409 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47d40106c0 con 0x7f47dc104050 2024-05-20T16:25:40.811 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:40.808+0000 7f47e0eeb640 1 -- 172.21.15.82:0/4036978409 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47dc1a15a0 con 0x7f47dc104050 2024-05-20T16:25:40.811 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:40.809+0000 7f47e0eeb640 1 -- 172.21.15.82:0/4036978409 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47dc1079e0 con 0x7f47dc104050 2024-05-20T16:25:40.811 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:40.811+0000 7f47d37fe640 1 -- 172.21.15.82:0/4036978409 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f47d4003970 con 0x7f47dc104050 2024-05-20T16:25:40.812 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:40.812+0000 7f47d37fe640 1 --2- 172.21.15.82:0/4036978409 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f47c80a2b90 0x7f47c80a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:25:40.812 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:40.812+0000 7f47d37fe640 1 -- 172.21.15.82:0/4036978409 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9150+0+0 (secure 0 0 0) 0x7f47d40cf210 con 0x7f47dc104050 2024-05-20T16:25:40.812 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:40.812+0000 7f47da59c640 1 --2- 172.21.15.82:0/4036978409 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f47c80a2b90 0x7f47c80a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:25:40.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:25:40 smithi160 ceph-mon[36347]: from='client.? 172.21.15.82:0/2382010358' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-20T16:25:40.813 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:25:40 smithi160 ceph-mon[36347]: pgmap v715: 129 pgs: 129 active+clean; 10 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 822 MiB/s wr, 1.48k op/s 2024-05-20T16:25:40.813 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:25:40 smithi160 ceph-mon[36347]: from='client.? 172.21.15.82:0/2558355108' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-20T16:25:40.813 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:25:40 smithi160 ceph-mon[36347]: from='client.? 172.21.15.82:0/3379085089' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-20T16:25:40.813 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:40.813+0000 7f47da59c640 1 --2- 172.21.15.82:0/4036978409 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f47c80a2b90 0x7f47c80a50a0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f47cc009230 tx=0x7f47cc000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:25:40.815 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:40.816+0000 7f47d37fe640 1 -- 172.21.15.82:0/4036978409 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f47d409bde0 con 0x7f47dc104050 2024-05-20T16:25:40.885 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:25:40 smithi184 ceph-mon[36209]: from='client.? 172.21.15.82:0/2382010358' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-20T16:25:40.885 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:25:40 smithi184 ceph-mon[36209]: pgmap v715: 129 pgs: 129 active+clean; 10 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 822 MiB/s wr, 1.48k op/s 2024-05-20T16:25:40.885 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:25:40 smithi184 ceph-mon[36209]: from='client.? 172.21.15.82:0/2558355108' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-20T16:25:40.885 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:25:40 smithi184 ceph-mon[36209]: from='client.? 172.21.15.82:0/3379085089' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-20T16:25:41.040 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:41.040+0000 7f47e0eeb640 1 -- 172.21.15.82:0/4036978409 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7f47dc02d470 con 0x7f47dc104050 2024-05-20T16:25:41.048 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:41.048+0000 7f47d37fe640 1 -- 172.21.15.82:0/4036978409 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v20) ==== 81+0+740368 (secure 0 0 0) 0x7f47d409b6e0 con 0x7f47dc104050 2024-05-20T16:25:41.057 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-05-20T16:25:41.057 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-05-20T16:25:41.057 INFO:teuthology.orchestra.run.smithi082.stdout: "epoch": 20, 2024-05-20T16:25:41.057 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.058 INFO:teuthology.orchestra.run.smithi082.stdout: "active_gid": 14150, 2024-05-20T16:25:41.058 INFO:teuthology.orchestra.run.smithi082.stdout: "active_name": "x", 2024-05-20T16:25:41.058 INFO:teuthology.orchestra.run.smithi082.stdout: "active_addrs": { 2024-05-20T16:25:41.058 INFO:teuthology.orchestra.run.smithi082.stdout: "addrvec": [ 2024-05-20T16:25:41.058 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-05-20T16:25:41.058 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "v2", 2024-05-20T16:25:41.058 INFO:teuthology.orchestra.run.smithi082.stdout: "addr": "172.21.15.82:6800", 2024-05-20T16:25:41.058 INFO:teuthology.orchestra.run.smithi082.stdout: "nonce": 379530823 2024-05-20T16:25:41.058 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.058 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-05-20T16:25:41.058 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "v1", 2024-05-20T16:25:41.058 INFO:teuthology.orchestra.run.smithi082.stdout: "addr": "172.21.15.82:6801", 2024-05-20T16:25:41.058 INFO:teuthology.orchestra.run.smithi082.stdout: "nonce": 379530823 2024-05-20T16:25:41.059 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-20T16:25:41.059 INFO:teuthology.orchestra.run.smithi082.stdout: ] 2024-05-20T16:25:41.059 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.059 INFO:teuthology.orchestra.run.smithi082.stdout: "active_addr": "172.21.15.82:6801/379530823", 2024-05-20T16:25:41.059 INFO:teuthology.orchestra.run.smithi082.stdout: "active_change": "2024-05-20T16:04:45.162085+0000", 2024-05-20T16:25:41.059 INFO:teuthology.orchestra.run.smithi082.stdout: "active_mgr_features": 4540701547738038271, 2024-05-20T16:25:41.059 INFO:teuthology.orchestra.run.smithi082.stdout: "available": true, 2024-05-20T16:25:41.059 INFO:teuthology.orchestra.run.smithi082.stdout: "standbys": [ 2024-05-20T16:25:41.059 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-05-20T16:25:41.059 INFO:teuthology.orchestra.run.smithi082.stdout: "gid": 24109, 2024-05-20T16:25:41.059 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "z", 2024-05-20T16:25:41.059 INFO:teuthology.orchestra.run.smithi082.stdout: "mgr_features": 4540701547738038271, 2024-05-20T16:25:41.059 INFO:teuthology.orchestra.run.smithi082.stdout: "available_modules": [ 2024-05-20T16:25:41.060 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-05-20T16:25:41.060 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "alerts", 2024-05-20T16:25:41.060 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-05-20T16:25:41.060 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-05-20T16:25:41.060 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-05-20T16:25:41.060 INFO:teuthology.orchestra.run.smithi082.stdout: "interval": { 2024-05-20T16:25:41.060 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "interval", 2024-05-20T16:25:41.060 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2024-05-20T16:25:41.060 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.060 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.060 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "60", 2024-05-20T16:25:41.060 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.061 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.061 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.061 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "How frequently to reexamine health status", 2024-05-20T16:25:41.061 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.061 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.061 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.061 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.061 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-05-20T16:25:41.061 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-05-20T16:25:41.061 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.061 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.062 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.062 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:41.062 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.062 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.062 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-05-20T16:25:41.062 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-05-20T16:25:41.062 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-05-20T16:25:41.062 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-05-20T16:25:41.062 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-05-20T16:25:41.062 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-05-20T16:25:41.062 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-05-20T16:25:41.063 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:41.063 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.063 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.063 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.063 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.063 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.063 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-05-20T16:25:41.063 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-05-20T16:25:41.063 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.063 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.063 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.063 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:41.063 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.064 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.064 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.064 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.064 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.064 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.064 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.064 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.064 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-05-20T16:25:41.064 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-05-20T16:25:41.065 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.065 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.065 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.065 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-05-20T16:25:41.065 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.065 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.065 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-05-20T16:25:41.065 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-05-20T16:25:41.065 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-05-20T16:25:41.065 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-05-20T16:25:41.065 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-05-20T16:25:41.065 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-05-20T16:25:41.065 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-05-20T16:25:41.066 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:41.066 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.066 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.066 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.066 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.066 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.066 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-05-20T16:25:41.066 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-05-20T16:25:41.066 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.066 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.066 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.066 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:41.066 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.067 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.067 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.067 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.067 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.067 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.067 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.067 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.067 INFO:teuthology.orchestra.run.smithi082.stdout: "smtp_destination": { 2024-05-20T16:25:41.067 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "smtp_destination", 2024-05-20T16:25:41.067 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.067 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.067 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.067 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:41.067 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.067 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.068 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.068 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Email address to send alerts to", 2024-05-20T16:25:41.068 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.068 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.068 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.068 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.068 INFO:teuthology.orchestra.run.smithi082.stdout: "smtp_from_name": { 2024-05-20T16:25:41.068 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "smtp_from_name", 2024-05-20T16:25:41.068 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.068 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.068 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.068 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "Ceph", 2024-05-20T16:25:41.068 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.068 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.068 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.069 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Email From: name", 2024-05-20T16:25:41.069 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.069 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.069 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.069 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.069 INFO:teuthology.orchestra.run.smithi082.stdout: "smtp_host": { 2024-05-20T16:25:41.069 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "smtp_host", 2024-05-20T16:25:41.069 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.069 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.069 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.069 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:41.069 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.069 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.069 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.069 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "SMTP server", 2024-05-20T16:25:41.069 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.070 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.070 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.070 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.070 INFO:teuthology.orchestra.run.smithi082.stdout: "smtp_password": { 2024-05-20T16:25:41.070 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "smtp_password", 2024-05-20T16:25:41.070 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.070 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.070 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.070 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:41.070 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.070 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.070 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.070 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Password to authenticate with", 2024-05-20T16:25:41.070 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.071 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.071 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.071 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.071 INFO:teuthology.orchestra.run.smithi082.stdout: "smtp_port": { 2024-05-20T16:25:41.071 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "smtp_port", 2024-05-20T16:25:41.071 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-05-20T16:25:41.071 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.071 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.071 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "465", 2024-05-20T16:25:41.071 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.071 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.071 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.072 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "SMTP port", 2024-05-20T16:25:41.072 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.072 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.072 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.072 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.072 INFO:teuthology.orchestra.run.smithi082.stdout: "smtp_sender": { 2024-05-20T16:25:41.072 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "smtp_sender", 2024-05-20T16:25:41.072 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.072 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.072 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.072 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:41.072 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.072 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.072 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.073 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "SMTP envelope sender", 2024-05-20T16:25:41.073 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.073 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.073 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.073 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.073 INFO:teuthology.orchestra.run.smithi082.stdout: "smtp_ssl": { 2024-05-20T16:25:41.073 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "smtp_ssl", 2024-05-20T16:25:41.073 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.073 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.073 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.073 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-05-20T16:25:41.073 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.073 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.074 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.074 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Use SSL to connect to SMTP server", 2024-05-20T16:25:41.074 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.074 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.074 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.074 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.074 INFO:teuthology.orchestra.run.smithi082.stdout: "smtp_user": { 2024-05-20T16:25:41.074 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "smtp_user", 2024-05-20T16:25:41.074 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.074 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.074 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.074 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:41.074 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.074 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.075 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.075 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "User to authenticate as", 2024-05-20T16:25:41.075 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.075 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.075 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.075 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-20T16:25:41.075 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-20T16:25:41.075 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.075 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-05-20T16:25:41.075 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "balancer", 2024-05-20T16:25:41.075 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-05-20T16:25:41.075 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-05-20T16:25:41.075 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-05-20T16:25:41.075 INFO:teuthology.orchestra.run.smithi082.stdout: "active": { 2024-05-20T16:25:41.075 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "active", 2024-05-20T16:25:41.076 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.076 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.076 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.076 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-05-20T16:25:41.076 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.076 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.076 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.076 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "automatically balance PGs across cluster", 2024-05-20T16:25:41.076 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.076 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.076 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.076 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.076 INFO:teuthology.orchestra.run.smithi082.stdout: "begin_time": { 2024-05-20T16:25:41.076 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "begin_time", 2024-05-20T16:25:41.077 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.077 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.077 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.077 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "0000", 2024-05-20T16:25:41.077 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.077 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.077 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.077 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "beginning time of day to automatically balance", 2024-05-20T16:25:41.077 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-20T16:25:41.077 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.077 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.077 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.078 INFO:teuthology.orchestra.run.smithi082.stdout: "begin_weekday": { 2024-05-20T16:25:41.078 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "begin_weekday", 2024-05-20T16:25:41.078 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "uint", 2024-05-20T16:25:41.078 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.078 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.078 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "0", 2024-05-20T16:25:41.078 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "0", 2024-05-20T16:25:41.078 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "6", 2024-05-20T16:25:41.078 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.078 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-05-20T16:25:41.078 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-20T16:25:41.079 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.079 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.079 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.079 INFO:teuthology.orchestra.run.smithi082.stdout: "crush_compat_max_iterations": { 2024-05-20T16:25:41.079 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "crush_compat_max_iterations", 2024-05-20T16:25:41.079 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "uint", 2024-05-20T16:25:41.079 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.079 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.079 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "25", 2024-05-20T16:25:41.079 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "1", 2024-05-20T16:25:41.079 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "250", 2024-05-20T16:25:41.079 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.079 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-05-20T16:25:41.080 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.080 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.080 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.080 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.080 INFO:teuthology.orchestra.run.smithi082.stdout: "crush_compat_metrics": { 2024-05-20T16:25:41.080 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "crush_compat_metrics", 2024-05-20T16:25:41.080 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.080 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.080 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.080 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "pgs,objects,bytes", 2024-05-20T16:25:41.080 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.080 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.080 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.081 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-05-20T16:25:41.081 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-05-20T16:25:41.081 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.081 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.081 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.081 INFO:teuthology.orchestra.run.smithi082.stdout: "crush_compat_step": { 2024-05-20T16:25:41.081 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "crush_compat_step", 2024-05-20T16:25:41.081 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "float", 2024-05-20T16:25:41.081 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.082 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.082 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "0.5", 2024-05-20T16:25:41.082 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "0.001", 2024-05-20T16:25:41.082 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "0.999", 2024-05-20T16:25:41.082 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.082 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "aggressiveness of optimization", 2024-05-20T16:25:41.082 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-05-20T16:25:41.082 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.082 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.082 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.082 INFO:teuthology.orchestra.run.smithi082.stdout: "end_time": { 2024-05-20T16:25:41.082 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "end_time", 2024-05-20T16:25:41.082 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.083 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.083 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.083 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "2359", 2024-05-20T16:25:41.083 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.083 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.083 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.083 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "ending time of day to automatically balance", 2024-05-20T16:25:41.083 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-20T16:25:41.083 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.083 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.083 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.083 INFO:teuthology.orchestra.run.smithi082.stdout: "end_weekday": { 2024-05-20T16:25:41.083 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "end_weekday", 2024-05-20T16:25:41.083 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "uint", 2024-05-20T16:25:41.084 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.084 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.084 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "0", 2024-05-20T16:25:41.084 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "0", 2024-05-20T16:25:41.084 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "6", 2024-05-20T16:25:41.084 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.084 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-05-20T16:25:41.084 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-20T16:25:41.084 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.084 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.084 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.084 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-05-20T16:25:41.084 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-05-20T16:25:41.084 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.085 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.085 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.085 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:41.085 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.085 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.085 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-05-20T16:25:41.085 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-05-20T16:25:41.085 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-05-20T16:25:41.085 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-05-20T16:25:41.085 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-05-20T16:25:41.085 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-05-20T16:25:41.085 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-05-20T16:25:41.085 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:41.085 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.086 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.086 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.086 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.086 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.086 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-05-20T16:25:41.086 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-05-20T16:25:41.086 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.086 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.086 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.086 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:41.086 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.086 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.086 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.087 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.087 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.087 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.087 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.087 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.087 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-05-20T16:25:41.087 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-05-20T16:25:41.087 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.087 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.087 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.087 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-05-20T16:25:41.087 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.087 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.087 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-05-20T16:25:41.087 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-05-20T16:25:41.088 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-05-20T16:25:41.088 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-05-20T16:25:41.088 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-05-20T16:25:41.088 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-05-20T16:25:41.088 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-05-20T16:25:41.088 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:41.088 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.088 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.088 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.088 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.088 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.088 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-05-20T16:25:41.088 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-05-20T16:25:41.088 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.088 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.089 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.089 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:41.089 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.089 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.089 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.089 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.089 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.089 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.089 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.089 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.089 INFO:teuthology.orchestra.run.smithi082.stdout: "min_score": { 2024-05-20T16:25:41.089 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "min_score", 2024-05-20T16:25:41.089 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "float", 2024-05-20T16:25:41.089 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.090 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.090 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "0", 2024-05-20T16:25:41.090 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.090 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.090 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.090 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-05-20T16:25:41.090 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.090 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.090 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.090 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.090 INFO:teuthology.orchestra.run.smithi082.stdout: "mode": { 2024-05-20T16:25:41.090 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "mode", 2024-05-20T16:25:41.090 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.090 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.091 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.091 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "upmap", 2024-05-20T16:25:41.091 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.091 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.091 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-05-20T16:25:41.091 INFO:teuthology.orchestra.run.smithi082.stdout: "crush-compat", 2024-05-20T16:25:41.091 INFO:teuthology.orchestra.run.smithi082.stdout: "none", 2024-05-20T16:25:41.091 INFO:teuthology.orchestra.run.smithi082.stdout: "read", 2024-05-20T16:25:41.091 INFO:teuthology.orchestra.run.smithi082.stdout: "upmap", 2024-05-20T16:25:41.091 INFO:teuthology.orchestra.run.smithi082.stdout: "upmap-read" 2024-05-20T16:25:41.091 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:41.091 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Balancer mode", 2024-05-20T16:25:41.092 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.092 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.092 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.092 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.092 INFO:teuthology.orchestra.run.smithi082.stdout: "pool_ids": { 2024-05-20T16:25:41.092 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "pool_ids", 2024-05-20T16:25:41.092 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.092 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.092 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.092 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:41.093 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.093 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.093 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.093 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-05-20T16:25:41.093 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.093 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.093 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.093 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.093 INFO:teuthology.orchestra.run.smithi082.stdout: "sleep_interval": { 2024-05-20T16:25:41.093 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "sleep_interval", 2024-05-20T16:25:41.093 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2024-05-20T16:25:41.094 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.094 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.094 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "60", 2024-05-20T16:25:41.094 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.094 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.094 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.094 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-05-20T16:25:41.094 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.094 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.094 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.094 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.094 INFO:teuthology.orchestra.run.smithi082.stdout: "upmap_max_deviation": { 2024-05-20T16:25:41.094 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "upmap_max_deviation", 2024-05-20T16:25:41.095 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-05-20T16:25:41.095 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.095 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.095 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "5", 2024-05-20T16:25:41.095 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "1", 2024-05-20T16:25:41.095 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.095 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.095 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "deviation below which no optimization is attempted", 2024-05-20T16:25:41.095 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-05-20T16:25:41.095 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.095 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.095 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.095 INFO:teuthology.orchestra.run.smithi082.stdout: "upmap_max_optimizations": { 2024-05-20T16:25:41.095 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "upmap_max_optimizations", 2024-05-20T16:25:41.095 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "uint", 2024-05-20T16:25:41.096 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.096 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.096 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "10", 2024-05-20T16:25:41.096 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.096 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.096 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.096 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-05-20T16:25:41.096 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.096 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.096 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.096 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-20T16:25:41.096 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-20T16:25:41.096 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.096 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-05-20T16:25:41.097 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "cephadm", 2024-05-20T16:25:41.097 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-05-20T16:25:41.097 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-05-20T16:25:41.097 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-05-20T16:25:41.097 INFO:teuthology.orchestra.run.smithi082.stdout: "agent_down_multiplier": { 2024-05-20T16:25:41.097 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "agent_down_multiplier", 2024-05-20T16:25:41.097 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "float", 2024-05-20T16:25:41.097 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.097 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.097 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "3.0", 2024-05-20T16:25:41.097 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.097 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.097 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.097 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-05-20T16:25:41.097 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.098 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.098 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.098 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.098 INFO:teuthology.orchestra.run.smithi082.stdout: "agent_refresh_rate": { 2024-05-20T16:25:41.098 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "agent_refresh_rate", 2024-05-20T16:25:41.098 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2024-05-20T16:25:41.098 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.098 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.098 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "20", 2024-05-20T16:25:41.098 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.098 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.098 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.098 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-05-20T16:25:41.098 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.099 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.099 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.099 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.099 INFO:teuthology.orchestra.run.smithi082.stdout: "agent_starting_port": { 2024-05-20T16:25:41.099 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "agent_starting_port", 2024-05-20T16:25:41.099 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-05-20T16:25:41.099 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.099 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.099 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "4721", 2024-05-20T16:25:41.099 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.099 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.099 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.099 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-05-20T16:25:41.099 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.100 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.100 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.100 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.100 INFO:teuthology.orchestra.run.smithi082.stdout: "allow_ptrace": { 2024-05-20T16:25:41.100 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "allow_ptrace", 2024-05-20T16:25:41.100 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.100 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.100 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.100 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:41.100 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.100 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.100 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.100 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-05-20T16:25:41.100 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-05-20T16:25:41.100 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.101 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.101 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.101 INFO:teuthology.orchestra.run.smithi082.stdout: "autotune_interval": { 2024-05-20T16:25:41.101 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "autotune_interval", 2024-05-20T16:25:41.101 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2024-05-20T16:25:41.101 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.101 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.101 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "600", 2024-05-20T16:25:41.101 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.101 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.101 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.101 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "how frequently to autotune daemon memory", 2024-05-20T16:25:41.101 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.101 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.101 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.102 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.102 INFO:teuthology.orchestra.run.smithi082.stdout: "autotune_memory_target_ratio": { 2024-05-20T16:25:41.102 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "autotune_memory_target_ratio", 2024-05-20T16:25:41.102 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "float", 2024-05-20T16:25:41.102 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.102 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.102 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "0.7", 2024-05-20T16:25:41.102 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.102 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.102 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.103 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-05-20T16:25:41.103 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.103 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.103 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.103 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.103 INFO:teuthology.orchestra.run.smithi082.stdout: "cephadm_log_destination": { 2024-05-20T16:25:41.103 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "cephadm_log_destination", 2024-05-20T16:25:41.103 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.103 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.103 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.103 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:41.103 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.103 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.104 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-05-20T16:25:41.104 INFO:teuthology.orchestra.run.smithi082.stdout: "file", 2024-05-20T16:25:41.104 INFO:teuthology.orchestra.run.smithi082.stdout: "file,syslog", 2024-05-20T16:25:41.104 INFO:teuthology.orchestra.run.smithi082.stdout: "syslog" 2024-05-20T16:25:41.104 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:41.104 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-05-20T16:25:41.104 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.104 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.104 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.104 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.104 INFO:teuthology.orchestra.run.smithi082.stdout: "cgroups_split": { 2024-05-20T16:25:41.104 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "cgroups_split", 2024-05-20T16:25:41.104 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.104 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.105 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.105 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-05-20T16:25:41.105 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.105 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.105 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.105 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-05-20T16:25:41.105 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.105 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.105 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.105 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.105 INFO:teuthology.orchestra.run.smithi082.stdout: "config_checks_enabled": { 2024-05-20T16:25:41.105 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "config_checks_enabled", 2024-05-20T16:25:41.105 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.105 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.106 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.106 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:41.106 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.106 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.106 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.106 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-05-20T16:25:41.106 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.106 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.106 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.106 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.106 INFO:teuthology.orchestra.run.smithi082.stdout: "config_dashboard": { 2024-05-20T16:25:41.106 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "config_dashboard", 2024-05-20T16:25:41.106 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.106 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.107 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.107 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-05-20T16:25:41.107 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.107 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.107 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.107 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-05-20T16:25:41.107 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.107 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.107 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.107 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.107 INFO:teuthology.orchestra.run.smithi082.stdout: "container_image_alertmanager": { 2024-05-20T16:25:41.107 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "container_image_alertmanager", 2024-05-20T16:25:41.107 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.107 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.108 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.108 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-05-20T16:25:41.108 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.108 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.108 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.108 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Prometheus container image", 2024-05-20T16:25:41.108 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.108 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.108 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.108 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.108 INFO:teuthology.orchestra.run.smithi082.stdout: "container_image_base": { 2024-05-20T16:25:41.108 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "container_image_base", 2024-05-20T16:25:41.108 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.108 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.109 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.109 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "quay.io/ceph/ceph", 2024-05-20T16:25:41.109 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.109 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.109 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.109 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Container image name, without the tag", 2024-05-20T16:25:41.109 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.109 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.109 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.109 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.109 INFO:teuthology.orchestra.run.smithi082.stdout: "container_image_elasticsearch": { 2024-05-20T16:25:41.109 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "container_image_elasticsearch", 2024-05-20T16:25:41.109 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.109 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.110 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.110 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-05-20T16:25:41.110 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.110 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.110 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.110 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "elasticsearch container image", 2024-05-20T16:25:41.110 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.110 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.110 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.110 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.110 INFO:teuthology.orchestra.run.smithi082.stdout: "container_image_grafana": { 2024-05-20T16:25:41.110 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "container_image_grafana", 2024-05-20T16:25:41.111 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.111 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.111 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.111 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-05-20T16:25:41.111 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.111 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.111 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.111 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Prometheus container image", 2024-05-20T16:25:41.111 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.111 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.111 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.111 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.111 INFO:teuthology.orchestra.run.smithi082.stdout: "container_image_haproxy": { 2024-05-20T16:25:41.111 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "container_image_haproxy", 2024-05-20T16:25:41.111 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.112 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.112 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.112 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-05-20T16:25:41.112 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.112 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.112 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.112 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "HAproxy container image", 2024-05-20T16:25:41.112 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.112 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.112 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.112 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.112 INFO:teuthology.orchestra.run.smithi082.stdout: "container_image_jaeger_agent": { 2024-05-20T16:25:41.113 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "container_image_jaeger_agent", 2024-05-20T16:25:41.113 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.113 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.113 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.113 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-05-20T16:25:41.113 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.113 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.113 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.113 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Jaeger agent container image", 2024-05-20T16:25:41.113 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.113 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.113 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.113 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.113 INFO:teuthology.orchestra.run.smithi082.stdout: "container_image_jaeger_collector": { 2024-05-20T16:25:41.113 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "container_image_jaeger_collector", 2024-05-20T16:25:41.114 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.114 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.114 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.114 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-05-20T16:25:41.114 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.114 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.114 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.114 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Jaeger collector container image", 2024-05-20T16:25:41.114 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.114 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.114 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.114 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.114 INFO:teuthology.orchestra.run.smithi082.stdout: "container_image_jaeger_query": { 2024-05-20T16:25:41.114 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "container_image_jaeger_query", 2024-05-20T16:25:41.114 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.114 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.115 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.115 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-05-20T16:25:41.115 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.115 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.115 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.115 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Jaeger query container image", 2024-05-20T16:25:41.115 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.115 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.115 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.115 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.115 INFO:teuthology.orchestra.run.smithi082.stdout: "container_image_keepalived": { 2024-05-20T16:25:41.115 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "container_image_keepalived", 2024-05-20T16:25:41.115 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.115 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.115 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.116 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-05-20T16:25:41.116 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.116 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.116 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.116 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Keepalived container image", 2024-05-20T16:25:41.116 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.116 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.116 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.116 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.116 INFO:teuthology.orchestra.run.smithi082.stdout: "container_image_loki": { 2024-05-20T16:25:41.116 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "container_image_loki", 2024-05-20T16:25:41.116 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.116 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.116 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.116 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-05-20T16:25:41.116 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.117 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.117 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.117 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Loki container image", 2024-05-20T16:25:41.117 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.117 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.117 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.117 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.117 INFO:teuthology.orchestra.run.smithi082.stdout: "container_image_node_exporter": { 2024-05-20T16:25:41.117 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "container_image_node_exporter", 2024-05-20T16:25:41.117 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.117 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.117 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.117 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-05-20T16:25:41.117 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.117 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.117 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.118 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Prometheus container image", 2024-05-20T16:25:41.118 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.118 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.118 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.118 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.118 INFO:teuthology.orchestra.run.smithi082.stdout: "container_image_nvmeof": { 2024-05-20T16:25:41.118 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "container_image_nvmeof", 2024-05-20T16:25:41.118 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.118 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.118 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.118 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-05-20T16:25:41.118 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.118 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.118 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.118 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Nvme-of container image", 2024-05-20T16:25:41.119 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.119 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.119 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.119 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.119 INFO:teuthology.orchestra.run.smithi082.stdout: "container_image_prometheus": { 2024-05-20T16:25:41.119 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "container_image_prometheus", 2024-05-20T16:25:41.119 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.119 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.119 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.119 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-05-20T16:25:41.119 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.119 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.119 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.119 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Prometheus container image", 2024-05-20T16:25:41.120 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.120 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.120 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.120 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.120 INFO:teuthology.orchestra.run.smithi082.stdout: "container_image_promtail": { 2024-05-20T16:25:41.120 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "container_image_promtail", 2024-05-20T16:25:41.120 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.120 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.120 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.120 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-05-20T16:25:41.120 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.120 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.120 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.120 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Promtail container image", 2024-05-20T16:25:41.120 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.121 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.121 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.121 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.121 INFO:teuthology.orchestra.run.smithi082.stdout: "container_image_samba": { 2024-05-20T16:25:41.121 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "container_image_samba", 2024-05-20T16:25:41.121 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.121 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.121 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.121 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-05-20T16:25:41.121 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.121 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.121 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.121 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Samba/SMB container image", 2024-05-20T16:25:41.121 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.122 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.122 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.122 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.122 INFO:teuthology.orchestra.run.smithi082.stdout: "container_image_snmp_gateway": { 2024-05-20T16:25:41.122 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "container_image_snmp_gateway", 2024-05-20T16:25:41.122 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.122 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.122 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.122 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-05-20T16:25:41.122 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.122 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.122 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.122 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "SNMP Gateway container image", 2024-05-20T16:25:41.123 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.123 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.123 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.123 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.123 INFO:teuthology.orchestra.run.smithi082.stdout: "container_init": { 2024-05-20T16:25:41.123 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "container_init", 2024-05-20T16:25:41.123 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.123 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.123 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.123 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-05-20T16:25:41.123 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.123 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.123 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.123 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Run podman/docker with `--init`", 2024-05-20T16:25:41.123 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.124 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.124 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.124 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.124 INFO:teuthology.orchestra.run.smithi082.stdout: "daemon_cache_timeout": { 2024-05-20T16:25:41.124 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "daemon_cache_timeout", 2024-05-20T16:25:41.124 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2024-05-20T16:25:41.124 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.124 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.124 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "600", 2024-05-20T16:25:41.124 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.124 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.124 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.125 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-05-20T16:25:41.125 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.125 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.125 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.125 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.125 INFO:teuthology.orchestra.run.smithi082.stdout: "default_cephadm_command_timeout": { 2024-05-20T16:25:41.125 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "default_cephadm_command_timeout", 2024-05-20T16:25:41.125 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-05-20T16:25:41.125 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.125 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.125 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "900", 2024-05-20T16:25:41.125 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.125 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.125 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.126 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-05-20T16:25:41.126 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.126 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.126 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.126 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.126 INFO:teuthology.orchestra.run.smithi082.stdout: "default_registry": { 2024-05-20T16:25:41.126 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "default_registry", 2024-05-20T16:25:41.126 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.126 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.126 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.126 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "docker.io", 2024-05-20T16:25:41.126 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.126 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.127 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.127 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-05-20T16:25:41.127 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.127 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.127 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.127 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.127 INFO:teuthology.orchestra.run.smithi082.stdout: "device_cache_timeout": { 2024-05-20T16:25:41.127 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "device_cache_timeout", 2024-05-20T16:25:41.127 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2024-05-20T16:25:41.127 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.127 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.127 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "1800", 2024-05-20T16:25:41.127 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.127 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.127 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.127 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "seconds to cache device inventory", 2024-05-20T16:25:41.128 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.128 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.128 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.128 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.128 INFO:teuthology.orchestra.run.smithi082.stdout: "device_enhanced_scan": { 2024-05-20T16:25:41.128 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "device_enhanced_scan", 2024-05-20T16:25:41.128 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.128 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.128 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.128 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:41.128 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.128 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.128 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.128 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Use libstoragemgmt during device scans", 2024-05-20T16:25:41.128 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.129 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.129 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.129 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.129 INFO:teuthology.orchestra.run.smithi082.stdout: "facts_cache_timeout": { 2024-05-20T16:25:41.129 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "facts_cache_timeout", 2024-05-20T16:25:41.129 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2024-05-20T16:25:41.129 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.129 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.129 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "60", 2024-05-20T16:25:41.129 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.129 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.129 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.129 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "seconds to cache host facts data", 2024-05-20T16:25:41.130 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.130 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.130 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.130 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.130 INFO:teuthology.orchestra.run.smithi082.stdout: "grafana_dashboards_path": { 2024-05-20T16:25:41.130 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "grafana_dashboards_path", 2024-05-20T16:25:41.130 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.130 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.130 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.130 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-05-20T16:25:41.130 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.130 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.130 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.131 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-05-20T16:25:41.131 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.131 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.131 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.131 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.131 INFO:teuthology.orchestra.run.smithi082.stdout: "host_check_interval": { 2024-05-20T16:25:41.131 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "host_check_interval", 2024-05-20T16:25:41.131 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2024-05-20T16:25:41.131 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.131 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.131 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "600", 2024-05-20T16:25:41.131 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.131 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.131 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.132 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "how frequently to perform a host check", 2024-05-20T16:25:41.132 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.132 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.132 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.132 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.132 INFO:teuthology.orchestra.run.smithi082.stdout: "hw_monitoring": { 2024-05-20T16:25:41.132 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "hw_monitoring", 2024-05-20T16:25:41.132 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.132 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.132 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.132 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:41.132 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.132 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.133 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.133 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-05-20T16:25:41.133 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.133 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.133 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.133 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.133 INFO:teuthology.orchestra.run.smithi082.stdout: "inventory_list_all": { 2024-05-20T16:25:41.133 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "inventory_list_all", 2024-05-20T16:25:41.133 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.133 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.133 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.133 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:41.133 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.133 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.133 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.134 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-05-20T16:25:41.134 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.134 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.134 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.134 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.134 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-05-20T16:25:41.134 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-05-20T16:25:41.134 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.134 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.134 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.134 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:41.134 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.134 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.135 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-05-20T16:25:41.135 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-05-20T16:25:41.135 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-05-20T16:25:41.135 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-05-20T16:25:41.135 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-05-20T16:25:41.135 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-05-20T16:25:41.135 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-05-20T16:25:41.135 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:41.135 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.135 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.135 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.135 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.135 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.135 INFO:teuthology.orchestra.run.smithi082.stdout: "log_refresh_metadata": { 2024-05-20T16:25:41.135 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_refresh_metadata", 2024-05-20T16:25:41.136 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.136 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.136 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.136 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:41.136 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.136 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.136 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.136 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-05-20T16:25:41.136 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.136 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.136 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.136 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.137 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-05-20T16:25:41.137 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-05-20T16:25:41.137 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.137 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.137 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.137 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-05-20T16:25:41.137 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.137 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.137 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.137 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-05-20T16:25:41.137 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.137 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.137 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.138 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.138 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-05-20T16:25:41.138 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-05-20T16:25:41.138 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.138 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.138 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.138 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-05-20T16:25:41.138 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.138 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.138 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-05-20T16:25:41.138 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-05-20T16:25:41.138 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-05-20T16:25:41.138 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-05-20T16:25:41.139 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-05-20T16:25:41.139 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-05-20T16:25:41.139 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-05-20T16:25:41.139 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:41.139 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.139 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.139 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.139 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.139 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.139 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-05-20T16:25:41.139 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-05-20T16:25:41.139 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.139 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.139 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.140 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:41.140 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.140 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.140 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.140 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.140 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.140 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.140 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.140 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.140 INFO:teuthology.orchestra.run.smithi082.stdout: "manage_etc_ceph_ceph_conf": { 2024-05-20T16:25:41.140 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-05-20T16:25:41.140 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.140 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.141 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.141 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:41.141 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.141 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.141 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.141 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-05-20T16:25:41.141 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.141 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.141 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.141 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.141 INFO:teuthology.orchestra.run.smithi082.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-05-20T16:25:41.141 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-05-20T16:25:41.141 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.142 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.142 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.142 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "*", 2024-05-20T16:25:41.142 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.142 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.142 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.142 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-05-20T16:25:41.142 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.142 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.142 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.142 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.142 INFO:teuthology.orchestra.run.smithi082.stdout: "max_count_per_host": { 2024-05-20T16:25:41.142 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "max_count_per_host", 2024-05-20T16:25:41.142 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-05-20T16:25:41.143 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.143 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.143 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "10", 2024-05-20T16:25:41.143 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.143 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.143 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.143 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "max number of daemons per service per host", 2024-05-20T16:25:41.143 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.143 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.143 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.143 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.143 INFO:teuthology.orchestra.run.smithi082.stdout: "max_osd_draining_count": { 2024-05-20T16:25:41.143 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "max_osd_draining_count", 2024-05-20T16:25:41.143 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-05-20T16:25:41.143 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.144 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.144 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "10", 2024-05-20T16:25:41.144 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.144 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.144 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.144 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-05-20T16:25:41.144 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.144 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.144 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.144 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.144 INFO:teuthology.orchestra.run.smithi082.stdout: "migration_current": { 2024-05-20T16:25:41.144 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "migration_current", 2024-05-20T16:25:41.144 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-05-20T16:25:41.144 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.144 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.144 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:41.144 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.145 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.145 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.145 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "internal - do not modify", 2024-05-20T16:25:41.145 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.145 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.145 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.145 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.145 INFO:teuthology.orchestra.run.smithi082.stdout: "mode": { 2024-05-20T16:25:41.145 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "mode", 2024-05-20T16:25:41.145 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.145 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.145 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.145 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "root", 2024-05-20T16:25:41.145 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.145 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.145 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-05-20T16:25:41.146 INFO:teuthology.orchestra.run.smithi082.stdout: "cephadm-package", 2024-05-20T16:25:41.146 INFO:teuthology.orchestra.run.smithi082.stdout: "root" 2024-05-20T16:25:41.146 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:41.146 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "mode for remote execution of cephadm", 2024-05-20T16:25:41.146 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.146 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.146 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.146 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.146 INFO:teuthology.orchestra.run.smithi082.stdout: "oob_default_addr": { 2024-05-20T16:25:41.146 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "oob_default_addr", 2024-05-20T16:25:41.146 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.146 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.146 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.146 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "169.254.1.1", 2024-05-20T16:25:41.146 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.146 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.147 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.147 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Default address for RedFish API (oob management).", 2024-05-20T16:25:41.147 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.147 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.147 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.147 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.147 INFO:teuthology.orchestra.run.smithi082.stdout: "prometheus_alerts_path": { 2024-05-20T16:25:41.147 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "prometheus_alerts_path", 2024-05-20T16:25:41.147 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.147 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.147 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.147 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-05-20T16:25:41.147 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.147 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.147 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.147 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-05-20T16:25:41.148 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.148 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.148 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.148 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.148 INFO:teuthology.orchestra.run.smithi082.stdout: "registry_insecure": { 2024-05-20T16:25:41.148 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "registry_insecure", 2024-05-20T16:25:41.148 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.148 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.148 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.148 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:41.148 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.148 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.148 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.148 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-05-20T16:25:41.148 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.149 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.149 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.149 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.149 INFO:teuthology.orchestra.run.smithi082.stdout: "registry_password": { 2024-05-20T16:25:41.149 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "registry_password", 2024-05-20T16:25:41.149 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.149 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.149 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.149 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:41.149 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.149 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.149 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.149 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-05-20T16:25:41.149 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.149 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.150 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.150 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.150 INFO:teuthology.orchestra.run.smithi082.stdout: "registry_url": { 2024-05-20T16:25:41.150 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "registry_url", 2024-05-20T16:25:41.150 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.150 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.150 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.150 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:41.150 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.150 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.150 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.150 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-05-20T16:25:41.150 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.150 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.150 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.151 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.151 INFO:teuthology.orchestra.run.smithi082.stdout: "registry_username": { 2024-05-20T16:25:41.151 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "registry_username", 2024-05-20T16:25:41.151 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.151 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.151 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.151 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:41.151 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.151 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.151 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.151 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-05-20T16:25:41.151 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.151 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.151 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.152 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.152 INFO:teuthology.orchestra.run.smithi082.stdout: "secure_monitoring_stack": { 2024-05-20T16:25:41.152 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "secure_monitoring_stack", 2024-05-20T16:25:41.152 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.152 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.152 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.152 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:41.152 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.152 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.152 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.152 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-05-20T16:25:41.152 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.152 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.152 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.152 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.153 INFO:teuthology.orchestra.run.smithi082.stdout: "service_discovery_port": { 2024-05-20T16:25:41.153 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "service_discovery_port", 2024-05-20T16:25:41.153 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-05-20T16:25:41.153 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.153 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.153 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "8765", 2024-05-20T16:25:41.153 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.153 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.153 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.153 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "cephadm service discovery port", 2024-05-20T16:25:41.153 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.153 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.153 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.153 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.154 INFO:teuthology.orchestra.run.smithi082.stdout: "ssh_config_file": { 2024-05-20T16:25:41.154 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "ssh_config_file", 2024-05-20T16:25:41.154 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.154 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.154 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.154 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:41.154 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.154 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.154 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.154 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-05-20T16:25:41.154 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.154 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.154 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.154 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.154 INFO:teuthology.orchestra.run.smithi082.stdout: "use_agent": { 2024-05-20T16:25:41.154 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "use_agent", 2024-05-20T16:25:41.155 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.155 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.155 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.155 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:41.155 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.155 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.155 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.155 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-05-20T16:25:41.155 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.155 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.155 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.155 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.155 INFO:teuthology.orchestra.run.smithi082.stdout: "use_repo_digest": { 2024-05-20T16:25:41.155 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "use_repo_digest", 2024-05-20T16:25:41.155 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.155 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.156 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.156 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-05-20T16:25:41.156 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.156 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.156 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.156 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-05-20T16:25:41.156 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.156 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.156 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.156 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.156 INFO:teuthology.orchestra.run.smithi082.stdout: "warn_on_failed_host_check": { 2024-05-20T16:25:41.156 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "warn_on_failed_host_check", 2024-05-20T16:25:41.156 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.156 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.156 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.156 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-05-20T16:25:41.157 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.157 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.157 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.157 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "raise a health warning if the host check fails", 2024-05-20T16:25:41.157 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.157 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.157 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.157 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.157 INFO:teuthology.orchestra.run.smithi082.stdout: "warn_on_stray_daemons": { 2024-05-20T16:25:41.157 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "warn_on_stray_daemons", 2024-05-20T16:25:41.157 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.157 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.157 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.157 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-05-20T16:25:41.157 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.157 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.158 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.158 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-05-20T16:25:41.158 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.158 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.158 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.158 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.158 INFO:teuthology.orchestra.run.smithi082.stdout: "warn_on_stray_hosts": { 2024-05-20T16:25:41.158 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "warn_on_stray_hosts", 2024-05-20T16:25:41.158 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.158 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.158 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.158 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-05-20T16:25:41.158 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.158 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.159 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.159 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-05-20T16:25:41.159 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.159 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.159 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.159 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-20T16:25:41.159 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-20T16:25:41.159 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.159 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-05-20T16:25:41.159 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "crash", 2024-05-20T16:25:41.159 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-05-20T16:25:41.159 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-05-20T16:25:41.159 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-05-20T16:25:41.159 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-05-20T16:25:41.159 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-05-20T16:25:41.160 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.160 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.160 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.160 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:41.160 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.160 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.160 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-05-20T16:25:41.160 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-05-20T16:25:41.160 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-05-20T16:25:41.160 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-05-20T16:25:41.160 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-05-20T16:25:41.160 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-05-20T16:25:41.160 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-05-20T16:25:41.160 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:41.161 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.161 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.161 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.161 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.161 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.161 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-05-20T16:25:41.161 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-05-20T16:25:41.161 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.161 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.161 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.161 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:41.161 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.161 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.161 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.161 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.162 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.162 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.162 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.162 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.162 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-05-20T16:25:41.162 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-05-20T16:25:41.162 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.162 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.162 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.162 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-05-20T16:25:41.162 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.162 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.162 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-05-20T16:25:41.163 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-05-20T16:25:41.163 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-05-20T16:25:41.163 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-05-20T16:25:41.163 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-05-20T16:25:41.163 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-05-20T16:25:41.163 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-05-20T16:25:41.163 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:41.163 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.163 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.163 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.163 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.163 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.163 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-05-20T16:25:41.163 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-05-20T16:25:41.164 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.164 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.164 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.164 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:41.164 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.164 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.164 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.164 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.164 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.164 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.164 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.164 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.164 INFO:teuthology.orchestra.run.smithi082.stdout: "retain_interval": { 2024-05-20T16:25:41.164 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "retain_interval", 2024-05-20T16:25:41.165 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2024-05-20T16:25:41.165 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.165 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.165 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "31536000", 2024-05-20T16:25:41.165 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.165 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.165 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.165 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "how long to retain crashes before pruning them", 2024-05-20T16:25:41.165 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.165 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.165 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.165 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.165 INFO:teuthology.orchestra.run.smithi082.stdout: "warn_recent_interval": { 2024-05-20T16:25:41.165 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "warn_recent_interval", 2024-05-20T16:25:41.165 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2024-05-20T16:25:41.165 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.166 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.166 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "1209600", 2024-05-20T16:25:41.166 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.166 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.166 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.166 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "time interval in which to warn about recent crashes", 2024-05-20T16:25:41.166 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.166 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.166 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.166 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-20T16:25:41.166 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-20T16:25:41.166 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.166 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-05-20T16:25:41.166 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "dashboard", 2024-05-20T16:25:41.166 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-05-20T16:25:41.167 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-05-20T16:25:41.167 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-05-20T16:25:41.167 INFO:teuthology.orchestra.run.smithi082.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-05-20T16:25:41.167 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-05-20T16:25:41.167 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-05-20T16:25:41.167 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.167 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.167 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "10", 2024-05-20T16:25:41.167 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.167 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.167 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.167 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.167 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.167 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.167 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.167 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.168 INFO:teuthology.orchestra.run.smithi082.stdout: "ALERTMANAGER_API_HOST": { 2024-05-20T16:25:41.168 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "ALERTMANAGER_API_HOST", 2024-05-20T16:25:41.168 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.168 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.168 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.168 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:41.168 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.168 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.168 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.168 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.168 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.168 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.168 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.168 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.168 INFO:teuthology.orchestra.run.smithi082.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-05-20T16:25:41.168 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-05-20T16:25:41.169 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.169 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.169 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.169 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-05-20T16:25:41.169 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.169 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.169 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.169 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.169 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.169 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.169 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.169 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.169 INFO:teuthology.orchestra.run.smithi082.stdout: "AUDIT_API_ENABLED": { 2024-05-20T16:25:41.169 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "AUDIT_API_ENABLED", 2024-05-20T16:25:41.169 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.169 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.170 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.170 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:41.170 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.170 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.170 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.170 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.170 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.170 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.170 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.170 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.170 INFO:teuthology.orchestra.run.smithi082.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-05-20T16:25:41.170 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-05-20T16:25:41.170 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.170 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.170 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.171 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-05-20T16:25:41.171 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.171 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.171 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.171 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.171 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.171 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.171 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.171 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.171 INFO:teuthology.orchestra.run.smithi082.stdout: "ENABLE_BROWSABLE_API": { 2024-05-20T16:25:41.171 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "ENABLE_BROWSABLE_API", 2024-05-20T16:25:41.171 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.171 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.171 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.171 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-05-20T16:25:41.172 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.172 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.172 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.172 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.172 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.172 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.172 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.172 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.172 INFO:teuthology.orchestra.run.smithi082.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-05-20T16:25:41.172 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-05-20T16:25:41.172 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.172 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.173 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.173 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-05-20T16:25:41.173 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.173 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.173 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.173 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.173 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.173 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.173 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.173 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.173 INFO:teuthology.orchestra.run.smithi082.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-05-20T16:25:41.173 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-05-20T16:25:41.173 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.174 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.174 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.174 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-05-20T16:25:41.174 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.174 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.174 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.174 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.174 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.174 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.174 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.174 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.174 INFO:teuthology.orchestra.run.smithi082.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-05-20T16:25:41.174 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-05-20T16:25:41.174 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.175 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.175 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.175 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-05-20T16:25:41.175 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.175 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.175 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.175 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.175 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.175 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.175 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.175 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.175 INFO:teuthology.orchestra.run.smithi082.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-05-20T16:25:41.175 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-05-20T16:25:41.175 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.176 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.176 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.176 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-05-20T16:25:41.176 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.176 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.176 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.176 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.176 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.176 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.176 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.176 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.176 INFO:teuthology.orchestra.run.smithi082.stdout: "FEATURE_TOGGLE_NFS": { 2024-05-20T16:25:41.176 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-05-20T16:25:41.176 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.176 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.177 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.177 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-05-20T16:25:41.177 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.177 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.177 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.177 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.177 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.177 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.177 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.177 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.177 INFO:teuthology.orchestra.run.smithi082.stdout: "FEATURE_TOGGLE_RBD": { 2024-05-20T16:25:41.177 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-05-20T16:25:41.177 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.177 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.177 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.177 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-05-20T16:25:41.178 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.178 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.178 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.178 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.178 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.178 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.178 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.178 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.178 INFO:teuthology.orchestra.run.smithi082.stdout: "FEATURE_TOGGLE_RGW": { 2024-05-20T16:25:41.178 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-05-20T16:25:41.178 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.178 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.178 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.178 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-05-20T16:25:41.178 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.179 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.179 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.179 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.179 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.179 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.179 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.179 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.179 INFO:teuthology.orchestra.run.smithi082.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-05-20T16:25:41.179 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-05-20T16:25:41.179 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.179 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.179 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.179 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:41.179 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.179 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.179 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.180 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.180 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.180 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.180 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.180 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.180 INFO:teuthology.orchestra.run.smithi082.stdout: "GRAFANA_API_PASSWORD": { 2024-05-20T16:25:41.180 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "GRAFANA_API_PASSWORD", 2024-05-20T16:25:41.180 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.180 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.180 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.180 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "admin", 2024-05-20T16:25:41.180 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.180 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.180 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.181 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.181 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.181 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.181 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.181 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.181 INFO:teuthology.orchestra.run.smithi082.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-05-20T16:25:41.181 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-05-20T16:25:41.181 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.181 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.181 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.181 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-05-20T16:25:41.181 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.181 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.181 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.182 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.182 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.182 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.182 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.182 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.182 INFO:teuthology.orchestra.run.smithi082.stdout: "GRAFANA_API_URL": { 2024-05-20T16:25:41.182 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "GRAFANA_API_URL", 2024-05-20T16:25:41.182 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.182 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.182 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.182 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:41.182 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.182 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.183 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.183 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.183 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.183 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.183 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.183 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.183 INFO:teuthology.orchestra.run.smithi082.stdout: "GRAFANA_API_USERNAME": { 2024-05-20T16:25:41.183 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "GRAFANA_API_USERNAME", 2024-05-20T16:25:41.183 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.183 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.183 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.183 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "admin", 2024-05-20T16:25:41.183 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.183 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.184 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.184 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.184 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.184 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.184 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.184 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.184 INFO:teuthology.orchestra.run.smithi082.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-05-20T16:25:41.184 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-05-20T16:25:41.184 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.184 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.184 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.184 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:41.184 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.184 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.184 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.184 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.185 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.185 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.185 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.185 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.185 INFO:teuthology.orchestra.run.smithi082.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-05-20T16:25:41.185 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-05-20T16:25:41.185 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.185 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.185 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.185 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:41.185 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.185 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.185 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.185 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.186 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.186 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.186 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.186 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.186 INFO:teuthology.orchestra.run.smithi082.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-05-20T16:25:41.186 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-05-20T16:25:41.186 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.186 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.186 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.186 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-05-20T16:25:41.186 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.186 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.186 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.187 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.187 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.187 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.187 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.187 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.187 INFO:teuthology.orchestra.run.smithi082.stdout: "ISSUE_TRACKER_API_KEY": { 2024-05-20T16:25:41.187 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-05-20T16:25:41.187 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.187 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.187 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.187 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:41.187 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.187 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.187 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.187 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.188 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.188 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.188 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.188 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.188 INFO:teuthology.orchestra.run.smithi082.stdout: "PROMETHEUS_API_HOST": { 2024-05-20T16:25:41.188 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "PROMETHEUS_API_HOST", 2024-05-20T16:25:41.188 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.188 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.188 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.188 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:41.188 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.189 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.189 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.189 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.189 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.189 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.189 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.189 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.189 INFO:teuthology.orchestra.run.smithi082.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-05-20T16:25:41.189 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-05-20T16:25:41.189 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.189 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.189 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.189 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-05-20T16:25:41.190 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.190 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.190 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.190 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.190 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.190 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.190 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.190 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.190 INFO:teuthology.orchestra.run.smithi082.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-05-20T16:25:41.190 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-05-20T16:25:41.190 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.190 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.190 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.190 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:41.191 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.191 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.191 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.191 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.191 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.191 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.191 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.191 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.191 INFO:teuthology.orchestra.run.smithi082.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-05-20T16:25:41.191 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-05-20T16:25:41.191 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.191 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.191 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.191 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:41.192 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.192 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.192 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.192 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.192 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.192 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.192 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.192 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.192 INFO:teuthology.orchestra.run.smithi082.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-05-20T16:25:41.192 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-05-20T16:25:41.192 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.192 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.192 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.192 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-05-20T16:25:41.193 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.193 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.193 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.193 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.193 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.193 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.193 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.193 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.193 INFO:teuthology.orchestra.run.smithi082.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-05-20T16:25:41.193 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-05-20T16:25:41.193 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.193 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.193 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.193 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-05-20T16:25:41.193 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.194 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.194 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.194 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.194 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.194 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.194 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.194 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.194 INFO:teuthology.orchestra.run.smithi082.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-05-20T16:25:41.194 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-05-20T16:25:41.194 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.194 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.194 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.194 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:41.194 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.194 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.195 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.195 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.195 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.195 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.195 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.195 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.195 INFO:teuthology.orchestra.run.smithi082.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-05-20T16:25:41.195 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-05-20T16:25:41.195 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.195 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.195 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.195 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:41.195 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.195 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.195 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.195 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.196 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.196 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.196 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.196 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.196 INFO:teuthology.orchestra.run.smithi082.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-05-20T16:25:41.196 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-05-20T16:25:41.196 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.196 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.196 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.196 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:41.196 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.196 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.196 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.196 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.196 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.197 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.197 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.197 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.197 INFO:teuthology.orchestra.run.smithi082.stdout: "PWD_POLICY_ENABLED": { 2024-05-20T16:25:41.197 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "PWD_POLICY_ENABLED", 2024-05-20T16:25:41.197 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.197 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.197 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.197 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-05-20T16:25:41.197 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.197 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.197 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.197 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.197 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.197 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.197 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.198 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.198 INFO:teuthology.orchestra.run.smithi082.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-05-20T16:25:41.198 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-05-20T16:25:41.198 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.198 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.198 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.198 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-05-20T16:25:41.198 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.198 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.198 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.198 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.198 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.198 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.198 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.199 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.199 INFO:teuthology.orchestra.run.smithi082.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-05-20T16:25:41.199 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-05-20T16:25:41.199 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-05-20T16:25:41.199 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.199 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.199 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "10", 2024-05-20T16:25:41.199 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.199 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.199 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.199 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.199 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.199 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.199 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.200 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.200 INFO:teuthology.orchestra.run.smithi082.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-05-20T16:25:41.200 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-05-20T16:25:41.200 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-05-20T16:25:41.200 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.200 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.200 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "8", 2024-05-20T16:25:41.200 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.200 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.200 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.200 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.200 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.200 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.201 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.201 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.201 INFO:teuthology.orchestra.run.smithi082.stdout: "REST_REQUESTS_TIMEOUT": { 2024-05-20T16:25:41.201 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-05-20T16:25:41.201 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-05-20T16:25:41.201 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.201 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.201 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "45", 2024-05-20T16:25:41.201 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.201 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.201 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.201 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.201 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.201 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.202 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.202 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.202 INFO:teuthology.orchestra.run.smithi082.stdout: "RGW_API_ACCESS_KEY": { 2024-05-20T16:25:41.202 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "RGW_API_ACCESS_KEY", 2024-05-20T16:25:41.202 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.202 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.202 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.202 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:41.202 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.202 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.202 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.202 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.202 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.203 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.203 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.203 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.203 INFO:teuthology.orchestra.run.smithi082.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-05-20T16:25:41.203 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-05-20T16:25:41.203 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.203 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.203 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.203 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "admin", 2024-05-20T16:25:41.203 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.203 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.203 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.203 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.204 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.204 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.204 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.204 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.204 INFO:teuthology.orchestra.run.smithi082.stdout: "RGW_API_SECRET_KEY": { 2024-05-20T16:25:41.204 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "RGW_API_SECRET_KEY", 2024-05-20T16:25:41.204 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.204 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.204 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.204 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:41.204 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.204 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.204 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.204 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.205 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.205 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.205 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.205 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.205 INFO:teuthology.orchestra.run.smithi082.stdout: "RGW_API_SSL_VERIFY": { 2024-05-20T16:25:41.205 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "RGW_API_SSL_VERIFY", 2024-05-20T16:25:41.205 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.205 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.205 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.205 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-05-20T16:25:41.205 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.205 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.205 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.205 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.206 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.206 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.206 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.206 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.206 INFO:teuthology.orchestra.run.smithi082.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-05-20T16:25:41.206 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-05-20T16:25:41.206 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-05-20T16:25:41.206 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.206 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.206 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "0", 2024-05-20T16:25:41.206 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.206 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.206 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.206 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.206 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.207 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.207 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.207 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.207 INFO:teuthology.orchestra.run.smithi082.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-05-20T16:25:41.207 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-05-20T16:25:41.207 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-05-20T16:25:41.207 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.207 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.207 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "10", 2024-05-20T16:25:41.207 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.207 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.207 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.207 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.207 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.207 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.208 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.208 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.208 INFO:teuthology.orchestra.run.smithi082.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-05-20T16:25:41.208 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-05-20T16:25:41.208 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-05-20T16:25:41.208 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.208 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.208 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "5", 2024-05-20T16:25:41.208 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.208 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.208 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.208 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.208 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.208 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.208 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.208 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.208 INFO:teuthology.orchestra.run.smithi082.stdout: "cross_origin_url": { 2024-05-20T16:25:41.209 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "cross_origin_url", 2024-05-20T16:25:41.209 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.209 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.209 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.209 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:41.209 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.209 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.209 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.209 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.209 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.209 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.209 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.209 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.209 INFO:teuthology.orchestra.run.smithi082.stdout: "crt_file": { 2024-05-20T16:25:41.209 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "crt_file", 2024-05-20T16:25:41.209 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.210 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.210 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.210 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:41.210 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.210 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.210 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.210 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.210 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.210 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.210 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.210 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.210 INFO:teuthology.orchestra.run.smithi082.stdout: "debug": { 2024-05-20T16:25:41.210 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "debug", 2024-05-20T16:25:41.210 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.210 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.211 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.211 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:41.211 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.211 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.211 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.211 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Enable/disable debug options", 2024-05-20T16:25:41.211 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.211 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.211 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.211 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.211 INFO:teuthology.orchestra.run.smithi082.stdout: "jwt_token_ttl": { 2024-05-20T16:25:41.211 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "jwt_token_ttl", 2024-05-20T16:25:41.211 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-05-20T16:25:41.211 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.211 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.212 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "28800", 2024-05-20T16:25:41.212 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.212 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.212 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.212 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.212 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.212 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.212 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.212 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.212 INFO:teuthology.orchestra.run.smithi082.stdout: "key_file": { 2024-05-20T16:25:41.212 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "key_file", 2024-05-20T16:25:41.212 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.212 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.212 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.212 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:41.213 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.213 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.213 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.213 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.213 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.213 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.213 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.213 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.213 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-05-20T16:25:41.213 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-05-20T16:25:41.213 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.213 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.213 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.213 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:41.213 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.213 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.214 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-05-20T16:25:41.214 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-05-20T16:25:41.214 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-05-20T16:25:41.214 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-05-20T16:25:41.214 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-05-20T16:25:41.214 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-05-20T16:25:41.214 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-05-20T16:25:41.214 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:41.214 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.214 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.214 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.214 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.214 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.214 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-05-20T16:25:41.215 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-05-20T16:25:41.215 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.215 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.215 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.215 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:41.215 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.215 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.215 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.215 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.215 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.215 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.215 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.215 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.215 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-05-20T16:25:41.215 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-05-20T16:25:41.216 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.216 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.216 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.216 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-05-20T16:25:41.216 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.216 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.216 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-05-20T16:25:41.216 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-05-20T16:25:41.216 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-05-20T16:25:41.216 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-05-20T16:25:41.216 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-05-20T16:25:41.216 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-05-20T16:25:41.216 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-05-20T16:25:41.216 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:41.217 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.217 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.217 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.217 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.217 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.217 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-05-20T16:25:41.217 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-05-20T16:25:41.217 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.217 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.217 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.217 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:41.217 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.217 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.217 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.217 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.218 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.218 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.218 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.218 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.218 INFO:teuthology.orchestra.run.smithi082.stdout: "motd": { 2024-05-20T16:25:41.218 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "motd", 2024-05-20T16:25:41.218 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.218 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.218 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.218 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:41.218 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.218 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.218 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.218 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "The message of the day", 2024-05-20T16:25:41.219 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.219 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.219 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.219 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.219 INFO:teuthology.orchestra.run.smithi082.stdout: "redirect_resolve_ip_addr": { 2024-05-20T16:25:41.219 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "redirect_resolve_ip_addr", 2024-05-20T16:25:41.219 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.219 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.219 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.219 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:41.219 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.219 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.219 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.219 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.220 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.220 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.220 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.220 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.220 INFO:teuthology.orchestra.run.smithi082.stdout: "server_addr": { 2024-05-20T16:25:41.220 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "server_addr", 2024-05-20T16:25:41.220 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.220 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.220 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.220 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "::", 2024-05-20T16:25:41.220 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.220 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.220 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.220 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.220 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.221 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.221 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.221 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.221 INFO:teuthology.orchestra.run.smithi082.stdout: "server_port": { 2024-05-20T16:25:41.221 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "server_port", 2024-05-20T16:25:41.221 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-05-20T16:25:41.221 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.221 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.221 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "8080", 2024-05-20T16:25:41.221 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.221 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.221 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.221 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.221 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.222 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.222 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.222 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.222 INFO:teuthology.orchestra.run.smithi082.stdout: "ssl": { 2024-05-20T16:25:41.222 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "ssl", 2024-05-20T16:25:41.222 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.222 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.222 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.222 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-05-20T16:25:41.222 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.222 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.222 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.223 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.223 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.223 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.223 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.223 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.223 INFO:teuthology.orchestra.run.smithi082.stdout: "ssl_server_port": { 2024-05-20T16:25:41.223 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "ssl_server_port", 2024-05-20T16:25:41.223 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-05-20T16:25:41.223 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.223 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.223 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "8443", 2024-05-20T16:25:41.223 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.223 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.223 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.223 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.224 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.224 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.224 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.224 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.224 INFO:teuthology.orchestra.run.smithi082.stdout: "standby_behaviour": { 2024-05-20T16:25:41.224 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "standby_behaviour", 2024-05-20T16:25:41.224 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.224 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.224 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.224 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "redirect", 2024-05-20T16:25:41.224 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.224 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.224 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-05-20T16:25:41.225 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-05-20T16:25:41.225 INFO:teuthology.orchestra.run.smithi082.stdout: "redirect" 2024-05-20T16:25:41.225 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:41.225 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.225 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.225 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.225 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.225 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.225 INFO:teuthology.orchestra.run.smithi082.stdout: "standby_error_status_code": { 2024-05-20T16:25:41.225 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "standby_error_status_code", 2024-05-20T16:25:41.225 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-05-20T16:25:41.225 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.225 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.225 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "500", 2024-05-20T16:25:41.226 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "400", 2024-05-20T16:25:41.226 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "599", 2024-05-20T16:25:41.226 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.226 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.226 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.226 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.226 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.226 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.226 INFO:teuthology.orchestra.run.smithi082.stdout: "url_prefix": { 2024-05-20T16:25:41.226 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "url_prefix", 2024-05-20T16:25:41.226 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.226 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.226 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.226 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:41.227 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.227 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.227 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.227 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.227 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.227 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.227 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.227 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-20T16:25:41.227 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-20T16:25:41.227 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.227 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-05-20T16:25:41.227 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "devicehealth", 2024-05-20T16:25:41.227 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-05-20T16:25:41.227 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-05-20T16:25:41.228 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-05-20T16:25:41.228 INFO:teuthology.orchestra.run.smithi082.stdout: "enable_monitoring": { 2024-05-20T16:25:41.228 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "enable_monitoring", 2024-05-20T16:25:41.228 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.228 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.228 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.228 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-05-20T16:25:41.228 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.228 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.228 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.228 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "monitor device health metrics", 2024-05-20T16:25:41.228 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.228 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.228 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.229 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.229 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-05-20T16:25:41.229 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-05-20T16:25:41.229 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.229 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.229 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.229 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:41.229 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.229 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.229 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-05-20T16:25:41.229 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-05-20T16:25:41.229 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-05-20T16:25:41.229 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-05-20T16:25:41.229 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-05-20T16:25:41.230 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-05-20T16:25:41.230 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-05-20T16:25:41.230 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:41.230 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.230 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.230 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.230 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.230 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.230 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-05-20T16:25:41.230 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-05-20T16:25:41.230 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.230 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.230 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.231 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:41.231 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.231 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.231 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.231 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.231 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.231 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.231 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.231 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.231 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-05-20T16:25:41.231 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-05-20T16:25:41.231 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.231 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.231 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.232 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-05-20T16:25:41.232 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.232 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.232 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-05-20T16:25:41.232 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-05-20T16:25:41.232 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-05-20T16:25:41.232 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-05-20T16:25:41.232 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-05-20T16:25:41.232 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-05-20T16:25:41.232 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-05-20T16:25:41.232 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:41.232 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.232 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.232 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.233 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.233 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.233 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-05-20T16:25:41.233 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-05-20T16:25:41.233 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.233 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.233 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.233 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:41.233 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.233 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.233 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.233 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.233 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.233 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.233 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.233 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.233 INFO:teuthology.orchestra.run.smithi082.stdout: "mark_out_threshold": { 2024-05-20T16:25:41.234 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "mark_out_threshold", 2024-05-20T16:25:41.234 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2024-05-20T16:25:41.234 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.234 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.234 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "2419200", 2024-05-20T16:25:41.234 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.234 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.234 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.234 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-05-20T16:25:41.234 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.234 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.234 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.234 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.234 INFO:teuthology.orchestra.run.smithi082.stdout: "pool_name": { 2024-05-20T16:25:41.235 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "pool_name", 2024-05-20T16:25:41.235 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.235 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.235 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.235 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "device_health_metrics", 2024-05-20T16:25:41.235 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.235 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.235 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.235 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "name of pool in which to store device health metrics", 2024-05-20T16:25:41.235 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.235 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.235 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.235 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.235 INFO:teuthology.orchestra.run.smithi082.stdout: "retention_period": { 2024-05-20T16:25:41.236 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "retention_period", 2024-05-20T16:25:41.236 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2024-05-20T16:25:41.236 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.236 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.236 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "15552000", 2024-05-20T16:25:41.236 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.236 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.236 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.236 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "how long to retain device health metrics", 2024-05-20T16:25:41.236 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.236 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.236 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.236 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.236 INFO:teuthology.orchestra.run.smithi082.stdout: "scrape_frequency": { 2024-05-20T16:25:41.237 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "scrape_frequency", 2024-05-20T16:25:41.237 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2024-05-20T16:25:41.237 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.237 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.237 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "86400", 2024-05-20T16:25:41.237 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.237 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.237 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.237 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "how frequently to scrape device health metrics", 2024-05-20T16:25:41.237 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.237 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.238 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.238 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.238 INFO:teuthology.orchestra.run.smithi082.stdout: "self_heal": { 2024-05-20T16:25:41.238 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "self_heal", 2024-05-20T16:25:41.238 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.238 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.238 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.238 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-05-20T16:25:41.238 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.238 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.238 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.238 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-05-20T16:25:41.238 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.239 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.239 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.239 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.239 INFO:teuthology.orchestra.run.smithi082.stdout: "sleep_interval": { 2024-05-20T16:25:41.239 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "sleep_interval", 2024-05-20T16:25:41.239 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2024-05-20T16:25:41.239 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.239 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.239 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "600", 2024-05-20T16:25:41.239 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.239 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.239 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.239 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "how frequently to wake up and check device health", 2024-05-20T16:25:41.239 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.240 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.240 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.240 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.240 INFO:teuthology.orchestra.run.smithi082.stdout: "warn_threshold": { 2024-05-20T16:25:41.240 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "warn_threshold", 2024-05-20T16:25:41.240 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2024-05-20T16:25:41.240 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.240 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.240 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "7257600", 2024-05-20T16:25:41.240 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.240 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.240 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.240 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-05-20T16:25:41.241 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.241 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.241 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.241 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-20T16:25:41.241 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-20T16:25:41.241 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.241 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-05-20T16:25:41.241 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "diskprediction_local", 2024-05-20T16:25:41.241 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-05-20T16:25:41.241 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-05-20T16:25:41.241 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-05-20T16:25:41.241 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-05-20T16:25:41.241 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-05-20T16:25:41.241 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.241 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.242 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.242 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:41.242 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.242 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.242 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-05-20T16:25:41.242 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-05-20T16:25:41.242 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-05-20T16:25:41.242 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-05-20T16:25:41.242 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-05-20T16:25:41.242 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-05-20T16:25:41.242 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-05-20T16:25:41.242 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:41.242 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.243 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.243 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.243 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.243 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.243 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-05-20T16:25:41.243 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-05-20T16:25:41.243 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.243 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.243 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.243 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:41.243 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.243 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.243 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.243 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.243 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.244 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.244 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.244 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.244 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-05-20T16:25:41.244 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-05-20T16:25:41.244 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.244 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.244 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.244 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-05-20T16:25:41.244 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.244 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.244 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-05-20T16:25:41.244 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-05-20T16:25:41.244 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-05-20T16:25:41.244 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-05-20T16:25:41.245 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-05-20T16:25:41.245 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-05-20T16:25:41.245 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-05-20T16:25:41.245 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:41.245 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.245 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.245 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.245 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.245 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.245 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-05-20T16:25:41.245 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-05-20T16:25:41.245 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.245 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.245 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.245 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:41.245 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.246 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.246 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.246 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.246 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.246 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.246 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.246 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.246 INFO:teuthology.orchestra.run.smithi082.stdout: "predict_interval": { 2024-05-20T16:25:41.246 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "predict_interval", 2024-05-20T16:25:41.246 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.246 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.246 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.246 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "86400", 2024-05-20T16:25:41.246 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.247 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.247 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.247 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.247 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.247 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.247 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.247 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.247 INFO:teuthology.orchestra.run.smithi082.stdout: "predictor_model": { 2024-05-20T16:25:41.247 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "predictor_model", 2024-05-20T16:25:41.247 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.247 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.247 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.247 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "prophetstor", 2024-05-20T16:25:41.248 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.248 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.248 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.248 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.248 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.248 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.248 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.248 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.248 INFO:teuthology.orchestra.run.smithi082.stdout: "sleep_interval": { 2024-05-20T16:25:41.248 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "sleep_interval", 2024-05-20T16:25:41.248 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.248 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.248 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.248 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "600", 2024-05-20T16:25:41.248 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.248 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.249 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.249 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.249 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.249 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.249 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.249 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-20T16:25:41.249 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-20T16:25:41.249 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.249 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-05-20T16:25:41.249 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "influx", 2024-05-20T16:25:41.249 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": false, 2024-05-20T16:25:41.249 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "influxdb python module not found", 2024-05-20T16:25:41.249 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-05-20T16:25:41.249 INFO:teuthology.orchestra.run.smithi082.stdout: "batch_size": { 2024-05-20T16:25:41.249 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "batch_size", 2024-05-20T16:25:41.250 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-05-20T16:25:41.250 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.250 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.250 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "5000", 2024-05-20T16:25:41.250 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.250 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.250 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.250 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-05-20T16:25:41.250 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.250 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.250 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.250 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.250 INFO:teuthology.orchestra.run.smithi082.stdout: "database": { 2024-05-20T16:25:41.250 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "database", 2024-05-20T16:25:41.251 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.251 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.251 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.251 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "ceph", 2024-05-20T16:25:41.251 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.251 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.251 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.251 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-05-20T16:25:41.251 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.251 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.251 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.251 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.251 INFO:teuthology.orchestra.run.smithi082.stdout: "hostname": { 2024-05-20T16:25:41.251 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "hostname", 2024-05-20T16:25:41.252 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.252 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.252 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.252 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:41.252 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.252 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.252 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.252 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "InfluxDB server hostname", 2024-05-20T16:25:41.252 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.252 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.252 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.252 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.252 INFO:teuthology.orchestra.run.smithi082.stdout: "interval": { 2024-05-20T16:25:41.252 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "interval", 2024-05-20T16:25:41.252 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2024-05-20T16:25:41.252 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.253 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.253 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "30", 2024-05-20T16:25:41.253 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "5", 2024-05-20T16:25:41.253 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.253 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.253 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-05-20T16:25:41.253 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.253 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.253 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.253 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.253 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-05-20T16:25:41.253 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-05-20T16:25:41.253 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.253 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.253 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.253 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:41.254 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.254 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.254 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-05-20T16:25:41.254 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-05-20T16:25:41.254 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-05-20T16:25:41.254 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-05-20T16:25:41.254 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-05-20T16:25:41.254 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-05-20T16:25:41.254 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-05-20T16:25:41.254 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:41.254 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.254 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.255 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.255 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.255 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.255 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-05-20T16:25:41.255 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-05-20T16:25:41.255 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.255 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.255 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.255 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:41.255 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.255 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.255 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.255 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.255 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.255 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.256 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.256 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.256 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-05-20T16:25:41.256 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-05-20T16:25:41.256 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.256 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.256 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.256 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-05-20T16:25:41.256 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.256 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.256 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-05-20T16:25:41.256 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-05-20T16:25:41.256 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-05-20T16:25:41.256 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-05-20T16:25:41.256 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-05-20T16:25:41.257 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-05-20T16:25:41.257 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-05-20T16:25:41.257 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:41.257 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.257 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.257 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.257 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.257 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.257 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-05-20T16:25:41.257 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-05-20T16:25:41.257 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.257 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.257 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.257 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:41.257 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.257 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.258 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.258 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.258 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.258 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.258 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.258 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.258 INFO:teuthology.orchestra.run.smithi082.stdout: "password": { 2024-05-20T16:25:41.258 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "password", 2024-05-20T16:25:41.258 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.258 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.258 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.258 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:41.258 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.258 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.259 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.259 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "password of InfluxDB server user", 2024-05-20T16:25:41.259 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.259 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.259 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.259 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.259 INFO:teuthology.orchestra.run.smithi082.stdout: "port": { 2024-05-20T16:25:41.259 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "port", 2024-05-20T16:25:41.259 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-05-20T16:25:41.259 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.259 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.259 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "8086", 2024-05-20T16:25:41.259 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.259 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.259 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.259 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "InfluxDB server port", 2024-05-20T16:25:41.260 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.260 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.260 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.260 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.260 INFO:teuthology.orchestra.run.smithi082.stdout: "ssl": { 2024-05-20T16:25:41.260 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "ssl", 2024-05-20T16:25:41.260 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.260 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.260 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.260 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "false", 2024-05-20T16:25:41.260 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.260 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.260 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.260 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-05-20T16:25:41.260 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.260 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.261 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.261 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.261 INFO:teuthology.orchestra.run.smithi082.stdout: "threads": { 2024-05-20T16:25:41.261 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "threads", 2024-05-20T16:25:41.261 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-05-20T16:25:41.261 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.261 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.261 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "5", 2024-05-20T16:25:41.261 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "1", 2024-05-20T16:25:41.261 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "32", 2024-05-20T16:25:41.261 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.261 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-05-20T16:25:41.261 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.261 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.261 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.261 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.262 INFO:teuthology.orchestra.run.smithi082.stdout: "username": { 2024-05-20T16:25:41.262 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "username", 2024-05-20T16:25:41.262 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.262 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.262 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.262 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:41.262 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.262 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.262 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.262 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "username of InfluxDB server user", 2024-05-20T16:25:41.262 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.262 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.262 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.262 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.263 INFO:teuthology.orchestra.run.smithi082.stdout: "verify_ssl": { 2024-05-20T16:25:41.263 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "verify_ssl", 2024-05-20T16:25:41.263 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.263 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.263 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.263 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "true", 2024-05-20T16:25:41.263 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.263 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.263 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.263 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-05-20T16:25:41.263 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.263 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.263 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.264 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-20T16:25:41.264 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-20T16:25:41.264 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.264 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-05-20T16:25:41.264 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "insights", 2024-05-20T16:25:41.264 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-05-20T16:25:41.264 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-05-20T16:25:41.264 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-05-20T16:25:41.264 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-05-20T16:25:41.264 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-05-20T16:25:41.264 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.264 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.264 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.264 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:41.264 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.265 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.265 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-05-20T16:25:41.265 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-05-20T16:25:41.265 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-05-20T16:25:41.265 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-05-20T16:25:41.265 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-05-20T16:25:41.265 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-05-20T16:25:41.265 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-05-20T16:25:41.265 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:41.265 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.265 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.265 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.265 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.265 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.266 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-05-20T16:25:41.266 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-05-20T16:25:41.266 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.266 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.266 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.266 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:41.266 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.266 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.266 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.266 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.266 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.266 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.266 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.267 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.267 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-05-20T16:25:41.267 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-05-20T16:25:41.267 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.267 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.267 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.267 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-05-20T16:25:41.267 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.267 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.267 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-05-20T16:25:41.267 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-05-20T16:25:41.267 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-05-20T16:25:41.267 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-05-20T16:25:41.267 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-05-20T16:25:41.268 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-05-20T16:25:41.268 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-05-20T16:25:41.268 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:41.268 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.268 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.268 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.268 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.268 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.268 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-05-20T16:25:41.268 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-05-20T16:25:41.268 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.268 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.268 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.268 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:41.268 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.269 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.269 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.269 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.269 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.269 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.269 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.269 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-20T16:25:41.269 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-20T16:25:41.269 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.269 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-05-20T16:25:41.269 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "iostat", 2024-05-20T16:25:41.269 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-05-20T16:25:41.269 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-05-20T16:25:41.269 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-05-20T16:25:41.269 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-05-20T16:25:41.270 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-05-20T16:25:41.270 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.270 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.270 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.270 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:41.270 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.270 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.270 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-05-20T16:25:41.270 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-05-20T16:25:41.270 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-05-20T16:25:41.270 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-05-20T16:25:41.270 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-05-20T16:25:41.270 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-05-20T16:25:41.270 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-05-20T16:25:41.271 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:41.273 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.273 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.274 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.274 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.274 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.274 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-05-20T16:25:41.274 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-05-20T16:25:41.274 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.274 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.274 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.274 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:41.274 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.274 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.274 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.274 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.274 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.275 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.275 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.275 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.275 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-05-20T16:25:41.275 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-05-20T16:25:41.275 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.275 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.275 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.275 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-05-20T16:25:41.275 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.275 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.275 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-05-20T16:25:41.275 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-05-20T16:25:41.275 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-05-20T16:25:41.275 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-05-20T16:25:41.276 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-05-20T16:25:41.276 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-05-20T16:25:41.276 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-05-20T16:25:41.276 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:41.276 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.276 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.276 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.276 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.276 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.276 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-05-20T16:25:41.276 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-05-20T16:25:41.276 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.276 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.276 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.276 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:41.276 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.277 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.277 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.277 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.277 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.277 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.277 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.277 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-20T16:25:41.277 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-20T16:25:41.277 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.277 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-05-20T16:25:41.277 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "k8sevents", 2024-05-20T16:25:41.277 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-05-20T16:25:41.277 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-05-20T16:25:41.277 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-05-20T16:25:41.277 INFO:teuthology.orchestra.run.smithi082.stdout: "ceph_event_retention_days": { 2024-05-20T16:25:41.278 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "ceph_event_retention_days", 2024-05-20T16:25:41.278 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-05-20T16:25:41.278 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.278 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.278 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "7", 2024-05-20T16:25:41.278 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.278 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.278 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.278 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Days to hold ceph event information within local cache", 2024-05-20T16:25:41.278 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.278 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.278 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.278 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.278 INFO:teuthology.orchestra.run.smithi082.stdout: "config_check_secs": { 2024-05-20T16:25:41.279 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "config_check_secs", 2024-05-20T16:25:41.279 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-05-20T16:25:41.279 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.279 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.279 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "10", 2024-05-20T16:25:41.279 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "10", 2024-05-20T16:25:41.279 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.279 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.279 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-05-20T16:25:41.279 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.279 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.279 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.279 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.280 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-05-20T16:25:41.280 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-05-20T16:25:41.280 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.280 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.280 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.280 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:41.280 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.280 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.280 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-05-20T16:25:41.280 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-05-20T16:25:41.280 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-05-20T16:25:41.280 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-05-20T16:25:41.280 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-05-20T16:25:41.280 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-05-20T16:25:41.280 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-05-20T16:25:41.280 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:41.281 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.281 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.281 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.281 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.281 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.281 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-05-20T16:25:41.281 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-05-20T16:25:41.281 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.281 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.281 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.281 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:41.281 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.281 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.281 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.281 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.281 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.282 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.282 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.282 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.282 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-05-20T16:25:41.282 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-05-20T16:25:41.282 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.282 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.282 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.282 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-05-20T16:25:41.282 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.282 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.282 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-05-20T16:25:41.282 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-05-20T16:25:41.282 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-05-20T16:25:41.283 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-05-20T16:25:41.283 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-05-20T16:25:41.283 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-05-20T16:25:41.283 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-05-20T16:25:41.283 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:41.283 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.283 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.283 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.283 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.283 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.283 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-05-20T16:25:41.283 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-05-20T16:25:41.283 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.283 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.283 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.283 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:41.284 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.284 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.284 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.284 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.284 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.284 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.284 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.284 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-20T16:25:41.284 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-20T16:25:41.284 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.284 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-05-20T16:25:41.284 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "localpool", 2024-05-20T16:25:41.284 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-05-20T16:25:41.284 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-05-20T16:25:41.284 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-05-20T16:25:41.285 INFO:teuthology.orchestra.run.smithi082.stdout: "failure_domain": { 2024-05-20T16:25:41.285 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "failure_domain", 2024-05-20T16:25:41.285 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.285 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.285 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.285 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "host", 2024-05-20T16:25:41.285 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.285 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.285 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.285 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "failure domain for any created local pool", 2024-05-20T16:25:41.285 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-05-20T16:25:41.285 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.285 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.285 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.285 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-05-20T16:25:41.285 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-05-20T16:25:41.286 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.286 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.286 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.286 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:41.286 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.286 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.286 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-05-20T16:25:41.286 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-05-20T16:25:41.286 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-05-20T16:25:41.286 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-05-20T16:25:41.286 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-05-20T16:25:41.286 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-05-20T16:25:41.286 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-05-20T16:25:41.286 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:41.287 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.287 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.287 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.287 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.287 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.287 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-05-20T16:25:41.287 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-05-20T16:25:41.287 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.287 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.287 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.287 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:41.287 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.287 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.287 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.287 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.287 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.288 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.288 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.288 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.288 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-05-20T16:25:41.288 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-05-20T16:25:41.288 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.288 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.288 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.288 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-05-20T16:25:41.288 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.288 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.288 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-05-20T16:25:41.288 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-05-20T16:25:41.288 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-05-20T16:25:41.288 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-05-20T16:25:41.288 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-05-20T16:25:41.289 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-05-20T16:25:41.289 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-05-20T16:25:41.289 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:41.289 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.289 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.289 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.289 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.289 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.289 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-05-20T16:25:41.289 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-05-20T16:25:41.289 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.289 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.289 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.289 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:41.289 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.289 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.289 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.290 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.290 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.290 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.290 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.290 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.290 INFO:teuthology.orchestra.run.smithi082.stdout: "min_size": { 2024-05-20T16:25:41.290 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "min_size", 2024-05-20T16:25:41.290 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-05-20T16:25:41.290 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.290 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.290 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:41.290 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.290 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.290 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.290 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "default min_size for any created local pool", 2024-05-20T16:25:41.291 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-05-20T16:25:41.291 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.291 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.291 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.291 INFO:teuthology.orchestra.run.smithi082.stdout: "num_rep": { 2024-05-20T16:25:41.291 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "num_rep", 2024-05-20T16:25:41.291 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-05-20T16:25:41.291 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.291 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.291 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "3", 2024-05-20T16:25:41.291 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.291 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.291 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.291 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "default replica count for any created local pool", 2024-05-20T16:25:41.291 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.292 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.292 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.292 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.292 INFO:teuthology.orchestra.run.smithi082.stdout: "pg_num": { 2024-05-20T16:25:41.292 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "pg_num", 2024-05-20T16:25:41.292 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-05-20T16:25:41.292 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.292 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.292 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "128", 2024-05-20T16:25:41.292 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.292 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.292 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.292 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "default pg_num for any created local pool", 2024-05-20T16:25:41.292 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.293 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.293 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.293 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.293 INFO:teuthology.orchestra.run.smithi082.stdout: "prefix": { 2024-05-20T16:25:41.293 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "prefix", 2024-05-20T16:25:41.293 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.293 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.293 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.293 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:41.293 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.293 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.293 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.293 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "name prefix for any created local pool", 2024-05-20T16:25:41.294 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.294 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.294 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.294 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.294 INFO:teuthology.orchestra.run.smithi082.stdout: "subtree": { 2024-05-20T16:25:41.294 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "subtree", 2024-05-20T16:25:41.294 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.294 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.294 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.294 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "rack", 2024-05-20T16:25:41.294 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.294 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.294 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.294 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "CRUSH level for which to create a local pool", 2024-05-20T16:25:41.295 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-05-20T16:25:41.295 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.295 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.295 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-20T16:25:41.295 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-20T16:25:41.295 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.295 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-05-20T16:25:41.295 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "mds_autoscaler", 2024-05-20T16:25:41.295 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-05-20T16:25:41.295 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-05-20T16:25:41.295 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-05-20T16:25:41.295 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-05-20T16:25:41.295 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-05-20T16:25:41.295 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.296 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.296 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.296 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:41.296 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.296 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.296 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-05-20T16:25:41.296 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-05-20T16:25:41.296 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-05-20T16:25:41.296 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-05-20T16:25:41.296 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-05-20T16:25:41.296 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-05-20T16:25:41.296 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-05-20T16:25:41.296 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:41.296 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.296 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.297 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.297 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.297 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.297 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-05-20T16:25:41.297 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-05-20T16:25:41.297 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.297 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.297 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.297 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:41.297 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.297 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.297 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.297 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.297 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.298 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.298 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.298 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.298 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-05-20T16:25:41.298 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-05-20T16:25:41.298 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.298 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.298 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.298 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-05-20T16:25:41.298 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.298 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.298 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-05-20T16:25:41.298 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-05-20T16:25:41.299 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-05-20T16:25:41.299 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-05-20T16:25:41.299 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-05-20T16:25:41.299 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-05-20T16:25:41.299 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-05-20T16:25:41.299 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:41.299 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.299 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.299 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.299 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.299 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.299 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-05-20T16:25:41.299 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-05-20T16:25:41.300 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.300 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.300 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.300 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:41.300 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.300 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.300 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.300 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.300 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.300 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.300 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.300 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-20T16:25:41.300 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-20T16:25:41.300 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.300 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-05-20T16:25:41.300 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "mirroring", 2024-05-20T16:25:41.301 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-05-20T16:25:41.301 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-05-20T16:25:41.301 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-05-20T16:25:41.301 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-05-20T16:25:41.301 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-05-20T16:25:41.301 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.301 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.301 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.301 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:41.301 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.301 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.301 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-05-20T16:25:41.301 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-05-20T16:25:41.301 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-05-20T16:25:41.301 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-05-20T16:25:41.301 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-05-20T16:25:41.302 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-05-20T16:25:41.302 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-05-20T16:25:41.302 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:41.302 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.302 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.302 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.302 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.302 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.302 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-05-20T16:25:41.302 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-05-20T16:25:41.302 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.302 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.302 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.302 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:41.303 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.303 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.303 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.303 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.303 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.303 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.303 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.303 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.303 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-05-20T16:25:41.303 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-05-20T16:25:41.303 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.303 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.303 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.303 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-05-20T16:25:41.303 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.303 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.304 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-05-20T16:25:41.304 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-05-20T16:25:41.304 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-05-20T16:25:41.304 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-05-20T16:25:41.304 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-05-20T16:25:41.304 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-05-20T16:25:41.304 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-05-20T16:25:41.304 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:41.304 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.304 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.304 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.304 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.304 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.304 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-05-20T16:25:41.304 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-05-20T16:25:41.304 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.305 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.305 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.305 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:41.305 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.305 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.305 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.305 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.305 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.305 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.305 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.305 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-20T16:25:41.305 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-20T16:25:41.305 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.305 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-05-20T16:25:41.306 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "nfs", 2024-05-20T16:25:41.306 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-05-20T16:25:41.306 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-05-20T16:25:41.306 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-05-20T16:25:41.306 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-05-20T16:25:41.306 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-05-20T16:25:41.306 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.306 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.306 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.306 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:41.306 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.306 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.306 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-05-20T16:25:41.306 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-05-20T16:25:41.306 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-05-20T16:25:41.307 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-05-20T16:25:41.307 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-05-20T16:25:41.307 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-05-20T16:25:41.307 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-05-20T16:25:41.307 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:41.307 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.307 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.307 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.307 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.307 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.307 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-05-20T16:25:41.307 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-05-20T16:25:41.307 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.307 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.307 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.307 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:41.308 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.308 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.308 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.308 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.308 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.308 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.308 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.308 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.308 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-05-20T16:25:41.308 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-05-20T16:25:41.308 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.308 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.308 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.308 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-05-20T16:25:41.308 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.308 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.309 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-05-20T16:25:41.309 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-05-20T16:25:41.309 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-05-20T16:25:41.309 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-05-20T16:25:41.309 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-05-20T16:25:41.309 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-05-20T16:25:41.309 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-05-20T16:25:41.309 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:41.309 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.309 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.309 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.309 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.309 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.309 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-05-20T16:25:41.309 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-05-20T16:25:41.309 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.310 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.310 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.310 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:41.310 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.310 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.310 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.310 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.310 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.310 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.310 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.310 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-20T16:25:41.310 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-20T16:25:41.310 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.310 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-05-20T16:25:41.311 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "orchestrator", 2024-05-20T16:25:41.311 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-05-20T16:25:41.311 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-05-20T16:25:41.311 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-05-20T16:25:41.311 INFO:teuthology.orchestra.run.smithi082.stdout: "fail_fs": { 2024-05-20T16:25:41.311 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "fail_fs", 2024-05-20T16:25:41.311 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.311 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.311 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.311 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:41.311 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.311 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.311 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.311 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-05-20T16:25:41.311 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.311 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.312 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.312 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.312 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-05-20T16:25:41.312 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-05-20T16:25:41.312 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.312 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.312 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.312 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:41.312 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.312 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.312 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-05-20T16:25:41.312 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-05-20T16:25:41.312 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-05-20T16:25:41.312 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-05-20T16:25:41.312 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-05-20T16:25:41.312 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-05-20T16:25:41.313 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-05-20T16:25:41.313 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:41.313 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.313 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.313 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.313 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.313 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.313 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-05-20T16:25:41.313 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-05-20T16:25:41.313 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.313 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.313 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.313 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:41.313 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.313 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.314 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.314 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.314 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.314 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.314 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.314 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.314 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-05-20T16:25:41.314 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-05-20T16:25:41.314 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.314 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.314 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.314 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-05-20T16:25:41.314 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.314 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.315 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-05-20T16:25:41.315 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-05-20T16:25:41.315 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-05-20T16:25:41.315 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-05-20T16:25:41.315 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-05-20T16:25:41.315 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-05-20T16:25:41.315 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-05-20T16:25:41.315 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:41.315 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.315 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.315 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.315 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.315 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.315 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-05-20T16:25:41.316 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-05-20T16:25:41.316 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.316 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.316 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.316 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:41.316 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.316 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.316 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.316 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.316 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.316 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.316 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.316 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.316 INFO:teuthology.orchestra.run.smithi082.stdout: "orchestrator": { 2024-05-20T16:25:41.316 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "orchestrator", 2024-05-20T16:25:41.317 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.317 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.317 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.317 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:41.317 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.317 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.317 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-05-20T16:25:41.317 INFO:teuthology.orchestra.run.smithi082.stdout: "cephadm", 2024-05-20T16:25:41.317 INFO:teuthology.orchestra.run.smithi082.stdout: "rook", 2024-05-20T16:25:41.317 INFO:teuthology.orchestra.run.smithi082.stdout: "test_orchestrator" 2024-05-20T16:25:41.317 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:41.317 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Orchestrator backend", 2024-05-20T16:25:41.317 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.317 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.318 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.318 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-20T16:25:41.318 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-20T16:25:41.318 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.318 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-05-20T16:25:41.318 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "osd_perf_query", 2024-05-20T16:25:41.318 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-05-20T16:25:41.318 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-05-20T16:25:41.318 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-05-20T16:25:41.318 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-05-20T16:25:41.318 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-05-20T16:25:41.318 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.318 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.319 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.319 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:41.319 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.319 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.319 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-05-20T16:25:41.319 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-05-20T16:25:41.319 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-05-20T16:25:41.319 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-05-20T16:25:41.319 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-05-20T16:25:41.319 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-05-20T16:25:41.319 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-05-20T16:25:41.319 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:41.319 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.319 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.319 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.320 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.320 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.320 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-05-20T16:25:41.320 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-05-20T16:25:41.320 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.320 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.320 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.320 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:41.320 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.320 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.320 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.320 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.320 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.320 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.321 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.321 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.321 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-05-20T16:25:41.321 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-05-20T16:25:41.321 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.321 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.321 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.321 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-05-20T16:25:41.321 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.321 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.321 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-05-20T16:25:41.321 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-05-20T16:25:41.321 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-05-20T16:25:41.321 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-05-20T16:25:41.321 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-05-20T16:25:41.322 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-05-20T16:25:41.322 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-05-20T16:25:41.322 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:41.322 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.322 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.322 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.322 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.322 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.322 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-05-20T16:25:41.322 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-05-20T16:25:41.322 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.322 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.322 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.322 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:41.322 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.323 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.323 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.323 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.323 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.323 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.323 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.323 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-20T16:25:41.323 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-20T16:25:41.323 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.323 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-05-20T16:25:41.323 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "osd_support", 2024-05-20T16:25:41.323 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-05-20T16:25:41.323 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-05-20T16:25:41.324 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-05-20T16:25:41.324 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-05-20T16:25:41.324 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-05-20T16:25:41.324 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.324 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.324 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.324 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:41.324 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.324 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.324 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-05-20T16:25:41.324 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-05-20T16:25:41.324 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-05-20T16:25:41.324 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-05-20T16:25:41.324 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-05-20T16:25:41.325 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-05-20T16:25:41.325 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-05-20T16:25:41.325 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:41.325 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.325 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.325 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.325 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.325 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.325 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-05-20T16:25:41.325 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-05-20T16:25:41.325 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.325 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.325 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.326 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:41.326 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.326 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.326 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.326 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.326 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.326 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.326 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.326 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.326 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-05-20T16:25:41.326 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-05-20T16:25:41.326 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.326 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.327 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.327 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-05-20T16:25:41.327 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.327 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.327 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-05-20T16:25:41.327 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-05-20T16:25:41.327 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-05-20T16:25:41.327 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-05-20T16:25:41.327 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-05-20T16:25:41.327 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-05-20T16:25:41.327 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-05-20T16:25:41.327 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:41.327 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.327 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.328 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.328 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.328 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.328 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-05-20T16:25:41.328 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-05-20T16:25:41.328 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.328 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.328 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.328 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:41.328 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.328 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.328 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.328 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.328 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.328 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.329 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.329 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-20T16:25:41.329 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-20T16:25:41.329 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.329 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-05-20T16:25:41.329 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "pg_autoscaler", 2024-05-20T16:25:41.329 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-05-20T16:25:41.329 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-05-20T16:25:41.329 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-05-20T16:25:41.329 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-05-20T16:25:41.329 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-05-20T16:25:41.329 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.329 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.329 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.329 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:41.330 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.330 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.330 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-05-20T16:25:41.330 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-05-20T16:25:41.330 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-05-20T16:25:41.330 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-05-20T16:25:41.330 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-05-20T16:25:41.330 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-05-20T16:25:41.330 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-05-20T16:25:41.330 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:41.330 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.330 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.330 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.330 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.331 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.331 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-05-20T16:25:41.331 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-05-20T16:25:41.331 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.331 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.331 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.331 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:41.331 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.331 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.331 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.331 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.331 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.331 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.331 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.332 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.332 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-05-20T16:25:41.332 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-05-20T16:25:41.332 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.332 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.332 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.332 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-05-20T16:25:41.332 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.332 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.332 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-05-20T16:25:41.332 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-05-20T16:25:41.332 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-05-20T16:25:41.332 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-05-20T16:25:41.333 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-05-20T16:25:41.333 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-05-20T16:25:41.333 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-05-20T16:25:41.333 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:41.333 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.333 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.333 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.333 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.333 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.333 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-05-20T16:25:41.333 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-05-20T16:25:41.333 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.333 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.333 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.334 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:41.334 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.334 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.334 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.334 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.334 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.334 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.334 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.334 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.334 INFO:teuthology.orchestra.run.smithi082.stdout: "sleep_interval": { 2024-05-20T16:25:41.334 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "sleep_interval", 2024-05-20T16:25:41.334 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2024-05-20T16:25:41.334 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.334 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.335 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "60", 2024-05-20T16:25:41.335 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.335 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.335 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.335 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.335 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.335 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.335 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.335 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.335 INFO:teuthology.orchestra.run.smithi082.stdout: "threshold": { 2024-05-20T16:25:41.335 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "threshold", 2024-05-20T16:25:41.335 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "float", 2024-05-20T16:25:41.335 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.335 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.335 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "3.0", 2024-05-20T16:25:41.336 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "1.0", 2024-05-20T16:25:41.336 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.336 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.336 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "scaling threshold", 2024-05-20T16:25:41.336 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-05-20T16:25:41.336 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.336 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.336 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-20T16:25:41.336 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-20T16:25:41.336 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.336 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-05-20T16:25:41.336 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "progress", 2024-05-20T16:25:41.336 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-05-20T16:25:41.336 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-05-20T16:25:41.337 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-05-20T16:25:41.337 INFO:teuthology.orchestra.run.smithi082.stdout: "allow_pg_recovery_event": { 2024-05-20T16:25:41.337 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "allow_pg_recovery_event", 2024-05-20T16:25:41.337 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.337 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.337 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.337 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:41.337 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.337 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.337 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.337 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "allow the module to show pg recovery progress", 2024-05-20T16:25:41.337 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.337 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.337 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.338 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.338 INFO:teuthology.orchestra.run.smithi082.stdout: "enabled": { 2024-05-20T16:25:41.338 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "enabled", 2024-05-20T16:25:41.338 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.338 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.338 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.338 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-05-20T16:25:41.338 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.338 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.338 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.338 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.338 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.338 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.339 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.339 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.339 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-05-20T16:25:41.339 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-05-20T16:25:41.339 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.339 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.339 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.339 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:41.339 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.339 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.339 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-05-20T16:25:41.339 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-05-20T16:25:41.339 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-05-20T16:25:41.339 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-05-20T16:25:41.340 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-05-20T16:25:41.340 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-05-20T16:25:41.340 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-05-20T16:25:41.340 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:41.340 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.340 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.340 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.340 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.340 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.340 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-05-20T16:25:41.340 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-05-20T16:25:41.340 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.340 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.340 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.340 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:41.341 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.341 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.341 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.341 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.341 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.341 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.341 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.341 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.341 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-05-20T16:25:41.341 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-05-20T16:25:41.341 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.341 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.341 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.341 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-05-20T16:25:41.342 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.342 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.342 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-05-20T16:25:41.342 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-05-20T16:25:41.342 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-05-20T16:25:41.342 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-05-20T16:25:41.342 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-05-20T16:25:41.342 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-05-20T16:25:41.342 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-05-20T16:25:41.342 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:41.342 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.342 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.342 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.343 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.343 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.343 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-05-20T16:25:41.343 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-05-20T16:25:41.343 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.343 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.343 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.343 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:41.343 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.343 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.343 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.343 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.343 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.343 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.343 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.344 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.344 INFO:teuthology.orchestra.run.smithi082.stdout: "max_completed_events": { 2024-05-20T16:25:41.344 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "max_completed_events", 2024-05-20T16:25:41.344 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-05-20T16:25:41.344 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.344 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.344 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "50", 2024-05-20T16:25:41.344 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.344 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.344 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.344 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "number of past completed events to remember", 2024-05-20T16:25:41.344 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.344 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.344 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.345 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.345 INFO:teuthology.orchestra.run.smithi082.stdout: "sleep_interval": { 2024-05-20T16:25:41.345 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "sleep_interval", 2024-05-20T16:25:41.345 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2024-05-20T16:25:41.345 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.345 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.345 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "5", 2024-05-20T16:25:41.345 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.345 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.345 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.345 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "how long the module is going to sleep", 2024-05-20T16:25:41.345 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.345 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.346 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.346 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-20T16:25:41.346 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-20T16:25:41.346 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.346 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-05-20T16:25:41.346 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "prometheus", 2024-05-20T16:25:41.346 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-05-20T16:25:41.346 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-05-20T16:25:41.346 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-05-20T16:25:41.346 INFO:teuthology.orchestra.run.smithi082.stdout: "cache": { 2024-05-20T16:25:41.346 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "cache", 2024-05-20T16:25:41.346 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.346 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.347 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.347 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-05-20T16:25:41.347 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.347 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.347 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.347 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.347 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.347 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.347 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.347 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.347 INFO:teuthology.orchestra.run.smithi082.stdout: "exclude_perf_counters": { 2024-05-20T16:25:41.347 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "exclude_perf_counters", 2024-05-20T16:25:41.347 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.347 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.348 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.348 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-05-20T16:25:41.348 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.348 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.348 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.348 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-05-20T16:25:41.348 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-05-20T16:25:41.348 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.348 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.348 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.348 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-05-20T16:25:41.348 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-05-20T16:25:41.348 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.349 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.349 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.349 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:41.349 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.349 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.349 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-05-20T16:25:41.349 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-05-20T16:25:41.349 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-05-20T16:25:41.349 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-05-20T16:25:41.349 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-05-20T16:25:41.349 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-05-20T16:25:41.349 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-05-20T16:25:41.349 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:41.349 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.350 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.350 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.350 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.350 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.350 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-05-20T16:25:41.350 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-05-20T16:25:41.350 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.350 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.350 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.350 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:41.350 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.350 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.350 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.350 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.351 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.351 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.351 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.351 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.351 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-05-20T16:25:41.351 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-05-20T16:25:41.351 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.351 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.351 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.351 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-05-20T16:25:41.351 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.351 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.351 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-05-20T16:25:41.351 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-05-20T16:25:41.352 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-05-20T16:25:41.352 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-05-20T16:25:41.352 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-05-20T16:25:41.352 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-05-20T16:25:41.352 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-05-20T16:25:41.352 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:41.352 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.352 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.352 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.352 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.352 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.352 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-05-20T16:25:41.352 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-05-20T16:25:41.352 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.352 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.353 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.353 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:41.353 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.353 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.353 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.353 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.353 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.353 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.353 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.353 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.353 INFO:teuthology.orchestra.run.smithi082.stdout: "rbd_stats_pools": { 2024-05-20T16:25:41.353 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "rbd_stats_pools", 2024-05-20T16:25:41.353 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.353 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.354 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.354 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:41.354 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.354 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.354 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.354 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.354 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.354 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.354 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.354 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.354 INFO:teuthology.orchestra.run.smithi082.stdout: "rbd_stats_pools_refresh_interval": { 2024-05-20T16:25:41.354 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-05-20T16:25:41.354 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-05-20T16:25:41.354 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.355 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.355 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "300", 2024-05-20T16:25:41.355 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.355 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.355 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.355 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.355 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.355 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.355 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.355 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.355 INFO:teuthology.orchestra.run.smithi082.stdout: "scrape_interval": { 2024-05-20T16:25:41.355 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "scrape_interval", 2024-05-20T16:25:41.355 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "float", 2024-05-20T16:25:41.355 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.356 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.356 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "15.0", 2024-05-20T16:25:41.356 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.356 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.356 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.356 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.356 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.356 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.356 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.356 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.356 INFO:teuthology.orchestra.run.smithi082.stdout: "server_addr": { 2024-05-20T16:25:41.356 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "server_addr", 2024-05-20T16:25:41.356 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.356 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.356 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.357 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "::", 2024-05-20T16:25:41.357 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.357 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.357 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.357 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-05-20T16:25:41.357 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.357 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.357 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.357 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.357 INFO:teuthology.orchestra.run.smithi082.stdout: "server_port": { 2024-05-20T16:25:41.357 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "server_port", 2024-05-20T16:25:41.357 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-05-20T16:25:41.357 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.357 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.357 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "9283", 2024-05-20T16:25:41.358 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.358 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.358 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.358 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-05-20T16:25:41.358 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.358 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.358 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.358 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.358 INFO:teuthology.orchestra.run.smithi082.stdout: "stale_cache_strategy": { 2024-05-20T16:25:41.358 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "stale_cache_strategy", 2024-05-20T16:25:41.358 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.358 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.358 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.359 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "log", 2024-05-20T16:25:41.359 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.359 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.359 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.359 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.359 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.359 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.359 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.359 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.359 INFO:teuthology.orchestra.run.smithi082.stdout: "standby_behaviour": { 2024-05-20T16:25:41.359 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "standby_behaviour", 2024-05-20T16:25:41.359 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.359 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.360 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.360 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "default", 2024-05-20T16:25:41.360 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.360 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.360 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-05-20T16:25:41.360 INFO:teuthology.orchestra.run.smithi082.stdout: "default", 2024-05-20T16:25:41.360 INFO:teuthology.orchestra.run.smithi082.stdout: "error" 2024-05-20T16:25:41.360 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:41.360 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.360 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.360 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.360 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.360 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.360 INFO:teuthology.orchestra.run.smithi082.stdout: "standby_error_status_code": { 2024-05-20T16:25:41.361 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "standby_error_status_code", 2024-05-20T16:25:41.361 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-05-20T16:25:41.361 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.361 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.361 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "500", 2024-05-20T16:25:41.361 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "400", 2024-05-20T16:25:41.361 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "599", 2024-05-20T16:25:41.361 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.361 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.361 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.361 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.361 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.361 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-20T16:25:41.361 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-20T16:25:41.362 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.362 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-05-20T16:25:41.362 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "rbd_support", 2024-05-20T16:25:41.362 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-05-20T16:25:41.362 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-05-20T16:25:41.362 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-05-20T16:25:41.362 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-05-20T16:25:41.362 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-05-20T16:25:41.362 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.362 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.362 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.362 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:41.362 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.362 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.363 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-05-20T16:25:41.363 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-05-20T16:25:41.363 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-05-20T16:25:41.363 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-05-20T16:25:41.363 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-05-20T16:25:41.363 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-05-20T16:25:41.363 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-05-20T16:25:41.363 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:41.363 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.363 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.363 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.363 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.363 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.364 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-05-20T16:25:41.364 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-05-20T16:25:41.364 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.364 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.364 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.364 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:41.364 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.364 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.364 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.364 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.364 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.364 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.364 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.364 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.365 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-05-20T16:25:41.365 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-05-20T16:25:41.365 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.365 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.365 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.365 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-05-20T16:25:41.365 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.365 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.365 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-05-20T16:25:41.365 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-05-20T16:25:41.365 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-05-20T16:25:41.365 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-05-20T16:25:41.365 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-05-20T16:25:41.365 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-05-20T16:25:41.366 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-05-20T16:25:41.366 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:41.366 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.366 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.366 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.366 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.366 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.366 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-05-20T16:25:41.366 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-05-20T16:25:41.366 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.366 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.366 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.366 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:41.366 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.367 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.367 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.367 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.367 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.367 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.367 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.367 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.367 INFO:teuthology.orchestra.run.smithi082.stdout: "max_concurrent_snap_create": { 2024-05-20T16:25:41.367 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "max_concurrent_snap_create", 2024-05-20T16:25:41.367 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-05-20T16:25:41.367 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.367 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.367 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "10", 2024-05-20T16:25:41.367 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.367 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.367 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.368 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.368 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.368 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.368 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.368 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.368 INFO:teuthology.orchestra.run.smithi082.stdout: "mirror_snapshot_schedule": { 2024-05-20T16:25:41.368 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "mirror_snapshot_schedule", 2024-05-20T16:25:41.368 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.368 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.368 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.368 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:41.368 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.368 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.368 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.368 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.369 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.369 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.369 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.369 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.369 INFO:teuthology.orchestra.run.smithi082.stdout: "trash_purge_schedule": { 2024-05-20T16:25:41.369 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "trash_purge_schedule", 2024-05-20T16:25:41.369 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.369 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.369 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.369 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:41.369 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.369 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.369 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.369 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.369 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.370 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.370 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.370 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-20T16:25:41.370 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-20T16:25:41.370 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.370 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-05-20T16:25:41.370 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "restful", 2024-05-20T16:25:41.370 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-05-20T16:25:41.370 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-05-20T16:25:41.370 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-05-20T16:25:41.370 INFO:teuthology.orchestra.run.smithi082.stdout: "enable_auth": { 2024-05-20T16:25:41.370 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "enable_auth", 2024-05-20T16:25:41.370 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.370 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.370 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.371 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-05-20T16:25:41.371 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.371 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.371 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.371 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.371 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.371 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.371 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.371 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.371 INFO:teuthology.orchestra.run.smithi082.stdout: "key_file": { 2024-05-20T16:25:41.371 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "key_file", 2024-05-20T16:25:41.371 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.371 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.371 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.372 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:41.372 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.372 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.372 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.372 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.372 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.372 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.372 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.372 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.372 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-05-20T16:25:41.373 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-05-20T16:25:41.373 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.373 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.373 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.373 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:41.373 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.373 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.373 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-05-20T16:25:41.373 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-05-20T16:25:41.373 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-05-20T16:25:41.373 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-05-20T16:25:41.373 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-05-20T16:25:41.373 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-05-20T16:25:41.374 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-05-20T16:25:41.374 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:41.374 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.374 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.374 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.374 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.374 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.374 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-05-20T16:25:41.374 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-05-20T16:25:41.374 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.374 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.374 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.375 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:41.375 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.375 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.375 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.375 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.375 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.375 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.375 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.375 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.375 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-05-20T16:25:41.375 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-05-20T16:25:41.375 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.375 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.375 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.376 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-05-20T16:25:41.376 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.376 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.376 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-05-20T16:25:41.376 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-05-20T16:25:41.376 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-05-20T16:25:41.376 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-05-20T16:25:41.376 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-05-20T16:25:41.376 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-05-20T16:25:41.376 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-05-20T16:25:41.376 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:41.376 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.376 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.376 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.376 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.377 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.377 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-05-20T16:25:41.377 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-05-20T16:25:41.377 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.377 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.377 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.377 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:41.377 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.377 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.377 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.377 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.377 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.377 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.377 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.378 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.378 INFO:teuthology.orchestra.run.smithi082.stdout: "server_addr": { 2024-05-20T16:25:41.378 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "server_addr", 2024-05-20T16:25:41.378 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.378 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.378 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.378 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:41.378 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.378 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.378 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.378 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.378 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.378 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.378 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.379 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.379 INFO:teuthology.orchestra.run.smithi082.stdout: "server_port": { 2024-05-20T16:25:41.379 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "server_port", 2024-05-20T16:25:41.379 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.379 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.379 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.379 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:41.379 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.379 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.379 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.379 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.379 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.379 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.379 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.380 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-20T16:25:41.380 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-20T16:25:41.380 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.380 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-05-20T16:25:41.380 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "rgw", 2024-05-20T16:25:41.380 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-05-20T16:25:41.380 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-05-20T16:25:41.380 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-05-20T16:25:41.380 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-05-20T16:25:41.380 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-05-20T16:25:41.380 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.380 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.380 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.380 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:41.381 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.381 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.381 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-05-20T16:25:41.381 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-05-20T16:25:41.381 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-05-20T16:25:41.381 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-05-20T16:25:41.381 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-05-20T16:25:41.381 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-05-20T16:25:41.381 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-05-20T16:25:41.381 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:41.381 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.381 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.381 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.381 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.382 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.382 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-05-20T16:25:41.382 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-05-20T16:25:41.382 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.382 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.382 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.382 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:41.382 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.382 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.382 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.382 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.382 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.382 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.383 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.383 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.383 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-05-20T16:25:41.383 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-05-20T16:25:41.383 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.383 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.383 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.383 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-05-20T16:25:41.383 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.383 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.383 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-05-20T16:25:41.383 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-05-20T16:25:41.383 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-05-20T16:25:41.383 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-05-20T16:25:41.384 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-05-20T16:25:41.384 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-05-20T16:25:41.384 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-05-20T16:25:41.384 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:41.384 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.384 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.384 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.384 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.384 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.384 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-05-20T16:25:41.384 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-05-20T16:25:41.384 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.384 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.384 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.385 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:41.385 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.385 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.385 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.385 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.385 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.385 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.385 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.385 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-20T16:25:41.385 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-20T16:25:41.385 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.385 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-05-20T16:25:41.385 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "rook", 2024-05-20T16:25:41.386 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-05-20T16:25:41.386 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-05-20T16:25:41.386 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-05-20T16:25:41.386 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-05-20T16:25:41.386 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-05-20T16:25:41.386 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.386 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.386 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.386 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:41.386 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.386 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.386 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-05-20T16:25:41.386 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-05-20T16:25:41.387 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-05-20T16:25:41.387 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-05-20T16:25:41.387 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-05-20T16:25:41.387 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-05-20T16:25:41.387 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-05-20T16:25:41.387 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:41.387 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.387 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.387 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.387 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.387 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.387 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-05-20T16:25:41.388 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-05-20T16:25:41.388 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.388 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.388 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.388 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:41.388 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.388 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.388 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.388 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.388 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.388 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.388 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.388 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.389 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-05-20T16:25:41.389 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-05-20T16:25:41.389 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.389 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.389 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.389 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-05-20T16:25:41.389 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.389 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.389 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-05-20T16:25:41.389 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-05-20T16:25:41.389 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-05-20T16:25:41.389 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-05-20T16:25:41.389 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-05-20T16:25:41.390 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-05-20T16:25:41.390 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-05-20T16:25:41.390 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:41.390 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.390 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.390 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.390 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.390 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.390 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-05-20T16:25:41.390 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-05-20T16:25:41.390 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.390 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.390 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.391 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:41.391 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.391 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.391 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.391 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.391 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.391 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.391 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.391 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.391 INFO:teuthology.orchestra.run.smithi082.stdout: "storage_class": { 2024-05-20T16:25:41.391 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "storage_class", 2024-05-20T16:25:41.391 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.391 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.392 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.392 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "local", 2024-05-20T16:25:41.392 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.392 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.392 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.392 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-05-20T16:25:41.392 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.392 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.392 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.392 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-20T16:25:41.392 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-20T16:25:41.392 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.392 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-05-20T16:25:41.392 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "selftest", 2024-05-20T16:25:41.393 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-05-20T16:25:41.393 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-05-20T16:25:41.393 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-05-20T16:25:41.393 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-05-20T16:25:41.393 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-05-20T16:25:41.393 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.393 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.393 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.393 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:41.393 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.393 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.393 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-05-20T16:25:41.393 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-05-20T16:25:41.394 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-05-20T16:25:41.394 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-05-20T16:25:41.394 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-05-20T16:25:41.394 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-05-20T16:25:41.394 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-05-20T16:25:41.394 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:41.394 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.394 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.394 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.394 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.394 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.394 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-05-20T16:25:41.394 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-05-20T16:25:41.395 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.395 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.395 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.395 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:41.395 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.395 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.395 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.395 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.395 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.395 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.395 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.395 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.395 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-05-20T16:25:41.395 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-05-20T16:25:41.396 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.396 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.396 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.396 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-05-20T16:25:41.396 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.396 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.396 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-05-20T16:25:41.396 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-05-20T16:25:41.396 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-05-20T16:25:41.396 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-05-20T16:25:41.396 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-05-20T16:25:41.396 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-05-20T16:25:41.396 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-05-20T16:25:41.396 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:41.396 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.397 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.397 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.397 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.397 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.397 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-05-20T16:25:41.397 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-05-20T16:25:41.397 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.397 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.397 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.397 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:41.397 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.397 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.397 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.397 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.398 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.398 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.398 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.398 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.398 INFO:teuthology.orchestra.run.smithi082.stdout: "roption1": { 2024-05-20T16:25:41.398 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "roption1", 2024-05-20T16:25:41.398 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.398 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.398 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.398 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:41.398 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.398 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.398 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.398 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.398 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.399 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.399 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.399 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.399 INFO:teuthology.orchestra.run.smithi082.stdout: "roption2": { 2024-05-20T16:25:41.399 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "roption2", 2024-05-20T16:25:41.399 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.399 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.399 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.399 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "xyz", 2024-05-20T16:25:41.399 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.399 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.399 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.399 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.399 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.399 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.399 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.400 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.400 INFO:teuthology.orchestra.run.smithi082.stdout: "rwoption1": { 2024-05-20T16:25:41.400 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "rwoption1", 2024-05-20T16:25:41.400 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.400 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.400 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.400 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:41.400 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.400 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.400 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.400 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.400 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.400 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.400 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.400 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.401 INFO:teuthology.orchestra.run.smithi082.stdout: "rwoption2": { 2024-05-20T16:25:41.401 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "rwoption2", 2024-05-20T16:25:41.401 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-05-20T16:25:41.401 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.401 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.401 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:41.401 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.401 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.401 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.401 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.401 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.401 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.401 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.401 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.402 INFO:teuthology.orchestra.run.smithi082.stdout: "rwoption3": { 2024-05-20T16:25:41.402 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "rwoption3", 2024-05-20T16:25:41.402 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "float", 2024-05-20T16:25:41.402 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.402 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.402 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:41.402 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.402 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.402 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.402 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.402 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.402 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.402 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.402 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.403 INFO:teuthology.orchestra.run.smithi082.stdout: "rwoption4": { 2024-05-20T16:25:41.403 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "rwoption4", 2024-05-20T16:25:41.403 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.403 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.403 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.403 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:41.403 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.403 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.403 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.403 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.403 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.403 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.403 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.403 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.403 INFO:teuthology.orchestra.run.smithi082.stdout: "rwoption5": { 2024-05-20T16:25:41.404 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "rwoption5", 2024-05-20T16:25:41.404 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.404 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.404 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.404 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:41.404 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.404 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.404 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.404 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.404 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.404 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.404 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.404 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.404 INFO:teuthology.orchestra.run.smithi082.stdout: "rwoption6": { 2024-05-20T16:25:41.404 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "rwoption6", 2024-05-20T16:25:41.404 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.405 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.405 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.405 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-05-20T16:25:41.405 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.405 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.405 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.405 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.405 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.405 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.405 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.405 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.405 INFO:teuthology.orchestra.run.smithi082.stdout: "rwoption7": { 2024-05-20T16:25:41.405 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "rwoption7", 2024-05-20T16:25:41.405 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-05-20T16:25:41.405 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.406 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.406 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:41.406 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "1", 2024-05-20T16:25:41.406 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "42", 2024-05-20T16:25:41.406 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.406 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.406 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.406 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.406 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.406 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.406 INFO:teuthology.orchestra.run.smithi082.stdout: "testkey": { 2024-05-20T16:25:41.406 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "testkey", 2024-05-20T16:25:41.406 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.407 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.407 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.407 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:41.407 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.407 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.407 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.407 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.407 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.407 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.407 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.407 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.407 INFO:teuthology.orchestra.run.smithi082.stdout: "testlkey": { 2024-05-20T16:25:41.407 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "testlkey", 2024-05-20T16:25:41.407 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.407 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.407 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.408 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:41.408 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.408 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.408 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.408 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.408 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.408 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.408 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.408 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.408 INFO:teuthology.orchestra.run.smithi082.stdout: "testnewline": { 2024-05-20T16:25:41.408 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "testnewline", 2024-05-20T16:25:41.408 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.408 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.408 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.408 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:41.409 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.409 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.409 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.409 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.409 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.409 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.409 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.409 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-20T16:25:41.409 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-20T16:25:41.409 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.409 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-05-20T16:25:41.409 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "snap_schedule", 2024-05-20T16:25:41.409 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-05-20T16:25:41.409 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-05-20T16:25:41.409 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-05-20T16:25:41.410 INFO:teuthology.orchestra.run.smithi082.stdout: "allow_m_granularity": { 2024-05-20T16:25:41.410 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "allow_m_granularity", 2024-05-20T16:25:41.410 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.410 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.410 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.410 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:41.410 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.410 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.410 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.410 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "allow minute scheduled snapshots", 2024-05-20T16:25:41.410 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.410 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.410 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.410 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.410 INFO:teuthology.orchestra.run.smithi082.stdout: "dump_on_update": { 2024-05-20T16:25:41.411 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "dump_on_update", 2024-05-20T16:25:41.411 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.411 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.411 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.411 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:41.411 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.411 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.411 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.411 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "dump database to debug log on update", 2024-05-20T16:25:41.411 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.411 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.411 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.411 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.411 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-05-20T16:25:41.412 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-05-20T16:25:41.412 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.412 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.412 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.412 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:41.412 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.412 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.412 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-05-20T16:25:41.412 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-05-20T16:25:41.412 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-05-20T16:25:41.412 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-05-20T16:25:41.412 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-05-20T16:25:41.412 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-05-20T16:25:41.412 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-05-20T16:25:41.413 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:41.413 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.413 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.413 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.413 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.413 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.413 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-05-20T16:25:41.413 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-05-20T16:25:41.413 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.413 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.413 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.413 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:41.413 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.413 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.414 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.414 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.414 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.414 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.414 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.414 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.414 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-05-20T16:25:41.414 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-05-20T16:25:41.414 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.414 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.414 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.414 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-05-20T16:25:41.415 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.415 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.415 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-05-20T16:25:41.415 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-05-20T16:25:41.415 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-05-20T16:25:41.415 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-05-20T16:25:41.415 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-05-20T16:25:41.415 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-05-20T16:25:41.415 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-05-20T16:25:41.415 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:41.415 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.415 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.415 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.415 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.416 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.416 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-05-20T16:25:41.416 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-05-20T16:25:41.416 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.416 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.416 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.416 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:41.416 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.416 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.416 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.416 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.416 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.417 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.417 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.417 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-20T16:25:41.417 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-20T16:25:41.417 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.417 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-05-20T16:25:41.417 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "stats", 2024-05-20T16:25:41.417 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-05-20T16:25:41.417 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-05-20T16:25:41.417 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-05-20T16:25:41.417 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-05-20T16:25:41.417 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-05-20T16:25:41.417 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.417 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.418 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.418 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:41.418 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.418 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.418 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-05-20T16:25:41.418 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-05-20T16:25:41.418 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-05-20T16:25:41.418 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-05-20T16:25:41.418 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-05-20T16:25:41.418 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-05-20T16:25:41.418 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-05-20T16:25:41.418 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:41.418 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.419 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.419 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.419 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.419 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.419 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-05-20T16:25:41.419 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-05-20T16:25:41.419 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.419 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.419 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.419 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:41.419 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.419 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.419 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.420 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.420 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.420 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.420 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.420 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.420 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-05-20T16:25:41.420 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-05-20T16:25:41.420 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.420 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.420 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.420 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-05-20T16:25:41.420 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.420 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.420 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-05-20T16:25:41.420 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-05-20T16:25:41.421 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-05-20T16:25:41.421 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-05-20T16:25:41.421 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-05-20T16:25:41.421 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-05-20T16:25:41.421 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-05-20T16:25:41.421 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:41.421 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.421 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.421 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.421 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.421 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.421 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-05-20T16:25:41.421 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-05-20T16:25:41.421 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.421 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.421 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.422 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:41.422 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.422 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.422 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.422 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.422 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.422 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.422 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.422 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-20T16:25:41.422 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-20T16:25:41.422 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.422 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-05-20T16:25:41.422 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "status", 2024-05-20T16:25:41.423 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-05-20T16:25:41.423 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-05-20T16:25:41.423 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-05-20T16:25:41.423 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-05-20T16:25:41.423 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-05-20T16:25:41.423 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.423 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.423 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.423 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:41.423 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.423 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.423 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-05-20T16:25:41.423 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-05-20T16:25:41.423 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-05-20T16:25:41.424 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-05-20T16:25:41.424 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-05-20T16:25:41.424 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-05-20T16:25:41.424 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-05-20T16:25:41.424 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:41.424 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.424 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.424 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.424 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.424 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.424 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-05-20T16:25:41.424 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-05-20T16:25:41.424 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.424 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.425 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.425 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:41.425 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.425 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.425 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.425 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.425 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.425 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.425 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.425 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.425 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-05-20T16:25:41.425 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-05-20T16:25:41.425 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.425 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.425 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.426 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-05-20T16:25:41.426 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.426 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.426 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-05-20T16:25:41.426 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-05-20T16:25:41.426 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-05-20T16:25:41.426 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-05-20T16:25:41.426 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-05-20T16:25:41.426 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-05-20T16:25:41.426 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-05-20T16:25:41.426 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:41.426 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.427 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.427 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.427 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.427 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.427 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-05-20T16:25:41.427 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-05-20T16:25:41.427 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.427 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.427 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.427 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:41.427 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.427 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.427 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.427 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.428 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.428 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.428 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.428 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-20T16:25:41.428 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-20T16:25:41.428 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.428 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-05-20T16:25:41.428 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "telegraf", 2024-05-20T16:25:41.428 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-05-20T16:25:41.428 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-05-20T16:25:41.428 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-05-20T16:25:41.428 INFO:teuthology.orchestra.run.smithi082.stdout: "address": { 2024-05-20T16:25:41.428 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "address", 2024-05-20T16:25:41.428 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.429 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.429 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.429 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-05-20T16:25:41.429 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.429 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.429 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.429 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.429 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.429 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.429 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.429 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.429 INFO:teuthology.orchestra.run.smithi082.stdout: "interval": { 2024-05-20T16:25:41.429 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "interval", 2024-05-20T16:25:41.430 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2024-05-20T16:25:41.430 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.430 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.430 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "15", 2024-05-20T16:25:41.430 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.430 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.430 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.430 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.430 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.430 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.430 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.430 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.430 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-05-20T16:25:41.430 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-05-20T16:25:41.431 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.431 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.431 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.431 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:41.431 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.431 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.431 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-05-20T16:25:41.431 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-05-20T16:25:41.431 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-05-20T16:25:41.431 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-05-20T16:25:41.431 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-05-20T16:25:41.431 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-05-20T16:25:41.431 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-05-20T16:25:41.432 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:41.432 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.432 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.432 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.432 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.432 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.432 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-05-20T16:25:41.432 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-05-20T16:25:41.432 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.432 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.432 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.432 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:41.432 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.433 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.433 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.433 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.433 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.433 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.433 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.433 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.433 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-05-20T16:25:41.433 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-05-20T16:25:41.433 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.433 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.433 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.433 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-05-20T16:25:41.433 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.434 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.434 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-05-20T16:25:41.434 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-05-20T16:25:41.434 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-05-20T16:25:41.434 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-05-20T16:25:41.434 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-05-20T16:25:41.434 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-05-20T16:25:41.434 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-05-20T16:25:41.434 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:41.434 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.434 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.434 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.434 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.435 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.435 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-05-20T16:25:41.435 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-05-20T16:25:41.435 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.435 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.435 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.435 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:41.435 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.435 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.435 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.435 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.435 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.435 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.435 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.436 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-20T16:25:41.436 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-20T16:25:41.436 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.436 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-05-20T16:25:41.436 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "telemetry", 2024-05-20T16:25:41.436 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-05-20T16:25:41.436 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-05-20T16:25:41.436 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-05-20T16:25:41.436 INFO:teuthology.orchestra.run.smithi082.stdout: "channel_basic": { 2024-05-20T16:25:41.436 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "channel_basic", 2024-05-20T16:25:41.436 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.436 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.436 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.437 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-05-20T16:25:41.437 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.437 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.437 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.437 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Share basic cluster information (size, version)", 2024-05-20T16:25:41.437 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.437 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.437 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.437 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.437 INFO:teuthology.orchestra.run.smithi082.stdout: "channel_crash": { 2024-05-20T16:25:41.437 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "channel_crash", 2024-05-20T16:25:41.437 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.437 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.437 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.438 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-05-20T16:25:41.438 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.438 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.438 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.438 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-05-20T16:25:41.438 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.438 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.438 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.438 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.438 INFO:teuthology.orchestra.run.smithi082.stdout: "channel_device": { 2024-05-20T16:25:41.438 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "channel_device", 2024-05-20T16:25:41.438 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.438 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.439 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.439 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-05-20T16:25:41.439 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.439 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.439 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.439 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-05-20T16:25:41.439 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.439 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.439 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.439 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.439 INFO:teuthology.orchestra.run.smithi082.stdout: "channel_ident": { 2024-05-20T16:25:41.439 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "channel_ident", 2024-05-20T16:25:41.439 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.439 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.439 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.440 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:41.440 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.440 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.440 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.440 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-05-20T16:25:41.440 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.440 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.440 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.440 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.440 INFO:teuthology.orchestra.run.smithi082.stdout: "channel_perf": { 2024-05-20T16:25:41.440 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "channel_perf", 2024-05-20T16:25:41.440 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.440 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.440 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.441 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:41.441 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.441 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.441 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.441 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Share various performance metrics of a cluster", 2024-05-20T16:25:41.441 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.441 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.441 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.441 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.441 INFO:teuthology.orchestra.run.smithi082.stdout: "contact": { 2024-05-20T16:25:41.441 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "contact", 2024-05-20T16:25:41.441 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.441 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.442 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.442 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:41.442 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.442 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.442 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.442 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.442 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.442 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.442 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.442 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.442 INFO:teuthology.orchestra.run.smithi082.stdout: "description": { 2024-05-20T16:25:41.442 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "description", 2024-05-20T16:25:41.442 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.443 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.443 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.443 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:41.443 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.443 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.443 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.443 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.443 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.443 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.443 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.443 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.443 INFO:teuthology.orchestra.run.smithi082.stdout: "device_url": { 2024-05-20T16:25:41.443 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "device_url", 2024-05-20T16:25:41.444 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.444 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.444 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.444 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-05-20T16:25:41.444 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.444 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.444 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.444 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.444 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.444 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.444 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.444 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.444 INFO:teuthology.orchestra.run.smithi082.stdout: "enabled": { 2024-05-20T16:25:41.444 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "enabled", 2024-05-20T16:25:41.445 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.445 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.445 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.445 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:41.445 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.445 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.445 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.445 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.445 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.445 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.445 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.445 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.445 INFO:teuthology.orchestra.run.smithi082.stdout: "interval": { 2024-05-20T16:25:41.445 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "interval", 2024-05-20T16:25:41.446 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-05-20T16:25:41.446 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.446 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.446 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "24", 2024-05-20T16:25:41.446 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "8", 2024-05-20T16:25:41.446 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.446 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.446 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.446 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.446 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.446 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.446 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.446 INFO:teuthology.orchestra.run.smithi082.stdout: "last_opt_revision": { 2024-05-20T16:25:41.446 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "last_opt_revision", 2024-05-20T16:25:41.447 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-05-20T16:25:41.447 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.447 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.447 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "1", 2024-05-20T16:25:41.447 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.447 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.447 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.447 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.447 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.447 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.447 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.447 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.447 INFO:teuthology.orchestra.run.smithi082.stdout: "leaderboard": { 2024-05-20T16:25:41.447 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "leaderboard", 2024-05-20T16:25:41.448 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.448 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.448 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.448 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:41.448 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.448 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.448 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.448 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.448 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.448 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.448 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.448 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.448 INFO:teuthology.orchestra.run.smithi082.stdout: "leaderboard_description": { 2024-05-20T16:25:41.448 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "leaderboard_description", 2024-05-20T16:25:41.449 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.449 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.449 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.449 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:41.449 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.449 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.449 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.449 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.449 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.449 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.449 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.449 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.449 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-05-20T16:25:41.449 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-05-20T16:25:41.449 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.450 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.450 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.450 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:41.450 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.450 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.450 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-05-20T16:25:41.450 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-05-20T16:25:41.450 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-05-20T16:25:41.450 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-05-20T16:25:41.450 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-05-20T16:25:41.450 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-05-20T16:25:41.450 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-05-20T16:25:41.450 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:41.450 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.451 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.451 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.451 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.451 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.451 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-05-20T16:25:41.451 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-05-20T16:25:41.451 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.451 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.451 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.451 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:41.451 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.451 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.451 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.451 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.451 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.452 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.452 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.452 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.452 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-05-20T16:25:41.452 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-05-20T16:25:41.452 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.452 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.452 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.452 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-05-20T16:25:41.452 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.452 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.452 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-05-20T16:25:41.452 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-05-20T16:25:41.452 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-05-20T16:25:41.453 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-05-20T16:25:41.453 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-05-20T16:25:41.453 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-05-20T16:25:41.453 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-05-20T16:25:41.453 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:41.453 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.453 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.453 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.453 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.453 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.453 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-05-20T16:25:41.453 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-05-20T16:25:41.453 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.453 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.453 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.453 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:41.454 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.454 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.454 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.454 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.454 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.454 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.454 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.454 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.454 INFO:teuthology.orchestra.run.smithi082.stdout: "organization": { 2024-05-20T16:25:41.454 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "organization", 2024-05-20T16:25:41.454 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.454 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.454 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.454 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:41.455 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.455 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.455 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.455 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.455 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.455 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.455 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.455 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.455 INFO:teuthology.orchestra.run.smithi082.stdout: "proxy": { 2024-05-20T16:25:41.455 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "proxy", 2024-05-20T16:25:41.455 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.455 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.455 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.455 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:41.455 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.455 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.456 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.456 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.456 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.456 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.456 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.456 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.456 INFO:teuthology.orchestra.run.smithi082.stdout: "url": { 2024-05-20T16:25:41.456 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "url", 2024-05-20T16:25:41.456 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.456 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.456 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.456 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-05-20T16:25:41.456 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.456 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.456 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.456 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.457 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.457 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.457 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.457 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-20T16:25:41.457 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-20T16:25:41.457 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.457 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-05-20T16:25:41.457 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "test_orchestrator", 2024-05-20T16:25:41.457 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-05-20T16:25:41.457 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-05-20T16:25:41.457 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-05-20T16:25:41.457 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-05-20T16:25:41.457 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-05-20T16:25:41.457 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.457 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.458 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.458 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:41.458 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.458 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.458 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-05-20T16:25:41.458 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-05-20T16:25:41.458 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-05-20T16:25:41.458 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-05-20T16:25:41.458 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-05-20T16:25:41.458 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-05-20T16:25:41.458 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-05-20T16:25:41.458 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:41.458 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.458 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.458 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.459 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.459 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.459 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-05-20T16:25:41.459 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-05-20T16:25:41.459 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.459 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.459 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.459 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:41.459 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.459 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.459 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.459 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.459 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.460 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.460 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.460 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.460 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-05-20T16:25:41.460 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-05-20T16:25:41.460 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.460 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.460 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.460 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-05-20T16:25:41.460 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.460 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.460 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-05-20T16:25:41.460 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-05-20T16:25:41.460 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-05-20T16:25:41.460 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-05-20T16:25:41.461 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-05-20T16:25:41.461 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-05-20T16:25:41.461 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-05-20T16:25:41.461 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:41.461 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.461 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.461 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.461 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.461 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.461 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-05-20T16:25:41.461 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-05-20T16:25:41.461 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.461 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.461 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.461 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:41.461 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.462 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.462 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.462 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.462 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.462 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.462 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.462 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-20T16:25:41.462 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-20T16:25:41.462 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.462 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-05-20T16:25:41.462 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "volumes", 2024-05-20T16:25:41.462 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-05-20T16:25:41.462 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-05-20T16:25:41.462 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-05-20T16:25:41.463 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-05-20T16:25:41.463 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-05-20T16:25:41.463 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.463 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.463 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.463 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:41.463 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.463 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.463 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-05-20T16:25:41.463 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-05-20T16:25:41.463 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-05-20T16:25:41.463 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-05-20T16:25:41.463 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-05-20T16:25:41.463 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-05-20T16:25:41.464 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-05-20T16:25:41.464 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:41.464 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.464 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.464 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.464 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.464 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.464 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-05-20T16:25:41.464 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-05-20T16:25:41.464 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.464 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.464 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.464 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:41.464 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.465 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.465 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.465 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.465 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.465 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.465 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.465 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.465 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-05-20T16:25:41.465 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-05-20T16:25:41.465 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.465 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.465 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.465 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-05-20T16:25:41.465 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.465 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.466 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-05-20T16:25:41.466 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-05-20T16:25:41.466 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-05-20T16:25:41.466 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-05-20T16:25:41.466 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-05-20T16:25:41.466 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-05-20T16:25:41.466 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-05-20T16:25:41.466 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:41.466 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.466 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.466 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.466 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.466 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.466 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-05-20T16:25:41.466 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-05-20T16:25:41.467 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.467 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.467 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.467 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:41.467 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.467 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.467 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.467 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.467 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.467 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.467 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.467 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.467 INFO:teuthology.orchestra.run.smithi082.stdout: "max_concurrent_clones": { 2024-05-20T16:25:41.467 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "max_concurrent_clones", 2024-05-20T16:25:41.467 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-05-20T16:25:41.468 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.468 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.468 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "4", 2024-05-20T16:25:41.468 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.468 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.468 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.468 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Number of asynchronous cloner threads", 2024-05-20T16:25:41.468 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.468 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.468 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.468 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.468 INFO:teuthology.orchestra.run.smithi082.stdout: "periodic_async_work": { 2024-05-20T16:25:41.468 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "periodic_async_work", 2024-05-20T16:25:41.468 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.468 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.468 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.469 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:41.469 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.469 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.469 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.469 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Periodically check for async work", 2024-05-20T16:25:41.469 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.469 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.469 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.469 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.469 INFO:teuthology.orchestra.run.smithi082.stdout: "snapshot_clone_delay": { 2024-05-20T16:25:41.469 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "snapshot_clone_delay", 2024-05-20T16:25:41.469 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-05-20T16:25:41.469 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.469 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.470 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "0", 2024-05-20T16:25:41.470 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.470 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.470 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.470 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-05-20T16:25:41.470 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.470 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.470 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.470 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.470 INFO:teuthology.orchestra.run.smithi082.stdout: "snapshot_clone_no_wait": { 2024-05-20T16:25:41.470 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "snapshot_clone_no_wait", 2024-05-20T16:25:41.470 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.473 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.473 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.474 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-05-20T16:25:41.474 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.474 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.474 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.474 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-05-20T16:25:41.474 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.474 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.474 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.474 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-20T16:25:41.474 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-20T16:25:41.474 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.474 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-05-20T16:25:41.475 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "zabbix", 2024-05-20T16:25:41.475 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-05-20T16:25:41.475 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-05-20T16:25:41.475 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-05-20T16:25:41.475 INFO:teuthology.orchestra.run.smithi082.stdout: "discovery_interval": { 2024-05-20T16:25:41.475 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "discovery_interval", 2024-05-20T16:25:41.475 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "uint", 2024-05-20T16:25:41.475 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.475 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.475 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "100", 2024-05-20T16:25:41.475 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.475 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.475 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.476 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.476 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.476 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.476 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.476 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.476 INFO:teuthology.orchestra.run.smithi082.stdout: "identifier": { 2024-05-20T16:25:41.476 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "identifier", 2024-05-20T16:25:41.476 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.476 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.476 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.476 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:41.476 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.476 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.476 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.477 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.477 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.477 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.477 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.477 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.477 INFO:teuthology.orchestra.run.smithi082.stdout: "interval": { 2024-05-20T16:25:41.477 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "interval", 2024-05-20T16:25:41.477 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2024-05-20T16:25:41.477 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.477 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.477 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "60", 2024-05-20T16:25:41.477 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.477 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.477 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.477 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.478 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.478 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.478 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.478 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.478 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-05-20T16:25:41.478 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-05-20T16:25:41.478 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.478 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.478 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.478 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:41.478 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.478 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.478 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-05-20T16:25:41.478 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-05-20T16:25:41.479 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-05-20T16:25:41.479 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-05-20T16:25:41.479 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-05-20T16:25:41.479 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-05-20T16:25:41.479 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-05-20T16:25:41.479 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:41.479 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.479 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.479 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.479 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.479 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.479 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-05-20T16:25:41.479 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-05-20T16:25:41.479 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.480 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.480 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.480 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:41.480 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.480 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.480 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.480 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.480 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.480 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.480 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.480 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.480 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-05-20T16:25:41.480 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-05-20T16:25:41.480 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.480 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.481 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.481 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-05-20T16:25:41.481 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.481 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.481 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-05-20T16:25:41.481 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-05-20T16:25:41.481 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-05-20T16:25:41.481 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-05-20T16:25:41.481 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-05-20T16:25:41.481 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-05-20T16:25:41.481 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-05-20T16:25:41.481 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:41.481 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.481 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.481 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.482 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.482 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.482 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-05-20T16:25:41.482 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-05-20T16:25:41.482 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.482 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.482 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.482 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:41.482 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.482 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.482 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.482 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.482 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.483 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.483 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.483 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.483 INFO:teuthology.orchestra.run.smithi082.stdout: "zabbix_host": { 2024-05-20T16:25:41.483 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "zabbix_host", 2024-05-20T16:25:41.483 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.483 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.483 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.483 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:41.483 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.483 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.483 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.483 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.483 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.483 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.484 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.484 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.484 INFO:teuthology.orchestra.run.smithi082.stdout: "zabbix_port": { 2024-05-20T16:25:41.484 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "zabbix_port", 2024-05-20T16:25:41.484 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-05-20T16:25:41.484 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.484 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.484 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "10051", 2024-05-20T16:25:41.484 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.484 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.484 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.484 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.484 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.484 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.484 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.485 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.485 INFO:teuthology.orchestra.run.smithi082.stdout: "zabbix_sender": { 2024-05-20T16:25:41.485 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "zabbix_sender", 2024-05-20T16:25:41.485 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.485 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.485 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.485 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-05-20T16:25:41.485 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.485 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.485 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.485 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.485 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.485 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.485 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.485 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-20T16:25:41.486 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-20T16:25:41.486 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-20T16:25:41.486 INFO:teuthology.orchestra.run.smithi082.stdout: ] 2024-05-20T16:25:41.486 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.486 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-05-20T16:25:41.486 INFO:teuthology.orchestra.run.smithi082.stdout: "gid": 24116, 2024-05-20T16:25:41.486 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "y", 2024-05-20T16:25:41.486 INFO:teuthology.orchestra.run.smithi082.stdout: "mgr_features": 4540701547738038271, 2024-05-20T16:25:41.486 INFO:teuthology.orchestra.run.smithi082.stdout: "available_modules": [ 2024-05-20T16:25:41.486 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-05-20T16:25:41.486 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "alerts", 2024-05-20T16:25:41.486 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-05-20T16:25:41.486 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-05-20T16:25:41.486 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-05-20T16:25:41.486 INFO:teuthology.orchestra.run.smithi082.stdout: "interval": { 2024-05-20T16:25:41.487 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "interval", 2024-05-20T16:25:41.487 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2024-05-20T16:25:41.487 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.487 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.487 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "60", 2024-05-20T16:25:41.487 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.487 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.487 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.487 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "How frequently to reexamine health status", 2024-05-20T16:25:41.487 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.487 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.487 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.487 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.487 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-05-20T16:25:41.487 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-05-20T16:25:41.488 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.488 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.488 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.488 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:41.488 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.488 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.488 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-05-20T16:25:41.488 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-05-20T16:25:41.488 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-05-20T16:25:41.488 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-05-20T16:25:41.488 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-05-20T16:25:41.488 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-05-20T16:25:41.488 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-05-20T16:25:41.488 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:41.489 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.489 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.489 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.489 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.489 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.489 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-05-20T16:25:41.489 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-05-20T16:25:41.489 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.489 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.489 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.489 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:41.489 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.489 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.489 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.490 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.490 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.490 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.490 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.490 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.490 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-05-20T16:25:41.490 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-05-20T16:25:41.490 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.490 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.490 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.490 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-05-20T16:25:41.490 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.490 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.490 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-05-20T16:25:41.491 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-05-20T16:25:41.491 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-05-20T16:25:41.491 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-05-20T16:25:41.491 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-05-20T16:25:41.491 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-05-20T16:25:41.491 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-05-20T16:25:41.491 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:41.491 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.491 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.491 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.491 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.491 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.491 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-05-20T16:25:41.491 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-05-20T16:25:41.491 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.491 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.492 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.492 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:41.492 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.492 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.492 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.492 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.492 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.492 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.492 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.492 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.492 INFO:teuthology.orchestra.run.smithi082.stdout: "smtp_destination": { 2024-05-20T16:25:41.492 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "smtp_destination", 2024-05-20T16:25:41.492 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.492 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.492 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.493 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:41.493 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.493 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.493 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.493 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Email address to send alerts to", 2024-05-20T16:25:41.493 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.493 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.493 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.493 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.493 INFO:teuthology.orchestra.run.smithi082.stdout: "smtp_from_name": { 2024-05-20T16:25:41.493 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "smtp_from_name", 2024-05-20T16:25:41.493 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.493 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.493 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.493 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "Ceph", 2024-05-20T16:25:41.494 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.494 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.494 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.494 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Email From: name", 2024-05-20T16:25:41.494 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.494 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.494 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.494 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.494 INFO:teuthology.orchestra.run.smithi082.stdout: "smtp_host": { 2024-05-20T16:25:41.494 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "smtp_host", 2024-05-20T16:25:41.494 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.494 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.494 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.494 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:41.495 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.495 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.495 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.495 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "SMTP server", 2024-05-20T16:25:41.495 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.495 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.495 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.495 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.495 INFO:teuthology.orchestra.run.smithi082.stdout: "smtp_password": { 2024-05-20T16:25:41.495 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "smtp_password", 2024-05-20T16:25:41.495 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.495 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.495 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.495 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:41.495 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.495 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.496 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.496 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Password to authenticate with", 2024-05-20T16:25:41.496 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.496 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.496 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.496 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.496 INFO:teuthology.orchestra.run.smithi082.stdout: "smtp_port": { 2024-05-20T16:25:41.496 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "smtp_port", 2024-05-20T16:25:41.496 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-05-20T16:25:41.496 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.496 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.496 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "465", 2024-05-20T16:25:41.496 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.496 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.496 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.496 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "SMTP port", 2024-05-20T16:25:41.497 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.497 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.497 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.497 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.497 INFO:teuthology.orchestra.run.smithi082.stdout: "smtp_sender": { 2024-05-20T16:25:41.497 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "smtp_sender", 2024-05-20T16:25:41.497 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.497 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.497 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.497 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:41.497 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.497 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.497 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.497 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "SMTP envelope sender", 2024-05-20T16:25:41.497 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.497 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.498 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.498 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.498 INFO:teuthology.orchestra.run.smithi082.stdout: "smtp_ssl": { 2024-05-20T16:25:41.498 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "smtp_ssl", 2024-05-20T16:25:41.498 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.498 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.498 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.498 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-05-20T16:25:41.498 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.498 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.498 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.498 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Use SSL to connect to SMTP server", 2024-05-20T16:25:41.498 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.498 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.498 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.498 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.499 INFO:teuthology.orchestra.run.smithi082.stdout: "smtp_user": { 2024-05-20T16:25:41.499 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "smtp_user", 2024-05-20T16:25:41.499 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.499 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.499 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.499 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:41.499 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.499 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.499 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.499 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "User to authenticate as", 2024-05-20T16:25:41.499 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.499 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.499 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.499 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-20T16:25:41.499 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-20T16:25:41.500 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.500 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-05-20T16:25:41.500 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "balancer", 2024-05-20T16:25:41.500 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-05-20T16:25:41.500 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-05-20T16:25:41.500 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-05-20T16:25:41.500 INFO:teuthology.orchestra.run.smithi082.stdout: "active": { 2024-05-20T16:25:41.500 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "active", 2024-05-20T16:25:41.500 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.500 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.500 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.500 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-05-20T16:25:41.500 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.500 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.500 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.500 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "automatically balance PGs across cluster", 2024-05-20T16:25:41.501 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.501 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.501 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.501 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.501 INFO:teuthology.orchestra.run.smithi082.stdout: "begin_time": { 2024-05-20T16:25:41.501 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "begin_time", 2024-05-20T16:25:41.501 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.501 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.501 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.501 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "0000", 2024-05-20T16:25:41.501 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.501 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.501 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.501 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "beginning time of day to automatically balance", 2024-05-20T16:25:41.502 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-20T16:25:41.502 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.502 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.502 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.502 INFO:teuthology.orchestra.run.smithi082.stdout: "begin_weekday": { 2024-05-20T16:25:41.502 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "begin_weekday", 2024-05-20T16:25:41.502 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "uint", 2024-05-20T16:25:41.502 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.502 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.502 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "0", 2024-05-20T16:25:41.502 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "0", 2024-05-20T16:25:41.502 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "6", 2024-05-20T16:25:41.502 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.503 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-05-20T16:25:41.503 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-20T16:25:41.503 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.503 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.503 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.503 INFO:teuthology.orchestra.run.smithi082.stdout: "crush_compat_max_iterations": { 2024-05-20T16:25:41.503 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "crush_compat_max_iterations", 2024-05-20T16:25:41.503 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "uint", 2024-05-20T16:25:41.503 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.503 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.503 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "25", 2024-05-20T16:25:41.503 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "1", 2024-05-20T16:25:41.503 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "250", 2024-05-20T16:25:41.503 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.503 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-05-20T16:25:41.503 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.504 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.504 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.504 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.504 INFO:teuthology.orchestra.run.smithi082.stdout: "crush_compat_metrics": { 2024-05-20T16:25:41.504 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "crush_compat_metrics", 2024-05-20T16:25:41.504 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.504 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.504 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.504 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "pgs,objects,bytes", 2024-05-20T16:25:41.504 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.504 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.504 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.504 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-05-20T16:25:41.504 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-05-20T16:25:41.504 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.504 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.505 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.505 INFO:teuthology.orchestra.run.smithi082.stdout: "crush_compat_step": { 2024-05-20T16:25:41.505 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "crush_compat_step", 2024-05-20T16:25:41.505 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "float", 2024-05-20T16:25:41.505 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.505 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.505 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "0.5", 2024-05-20T16:25:41.505 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "0.001", 2024-05-20T16:25:41.505 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "0.999", 2024-05-20T16:25:41.505 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.505 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "aggressiveness of optimization", 2024-05-20T16:25:41.505 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-05-20T16:25:41.505 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.505 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.505 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.505 INFO:teuthology.orchestra.run.smithi082.stdout: "end_time": { 2024-05-20T16:25:41.506 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "end_time", 2024-05-20T16:25:41.506 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.506 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.506 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.506 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "2359", 2024-05-20T16:25:41.506 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.506 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.506 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.506 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "ending time of day to automatically balance", 2024-05-20T16:25:41.506 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-20T16:25:41.506 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.506 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.506 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.506 INFO:teuthology.orchestra.run.smithi082.stdout: "end_weekday": { 2024-05-20T16:25:41.507 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "end_weekday", 2024-05-20T16:25:41.507 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "uint", 2024-05-20T16:25:41.507 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.507 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.507 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "0", 2024-05-20T16:25:41.507 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "0", 2024-05-20T16:25:41.507 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "6", 2024-05-20T16:25:41.507 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.507 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-05-20T16:25:41.507 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-20T16:25:41.507 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.507 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.507 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.507 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-05-20T16:25:41.508 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-05-20T16:25:41.508 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.508 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.508 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.508 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:41.508 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.508 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.508 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-05-20T16:25:41.508 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-05-20T16:25:41.508 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-05-20T16:25:41.508 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-05-20T16:25:41.508 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-05-20T16:25:41.508 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-05-20T16:25:41.508 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-05-20T16:25:41.508 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:41.509 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.509 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.509 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.509 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.509 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.509 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-05-20T16:25:41.509 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-05-20T16:25:41.509 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.509 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.509 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.509 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:41.509 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.509 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.509 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.509 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.509 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.510 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.510 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.510 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.510 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-05-20T16:25:41.510 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-05-20T16:25:41.510 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.510 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.510 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.510 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-05-20T16:25:41.510 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.510 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.510 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-05-20T16:25:41.510 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-05-20T16:25:41.510 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-05-20T16:25:41.511 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-05-20T16:25:41.511 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-05-20T16:25:41.511 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-05-20T16:25:41.511 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-05-20T16:25:41.511 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:41.511 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.511 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.511 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.511 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.511 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.511 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-05-20T16:25:41.511 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-05-20T16:25:41.511 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.511 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.511 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.511 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:41.512 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.512 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.512 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.512 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.512 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.512 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.512 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.512 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.512 INFO:teuthology.orchestra.run.smithi082.stdout: "min_score": { 2024-05-20T16:25:41.512 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "min_score", 2024-05-20T16:25:41.512 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "float", 2024-05-20T16:25:41.512 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.512 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.512 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "0", 2024-05-20T16:25:41.513 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.513 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.513 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.513 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-05-20T16:25:41.513 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.513 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.513 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.513 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.513 INFO:teuthology.orchestra.run.smithi082.stdout: "mode": { 2024-05-20T16:25:41.513 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "mode", 2024-05-20T16:25:41.513 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.513 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.513 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.513 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "upmap", 2024-05-20T16:25:41.513 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.513 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.514 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-05-20T16:25:41.514 INFO:teuthology.orchestra.run.smithi082.stdout: "crush-compat", 2024-05-20T16:25:41.514 INFO:teuthology.orchestra.run.smithi082.stdout: "none", 2024-05-20T16:25:41.514 INFO:teuthology.orchestra.run.smithi082.stdout: "read", 2024-05-20T16:25:41.514 INFO:teuthology.orchestra.run.smithi082.stdout: "upmap", 2024-05-20T16:25:41.514 INFO:teuthology.orchestra.run.smithi082.stdout: "upmap-read" 2024-05-20T16:25:41.514 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:41.514 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Balancer mode", 2024-05-20T16:25:41.514 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.514 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.514 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.514 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.514 INFO:teuthology.orchestra.run.smithi082.stdout: "pool_ids": { 2024-05-20T16:25:41.514 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "pool_ids", 2024-05-20T16:25:41.514 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.515 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.515 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.515 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:41.515 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.515 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.515 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.515 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-05-20T16:25:41.515 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.515 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.515 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.515 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.515 INFO:teuthology.orchestra.run.smithi082.stdout: "sleep_interval": { 2024-05-20T16:25:41.515 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "sleep_interval", 2024-05-20T16:25:41.515 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2024-05-20T16:25:41.515 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.515 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.516 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "60", 2024-05-20T16:25:41.516 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.516 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.516 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.516 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-05-20T16:25:41.516 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.516 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.516 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.516 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.516 INFO:teuthology.orchestra.run.smithi082.stdout: "upmap_max_deviation": { 2024-05-20T16:25:41.516 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "upmap_max_deviation", 2024-05-20T16:25:41.516 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-05-20T16:25:41.516 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.516 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.516 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "5", 2024-05-20T16:25:41.516 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "1", 2024-05-20T16:25:41.517 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.517 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.517 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "deviation below which no optimization is attempted", 2024-05-20T16:25:41.517 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-05-20T16:25:41.517 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.517 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.517 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.517 INFO:teuthology.orchestra.run.smithi082.stdout: "upmap_max_optimizations": { 2024-05-20T16:25:41.517 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "upmap_max_optimizations", 2024-05-20T16:25:41.517 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "uint", 2024-05-20T16:25:41.517 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.517 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.517 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "10", 2024-05-20T16:25:41.517 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.517 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.517 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.518 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-05-20T16:25:41.518 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.518 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.518 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.518 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-20T16:25:41.518 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-20T16:25:41.518 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.518 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-05-20T16:25:41.518 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "cephadm", 2024-05-20T16:25:41.518 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-05-20T16:25:41.518 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-05-20T16:25:41.518 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-05-20T16:25:41.518 INFO:teuthology.orchestra.run.smithi082.stdout: "agent_down_multiplier": { 2024-05-20T16:25:41.519 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "agent_down_multiplier", 2024-05-20T16:25:41.519 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "float", 2024-05-20T16:25:41.519 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.519 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.519 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "3.0", 2024-05-20T16:25:41.519 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.519 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.519 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.519 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-05-20T16:25:41.519 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.519 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.519 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.519 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.519 INFO:teuthology.orchestra.run.smithi082.stdout: "agent_refresh_rate": { 2024-05-20T16:25:41.519 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "agent_refresh_rate", 2024-05-20T16:25:41.519 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2024-05-20T16:25:41.519 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.520 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.520 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "20", 2024-05-20T16:25:41.520 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.520 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.520 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.520 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-05-20T16:25:41.520 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.520 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.520 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.520 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.520 INFO:teuthology.orchestra.run.smithi082.stdout: "agent_starting_port": { 2024-05-20T16:25:41.520 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "agent_starting_port", 2024-05-20T16:25:41.520 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-05-20T16:25:41.520 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.520 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.521 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "4721", 2024-05-20T16:25:41.521 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.521 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.521 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.521 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-05-20T16:25:41.521 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.521 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.521 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.521 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.521 INFO:teuthology.orchestra.run.smithi082.stdout: "allow_ptrace": { 2024-05-20T16:25:41.521 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "allow_ptrace", 2024-05-20T16:25:41.521 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.521 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.521 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.521 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:41.522 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.522 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.522 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.522 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-05-20T16:25:41.522 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-05-20T16:25:41.522 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.522 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.522 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.522 INFO:teuthology.orchestra.run.smithi082.stdout: "autotune_interval": { 2024-05-20T16:25:41.522 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "autotune_interval", 2024-05-20T16:25:41.522 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2024-05-20T16:25:41.522 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.522 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.522 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "600", 2024-05-20T16:25:41.523 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.523 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.523 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.523 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "how frequently to autotune daemon memory", 2024-05-20T16:25:41.523 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.523 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.523 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.523 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.523 INFO:teuthology.orchestra.run.smithi082.stdout: "autotune_memory_target_ratio": { 2024-05-20T16:25:41.523 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "autotune_memory_target_ratio", 2024-05-20T16:25:41.523 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "float", 2024-05-20T16:25:41.523 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.523 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.523 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "0.7", 2024-05-20T16:25:41.523 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.524 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.524 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.524 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-05-20T16:25:41.524 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.524 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.524 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.524 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.524 INFO:teuthology.orchestra.run.smithi082.stdout: "cephadm_log_destination": { 2024-05-20T16:25:41.524 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "cephadm_log_destination", 2024-05-20T16:25:41.524 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.524 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.524 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.524 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:41.524 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.524 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.525 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-05-20T16:25:41.525 INFO:teuthology.orchestra.run.smithi082.stdout: "file", 2024-05-20T16:25:41.525 INFO:teuthology.orchestra.run.smithi082.stdout: "file,syslog", 2024-05-20T16:25:41.525 INFO:teuthology.orchestra.run.smithi082.stdout: "syslog" 2024-05-20T16:25:41.525 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:41.525 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-05-20T16:25:41.525 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.525 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.525 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.525 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.525 INFO:teuthology.orchestra.run.smithi082.stdout: "cgroups_split": { 2024-05-20T16:25:41.525 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "cgroups_split", 2024-05-20T16:25:41.525 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.525 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.525 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.525 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-05-20T16:25:41.526 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.526 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.526 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.526 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-05-20T16:25:41.526 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.526 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.526 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.526 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.526 INFO:teuthology.orchestra.run.smithi082.stdout: "config_checks_enabled": { 2024-05-20T16:25:41.526 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "config_checks_enabled", 2024-05-20T16:25:41.526 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.526 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.526 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.526 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:41.526 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.527 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.527 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.527 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-05-20T16:25:41.527 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.527 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.527 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.527 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.527 INFO:teuthology.orchestra.run.smithi082.stdout: "config_dashboard": { 2024-05-20T16:25:41.527 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "config_dashboard", 2024-05-20T16:25:41.527 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.527 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.527 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.527 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-05-20T16:25:41.527 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.527 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.527 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.528 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-05-20T16:25:41.528 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.528 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.528 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.528 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.528 INFO:teuthology.orchestra.run.smithi082.stdout: "container_image_alertmanager": { 2024-05-20T16:25:41.528 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "container_image_alertmanager", 2024-05-20T16:25:41.528 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.528 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.528 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.528 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-05-20T16:25:41.528 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.528 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.528 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.528 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Prometheus container image", 2024-05-20T16:25:41.528 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.529 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.529 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.529 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.529 INFO:teuthology.orchestra.run.smithi082.stdout: "container_image_base": { 2024-05-20T16:25:41.529 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "container_image_base", 2024-05-20T16:25:41.529 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.529 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.529 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.529 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "quay.io/ceph/ceph", 2024-05-20T16:25:41.529 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.529 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.529 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.529 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Container image name, without the tag", 2024-05-20T16:25:41.529 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.529 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.530 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.530 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.530 INFO:teuthology.orchestra.run.smithi082.stdout: "container_image_elasticsearch": { 2024-05-20T16:25:41.530 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "container_image_elasticsearch", 2024-05-20T16:25:41.530 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.530 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.530 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.530 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-05-20T16:25:41.530 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.530 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.530 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.530 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "elasticsearch container image", 2024-05-20T16:25:41.530 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.530 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.531 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.531 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.531 INFO:teuthology.orchestra.run.smithi082.stdout: "container_image_grafana": { 2024-05-20T16:25:41.531 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "container_image_grafana", 2024-05-20T16:25:41.531 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.531 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.531 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.531 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-05-20T16:25:41.531 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.531 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.531 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.531 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Prometheus container image", 2024-05-20T16:25:41.531 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.531 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.532 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.532 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.532 INFO:teuthology.orchestra.run.smithi082.stdout: "container_image_haproxy": { 2024-05-20T16:25:41.532 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "container_image_haproxy", 2024-05-20T16:25:41.532 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.532 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.532 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.532 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-05-20T16:25:41.532 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.532 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.532 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.532 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "HAproxy container image", 2024-05-20T16:25:41.532 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.532 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.533 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.533 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.533 INFO:teuthology.orchestra.run.smithi082.stdout: "container_image_jaeger_agent": { 2024-05-20T16:25:41.533 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "container_image_jaeger_agent", 2024-05-20T16:25:41.533 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.533 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.533 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.533 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-05-20T16:25:41.533 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.533 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.533 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.533 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Jaeger agent container image", 2024-05-20T16:25:41.533 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.534 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.534 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.534 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.534 INFO:teuthology.orchestra.run.smithi082.stdout: "container_image_jaeger_collector": { 2024-05-20T16:25:41.534 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "container_image_jaeger_collector", 2024-05-20T16:25:41.534 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.534 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.534 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.534 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-05-20T16:25:41.534 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.534 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.534 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.534 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Jaeger collector container image", 2024-05-20T16:25:41.534 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.535 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.535 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.535 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.535 INFO:teuthology.orchestra.run.smithi082.stdout: "container_image_jaeger_query": { 2024-05-20T16:25:41.535 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "container_image_jaeger_query", 2024-05-20T16:25:41.535 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.535 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.535 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.535 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-05-20T16:25:41.535 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.535 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.535 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.535 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Jaeger query container image", 2024-05-20T16:25:41.535 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.535 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.536 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.536 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.536 INFO:teuthology.orchestra.run.smithi082.stdout: "container_image_keepalived": { 2024-05-20T16:25:41.536 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "container_image_keepalived", 2024-05-20T16:25:41.536 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.536 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.536 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.536 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-05-20T16:25:41.536 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.536 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.536 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.536 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Keepalived container image", 2024-05-20T16:25:41.536 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.536 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.536 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.536 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.537 INFO:teuthology.orchestra.run.smithi082.stdout: "container_image_loki": { 2024-05-20T16:25:41.537 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "container_image_loki", 2024-05-20T16:25:41.537 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.537 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.537 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.537 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-05-20T16:25:41.537 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.537 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.537 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.537 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Loki container image", 2024-05-20T16:25:41.537 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.537 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.537 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.537 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.538 INFO:teuthology.orchestra.run.smithi082.stdout: "container_image_node_exporter": { 2024-05-20T16:25:41.538 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "container_image_node_exporter", 2024-05-20T16:25:41.538 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.538 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.538 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.538 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-05-20T16:25:41.538 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.538 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.538 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.538 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Prometheus container image", 2024-05-20T16:25:41.538 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.538 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.538 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.539 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.539 INFO:teuthology.orchestra.run.smithi082.stdout: "container_image_nvmeof": { 2024-05-20T16:25:41.539 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "container_image_nvmeof", 2024-05-20T16:25:41.539 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.539 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.539 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.539 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-05-20T16:25:41.539 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.539 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.539 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.539 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Nvme-of container image", 2024-05-20T16:25:41.539 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.539 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.539 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.539 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.540 INFO:teuthology.orchestra.run.smithi082.stdout: "container_image_prometheus": { 2024-05-20T16:25:41.540 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "container_image_prometheus", 2024-05-20T16:25:41.540 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.540 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.540 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.540 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-05-20T16:25:41.540 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.540 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.540 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.540 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Prometheus container image", 2024-05-20T16:25:41.540 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.540 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.540 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.540 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.541 INFO:teuthology.orchestra.run.smithi082.stdout: "container_image_promtail": { 2024-05-20T16:25:41.541 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "container_image_promtail", 2024-05-20T16:25:41.541 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.541 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.541 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.541 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-05-20T16:25:41.541 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.541 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.541 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.541 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Promtail container image", 2024-05-20T16:25:41.541 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.541 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.541 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.541 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.541 INFO:teuthology.orchestra.run.smithi082.stdout: "container_image_samba": { 2024-05-20T16:25:41.542 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "container_image_samba", 2024-05-20T16:25:41.542 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.542 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.542 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.542 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-05-20T16:25:41.542 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.542 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.542 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.542 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Samba/SMB container image", 2024-05-20T16:25:41.542 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.542 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.542 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.542 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.543 INFO:teuthology.orchestra.run.smithi082.stdout: "container_image_snmp_gateway": { 2024-05-20T16:25:41.543 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "container_image_snmp_gateway", 2024-05-20T16:25:41.543 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.543 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.543 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.543 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-05-20T16:25:41.543 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.543 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.543 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.543 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "SNMP Gateway container image", 2024-05-20T16:25:41.543 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.543 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.543 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.543 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.544 INFO:teuthology.orchestra.run.smithi082.stdout: "container_init": { 2024-05-20T16:25:41.544 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "container_init", 2024-05-20T16:25:41.544 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.544 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.544 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.544 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-05-20T16:25:41.544 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.544 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.544 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.544 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Run podman/docker with `--init`", 2024-05-20T16:25:41.544 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.544 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.544 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.544 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.544 INFO:teuthology.orchestra.run.smithi082.stdout: "daemon_cache_timeout": { 2024-05-20T16:25:41.544 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "daemon_cache_timeout", 2024-05-20T16:25:41.545 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2024-05-20T16:25:41.545 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.545 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.545 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "600", 2024-05-20T16:25:41.545 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.545 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.545 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.545 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-05-20T16:25:41.545 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.545 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.545 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.545 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.545 INFO:teuthology.orchestra.run.smithi082.stdout: "default_cephadm_command_timeout": { 2024-05-20T16:25:41.545 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "default_cephadm_command_timeout", 2024-05-20T16:25:41.545 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-05-20T16:25:41.546 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.546 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.546 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "900", 2024-05-20T16:25:41.546 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.546 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.546 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.546 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-05-20T16:25:41.546 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.546 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.546 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.546 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.546 INFO:teuthology.orchestra.run.smithi082.stdout: "default_registry": { 2024-05-20T16:25:41.546 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "default_registry", 2024-05-20T16:25:41.546 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.546 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.547 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.547 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "docker.io", 2024-05-20T16:25:41.547 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.547 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.547 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.547 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-05-20T16:25:41.547 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.547 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.547 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.547 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.547 INFO:teuthology.orchestra.run.smithi082.stdout: "device_cache_timeout": { 2024-05-20T16:25:41.547 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "device_cache_timeout", 2024-05-20T16:25:41.547 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2024-05-20T16:25:41.547 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.548 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.548 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "1800", 2024-05-20T16:25:41.548 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.548 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.548 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.548 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "seconds to cache device inventory", 2024-05-20T16:25:41.548 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.548 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.548 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.548 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.548 INFO:teuthology.orchestra.run.smithi082.stdout: "device_enhanced_scan": { 2024-05-20T16:25:41.548 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "device_enhanced_scan", 2024-05-20T16:25:41.548 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.548 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.549 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.549 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:41.549 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.549 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.549 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.549 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Use libstoragemgmt during device scans", 2024-05-20T16:25:41.549 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.549 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.549 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.549 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.549 INFO:teuthology.orchestra.run.smithi082.stdout: "facts_cache_timeout": { 2024-05-20T16:25:41.549 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "facts_cache_timeout", 2024-05-20T16:25:41.549 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2024-05-20T16:25:41.550 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.550 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.550 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "60", 2024-05-20T16:25:41.550 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.550 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.550 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.550 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "seconds to cache host facts data", 2024-05-20T16:25:41.550 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.550 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.550 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.550 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.550 INFO:teuthology.orchestra.run.smithi082.stdout: "grafana_dashboards_path": { 2024-05-20T16:25:41.550 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "grafana_dashboards_path", 2024-05-20T16:25:41.551 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.551 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.551 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.551 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-05-20T16:25:41.551 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.551 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.551 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.551 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-05-20T16:25:41.551 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.551 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.551 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.551 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.551 INFO:teuthology.orchestra.run.smithi082.stdout: "host_check_interval": { 2024-05-20T16:25:41.551 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "host_check_interval", 2024-05-20T16:25:41.551 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2024-05-20T16:25:41.552 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.552 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.552 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "600", 2024-05-20T16:25:41.552 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.552 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.552 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.552 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "how frequently to perform a host check", 2024-05-20T16:25:41.552 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.552 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.552 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.552 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.552 INFO:teuthology.orchestra.run.smithi082.stdout: "hw_monitoring": { 2024-05-20T16:25:41.552 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "hw_monitoring", 2024-05-20T16:25:41.552 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.553 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.553 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.553 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:41.553 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.553 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.553 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.553 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-05-20T16:25:41.553 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.553 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.553 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.553 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.553 INFO:teuthology.orchestra.run.smithi082.stdout: "inventory_list_all": { 2024-05-20T16:25:41.553 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "inventory_list_all", 2024-05-20T16:25:41.553 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.553 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.553 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.554 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:41.554 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.554 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.554 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.554 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-05-20T16:25:41.554 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.554 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.554 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.554 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.554 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-05-20T16:25:41.554 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-05-20T16:25:41.554 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.554 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.555 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.555 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:41.555 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.555 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.555 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-05-20T16:25:41.555 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-05-20T16:25:41.555 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-05-20T16:25:41.555 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-05-20T16:25:41.555 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-05-20T16:25:41.555 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-05-20T16:25:41.555 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-05-20T16:25:41.555 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:41.555 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.555 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.555 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.555 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.556 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.556 INFO:teuthology.orchestra.run.smithi082.stdout: "log_refresh_metadata": { 2024-05-20T16:25:41.556 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_refresh_metadata", 2024-05-20T16:25:41.556 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.556 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.556 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.556 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:41.556 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.556 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.556 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.556 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-05-20T16:25:41.556 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.556 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.556 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.556 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.557 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-05-20T16:25:41.557 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-05-20T16:25:41.557 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.557 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.557 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.557 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-05-20T16:25:41.557 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.557 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.557 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.557 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-05-20T16:25:41.557 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.557 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.557 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.558 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.558 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-05-20T16:25:41.558 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-05-20T16:25:41.558 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.558 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.558 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.558 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-05-20T16:25:41.558 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.558 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.558 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-05-20T16:25:41.558 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-05-20T16:25:41.558 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-05-20T16:25:41.558 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-05-20T16:25:41.558 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-05-20T16:25:41.559 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-05-20T16:25:41.559 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-05-20T16:25:41.559 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:41.559 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.559 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.559 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.559 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.559 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.559 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-05-20T16:25:41.559 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-05-20T16:25:41.559 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.559 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.559 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.559 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:41.560 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.560 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.560 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.560 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.560 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.560 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.560 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.560 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.560 INFO:teuthology.orchestra.run.smithi082.stdout: "manage_etc_ceph_ceph_conf": { 2024-05-20T16:25:41.560 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-05-20T16:25:41.560 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.560 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.560 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.560 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:41.560 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.561 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.561 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.561 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-05-20T16:25:41.561 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.561 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.561 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.561 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.561 INFO:teuthology.orchestra.run.smithi082.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-05-20T16:25:41.561 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-05-20T16:25:41.561 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.561 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.561 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.561 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "*", 2024-05-20T16:25:41.561 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.561 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.562 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.562 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-05-20T16:25:41.562 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.562 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.562 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.562 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.562 INFO:teuthology.orchestra.run.smithi082.stdout: "max_count_per_host": { 2024-05-20T16:25:41.562 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "max_count_per_host", 2024-05-20T16:25:41.562 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-05-20T16:25:41.562 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.562 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.562 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "10", 2024-05-20T16:25:41.562 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.563 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.563 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.563 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "max number of daemons per service per host", 2024-05-20T16:25:41.563 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.563 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.563 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.563 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.563 INFO:teuthology.orchestra.run.smithi082.stdout: "max_osd_draining_count": { 2024-05-20T16:25:41.563 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "max_osd_draining_count", 2024-05-20T16:25:41.563 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-05-20T16:25:41.563 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.563 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.563 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "10", 2024-05-20T16:25:41.563 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.564 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.564 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.564 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-05-20T16:25:41.564 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.564 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.564 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.564 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.564 INFO:teuthology.orchestra.run.smithi082.stdout: "migration_current": { 2024-05-20T16:25:41.564 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "migration_current", 2024-05-20T16:25:41.564 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-05-20T16:25:41.564 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.564 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.564 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:41.565 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.565 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.565 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.565 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "internal - do not modify", 2024-05-20T16:25:41.565 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.565 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.565 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.565 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.565 INFO:teuthology.orchestra.run.smithi082.stdout: "mode": { 2024-05-20T16:25:41.565 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "mode", 2024-05-20T16:25:41.565 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.565 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.565 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.565 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "root", 2024-05-20T16:25:41.566 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.566 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.566 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-05-20T16:25:41.566 INFO:teuthology.orchestra.run.smithi082.stdout: "cephadm-package", 2024-05-20T16:25:41.566 INFO:teuthology.orchestra.run.smithi082.stdout: "root" 2024-05-20T16:25:41.566 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:41.566 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "mode for remote execution of cephadm", 2024-05-20T16:25:41.566 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.566 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.566 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.566 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.566 INFO:teuthology.orchestra.run.smithi082.stdout: "oob_default_addr": { 2024-05-20T16:25:41.566 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "oob_default_addr", 2024-05-20T16:25:41.566 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.567 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.567 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.567 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "169.254.1.1", 2024-05-20T16:25:41.567 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.567 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.567 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.567 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Default address for RedFish API (oob management).", 2024-05-20T16:25:41.567 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.567 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.567 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.567 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.567 INFO:teuthology.orchestra.run.smithi082.stdout: "prometheus_alerts_path": { 2024-05-20T16:25:41.567 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "prometheus_alerts_path", 2024-05-20T16:25:41.567 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.568 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.568 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.568 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-05-20T16:25:41.568 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.568 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.568 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.568 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-05-20T16:25:41.568 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.568 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.568 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.568 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.568 INFO:teuthology.orchestra.run.smithi082.stdout: "registry_insecure": { 2024-05-20T16:25:41.568 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "registry_insecure", 2024-05-20T16:25:41.568 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.568 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.569 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.569 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:41.569 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.569 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.569 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.569 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-05-20T16:25:41.569 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.569 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.569 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.569 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.569 INFO:teuthology.orchestra.run.smithi082.stdout: "registry_password": { 2024-05-20T16:25:41.569 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "registry_password", 2024-05-20T16:25:41.569 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.570 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.570 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.570 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:41.570 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.570 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.570 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.570 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-05-20T16:25:41.570 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.570 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.570 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.570 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.570 INFO:teuthology.orchestra.run.smithi082.stdout: "registry_url": { 2024-05-20T16:25:41.570 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "registry_url", 2024-05-20T16:25:41.571 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.571 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.571 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.571 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:41.571 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.571 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.571 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.571 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-05-20T16:25:41.571 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.571 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.571 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.571 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.571 INFO:teuthology.orchestra.run.smithi082.stdout: "registry_username": { 2024-05-20T16:25:41.571 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "registry_username", 2024-05-20T16:25:41.572 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.572 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.572 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.572 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:41.572 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.572 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.572 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.572 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-05-20T16:25:41.572 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.572 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.572 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.572 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.572 INFO:teuthology.orchestra.run.smithi082.stdout: "secure_monitoring_stack": { 2024-05-20T16:25:41.572 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "secure_monitoring_stack", 2024-05-20T16:25:41.573 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.573 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.573 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.573 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:41.573 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.573 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.573 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.573 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-05-20T16:25:41.573 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.573 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.573 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.573 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.573 INFO:teuthology.orchestra.run.smithi082.stdout: "service_discovery_port": { 2024-05-20T16:25:41.573 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "service_discovery_port", 2024-05-20T16:25:41.573 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-05-20T16:25:41.573 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.574 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.574 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "8765", 2024-05-20T16:25:41.574 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.574 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.574 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.574 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "cephadm service discovery port", 2024-05-20T16:25:41.574 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.574 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.574 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.574 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.574 INFO:teuthology.orchestra.run.smithi082.stdout: "ssh_config_file": { 2024-05-20T16:25:41.574 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "ssh_config_file", 2024-05-20T16:25:41.574 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.575 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.575 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.575 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:41.575 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.575 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.575 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.575 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-05-20T16:25:41.575 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.575 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.575 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.575 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.575 INFO:teuthology.orchestra.run.smithi082.stdout: "use_agent": { 2024-05-20T16:25:41.575 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "use_agent", 2024-05-20T16:25:41.575 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.575 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.576 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.576 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:41.576 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.576 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.576 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.576 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-05-20T16:25:41.576 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.576 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.576 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.576 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.576 INFO:teuthology.orchestra.run.smithi082.stdout: "use_repo_digest": { 2024-05-20T16:25:41.576 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "use_repo_digest", 2024-05-20T16:25:41.576 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.576 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.576 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.576 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-05-20T16:25:41.577 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.577 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.577 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.577 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-05-20T16:25:41.577 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.577 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.577 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.577 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.577 INFO:teuthology.orchestra.run.smithi082.stdout: "warn_on_failed_host_check": { 2024-05-20T16:25:41.577 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "warn_on_failed_host_check", 2024-05-20T16:25:41.577 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.577 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.577 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.577 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-05-20T16:25:41.577 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.577 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.578 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.578 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "raise a health warning if the host check fails", 2024-05-20T16:25:41.578 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.578 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.578 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.578 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.578 INFO:teuthology.orchestra.run.smithi082.stdout: "warn_on_stray_daemons": { 2024-05-20T16:25:41.578 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "warn_on_stray_daemons", 2024-05-20T16:25:41.578 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.578 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.578 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.578 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-05-20T16:25:41.578 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.578 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.578 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.578 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-05-20T16:25:41.579 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.579 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.579 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.579 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.579 INFO:teuthology.orchestra.run.smithi082.stdout: "warn_on_stray_hosts": { 2024-05-20T16:25:41.579 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "warn_on_stray_hosts", 2024-05-20T16:25:41.579 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.579 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.579 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.579 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-05-20T16:25:41.579 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.579 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.579 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.579 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-05-20T16:25:41.579 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.580 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.580 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.580 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-20T16:25:41.580 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-20T16:25:41.580 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.580 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-05-20T16:25:41.580 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "crash", 2024-05-20T16:25:41.580 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-05-20T16:25:41.580 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-05-20T16:25:41.580 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-05-20T16:25:41.580 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-05-20T16:25:41.580 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-05-20T16:25:41.580 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.580 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.580 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.581 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:41.581 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.581 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.581 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-05-20T16:25:41.581 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-05-20T16:25:41.581 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-05-20T16:25:41.581 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-05-20T16:25:41.581 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-05-20T16:25:41.581 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-05-20T16:25:41.581 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-05-20T16:25:41.581 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:41.581 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.581 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.582 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.582 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.582 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.582 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-05-20T16:25:41.582 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-05-20T16:25:41.582 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.582 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.582 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.582 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:41.582 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.582 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.582 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.582 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.582 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.583 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.583 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.583 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.583 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-05-20T16:25:41.583 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-05-20T16:25:41.583 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.583 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.583 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.583 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-05-20T16:25:41.583 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.583 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.583 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-05-20T16:25:41.583 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-05-20T16:25:41.583 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-05-20T16:25:41.584 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-05-20T16:25:41.584 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-05-20T16:25:41.584 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-05-20T16:25:41.584 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-05-20T16:25:41.584 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:41.584 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.584 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.584 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.584 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.584 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.584 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-05-20T16:25:41.584 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-05-20T16:25:41.584 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.584 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.585 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.585 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:41.585 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.585 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.585 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.585 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.585 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.585 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.585 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.585 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.585 INFO:teuthology.orchestra.run.smithi082.stdout: "retain_interval": { 2024-05-20T16:25:41.585 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "retain_interval", 2024-05-20T16:25:41.585 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2024-05-20T16:25:41.585 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.585 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.585 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "31536000", 2024-05-20T16:25:41.586 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.586 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.586 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.586 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "how long to retain crashes before pruning them", 2024-05-20T16:25:41.586 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.586 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.586 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.586 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.586 INFO:teuthology.orchestra.run.smithi082.stdout: "warn_recent_interval": { 2024-05-20T16:25:41.586 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "warn_recent_interval", 2024-05-20T16:25:41.586 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2024-05-20T16:25:41.586 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.586 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.587 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "1209600", 2024-05-20T16:25:41.587 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.587 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.587 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.587 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "time interval in which to warn about recent crashes", 2024-05-20T16:25:41.587 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.587 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.587 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.587 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-20T16:25:41.587 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-20T16:25:41.587 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.587 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-05-20T16:25:41.587 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "dashboard", 2024-05-20T16:25:41.587 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-05-20T16:25:41.588 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-05-20T16:25:41.588 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-05-20T16:25:41.588 INFO:teuthology.orchestra.run.smithi082.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-05-20T16:25:41.588 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-05-20T16:25:41.588 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-05-20T16:25:41.588 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.588 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.588 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "10", 2024-05-20T16:25:41.588 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.588 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.588 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.588 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.588 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.588 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.589 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.589 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.589 INFO:teuthology.orchestra.run.smithi082.stdout: "ALERTMANAGER_API_HOST": { 2024-05-20T16:25:41.589 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "ALERTMANAGER_API_HOST", 2024-05-20T16:25:41.589 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.589 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.589 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.589 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:41.589 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.589 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.589 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.589 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.589 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.589 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.589 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.590 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.590 INFO:teuthology.orchestra.run.smithi082.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-05-20T16:25:41.590 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-05-20T16:25:41.590 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.590 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.590 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.590 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-05-20T16:25:41.590 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.590 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.590 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.590 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.590 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.590 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.591 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.591 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.591 INFO:teuthology.orchestra.run.smithi082.stdout: "AUDIT_API_ENABLED": { 2024-05-20T16:25:41.591 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "AUDIT_API_ENABLED", 2024-05-20T16:25:41.591 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.591 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.591 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.591 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:41.591 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.591 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.591 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.591 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.591 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.591 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.591 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.592 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.592 INFO:teuthology.orchestra.run.smithi082.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-05-20T16:25:41.592 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-05-20T16:25:41.592 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.592 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.592 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.592 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-05-20T16:25:41.592 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.592 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.592 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.592 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.592 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.592 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.592 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.593 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.593 INFO:teuthology.orchestra.run.smithi082.stdout: "ENABLE_BROWSABLE_API": { 2024-05-20T16:25:41.593 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "ENABLE_BROWSABLE_API", 2024-05-20T16:25:41.593 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.593 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.593 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.593 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-05-20T16:25:41.593 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.593 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.593 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.593 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.593 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.593 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.593 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.594 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.594 INFO:teuthology.orchestra.run.smithi082.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-05-20T16:25:41.594 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-05-20T16:25:41.594 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.594 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.594 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.594 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-05-20T16:25:41.594 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.594 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.594 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.594 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.594 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.594 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.595 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.595 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.595 INFO:teuthology.orchestra.run.smithi082.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-05-20T16:25:41.595 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-05-20T16:25:41.595 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.595 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.595 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.595 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-05-20T16:25:41.595 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.595 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.595 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.595 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.595 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.595 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.596 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.596 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.596 INFO:teuthology.orchestra.run.smithi082.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-05-20T16:25:41.596 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-05-20T16:25:41.596 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.596 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.596 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.596 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-05-20T16:25:41.596 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.596 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.596 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.596 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.596 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.597 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.597 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.597 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.597 INFO:teuthology.orchestra.run.smithi082.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-05-20T16:25:41.597 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-05-20T16:25:41.597 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.597 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.597 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.597 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-05-20T16:25:41.597 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.597 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.597 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.597 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.598 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.598 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.598 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.598 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.598 INFO:teuthology.orchestra.run.smithi082.stdout: "FEATURE_TOGGLE_NFS": { 2024-05-20T16:25:41.598 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-05-20T16:25:41.598 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.598 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.598 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.598 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-05-20T16:25:41.598 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.598 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.598 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.599 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.599 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.599 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.599 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.599 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.599 INFO:teuthology.orchestra.run.smithi082.stdout: "FEATURE_TOGGLE_RBD": { 2024-05-20T16:25:41.599 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-05-20T16:25:41.599 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.599 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.599 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.599 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-05-20T16:25:41.599 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.599 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.599 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.599 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.600 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.600 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.600 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.600 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.600 INFO:teuthology.orchestra.run.smithi082.stdout: "FEATURE_TOGGLE_RGW": { 2024-05-20T16:25:41.600 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-05-20T16:25:41.600 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.600 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.600 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.600 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-05-20T16:25:41.600 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.600 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.600 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.600 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.600 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.601 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.601 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.601 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.601 INFO:teuthology.orchestra.run.smithi082.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-05-20T16:25:41.601 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-05-20T16:25:41.601 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.601 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.601 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.601 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:41.601 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.601 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.601 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.601 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.601 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.601 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.602 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.602 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.602 INFO:teuthology.orchestra.run.smithi082.stdout: "GRAFANA_API_PASSWORD": { 2024-05-20T16:25:41.602 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "GRAFANA_API_PASSWORD", 2024-05-20T16:25:41.602 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.602 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.602 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.602 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "admin", 2024-05-20T16:25:41.602 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.602 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.602 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.602 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.602 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.602 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.603 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.603 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.603 INFO:teuthology.orchestra.run.smithi082.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-05-20T16:25:41.603 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-05-20T16:25:41.603 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.603 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.603 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.603 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-05-20T16:25:41.603 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.603 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.603 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.603 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.603 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.603 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.604 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.604 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.604 INFO:teuthology.orchestra.run.smithi082.stdout: "GRAFANA_API_URL": { 2024-05-20T16:25:41.604 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "GRAFANA_API_URL", 2024-05-20T16:25:41.604 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.604 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.604 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.604 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:41.604 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.604 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.604 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.604 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.604 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.604 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.605 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.605 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.605 INFO:teuthology.orchestra.run.smithi082.stdout: "GRAFANA_API_USERNAME": { 2024-05-20T16:25:41.605 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "GRAFANA_API_USERNAME", 2024-05-20T16:25:41.605 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.605 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.605 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.605 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "admin", 2024-05-20T16:25:41.605 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.605 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.605 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.605 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.605 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.605 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.606 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.606 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.606 INFO:teuthology.orchestra.run.smithi082.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-05-20T16:25:41.606 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-05-20T16:25:41.606 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.606 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.606 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.606 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:41.606 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.606 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.606 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.606 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.606 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.606 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.607 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.607 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.607 INFO:teuthology.orchestra.run.smithi082.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-05-20T16:25:41.607 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-05-20T16:25:41.607 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.607 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.607 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.607 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:41.607 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.607 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.607 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.607 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.607 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.608 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.608 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.608 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.608 INFO:teuthology.orchestra.run.smithi082.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-05-20T16:25:41.608 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-05-20T16:25:41.608 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.608 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.608 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.608 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-05-20T16:25:41.608 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.608 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.608 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.608 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.609 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.609 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.609 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.609 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.609 INFO:teuthology.orchestra.run.smithi082.stdout: "ISSUE_TRACKER_API_KEY": { 2024-05-20T16:25:41.609 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-05-20T16:25:41.609 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.609 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.609 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.609 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:41.609 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.609 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.609 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.609 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.609 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.610 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.610 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.610 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.610 INFO:teuthology.orchestra.run.smithi082.stdout: "PROMETHEUS_API_HOST": { 2024-05-20T16:25:41.610 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "PROMETHEUS_API_HOST", 2024-05-20T16:25:41.610 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.610 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.610 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.610 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:41.610 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.610 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.610 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.610 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.610 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.611 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.611 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.611 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.611 INFO:teuthology.orchestra.run.smithi082.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-05-20T16:25:41.611 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-05-20T16:25:41.611 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.611 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.611 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.611 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-05-20T16:25:41.611 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.611 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.611 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.611 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.611 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.611 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.612 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.612 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.612 INFO:teuthology.orchestra.run.smithi082.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-05-20T16:25:41.612 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-05-20T16:25:41.612 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.612 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.612 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.612 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:41.612 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.612 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.612 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.612 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.612 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.612 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.613 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.613 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.613 INFO:teuthology.orchestra.run.smithi082.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-05-20T16:25:41.613 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-05-20T16:25:41.613 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.613 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.613 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.613 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:41.613 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.613 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.613 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.613 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.613 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.613 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.614 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.614 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.614 INFO:teuthology.orchestra.run.smithi082.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-05-20T16:25:41.614 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-05-20T16:25:41.614 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.614 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.614 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.614 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-05-20T16:25:41.614 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.614 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.614 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.614 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.614 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.614 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.615 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.615 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.615 INFO:teuthology.orchestra.run.smithi082.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-05-20T16:25:41.615 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-05-20T16:25:41.615 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.615 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.615 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.615 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-05-20T16:25:41.615 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.615 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.615 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.615 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.615 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.615 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.616 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.616 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.616 INFO:teuthology.orchestra.run.smithi082.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-05-20T16:25:41.616 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-05-20T16:25:41.616 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.616 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.616 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.616 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:41.616 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.616 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.616 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.616 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.616 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.616 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.617 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.617 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.617 INFO:teuthology.orchestra.run.smithi082.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-05-20T16:25:41.617 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-05-20T16:25:41.617 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.617 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.617 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.617 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:41.617 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.617 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.617 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.617 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.617 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.617 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.618 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.618 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.618 INFO:teuthology.orchestra.run.smithi082.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-05-20T16:25:41.618 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-05-20T16:25:41.618 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.618 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.618 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.618 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:41.618 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.618 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.618 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.618 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.618 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.618 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.618 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.619 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.619 INFO:teuthology.orchestra.run.smithi082.stdout: "PWD_POLICY_ENABLED": { 2024-05-20T16:25:41.619 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "PWD_POLICY_ENABLED", 2024-05-20T16:25:41.619 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.619 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.619 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.619 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-05-20T16:25:41.619 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.619 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.619 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.619 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.619 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.619 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.619 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.619 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.620 INFO:teuthology.orchestra.run.smithi082.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-05-20T16:25:41.620 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-05-20T16:25:41.620 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.620 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.620 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.620 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-05-20T16:25:41.620 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.620 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.620 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.620 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.620 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.620 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.620 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.620 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.620 INFO:teuthology.orchestra.run.smithi082.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-05-20T16:25:41.621 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-05-20T16:25:41.621 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-05-20T16:25:41.621 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.621 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.621 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "10", 2024-05-20T16:25:41.621 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.621 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.621 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.621 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.621 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.621 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.621 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.621 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.621 INFO:teuthology.orchestra.run.smithi082.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-05-20T16:25:41.622 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-05-20T16:25:41.622 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-05-20T16:25:41.622 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.622 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.622 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "8", 2024-05-20T16:25:41.622 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.622 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.622 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.622 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.622 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.622 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.622 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.622 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.623 INFO:teuthology.orchestra.run.smithi082.stdout: "REST_REQUESTS_TIMEOUT": { 2024-05-20T16:25:41.623 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-05-20T16:25:41.623 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-05-20T16:25:41.623 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.623 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.623 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "45", 2024-05-20T16:25:41.623 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.623 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.623 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.623 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.623 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.623 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.623 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.623 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.623 INFO:teuthology.orchestra.run.smithi082.stdout: "RGW_API_ACCESS_KEY": { 2024-05-20T16:25:41.624 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "RGW_API_ACCESS_KEY", 2024-05-20T16:25:41.624 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.624 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.624 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.624 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:41.624 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.624 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.624 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.624 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.624 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.624 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.624 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.624 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.624 INFO:teuthology.orchestra.run.smithi082.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-05-20T16:25:41.624 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-05-20T16:25:41.624 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.625 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.625 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.625 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "admin", 2024-05-20T16:25:41.625 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.625 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.625 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.625 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.625 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.625 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.625 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.625 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.625 INFO:teuthology.orchestra.run.smithi082.stdout: "RGW_API_SECRET_KEY": { 2024-05-20T16:25:41.625 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "RGW_API_SECRET_KEY", 2024-05-20T16:25:41.625 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.625 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.626 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.626 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:41.626 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.626 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.626 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.626 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.626 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.626 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.626 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.626 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.626 INFO:teuthology.orchestra.run.smithi082.stdout: "RGW_API_SSL_VERIFY": { 2024-05-20T16:25:41.626 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "RGW_API_SSL_VERIFY", 2024-05-20T16:25:41.626 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.626 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.627 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.627 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-05-20T16:25:41.627 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.627 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.627 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.627 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.627 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.627 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.627 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.627 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.627 INFO:teuthology.orchestra.run.smithi082.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-05-20T16:25:41.627 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-05-20T16:25:41.627 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-05-20T16:25:41.627 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.628 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.628 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "0", 2024-05-20T16:25:41.628 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.628 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.628 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.628 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.628 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.628 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.628 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.628 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.628 INFO:teuthology.orchestra.run.smithi082.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-05-20T16:25:41.628 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-05-20T16:25:41.628 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-05-20T16:25:41.628 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.628 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.628 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "10", 2024-05-20T16:25:41.629 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.629 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.629 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.629 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.629 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.629 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.629 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.629 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.629 INFO:teuthology.orchestra.run.smithi082.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-05-20T16:25:41.629 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-05-20T16:25:41.629 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-05-20T16:25:41.629 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.629 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.629 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "5", 2024-05-20T16:25:41.630 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.630 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.630 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.630 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.630 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.630 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.630 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.630 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.630 INFO:teuthology.orchestra.run.smithi082.stdout: "cross_origin_url": { 2024-05-20T16:25:41.630 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "cross_origin_url", 2024-05-20T16:25:41.630 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.630 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.630 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.630 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:41.631 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.631 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.631 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.631 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.631 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.631 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.631 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.631 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.631 INFO:teuthology.orchestra.run.smithi082.stdout: "crt_file": { 2024-05-20T16:25:41.631 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "crt_file", 2024-05-20T16:25:41.631 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.631 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.631 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.631 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:41.631 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.631 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.632 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.632 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.632 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.632 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.632 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.632 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.632 INFO:teuthology.orchestra.run.smithi082.stdout: "debug": { 2024-05-20T16:25:41.632 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "debug", 2024-05-20T16:25:41.632 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.632 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.632 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.632 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:41.632 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.633 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.633 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.633 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Enable/disable debug options", 2024-05-20T16:25:41.633 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.633 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.633 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.633 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.633 INFO:teuthology.orchestra.run.smithi082.stdout: "jwt_token_ttl": { 2024-05-20T16:25:41.633 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "jwt_token_ttl", 2024-05-20T16:25:41.633 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-05-20T16:25:41.633 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.633 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.633 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "28800", 2024-05-20T16:25:41.633 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.634 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.634 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.634 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.634 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.634 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.634 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.634 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.634 INFO:teuthology.orchestra.run.smithi082.stdout: "key_file": { 2024-05-20T16:25:41.634 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "key_file", 2024-05-20T16:25:41.634 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.634 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.634 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.634 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:41.634 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.635 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.635 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.635 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.635 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.635 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.635 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.635 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.635 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-05-20T16:25:41.635 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-05-20T16:25:41.635 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.635 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.635 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.635 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:41.635 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.635 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.635 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-05-20T16:25:41.636 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-05-20T16:25:41.636 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-05-20T16:25:41.636 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-05-20T16:25:41.636 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-05-20T16:25:41.636 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-05-20T16:25:41.636 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-05-20T16:25:41.636 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:41.636 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.636 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.636 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.636 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.636 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.636 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-05-20T16:25:41.636 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-05-20T16:25:41.636 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.636 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.637 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.637 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:41.637 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.637 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.637 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.637 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.637 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.637 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.637 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.637 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.637 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-05-20T16:25:41.637 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-05-20T16:25:41.638 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.638 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.638 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.638 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-05-20T16:25:41.638 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.638 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.638 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-05-20T16:25:41.638 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-05-20T16:25:41.638 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-05-20T16:25:41.638 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-05-20T16:25:41.638 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-05-20T16:25:41.638 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-05-20T16:25:41.638 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-05-20T16:25:41.639 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:41.639 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.639 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.639 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.639 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.639 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.639 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-05-20T16:25:41.639 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-05-20T16:25:41.639 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.639 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.639 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.639 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:41.639 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.639 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.640 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.640 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.640 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.640 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.640 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.640 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.640 INFO:teuthology.orchestra.run.smithi082.stdout: "motd": { 2024-05-20T16:25:41.640 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "motd", 2024-05-20T16:25:41.640 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.640 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.640 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.640 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:41.640 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.640 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.641 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.641 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "The message of the day", 2024-05-20T16:25:41.641 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.641 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.641 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.641 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.641 INFO:teuthology.orchestra.run.smithi082.stdout: "redirect_resolve_ip_addr": { 2024-05-20T16:25:41.641 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "redirect_resolve_ip_addr", 2024-05-20T16:25:41.641 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.641 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.641 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.641 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:41.641 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.641 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.642 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.642 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.642 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.642 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.642 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.642 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.642 INFO:teuthology.orchestra.run.smithi082.stdout: "server_addr": { 2024-05-20T16:25:41.642 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "server_addr", 2024-05-20T16:25:41.642 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.642 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.642 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.642 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "::", 2024-05-20T16:25:41.642 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.642 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.643 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.643 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.643 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.643 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.643 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.643 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.643 INFO:teuthology.orchestra.run.smithi082.stdout: "server_port": { 2024-05-20T16:25:41.643 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "server_port", 2024-05-20T16:25:41.643 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-05-20T16:25:41.643 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.643 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.643 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "8080", 2024-05-20T16:25:41.643 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.643 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.644 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.644 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.644 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.644 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.644 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.644 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.644 INFO:teuthology.orchestra.run.smithi082.stdout: "ssl": { 2024-05-20T16:25:41.644 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "ssl", 2024-05-20T16:25:41.644 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.644 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.644 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.644 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-05-20T16:25:41.644 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.644 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.644 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.645 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.645 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.645 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.645 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.645 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.645 INFO:teuthology.orchestra.run.smithi082.stdout: "ssl_server_port": { 2024-05-20T16:25:41.645 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "ssl_server_port", 2024-05-20T16:25:41.645 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-05-20T16:25:41.645 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.645 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.645 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "8443", 2024-05-20T16:25:41.645 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.645 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.645 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.645 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.645 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.646 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.646 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.646 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.646 INFO:teuthology.orchestra.run.smithi082.stdout: "standby_behaviour": { 2024-05-20T16:25:41.646 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "standby_behaviour", 2024-05-20T16:25:41.646 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.646 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.646 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.646 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "redirect", 2024-05-20T16:25:41.646 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.646 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.646 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-05-20T16:25:41.646 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-05-20T16:25:41.647 INFO:teuthology.orchestra.run.smithi082.stdout: "redirect" 2024-05-20T16:25:41.647 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:41.647 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.647 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.647 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.647 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.647 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.647 INFO:teuthology.orchestra.run.smithi082.stdout: "standby_error_status_code": { 2024-05-20T16:25:41.647 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "standby_error_status_code", 2024-05-20T16:25:41.647 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-05-20T16:25:41.647 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.647 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.647 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "500", 2024-05-20T16:25:41.647 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "400", 2024-05-20T16:25:41.648 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "599", 2024-05-20T16:25:41.648 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.648 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.648 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.648 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.648 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.648 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.648 INFO:teuthology.orchestra.run.smithi082.stdout: "url_prefix": { 2024-05-20T16:25:41.648 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "url_prefix", 2024-05-20T16:25:41.648 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.648 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.648 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.648 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:41.648 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.649 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.649 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.649 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.649 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.649 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.649 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.649 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-20T16:25:41.649 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-20T16:25:41.649 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.649 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-05-20T16:25:41.649 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "devicehealth", 2024-05-20T16:25:41.649 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-05-20T16:25:41.649 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-05-20T16:25:41.649 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-05-20T16:25:41.649 INFO:teuthology.orchestra.run.smithi082.stdout: "enable_monitoring": { 2024-05-20T16:25:41.650 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "enable_monitoring", 2024-05-20T16:25:41.650 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.650 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.650 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.650 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-05-20T16:25:41.650 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.650 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.650 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.650 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "monitor device health metrics", 2024-05-20T16:25:41.650 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.650 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.650 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.650 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.650 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-05-20T16:25:41.651 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-05-20T16:25:41.651 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.651 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.651 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.651 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:41.651 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.651 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.651 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-05-20T16:25:41.651 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-05-20T16:25:41.651 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-05-20T16:25:41.651 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-05-20T16:25:41.651 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-05-20T16:25:41.651 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-05-20T16:25:41.651 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-05-20T16:25:41.652 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:41.652 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.652 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.652 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.652 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.652 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.652 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-05-20T16:25:41.652 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-05-20T16:25:41.652 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.652 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.652 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.652 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:41.652 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.653 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.653 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.653 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.653 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.653 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.653 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.653 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.653 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-05-20T16:25:41.653 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-05-20T16:25:41.653 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.653 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.653 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.653 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-05-20T16:25:41.653 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.654 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.654 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-05-20T16:25:41.654 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-05-20T16:25:41.654 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-05-20T16:25:41.654 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-05-20T16:25:41.654 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-05-20T16:25:41.654 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-05-20T16:25:41.654 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-05-20T16:25:41.654 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:41.654 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.654 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.654 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.654 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.654 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.654 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-05-20T16:25:41.655 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-05-20T16:25:41.655 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.655 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.655 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.655 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:41.655 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.655 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.655 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.655 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.655 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.655 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.655 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.655 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.655 INFO:teuthology.orchestra.run.smithi082.stdout: "mark_out_threshold": { 2024-05-20T16:25:41.656 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "mark_out_threshold", 2024-05-20T16:25:41.656 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2024-05-20T16:25:41.656 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.656 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.656 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "2419200", 2024-05-20T16:25:41.656 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.656 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.656 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.656 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-05-20T16:25:41.656 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.656 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.656 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.656 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.656 INFO:teuthology.orchestra.run.smithi082.stdout: "pool_name": { 2024-05-20T16:25:41.656 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "pool_name", 2024-05-20T16:25:41.657 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.657 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.657 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.657 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "device_health_metrics", 2024-05-20T16:25:41.657 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.657 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.657 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.657 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "name of pool in which to store device health metrics", 2024-05-20T16:25:41.657 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.657 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.657 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.657 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.657 INFO:teuthology.orchestra.run.smithi082.stdout: "retention_period": { 2024-05-20T16:25:41.657 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "retention_period", 2024-05-20T16:25:41.657 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2024-05-20T16:25:41.658 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.658 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.658 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "15552000", 2024-05-20T16:25:41.658 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.658 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.658 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.658 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "how long to retain device health metrics", 2024-05-20T16:25:41.658 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.658 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.658 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.658 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.658 INFO:teuthology.orchestra.run.smithi082.stdout: "scrape_frequency": { 2024-05-20T16:25:41.658 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "scrape_frequency", 2024-05-20T16:25:41.658 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2024-05-20T16:25:41.659 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.659 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.659 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "86400", 2024-05-20T16:25:41.659 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.659 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.659 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.659 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "how frequently to scrape device health metrics", 2024-05-20T16:25:41.659 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.659 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.659 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.659 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.659 INFO:teuthology.orchestra.run.smithi082.stdout: "self_heal": { 2024-05-20T16:25:41.659 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "self_heal", 2024-05-20T16:25:41.659 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.659 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.660 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.660 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-05-20T16:25:41.660 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.660 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.660 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.660 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-05-20T16:25:41.660 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.660 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.660 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.660 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.660 INFO:teuthology.orchestra.run.smithi082.stdout: "sleep_interval": { 2024-05-20T16:25:41.660 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "sleep_interval", 2024-05-20T16:25:41.660 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2024-05-20T16:25:41.660 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.661 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.661 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "600", 2024-05-20T16:25:41.661 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.661 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.661 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.661 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "how frequently to wake up and check device health", 2024-05-20T16:25:41.661 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.661 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.661 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.661 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.661 INFO:teuthology.orchestra.run.smithi082.stdout: "warn_threshold": { 2024-05-20T16:25:41.661 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "warn_threshold", 2024-05-20T16:25:41.661 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2024-05-20T16:25:41.661 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.661 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.662 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "7257600", 2024-05-20T16:25:41.662 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.662 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.662 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.662 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-05-20T16:25:41.662 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.662 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.662 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.662 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-20T16:25:41.662 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-20T16:25:41.662 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.662 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-05-20T16:25:41.663 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "diskprediction_local", 2024-05-20T16:25:41.663 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-05-20T16:25:41.663 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-05-20T16:25:41.663 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-05-20T16:25:41.663 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-05-20T16:25:41.663 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-05-20T16:25:41.663 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.663 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.663 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.663 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:41.663 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.663 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.663 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-05-20T16:25:41.663 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-05-20T16:25:41.664 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-05-20T16:25:41.664 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-05-20T16:25:41.664 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-05-20T16:25:41.664 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-05-20T16:25:41.664 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-05-20T16:25:41.664 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:41.664 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.664 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.664 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.664 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.664 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.664 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-05-20T16:25:41.664 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-05-20T16:25:41.665 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.665 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.665 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.665 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:41.665 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.665 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.665 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.665 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.665 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.665 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.665 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.665 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.665 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-05-20T16:25:41.665 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-05-20T16:25:41.666 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.666 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.666 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.666 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-05-20T16:25:41.666 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.666 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.666 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-05-20T16:25:41.666 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-05-20T16:25:41.666 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-05-20T16:25:41.666 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-05-20T16:25:41.666 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-05-20T16:25:41.666 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-05-20T16:25:41.666 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-05-20T16:25:41.666 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:41.667 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.667 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.667 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.667 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.667 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.667 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-05-20T16:25:41.667 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-05-20T16:25:41.667 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.667 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.667 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.667 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:41.667 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.667 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.667 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.667 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.667 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.668 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.668 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.668 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.668 INFO:teuthology.orchestra.run.smithi082.stdout: "predict_interval": { 2024-05-20T16:25:41.668 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "predict_interval", 2024-05-20T16:25:41.668 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.668 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.668 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.668 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "86400", 2024-05-20T16:25:41.668 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.668 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.668 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.668 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.668 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.668 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.669 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.669 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.669 INFO:teuthology.orchestra.run.smithi082.stdout: "predictor_model": { 2024-05-20T16:25:41.669 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "predictor_model", 2024-05-20T16:25:41.669 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.669 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.669 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.669 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "prophetstor", 2024-05-20T16:25:41.669 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.669 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.669 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.669 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.669 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.669 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.669 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.669 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.669 INFO:teuthology.orchestra.run.smithi082.stdout: "sleep_interval": { 2024-05-20T16:25:41.670 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "sleep_interval", 2024-05-20T16:25:41.670 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.670 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.670 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.670 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "600", 2024-05-20T16:25:41.670 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.670 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.670 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.670 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.670 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.670 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.670 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.670 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-20T16:25:41.670 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-20T16:25:41.671 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.671 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-05-20T16:25:41.671 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "influx", 2024-05-20T16:25:41.671 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": false, 2024-05-20T16:25:41.671 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "influxdb python module not found", 2024-05-20T16:25:41.671 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-05-20T16:25:41.671 INFO:teuthology.orchestra.run.smithi082.stdout: "batch_size": { 2024-05-20T16:25:41.671 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "batch_size", 2024-05-20T16:25:41.671 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-05-20T16:25:41.671 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.671 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.671 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "5000", 2024-05-20T16:25:41.671 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.671 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.671 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.671 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-05-20T16:25:41.672 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.672 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.672 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.672 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.672 INFO:teuthology.orchestra.run.smithi082.stdout: "database": { 2024-05-20T16:25:41.672 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "database", 2024-05-20T16:25:41.672 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.672 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.672 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.672 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "ceph", 2024-05-20T16:25:41.672 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.672 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.672 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.672 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-05-20T16:25:41.672 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.673 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.673 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.673 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.673 INFO:teuthology.orchestra.run.smithi082.stdout: "hostname": { 2024-05-20T16:25:41.673 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "hostname", 2024-05-20T16:25:41.673 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.673 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.673 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.673 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:41.673 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.673 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.673 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.673 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "InfluxDB server hostname", 2024-05-20T16:25:41.673 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.673 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.674 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.674 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.674 INFO:teuthology.orchestra.run.smithi082.stdout: "interval": { 2024-05-20T16:25:41.674 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "interval", 2024-05-20T16:25:41.674 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2024-05-20T16:25:41.674 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.674 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.674 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "30", 2024-05-20T16:25:41.674 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "5", 2024-05-20T16:25:41.674 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.674 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.674 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-05-20T16:25:41.674 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.674 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.675 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.675 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.675 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-05-20T16:25:41.675 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-05-20T16:25:41.675 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.675 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.675 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.675 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:41.675 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.675 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.675 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-05-20T16:25:41.675 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-05-20T16:25:41.675 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-05-20T16:25:41.675 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-05-20T16:25:41.676 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-05-20T16:25:41.676 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-05-20T16:25:41.676 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-05-20T16:25:41.676 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:41.676 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.676 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.676 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.676 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.676 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.676 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-05-20T16:25:41.676 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-05-20T16:25:41.676 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.676 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.676 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.676 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:41.676 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.677 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.677 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.677 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.677 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.677 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.677 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.677 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.677 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-05-20T16:25:41.677 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-05-20T16:25:41.677 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.677 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.677 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.677 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-05-20T16:25:41.677 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.678 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.678 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-05-20T16:25:41.678 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-05-20T16:25:41.678 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-05-20T16:25:41.678 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-05-20T16:25:41.678 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-05-20T16:25:41.678 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-05-20T16:25:41.678 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-05-20T16:25:41.678 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:41.678 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.678 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.678 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.678 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.678 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.679 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-05-20T16:25:41.679 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-05-20T16:25:41.679 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.679 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.679 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.679 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:41.679 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.679 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.679 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.679 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.679 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.679 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.679 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.679 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.679 INFO:teuthology.orchestra.run.smithi082.stdout: "password": { 2024-05-20T16:25:41.679 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "password", 2024-05-20T16:25:41.679 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.680 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.680 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.680 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:41.680 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.680 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.680 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.680 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "password of InfluxDB server user", 2024-05-20T16:25:41.680 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.680 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.680 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.680 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.680 INFO:teuthology.orchestra.run.smithi082.stdout: "port": { 2024-05-20T16:25:41.680 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "port", 2024-05-20T16:25:41.680 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-05-20T16:25:41.680 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.680 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.681 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "8086", 2024-05-20T16:25:41.681 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.681 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.681 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.681 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "InfluxDB server port", 2024-05-20T16:25:41.681 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.681 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.681 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.681 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.681 INFO:teuthology.orchestra.run.smithi082.stdout: "ssl": { 2024-05-20T16:25:41.681 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "ssl", 2024-05-20T16:25:41.681 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.681 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.681 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.681 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "false", 2024-05-20T16:25:41.681 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.682 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.682 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.682 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-05-20T16:25:41.682 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.682 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.682 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.682 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.682 INFO:teuthology.orchestra.run.smithi082.stdout: "threads": { 2024-05-20T16:25:41.682 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "threads", 2024-05-20T16:25:41.682 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-05-20T16:25:41.682 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.682 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.682 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "5", 2024-05-20T16:25:41.683 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "1", 2024-05-20T16:25:41.683 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "32", 2024-05-20T16:25:41.683 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.683 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-05-20T16:25:41.683 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.683 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.683 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.683 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.683 INFO:teuthology.orchestra.run.smithi082.stdout: "username": { 2024-05-20T16:25:41.683 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "username", 2024-05-20T16:25:41.683 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.683 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.683 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.683 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:41.684 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.684 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.684 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.684 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "username of InfluxDB server user", 2024-05-20T16:25:41.684 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.684 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.684 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.684 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.684 INFO:teuthology.orchestra.run.smithi082.stdout: "verify_ssl": { 2024-05-20T16:25:41.684 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "verify_ssl", 2024-05-20T16:25:41.684 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.684 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.684 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.684 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "true", 2024-05-20T16:25:41.684 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.685 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.685 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.685 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-05-20T16:25:41.685 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.685 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.685 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.685 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-20T16:25:41.685 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-20T16:25:41.685 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.685 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-05-20T16:25:41.685 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "insights", 2024-05-20T16:25:41.685 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-05-20T16:25:41.685 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-05-20T16:25:41.685 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-05-20T16:25:41.685 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-05-20T16:25:41.686 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-05-20T16:25:41.686 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.686 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.686 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.686 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:41.686 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.686 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.686 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-05-20T16:25:41.686 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-05-20T16:25:41.686 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-05-20T16:25:41.686 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-05-20T16:25:41.686 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-05-20T16:25:41.686 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-05-20T16:25:41.686 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-05-20T16:25:41.686 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:41.687 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.687 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.687 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.687 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.687 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.687 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-05-20T16:25:41.687 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-05-20T16:25:41.687 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.687 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.687 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.687 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:41.687 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.687 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.687 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.687 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.687 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.688 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.688 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.688 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.688 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-05-20T16:25:41.688 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-05-20T16:25:41.688 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.688 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.688 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.688 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-05-20T16:25:41.688 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.688 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.688 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-05-20T16:25:41.688 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-05-20T16:25:41.688 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-05-20T16:25:41.689 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-05-20T16:25:41.689 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-05-20T16:25:41.689 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-05-20T16:25:41.689 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-05-20T16:25:41.689 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:41.689 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.689 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.689 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.689 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.689 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.689 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-05-20T16:25:41.689 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-05-20T16:25:41.689 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.689 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.689 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.689 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:41.690 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.690 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.690 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.690 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.690 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.690 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.690 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.690 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-20T16:25:41.690 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-20T16:25:41.690 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.690 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-05-20T16:25:41.690 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "iostat", 2024-05-20T16:25:41.690 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-05-20T16:25:41.690 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-05-20T16:25:41.690 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-05-20T16:25:41.691 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-05-20T16:25:41.691 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-05-20T16:25:41.691 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.691 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.691 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.691 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:41.691 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.691 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.691 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-05-20T16:25:41.691 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-05-20T16:25:41.691 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-05-20T16:25:41.691 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-05-20T16:25:41.691 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-05-20T16:25:41.691 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-05-20T16:25:41.691 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-05-20T16:25:41.691 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:41.692 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.692 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.692 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.692 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.692 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.692 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-05-20T16:25:41.692 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-05-20T16:25:41.692 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.692 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.692 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.692 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:41.692 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.692 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.692 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.692 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.692 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.693 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.693 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.693 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.693 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-05-20T16:25:41.693 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-05-20T16:25:41.693 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.693 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.693 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.693 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-05-20T16:25:41.693 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.693 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.693 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-05-20T16:25:41.693 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-05-20T16:25:41.694 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-05-20T16:25:41.694 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-05-20T16:25:41.694 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-05-20T16:25:41.694 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-05-20T16:25:41.694 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-05-20T16:25:41.694 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:41.694 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.694 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.694 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.694 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.694 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.694 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-05-20T16:25:41.694 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-05-20T16:25:41.695 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.695 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.695 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.695 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:41.695 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.695 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.695 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.695 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.695 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.695 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.695 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.695 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-20T16:25:41.695 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-20T16:25:41.695 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.695 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-05-20T16:25:41.696 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "k8sevents", 2024-05-20T16:25:41.696 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-05-20T16:25:41.696 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-05-20T16:25:41.696 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-05-20T16:25:41.696 INFO:teuthology.orchestra.run.smithi082.stdout: "ceph_event_retention_days": { 2024-05-20T16:25:41.696 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "ceph_event_retention_days", 2024-05-20T16:25:41.696 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-05-20T16:25:41.696 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.696 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.696 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "7", 2024-05-20T16:25:41.696 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.696 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.696 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.696 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Days to hold ceph event information within local cache", 2024-05-20T16:25:41.696 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.697 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.697 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.697 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.697 INFO:teuthology.orchestra.run.smithi082.stdout: "config_check_secs": { 2024-05-20T16:25:41.697 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "config_check_secs", 2024-05-20T16:25:41.697 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-05-20T16:25:41.697 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.697 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.697 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "10", 2024-05-20T16:25:41.697 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "10", 2024-05-20T16:25:41.697 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.697 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.697 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-05-20T16:25:41.697 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.697 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.697 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.698 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.698 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-05-20T16:25:41.698 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-05-20T16:25:41.698 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.698 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.698 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.698 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:41.698 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.698 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.698 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-05-20T16:25:41.698 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-05-20T16:25:41.698 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-05-20T16:25:41.698 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-05-20T16:25:41.698 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-05-20T16:25:41.699 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-05-20T16:25:41.699 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-05-20T16:25:41.699 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:41.699 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.699 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.699 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.699 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.699 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.699 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-05-20T16:25:41.699 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-05-20T16:25:41.699 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.699 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.699 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.699 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:41.699 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.700 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.700 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.700 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.700 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.700 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.700 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.700 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.700 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-05-20T16:25:41.700 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-05-20T16:25:41.700 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.700 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.700 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.700 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-05-20T16:25:41.700 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.700 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.700 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-05-20T16:25:41.701 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-05-20T16:25:41.701 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-05-20T16:25:41.701 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-05-20T16:25:41.701 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-05-20T16:25:41.701 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-05-20T16:25:41.701 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-05-20T16:25:41.701 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:41.701 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.701 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.701 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.701 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.701 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.701 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-05-20T16:25:41.701 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-05-20T16:25:41.702 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.702 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.702 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.702 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:41.702 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.702 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.702 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.702 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.702 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.702 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.702 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.702 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-20T16:25:41.702 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-20T16:25:41.702 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.703 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-05-20T16:25:41.703 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "localpool", 2024-05-20T16:25:41.703 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-05-20T16:25:41.703 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-05-20T16:25:41.703 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-05-20T16:25:41.703 INFO:teuthology.orchestra.run.smithi082.stdout: "failure_domain": { 2024-05-20T16:25:41.703 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "failure_domain", 2024-05-20T16:25:41.703 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.703 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.703 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.703 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "host", 2024-05-20T16:25:41.703 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.703 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.703 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.703 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "failure domain for any created local pool", 2024-05-20T16:25:41.704 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-05-20T16:25:41.704 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.704 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.704 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.704 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-05-20T16:25:41.704 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-05-20T16:25:41.704 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.704 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.704 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.704 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:41.704 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.704 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.704 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-05-20T16:25:41.704 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-05-20T16:25:41.705 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-05-20T16:25:41.705 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-05-20T16:25:41.705 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-05-20T16:25:41.705 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-05-20T16:25:41.705 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-05-20T16:25:41.705 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:41.705 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.705 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.705 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.705 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.705 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.705 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-05-20T16:25:41.705 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-05-20T16:25:41.705 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.705 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.706 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.706 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:41.706 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.706 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.706 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.706 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.706 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.706 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.706 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.706 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.706 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-05-20T16:25:41.706 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-05-20T16:25:41.706 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.707 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.707 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.707 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-05-20T16:25:41.707 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.707 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.707 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-05-20T16:25:41.707 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-05-20T16:25:41.707 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-05-20T16:25:41.707 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-05-20T16:25:41.707 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-05-20T16:25:41.707 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-05-20T16:25:41.707 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-05-20T16:25:41.707 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:41.707 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.708 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.708 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.708 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.708 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.708 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-05-20T16:25:41.708 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-05-20T16:25:41.708 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.708 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.708 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.708 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:41.708 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.708 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.708 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.708 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.708 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.709 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.709 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.709 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.709 INFO:teuthology.orchestra.run.smithi082.stdout: "min_size": { 2024-05-20T16:25:41.709 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "min_size", 2024-05-20T16:25:41.709 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-05-20T16:25:41.709 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.709 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.709 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:41.709 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.709 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.709 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.709 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "default min_size for any created local pool", 2024-05-20T16:25:41.709 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-05-20T16:25:41.710 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.710 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.710 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.710 INFO:teuthology.orchestra.run.smithi082.stdout: "num_rep": { 2024-05-20T16:25:41.710 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "num_rep", 2024-05-20T16:25:41.710 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-05-20T16:25:41.710 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.710 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.710 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "3", 2024-05-20T16:25:41.710 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.710 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.710 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.710 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "default replica count for any created local pool", 2024-05-20T16:25:41.710 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.711 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.711 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.711 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.711 INFO:teuthology.orchestra.run.smithi082.stdout: "pg_num": { 2024-05-20T16:25:41.711 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "pg_num", 2024-05-20T16:25:41.711 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-05-20T16:25:41.711 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.711 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.711 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "128", 2024-05-20T16:25:41.711 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.712 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.712 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.712 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "default pg_num for any created local pool", 2024-05-20T16:25:41.712 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.712 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.712 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.712 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.712 INFO:teuthology.orchestra.run.smithi082.stdout: "prefix": { 2024-05-20T16:25:41.712 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "prefix", 2024-05-20T16:25:41.712 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.712 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.712 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.712 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:41.713 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.713 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.713 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.713 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "name prefix for any created local pool", 2024-05-20T16:25:41.713 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.713 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.713 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.713 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.713 INFO:teuthology.orchestra.run.smithi082.stdout: "subtree": { 2024-05-20T16:25:41.713 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "subtree", 2024-05-20T16:25:41.713 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.713 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.713 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.713 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "rack", 2024-05-20T16:25:41.713 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.714 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.714 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.714 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "CRUSH level for which to create a local pool", 2024-05-20T16:25:41.714 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-05-20T16:25:41.714 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.714 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.714 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-20T16:25:41.714 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-20T16:25:41.714 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.714 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-05-20T16:25:41.714 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "mds_autoscaler", 2024-05-20T16:25:41.714 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-05-20T16:25:41.714 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-05-20T16:25:41.715 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-05-20T16:25:41.715 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-05-20T16:25:41.715 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-05-20T16:25:41.715 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.715 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.715 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.715 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:41.715 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.715 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.715 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-05-20T16:25:41.715 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-05-20T16:25:41.715 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-05-20T16:25:41.715 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-05-20T16:25:41.715 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-05-20T16:25:41.715 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-05-20T16:25:41.716 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-05-20T16:25:41.716 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:41.716 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.716 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.716 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.716 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.716 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.716 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-05-20T16:25:41.716 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-05-20T16:25:41.716 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.716 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.716 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.716 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:41.716 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.717 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.717 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.717 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.717 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.717 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.717 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.717 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.717 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-05-20T16:25:41.717 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-05-20T16:25:41.717 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.717 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.717 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.717 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-05-20T16:25:41.718 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.718 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.718 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-05-20T16:25:41.718 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-05-20T16:25:41.718 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-05-20T16:25:41.718 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-05-20T16:25:41.718 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-05-20T16:25:41.718 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-05-20T16:25:41.718 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-05-20T16:25:41.718 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:41.718 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.718 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.718 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.718 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.719 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.719 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-05-20T16:25:41.719 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-05-20T16:25:41.719 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.719 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.719 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.719 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:41.719 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.719 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.719 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.719 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.719 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.719 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.719 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.720 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-20T16:25:41.720 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-20T16:25:41.720 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.720 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-05-20T16:25:41.720 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "mirroring", 2024-05-20T16:25:41.720 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-05-20T16:25:41.720 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-05-20T16:25:41.720 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-05-20T16:25:41.720 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-05-20T16:25:41.720 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-05-20T16:25:41.720 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.720 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.720 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.720 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:41.721 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.721 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.721 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-05-20T16:25:41.721 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-05-20T16:25:41.721 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-05-20T16:25:41.721 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-05-20T16:25:41.721 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-05-20T16:25:41.721 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-05-20T16:25:41.721 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-05-20T16:25:41.721 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:41.721 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.721 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.721 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.721 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.721 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.722 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-05-20T16:25:41.722 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-05-20T16:25:41.722 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.722 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.722 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.722 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:41.722 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.722 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.722 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.722 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.722 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.722 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.722 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.722 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.723 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-05-20T16:25:41.723 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-05-20T16:25:41.723 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.723 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.723 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.723 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-05-20T16:25:41.723 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.723 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.723 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-05-20T16:25:41.723 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-05-20T16:25:41.723 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-05-20T16:25:41.723 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-05-20T16:25:41.723 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-05-20T16:25:41.723 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-05-20T16:25:41.723 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-05-20T16:25:41.724 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:41.724 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.724 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.724 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.724 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.724 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.724 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-05-20T16:25:41.724 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-05-20T16:25:41.724 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.724 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.724 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.724 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:41.724 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.724 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.724 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.725 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.725 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.725 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.725 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.725 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-20T16:25:41.725 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-20T16:25:41.725 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.725 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-05-20T16:25:41.725 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "nfs", 2024-05-20T16:25:41.725 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-05-20T16:25:41.725 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-05-20T16:25:41.725 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-05-20T16:25:41.725 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-05-20T16:25:41.725 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-05-20T16:25:41.725 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.726 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.726 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.726 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:41.726 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.726 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.726 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-05-20T16:25:41.726 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-05-20T16:25:41.726 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-05-20T16:25:41.726 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-05-20T16:25:41.726 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-05-20T16:25:41.726 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-05-20T16:25:41.726 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-05-20T16:25:41.726 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:41.727 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.727 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.727 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.727 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.727 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.727 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-05-20T16:25:41.727 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-05-20T16:25:41.727 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.727 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.727 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.727 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:41.727 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.727 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.727 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.727 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.728 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.728 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.728 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.728 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.728 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-05-20T16:25:41.728 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-05-20T16:25:41.728 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.728 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.728 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.728 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-05-20T16:25:41.728 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.728 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.728 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-05-20T16:25:41.728 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-05-20T16:25:41.728 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-05-20T16:25:41.729 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-05-20T16:25:41.729 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-05-20T16:25:41.729 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-05-20T16:25:41.729 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-05-20T16:25:41.729 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:41.729 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.729 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.729 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.729 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.729 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.729 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-05-20T16:25:41.729 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-05-20T16:25:41.729 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.729 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.729 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.729 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:41.730 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.730 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.730 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.730 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.730 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.730 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.730 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.730 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-20T16:25:41.730 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-20T16:25:41.730 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.730 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-05-20T16:25:41.730 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "orchestrator", 2024-05-20T16:25:41.730 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-05-20T16:25:41.730 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-05-20T16:25:41.730 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-05-20T16:25:41.731 INFO:teuthology.orchestra.run.smithi082.stdout: "fail_fs": { 2024-05-20T16:25:41.731 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "fail_fs", 2024-05-20T16:25:41.731 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.731 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.731 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.731 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:41.731 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.731 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.731 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.731 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-05-20T16:25:41.731 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.731 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.731 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.731 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.731 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-05-20T16:25:41.732 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-05-20T16:25:41.732 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.732 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.732 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.732 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:41.732 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.732 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.732 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-05-20T16:25:41.732 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-05-20T16:25:41.732 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-05-20T16:25:41.732 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-05-20T16:25:41.732 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-05-20T16:25:41.732 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-05-20T16:25:41.732 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-05-20T16:25:41.732 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:41.733 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.733 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.733 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.733 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.733 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.733 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-05-20T16:25:41.733 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-05-20T16:25:41.733 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.733 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.733 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.733 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:41.733 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.733 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.733 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.733 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.734 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.734 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.734 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.734 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.734 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-05-20T16:25:41.734 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-05-20T16:25:41.734 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.734 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.734 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.734 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-05-20T16:25:41.734 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.734 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.734 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-05-20T16:25:41.734 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-05-20T16:25:41.735 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-05-20T16:25:41.735 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-05-20T16:25:41.735 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-05-20T16:25:41.735 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-05-20T16:25:41.735 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-05-20T16:25:41.735 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:41.735 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.735 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.735 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.735 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.735 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.735 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-05-20T16:25:41.735 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-05-20T16:25:41.735 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.735 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.736 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.736 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:41.736 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.736 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.736 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.736 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.736 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.736 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.736 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.736 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.736 INFO:teuthology.orchestra.run.smithi082.stdout: "orchestrator": { 2024-05-20T16:25:41.736 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "orchestrator", 2024-05-20T16:25:41.736 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.736 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.736 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.737 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:41.737 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.737 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.737 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-05-20T16:25:41.737 INFO:teuthology.orchestra.run.smithi082.stdout: "cephadm", 2024-05-20T16:25:41.737 INFO:teuthology.orchestra.run.smithi082.stdout: "rook", 2024-05-20T16:25:41.737 INFO:teuthology.orchestra.run.smithi082.stdout: "test_orchestrator" 2024-05-20T16:25:41.737 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:41.737 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Orchestrator backend", 2024-05-20T16:25:41.737 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.737 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.737 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.737 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-20T16:25:41.738 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-20T16:25:41.738 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.738 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-05-20T16:25:41.738 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "osd_perf_query", 2024-05-20T16:25:41.738 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-05-20T16:25:41.738 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-05-20T16:25:41.738 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-05-20T16:25:41.738 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-05-20T16:25:41.738 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-05-20T16:25:41.738 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.738 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.738 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.738 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:41.738 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.739 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.739 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-05-20T16:25:41.739 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-05-20T16:25:41.739 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-05-20T16:25:41.739 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-05-20T16:25:41.739 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-05-20T16:25:41.739 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-05-20T16:25:41.739 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-05-20T16:25:41.739 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:41.739 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.739 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.739 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.739 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.740 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.740 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-05-20T16:25:41.740 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-05-20T16:25:41.740 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.740 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.740 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.740 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:41.740 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.740 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.740 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.740 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.740 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.740 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.741 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.741 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.741 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-05-20T16:25:41.741 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-05-20T16:25:41.741 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.741 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.741 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.741 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-05-20T16:25:41.741 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.741 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.741 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-05-20T16:25:41.741 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-05-20T16:25:41.741 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-05-20T16:25:41.741 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-05-20T16:25:41.742 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-05-20T16:25:41.742 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-05-20T16:25:41.742 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-05-20T16:25:41.742 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:41.742 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.742 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.742 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.742 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.742 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.742 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-05-20T16:25:41.742 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-05-20T16:25:41.742 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.742 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.743 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.743 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:41.743 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.743 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.743 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.743 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.743 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.743 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.743 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.743 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-20T16:25:41.743 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-20T16:25:41.743 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.743 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-05-20T16:25:41.744 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "osd_support", 2024-05-20T16:25:41.744 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-05-20T16:25:41.744 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-05-20T16:25:41.744 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-05-20T16:25:41.744 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-05-20T16:25:41.744 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-05-20T16:25:41.744 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.744 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.744 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.744 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:41.744 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.744 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.744 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-05-20T16:25:41.744 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-05-20T16:25:41.745 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-05-20T16:25:41.745 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-05-20T16:25:41.745 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-05-20T16:25:41.745 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-05-20T16:25:41.745 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-05-20T16:25:41.745 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:41.745 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.745 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.745 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.745 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.745 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.745 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-05-20T16:25:41.745 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-05-20T16:25:41.745 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.746 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.746 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.746 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:41.746 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.746 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.746 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.746 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.746 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.746 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.746 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.746 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.746 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-05-20T16:25:41.746 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-05-20T16:25:41.746 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.747 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.747 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.747 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-05-20T16:25:41.747 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.747 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.747 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-05-20T16:25:41.747 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-05-20T16:25:41.747 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-05-20T16:25:41.747 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-05-20T16:25:41.747 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-05-20T16:25:41.747 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-05-20T16:25:41.747 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-05-20T16:25:41.747 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:41.747 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.747 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.748 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.748 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.748 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.748 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-05-20T16:25:41.748 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-05-20T16:25:41.748 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.748 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.748 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.748 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:41.748 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.748 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.748 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.748 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.748 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.749 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.749 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.749 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-20T16:25:41.749 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-20T16:25:41.749 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.749 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-05-20T16:25:41.749 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "pg_autoscaler", 2024-05-20T16:25:41.749 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-05-20T16:25:41.749 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-05-20T16:25:41.749 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-05-20T16:25:41.749 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-05-20T16:25:41.749 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-05-20T16:25:41.749 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.749 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.749 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.749 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:41.750 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.750 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.750 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-05-20T16:25:41.750 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-05-20T16:25:41.750 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-05-20T16:25:41.750 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-05-20T16:25:41.750 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-05-20T16:25:41.750 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-05-20T16:25:41.750 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-05-20T16:25:41.750 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:41.750 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.750 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.750 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.750 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.751 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.751 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-05-20T16:25:41.751 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-05-20T16:25:41.751 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.751 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.751 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.751 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:41.751 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.751 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.751 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.751 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.751 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.751 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.751 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.752 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.752 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-05-20T16:25:41.752 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-05-20T16:25:41.752 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.752 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.752 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.752 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-05-20T16:25:41.752 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.752 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.752 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-05-20T16:25:41.752 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-05-20T16:25:41.752 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-05-20T16:25:41.752 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-05-20T16:25:41.752 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-05-20T16:25:41.753 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-05-20T16:25:41.753 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-05-20T16:25:41.753 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:41.753 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.753 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.753 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.753 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.753 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.753 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-05-20T16:25:41.753 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-05-20T16:25:41.753 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.753 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.753 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.754 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:41.754 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.754 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.754 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.754 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.754 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.754 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.754 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.754 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.754 INFO:teuthology.orchestra.run.smithi082.stdout: "sleep_interval": { 2024-05-20T16:25:41.754 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "sleep_interval", 2024-05-20T16:25:41.754 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2024-05-20T16:25:41.754 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.754 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.755 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "60", 2024-05-20T16:25:41.755 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.755 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.755 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.755 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.755 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.755 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.755 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.755 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.755 INFO:teuthology.orchestra.run.smithi082.stdout: "threshold": { 2024-05-20T16:25:41.755 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "threshold", 2024-05-20T16:25:41.755 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "float", 2024-05-20T16:25:41.755 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.756 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.756 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "3.0", 2024-05-20T16:25:41.756 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "1.0", 2024-05-20T16:25:41.756 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.756 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.756 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "scaling threshold", 2024-05-20T16:25:41.756 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-05-20T16:25:41.756 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.756 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.756 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-20T16:25:41.756 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-20T16:25:41.756 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.756 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-05-20T16:25:41.756 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "progress", 2024-05-20T16:25:41.757 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-05-20T16:25:41.757 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-05-20T16:25:41.757 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-05-20T16:25:41.757 INFO:teuthology.orchestra.run.smithi082.stdout: "allow_pg_recovery_event": { 2024-05-20T16:25:41.757 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "allow_pg_recovery_event", 2024-05-20T16:25:41.757 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.757 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.757 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.757 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:41.757 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.757 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.757 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.757 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "allow the module to show pg recovery progress", 2024-05-20T16:25:41.757 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.758 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.758 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.758 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.758 INFO:teuthology.orchestra.run.smithi082.stdout: "enabled": { 2024-05-20T16:25:41.758 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "enabled", 2024-05-20T16:25:41.758 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.758 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.758 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.758 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-05-20T16:25:41.758 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.758 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.758 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.758 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.758 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.759 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.759 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.759 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.759 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-05-20T16:25:41.759 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-05-20T16:25:41.759 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.759 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.759 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.759 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:41.759 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.759 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.759 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-05-20T16:25:41.759 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-05-20T16:25:41.759 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-05-20T16:25:41.760 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-05-20T16:25:41.760 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-05-20T16:25:41.760 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-05-20T16:25:41.760 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-05-20T16:25:41.760 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:41.760 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.760 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.760 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.760 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.760 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.760 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-05-20T16:25:41.760 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-05-20T16:25:41.760 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.760 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.761 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.761 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:41.761 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.761 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.761 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.761 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.761 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.761 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.761 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.761 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.761 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-05-20T16:25:41.761 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-05-20T16:25:41.761 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.762 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.762 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.762 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-05-20T16:25:41.762 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.762 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.762 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-05-20T16:25:41.762 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-05-20T16:25:41.762 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-05-20T16:25:41.762 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-05-20T16:25:41.762 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-05-20T16:25:41.762 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-05-20T16:25:41.762 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-05-20T16:25:41.762 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:41.762 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.763 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.763 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.763 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.763 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.763 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-05-20T16:25:41.763 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-05-20T16:25:41.763 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.763 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.763 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.763 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:41.763 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.763 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.763 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.763 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.764 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.764 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.764 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.764 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.764 INFO:teuthology.orchestra.run.smithi082.stdout: "max_completed_events": { 2024-05-20T16:25:41.764 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "max_completed_events", 2024-05-20T16:25:41.764 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-05-20T16:25:41.764 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.764 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.764 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "50", 2024-05-20T16:25:41.764 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.764 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.764 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.764 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "number of past completed events to remember", 2024-05-20T16:25:41.765 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.765 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.765 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.765 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.765 INFO:teuthology.orchestra.run.smithi082.stdout: "sleep_interval": { 2024-05-20T16:25:41.765 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "sleep_interval", 2024-05-20T16:25:41.765 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2024-05-20T16:25:41.765 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.765 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.765 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "5", 2024-05-20T16:25:41.765 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.765 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.765 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.765 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "how long the module is going to sleep", 2024-05-20T16:25:41.766 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.766 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.766 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.766 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-20T16:25:41.766 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-20T16:25:41.766 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.766 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-05-20T16:25:41.766 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "prometheus", 2024-05-20T16:25:41.766 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-05-20T16:25:41.766 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-05-20T16:25:41.766 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-05-20T16:25:41.766 INFO:teuthology.orchestra.run.smithi082.stdout: "cache": { 2024-05-20T16:25:41.766 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "cache", 2024-05-20T16:25:41.767 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.767 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.767 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.767 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-05-20T16:25:41.767 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.767 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.767 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.767 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.767 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.767 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.767 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.767 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.767 INFO:teuthology.orchestra.run.smithi082.stdout: "exclude_perf_counters": { 2024-05-20T16:25:41.768 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "exclude_perf_counters", 2024-05-20T16:25:41.768 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.768 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.768 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.768 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-05-20T16:25:41.768 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.768 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.768 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.768 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-05-20T16:25:41.768 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-05-20T16:25:41.768 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.768 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.768 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.768 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-05-20T16:25:41.768 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-05-20T16:25:41.769 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.769 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.769 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.769 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:41.769 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.769 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.769 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-05-20T16:25:41.769 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-05-20T16:25:41.769 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-05-20T16:25:41.769 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-05-20T16:25:41.769 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-05-20T16:25:41.769 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-05-20T16:25:41.769 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-05-20T16:25:41.769 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:41.769 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.769 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.770 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.770 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.770 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.770 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-05-20T16:25:41.770 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-05-20T16:25:41.770 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.770 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.770 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.770 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:41.770 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.770 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.770 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.770 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.770 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.770 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.771 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.771 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.771 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-05-20T16:25:41.771 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-05-20T16:25:41.771 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.771 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.771 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.771 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-05-20T16:25:41.771 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.771 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.771 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-05-20T16:25:41.771 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-05-20T16:25:41.771 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-05-20T16:25:41.771 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-05-20T16:25:41.772 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-05-20T16:25:41.772 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-05-20T16:25:41.772 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-05-20T16:25:41.772 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:41.772 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.772 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.772 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.772 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.772 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.772 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-05-20T16:25:41.772 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-05-20T16:25:41.772 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.772 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.772 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.772 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:41.773 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.773 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.773 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.773 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.773 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.773 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.773 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.773 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.773 INFO:teuthology.orchestra.run.smithi082.stdout: "rbd_stats_pools": { 2024-05-20T16:25:41.773 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "rbd_stats_pools", 2024-05-20T16:25:41.773 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.773 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.773 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.773 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:41.773 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.774 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.774 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.774 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.774 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.774 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.774 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.774 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.774 INFO:teuthology.orchestra.run.smithi082.stdout: "rbd_stats_pools_refresh_interval": { 2024-05-20T16:25:41.774 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-05-20T16:25:41.774 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-05-20T16:25:41.774 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.774 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.774 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "300", 2024-05-20T16:25:41.774 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.774 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.775 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.775 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.775 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.775 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.775 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.775 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.775 INFO:teuthology.orchestra.run.smithi082.stdout: "scrape_interval": { 2024-05-20T16:25:41.775 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "scrape_interval", 2024-05-20T16:25:41.775 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "float", 2024-05-20T16:25:41.775 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.775 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.775 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "15.0", 2024-05-20T16:25:41.775 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.775 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.775 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.776 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.776 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.776 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.776 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.776 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.776 INFO:teuthology.orchestra.run.smithi082.stdout: "server_addr": { 2024-05-20T16:25:41.776 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "server_addr", 2024-05-20T16:25:41.776 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.776 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.776 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.776 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "::", 2024-05-20T16:25:41.776 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.776 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.776 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.776 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-05-20T16:25:41.777 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.777 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.777 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.777 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.777 INFO:teuthology.orchestra.run.smithi082.stdout: "server_port": { 2024-05-20T16:25:41.777 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "server_port", 2024-05-20T16:25:41.777 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-05-20T16:25:41.777 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.777 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.777 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "9283", 2024-05-20T16:25:41.777 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.777 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.777 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.777 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-05-20T16:25:41.777 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.778 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.778 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.778 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.778 INFO:teuthology.orchestra.run.smithi082.stdout: "stale_cache_strategy": { 2024-05-20T16:25:41.778 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "stale_cache_strategy", 2024-05-20T16:25:41.778 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.778 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.778 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.778 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "log", 2024-05-20T16:25:41.778 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.778 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.778 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.778 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.778 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.779 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.779 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.779 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.779 INFO:teuthology.orchestra.run.smithi082.stdout: "standby_behaviour": { 2024-05-20T16:25:41.779 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "standby_behaviour", 2024-05-20T16:25:41.779 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.779 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.779 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.779 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "default", 2024-05-20T16:25:41.779 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.779 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.779 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-05-20T16:25:41.779 INFO:teuthology.orchestra.run.smithi082.stdout: "default", 2024-05-20T16:25:41.779 INFO:teuthology.orchestra.run.smithi082.stdout: "error" 2024-05-20T16:25:41.779 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:41.779 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.780 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.780 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.780 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.780 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.780 INFO:teuthology.orchestra.run.smithi082.stdout: "standby_error_status_code": { 2024-05-20T16:25:41.780 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "standby_error_status_code", 2024-05-20T16:25:41.780 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-05-20T16:25:41.780 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.780 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.780 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "500", 2024-05-20T16:25:41.780 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "400", 2024-05-20T16:25:41.780 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "599", 2024-05-20T16:25:41.780 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.780 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.781 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.781 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.781 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.781 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-20T16:25:41.781 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-20T16:25:41.781 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.781 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-05-20T16:25:41.781 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "rbd_support", 2024-05-20T16:25:41.781 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-05-20T16:25:41.781 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-05-20T16:25:41.781 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-05-20T16:25:41.781 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-05-20T16:25:41.781 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-05-20T16:25:41.781 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.782 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.782 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.782 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:41.782 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.782 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.782 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-05-20T16:25:41.782 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-05-20T16:25:41.782 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-05-20T16:25:41.782 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-05-20T16:25:41.782 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-05-20T16:25:41.782 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-05-20T16:25:41.782 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-05-20T16:25:41.782 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:41.782 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.783 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.783 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.783 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.783 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.783 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-05-20T16:25:41.783 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-05-20T16:25:41.783 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.783 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.783 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.783 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:41.783 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.783 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.783 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.784 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.784 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.784 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.784 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.784 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.784 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-05-20T16:25:41.784 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-05-20T16:25:41.784 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.784 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.784 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.784 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-05-20T16:25:41.784 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.784 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.784 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-05-20T16:25:41.785 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-05-20T16:25:41.785 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-05-20T16:25:41.785 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-05-20T16:25:41.785 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-05-20T16:25:41.785 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-05-20T16:25:41.785 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-05-20T16:25:41.785 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:41.785 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.785 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.785 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.785 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.785 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.785 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-05-20T16:25:41.785 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-05-20T16:25:41.786 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.786 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.786 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.786 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:41.786 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.786 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.786 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.786 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.786 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.786 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.786 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.786 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.786 INFO:teuthology.orchestra.run.smithi082.stdout: "max_concurrent_snap_create": { 2024-05-20T16:25:41.786 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "max_concurrent_snap_create", 2024-05-20T16:25:41.787 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-05-20T16:25:41.787 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.787 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.787 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "10", 2024-05-20T16:25:41.787 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.787 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.787 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.787 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.787 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.787 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.787 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.787 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.787 INFO:teuthology.orchestra.run.smithi082.stdout: "mirror_snapshot_schedule": { 2024-05-20T16:25:41.787 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "mirror_snapshot_schedule", 2024-05-20T16:25:41.788 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.788 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.788 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.788 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:41.788 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.788 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.788 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.788 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.788 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.788 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.788 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.788 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.788 INFO:teuthology.orchestra.run.smithi082.stdout: "trash_purge_schedule": { 2024-05-20T16:25:41.788 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "trash_purge_schedule", 2024-05-20T16:25:41.788 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.789 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.789 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.789 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:41.789 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.789 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.789 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.789 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.789 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.789 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.789 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.789 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-20T16:25:41.789 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-20T16:25:41.789 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.789 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-05-20T16:25:41.790 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "restful", 2024-05-20T16:25:41.790 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-05-20T16:25:41.790 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-05-20T16:25:41.790 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-05-20T16:25:41.790 INFO:teuthology.orchestra.run.smithi082.stdout: "enable_auth": { 2024-05-20T16:25:41.790 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "enable_auth", 2024-05-20T16:25:41.790 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.790 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.790 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.790 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-05-20T16:25:41.790 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.790 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.790 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.790 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.791 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.791 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.791 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.791 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.791 INFO:teuthology.orchestra.run.smithi082.stdout: "key_file": { 2024-05-20T16:25:41.791 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "key_file", 2024-05-20T16:25:41.791 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.791 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.791 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.791 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:41.791 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.791 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.791 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.791 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.791 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.791 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.792 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.792 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.792 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-05-20T16:25:41.792 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-05-20T16:25:41.792 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.792 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.792 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.792 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:41.792 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.792 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.792 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-05-20T16:25:41.792 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-05-20T16:25:41.792 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-05-20T16:25:41.792 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-05-20T16:25:41.793 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-05-20T16:25:41.793 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-05-20T16:25:41.793 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-05-20T16:25:41.793 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:41.793 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.793 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.793 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.793 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.793 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.793 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-05-20T16:25:41.793 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-05-20T16:25:41.793 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.793 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.794 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.794 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:41.794 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.794 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.794 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.794 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.794 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.794 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.794 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.794 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.794 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-05-20T16:25:41.794 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-05-20T16:25:41.794 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.795 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.795 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.795 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-05-20T16:25:41.795 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.795 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.795 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-05-20T16:25:41.795 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-05-20T16:25:41.795 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-05-20T16:25:41.795 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-05-20T16:25:41.795 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-05-20T16:25:41.795 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-05-20T16:25:41.795 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-05-20T16:25:41.795 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:41.796 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.796 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.796 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.796 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.796 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.796 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-05-20T16:25:41.796 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-05-20T16:25:41.796 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.796 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.796 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.796 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:41.796 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.796 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.796 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.797 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.797 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.797 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.797 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.797 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.797 INFO:teuthology.orchestra.run.smithi082.stdout: "server_addr": { 2024-05-20T16:25:41.797 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "server_addr", 2024-05-20T16:25:41.797 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.797 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.797 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.797 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:41.797 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.797 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.797 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.798 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.798 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.798 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.798 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.798 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.798 INFO:teuthology.orchestra.run.smithi082.stdout: "server_port": { 2024-05-20T16:25:41.798 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "server_port", 2024-05-20T16:25:41.798 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.798 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.798 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.798 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:41.798 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.798 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.798 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.799 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.799 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.799 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.799 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.799 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-20T16:25:41.799 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-20T16:25:41.799 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.799 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-05-20T16:25:41.799 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "rgw", 2024-05-20T16:25:41.799 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-05-20T16:25:41.799 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-05-20T16:25:41.799 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-05-20T16:25:41.799 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-05-20T16:25:41.799 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-05-20T16:25:41.799 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.800 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.800 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.800 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:41.800 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.800 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.800 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-05-20T16:25:41.800 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-05-20T16:25:41.800 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-05-20T16:25:41.800 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-05-20T16:25:41.800 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-05-20T16:25:41.800 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-05-20T16:25:41.800 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-05-20T16:25:41.800 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:41.800 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.800 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.800 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.801 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.801 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.801 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-05-20T16:25:41.801 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-05-20T16:25:41.801 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.801 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.801 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.801 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:41.801 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.801 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.801 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.801 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.801 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.801 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.801 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.801 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.802 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-05-20T16:25:41.802 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-05-20T16:25:41.802 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.802 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.802 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.802 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-05-20T16:25:41.802 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.802 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.802 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-05-20T16:25:41.802 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-05-20T16:25:41.802 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-05-20T16:25:41.802 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-05-20T16:25:41.802 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-05-20T16:25:41.802 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-05-20T16:25:41.802 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-05-20T16:25:41.803 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:41.803 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.803 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.803 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.803 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.803 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.803 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-05-20T16:25:41.803 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-05-20T16:25:41.803 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.803 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.803 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.803 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:41.803 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.803 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.804 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.804 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.804 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.804 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.804 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.804 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-20T16:25:41.804 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-20T16:25:41.804 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.804 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-05-20T16:25:41.804 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "rook", 2024-05-20T16:25:41.804 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-05-20T16:25:41.804 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-05-20T16:25:41.804 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-05-20T16:25:41.804 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-05-20T16:25:41.804 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-05-20T16:25:41.804 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.805 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.805 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.805 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:41.805 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.805 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.805 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-05-20T16:25:41.805 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-05-20T16:25:41.805 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-05-20T16:25:41.805 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-05-20T16:25:41.805 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-05-20T16:25:41.805 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-05-20T16:25:41.805 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-05-20T16:25:41.805 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:41.805 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.805 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.805 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.806 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.806 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.806 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-05-20T16:25:41.806 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-05-20T16:25:41.806 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.806 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.806 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.806 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:41.806 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.806 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.806 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.806 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.806 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.806 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.806 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.807 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.807 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-05-20T16:25:41.807 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-05-20T16:25:41.807 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.807 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.807 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.807 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-05-20T16:25:41.807 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.807 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.807 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-05-20T16:25:41.807 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-05-20T16:25:41.807 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-05-20T16:25:41.807 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-05-20T16:25:41.807 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-05-20T16:25:41.807 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-05-20T16:25:41.807 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-05-20T16:25:41.808 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:41.808 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.808 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.808 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.808 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.808 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.808 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-05-20T16:25:41.808 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-05-20T16:25:41.808 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.808 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.808 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.808 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:41.808 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.808 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.809 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.809 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.809 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.809 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.809 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.809 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.809 INFO:teuthology.orchestra.run.smithi082.stdout: "storage_class": { 2024-05-20T16:25:41.809 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "storage_class", 2024-05-20T16:25:41.809 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.809 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.809 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.809 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "local", 2024-05-20T16:25:41.809 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.809 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.809 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.810 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-05-20T16:25:41.810 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.810 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.810 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.810 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-20T16:25:41.810 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-20T16:25:41.810 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.810 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-05-20T16:25:41.810 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "selftest", 2024-05-20T16:25:41.810 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-05-20T16:25:41.810 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-05-20T16:25:41.810 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-05-20T16:25:41.810 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-05-20T16:25:41.810 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-05-20T16:25:41.810 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.811 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.811 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.811 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:41.811 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.811 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.811 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-05-20T16:25:41.811 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-05-20T16:25:41.811 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-05-20T16:25:41.811 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-05-20T16:25:41.811 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-05-20T16:25:41.811 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-05-20T16:25:41.811 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-05-20T16:25:41.811 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:41.811 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.812 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.812 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.812 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.812 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.812 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-05-20T16:25:41.812 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-05-20T16:25:41.812 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.812 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.812 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.812 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:41.812 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.812 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.812 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.812 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.812 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.813 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.813 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.813 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.813 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-05-20T16:25:41.813 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-05-20T16:25:41.813 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.813 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.813 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.813 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-05-20T16:25:41.813 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.813 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.813 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-05-20T16:25:41.813 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-05-20T16:25:41.813 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-05-20T16:25:41.814 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-05-20T16:25:41.814 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-05-20T16:25:41.814 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-05-20T16:25:41.814 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-05-20T16:25:41.814 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:41.814 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.814 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.814 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.814 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.814 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.814 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-05-20T16:25:41.814 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-05-20T16:25:41.814 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.814 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.814 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.814 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:41.815 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.815 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.815 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.815 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.815 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.815 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.815 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.815 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.815 INFO:teuthology.orchestra.run.smithi082.stdout: "roption1": { 2024-05-20T16:25:41.815 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "roption1", 2024-05-20T16:25:41.815 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.815 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.815 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.815 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:41.815 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.815 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.816 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.816 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.816 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.816 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.816 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.816 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.816 INFO:teuthology.orchestra.run.smithi082.stdout: "roption2": { 2024-05-20T16:25:41.816 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "roption2", 2024-05-20T16:25:41.816 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.816 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.816 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.816 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "xyz", 2024-05-20T16:25:41.816 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.816 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.816 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.816 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.817 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.817 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.817 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.817 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.817 INFO:teuthology.orchestra.run.smithi082.stdout: "rwoption1": { 2024-05-20T16:25:41.817 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "rwoption1", 2024-05-20T16:25:41.817 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.817 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.817 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.817 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:41.817 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.817 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.817 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.817 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.818 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.818 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.818 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.818 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.818 INFO:teuthology.orchestra.run.smithi082.stdout: "rwoption2": { 2024-05-20T16:25:41.818 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "rwoption2", 2024-05-20T16:25:41.818 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-05-20T16:25:41.818 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.818 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.818 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:41.818 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.818 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.818 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.818 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.818 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.819 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.819 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.819 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.819 INFO:teuthology.orchestra.run.smithi082.stdout: "rwoption3": { 2024-05-20T16:25:41.819 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "rwoption3", 2024-05-20T16:25:41.819 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "float", 2024-05-20T16:25:41.819 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.819 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.819 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:41.819 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.819 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.819 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.819 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.819 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.819 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.820 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.820 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.820 INFO:teuthology.orchestra.run.smithi082.stdout: "rwoption4": { 2024-05-20T16:25:41.820 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "rwoption4", 2024-05-20T16:25:41.820 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.820 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.820 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.820 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:41.820 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.820 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.820 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.820 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.820 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.820 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.820 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.820 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.821 INFO:teuthology.orchestra.run.smithi082.stdout: "rwoption5": { 2024-05-20T16:25:41.821 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "rwoption5", 2024-05-20T16:25:41.821 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.821 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.821 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.821 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:41.821 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.821 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.821 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.821 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.821 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.821 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.821 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.821 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.822 INFO:teuthology.orchestra.run.smithi082.stdout: "rwoption6": { 2024-05-20T16:25:41.822 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "rwoption6", 2024-05-20T16:25:41.822 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.822 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.822 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.822 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-05-20T16:25:41.822 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.822 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.822 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.822 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.822 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.822 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.822 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.822 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.823 INFO:teuthology.orchestra.run.smithi082.stdout: "rwoption7": { 2024-05-20T16:25:41.823 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "rwoption7", 2024-05-20T16:25:41.823 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-05-20T16:25:41.823 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.823 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.823 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:41.823 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "1", 2024-05-20T16:25:41.823 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "42", 2024-05-20T16:25:41.823 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.823 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.823 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.823 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.823 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.823 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.824 INFO:teuthology.orchestra.run.smithi082.stdout: "testkey": { 2024-05-20T16:25:41.824 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "testkey", 2024-05-20T16:25:41.824 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.824 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.824 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.824 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:41.824 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.824 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.824 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.824 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.824 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.824 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.824 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.824 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.825 INFO:teuthology.orchestra.run.smithi082.stdout: "testlkey": { 2024-05-20T16:25:41.825 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "testlkey", 2024-05-20T16:25:41.825 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.825 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.825 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.825 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:41.825 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.825 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.825 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.825 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.825 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.825 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.825 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.825 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.825 INFO:teuthology.orchestra.run.smithi082.stdout: "testnewline": { 2024-05-20T16:25:41.825 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "testnewline", 2024-05-20T16:25:41.826 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.826 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.826 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.826 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:41.826 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.826 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.826 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.826 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.826 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.826 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.826 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.826 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-20T16:25:41.826 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-20T16:25:41.826 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.826 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-05-20T16:25:41.826 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "snap_schedule", 2024-05-20T16:25:41.827 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-05-20T16:25:41.827 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-05-20T16:25:41.827 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-05-20T16:25:41.827 INFO:teuthology.orchestra.run.smithi082.stdout: "allow_m_granularity": { 2024-05-20T16:25:41.827 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "allow_m_granularity", 2024-05-20T16:25:41.827 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.827 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.827 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.827 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:41.827 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.827 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.827 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.827 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "allow minute scheduled snapshots", 2024-05-20T16:25:41.827 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.827 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.828 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.828 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.828 INFO:teuthology.orchestra.run.smithi082.stdout: "dump_on_update": { 2024-05-20T16:25:41.828 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "dump_on_update", 2024-05-20T16:25:41.828 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.828 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.828 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.828 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:41.828 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.828 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.828 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.828 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "dump database to debug log on update", 2024-05-20T16:25:41.828 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.828 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.828 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.828 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.829 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-05-20T16:25:41.829 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-05-20T16:25:41.829 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.829 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.829 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.829 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:41.829 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.829 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.829 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-05-20T16:25:41.829 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-05-20T16:25:41.829 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-05-20T16:25:41.829 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-05-20T16:25:41.829 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-05-20T16:25:41.829 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-05-20T16:25:41.829 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-05-20T16:25:41.829 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:41.830 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.830 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.830 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.830 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.830 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.830 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-05-20T16:25:41.830 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-05-20T16:25:41.830 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.830 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.830 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.830 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:41.830 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.830 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.830 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.830 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.831 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.831 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.831 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.831 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.831 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-05-20T16:25:41.831 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-05-20T16:25:41.831 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.831 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.831 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.831 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-05-20T16:25:41.831 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.831 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.831 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-05-20T16:25:41.831 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-05-20T16:25:41.831 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-05-20T16:25:41.832 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-05-20T16:25:41.832 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-05-20T16:25:41.832 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-05-20T16:25:41.832 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-05-20T16:25:41.832 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:41.832 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.832 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.832 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.832 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.832 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.832 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-05-20T16:25:41.832 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-05-20T16:25:41.832 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.833 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.833 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.833 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:41.833 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.833 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.833 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.833 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.833 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.833 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.833 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.833 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-20T16:25:41.833 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-20T16:25:41.833 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.833 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-05-20T16:25:41.833 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "stats", 2024-05-20T16:25:41.834 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-05-20T16:25:41.834 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-05-20T16:25:41.834 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-05-20T16:25:41.834 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-05-20T16:25:41.834 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-05-20T16:25:41.834 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.834 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.834 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.834 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:41.834 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.834 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.834 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-05-20T16:25:41.834 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-05-20T16:25:41.834 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-05-20T16:25:41.834 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-05-20T16:25:41.835 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-05-20T16:25:41.835 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-05-20T16:25:41.835 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-05-20T16:25:41.835 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:41.835 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.835 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.835 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.835 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.835 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.835 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-05-20T16:25:41.835 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-05-20T16:25:41.835 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.835 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.836 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.836 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:41.836 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.836 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.836 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.836 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.836 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.836 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.836 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.836 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.836 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-05-20T16:25:41.836 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-05-20T16:25:41.836 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.836 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.837 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.837 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-05-20T16:25:41.837 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.837 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.837 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-05-20T16:25:41.837 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-05-20T16:25:41.837 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-05-20T16:25:41.837 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-05-20T16:25:41.837 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-05-20T16:25:41.837 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-05-20T16:25:41.837 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-05-20T16:25:41.837 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:41.837 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.837 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.838 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.838 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.838 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.838 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-05-20T16:25:41.838 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-05-20T16:25:41.838 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.838 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.838 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.838 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:41.838 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.838 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.838 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.838 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.839 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.839 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.839 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.839 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-20T16:25:41.839 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-20T16:25:41.839 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.839 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-05-20T16:25:41.839 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "status", 2024-05-20T16:25:41.839 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-05-20T16:25:41.839 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-05-20T16:25:41.839 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-05-20T16:25:41.839 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-05-20T16:25:41.839 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-05-20T16:25:41.839 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.839 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.840 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.840 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:41.840 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.840 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.840 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-05-20T16:25:41.840 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-05-20T16:25:41.840 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-05-20T16:25:41.840 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-05-20T16:25:41.840 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-05-20T16:25:41.840 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-05-20T16:25:41.840 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-05-20T16:25:41.840 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:41.840 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.840 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.840 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.840 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.841 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.841 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-05-20T16:25:41.841 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-05-20T16:25:41.841 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.841 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.841 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.841 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:41.841 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.841 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.841 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.841 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.841 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.841 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.841 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.841 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.842 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-05-20T16:25:41.842 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-05-20T16:25:41.842 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.842 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.842 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.842 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-05-20T16:25:41.842 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.842 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.842 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-05-20T16:25:41.842 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-05-20T16:25:41.842 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-05-20T16:25:41.842 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-05-20T16:25:41.842 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-05-20T16:25:41.843 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-05-20T16:25:41.843 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-05-20T16:25:41.843 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:41.843 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.843 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.843 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.843 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.843 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.843 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-05-20T16:25:41.843 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-05-20T16:25:41.843 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.843 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.843 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.843 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:41.844 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.844 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.844 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.844 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.844 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.844 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.844 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.844 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-20T16:25:41.844 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-20T16:25:41.844 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.844 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-05-20T16:25:41.844 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "telegraf", 2024-05-20T16:25:41.844 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-05-20T16:25:41.844 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-05-20T16:25:41.844 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-05-20T16:25:41.844 INFO:teuthology.orchestra.run.smithi082.stdout: "address": { 2024-05-20T16:25:41.845 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "address", 2024-05-20T16:25:41.845 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.845 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.845 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.845 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-05-20T16:25:41.845 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.845 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.845 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.845 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.845 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.845 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.845 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.845 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.845 INFO:teuthology.orchestra.run.smithi082.stdout: "interval": { 2024-05-20T16:25:41.845 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "interval", 2024-05-20T16:25:41.845 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2024-05-20T16:25:41.846 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.846 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.846 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "15", 2024-05-20T16:25:41.846 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.846 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.846 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.846 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.846 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.846 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.846 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.846 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.846 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-05-20T16:25:41.846 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-05-20T16:25:41.847 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.847 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.847 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.847 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:41.847 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.847 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.847 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-05-20T16:25:41.847 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-05-20T16:25:41.847 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-05-20T16:25:41.847 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-05-20T16:25:41.847 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-05-20T16:25:41.847 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-05-20T16:25:41.847 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-05-20T16:25:41.847 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:41.847 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.848 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.848 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.848 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.848 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.848 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-05-20T16:25:41.848 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-05-20T16:25:41.848 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.848 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.848 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.848 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:41.848 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.848 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.848 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.848 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.848 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.848 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.849 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.849 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.849 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-05-20T16:25:41.849 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-05-20T16:25:41.849 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.849 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.849 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.849 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-05-20T16:25:41.849 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.849 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.849 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-05-20T16:25:41.849 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-05-20T16:25:41.849 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-05-20T16:25:41.849 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-05-20T16:25:41.849 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-05-20T16:25:41.850 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-05-20T16:25:41.850 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-05-20T16:25:41.850 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:41.850 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.850 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.850 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.850 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.850 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.850 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-05-20T16:25:41.850 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-05-20T16:25:41.850 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.850 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.850 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.850 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:41.851 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.851 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.851 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.851 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.851 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.851 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.851 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.851 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-20T16:25:41.851 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-20T16:25:41.851 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.851 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-05-20T16:25:41.851 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "telemetry", 2024-05-20T16:25:41.851 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-05-20T16:25:41.851 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-05-20T16:25:41.851 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-05-20T16:25:41.852 INFO:teuthology.orchestra.run.smithi082.stdout: "channel_basic": { 2024-05-20T16:25:41.852 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "channel_basic", 2024-05-20T16:25:41.852 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.852 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.852 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.852 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-05-20T16:25:41.852 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.852 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.852 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.852 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Share basic cluster information (size, version)", 2024-05-20T16:25:41.852 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.852 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.852 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.852 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.852 INFO:teuthology.orchestra.run.smithi082.stdout: "channel_crash": { 2024-05-20T16:25:41.853 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "channel_crash", 2024-05-20T16:25:41.853 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.853 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.853 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.853 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-05-20T16:25:41.853 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.853 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.853 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.853 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-05-20T16:25:41.853 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.853 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.853 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.853 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.853 INFO:teuthology.orchestra.run.smithi082.stdout: "channel_device": { 2024-05-20T16:25:41.854 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "channel_device", 2024-05-20T16:25:41.854 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.854 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.854 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.854 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-05-20T16:25:41.854 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.854 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.854 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.854 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-05-20T16:25:41.854 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.854 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.854 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.854 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.854 INFO:teuthology.orchestra.run.smithi082.stdout: "channel_ident": { 2024-05-20T16:25:41.855 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "channel_ident", 2024-05-20T16:25:41.855 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.855 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.855 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.855 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:41.855 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.855 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.855 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.855 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-05-20T16:25:41.855 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.855 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.855 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.855 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.855 INFO:teuthology.orchestra.run.smithi082.stdout: "channel_perf": { 2024-05-20T16:25:41.856 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "channel_perf", 2024-05-20T16:25:41.856 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.856 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.856 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.856 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:41.856 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.856 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.856 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.856 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Share various performance metrics of a cluster", 2024-05-20T16:25:41.856 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.856 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.856 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.856 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.856 INFO:teuthology.orchestra.run.smithi082.stdout: "contact": { 2024-05-20T16:25:41.856 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "contact", 2024-05-20T16:25:41.857 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.857 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.857 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.857 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:41.857 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.857 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.857 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.857 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.857 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.857 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.857 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.857 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.857 INFO:teuthology.orchestra.run.smithi082.stdout: "description": { 2024-05-20T16:25:41.857 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "description", 2024-05-20T16:25:41.857 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.858 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.858 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.858 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:41.858 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.858 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.858 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.858 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.858 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.858 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.858 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.858 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.858 INFO:teuthology.orchestra.run.smithi082.stdout: "device_url": { 2024-05-20T16:25:41.859 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "device_url", 2024-05-20T16:25:41.859 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.859 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.859 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.859 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-05-20T16:25:41.859 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.859 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.859 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.859 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.859 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.859 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.859 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.859 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.859 INFO:teuthology.orchestra.run.smithi082.stdout: "enabled": { 2024-05-20T16:25:41.859 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "enabled", 2024-05-20T16:25:41.860 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.860 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.860 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.860 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:41.860 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.860 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.860 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.860 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.860 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.860 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.860 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.860 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.860 INFO:teuthology.orchestra.run.smithi082.stdout: "interval": { 2024-05-20T16:25:41.860 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "interval", 2024-05-20T16:25:41.860 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-05-20T16:25:41.861 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.861 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.861 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "24", 2024-05-20T16:25:41.861 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "8", 2024-05-20T16:25:41.861 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.861 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.861 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.861 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.861 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.861 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.861 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.861 INFO:teuthology.orchestra.run.smithi082.stdout: "last_opt_revision": { 2024-05-20T16:25:41.861 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "last_opt_revision", 2024-05-20T16:25:41.861 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-05-20T16:25:41.862 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.862 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.862 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "1", 2024-05-20T16:25:41.862 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.862 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.862 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.862 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.862 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.862 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.862 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.862 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.862 INFO:teuthology.orchestra.run.smithi082.stdout: "leaderboard": { 2024-05-20T16:25:41.862 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "leaderboard", 2024-05-20T16:25:41.862 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.863 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.863 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.863 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:41.863 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.863 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.863 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.863 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.863 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.863 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.863 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.863 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.863 INFO:teuthology.orchestra.run.smithi082.stdout: "leaderboard_description": { 2024-05-20T16:25:41.863 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "leaderboard_description", 2024-05-20T16:25:41.864 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.864 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.864 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.864 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:41.864 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.864 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.864 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.864 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.864 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.864 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.864 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.864 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.864 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-05-20T16:25:41.864 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-05-20T16:25:41.865 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.865 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.865 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.865 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:41.865 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.865 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.865 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-05-20T16:25:41.865 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-05-20T16:25:41.865 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-05-20T16:25:41.865 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-05-20T16:25:41.865 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-05-20T16:25:41.865 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-05-20T16:25:41.865 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-05-20T16:25:41.865 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:41.865 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.866 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.866 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.866 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.866 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.866 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-05-20T16:25:41.866 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-05-20T16:25:41.866 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.866 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.866 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.866 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:41.866 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.866 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.866 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.867 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.867 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.867 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.867 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.867 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.867 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-05-20T16:25:41.867 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-05-20T16:25:41.867 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.867 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.867 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.867 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-05-20T16:25:41.867 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.867 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.867 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-05-20T16:25:41.868 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-05-20T16:25:41.868 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-05-20T16:25:41.868 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-05-20T16:25:41.868 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-05-20T16:25:41.868 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-05-20T16:25:41.868 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-05-20T16:25:41.868 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:41.868 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.868 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.868 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.868 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.868 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.868 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-05-20T16:25:41.868 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-05-20T16:25:41.868 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.868 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.869 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.869 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:41.869 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.869 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.869 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.869 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.869 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.869 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.869 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.869 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.869 INFO:teuthology.orchestra.run.smithi082.stdout: "organization": { 2024-05-20T16:25:41.869 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "organization", 2024-05-20T16:25:41.869 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.869 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.870 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.870 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:41.870 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.870 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.870 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.870 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.870 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.870 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.870 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.870 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.870 INFO:teuthology.orchestra.run.smithi082.stdout: "proxy": { 2024-05-20T16:25:41.870 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "proxy", 2024-05-20T16:25:41.870 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.870 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.870 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.871 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:41.871 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.871 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.871 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.871 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.871 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.871 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.871 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.871 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.871 INFO:teuthology.orchestra.run.smithi082.stdout: "url": { 2024-05-20T16:25:41.871 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "url", 2024-05-20T16:25:41.871 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.871 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.871 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.871 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-05-20T16:25:41.872 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.872 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.872 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.872 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.872 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.872 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.872 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.872 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-20T16:25:41.872 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-20T16:25:41.872 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.872 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-05-20T16:25:41.872 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "test_orchestrator", 2024-05-20T16:25:41.872 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-05-20T16:25:41.872 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-05-20T16:25:41.873 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-05-20T16:25:41.873 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-05-20T16:25:41.873 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-05-20T16:25:41.873 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.873 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.873 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.873 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:41.873 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.873 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.873 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-05-20T16:25:41.873 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-05-20T16:25:41.873 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-05-20T16:25:41.873 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-05-20T16:25:41.873 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-05-20T16:25:41.873 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-05-20T16:25:41.873 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-05-20T16:25:41.874 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:41.874 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.874 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.874 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.874 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.874 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.874 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-05-20T16:25:41.874 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-05-20T16:25:41.874 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.874 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.874 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.874 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:41.874 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.875 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.875 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.875 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.875 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.875 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.875 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.875 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.875 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-05-20T16:25:41.875 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-05-20T16:25:41.875 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.875 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.875 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.875 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-05-20T16:25:41.875 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.876 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.876 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-05-20T16:25:41.876 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-05-20T16:25:41.876 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-05-20T16:25:41.876 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-05-20T16:25:41.876 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-05-20T16:25:41.876 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-05-20T16:25:41.876 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-05-20T16:25:41.876 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:41.876 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.876 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.876 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.876 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.876 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.877 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-05-20T16:25:41.877 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-05-20T16:25:41.877 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.877 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.877 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.877 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:41.877 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.877 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.877 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.877 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.877 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.877 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.877 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.877 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-20T16:25:41.878 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-20T16:25:41.878 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.878 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-05-20T16:25:41.878 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "volumes", 2024-05-20T16:25:41.878 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-05-20T16:25:41.878 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-05-20T16:25:41.878 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-05-20T16:25:41.878 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-05-20T16:25:41.878 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-05-20T16:25:41.878 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.878 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.878 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.878 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:41.878 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.879 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.879 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-05-20T16:25:41.879 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-05-20T16:25:41.879 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-05-20T16:25:41.879 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-05-20T16:25:41.879 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-05-20T16:25:41.879 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-05-20T16:25:41.879 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-05-20T16:25:41.879 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:41.879 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.879 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.879 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.879 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.879 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.880 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-05-20T16:25:41.880 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-05-20T16:25:41.880 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.880 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.880 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.880 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:41.880 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.880 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.880 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.880 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.880 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.880 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.880 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.880 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.881 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-05-20T16:25:41.881 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-05-20T16:25:41.881 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.881 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.881 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.881 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-05-20T16:25:41.881 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.881 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.881 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-05-20T16:25:41.881 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-05-20T16:25:41.881 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-05-20T16:25:41.881 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-05-20T16:25:41.881 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-05-20T16:25:41.881 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-05-20T16:25:41.882 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-05-20T16:25:41.882 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:41.882 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.882 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.882 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.882 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.882 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.882 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-05-20T16:25:41.882 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-05-20T16:25:41.882 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.882 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.882 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.882 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:41.882 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.883 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.883 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.883 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.883 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.883 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.883 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.883 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.883 INFO:teuthology.orchestra.run.smithi082.stdout: "max_concurrent_clones": { 2024-05-20T16:25:41.883 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "max_concurrent_clones", 2024-05-20T16:25:41.883 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-05-20T16:25:41.883 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.883 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.883 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "4", 2024-05-20T16:25:41.883 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.884 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.884 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.884 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Number of asynchronous cloner threads", 2024-05-20T16:25:41.884 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.884 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.884 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.884 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.884 INFO:teuthology.orchestra.run.smithi082.stdout: "periodic_async_work": { 2024-05-20T16:25:41.884 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "periodic_async_work", 2024-05-20T16:25:41.884 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.884 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.884 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.884 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:41.884 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.884 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.885 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.885 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Periodically check for async work", 2024-05-20T16:25:41.885 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.885 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.885 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.885 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.885 INFO:teuthology.orchestra.run.smithi082.stdout: "snapshot_clone_delay": { 2024-05-20T16:25:41.885 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "snapshot_clone_delay", 2024-05-20T16:25:41.885 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-05-20T16:25:41.885 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.885 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.885 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "0", 2024-05-20T16:25:41.885 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.885 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.886 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.886 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-05-20T16:25:41.886 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.886 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.886 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.886 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.886 INFO:teuthology.orchestra.run.smithi082.stdout: "snapshot_clone_no_wait": { 2024-05-20T16:25:41.886 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "snapshot_clone_no_wait", 2024-05-20T16:25:41.886 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.886 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.886 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.886 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-05-20T16:25:41.886 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.887 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.887 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.887 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-05-20T16:25:41.887 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.887 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.887 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.887 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-20T16:25:41.887 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-20T16:25:41.887 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.887 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-05-20T16:25:41.887 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "zabbix", 2024-05-20T16:25:41.887 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-05-20T16:25:41.887 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-05-20T16:25:41.887 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-05-20T16:25:41.888 INFO:teuthology.orchestra.run.smithi082.stdout: "discovery_interval": { 2024-05-20T16:25:41.888 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "discovery_interval", 2024-05-20T16:25:41.888 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "uint", 2024-05-20T16:25:41.888 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.888 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.888 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "100", 2024-05-20T16:25:41.888 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.888 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.888 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.888 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.888 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.888 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.888 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.888 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.889 INFO:teuthology.orchestra.run.smithi082.stdout: "identifier": { 2024-05-20T16:25:41.889 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "identifier", 2024-05-20T16:25:41.889 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.889 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.889 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.889 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:41.889 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.889 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.889 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.889 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.889 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.889 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.889 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.889 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.889 INFO:teuthology.orchestra.run.smithi082.stdout: "interval": { 2024-05-20T16:25:41.890 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "interval", 2024-05-20T16:25:41.890 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2024-05-20T16:25:41.890 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.890 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.890 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "60", 2024-05-20T16:25:41.890 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.890 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.890 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.890 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.890 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.890 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.890 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.890 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.890 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-05-20T16:25:41.891 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-05-20T16:25:41.891 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.891 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.891 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.891 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:41.891 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.891 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.891 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-05-20T16:25:41.891 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-05-20T16:25:41.891 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-05-20T16:25:41.891 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-05-20T16:25:41.891 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-05-20T16:25:41.891 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-05-20T16:25:41.891 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-05-20T16:25:41.892 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:41.892 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.892 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.892 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.892 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.892 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.892 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-05-20T16:25:41.892 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-05-20T16:25:41.892 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.892 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.892 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.892 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:41.892 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.892 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.892 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.893 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.893 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.893 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.893 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.893 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.893 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-05-20T16:25:41.893 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-05-20T16:25:41.893 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.893 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.893 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.893 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-05-20T16:25:41.893 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.893 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.893 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-05-20T16:25:41.893 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-05-20T16:25:41.894 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-05-20T16:25:41.894 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-05-20T16:25:41.894 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-05-20T16:25:41.894 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-05-20T16:25:41.894 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-05-20T16:25:41.894 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:41.894 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.894 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.894 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.894 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.894 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.894 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-05-20T16:25:41.894 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-05-20T16:25:41.894 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.895 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.895 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.895 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:41.895 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.895 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.895 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.895 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.895 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.895 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.895 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.895 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.895 INFO:teuthology.orchestra.run.smithi082.stdout: "zabbix_host": { 2024-05-20T16:25:41.895 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "zabbix_host", 2024-05-20T16:25:41.895 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.895 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.896 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.896 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:41.896 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.896 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.896 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.896 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.896 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.896 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.896 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.896 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.896 INFO:teuthology.orchestra.run.smithi082.stdout: "zabbix_port": { 2024-05-20T16:25:41.896 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "zabbix_port", 2024-05-20T16:25:41.896 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-05-20T16:25:41.896 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.896 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.897 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "10051", 2024-05-20T16:25:41.897 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.897 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.897 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.897 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.897 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.897 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.897 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.897 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.897 INFO:teuthology.orchestra.run.smithi082.stdout: "zabbix_sender": { 2024-05-20T16:25:41.897 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "zabbix_sender", 2024-05-20T16:25:41.897 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.897 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.897 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.898 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-05-20T16:25:41.898 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.898 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.898 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.898 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.898 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.898 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.898 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.898 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-20T16:25:41.898 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-20T16:25:41.898 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-20T16:25:41.898 INFO:teuthology.orchestra.run.smithi082.stdout: ] 2024-05-20T16:25:41.898 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-20T16:25:41.898 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:41.899 INFO:teuthology.orchestra.run.smithi082.stdout: "modules": [ 2024-05-20T16:25:41.899 INFO:teuthology.orchestra.run.smithi082.stdout: "cephadm", 2024-05-20T16:25:41.899 INFO:teuthology.orchestra.run.smithi082.stdout: "dashboard", 2024-05-20T16:25:41.899 INFO:teuthology.orchestra.run.smithi082.stdout: "iostat", 2024-05-20T16:25:41.899 INFO:teuthology.orchestra.run.smithi082.stdout: "nfs", 2024-05-20T16:25:41.899 INFO:teuthology.orchestra.run.smithi082.stdout: "restful" 2024-05-20T16:25:41.899 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:41.899 INFO:teuthology.orchestra.run.smithi082.stdout: "available_modules": [ 2024-05-20T16:25:41.899 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-05-20T16:25:41.899 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "alerts", 2024-05-20T16:25:41.899 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-05-20T16:25:41.899 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-05-20T16:25:41.899 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-05-20T16:25:41.899 INFO:teuthology.orchestra.run.smithi082.stdout: "interval": { 2024-05-20T16:25:41.899 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "interval", 2024-05-20T16:25:41.900 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2024-05-20T16:25:41.900 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.900 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.900 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "60", 2024-05-20T16:25:41.900 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.900 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.900 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.900 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "How frequently to reexamine health status", 2024-05-20T16:25:41.900 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.900 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.900 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.900 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.900 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-05-20T16:25:41.901 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-05-20T16:25:41.901 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.901 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.901 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.901 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:41.901 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.901 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.901 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-05-20T16:25:41.901 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-05-20T16:25:41.901 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-05-20T16:25:41.901 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-05-20T16:25:41.901 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-05-20T16:25:41.901 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-05-20T16:25:41.901 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-05-20T16:25:41.902 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:41.902 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.902 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.902 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.902 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.902 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.902 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-05-20T16:25:41.902 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-05-20T16:25:41.902 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.902 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.902 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.902 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:41.902 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.902 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.903 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.903 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.903 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.903 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.903 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.903 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.903 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-05-20T16:25:41.903 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-05-20T16:25:41.903 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.903 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.903 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.903 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-05-20T16:25:41.903 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.904 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.904 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-05-20T16:25:41.904 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-05-20T16:25:41.904 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-05-20T16:25:41.904 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-05-20T16:25:41.904 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-05-20T16:25:41.904 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-05-20T16:25:41.904 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-05-20T16:25:41.904 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:41.904 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.904 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.904 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.904 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.904 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.905 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-05-20T16:25:41.905 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-05-20T16:25:41.905 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.905 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.905 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.905 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:41.905 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.905 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.905 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.905 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.905 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.905 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.905 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.905 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.905 INFO:teuthology.orchestra.run.smithi082.stdout: "smtp_destination": { 2024-05-20T16:25:41.906 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "smtp_destination", 2024-05-20T16:25:41.906 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.906 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.906 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.906 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:41.906 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.906 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.906 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.906 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Email address to send alerts to", 2024-05-20T16:25:41.906 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.906 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.906 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.906 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.906 INFO:teuthology.orchestra.run.smithi082.stdout: "smtp_from_name": { 2024-05-20T16:25:41.907 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "smtp_from_name", 2024-05-20T16:25:41.907 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.907 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.907 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.907 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "Ceph", 2024-05-20T16:25:41.907 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.907 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.907 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.907 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Email From: name", 2024-05-20T16:25:41.907 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.907 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.907 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.907 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.907 INFO:teuthology.orchestra.run.smithi082.stdout: "smtp_host": { 2024-05-20T16:25:41.907 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "smtp_host", 2024-05-20T16:25:41.908 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.908 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.908 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.908 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:41.908 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.908 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.908 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.908 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "SMTP server", 2024-05-20T16:25:41.908 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.908 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.908 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.908 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.908 INFO:teuthology.orchestra.run.smithi082.stdout: "smtp_password": { 2024-05-20T16:25:41.908 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "smtp_password", 2024-05-20T16:25:41.909 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.909 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.909 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.909 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:41.909 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.909 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.909 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.909 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Password to authenticate with", 2024-05-20T16:25:41.909 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.909 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.909 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.909 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.909 INFO:teuthology.orchestra.run.smithi082.stdout: "smtp_port": { 2024-05-20T16:25:41.909 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "smtp_port", 2024-05-20T16:25:41.910 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-05-20T16:25:41.910 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.910 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.910 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "465", 2024-05-20T16:25:41.910 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.910 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.910 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.910 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "SMTP port", 2024-05-20T16:25:41.910 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.910 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.910 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.910 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.910 INFO:teuthology.orchestra.run.smithi082.stdout: "smtp_sender": { 2024-05-20T16:25:41.910 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "smtp_sender", 2024-05-20T16:25:41.911 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.911 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.911 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.911 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:41.911 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.911 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.911 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.911 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "SMTP envelope sender", 2024-05-20T16:25:41.911 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.911 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.911 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.911 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.911 INFO:teuthology.orchestra.run.smithi082.stdout: "smtp_ssl": { 2024-05-20T16:25:41.911 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "smtp_ssl", 2024-05-20T16:25:41.912 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.912 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.912 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.912 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-05-20T16:25:41.912 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.912 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.913 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.913 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Use SSL to connect to SMTP server", 2024-05-20T16:25:41.913 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.913 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.913 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.913 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.913 INFO:teuthology.orchestra.run.smithi082.stdout: "smtp_user": { 2024-05-20T16:25:41.913 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "smtp_user", 2024-05-20T16:25:41.913 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.913 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.913 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.913 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:41.913 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.913 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.914 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.914 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "User to authenticate as", 2024-05-20T16:25:41.914 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.914 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.914 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.914 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-20T16:25:41.914 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-20T16:25:41.914 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.914 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-05-20T16:25:41.914 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "balancer", 2024-05-20T16:25:41.914 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-05-20T16:25:41.914 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-05-20T16:25:41.914 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-05-20T16:25:41.914 INFO:teuthology.orchestra.run.smithi082.stdout: "active": { 2024-05-20T16:25:41.915 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "active", 2024-05-20T16:25:41.915 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.915 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.915 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.915 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-05-20T16:25:41.915 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.915 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.915 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.915 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "automatically balance PGs across cluster", 2024-05-20T16:25:41.915 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.915 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.915 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.915 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.915 INFO:teuthology.orchestra.run.smithi082.stdout: "begin_time": { 2024-05-20T16:25:41.916 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "begin_time", 2024-05-20T16:25:41.916 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.916 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.916 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.916 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "0000", 2024-05-20T16:25:41.916 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.916 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.916 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.916 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "beginning time of day to automatically balance", 2024-05-20T16:25:41.916 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-20T16:25:41.916 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.916 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.916 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.916 INFO:teuthology.orchestra.run.smithi082.stdout: "begin_weekday": { 2024-05-20T16:25:41.917 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "begin_weekday", 2024-05-20T16:25:41.917 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "uint", 2024-05-20T16:25:41.917 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.917 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.917 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "0", 2024-05-20T16:25:41.917 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "0", 2024-05-20T16:25:41.917 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "6", 2024-05-20T16:25:41.917 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.917 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-05-20T16:25:41.917 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-20T16:25:41.917 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.917 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.917 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.918 INFO:teuthology.orchestra.run.smithi082.stdout: "crush_compat_max_iterations": { 2024-05-20T16:25:41.918 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "crush_compat_max_iterations", 2024-05-20T16:25:41.918 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "uint", 2024-05-20T16:25:41.918 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.918 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.918 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "25", 2024-05-20T16:25:41.918 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "1", 2024-05-20T16:25:41.918 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "250", 2024-05-20T16:25:41.918 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.918 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-05-20T16:25:41.918 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.918 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.918 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.919 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.919 INFO:teuthology.orchestra.run.smithi082.stdout: "crush_compat_metrics": { 2024-05-20T16:25:41.919 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "crush_compat_metrics", 2024-05-20T16:25:41.919 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.919 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.919 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.919 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "pgs,objects,bytes", 2024-05-20T16:25:41.919 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.919 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.919 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.919 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-05-20T16:25:41.919 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-05-20T16:25:41.919 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.919 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.920 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.920 INFO:teuthology.orchestra.run.smithi082.stdout: "crush_compat_step": { 2024-05-20T16:25:41.920 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "crush_compat_step", 2024-05-20T16:25:41.920 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "float", 2024-05-20T16:25:41.920 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.920 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.920 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "0.5", 2024-05-20T16:25:41.920 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "0.001", 2024-05-20T16:25:41.920 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "0.999", 2024-05-20T16:25:41.920 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.920 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "aggressiveness of optimization", 2024-05-20T16:25:41.920 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-05-20T16:25:41.920 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.920 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.920 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.921 INFO:teuthology.orchestra.run.smithi082.stdout: "end_time": { 2024-05-20T16:25:41.921 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "end_time", 2024-05-20T16:25:41.921 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.921 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.921 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.921 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "2359", 2024-05-20T16:25:41.921 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.921 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.921 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.921 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "ending time of day to automatically balance", 2024-05-20T16:25:41.921 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-20T16:25:41.921 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.921 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.922 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.922 INFO:teuthology.orchestra.run.smithi082.stdout: "end_weekday": { 2024-05-20T16:25:41.922 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "end_weekday", 2024-05-20T16:25:41.922 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "uint", 2024-05-20T16:25:41.922 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.922 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.922 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "0", 2024-05-20T16:25:41.922 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "0", 2024-05-20T16:25:41.922 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "6", 2024-05-20T16:25:41.922 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.922 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-05-20T16:25:41.922 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-20T16:25:41.922 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.922 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.923 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.923 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-05-20T16:25:41.923 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-05-20T16:25:41.923 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.923 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.923 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.923 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:41.923 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.923 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.923 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-05-20T16:25:41.923 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-05-20T16:25:41.923 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-05-20T16:25:41.923 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-05-20T16:25:41.923 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-05-20T16:25:41.924 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-05-20T16:25:41.924 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-05-20T16:25:41.924 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:41.924 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.924 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.924 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.924 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.924 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.924 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-05-20T16:25:41.924 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-05-20T16:25:41.924 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.924 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.924 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.924 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:41.925 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.925 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.925 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.925 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.925 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.925 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.925 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.925 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.925 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-05-20T16:25:41.925 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-05-20T16:25:41.925 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.925 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.925 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.925 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-05-20T16:25:41.926 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.926 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.926 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-05-20T16:25:41.926 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-05-20T16:25:41.926 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-05-20T16:25:41.926 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-05-20T16:25:41.926 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-05-20T16:25:41.926 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-05-20T16:25:41.926 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-05-20T16:25:41.926 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:41.926 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.926 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.926 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.926 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.927 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.927 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-05-20T16:25:41.927 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-05-20T16:25:41.927 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.927 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.927 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.927 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:41.927 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.927 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.927 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.927 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.927 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.927 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.927 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.927 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.928 INFO:teuthology.orchestra.run.smithi082.stdout: "min_score": { 2024-05-20T16:25:41.928 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "min_score", 2024-05-20T16:25:41.928 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "float", 2024-05-20T16:25:41.928 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.928 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.928 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "0", 2024-05-20T16:25:41.928 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.928 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.928 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.928 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-05-20T16:25:41.928 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.928 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.928 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.929 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.929 INFO:teuthology.orchestra.run.smithi082.stdout: "mode": { 2024-05-20T16:25:41.929 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "mode", 2024-05-20T16:25:41.929 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.929 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.929 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.929 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "upmap", 2024-05-20T16:25:41.929 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.929 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.929 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-05-20T16:25:41.929 INFO:teuthology.orchestra.run.smithi082.stdout: "crush-compat", 2024-05-20T16:25:41.929 INFO:teuthology.orchestra.run.smithi082.stdout: "none", 2024-05-20T16:25:41.929 INFO:teuthology.orchestra.run.smithi082.stdout: "read", 2024-05-20T16:25:41.929 INFO:teuthology.orchestra.run.smithi082.stdout: "upmap", 2024-05-20T16:25:41.930 INFO:teuthology.orchestra.run.smithi082.stdout: "upmap-read" 2024-05-20T16:25:41.930 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:41.930 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Balancer mode", 2024-05-20T16:25:41.930 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.930 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.930 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.930 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.930 INFO:teuthology.orchestra.run.smithi082.stdout: "pool_ids": { 2024-05-20T16:25:41.930 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "pool_ids", 2024-05-20T16:25:41.930 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.930 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.930 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.930 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:41.930 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.931 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.931 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.931 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-05-20T16:25:41.931 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.931 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.931 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.931 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.931 INFO:teuthology.orchestra.run.smithi082.stdout: "sleep_interval": { 2024-05-20T16:25:41.931 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "sleep_interval", 2024-05-20T16:25:41.931 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2024-05-20T16:25:41.931 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.931 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.931 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "60", 2024-05-20T16:25:41.931 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.932 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.932 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.932 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-05-20T16:25:41.932 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.932 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.932 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.932 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.932 INFO:teuthology.orchestra.run.smithi082.stdout: "upmap_max_deviation": { 2024-05-20T16:25:41.932 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "upmap_max_deviation", 2024-05-20T16:25:41.932 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-05-20T16:25:41.932 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.932 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.932 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "5", 2024-05-20T16:25:41.933 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "1", 2024-05-20T16:25:41.933 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.933 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.933 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "deviation below which no optimization is attempted", 2024-05-20T16:25:41.933 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-05-20T16:25:41.933 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.933 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.933 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.933 INFO:teuthology.orchestra.run.smithi082.stdout: "upmap_max_optimizations": { 2024-05-20T16:25:41.933 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "upmap_max_optimizations", 2024-05-20T16:25:41.933 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "uint", 2024-05-20T16:25:41.933 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.933 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.933 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "10", 2024-05-20T16:25:41.933 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.934 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.934 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.934 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-05-20T16:25:41.934 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.934 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.934 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.934 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-20T16:25:41.934 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-20T16:25:41.934 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.934 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-05-20T16:25:41.934 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "cephadm", 2024-05-20T16:25:41.934 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-05-20T16:25:41.934 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-05-20T16:25:41.935 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-05-20T16:25:41.935 INFO:teuthology.orchestra.run.smithi082.stdout: "agent_down_multiplier": { 2024-05-20T16:25:41.935 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "agent_down_multiplier", 2024-05-20T16:25:41.935 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "float", 2024-05-20T16:25:41.935 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.935 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.935 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "3.0", 2024-05-20T16:25:41.935 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.935 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.935 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.935 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-05-20T16:25:41.935 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.935 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.935 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.936 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.936 INFO:teuthology.orchestra.run.smithi082.stdout: "agent_refresh_rate": { 2024-05-20T16:25:41.936 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "agent_refresh_rate", 2024-05-20T16:25:41.936 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2024-05-20T16:25:41.936 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.936 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.936 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "20", 2024-05-20T16:25:41.936 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.936 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.936 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.936 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-05-20T16:25:41.936 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.936 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.936 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.936 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.937 INFO:teuthology.orchestra.run.smithi082.stdout: "agent_starting_port": { 2024-05-20T16:25:41.937 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "agent_starting_port", 2024-05-20T16:25:41.937 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-05-20T16:25:41.937 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.937 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.937 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "4721", 2024-05-20T16:25:41.937 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.937 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.937 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.937 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-05-20T16:25:41.937 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.937 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.937 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.938 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.938 INFO:teuthology.orchestra.run.smithi082.stdout: "allow_ptrace": { 2024-05-20T16:25:41.938 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "allow_ptrace", 2024-05-20T16:25:41.938 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.938 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.938 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.938 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:41.938 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.938 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.938 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.938 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-05-20T16:25:41.938 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-05-20T16:25:41.938 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.939 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.939 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.939 INFO:teuthology.orchestra.run.smithi082.stdout: "autotune_interval": { 2024-05-20T16:25:41.939 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "autotune_interval", 2024-05-20T16:25:41.939 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2024-05-20T16:25:41.939 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.939 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.939 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "600", 2024-05-20T16:25:41.939 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.939 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.939 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.939 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "how frequently to autotune daemon memory", 2024-05-20T16:25:41.939 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.939 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.940 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.940 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.940 INFO:teuthology.orchestra.run.smithi082.stdout: "autotune_memory_target_ratio": { 2024-05-20T16:25:41.940 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "autotune_memory_target_ratio", 2024-05-20T16:25:41.940 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "float", 2024-05-20T16:25:41.940 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.940 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.940 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "0.7", 2024-05-20T16:25:41.940 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.940 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.940 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.940 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-05-20T16:25:41.940 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.940 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.941 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.941 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.941 INFO:teuthology.orchestra.run.smithi082.stdout: "cephadm_log_destination": { 2024-05-20T16:25:41.941 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "cephadm_log_destination", 2024-05-20T16:25:41.941 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.941 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.941 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.941 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:41.941 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.941 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.941 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-05-20T16:25:41.941 INFO:teuthology.orchestra.run.smithi082.stdout: "file", 2024-05-20T16:25:41.941 INFO:teuthology.orchestra.run.smithi082.stdout: "file,syslog", 2024-05-20T16:25:41.941 INFO:teuthology.orchestra.run.smithi082.stdout: "syslog" 2024-05-20T16:25:41.942 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:41.942 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-05-20T16:25:41.942 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.942 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.942 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.942 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.942 INFO:teuthology.orchestra.run.smithi082.stdout: "cgroups_split": { 2024-05-20T16:25:41.942 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "cgroups_split", 2024-05-20T16:25:41.942 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.942 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.942 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.942 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-05-20T16:25:41.942 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.942 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.943 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.943 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-05-20T16:25:41.943 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.943 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.943 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.943 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.943 INFO:teuthology.orchestra.run.smithi082.stdout: "config_checks_enabled": { 2024-05-20T16:25:41.943 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "config_checks_enabled", 2024-05-20T16:25:41.943 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.943 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.943 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.943 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:41.943 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.943 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.944 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.944 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-05-20T16:25:41.944 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.944 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.944 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.944 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.944 INFO:teuthology.orchestra.run.smithi082.stdout: "config_dashboard": { 2024-05-20T16:25:41.944 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "config_dashboard", 2024-05-20T16:25:41.944 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.944 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.944 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.944 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-05-20T16:25:41.944 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.944 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.945 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.945 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-05-20T16:25:41.945 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.945 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.945 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.945 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.945 INFO:teuthology.orchestra.run.smithi082.stdout: "container_image_alertmanager": { 2024-05-20T16:25:41.945 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "container_image_alertmanager", 2024-05-20T16:25:41.945 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.945 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.945 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.945 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-05-20T16:25:41.945 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.945 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.946 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.946 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Prometheus container image", 2024-05-20T16:25:41.946 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.946 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.946 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.946 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.946 INFO:teuthology.orchestra.run.smithi082.stdout: "container_image_base": { 2024-05-20T16:25:41.946 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "container_image_base", 2024-05-20T16:25:41.946 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.946 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.946 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.946 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "quay.io/ceph/ceph", 2024-05-20T16:25:41.946 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.946 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.946 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.947 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Container image name, without the tag", 2024-05-20T16:25:41.947 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.947 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.947 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.947 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.947 INFO:teuthology.orchestra.run.smithi082.stdout: "container_image_elasticsearch": { 2024-05-20T16:25:41.947 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "container_image_elasticsearch", 2024-05-20T16:25:41.947 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.947 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.947 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.947 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-05-20T16:25:41.947 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.947 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.948 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.948 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "elasticsearch container image", 2024-05-20T16:25:41.948 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.948 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.948 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.948 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.948 INFO:teuthology.orchestra.run.smithi082.stdout: "container_image_grafana": { 2024-05-20T16:25:41.948 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "container_image_grafana", 2024-05-20T16:25:41.948 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.948 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.948 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.948 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-05-20T16:25:41.948 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.948 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.949 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.949 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Prometheus container image", 2024-05-20T16:25:41.949 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.949 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.949 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.949 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.949 INFO:teuthology.orchestra.run.smithi082.stdout: "container_image_haproxy": { 2024-05-20T16:25:41.949 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "container_image_haproxy", 2024-05-20T16:25:41.949 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.949 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.949 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.949 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-05-20T16:25:41.949 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.949 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.950 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.950 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "HAproxy container image", 2024-05-20T16:25:41.950 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.950 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.950 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.950 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.950 INFO:teuthology.orchestra.run.smithi082.stdout: "container_image_jaeger_agent": { 2024-05-20T16:25:41.950 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "container_image_jaeger_agent", 2024-05-20T16:25:41.950 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.950 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.950 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.950 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-05-20T16:25:41.950 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.950 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.951 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.951 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Jaeger agent container image", 2024-05-20T16:25:41.951 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.951 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.951 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.951 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.951 INFO:teuthology.orchestra.run.smithi082.stdout: "container_image_jaeger_collector": { 2024-05-20T16:25:41.951 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "container_image_jaeger_collector", 2024-05-20T16:25:41.951 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.951 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.951 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.951 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-05-20T16:25:41.951 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.952 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.952 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.952 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Jaeger collector container image", 2024-05-20T16:25:41.952 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.952 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.952 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.952 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.952 INFO:teuthology.orchestra.run.smithi082.stdout: "container_image_jaeger_query": { 2024-05-20T16:25:41.952 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "container_image_jaeger_query", 2024-05-20T16:25:41.952 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.952 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.952 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.952 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-05-20T16:25:41.952 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.953 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.953 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.953 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Jaeger query container image", 2024-05-20T16:25:41.953 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.953 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.953 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.953 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.953 INFO:teuthology.orchestra.run.smithi082.stdout: "container_image_keepalived": { 2024-05-20T16:25:41.953 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "container_image_keepalived", 2024-05-20T16:25:41.953 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.953 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.953 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.953 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-05-20T16:25:41.953 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.954 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.954 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.954 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Keepalived container image", 2024-05-20T16:25:41.954 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.954 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.954 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.954 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.954 INFO:teuthology.orchestra.run.smithi082.stdout: "container_image_loki": { 2024-05-20T16:25:41.954 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "container_image_loki", 2024-05-20T16:25:41.954 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.954 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.954 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.954 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-05-20T16:25:41.954 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.955 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.955 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.955 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Loki container image", 2024-05-20T16:25:41.955 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.955 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.955 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.955 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.955 INFO:teuthology.orchestra.run.smithi082.stdout: "container_image_node_exporter": { 2024-05-20T16:25:41.955 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "container_image_node_exporter", 2024-05-20T16:25:41.955 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.955 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.955 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.955 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-05-20T16:25:41.955 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.956 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.956 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.956 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Prometheus container image", 2024-05-20T16:25:41.956 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.956 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.956 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.956 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.956 INFO:teuthology.orchestra.run.smithi082.stdout: "container_image_nvmeof": { 2024-05-20T16:25:41.956 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "container_image_nvmeof", 2024-05-20T16:25:41.956 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.956 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.956 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.956 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-05-20T16:25:41.956 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.957 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.957 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.957 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Nvme-of container image", 2024-05-20T16:25:41.957 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.957 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.957 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.957 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.957 INFO:teuthology.orchestra.run.smithi082.stdout: "container_image_prometheus": { 2024-05-20T16:25:41.957 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "container_image_prometheus", 2024-05-20T16:25:41.957 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.957 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.957 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.957 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-05-20T16:25:41.957 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.957 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.958 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.958 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Prometheus container image", 2024-05-20T16:25:41.958 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.958 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.958 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.958 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.958 INFO:teuthology.orchestra.run.smithi082.stdout: "container_image_promtail": { 2024-05-20T16:25:41.958 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "container_image_promtail", 2024-05-20T16:25:41.958 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.958 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.958 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.958 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-05-20T16:25:41.958 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.958 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.959 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.959 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Promtail container image", 2024-05-20T16:25:41.959 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.959 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.959 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.959 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.959 INFO:teuthology.orchestra.run.smithi082.stdout: "container_image_samba": { 2024-05-20T16:25:41.959 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "container_image_samba", 2024-05-20T16:25:41.959 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.959 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.959 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.959 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-05-20T16:25:41.959 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.959 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.960 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.960 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Samba/SMB container image", 2024-05-20T16:25:41.960 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.960 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.960 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.960 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.960 INFO:teuthology.orchestra.run.smithi082.stdout: "container_image_snmp_gateway": { 2024-05-20T16:25:41.960 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "container_image_snmp_gateway", 2024-05-20T16:25:41.960 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.960 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.960 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.960 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-05-20T16:25:41.960 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.960 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.960 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.961 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "SNMP Gateway container image", 2024-05-20T16:25:41.961 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.961 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.961 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.961 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.961 INFO:teuthology.orchestra.run.smithi082.stdout: "container_init": { 2024-05-20T16:25:41.961 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "container_init", 2024-05-20T16:25:41.961 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.961 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.961 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.961 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-05-20T16:25:41.961 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.961 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.961 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.962 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Run podman/docker with `--init`", 2024-05-20T16:25:41.962 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.962 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.962 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.962 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.962 INFO:teuthology.orchestra.run.smithi082.stdout: "daemon_cache_timeout": { 2024-05-20T16:25:41.962 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "daemon_cache_timeout", 2024-05-20T16:25:41.962 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2024-05-20T16:25:41.962 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.962 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.962 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "600", 2024-05-20T16:25:41.962 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.962 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.963 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.963 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-05-20T16:25:41.963 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.963 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.963 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.963 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.963 INFO:teuthology.orchestra.run.smithi082.stdout: "default_cephadm_command_timeout": { 2024-05-20T16:25:41.963 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "default_cephadm_command_timeout", 2024-05-20T16:25:41.963 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-05-20T16:25:41.963 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.963 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.963 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "900", 2024-05-20T16:25:41.963 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.963 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.964 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.964 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-05-20T16:25:41.964 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.964 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.964 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.964 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.964 INFO:teuthology.orchestra.run.smithi082.stdout: "default_registry": { 2024-05-20T16:25:41.964 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "default_registry", 2024-05-20T16:25:41.964 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.964 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.964 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.964 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "docker.io", 2024-05-20T16:25:41.964 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.965 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.965 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.965 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-05-20T16:25:41.965 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.965 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.965 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.965 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.965 INFO:teuthology.orchestra.run.smithi082.stdout: "device_cache_timeout": { 2024-05-20T16:25:41.965 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "device_cache_timeout", 2024-05-20T16:25:41.965 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2024-05-20T16:25:41.965 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.965 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.965 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "1800", 2024-05-20T16:25:41.966 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.966 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.966 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.966 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "seconds to cache device inventory", 2024-05-20T16:25:41.966 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.966 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.966 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.966 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.966 INFO:teuthology.orchestra.run.smithi082.stdout: "device_enhanced_scan": { 2024-05-20T16:25:41.966 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "device_enhanced_scan", 2024-05-20T16:25:41.966 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.966 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.966 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.966 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:41.967 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.967 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.967 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.967 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Use libstoragemgmt during device scans", 2024-05-20T16:25:41.967 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.967 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.967 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.967 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.967 INFO:teuthology.orchestra.run.smithi082.stdout: "facts_cache_timeout": { 2024-05-20T16:25:41.967 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "facts_cache_timeout", 2024-05-20T16:25:41.967 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2024-05-20T16:25:41.967 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.968 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.968 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "60", 2024-05-20T16:25:41.968 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.968 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.968 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.968 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "seconds to cache host facts data", 2024-05-20T16:25:41.968 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.968 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.968 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.968 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.968 INFO:teuthology.orchestra.run.smithi082.stdout: "grafana_dashboards_path": { 2024-05-20T16:25:41.968 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "grafana_dashboards_path", 2024-05-20T16:25:41.969 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.969 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.969 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.969 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-05-20T16:25:41.969 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.969 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.969 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.969 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-05-20T16:25:41.969 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.969 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.969 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.969 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.970 INFO:teuthology.orchestra.run.smithi082.stdout: "host_check_interval": { 2024-05-20T16:25:41.970 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "host_check_interval", 2024-05-20T16:25:41.970 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2024-05-20T16:25:41.970 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.970 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.970 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "600", 2024-05-20T16:25:41.970 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.970 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.970 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.970 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "how frequently to perform a host check", 2024-05-20T16:25:41.971 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.971 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.971 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.971 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.971 INFO:teuthology.orchestra.run.smithi082.stdout: "hw_monitoring": { 2024-05-20T16:25:41.971 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "hw_monitoring", 2024-05-20T16:25:41.971 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.971 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.971 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.971 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:41.971 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.971 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.971 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.972 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-05-20T16:25:41.972 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.972 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.972 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.972 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.972 INFO:teuthology.orchestra.run.smithi082.stdout: "inventory_list_all": { 2024-05-20T16:25:41.972 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "inventory_list_all", 2024-05-20T16:25:41.972 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.972 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.972 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.973 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:41.973 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.973 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.973 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.973 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-05-20T16:25:41.973 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.973 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.973 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.973 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.973 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-05-20T16:25:41.973 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-05-20T16:25:41.973 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.973 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.974 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.974 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:41.974 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.974 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.974 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-05-20T16:25:41.974 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-05-20T16:25:41.974 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-05-20T16:25:41.974 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-05-20T16:25:41.974 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-05-20T16:25:41.974 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-05-20T16:25:41.974 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-05-20T16:25:41.974 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:41.974 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.974 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.975 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.975 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.975 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.975 INFO:teuthology.orchestra.run.smithi082.stdout: "log_refresh_metadata": { 2024-05-20T16:25:41.975 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_refresh_metadata", 2024-05-20T16:25:41.975 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.975 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.975 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.975 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:41.975 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.975 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.975 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.975 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-05-20T16:25:41.975 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.976 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.976 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.976 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.976 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-05-20T16:25:41.976 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-05-20T16:25:41.976 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.976 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.976 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.976 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-05-20T16:25:41.976 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.976 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.976 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.976 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-05-20T16:25:41.976 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.977 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.977 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.977 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.977 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-05-20T16:25:41.977 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-05-20T16:25:41.977 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.977 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.977 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.977 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-05-20T16:25:41.977 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.977 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.977 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-05-20T16:25:41.977 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-05-20T16:25:41.977 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-05-20T16:25:41.978 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-05-20T16:25:41.978 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-05-20T16:25:41.978 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-05-20T16:25:41.978 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-05-20T16:25:41.978 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:41.978 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.978 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.978 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.978 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.978 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.978 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-05-20T16:25:41.978 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-05-20T16:25:41.978 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.979 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.979 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:41.979 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:41.979 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.979 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.979 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.979 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:41.979 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.979 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.979 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.979 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.979 INFO:teuthology.orchestra.run.smithi082.stdout: "manage_etc_ceph_ceph_conf": { 2024-05-20T16:25:41.979 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-05-20T16:25:41.979 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.980 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.980 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.980 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:41.980 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.980 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.980 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.980 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-05-20T16:25:41.980 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.980 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.980 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.980 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.980 INFO:teuthology.orchestra.run.smithi082.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-05-20T16:25:41.980 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-05-20T16:25:41.980 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.981 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.981 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.981 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "*", 2024-05-20T16:25:41.981 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.981 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.981 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.981 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-05-20T16:25:41.981 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.981 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.981 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.981 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.981 INFO:teuthology.orchestra.run.smithi082.stdout: "max_count_per_host": { 2024-05-20T16:25:41.981 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "max_count_per_host", 2024-05-20T16:25:41.981 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-05-20T16:25:41.982 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.982 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.982 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "10", 2024-05-20T16:25:41.982 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.982 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.982 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.982 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "max number of daemons per service per host", 2024-05-20T16:25:41.982 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.982 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.982 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.982 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.982 INFO:teuthology.orchestra.run.smithi082.stdout: "max_osd_draining_count": { 2024-05-20T16:25:41.982 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "max_osd_draining_count", 2024-05-20T16:25:41.983 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-05-20T16:25:41.983 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.983 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.983 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "10", 2024-05-20T16:25:41.983 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.983 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.983 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.983 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-05-20T16:25:41.983 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.983 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.983 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.983 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.983 INFO:teuthology.orchestra.run.smithi082.stdout: "migration_current": { 2024-05-20T16:25:41.983 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "migration_current", 2024-05-20T16:25:41.984 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-05-20T16:25:41.984 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.984 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.984 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:41.984 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.984 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.984 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.984 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "internal - do not modify", 2024-05-20T16:25:41.984 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.984 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.984 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.984 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.984 INFO:teuthology.orchestra.run.smithi082.stdout: "mode": { 2024-05-20T16:25:41.984 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "mode", 2024-05-20T16:25:41.985 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.985 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.985 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.985 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "root", 2024-05-20T16:25:41.985 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.985 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.985 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-05-20T16:25:41.985 INFO:teuthology.orchestra.run.smithi082.stdout: "cephadm-package", 2024-05-20T16:25:41.985 INFO:teuthology.orchestra.run.smithi082.stdout: "root" 2024-05-20T16:25:41.985 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:41.985 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "mode for remote execution of cephadm", 2024-05-20T16:25:41.985 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.985 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.985 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.986 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.986 INFO:teuthology.orchestra.run.smithi082.stdout: "oob_default_addr": { 2024-05-20T16:25:41.986 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "oob_default_addr", 2024-05-20T16:25:41.986 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.986 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.986 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.986 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "169.254.1.1", 2024-05-20T16:25:41.986 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.986 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.986 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.986 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Default address for RedFish API (oob management).", 2024-05-20T16:25:41.986 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.986 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.986 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.987 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.987 INFO:teuthology.orchestra.run.smithi082.stdout: "prometheus_alerts_path": { 2024-05-20T16:25:41.987 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "prometheus_alerts_path", 2024-05-20T16:25:41.987 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.987 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.987 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.987 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-05-20T16:25:41.987 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.987 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.987 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.987 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-05-20T16:25:41.987 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.987 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.987 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.988 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.988 INFO:teuthology.orchestra.run.smithi082.stdout: "registry_insecure": { 2024-05-20T16:25:41.988 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "registry_insecure", 2024-05-20T16:25:41.988 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.988 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.988 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.988 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:41.988 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.988 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.988 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.988 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-05-20T16:25:41.988 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.988 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.988 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.989 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.989 INFO:teuthology.orchestra.run.smithi082.stdout: "registry_password": { 2024-05-20T16:25:41.989 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "registry_password", 2024-05-20T16:25:41.989 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.989 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.989 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.989 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:41.989 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.989 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.989 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.989 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-05-20T16:25:41.989 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.989 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.989 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.990 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.990 INFO:teuthology.orchestra.run.smithi082.stdout: "registry_url": { 2024-05-20T16:25:41.990 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "registry_url", 2024-05-20T16:25:41.990 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.990 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.990 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.990 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:41.990 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.990 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.990 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.990 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-05-20T16:25:41.990 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.990 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.990 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.991 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.991 INFO:teuthology.orchestra.run.smithi082.stdout: "registry_username": { 2024-05-20T16:25:41.991 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "registry_username", 2024-05-20T16:25:41.991 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.991 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.991 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.991 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:41.991 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.991 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.991 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.991 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-05-20T16:25:41.991 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.992 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.992 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.992 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.992 INFO:teuthology.orchestra.run.smithi082.stdout: "secure_monitoring_stack": { 2024-05-20T16:25:41.992 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "secure_monitoring_stack", 2024-05-20T16:25:41.992 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.992 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.992 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.992 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:41.992 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.992 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.992 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.992 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-05-20T16:25:41.992 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.993 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.993 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.993 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.993 INFO:teuthology.orchestra.run.smithi082.stdout: "service_discovery_port": { 2024-05-20T16:25:41.993 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "service_discovery_port", 2024-05-20T16:25:41.993 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-05-20T16:25:41.993 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.993 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.993 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "8765", 2024-05-20T16:25:41.993 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.993 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.993 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.993 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "cephadm service discovery port", 2024-05-20T16:25:41.993 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.994 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.994 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.994 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.994 INFO:teuthology.orchestra.run.smithi082.stdout: "ssh_config_file": { 2024-05-20T16:25:41.994 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "ssh_config_file", 2024-05-20T16:25:41.994 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:41.994 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.994 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.994 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:41.994 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.994 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.994 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.994 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-05-20T16:25:41.995 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.995 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.995 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.995 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.995 INFO:teuthology.orchestra.run.smithi082.stdout: "use_agent": { 2024-05-20T16:25:41.995 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "use_agent", 2024-05-20T16:25:41.995 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.995 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.995 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.995 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:41.995 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.995 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.995 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.995 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-05-20T16:25:41.996 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.996 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.996 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.996 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.996 INFO:teuthology.orchestra.run.smithi082.stdout: "use_repo_digest": { 2024-05-20T16:25:41.996 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "use_repo_digest", 2024-05-20T16:25:41.996 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.996 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.996 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.996 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-05-20T16:25:41.996 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.996 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.996 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.996 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-05-20T16:25:41.997 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.997 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.997 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.997 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.997 INFO:teuthology.orchestra.run.smithi082.stdout: "warn_on_failed_host_check": { 2024-05-20T16:25:41.997 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "warn_on_failed_host_check", 2024-05-20T16:25:41.997 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.997 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.997 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.997 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-05-20T16:25:41.997 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.997 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.997 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.997 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "raise a health warning if the host check fails", 2024-05-20T16:25:41.998 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.998 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.998 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.998 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.998 INFO:teuthology.orchestra.run.smithi082.stdout: "warn_on_stray_daemons": { 2024-05-20T16:25:41.998 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "warn_on_stray_daemons", 2024-05-20T16:25:41.998 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.998 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.998 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.998 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-05-20T16:25:41.998 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.998 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.998 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:41.999 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-05-20T16:25:41.999 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:41.999 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:41.999 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:41.999 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:41.999 INFO:teuthology.orchestra.run.smithi082.stdout: "warn_on_stray_hosts": { 2024-05-20T16:25:41.999 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "warn_on_stray_hosts", 2024-05-20T16:25:41.999 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:41.999 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:41.999 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:41.999 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-05-20T16:25:41.999 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:41.999 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:41.999 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:42.000 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-05-20T16:25:42.000 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.000 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.000 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.000 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-20T16:25:42.000 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-20T16:25:42.000 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.000 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-05-20T16:25:42.000 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "crash", 2024-05-20T16:25:42.000 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-05-20T16:25:42.000 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-05-20T16:25:42.000 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-05-20T16:25:42.000 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-05-20T16:25:42.001 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-05-20T16:25:42.001 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:42.001 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.001 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:42.001 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:42.001 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.001 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.001 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-05-20T16:25:42.001 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-05-20T16:25:42.001 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-05-20T16:25:42.001 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-05-20T16:25:42.001 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-05-20T16:25:42.001 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-05-20T16:25:42.001 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-05-20T16:25:42.002 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:42.002 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.002 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.002 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.002 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.002 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.002 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-05-20T16:25:42.002 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-05-20T16:25:42.002 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:42.002 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.002 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:42.002 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:42.002 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.003 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.003 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:42.003 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.003 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.003 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.003 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.003 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.003 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-05-20T16:25:42.003 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-05-20T16:25:42.003 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:42.003 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.003 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:42.003 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-05-20T16:25:42.003 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.004 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.004 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-05-20T16:25:42.004 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-05-20T16:25:42.004 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-05-20T16:25:42.004 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-05-20T16:25:42.004 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-05-20T16:25:42.004 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-05-20T16:25:42.004 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-05-20T16:25:42.004 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:42.004 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.004 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.004 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.004 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.004 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.005 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-05-20T16:25:42.005 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-05-20T16:25:42.005 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:42.005 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.005 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:42.005 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:42.005 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.005 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.005 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:42.005 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.005 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.005 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.005 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.005 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.006 INFO:teuthology.orchestra.run.smithi082.stdout: "retain_interval": { 2024-05-20T16:25:42.006 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "retain_interval", 2024-05-20T16:25:42.006 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2024-05-20T16:25:42.006 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.006 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:42.006 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "31536000", 2024-05-20T16:25:42.006 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.006 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.006 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:42.006 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "how long to retain crashes before pruning them", 2024-05-20T16:25:42.006 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.006 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.006 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.007 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.007 INFO:teuthology.orchestra.run.smithi082.stdout: "warn_recent_interval": { 2024-05-20T16:25:42.007 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "warn_recent_interval", 2024-05-20T16:25:42.007 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2024-05-20T16:25:42.007 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.007 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:42.007 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "1209600", 2024-05-20T16:25:42.007 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.007 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.007 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:42.007 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "time interval in which to warn about recent crashes", 2024-05-20T16:25:42.007 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.008 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.008 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.008 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-20T16:25:42.008 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-20T16:25:42.008 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.008 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-05-20T16:25:42.008 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "dashboard", 2024-05-20T16:25:42.008 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-05-20T16:25:42.008 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-05-20T16:25:42.008 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-05-20T16:25:42.008 INFO:teuthology.orchestra.run.smithi082.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-05-20T16:25:42.008 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-05-20T16:25:42.009 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-05-20T16:25:42.009 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.009 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:42.009 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "10", 2024-05-20T16:25:42.009 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.009 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.009 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:42.009 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.009 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.009 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.009 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.009 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.009 INFO:teuthology.orchestra.run.smithi082.stdout: "ALERTMANAGER_API_HOST": { 2024-05-20T16:25:42.010 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "ALERTMANAGER_API_HOST", 2024-05-20T16:25:42.010 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:42.010 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.010 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:42.010 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:42.010 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.010 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.010 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:42.010 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.010 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.010 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.010 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.011 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.011 INFO:teuthology.orchestra.run.smithi082.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-05-20T16:25:42.011 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-05-20T16:25:42.011 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:42.011 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.011 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:42.011 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-05-20T16:25:42.011 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.011 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.011 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:42.011 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.011 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.011 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.011 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.012 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.012 INFO:teuthology.orchestra.run.smithi082.stdout: "AUDIT_API_ENABLED": { 2024-05-20T16:25:42.012 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "AUDIT_API_ENABLED", 2024-05-20T16:25:42.012 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:42.012 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.012 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:42.012 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:42.012 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.012 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.012 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:42.013 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.013 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.013 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.013 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.013 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.013 INFO:teuthology.orchestra.run.smithi082.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-05-20T16:25:42.013 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-05-20T16:25:42.013 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:42.013 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.013 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:42.013 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-05-20T16:25:42.014 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.014 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.014 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:42.014 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.014 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.014 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.014 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.014 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.014 INFO:teuthology.orchestra.run.smithi082.stdout: "ENABLE_BROWSABLE_API": { 2024-05-20T16:25:42.014 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "ENABLE_BROWSABLE_API", 2024-05-20T16:25:42.014 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:42.015 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.015 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:42.015 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-05-20T16:25:42.015 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.015 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.015 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:42.015 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.015 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.015 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.015 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.016 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.016 INFO:teuthology.orchestra.run.smithi082.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-05-20T16:25:42.016 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-05-20T16:25:42.016 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:42.016 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.016 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:42.016 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-05-20T16:25:42.016 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.016 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.016 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:42.017 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.017 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.017 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.017 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.017 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.017 INFO:teuthology.orchestra.run.smithi082.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-05-20T16:25:42.017 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-05-20T16:25:42.017 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:42.017 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.017 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:42.017 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-05-20T16:25:42.017 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.017 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.018 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:42.018 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.018 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.018 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.018 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.018 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.018 INFO:teuthology.orchestra.run.smithi082.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-05-20T16:25:42.018 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-05-20T16:25:42.018 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:42.018 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.018 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:42.018 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-05-20T16:25:42.018 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.019 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.019 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:42.019 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.019 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.019 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.019 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.019 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.019 INFO:teuthology.orchestra.run.smithi082.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-05-20T16:25:42.019 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-05-20T16:25:42.019 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:42.019 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.019 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:42.019 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-05-20T16:25:42.019 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.020 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.020 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:42.020 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.020 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.020 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.020 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.020 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.020 INFO:teuthology.orchestra.run.smithi082.stdout: "FEATURE_TOGGLE_NFS": { 2024-05-20T16:25:42.020 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-05-20T16:25:42.020 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:42.020 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.020 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:42.020 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-05-20T16:25:42.020 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.021 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.021 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:42.021 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.021 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.021 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.021 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.021 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.021 INFO:teuthology.orchestra.run.smithi082.stdout: "FEATURE_TOGGLE_RBD": { 2024-05-20T16:25:42.021 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-05-20T16:25:42.021 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:42.021 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.021 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:42.021 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-05-20T16:25:42.022 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.022 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.022 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:42.022 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.022 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.022 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.022 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.022 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.022 INFO:teuthology.orchestra.run.smithi082.stdout: "FEATURE_TOGGLE_RGW": { 2024-05-20T16:25:42.022 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-05-20T16:25:42.022 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:42.022 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.022 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:42.023 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-05-20T16:25:42.023 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.023 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.023 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:42.023 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.023 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.023 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.023 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.023 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.023 INFO:teuthology.orchestra.run.smithi082.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-05-20T16:25:42.024 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-05-20T16:25:42.024 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:42.024 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.024 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:42.024 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:42.024 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.024 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.024 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:42.024 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.024 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.024 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.025 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.025 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.025 INFO:teuthology.orchestra.run.smithi082.stdout: "GRAFANA_API_PASSWORD": { 2024-05-20T16:25:42.025 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "GRAFANA_API_PASSWORD", 2024-05-20T16:25:42.025 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:42.025 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.025 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:42.025 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "admin", 2024-05-20T16:25:42.025 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.025 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.025 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:42.026 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.026 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.026 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.026 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.026 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.026 INFO:teuthology.orchestra.run.smithi082.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-05-20T16:25:42.026 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-05-20T16:25:42.026 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:42.026 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.027 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:42.027 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-05-20T16:25:42.027 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.027 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.027 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:42.027 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.027 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.027 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.027 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.027 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.028 INFO:teuthology.orchestra.run.smithi082.stdout: "GRAFANA_API_URL": { 2024-05-20T16:25:42.028 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "GRAFANA_API_URL", 2024-05-20T16:25:42.028 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:42.028 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.028 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:42.028 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:42.028 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.028 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.028 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:42.028 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.028 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.029 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.029 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.029 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.029 INFO:teuthology.orchestra.run.smithi082.stdout: "GRAFANA_API_USERNAME": { 2024-05-20T16:25:42.029 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "GRAFANA_API_USERNAME", 2024-05-20T16:25:42.029 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:42.029 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.029 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:42.029 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "admin", 2024-05-20T16:25:42.029 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.029 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.029 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:42.029 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.030 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.030 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.030 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.030 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.030 INFO:teuthology.orchestra.run.smithi082.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-05-20T16:25:42.030 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-05-20T16:25:42.030 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:42.030 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.030 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:42.030 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:42.030 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.030 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.030 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:42.030 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.031 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.031 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.031 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.031 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.031 INFO:teuthology.orchestra.run.smithi082.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-05-20T16:25:42.031 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-05-20T16:25:42.031 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:42.031 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.031 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:42.031 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:42.031 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.031 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.032 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:42.032 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.032 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.032 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.032 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.032 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.032 INFO:teuthology.orchestra.run.smithi082.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-05-20T16:25:42.032 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-05-20T16:25:42.032 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:42.032 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.032 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:42.032 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-05-20T16:25:42.032 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.032 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.033 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:42.033 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.033 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.033 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.033 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.033 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.033 INFO:teuthology.orchestra.run.smithi082.stdout: "ISSUE_TRACKER_API_KEY": { 2024-05-20T16:25:42.033 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-05-20T16:25:42.033 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:42.033 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.033 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:42.033 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:42.033 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.034 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.034 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:42.034 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.034 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.034 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.034 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.034 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.034 INFO:teuthology.orchestra.run.smithi082.stdout: "PROMETHEUS_API_HOST": { 2024-05-20T16:25:42.034 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "PROMETHEUS_API_HOST", 2024-05-20T16:25:42.034 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:42.034 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.034 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:42.034 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:42.034 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.035 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.035 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:42.035 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.035 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.035 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.035 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.035 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.035 INFO:teuthology.orchestra.run.smithi082.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-05-20T16:25:42.035 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-05-20T16:25:42.035 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:42.035 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.035 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:42.035 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-05-20T16:25:42.036 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.036 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.036 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:42.036 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.036 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.036 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.036 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.036 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.036 INFO:teuthology.orchestra.run.smithi082.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-05-20T16:25:42.036 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-05-20T16:25:42.036 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:42.037 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.037 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:42.037 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:42.037 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.037 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.037 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:42.037 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.037 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.037 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.037 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.037 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.038 INFO:teuthology.orchestra.run.smithi082.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-05-20T16:25:42.038 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-05-20T16:25:42.038 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:42.038 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.038 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:42.038 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:42.038 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.038 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.038 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:42.038 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.038 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.039 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.039 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.039 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.039 INFO:teuthology.orchestra.run.smithi082.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-05-20T16:25:42.039 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-05-20T16:25:42.039 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:42.039 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.039 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:42.039 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-05-20T16:25:42.040 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.040 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.040 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:42.040 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.040 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.040 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.040 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.040 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.040 INFO:teuthology.orchestra.run.smithi082.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-05-20T16:25:42.040 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-05-20T16:25:42.040 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:42.040 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.040 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:42.040 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-05-20T16:25:42.041 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.041 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.041 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:42.041 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.041 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.041 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.041 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.041 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.041 INFO:teuthology.orchestra.run.smithi082.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-05-20T16:25:42.041 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-05-20T16:25:42.041 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:42.041 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.041 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:42.041 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:42.042 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.042 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.042 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:42.042 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.042 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.042 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.042 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.042 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.042 INFO:teuthology.orchestra.run.smithi082.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-05-20T16:25:42.042 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-05-20T16:25:42.042 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:42.042 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.042 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:42.042 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:42.043 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.043 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.043 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:42.043 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.043 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.043 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.043 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.043 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.043 INFO:teuthology.orchestra.run.smithi082.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-05-20T16:25:42.043 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-05-20T16:25:42.043 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:42.043 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.043 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:42.043 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:42.044 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.044 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.044 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:42.044 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.044 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.044 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.044 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.044 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.044 INFO:teuthology.orchestra.run.smithi082.stdout: "PWD_POLICY_ENABLED": { 2024-05-20T16:25:42.044 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "PWD_POLICY_ENABLED", 2024-05-20T16:25:42.044 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:42.044 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.044 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:42.044 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-05-20T16:25:42.045 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.045 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.045 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:42.045 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.045 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.045 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.045 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.045 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.045 INFO:teuthology.orchestra.run.smithi082.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-05-20T16:25:42.045 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-05-20T16:25:42.045 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:42.045 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.045 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:42.045 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-05-20T16:25:42.046 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.046 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.046 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:42.046 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.046 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.046 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.046 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.046 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.046 INFO:teuthology.orchestra.run.smithi082.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-05-20T16:25:42.046 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-05-20T16:25:42.046 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-05-20T16:25:42.046 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.046 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:42.047 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "10", 2024-05-20T16:25:42.047 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.047 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.047 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:42.047 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.047 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.047 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.047 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.047 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.047 INFO:teuthology.orchestra.run.smithi082.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-05-20T16:25:42.047 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-05-20T16:25:42.047 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-05-20T16:25:42.047 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.047 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:42.048 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "8", 2024-05-20T16:25:42.048 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.048 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.048 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:42.048 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.048 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.048 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.048 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.048 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.048 INFO:teuthology.orchestra.run.smithi082.stdout: "REST_REQUESTS_TIMEOUT": { 2024-05-20T16:25:42.048 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-05-20T16:25:42.048 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-05-20T16:25:42.048 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.048 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:42.049 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "45", 2024-05-20T16:25:42.049 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.049 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.049 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:42.049 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.049 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.049 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.049 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.049 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.049 INFO:teuthology.orchestra.run.smithi082.stdout: "RGW_API_ACCESS_KEY": { 2024-05-20T16:25:42.049 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "RGW_API_ACCESS_KEY", 2024-05-20T16:25:42.049 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:42.049 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.049 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:42.050 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:42.050 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.050 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.050 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:42.050 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.050 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.050 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.050 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.050 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.050 INFO:teuthology.orchestra.run.smithi082.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-05-20T16:25:42.050 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-05-20T16:25:42.050 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:42.050 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.051 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:42.051 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "admin", 2024-05-20T16:25:42.051 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.051 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.051 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:42.051 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.051 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.051 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.051 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.051 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.051 INFO:teuthology.orchestra.run.smithi082.stdout: "RGW_API_SECRET_KEY": { 2024-05-20T16:25:42.051 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "RGW_API_SECRET_KEY", 2024-05-20T16:25:42.051 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:42.051 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.052 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:42.052 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:42.052 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.052 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.052 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:42.052 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.052 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.052 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.052 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.052 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.052 INFO:teuthology.orchestra.run.smithi082.stdout: "RGW_API_SSL_VERIFY": { 2024-05-20T16:25:42.052 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "RGW_API_SSL_VERIFY", 2024-05-20T16:25:42.052 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:42.053 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.053 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:42.053 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-05-20T16:25:42.053 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.053 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.053 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:42.053 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.053 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.053 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.053 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.053 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.053 INFO:teuthology.orchestra.run.smithi082.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-05-20T16:25:42.053 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-05-20T16:25:42.053 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-05-20T16:25:42.054 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.054 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:42.054 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "0", 2024-05-20T16:25:42.054 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.054 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.054 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:42.054 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.054 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.054 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.054 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.054 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.054 INFO:teuthology.orchestra.run.smithi082.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-05-20T16:25:42.054 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-05-20T16:25:42.055 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-05-20T16:25:42.055 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.055 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:42.055 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "10", 2024-05-20T16:25:42.055 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.055 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.055 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:42.055 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.055 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.055 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.055 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.055 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.055 INFO:teuthology.orchestra.run.smithi082.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-05-20T16:25:42.055 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-05-20T16:25:42.056 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-05-20T16:25:42.056 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.056 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:42.056 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "5", 2024-05-20T16:25:42.056 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.056 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.056 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:42.056 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.056 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.056 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.056 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.056 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.056 INFO:teuthology.orchestra.run.smithi082.stdout: "cross_origin_url": { 2024-05-20T16:25:42.056 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "cross_origin_url", 2024-05-20T16:25:42.057 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:42.057 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.057 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:42.057 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:42.057 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.057 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.057 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:42.057 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.057 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.057 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.057 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.057 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.057 INFO:teuthology.orchestra.run.smithi082.stdout: "crt_file": { 2024-05-20T16:25:42.058 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "crt_file", 2024-05-20T16:25:42.058 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:42.058 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.058 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:42.058 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:42.058 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.058 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.058 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:42.058 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.058 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.058 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.058 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.058 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.058 INFO:teuthology.orchestra.run.smithi082.stdout: "debug": { 2024-05-20T16:25:42.059 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "debug", 2024-05-20T16:25:42.059 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:42.059 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.059 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:42.059 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:42.059 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.059 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.059 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:42.059 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Enable/disable debug options", 2024-05-20T16:25:42.059 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.059 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.059 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.059 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.059 INFO:teuthology.orchestra.run.smithi082.stdout: "jwt_token_ttl": { 2024-05-20T16:25:42.060 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "jwt_token_ttl", 2024-05-20T16:25:42.060 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-05-20T16:25:42.060 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.060 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:42.060 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "28800", 2024-05-20T16:25:42.060 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.060 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.060 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:42.060 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.060 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.060 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.060 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.060 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.060 INFO:teuthology.orchestra.run.smithi082.stdout: "key_file": { 2024-05-20T16:25:42.061 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "key_file", 2024-05-20T16:25:42.061 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:42.061 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.061 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:42.061 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:42.061 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.061 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.061 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:42.061 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.061 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.061 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.061 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.061 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.061 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-05-20T16:25:42.062 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-05-20T16:25:42.062 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:42.062 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.062 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:42.062 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:42.062 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.062 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.062 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-05-20T16:25:42.062 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-05-20T16:25:42.062 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-05-20T16:25:42.062 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-05-20T16:25:42.062 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-05-20T16:25:42.062 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-05-20T16:25:42.063 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-05-20T16:25:42.063 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:42.063 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.063 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.063 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.063 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.063 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.063 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-05-20T16:25:42.063 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-05-20T16:25:42.063 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:42.063 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.063 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:42.063 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:42.063 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.064 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.064 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:42.064 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.064 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.064 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.064 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.064 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.064 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-05-20T16:25:42.064 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-05-20T16:25:42.064 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:42.064 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.064 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:42.064 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-05-20T16:25:42.065 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.065 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.065 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-05-20T16:25:42.065 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-05-20T16:25:42.065 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-05-20T16:25:42.065 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-05-20T16:25:42.065 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-05-20T16:25:42.065 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-05-20T16:25:42.065 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-05-20T16:25:42.065 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:42.065 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.065 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.065 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.065 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.066 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.066 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-05-20T16:25:42.066 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-05-20T16:25:42.066 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:42.066 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.066 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:42.066 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:42.066 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.066 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.066 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:42.066 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.066 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.066 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.067 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.067 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.067 INFO:teuthology.orchestra.run.smithi082.stdout: "motd": { 2024-05-20T16:25:42.067 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "motd", 2024-05-20T16:25:42.067 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:42.067 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.067 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:42.067 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:42.067 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.067 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.067 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:42.067 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "The message of the day", 2024-05-20T16:25:42.067 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.068 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.068 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.068 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.068 INFO:teuthology.orchestra.run.smithi082.stdout: "redirect_resolve_ip_addr": { 2024-05-20T16:25:42.068 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "redirect_resolve_ip_addr", 2024-05-20T16:25:42.068 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:42.068 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.068 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:42.068 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:42.068 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.068 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.068 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:42.068 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.068 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.069 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.069 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.069 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.069 INFO:teuthology.orchestra.run.smithi082.stdout: "server_addr": { 2024-05-20T16:25:42.069 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "server_addr", 2024-05-20T16:25:42.069 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:42.069 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.069 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:42.069 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "::", 2024-05-20T16:25:42.069 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.069 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.069 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:42.069 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.069 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.070 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.070 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.070 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.070 INFO:teuthology.orchestra.run.smithi082.stdout: "server_port": { 2024-05-20T16:25:42.070 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "server_port", 2024-05-20T16:25:42.070 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-05-20T16:25:42.070 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.070 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:42.070 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "8080", 2024-05-20T16:25:42.070 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.070 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.070 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:42.070 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.070 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.071 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.071 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.071 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.071 INFO:teuthology.orchestra.run.smithi082.stdout: "ssl": { 2024-05-20T16:25:42.071 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "ssl", 2024-05-20T16:25:42.071 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:42.071 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.071 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:42.071 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-05-20T16:25:42.071 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.071 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.071 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:42.071 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.071 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.072 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.072 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.072 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.072 INFO:teuthology.orchestra.run.smithi082.stdout: "ssl_server_port": { 2024-05-20T16:25:42.072 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "ssl_server_port", 2024-05-20T16:25:42.072 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-05-20T16:25:42.072 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.072 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:42.072 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "8443", 2024-05-20T16:25:42.072 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.072 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.072 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:42.072 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.072 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.073 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.073 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.073 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.073 INFO:teuthology.orchestra.run.smithi082.stdout: "standby_behaviour": { 2024-05-20T16:25:42.073 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "standby_behaviour", 2024-05-20T16:25:42.073 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:42.073 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.073 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:42.073 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "redirect", 2024-05-20T16:25:42.073 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.073 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.073 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-05-20T16:25:42.073 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-05-20T16:25:42.073 INFO:teuthology.orchestra.run.smithi082.stdout: "redirect" 2024-05-20T16:25:42.074 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:42.074 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.074 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.074 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.074 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.074 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.074 INFO:teuthology.orchestra.run.smithi082.stdout: "standby_error_status_code": { 2024-05-20T16:25:42.074 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "standby_error_status_code", 2024-05-20T16:25:42.074 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-05-20T16:25:42.074 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.074 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:42.074 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "500", 2024-05-20T16:25:42.075 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "400", 2024-05-20T16:25:42.075 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "599", 2024-05-20T16:25:42.075 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:42.075 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.075 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.075 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.075 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.075 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.075 INFO:teuthology.orchestra.run.smithi082.stdout: "url_prefix": { 2024-05-20T16:25:42.075 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "url_prefix", 2024-05-20T16:25:42.075 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:42.075 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.075 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:42.075 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:42.075 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.076 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.076 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:42.076 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.076 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.076 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.076 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.076 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-20T16:25:42.076 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-20T16:25:42.076 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.076 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-05-20T16:25:42.076 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "devicehealth", 2024-05-20T16:25:42.076 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-05-20T16:25:42.076 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-05-20T16:25:42.076 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-05-20T16:25:42.077 INFO:teuthology.orchestra.run.smithi082.stdout: "enable_monitoring": { 2024-05-20T16:25:42.077 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "enable_monitoring", 2024-05-20T16:25:42.077 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:42.077 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.077 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:42.077 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-05-20T16:25:42.077 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.077 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.077 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:42.077 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "monitor device health metrics", 2024-05-20T16:25:42.077 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.077 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.077 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.077 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.077 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-05-20T16:25:42.078 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-05-20T16:25:42.078 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:42.078 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.078 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:42.078 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:42.078 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.078 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.078 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-05-20T16:25:42.078 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-05-20T16:25:42.078 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-05-20T16:25:42.078 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-05-20T16:25:42.078 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-05-20T16:25:42.078 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-05-20T16:25:42.079 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-05-20T16:25:42.079 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:42.079 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.079 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.079 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.079 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.079 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.079 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-05-20T16:25:42.079 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-05-20T16:25:42.079 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:42.079 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.079 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:42.079 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:42.080 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.080 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.080 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:42.080 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.080 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.080 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.080 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.080 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.080 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-05-20T16:25:42.080 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-05-20T16:25:42.080 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:42.080 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.080 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:42.080 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-05-20T16:25:42.081 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.081 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.081 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-05-20T16:25:42.081 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-05-20T16:25:42.081 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-05-20T16:25:42.081 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-05-20T16:25:42.081 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-05-20T16:25:42.081 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-05-20T16:25:42.081 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-05-20T16:25:42.081 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:42.081 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.081 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.081 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.081 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.082 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.082 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-05-20T16:25:42.082 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-05-20T16:25:42.082 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:42.082 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.082 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:42.082 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:42.082 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.082 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.082 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:42.082 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.082 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.083 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.083 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.083 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.083 INFO:teuthology.orchestra.run.smithi082.stdout: "mark_out_threshold": { 2024-05-20T16:25:42.083 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "mark_out_threshold", 2024-05-20T16:25:42.083 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2024-05-20T16:25:42.083 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.083 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:42.083 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "2419200", 2024-05-20T16:25:42.083 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.083 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.083 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:42.083 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-05-20T16:25:42.083 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.084 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.084 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.084 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.084 INFO:teuthology.orchestra.run.smithi082.stdout: "pool_name": { 2024-05-20T16:25:42.084 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "pool_name", 2024-05-20T16:25:42.084 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:42.084 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.084 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:42.084 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "device_health_metrics", 2024-05-20T16:25:42.084 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.084 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.084 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:42.084 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "name of pool in which to store device health metrics", 2024-05-20T16:25:42.084 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.085 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.085 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.085 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.085 INFO:teuthology.orchestra.run.smithi082.stdout: "retention_period": { 2024-05-20T16:25:42.085 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "retention_period", 2024-05-20T16:25:42.085 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2024-05-20T16:25:42.085 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.085 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:42.085 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "15552000", 2024-05-20T16:25:42.085 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.085 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.085 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:42.085 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "how long to retain device health metrics", 2024-05-20T16:25:42.086 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.086 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.086 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.086 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.086 INFO:teuthology.orchestra.run.smithi082.stdout: "scrape_frequency": { 2024-05-20T16:25:42.086 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "scrape_frequency", 2024-05-20T16:25:42.086 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2024-05-20T16:25:42.086 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.086 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:42.086 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "86400", 2024-05-20T16:25:42.086 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.086 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.086 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:42.086 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "how frequently to scrape device health metrics", 2024-05-20T16:25:42.087 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.087 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.087 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.087 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.087 INFO:teuthology.orchestra.run.smithi082.stdout: "self_heal": { 2024-05-20T16:25:42.087 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "self_heal", 2024-05-20T16:25:42.087 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:42.087 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.087 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:42.087 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-05-20T16:25:42.087 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.087 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.087 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:42.087 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-05-20T16:25:42.088 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.088 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.088 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.088 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.088 INFO:teuthology.orchestra.run.smithi082.stdout: "sleep_interval": { 2024-05-20T16:25:42.088 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "sleep_interval", 2024-05-20T16:25:42.088 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2024-05-20T16:25:42.088 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.088 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:42.088 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "600", 2024-05-20T16:25:42.088 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.088 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.088 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:42.088 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "how frequently to wake up and check device health", 2024-05-20T16:25:42.089 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.089 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.089 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.089 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.089 INFO:teuthology.orchestra.run.smithi082.stdout: "warn_threshold": { 2024-05-20T16:25:42.089 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "warn_threshold", 2024-05-20T16:25:42.089 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2024-05-20T16:25:42.089 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.089 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:42.089 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "7257600", 2024-05-20T16:25:42.089 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.089 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.089 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:42.089 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-05-20T16:25:42.090 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.090 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.090 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.090 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-20T16:25:42.090 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-20T16:25:42.090 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.090 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-05-20T16:25:42.090 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "diskprediction_local", 2024-05-20T16:25:42.090 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-05-20T16:25:42.090 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-05-20T16:25:42.090 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-05-20T16:25:42.090 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-05-20T16:25:42.090 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-05-20T16:25:42.090 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:42.091 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.091 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:42.091 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:42.091 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.091 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.091 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-05-20T16:25:42.091 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-05-20T16:25:42.091 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-05-20T16:25:42.091 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-05-20T16:25:42.091 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-05-20T16:25:42.091 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-05-20T16:25:42.091 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-05-20T16:25:42.091 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:42.091 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.092 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.092 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.092 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.092 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.092 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-05-20T16:25:42.092 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-05-20T16:25:42.092 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:42.092 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.092 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:42.092 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:42.092 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.092 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.092 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:42.092 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.093 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.093 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.093 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.093 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.093 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-05-20T16:25:42.093 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-05-20T16:25:42.093 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:42.093 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.093 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:42.093 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-05-20T16:25:42.093 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.093 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.093 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-05-20T16:25:42.093 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-05-20T16:25:42.093 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-05-20T16:25:42.094 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-05-20T16:25:42.094 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-05-20T16:25:42.094 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-05-20T16:25:42.094 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-05-20T16:25:42.094 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:42.094 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.094 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.094 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.094 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.094 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.094 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-05-20T16:25:42.094 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-05-20T16:25:42.094 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:42.094 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.094 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:42.095 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:42.095 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.095 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.095 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:42.095 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.095 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.095 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.095 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.095 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.095 INFO:teuthology.orchestra.run.smithi082.stdout: "predict_interval": { 2024-05-20T16:25:42.095 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "predict_interval", 2024-05-20T16:25:42.095 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:42.095 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.095 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:42.096 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "86400", 2024-05-20T16:25:42.096 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.096 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.096 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:42.096 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.096 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.096 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.096 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.096 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.097 INFO:teuthology.orchestra.run.smithi082.stdout: "predictor_model": { 2024-05-20T16:25:42.097 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "predictor_model", 2024-05-20T16:25:42.097 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:42.097 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.097 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:42.097 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "prophetstor", 2024-05-20T16:25:42.097 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.097 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.097 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:42.097 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.097 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.097 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.097 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.097 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.097 INFO:teuthology.orchestra.run.smithi082.stdout: "sleep_interval": { 2024-05-20T16:25:42.098 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "sleep_interval", 2024-05-20T16:25:42.098 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:42.098 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.098 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:42.098 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "600", 2024-05-20T16:25:42.098 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.098 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.098 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:42.098 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.098 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.098 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.098 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.098 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-20T16:25:42.099 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-20T16:25:42.099 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.099 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-05-20T16:25:42.099 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "influx", 2024-05-20T16:25:42.099 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": false, 2024-05-20T16:25:42.099 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "influxdb python module not found", 2024-05-20T16:25:42.099 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-05-20T16:25:42.099 INFO:teuthology.orchestra.run.smithi082.stdout: "batch_size": { 2024-05-20T16:25:42.099 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "batch_size", 2024-05-20T16:25:42.099 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-05-20T16:25:42.099 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.099 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:42.099 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "5000", 2024-05-20T16:25:42.099 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.100 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.100 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:42.100 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-05-20T16:25:42.100 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.100 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.100 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.100 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.100 INFO:teuthology.orchestra.run.smithi082.stdout: "database": { 2024-05-20T16:25:42.100 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "database", 2024-05-20T16:25:42.100 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:42.100 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.100 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:42.100 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "ceph", 2024-05-20T16:25:42.100 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.101 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.101 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:42.101 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-05-20T16:25:42.101 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.101 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.101 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.101 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.101 INFO:teuthology.orchestra.run.smithi082.stdout: "hostname": { 2024-05-20T16:25:42.101 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "hostname", 2024-05-20T16:25:42.101 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:42.101 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.101 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:42.101 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:42.101 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.102 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.102 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:42.102 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "InfluxDB server hostname", 2024-05-20T16:25:42.102 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.102 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.102 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.102 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.102 INFO:teuthology.orchestra.run.smithi082.stdout: "interval": { 2024-05-20T16:25:42.102 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "interval", 2024-05-20T16:25:42.102 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2024-05-20T16:25:42.102 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.102 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:42.102 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "30", 2024-05-20T16:25:42.103 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "5", 2024-05-20T16:25:42.103 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.103 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:42.103 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-05-20T16:25:42.103 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.103 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.103 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.103 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.103 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-05-20T16:25:42.103 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-05-20T16:25:42.103 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:42.103 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.103 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:42.103 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:42.103 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.104 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.104 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-05-20T16:25:42.104 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-05-20T16:25:42.104 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-05-20T16:25:42.104 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-05-20T16:25:42.104 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-05-20T16:25:42.104 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-05-20T16:25:42.104 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-05-20T16:25:42.104 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:42.104 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.104 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.104 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.104 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.104 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.105 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-05-20T16:25:42.105 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-05-20T16:25:42.105 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:42.105 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.105 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:42.105 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:42.105 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.105 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.105 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:42.105 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.105 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.105 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.105 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.105 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.105 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-05-20T16:25:42.106 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-05-20T16:25:42.106 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:42.106 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.106 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:42.106 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-05-20T16:25:42.106 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.106 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.106 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-05-20T16:25:42.106 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-05-20T16:25:42.106 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-05-20T16:25:42.106 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-05-20T16:25:42.106 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-05-20T16:25:42.106 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-05-20T16:25:42.106 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-05-20T16:25:42.107 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:42.107 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.107 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.107 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.107 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.107 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.107 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-05-20T16:25:42.107 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-05-20T16:25:42.107 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:42.107 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.107 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:42.107 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:42.107 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.107 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.107 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:42.108 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.108 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.108 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.108 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.108 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.108 INFO:teuthology.orchestra.run.smithi082.stdout: "password": { 2024-05-20T16:25:42.108 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "password", 2024-05-20T16:25:42.108 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:42.108 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.108 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:42.108 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:42.108 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.108 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.108 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:42.108 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "password of InfluxDB server user", 2024-05-20T16:25:42.109 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.109 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.109 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.109 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.109 INFO:teuthology.orchestra.run.smithi082.stdout: "port": { 2024-05-20T16:25:42.109 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "port", 2024-05-20T16:25:42.109 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-05-20T16:25:42.109 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.109 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:42.109 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "8086", 2024-05-20T16:25:42.109 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.109 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.109 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:42.109 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "InfluxDB server port", 2024-05-20T16:25:42.109 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.110 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.110 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.110 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.110 INFO:teuthology.orchestra.run.smithi082.stdout: "ssl": { 2024-05-20T16:25:42.110 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "ssl", 2024-05-20T16:25:42.110 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:42.110 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.110 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:42.110 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "false", 2024-05-20T16:25:42.110 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.110 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.110 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:42.110 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-05-20T16:25:42.111 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.111 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.111 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.111 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.111 INFO:teuthology.orchestra.run.smithi082.stdout: "threads": { 2024-05-20T16:25:42.111 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "threads", 2024-05-20T16:25:42.111 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-05-20T16:25:42.111 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.111 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:42.111 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "5", 2024-05-20T16:25:42.111 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "1", 2024-05-20T16:25:42.111 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "32", 2024-05-20T16:25:42.111 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:42.111 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-05-20T16:25:42.111 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.112 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.112 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.112 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.112 INFO:teuthology.orchestra.run.smithi082.stdout: "username": { 2024-05-20T16:25:42.112 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "username", 2024-05-20T16:25:42.112 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:42.112 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.112 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:42.112 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:42.112 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.112 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.112 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:42.112 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "username of InfluxDB server user", 2024-05-20T16:25:42.112 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.113 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.113 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.113 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.113 INFO:teuthology.orchestra.run.smithi082.stdout: "verify_ssl": { 2024-05-20T16:25:42.113 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "verify_ssl", 2024-05-20T16:25:42.113 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:42.113 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.113 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:42.113 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "true", 2024-05-20T16:25:42.113 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.113 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.113 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:42.113 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-05-20T16:25:42.113 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.114 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.114 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.114 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-20T16:25:42.114 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-20T16:25:42.114 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.114 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-05-20T16:25:42.114 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "insights", 2024-05-20T16:25:42.114 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-05-20T16:25:42.114 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-05-20T16:25:42.114 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-05-20T16:25:42.114 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-05-20T16:25:42.114 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-05-20T16:25:42.114 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:42.114 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.114 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:42.115 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:42.115 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.115 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.115 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-05-20T16:25:42.115 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-05-20T16:25:42.115 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-05-20T16:25:42.115 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-05-20T16:25:42.115 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-05-20T16:25:42.115 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-05-20T16:25:42.115 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-05-20T16:25:42.115 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:42.115 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.115 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.115 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.116 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.116 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.116 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-05-20T16:25:42.116 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-05-20T16:25:42.116 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:42.116 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.116 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:42.116 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:42.116 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.116 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.116 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:42.116 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.116 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.116 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.117 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.117 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.117 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-05-20T16:25:42.117 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-05-20T16:25:42.117 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:42.117 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.117 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:42.117 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-05-20T16:25:42.117 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.117 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.117 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-05-20T16:25:42.117 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-05-20T16:25:42.117 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-05-20T16:25:42.118 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-05-20T16:25:42.118 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-05-20T16:25:42.118 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-05-20T16:25:42.118 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-05-20T16:25:42.118 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:42.118 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.118 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.118 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.118 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.118 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.118 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-05-20T16:25:42.118 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-05-20T16:25:42.118 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:42.119 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.119 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:42.119 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:42.119 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.119 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.119 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:42.119 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.119 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.119 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.119 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.119 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-20T16:25:42.119 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-20T16:25:42.119 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.119 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-05-20T16:25:42.119 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "iostat", 2024-05-20T16:25:42.120 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-05-20T16:25:42.120 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-05-20T16:25:42.120 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-05-20T16:25:42.120 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-05-20T16:25:42.120 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-05-20T16:25:42.120 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:42.120 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.120 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:42.120 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:42.120 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.120 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.120 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-05-20T16:25:42.120 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-05-20T16:25:42.120 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-05-20T16:25:42.120 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-05-20T16:25:42.121 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-05-20T16:25:42.121 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-05-20T16:25:42.121 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-05-20T16:25:42.121 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:42.121 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.121 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.121 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.121 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.121 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.121 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-05-20T16:25:42.121 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-05-20T16:25:42.121 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:42.121 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.121 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:42.121 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:42.122 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.122 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.122 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:42.122 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.122 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.122 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.122 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.122 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.122 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-05-20T16:25:42.122 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-05-20T16:25:42.122 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:42.122 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.122 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:42.123 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-05-20T16:25:42.123 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.123 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.123 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-05-20T16:25:42.123 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-05-20T16:25:42.123 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-05-20T16:25:42.123 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-05-20T16:25:42.123 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-05-20T16:25:42.123 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-05-20T16:25:42.123 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-05-20T16:25:42.123 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:42.123 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.123 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.123 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.123 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.124 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.124 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-05-20T16:25:42.124 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-05-20T16:25:42.124 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:42.124 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.124 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:42.124 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:42.124 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.124 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.124 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:42.124 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.124 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.124 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.124 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.125 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-20T16:25:42.125 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-20T16:25:42.125 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.125 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-05-20T16:25:42.125 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "k8sevents", 2024-05-20T16:25:42.125 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-05-20T16:25:42.125 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-05-20T16:25:42.125 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-05-20T16:25:42.125 INFO:teuthology.orchestra.run.smithi082.stdout: "ceph_event_retention_days": { 2024-05-20T16:25:42.125 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "ceph_event_retention_days", 2024-05-20T16:25:42.125 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-05-20T16:25:42.125 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.125 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:42.125 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "7", 2024-05-20T16:25:42.125 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.126 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.126 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:42.126 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Days to hold ceph event information within local cache", 2024-05-20T16:25:42.126 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.126 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.126 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.126 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.126 INFO:teuthology.orchestra.run.smithi082.stdout: "config_check_secs": { 2024-05-20T16:25:42.126 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "config_check_secs", 2024-05-20T16:25:42.126 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-05-20T16:25:42.126 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.126 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:42.126 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "10", 2024-05-20T16:25:42.126 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "10", 2024-05-20T16:25:42.127 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.127 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:42.127 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-05-20T16:25:42.127 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.127 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.127 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.127 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.127 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-05-20T16:25:42.127 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-05-20T16:25:42.127 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:42.127 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.127 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:42.127 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:42.127 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.128 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.128 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-05-20T16:25:42.128 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-05-20T16:25:42.128 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-05-20T16:25:42.128 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-05-20T16:25:42.128 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-05-20T16:25:42.128 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-05-20T16:25:42.128 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-05-20T16:25:42.128 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:42.128 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.128 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.128 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.128 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.128 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.128 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-05-20T16:25:42.129 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-05-20T16:25:42.129 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:42.129 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.129 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:42.129 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:42.129 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.129 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.129 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:42.129 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.129 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.129 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.129 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.129 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.130 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-05-20T16:25:42.130 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-05-20T16:25:42.130 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:42.130 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.130 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:42.130 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-05-20T16:25:42.130 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.130 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.130 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-05-20T16:25:42.130 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-05-20T16:25:42.130 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-05-20T16:25:42.130 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-05-20T16:25:42.130 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-05-20T16:25:42.130 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-05-20T16:25:42.130 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-05-20T16:25:42.131 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:42.131 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.131 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.131 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.131 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.131 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.131 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-05-20T16:25:42.131 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-05-20T16:25:42.131 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:42.131 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.131 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:42.131 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:42.131 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.131 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.131 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:42.131 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.132 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.132 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.132 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.132 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-20T16:25:42.132 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-20T16:25:42.132 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.132 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-05-20T16:25:42.132 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "localpool", 2024-05-20T16:25:42.132 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-05-20T16:25:42.132 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-05-20T16:25:42.132 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-05-20T16:25:42.132 INFO:teuthology.orchestra.run.smithi082.stdout: "failure_domain": { 2024-05-20T16:25:42.132 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "failure_domain", 2024-05-20T16:25:42.133 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:42.133 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.133 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:42.133 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "host", 2024-05-20T16:25:42.133 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.133 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.133 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:42.133 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "failure domain for any created local pool", 2024-05-20T16:25:42.133 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-05-20T16:25:42.133 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.133 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.133 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.133 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-05-20T16:25:42.133 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-05-20T16:25:42.133 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:42.134 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.134 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:42.134 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:42.134 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.134 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.134 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-05-20T16:25:42.134 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-05-20T16:25:42.134 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-05-20T16:25:42.134 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-05-20T16:25:42.134 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-05-20T16:25:42.134 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-05-20T16:25:42.134 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-05-20T16:25:42.134 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:42.134 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.134 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.135 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.135 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.135 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.135 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-05-20T16:25:42.135 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-05-20T16:25:42.135 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:42.135 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.135 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:42.135 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:42.135 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.135 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.135 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:42.135 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.135 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.136 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.136 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.136 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.136 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-05-20T16:25:42.136 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-05-20T16:25:42.136 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:42.136 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.136 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:42.136 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-05-20T16:25:42.136 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.136 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.136 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-05-20T16:25:42.136 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-05-20T16:25:42.136 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-05-20T16:25:42.136 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-05-20T16:25:42.137 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-05-20T16:25:42.137 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-05-20T16:25:42.137 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-05-20T16:25:42.137 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:42.137 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.137 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.137 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.137 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.137 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.137 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-05-20T16:25:42.137 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-05-20T16:25:42.137 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:42.137 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.137 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:42.137 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:42.137 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.138 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.138 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:42.138 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.138 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.138 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.138 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.138 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.138 INFO:teuthology.orchestra.run.smithi082.stdout: "min_size": { 2024-05-20T16:25:42.138 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "min_size", 2024-05-20T16:25:42.138 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-05-20T16:25:42.138 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.138 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:42.138 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:42.139 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.139 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.139 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:42.139 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "default min_size for any created local pool", 2024-05-20T16:25:42.139 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-05-20T16:25:42.139 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.139 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.139 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.139 INFO:teuthology.orchestra.run.smithi082.stdout: "num_rep": { 2024-05-20T16:25:42.139 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "num_rep", 2024-05-20T16:25:42.139 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-05-20T16:25:42.139 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.139 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:42.140 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "3", 2024-05-20T16:25:42.140 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.140 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.140 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:42.140 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "default replica count for any created local pool", 2024-05-20T16:25:42.140 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.140 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.140 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.140 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.140 INFO:teuthology.orchestra.run.smithi082.stdout: "pg_num": { 2024-05-20T16:25:42.140 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "pg_num", 2024-05-20T16:25:42.140 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-05-20T16:25:42.140 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.140 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:42.141 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "128", 2024-05-20T16:25:42.141 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.141 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.141 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:42.141 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "default pg_num for any created local pool", 2024-05-20T16:25:42.141 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.141 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.141 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.141 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.141 INFO:teuthology.orchestra.run.smithi082.stdout: "prefix": { 2024-05-20T16:25:42.141 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "prefix", 2024-05-20T16:25:42.141 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:42.141 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.141 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:42.141 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:42.142 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.142 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.142 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:42.142 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "name prefix for any created local pool", 2024-05-20T16:25:42.142 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.142 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.142 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.142 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.142 INFO:teuthology.orchestra.run.smithi082.stdout: "subtree": { 2024-05-20T16:25:42.142 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "subtree", 2024-05-20T16:25:42.142 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:42.142 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.142 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:42.142 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "rack", 2024-05-20T16:25:42.143 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.143 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.143 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:42.143 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "CRUSH level for which to create a local pool", 2024-05-20T16:25:42.143 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-05-20T16:25:42.143 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.143 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.143 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-20T16:25:42.143 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-20T16:25:42.143 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.143 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-05-20T16:25:42.143 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "mds_autoscaler", 2024-05-20T16:25:42.143 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-05-20T16:25:42.143 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-05-20T16:25:42.143 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-05-20T16:25:42.144 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-05-20T16:25:42.144 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-05-20T16:25:42.144 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:42.144 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.144 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:42.144 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:42.144 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.144 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.144 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-05-20T16:25:42.144 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-05-20T16:25:42.144 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-05-20T16:25:42.144 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-05-20T16:25:42.144 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-05-20T16:25:42.144 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-05-20T16:25:42.145 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-05-20T16:25:42.145 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:42.145 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.145 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.145 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.145 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.145 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.145 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-05-20T16:25:42.145 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-05-20T16:25:42.145 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:42.145 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.145 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:42.145 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:42.145 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.145 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.146 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:42.146 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.146 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.146 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.146 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.146 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.146 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-05-20T16:25:42.146 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-05-20T16:25:42.146 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:42.146 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.146 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:42.146 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-05-20T16:25:42.146 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.147 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.147 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-05-20T16:25:42.147 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-05-20T16:25:42.147 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-05-20T16:25:42.147 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-05-20T16:25:42.147 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-05-20T16:25:42.147 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-05-20T16:25:42.147 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-05-20T16:25:42.147 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:42.147 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.147 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.147 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.147 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.147 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.147 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-05-20T16:25:42.148 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-05-20T16:25:42.148 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:42.148 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.148 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:42.148 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:42.148 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.148 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.148 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:42.148 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.148 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.148 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.148 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.148 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-20T16:25:42.148 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-20T16:25:42.148 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.149 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-05-20T16:25:42.149 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "mirroring", 2024-05-20T16:25:42.149 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-05-20T16:25:42.149 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-05-20T16:25:42.149 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-05-20T16:25:42.149 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-05-20T16:25:42.149 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-05-20T16:25:42.149 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:42.149 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.149 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:42.149 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:42.149 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.149 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.149 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-05-20T16:25:42.149 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-05-20T16:25:42.150 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-05-20T16:25:42.150 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-05-20T16:25:42.150 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-05-20T16:25:42.150 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-05-20T16:25:42.150 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-05-20T16:25:42.150 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:42.150 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.150 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.150 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.150 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.150 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.150 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-05-20T16:25:42.150 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-05-20T16:25:42.150 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:42.151 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.151 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:42.151 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:42.151 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.151 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.151 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:42.151 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.151 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.151 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.151 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.151 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.151 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-05-20T16:25:42.151 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-05-20T16:25:42.151 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:42.151 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.152 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:42.152 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-05-20T16:25:42.152 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.152 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.152 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-05-20T16:25:42.152 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-05-20T16:25:42.152 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-05-20T16:25:42.152 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-05-20T16:25:42.152 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-05-20T16:25:42.152 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-05-20T16:25:42.152 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-05-20T16:25:42.152 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:42.152 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.153 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.153 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.153 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.153 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.153 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-05-20T16:25:42.153 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-05-20T16:25:42.153 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:42.153 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.153 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:42.153 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:42.153 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.153 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.153 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:42.153 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.153 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.154 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.154 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.154 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-20T16:25:42.154 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-20T16:25:42.154 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.154 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-05-20T16:25:42.154 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "nfs", 2024-05-20T16:25:42.154 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-05-20T16:25:42.154 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-05-20T16:25:42.154 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-05-20T16:25:42.154 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-05-20T16:25:42.154 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-05-20T16:25:42.154 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:42.154 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.154 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:42.155 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:42.155 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.155 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.155 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-05-20T16:25:42.155 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-05-20T16:25:42.155 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-05-20T16:25:42.155 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-05-20T16:25:42.155 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-05-20T16:25:42.155 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-05-20T16:25:42.155 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-05-20T16:25:42.155 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:42.155 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.155 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.155 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.156 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.156 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.156 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-05-20T16:25:42.156 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-05-20T16:25:42.156 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:42.156 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.156 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:42.156 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:42.156 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.156 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.156 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:42.156 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.156 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.156 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.157 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.157 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.157 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-05-20T16:25:42.157 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-05-20T16:25:42.157 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:42.157 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.157 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:42.157 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-05-20T16:25:42.157 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.157 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.157 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-05-20T16:25:42.157 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-05-20T16:25:42.157 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-05-20T16:25:42.157 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-05-20T16:25:42.158 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-05-20T16:25:42.158 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-05-20T16:25:42.158 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-05-20T16:25:42.158 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:42.158 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.158 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.158 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.158 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.158 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.158 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-05-20T16:25:42.158 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-05-20T16:25:42.158 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:42.158 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.159 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:42.159 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:42.159 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.159 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.159 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:42.159 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.159 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.159 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.159 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.159 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-20T16:25:42.159 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-20T16:25:42.159 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.159 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-05-20T16:25:42.159 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "orchestrator", 2024-05-20T16:25:42.160 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-05-20T16:25:42.160 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-05-20T16:25:42.160 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-05-20T16:25:42.160 INFO:teuthology.orchestra.run.smithi082.stdout: "fail_fs": { 2024-05-20T16:25:42.160 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "fail_fs", 2024-05-20T16:25:42.160 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:42.160 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.160 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:42.160 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:42.160 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.160 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.160 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:42.160 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-05-20T16:25:42.160 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.160 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.161 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.161 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.161 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-05-20T16:25:42.161 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-05-20T16:25:42.161 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:42.161 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.161 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:42.161 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:42.161 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.161 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.161 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-05-20T16:25:42.161 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-05-20T16:25:42.161 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-05-20T16:25:42.161 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-05-20T16:25:42.162 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-05-20T16:25:42.162 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-05-20T16:25:42.162 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-05-20T16:25:42.162 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:42.162 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.162 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.162 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.162 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.162 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.162 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-05-20T16:25:42.162 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-05-20T16:25:42.162 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:42.162 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.163 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:42.163 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:42.163 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.163 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.163 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:42.163 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.163 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.163 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.163 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.163 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.163 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-05-20T16:25:42.163 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-05-20T16:25:42.163 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:42.164 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.164 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:42.164 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-05-20T16:25:42.164 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.164 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.164 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-05-20T16:25:42.164 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-05-20T16:25:42.164 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-05-20T16:25:42.164 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-05-20T16:25:42.164 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-05-20T16:25:42.164 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-05-20T16:25:42.164 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-05-20T16:25:42.164 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:42.164 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.165 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.165 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.165 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.165 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.165 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-05-20T16:25:42.165 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-05-20T16:25:42.165 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:42.165 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.165 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:42.165 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:42.165 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.165 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.165 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:42.166 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.166 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.166 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.166 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.166 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.166 INFO:teuthology.orchestra.run.smithi082.stdout: "orchestrator": { 2024-05-20T16:25:42.166 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "orchestrator", 2024-05-20T16:25:42.166 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:42.166 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.166 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:42.166 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:42.166 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.166 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.166 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-05-20T16:25:42.167 INFO:teuthology.orchestra.run.smithi082.stdout: "cephadm", 2024-05-20T16:25:42.167 INFO:teuthology.orchestra.run.smithi082.stdout: "rook", 2024-05-20T16:25:42.167 INFO:teuthology.orchestra.run.smithi082.stdout: "test_orchestrator" 2024-05-20T16:25:42.167 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:42.167 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Orchestrator backend", 2024-05-20T16:25:42.167 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.167 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.167 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.167 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-20T16:25:42.167 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-20T16:25:42.167 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.167 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-05-20T16:25:42.167 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "osd_perf_query", 2024-05-20T16:25:42.167 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-05-20T16:25:42.168 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-05-20T16:25:42.168 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-05-20T16:25:42.168 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-05-20T16:25:42.168 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-05-20T16:25:42.168 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:42.168 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.168 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:42.168 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:42.168 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.168 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.168 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-05-20T16:25:42.168 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-05-20T16:25:42.168 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-05-20T16:25:42.168 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-05-20T16:25:42.169 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-05-20T16:25:42.169 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-05-20T16:25:42.169 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-05-20T16:25:42.169 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:42.169 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.169 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.169 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.169 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.169 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.169 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-05-20T16:25:42.169 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-05-20T16:25:42.169 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:42.169 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.169 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:42.169 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:42.170 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.170 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.170 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:42.170 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.170 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.170 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.170 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.170 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.170 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-05-20T16:25:42.170 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-05-20T16:25:42.170 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:42.170 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.170 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:42.171 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-05-20T16:25:42.171 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.171 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.171 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-05-20T16:25:42.171 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-05-20T16:25:42.171 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-05-20T16:25:42.171 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-05-20T16:25:42.171 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-05-20T16:25:42.171 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-05-20T16:25:42.171 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-05-20T16:25:42.171 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:42.171 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.171 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.172 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.172 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.172 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.172 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-05-20T16:25:42.172 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-05-20T16:25:42.172 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:42.172 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.172 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:42.172 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:42.172 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.172 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.172 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:42.172 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.173 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.173 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.173 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.173 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-20T16:25:42.173 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-20T16:25:42.173 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.173 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-05-20T16:25:42.173 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "osd_support", 2024-05-20T16:25:42.173 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-05-20T16:25:42.173 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-05-20T16:25:42.173 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-05-20T16:25:42.173 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-05-20T16:25:42.173 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-05-20T16:25:42.173 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:42.174 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.174 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:42.174 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:42.174 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.174 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.174 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-05-20T16:25:42.174 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-05-20T16:25:42.174 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-05-20T16:25:42.174 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-05-20T16:25:42.174 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-05-20T16:25:42.174 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-05-20T16:25:42.174 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-05-20T16:25:42.174 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:42.174 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.175 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.175 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.175 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.175 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.175 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-05-20T16:25:42.175 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-05-20T16:25:42.175 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:42.175 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.175 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:42.175 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:42.175 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.175 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.175 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:42.175 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.175 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.176 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.176 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.176 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.176 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-05-20T16:25:42.176 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-05-20T16:25:42.176 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:42.176 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.176 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:42.176 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-05-20T16:25:42.176 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.176 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.176 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-05-20T16:25:42.176 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-05-20T16:25:42.176 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-05-20T16:25:42.177 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-05-20T16:25:42.177 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-05-20T16:25:42.177 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-05-20T16:25:42.177 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-05-20T16:25:42.177 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:42.177 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.177 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.177 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.177 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.177 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.177 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-05-20T16:25:42.177 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-05-20T16:25:42.177 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:42.177 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.177 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:42.178 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:42.178 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.178 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.178 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:42.178 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.178 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.178 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.178 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.178 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-20T16:25:42.178 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-20T16:25:42.178 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.178 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-05-20T16:25:42.178 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "pg_autoscaler", 2024-05-20T16:25:42.178 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-05-20T16:25:42.179 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-05-20T16:25:42.179 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-05-20T16:25:42.179 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-05-20T16:25:42.179 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-05-20T16:25:42.179 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:42.179 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.179 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:42.179 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:42.179 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.179 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.179 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-05-20T16:25:42.179 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-05-20T16:25:42.179 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-05-20T16:25:42.179 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-05-20T16:25:42.180 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-05-20T16:25:42.180 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-05-20T16:25:42.180 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-05-20T16:25:42.180 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:42.180 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.180 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.180 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.180 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.180 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.180 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-05-20T16:25:42.180 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-05-20T16:25:42.180 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:42.180 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.180 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:42.181 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:42.181 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.181 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.181 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:42.181 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.181 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.181 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.181 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.181 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.181 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-05-20T16:25:42.181 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-05-20T16:25:42.181 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:42.181 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.181 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:42.181 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-05-20T16:25:42.182 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.182 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.182 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-05-20T16:25:42.182 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-05-20T16:25:42.182 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-05-20T16:25:42.182 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-05-20T16:25:42.182 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-05-20T16:25:42.182 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-05-20T16:25:42.182 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-05-20T16:25:42.182 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:42.182 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.182 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.182 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.182 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.183 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.183 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-05-20T16:25:42.183 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-05-20T16:25:42.183 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:42.183 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.183 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:42.183 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:42.183 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.183 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.183 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:42.183 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.183 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.183 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.183 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.184 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.184 INFO:teuthology.orchestra.run.smithi082.stdout: "sleep_interval": { 2024-05-20T16:25:42.184 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "sleep_interval", 2024-05-20T16:25:42.184 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2024-05-20T16:25:42.184 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.184 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:42.184 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "60", 2024-05-20T16:25:42.184 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.184 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.184 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:42.184 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.184 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.184 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.184 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.185 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.185 INFO:teuthology.orchestra.run.smithi082.stdout: "threshold": { 2024-05-20T16:25:42.185 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "threshold", 2024-05-20T16:25:42.185 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "float", 2024-05-20T16:25:42.185 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.185 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:42.185 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "3.0", 2024-05-20T16:25:42.185 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "1.0", 2024-05-20T16:25:42.185 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.185 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:42.185 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "scaling threshold", 2024-05-20T16:25:42.185 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-05-20T16:25:42.185 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.185 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.186 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-20T16:25:42.186 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-20T16:25:42.186 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.186 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-05-20T16:25:42.186 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "progress", 2024-05-20T16:25:42.186 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-05-20T16:25:42.186 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-05-20T16:25:42.186 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-05-20T16:25:42.186 INFO:teuthology.orchestra.run.smithi082.stdout: "allow_pg_recovery_event": { 2024-05-20T16:25:42.186 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "allow_pg_recovery_event", 2024-05-20T16:25:42.186 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:42.186 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.186 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:42.186 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:42.187 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.187 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.187 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:42.187 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "allow the module to show pg recovery progress", 2024-05-20T16:25:42.187 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.187 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.187 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.187 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.187 INFO:teuthology.orchestra.run.smithi082.stdout: "enabled": { 2024-05-20T16:25:42.187 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "enabled", 2024-05-20T16:25:42.187 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:42.187 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.187 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:42.187 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-05-20T16:25:42.187 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.188 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.188 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:42.188 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.188 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.188 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.188 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.188 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.188 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-05-20T16:25:42.188 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-05-20T16:25:42.188 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:42.188 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.188 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:42.188 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:42.188 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.189 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.189 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-05-20T16:25:42.189 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-05-20T16:25:42.189 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-05-20T16:25:42.189 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-05-20T16:25:42.189 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-05-20T16:25:42.189 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-05-20T16:25:42.189 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-05-20T16:25:42.189 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:42.189 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.189 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.189 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.189 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.189 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.190 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-05-20T16:25:42.190 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-05-20T16:25:42.190 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:42.190 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.190 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:42.190 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:42.190 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.190 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.190 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:42.190 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.190 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.190 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.190 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.190 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.191 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-05-20T16:25:42.191 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-05-20T16:25:42.191 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:42.191 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.191 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:42.191 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-05-20T16:25:42.191 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.191 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.191 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-05-20T16:25:42.191 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-05-20T16:25:42.191 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-05-20T16:25:42.191 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-05-20T16:25:42.191 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-05-20T16:25:42.191 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-05-20T16:25:42.191 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-05-20T16:25:42.192 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:42.192 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.192 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.192 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.192 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.192 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.192 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-05-20T16:25:42.192 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-05-20T16:25:42.192 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:42.192 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.192 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:42.192 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:42.192 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.192 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.193 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:42.193 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.193 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.193 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.193 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.193 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.193 INFO:teuthology.orchestra.run.smithi082.stdout: "max_completed_events": { 2024-05-20T16:25:42.193 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "max_completed_events", 2024-05-20T16:25:42.193 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-05-20T16:25:42.193 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.193 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:42.193 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "50", 2024-05-20T16:25:42.193 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.193 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.193 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:42.194 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "number of past completed events to remember", 2024-05-20T16:25:42.194 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.194 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.194 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.194 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.194 INFO:teuthology.orchestra.run.smithi082.stdout: "sleep_interval": { 2024-05-20T16:25:42.194 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "sleep_interval", 2024-05-20T16:25:42.194 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2024-05-20T16:25:42.194 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.194 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:42.194 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "5", 2024-05-20T16:25:42.194 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.194 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.195 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:42.195 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "how long the module is going to sleep", 2024-05-20T16:25:42.195 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.195 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.195 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.195 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-20T16:25:42.195 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-20T16:25:42.195 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.195 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-05-20T16:25:42.195 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "prometheus", 2024-05-20T16:25:42.195 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-05-20T16:25:42.195 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-05-20T16:25:42.195 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-05-20T16:25:42.195 INFO:teuthology.orchestra.run.smithi082.stdout: "cache": { 2024-05-20T16:25:42.195 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "cache", 2024-05-20T16:25:42.196 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:42.196 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.196 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:42.196 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-05-20T16:25:42.196 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.196 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.196 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:42.196 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.196 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.196 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.196 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.196 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.196 INFO:teuthology.orchestra.run.smithi082.stdout: "exclude_perf_counters": { 2024-05-20T16:25:42.196 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "exclude_perf_counters", 2024-05-20T16:25:42.197 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:42.197 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.197 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:42.197 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-05-20T16:25:42.197 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.197 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.197 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:42.197 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-05-20T16:25:42.197 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-05-20T16:25:42.197 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.197 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.197 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.197 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-05-20T16:25:42.197 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-05-20T16:25:42.198 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:42.198 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.198 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:42.198 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:42.198 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.198 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.198 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-05-20T16:25:42.198 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-05-20T16:25:42.198 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-05-20T16:25:42.198 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-05-20T16:25:42.198 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-05-20T16:25:42.198 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-05-20T16:25:42.198 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-05-20T16:25:42.198 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:42.199 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.199 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.199 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.199 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.199 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.199 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-05-20T16:25:42.199 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-05-20T16:25:42.199 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:42.199 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.199 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:42.199 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:42.199 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.199 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.199 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:42.200 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.200 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.200 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.200 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.200 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.200 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-05-20T16:25:42.200 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-05-20T16:25:42.200 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:42.200 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.200 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:42.200 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-05-20T16:25:42.200 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.200 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.200 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-05-20T16:25:42.201 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-05-20T16:25:42.201 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-05-20T16:25:42.201 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-05-20T16:25:42.201 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-05-20T16:25:42.201 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-05-20T16:25:42.201 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-05-20T16:25:42.201 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:42.201 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.201 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.201 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.201 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.201 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.201 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-05-20T16:25:42.201 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-05-20T16:25:42.201 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:42.202 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.202 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:42.202 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:42.202 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.202 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.202 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:42.202 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.202 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.202 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.202 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.202 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.202 INFO:teuthology.orchestra.run.smithi082.stdout: "rbd_stats_pools": { 2024-05-20T16:25:42.202 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "rbd_stats_pools", 2024-05-20T16:25:42.202 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:42.203 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.203 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:42.203 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:42.203 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.203 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.203 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:42.203 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.203 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.203 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.203 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.203 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.203 INFO:teuthology.orchestra.run.smithi082.stdout: "rbd_stats_pools_refresh_interval": { 2024-05-20T16:25:42.203 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-05-20T16:25:42.203 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-05-20T16:25:42.204 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.204 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:42.204 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "300", 2024-05-20T16:25:42.204 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.204 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.204 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:42.204 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.204 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.204 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.204 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.204 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.204 INFO:teuthology.orchestra.run.smithi082.stdout: "scrape_interval": { 2024-05-20T16:25:42.204 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "scrape_interval", 2024-05-20T16:25:42.204 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "float", 2024-05-20T16:25:42.205 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.205 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:42.205 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "15.0", 2024-05-20T16:25:42.205 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.205 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.205 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:42.205 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.205 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.205 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.205 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.205 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.205 INFO:teuthology.orchestra.run.smithi082.stdout: "server_addr": { 2024-05-20T16:25:42.205 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "server_addr", 2024-05-20T16:25:42.205 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:42.205 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.206 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:42.206 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "::", 2024-05-20T16:25:42.206 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.206 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.206 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:42.206 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-05-20T16:25:42.206 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.206 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.206 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.206 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.206 INFO:teuthology.orchestra.run.smithi082.stdout: "server_port": { 2024-05-20T16:25:42.206 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "server_port", 2024-05-20T16:25:42.206 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-05-20T16:25:42.206 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.207 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:42.207 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "9283", 2024-05-20T16:25:42.207 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.207 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.207 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:42.207 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-05-20T16:25:42.207 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.207 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.207 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.207 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.207 INFO:teuthology.orchestra.run.smithi082.stdout: "stale_cache_strategy": { 2024-05-20T16:25:42.207 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "stale_cache_strategy", 2024-05-20T16:25:42.207 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:42.207 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.208 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:42.208 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "log", 2024-05-20T16:25:42.208 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.208 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.208 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:42.208 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.208 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.208 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.208 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.208 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.208 INFO:teuthology.orchestra.run.smithi082.stdout: "standby_behaviour": { 2024-05-20T16:25:42.208 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "standby_behaviour", 2024-05-20T16:25:42.208 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:42.208 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.209 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:42.209 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "default", 2024-05-20T16:25:42.209 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.209 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.209 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-05-20T16:25:42.209 INFO:teuthology.orchestra.run.smithi082.stdout: "default", 2024-05-20T16:25:42.209 INFO:teuthology.orchestra.run.smithi082.stdout: "error" 2024-05-20T16:25:42.209 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:42.209 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.209 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.209 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.209 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.209 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.209 INFO:teuthology.orchestra.run.smithi082.stdout: "standby_error_status_code": { 2024-05-20T16:25:42.209 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "standby_error_status_code", 2024-05-20T16:25:42.210 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-05-20T16:25:42.210 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.210 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:42.210 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "500", 2024-05-20T16:25:42.210 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "400", 2024-05-20T16:25:42.210 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "599", 2024-05-20T16:25:42.210 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:42.210 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.210 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.210 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.210 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.210 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-20T16:25:42.210 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-20T16:25:42.211 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.211 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-05-20T16:25:42.211 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "rbd_support", 2024-05-20T16:25:42.211 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-05-20T16:25:42.211 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-05-20T16:25:42.211 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-05-20T16:25:42.211 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-05-20T16:25:42.211 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-05-20T16:25:42.211 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:42.211 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.211 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:42.211 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:42.211 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.211 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.211 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-05-20T16:25:42.212 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-05-20T16:25:42.212 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-05-20T16:25:42.212 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-05-20T16:25:42.212 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-05-20T16:25:42.212 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-05-20T16:25:42.212 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-05-20T16:25:42.212 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:42.212 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.212 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.212 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.212 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.212 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.212 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-05-20T16:25:42.213 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-05-20T16:25:42.213 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:42.213 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.213 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:42.213 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:42.213 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.213 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.213 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:42.213 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.213 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.213 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.213 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.213 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.213 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-05-20T16:25:42.213 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-05-20T16:25:42.214 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:42.214 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.214 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:42.214 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-05-20T16:25:42.214 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.214 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.214 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-05-20T16:25:42.214 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-05-20T16:25:42.214 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-05-20T16:25:42.214 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-05-20T16:25:42.214 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-05-20T16:25:42.214 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-05-20T16:25:42.214 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-05-20T16:25:42.215 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:42.215 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.215 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.215 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.215 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.215 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.215 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-05-20T16:25:42.215 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-05-20T16:25:42.215 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:42.215 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.215 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:42.215 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:42.215 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.215 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.216 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:42.216 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.216 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.216 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.216 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.216 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.216 INFO:teuthology.orchestra.run.smithi082.stdout: "max_concurrent_snap_create": { 2024-05-20T16:25:42.216 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "max_concurrent_snap_create", 2024-05-20T16:25:42.216 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-05-20T16:25:42.216 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.216 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:42.216 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "10", 2024-05-20T16:25:42.216 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.216 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.217 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:42.217 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.217 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.217 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.217 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.217 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.217 INFO:teuthology.orchestra.run.smithi082.stdout: "mirror_snapshot_schedule": { 2024-05-20T16:25:42.217 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "mirror_snapshot_schedule", 2024-05-20T16:25:42.217 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:42.217 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.217 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:42.217 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:42.217 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.217 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.218 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:42.218 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.218 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.218 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.218 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.218 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.218 INFO:teuthology.orchestra.run.smithi082.stdout: "trash_purge_schedule": { 2024-05-20T16:25:42.218 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "trash_purge_schedule", 2024-05-20T16:25:42.218 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:42.218 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.218 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:42.218 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:42.218 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.218 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.219 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:42.219 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.219 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.219 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.219 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.219 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-20T16:25:42.219 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-20T16:25:42.219 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.219 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-05-20T16:25:42.219 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "restful", 2024-05-20T16:25:42.219 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-05-20T16:25:42.219 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-05-20T16:25:42.219 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-05-20T16:25:42.219 INFO:teuthology.orchestra.run.smithi082.stdout: "enable_auth": { 2024-05-20T16:25:42.220 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "enable_auth", 2024-05-20T16:25:42.220 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:42.220 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.220 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:42.220 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-05-20T16:25:42.220 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.220 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.220 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:42.220 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.220 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.220 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.220 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.220 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.220 INFO:teuthology.orchestra.run.smithi082.stdout: "key_file": { 2024-05-20T16:25:42.221 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "key_file", 2024-05-20T16:25:42.221 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:42.221 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.221 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:42.221 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:42.221 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.221 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.221 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:42.221 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.221 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.221 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.221 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.221 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.221 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-05-20T16:25:42.221 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-05-20T16:25:42.222 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:42.222 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.222 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:42.222 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:42.222 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.222 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.222 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-05-20T16:25:42.222 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-05-20T16:25:42.222 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-05-20T16:25:42.222 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-05-20T16:25:42.222 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-05-20T16:25:42.222 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-05-20T16:25:42.222 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-05-20T16:25:42.222 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:42.223 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.223 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.223 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.223 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.223 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.223 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-05-20T16:25:42.223 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-05-20T16:25:42.223 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:42.223 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.223 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:42.223 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:42.223 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.223 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.223 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:42.223 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.224 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.224 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.224 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.224 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.224 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-05-20T16:25:42.224 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-05-20T16:25:42.224 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:42.224 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.224 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:42.224 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-05-20T16:25:42.224 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.224 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.224 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-05-20T16:25:42.224 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-05-20T16:25:42.225 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-05-20T16:25:42.225 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-05-20T16:25:42.225 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-05-20T16:25:42.225 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-05-20T16:25:42.225 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-05-20T16:25:42.225 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:42.225 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.225 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.225 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.225 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.225 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.225 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-05-20T16:25:42.225 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-05-20T16:25:42.225 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:42.225 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.226 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:42.226 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:42.226 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.226 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.226 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:42.226 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.226 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.226 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.226 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.226 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.226 INFO:teuthology.orchestra.run.smithi082.stdout: "server_addr": { 2024-05-20T16:25:42.226 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "server_addr", 2024-05-20T16:25:42.226 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:42.227 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.227 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:42.227 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:42.227 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.227 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.227 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:42.227 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.227 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.227 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.227 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.227 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.227 INFO:teuthology.orchestra.run.smithi082.stdout: "server_port": { 2024-05-20T16:25:42.227 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "server_port", 2024-05-20T16:25:42.227 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:42.227 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.228 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:42.228 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:42.228 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.228 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.228 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:42.228 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.228 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.228 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.228 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.228 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-20T16:25:42.228 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-20T16:25:42.228 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.228 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-05-20T16:25:42.228 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "rgw", 2024-05-20T16:25:42.228 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-05-20T16:25:42.229 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-05-20T16:25:42.229 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-05-20T16:25:42.229 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-05-20T16:25:42.229 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-05-20T16:25:42.229 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:42.229 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.229 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:42.229 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:42.229 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.229 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.229 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-05-20T16:25:42.229 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-05-20T16:25:42.229 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-05-20T16:25:42.229 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-05-20T16:25:42.230 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-05-20T16:25:42.230 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-05-20T16:25:42.230 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-05-20T16:25:42.230 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:42.230 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.230 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.230 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.230 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.230 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.230 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-05-20T16:25:42.230 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-05-20T16:25:42.230 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:42.230 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.230 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:42.231 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:42.231 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.231 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.231 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:42.231 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.231 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.231 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.231 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.231 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.231 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-05-20T16:25:42.231 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-05-20T16:25:42.231 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:42.231 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.231 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:42.232 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-05-20T16:25:42.232 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.232 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.232 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-05-20T16:25:42.232 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-05-20T16:25:42.232 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-05-20T16:25:42.232 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-05-20T16:25:42.232 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-05-20T16:25:42.232 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-05-20T16:25:42.232 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-05-20T16:25:42.232 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:42.232 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.232 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.233 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.233 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.233 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.233 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-05-20T16:25:42.233 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-05-20T16:25:42.233 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:42.233 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.233 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:42.233 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:42.233 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.233 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.233 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:42.233 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.233 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.234 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.234 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.234 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-20T16:25:42.234 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-20T16:25:42.234 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.234 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-05-20T16:25:42.234 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "rook", 2024-05-20T16:25:42.234 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-05-20T16:25:42.234 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-05-20T16:25:42.234 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-05-20T16:25:42.234 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-05-20T16:25:42.234 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-05-20T16:25:42.234 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:42.234 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.235 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:42.235 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:42.235 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.235 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.235 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-05-20T16:25:42.235 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-05-20T16:25:42.235 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-05-20T16:25:42.235 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-05-20T16:25:42.235 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-05-20T16:25:42.235 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-05-20T16:25:42.235 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-05-20T16:25:42.235 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:42.235 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.235 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.235 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.236 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.236 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.236 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-05-20T16:25:42.236 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-05-20T16:25:42.236 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:42.236 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.236 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:42.236 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:42.236 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.236 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.236 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:42.236 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.236 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.236 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.237 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.237 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.237 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-05-20T16:25:42.237 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-05-20T16:25:42.237 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:42.237 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.237 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:42.237 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-05-20T16:25:42.237 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.237 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.237 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-05-20T16:25:42.237 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-05-20T16:25:42.237 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-05-20T16:25:42.237 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-05-20T16:25:42.238 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-05-20T16:25:42.238 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-05-20T16:25:42.238 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-05-20T16:25:42.238 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:42.238 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.238 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.238 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.238 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.238 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.238 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-05-20T16:25:42.238 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-05-20T16:25:42.238 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:42.238 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.238 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:42.239 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:42.239 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.239 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.239 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:42.239 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.239 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.239 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.239 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.239 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.239 INFO:teuthology.orchestra.run.smithi082.stdout: "storage_class": { 2024-05-20T16:25:42.239 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "storage_class", 2024-05-20T16:25:42.239 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:42.239 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.239 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:42.240 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "local", 2024-05-20T16:25:42.240 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.240 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.240 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:42.240 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-05-20T16:25:42.240 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.240 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.240 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.240 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-20T16:25:42.240 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-20T16:25:42.240 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.240 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-05-20T16:25:42.240 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "selftest", 2024-05-20T16:25:42.240 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-05-20T16:25:42.240 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-05-20T16:25:42.241 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-05-20T16:25:42.241 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-05-20T16:25:42.241 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-05-20T16:25:42.241 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:42.241 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.241 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:42.241 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:42.241 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.241 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.241 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-05-20T16:25:42.241 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-05-20T16:25:42.241 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-05-20T16:25:42.241 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-05-20T16:25:42.241 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-05-20T16:25:42.242 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-05-20T16:25:42.242 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-05-20T16:25:42.242 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:42.242 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.242 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.242 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.242 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.242 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.242 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-05-20T16:25:42.242 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-05-20T16:25:42.242 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:42.242 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.242 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:42.243 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:42.243 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.243 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.243 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:42.243 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.243 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.243 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.243 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.243 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.243 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-05-20T16:25:42.243 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-05-20T16:25:42.243 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:42.243 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.244 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:42.244 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-05-20T16:25:42.244 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.244 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.244 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-05-20T16:25:42.244 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-05-20T16:25:42.244 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-05-20T16:25:42.244 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-05-20T16:25:42.244 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-05-20T16:25:42.244 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-05-20T16:25:42.244 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-05-20T16:25:42.244 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:42.244 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.244 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.245 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.245 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.245 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.245 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-05-20T16:25:42.245 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-05-20T16:25:42.245 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:42.245 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.245 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:42.245 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:42.245 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.245 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.245 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:42.245 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.245 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.246 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.246 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.246 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.246 INFO:teuthology.orchestra.run.smithi082.stdout: "roption1": { 2024-05-20T16:25:42.246 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "roption1", 2024-05-20T16:25:42.246 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:42.246 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.246 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:42.246 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:42.246 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.246 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.246 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:42.246 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.247 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.247 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.247 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.247 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.247 INFO:teuthology.orchestra.run.smithi082.stdout: "roption2": { 2024-05-20T16:25:42.247 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "roption2", 2024-05-20T16:25:42.247 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:42.247 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.247 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:42.247 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "xyz", 2024-05-20T16:25:42.247 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.247 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.247 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:42.247 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.247 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.248 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.248 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.248 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.248 INFO:teuthology.orchestra.run.smithi082.stdout: "rwoption1": { 2024-05-20T16:25:42.248 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "rwoption1", 2024-05-20T16:25:42.248 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:42.248 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.248 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:42.248 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:42.248 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.248 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.248 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:42.248 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.248 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.249 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.249 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.249 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.249 INFO:teuthology.orchestra.run.smithi082.stdout: "rwoption2": { 2024-05-20T16:25:42.249 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "rwoption2", 2024-05-20T16:25:42.249 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-05-20T16:25:42.249 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.249 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:42.249 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:42.249 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.249 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.249 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:42.249 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.249 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.250 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.250 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.250 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.250 INFO:teuthology.orchestra.run.smithi082.stdout: "rwoption3": { 2024-05-20T16:25:42.250 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "rwoption3", 2024-05-20T16:25:42.250 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "float", 2024-05-20T16:25:42.250 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.250 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:42.250 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:42.250 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.250 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.250 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:42.250 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.250 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.251 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.251 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.251 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.251 INFO:teuthology.orchestra.run.smithi082.stdout: "rwoption4": { 2024-05-20T16:25:42.251 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "rwoption4", 2024-05-20T16:25:42.251 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:42.251 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.251 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:42.251 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:42.251 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.251 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.251 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:42.251 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.251 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.252 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.252 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.252 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.252 INFO:teuthology.orchestra.run.smithi082.stdout: "rwoption5": { 2024-05-20T16:25:42.252 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "rwoption5", 2024-05-20T16:25:42.252 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:42.252 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.252 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:42.252 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:42.252 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.252 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.252 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:42.252 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.252 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.253 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.253 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.253 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.253 INFO:teuthology.orchestra.run.smithi082.stdout: "rwoption6": { 2024-05-20T16:25:42.253 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "rwoption6", 2024-05-20T16:25:42.253 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:42.253 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.253 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:42.253 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-05-20T16:25:42.253 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.253 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.253 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:42.253 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.253 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.253 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.254 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.254 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.254 INFO:teuthology.orchestra.run.smithi082.stdout: "rwoption7": { 2024-05-20T16:25:42.254 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "rwoption7", 2024-05-20T16:25:42.254 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-05-20T16:25:42.254 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.254 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:42.254 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:42.254 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "1", 2024-05-20T16:25:42.254 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "42", 2024-05-20T16:25:42.254 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:42.254 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.254 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.255 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.255 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.255 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.255 INFO:teuthology.orchestra.run.smithi082.stdout: "testkey": { 2024-05-20T16:25:42.255 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "testkey", 2024-05-20T16:25:42.255 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:42.255 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.255 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:42.255 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:42.255 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.255 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.255 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:42.255 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.255 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.255 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.256 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.256 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.256 INFO:teuthology.orchestra.run.smithi082.stdout: "testlkey": { 2024-05-20T16:25:42.256 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "testlkey", 2024-05-20T16:25:42.256 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:42.256 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.256 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:42.256 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:42.256 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.256 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.256 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:42.256 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.256 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.256 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.257 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.257 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.257 INFO:teuthology.orchestra.run.smithi082.stdout: "testnewline": { 2024-05-20T16:25:42.257 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "testnewline", 2024-05-20T16:25:42.257 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:42.257 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.257 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:42.257 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:42.257 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.257 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.257 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:42.257 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.257 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.257 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.257 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.258 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-20T16:25:42.258 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-20T16:25:42.258 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.258 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-05-20T16:25:42.258 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "snap_schedule", 2024-05-20T16:25:42.258 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-05-20T16:25:42.258 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-05-20T16:25:42.258 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-05-20T16:25:42.258 INFO:teuthology.orchestra.run.smithi082.stdout: "allow_m_granularity": { 2024-05-20T16:25:42.258 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "allow_m_granularity", 2024-05-20T16:25:42.258 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:42.258 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.258 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:42.258 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:42.259 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.259 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.259 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:42.259 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "allow minute scheduled snapshots", 2024-05-20T16:25:42.259 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.259 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.259 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.259 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.259 INFO:teuthology.orchestra.run.smithi082.stdout: "dump_on_update": { 2024-05-20T16:25:42.259 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "dump_on_update", 2024-05-20T16:25:42.259 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:42.259 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.259 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:42.259 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:42.260 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.260 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.260 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:42.260 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "dump database to debug log on update", 2024-05-20T16:25:42.260 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.260 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.260 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.260 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.260 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-05-20T16:25:42.260 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-05-20T16:25:42.260 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:42.260 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.260 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:42.260 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:42.261 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.261 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.261 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-05-20T16:25:42.261 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-05-20T16:25:42.261 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-05-20T16:25:42.261 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-05-20T16:25:42.261 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-05-20T16:25:42.261 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-05-20T16:25:42.261 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-05-20T16:25:42.261 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:42.261 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.261 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.261 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.261 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.261 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.261 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-05-20T16:25:42.262 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-05-20T16:25:42.262 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:42.262 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.262 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:42.262 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:42.262 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.262 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.262 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:42.262 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.262 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.262 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.262 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.262 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.263 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-05-20T16:25:42.263 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-05-20T16:25:42.263 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:42.263 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.263 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:42.263 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-05-20T16:25:42.263 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.263 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.263 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-05-20T16:25:42.263 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-05-20T16:25:42.263 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-05-20T16:25:42.263 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-05-20T16:25:42.263 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-05-20T16:25:42.263 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-05-20T16:25:42.264 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-05-20T16:25:42.264 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:42.264 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.264 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.264 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.264 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.264 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.264 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-05-20T16:25:42.264 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-05-20T16:25:42.264 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:42.264 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.264 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:42.264 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:42.264 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.265 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.265 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:42.265 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.265 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.265 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.265 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.265 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-20T16:25:42.265 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-20T16:25:42.265 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.265 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-05-20T16:25:42.265 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "stats", 2024-05-20T16:25:42.265 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-05-20T16:25:42.265 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-05-20T16:25:42.265 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-05-20T16:25:42.266 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-05-20T16:25:42.266 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-05-20T16:25:42.266 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:42.266 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.266 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:42.266 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:42.266 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.266 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.266 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-05-20T16:25:42.266 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-05-20T16:25:42.266 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-05-20T16:25:42.266 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-05-20T16:25:42.266 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-05-20T16:25:42.267 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-05-20T16:25:42.267 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-05-20T16:25:42.267 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:42.267 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.267 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.267 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.267 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.267 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.267 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-05-20T16:25:42.267 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-05-20T16:25:42.267 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:42.267 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.267 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:42.267 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:42.268 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.268 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.268 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:42.268 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.268 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.268 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.268 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.268 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.268 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-05-20T16:25:42.268 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-05-20T16:25:42.268 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:42.268 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.268 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:42.268 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-05-20T16:25:42.269 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.269 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.269 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-05-20T16:25:42.269 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-05-20T16:25:42.269 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-05-20T16:25:42.269 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-05-20T16:25:42.269 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-05-20T16:25:42.269 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-05-20T16:25:42.269 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-05-20T16:25:42.269 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:42.269 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.269 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.269 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.269 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.270 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.270 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-05-20T16:25:42.270 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-05-20T16:25:42.270 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:42.270 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.270 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:42.270 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:42.270 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.270 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.270 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:42.270 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.270 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.270 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.270 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.271 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-20T16:25:42.271 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-20T16:25:42.271 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.271 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-05-20T16:25:42.271 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "status", 2024-05-20T16:25:42.271 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-05-20T16:25:42.271 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-05-20T16:25:42.271 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-05-20T16:25:42.271 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-05-20T16:25:42.271 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-05-20T16:25:42.271 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:42.271 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.272 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:42.272 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:42.272 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.272 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.272 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-05-20T16:25:42.272 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-05-20T16:25:42.272 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-05-20T16:25:42.272 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-05-20T16:25:42.272 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-05-20T16:25:42.272 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-05-20T16:25:42.272 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-05-20T16:25:42.272 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:42.272 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.272 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.273 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.273 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.273 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.273 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-05-20T16:25:42.273 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-05-20T16:25:42.273 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:42.273 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.273 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:42.273 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:42.273 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.273 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.273 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:42.273 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.273 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.274 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.274 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.274 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.274 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-05-20T16:25:42.274 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-05-20T16:25:42.274 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:42.274 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.274 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:42.274 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-05-20T16:25:42.274 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.274 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.274 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-05-20T16:25:42.274 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-05-20T16:25:42.275 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-05-20T16:25:42.275 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-05-20T16:25:42.275 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-05-20T16:25:42.275 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-05-20T16:25:42.275 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-05-20T16:25:42.275 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:42.275 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.275 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.275 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.275 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.275 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.275 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-05-20T16:25:42.275 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-05-20T16:25:42.275 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:42.275 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.276 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:42.276 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:42.276 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.276 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.276 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:42.276 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.276 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.276 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.276 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.276 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-20T16:25:42.276 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-20T16:25:42.276 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.276 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-05-20T16:25:42.276 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "telegraf", 2024-05-20T16:25:42.277 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-05-20T16:25:42.277 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-05-20T16:25:42.277 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-05-20T16:25:42.277 INFO:teuthology.orchestra.run.smithi082.stdout: "address": { 2024-05-20T16:25:42.277 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "address", 2024-05-20T16:25:42.277 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:42.277 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.277 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:42.277 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-05-20T16:25:42.277 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.277 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.277 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:42.277 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.277 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.277 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.278 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.278 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.278 INFO:teuthology.orchestra.run.smithi082.stdout: "interval": { 2024-05-20T16:25:42.278 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "interval", 2024-05-20T16:25:42.278 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2024-05-20T16:25:42.278 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.278 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:42.278 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "15", 2024-05-20T16:25:42.278 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.278 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.278 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:42.278 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.278 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.279 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.279 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.279 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.279 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-05-20T16:25:42.279 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-05-20T16:25:42.279 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:42.279 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.279 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:42.279 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:42.279 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.279 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.279 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-05-20T16:25:42.279 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-05-20T16:25:42.279 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-05-20T16:25:42.279 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-05-20T16:25:42.280 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-05-20T16:25:42.280 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-05-20T16:25:42.280 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-05-20T16:25:42.280 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:42.280 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.280 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.280 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.280 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.280 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.280 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-05-20T16:25:42.280 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-05-20T16:25:42.280 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:42.280 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.281 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:42.281 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:42.281 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.281 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.281 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:42.281 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.281 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.281 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.281 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.281 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.281 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-05-20T16:25:42.281 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-05-20T16:25:42.281 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:42.281 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.282 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:42.282 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-05-20T16:25:42.282 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.282 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.282 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-05-20T16:25:42.282 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-05-20T16:25:42.282 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-05-20T16:25:42.282 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-05-20T16:25:42.282 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-05-20T16:25:42.282 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-05-20T16:25:42.282 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-05-20T16:25:42.282 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:42.282 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.282 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.283 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.283 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.283 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.283 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-05-20T16:25:42.283 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-05-20T16:25:42.283 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:42.283 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.283 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:42.283 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:42.283 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.283 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.283 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:42.283 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.283 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.284 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.284 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.284 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-20T16:25:42.284 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-20T16:25:42.284 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.284 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-05-20T16:25:42.284 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "telemetry", 2024-05-20T16:25:42.284 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-05-20T16:25:42.284 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-05-20T16:25:42.284 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-05-20T16:25:42.284 INFO:teuthology.orchestra.run.smithi082.stdout: "channel_basic": { 2024-05-20T16:25:42.284 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "channel_basic", 2024-05-20T16:25:42.284 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:42.284 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.284 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:42.285 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-05-20T16:25:42.285 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.285 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.285 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:42.285 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Share basic cluster information (size, version)", 2024-05-20T16:25:42.285 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.285 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.285 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.285 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.285 INFO:teuthology.orchestra.run.smithi082.stdout: "channel_crash": { 2024-05-20T16:25:42.285 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "channel_crash", 2024-05-20T16:25:42.285 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:42.285 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.285 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:42.286 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-05-20T16:25:42.286 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.286 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.286 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:42.286 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-05-20T16:25:42.286 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.286 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.286 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.286 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.286 INFO:teuthology.orchestra.run.smithi082.stdout: "channel_device": { 2024-05-20T16:25:42.286 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "channel_device", 2024-05-20T16:25:42.286 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:42.286 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.286 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:42.287 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-05-20T16:25:42.287 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.287 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.287 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:42.287 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-05-20T16:25:42.287 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.287 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.287 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.287 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.287 INFO:teuthology.orchestra.run.smithi082.stdout: "channel_ident": { 2024-05-20T16:25:42.287 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "channel_ident", 2024-05-20T16:25:42.287 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:42.287 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.288 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:42.288 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:42.288 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.288 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.288 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:42.288 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-05-20T16:25:42.288 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.288 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.288 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.288 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.288 INFO:teuthology.orchestra.run.smithi082.stdout: "channel_perf": { 2024-05-20T16:25:42.288 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "channel_perf", 2024-05-20T16:25:42.288 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:42.288 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.289 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:42.289 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:42.289 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.289 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.289 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:42.289 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Share various performance metrics of a cluster", 2024-05-20T16:25:42.289 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.289 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.289 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.289 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.289 INFO:teuthology.orchestra.run.smithi082.stdout: "contact": { 2024-05-20T16:25:42.289 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "contact", 2024-05-20T16:25:42.289 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:42.289 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.289 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:42.290 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:42.290 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.290 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.290 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:42.290 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.290 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.290 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.290 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.290 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.290 INFO:teuthology.orchestra.run.smithi082.stdout: "description": { 2024-05-20T16:25:42.290 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "description", 2024-05-20T16:25:42.290 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:42.290 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.290 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:42.291 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:42.291 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.291 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.291 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:42.291 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.291 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.291 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.291 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.291 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.291 INFO:teuthology.orchestra.run.smithi082.stdout: "device_url": { 2024-05-20T16:25:42.291 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "device_url", 2024-05-20T16:25:42.291 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:42.291 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.291 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:42.292 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-05-20T16:25:42.292 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.292 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.292 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:42.292 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.292 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.292 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.293 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.293 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.293 INFO:teuthology.orchestra.run.smithi082.stdout: "enabled": { 2024-05-20T16:25:42.294 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "enabled", 2024-05-20T16:25:42.294 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:42.294 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.294 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:42.294 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:42.294 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.294 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.294 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:42.294 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.294 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.294 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.294 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.294 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.295 INFO:teuthology.orchestra.run.smithi082.stdout: "interval": { 2024-05-20T16:25:42.295 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "interval", 2024-05-20T16:25:42.295 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-05-20T16:25:42.295 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.295 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:42.295 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "24", 2024-05-20T16:25:42.295 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "8", 2024-05-20T16:25:42.295 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.295 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:42.295 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.295 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.295 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.295 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.295 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.296 INFO:teuthology.orchestra.run.smithi082.stdout: "last_opt_revision": { 2024-05-20T16:25:42.296 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "last_opt_revision", 2024-05-20T16:25:42.296 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-05-20T16:25:42.296 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.296 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:42.296 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "1", 2024-05-20T16:25:42.296 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.296 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.296 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:42.296 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.296 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.296 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.296 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.296 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.296 INFO:teuthology.orchestra.run.smithi082.stdout: "leaderboard": { 2024-05-20T16:25:42.297 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "leaderboard", 2024-05-20T16:25:42.297 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:42.297 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.297 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:42.297 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:42.297 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.297 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.297 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:42.297 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.297 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.297 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.297 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.297 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.298 INFO:teuthology.orchestra.run.smithi082.stdout: "leaderboard_description": { 2024-05-20T16:25:42.298 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "leaderboard_description", 2024-05-20T16:25:42.298 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:42.298 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.298 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:42.298 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:42.298 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.298 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.298 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:42.298 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.298 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.298 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.298 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.298 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.299 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-05-20T16:25:42.299 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-05-20T16:25:42.299 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:42.299 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.299 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:42.299 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:42.299 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.299 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.299 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-05-20T16:25:42.299 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-05-20T16:25:42.299 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-05-20T16:25:42.299 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-05-20T16:25:42.299 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-05-20T16:25:42.299 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-05-20T16:25:42.300 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-05-20T16:25:42.300 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:42.300 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.300 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.300 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.300 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.300 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.300 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-05-20T16:25:42.300 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-05-20T16:25:42.300 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:42.300 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.300 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:42.300 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:42.300 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.301 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.301 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:42.301 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.301 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.301 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.301 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.301 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.301 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-05-20T16:25:42.301 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-05-20T16:25:42.301 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:42.301 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.301 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:42.301 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-05-20T16:25:42.301 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.302 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.302 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-05-20T16:25:42.302 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-05-20T16:25:42.302 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-05-20T16:25:42.302 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-05-20T16:25:42.302 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-05-20T16:25:42.302 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-05-20T16:25:42.302 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-05-20T16:25:42.302 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:42.302 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.302 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.302 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.302 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.303 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.303 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-05-20T16:25:42.303 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-05-20T16:25:42.303 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:42.303 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.303 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:42.303 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:42.303 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.303 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.303 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:42.303 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.303 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.303 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.303 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.303 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.304 INFO:teuthology.orchestra.run.smithi082.stdout: "organization": { 2024-05-20T16:25:42.304 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "organization", 2024-05-20T16:25:42.304 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:42.304 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.304 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:42.304 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:42.304 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.304 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.304 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:42.304 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.304 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.304 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.304 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.304 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.304 INFO:teuthology.orchestra.run.smithi082.stdout: "proxy": { 2024-05-20T16:25:42.304 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "proxy", 2024-05-20T16:25:42.305 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:42.305 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.305 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:42.305 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:42.305 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.305 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.305 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:42.305 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.305 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.305 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.305 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.305 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.305 INFO:teuthology.orchestra.run.smithi082.stdout: "url": { 2024-05-20T16:25:42.305 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "url", 2024-05-20T16:25:42.305 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:42.306 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.306 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:42.306 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-05-20T16:25:42.306 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.306 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.306 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:42.306 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.306 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.306 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.306 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.306 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-20T16:25:42.306 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-20T16:25:42.306 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.306 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-05-20T16:25:42.306 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "test_orchestrator", 2024-05-20T16:25:42.306 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-05-20T16:25:42.307 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-05-20T16:25:42.307 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-05-20T16:25:42.307 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-05-20T16:25:42.307 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-05-20T16:25:42.307 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:42.307 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.307 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:42.307 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:42.307 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.307 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.307 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-05-20T16:25:42.307 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-05-20T16:25:42.307 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-05-20T16:25:42.307 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-05-20T16:25:42.307 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-05-20T16:25:42.307 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-05-20T16:25:42.308 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-05-20T16:25:42.308 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:42.308 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.308 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.308 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.308 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.308 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.308 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-05-20T16:25:42.308 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-05-20T16:25:42.308 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:42.308 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.308 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:42.308 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:42.308 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.308 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.309 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:42.309 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.309 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.309 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.309 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.309 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.309 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-05-20T16:25:42.309 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-05-20T16:25:42.309 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:42.309 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.309 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:42.309 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-05-20T16:25:42.309 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.309 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.309 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-05-20T16:25:42.310 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-05-20T16:25:42.310 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-05-20T16:25:42.310 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-05-20T16:25:42.310 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-05-20T16:25:42.310 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-05-20T16:25:42.310 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-05-20T16:25:42.310 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:42.310 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.310 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.310 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.310 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.310 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.310 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-05-20T16:25:42.310 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-05-20T16:25:42.310 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:42.311 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.311 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:42.311 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:42.311 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.311 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.311 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:42.311 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.311 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.311 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.311 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.311 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-20T16:25:42.311 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-20T16:25:42.311 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.311 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-05-20T16:25:42.311 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "volumes", 2024-05-20T16:25:42.311 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-05-20T16:25:42.312 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-05-20T16:25:42.312 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-05-20T16:25:42.312 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-05-20T16:25:42.312 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-05-20T16:25:42.312 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:42.312 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.312 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:42.312 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:42.312 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.312 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.312 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-05-20T16:25:42.312 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-05-20T16:25:42.312 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-05-20T16:25:42.312 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-05-20T16:25:42.312 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-05-20T16:25:42.312 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-05-20T16:25:42.313 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-05-20T16:25:42.313 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:42.313 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.313 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.313 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.313 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.313 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.313 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-05-20T16:25:42.313 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-05-20T16:25:42.313 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:42.313 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.313 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:42.313 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:42.313 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.314 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.314 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:42.314 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.314 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.314 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.314 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.314 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.314 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-05-20T16:25:42.314 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-05-20T16:25:42.314 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:42.314 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.314 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:42.314 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-05-20T16:25:42.314 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.314 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.315 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-05-20T16:25:42.315 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-05-20T16:25:42.315 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-05-20T16:25:42.315 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-05-20T16:25:42.315 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-05-20T16:25:42.315 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-05-20T16:25:42.315 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-05-20T16:25:42.315 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:42.315 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.315 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.315 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.315 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.315 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.315 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-05-20T16:25:42.316 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-05-20T16:25:42.316 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:42.316 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.316 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:42.316 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:42.316 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.316 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.316 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:42.316 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.316 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.316 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.316 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.317 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.317 INFO:teuthology.orchestra.run.smithi082.stdout: "max_concurrent_clones": { 2024-05-20T16:25:42.317 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "max_concurrent_clones", 2024-05-20T16:25:42.317 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-05-20T16:25:42.317 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.317 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:42.317 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "4", 2024-05-20T16:25:42.317 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.317 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.317 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:42.317 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Number of asynchronous cloner threads", 2024-05-20T16:25:42.317 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.317 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.317 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.317 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.318 INFO:teuthology.orchestra.run.smithi082.stdout: "periodic_async_work": { 2024-05-20T16:25:42.318 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "periodic_async_work", 2024-05-20T16:25:42.318 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:42.318 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.318 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:42.318 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:42.318 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.318 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.318 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:42.318 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Periodically check for async work", 2024-05-20T16:25:42.318 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.318 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.318 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.319 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.319 INFO:teuthology.orchestra.run.smithi082.stdout: "snapshot_clone_delay": { 2024-05-20T16:25:42.319 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "snapshot_clone_delay", 2024-05-20T16:25:42.319 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-05-20T16:25:42.319 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.319 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:42.319 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "0", 2024-05-20T16:25:42.319 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.319 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.319 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:42.319 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-05-20T16:25:42.319 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.319 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.319 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.319 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.320 INFO:teuthology.orchestra.run.smithi082.stdout: "snapshot_clone_no_wait": { 2024-05-20T16:25:42.320 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "snapshot_clone_no_wait", 2024-05-20T16:25:42.320 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:42.320 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.320 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:42.320 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-05-20T16:25:42.320 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.320 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.320 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:42.320 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-05-20T16:25:42.320 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.320 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.320 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.321 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-20T16:25:42.321 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-20T16:25:42.321 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.321 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-05-20T16:25:42.321 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "zabbix", 2024-05-20T16:25:42.321 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-05-20T16:25:42.321 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-05-20T16:25:42.321 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-05-20T16:25:42.321 INFO:teuthology.orchestra.run.smithi082.stdout: "discovery_interval": { 2024-05-20T16:25:42.321 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "discovery_interval", 2024-05-20T16:25:42.321 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "uint", 2024-05-20T16:25:42.321 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.321 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:42.321 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "100", 2024-05-20T16:25:42.322 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.322 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.322 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:42.322 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.322 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.322 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.322 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.322 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.322 INFO:teuthology.orchestra.run.smithi082.stdout: "identifier": { 2024-05-20T16:25:42.322 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "identifier", 2024-05-20T16:25:42.322 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:42.322 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.322 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:42.323 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:42.323 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.323 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.323 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:42.323 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.323 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.323 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.323 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.323 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.323 INFO:teuthology.orchestra.run.smithi082.stdout: "interval": { 2024-05-20T16:25:42.323 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "interval", 2024-05-20T16:25:42.323 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2024-05-20T16:25:42.323 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.323 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:42.324 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "60", 2024-05-20T16:25:42.324 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.324 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.324 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:42.324 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.324 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.324 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.324 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.324 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.324 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-05-20T16:25:42.324 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-05-20T16:25:42.324 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:42.324 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.324 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:42.324 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:42.325 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.325 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.325 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-05-20T16:25:42.325 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-05-20T16:25:42.325 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-05-20T16:25:42.325 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-05-20T16:25:42.325 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-05-20T16:25:42.325 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-05-20T16:25:42.325 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-05-20T16:25:42.325 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:42.325 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.325 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.325 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.325 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.325 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.326 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-05-20T16:25:42.326 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-05-20T16:25:42.326 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:42.326 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.326 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:42.326 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:42.326 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.326 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.326 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:42.326 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.326 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.326 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.326 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.327 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.327 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-05-20T16:25:42.327 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-05-20T16:25:42.327 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:42.327 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.327 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:42.327 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-05-20T16:25:42.327 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.327 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.327 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-05-20T16:25:42.327 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-05-20T16:25:42.327 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-05-20T16:25:42.327 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-05-20T16:25:42.327 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-05-20T16:25:42.327 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-05-20T16:25:42.328 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-05-20T16:25:42.328 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:42.328 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.328 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.328 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.328 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.328 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.328 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-05-20T16:25:42.328 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-05-20T16:25:42.328 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-05-20T16:25:42.328 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.328 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-05-20T16:25:42.328 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-05-20T16:25:42.328 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.328 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.329 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:42.329 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.329 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.329 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.329 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.329 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.329 INFO:teuthology.orchestra.run.smithi082.stdout: "zabbix_host": { 2024-05-20T16:25:42.329 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "zabbix_host", 2024-05-20T16:25:42.329 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:42.329 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.329 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:42.329 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-05-20T16:25:42.329 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.329 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.329 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:42.330 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.330 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.330 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.330 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.330 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.330 INFO:teuthology.orchestra.run.smithi082.stdout: "zabbix_port": { 2024-05-20T16:25:42.330 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "zabbix_port", 2024-05-20T16:25:42.330 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-05-20T16:25:42.330 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.330 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:42.330 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "10051", 2024-05-20T16:25:42.330 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.330 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.330 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:42.331 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.331 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.331 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.331 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.331 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.331 INFO:teuthology.orchestra.run.smithi082.stdout: "zabbix_sender": { 2024-05-20T16:25:42.331 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "zabbix_sender", 2024-05-20T16:25:42.331 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-05-20T16:25:42.331 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-05-20T16:25:42.331 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-05-20T16:25:42.331 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-05-20T16:25:42.331 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-05-20T16:25:42.331 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-05-20T16:25:42.331 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-05-20T16:25:42.332 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-05-20T16:25:42.332 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-05-20T16:25:42.332 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-05-20T16:25:42.332 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-05-20T16:25:42.332 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-20T16:25:42.332 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-20T16:25:42.332 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-20T16:25:42.332 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:42.332 INFO:teuthology.orchestra.run.smithi082.stdout: "services": { 2024-05-20T16:25:42.332 INFO:teuthology.orchestra.run.smithi082.stdout: "dashboard": "https://172.21.15.82:8443/" 2024-05-20T16:25:42.332 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.332 INFO:teuthology.orchestra.run.smithi082.stdout: "always_on_modules": { 2024-05-20T16:25:42.332 INFO:teuthology.orchestra.run.smithi082.stdout: "octopus": [ 2024-05-20T16:25:42.332 INFO:teuthology.orchestra.run.smithi082.stdout: "balancer", 2024-05-20T16:25:42.332 INFO:teuthology.orchestra.run.smithi082.stdout: "crash", 2024-05-20T16:25:42.333 INFO:teuthology.orchestra.run.smithi082.stdout: "devicehealth", 2024-05-20T16:25:42.333 INFO:teuthology.orchestra.run.smithi082.stdout: "orchestrator", 2024-05-20T16:25:42.333 INFO:teuthology.orchestra.run.smithi082.stdout: "pg_autoscaler", 2024-05-20T16:25:42.333 INFO:teuthology.orchestra.run.smithi082.stdout: "progress", 2024-05-20T16:25:42.333 INFO:teuthology.orchestra.run.smithi082.stdout: "rbd_support", 2024-05-20T16:25:42.333 INFO:teuthology.orchestra.run.smithi082.stdout: "status", 2024-05-20T16:25:42.333 INFO:teuthology.orchestra.run.smithi082.stdout: "telemetry", 2024-05-20T16:25:42.333 INFO:teuthology.orchestra.run.smithi082.stdout: "volumes" 2024-05-20T16:25:42.333 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:42.333 INFO:teuthology.orchestra.run.smithi082.stdout: "pacific": [ 2024-05-20T16:25:42.333 INFO:teuthology.orchestra.run.smithi082.stdout: "balancer", 2024-05-20T16:25:42.333 INFO:teuthology.orchestra.run.smithi082.stdout: "crash", 2024-05-20T16:25:42.333 INFO:teuthology.orchestra.run.smithi082.stdout: "devicehealth", 2024-05-20T16:25:42.333 INFO:teuthology.orchestra.run.smithi082.stdout: "orchestrator", 2024-05-20T16:25:42.333 INFO:teuthology.orchestra.run.smithi082.stdout: "pg_autoscaler", 2024-05-20T16:25:42.334 INFO:teuthology.orchestra.run.smithi082.stdout: "progress", 2024-05-20T16:25:42.334 INFO:teuthology.orchestra.run.smithi082.stdout: "rbd_support", 2024-05-20T16:25:42.334 INFO:teuthology.orchestra.run.smithi082.stdout: "status", 2024-05-20T16:25:42.334 INFO:teuthology.orchestra.run.smithi082.stdout: "telemetry", 2024-05-20T16:25:42.334 INFO:teuthology.orchestra.run.smithi082.stdout: "volumes" 2024-05-20T16:25:42.334 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:42.334 INFO:teuthology.orchestra.run.smithi082.stdout: "quincy": [ 2024-05-20T16:25:42.334 INFO:teuthology.orchestra.run.smithi082.stdout: "balancer", 2024-05-20T16:25:42.334 INFO:teuthology.orchestra.run.smithi082.stdout: "crash", 2024-05-20T16:25:42.334 INFO:teuthology.orchestra.run.smithi082.stdout: "devicehealth", 2024-05-20T16:25:42.334 INFO:teuthology.orchestra.run.smithi082.stdout: "orchestrator", 2024-05-20T16:25:42.334 INFO:teuthology.orchestra.run.smithi082.stdout: "pg_autoscaler", 2024-05-20T16:25:42.334 INFO:teuthology.orchestra.run.smithi082.stdout: "progress", 2024-05-20T16:25:42.334 INFO:teuthology.orchestra.run.smithi082.stdout: "rbd_support", 2024-05-20T16:25:42.334 INFO:teuthology.orchestra.run.smithi082.stdout: "status", 2024-05-20T16:25:42.335 INFO:teuthology.orchestra.run.smithi082.stdout: "telemetry", 2024-05-20T16:25:42.335 INFO:teuthology.orchestra.run.smithi082.stdout: "volumes" 2024-05-20T16:25:42.335 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:42.335 INFO:teuthology.orchestra.run.smithi082.stdout: "reef": [ 2024-05-20T16:25:42.335 INFO:teuthology.orchestra.run.smithi082.stdout: "balancer", 2024-05-20T16:25:42.335 INFO:teuthology.orchestra.run.smithi082.stdout: "crash", 2024-05-20T16:25:42.335 INFO:teuthology.orchestra.run.smithi082.stdout: "devicehealth", 2024-05-20T16:25:42.335 INFO:teuthology.orchestra.run.smithi082.stdout: "orchestrator", 2024-05-20T16:25:42.335 INFO:teuthology.orchestra.run.smithi082.stdout: "pg_autoscaler", 2024-05-20T16:25:42.335 INFO:teuthology.orchestra.run.smithi082.stdout: "progress", 2024-05-20T16:25:42.335 INFO:teuthology.orchestra.run.smithi082.stdout: "rbd_support", 2024-05-20T16:25:42.335 INFO:teuthology.orchestra.run.smithi082.stdout: "status", 2024-05-20T16:25:42.335 INFO:teuthology.orchestra.run.smithi082.stdout: "telemetry", 2024-05-20T16:25:42.335 INFO:teuthology.orchestra.run.smithi082.stdout: "volumes" 2024-05-20T16:25:42.336 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-05-20T16:25:42.336 INFO:teuthology.orchestra.run.smithi082.stdout: "squid": [ 2024-05-20T16:25:42.336 INFO:teuthology.orchestra.run.smithi082.stdout: "balancer", 2024-05-20T16:25:42.336 INFO:teuthology.orchestra.run.smithi082.stdout: "crash", 2024-05-20T16:25:42.336 INFO:teuthology.orchestra.run.smithi082.stdout: "devicehealth", 2024-05-20T16:25:42.336 INFO:teuthology.orchestra.run.smithi082.stdout: "orchestrator", 2024-05-20T16:25:42.336 INFO:teuthology.orchestra.run.smithi082.stdout: "pg_autoscaler", 2024-05-20T16:25:42.336 INFO:teuthology.orchestra.run.smithi082.stdout: "progress", 2024-05-20T16:25:42.336 INFO:teuthology.orchestra.run.smithi082.stdout: "rbd_support", 2024-05-20T16:25:42.336 INFO:teuthology.orchestra.run.smithi082.stdout: "status", 2024-05-20T16:25:42.336 INFO:teuthology.orchestra.run.smithi082.stdout: "telemetry", 2024-05-20T16:25:42.336 INFO:teuthology.orchestra.run.smithi082.stdout: "volumes" 2024-05-20T16:25:42.336 INFO:teuthology.orchestra.run.smithi082.stdout: ] 2024-05-20T16:25:42.337 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.337 INFO:teuthology.orchestra.run.smithi082.stdout: "last_failure_osd_epoch": 3, 2024-05-20T16:25:42.337 INFO:teuthology.orchestra.run.smithi082.stdout: "active_clients": [ 2024-05-20T16:25:42.337 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-05-20T16:25:42.337 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "devicehealth", 2024-05-20T16:25:42.337 INFO:teuthology.orchestra.run.smithi082.stdout: "addrvec": [ 2024-05-20T16:25:42.337 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-05-20T16:25:42.337 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "v2", 2024-05-20T16:25:42.337 INFO:teuthology.orchestra.run.smithi082.stdout: "addr": "172.21.15.82:0", 2024-05-20T16:25:42.337 INFO:teuthology.orchestra.run.smithi082.stdout: "nonce": 3429482372 2024-05-20T16:25:42.337 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-20T16:25:42.337 INFO:teuthology.orchestra.run.smithi082.stdout: ] 2024-05-20T16:25:42.337 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.337 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-05-20T16:25:42.337 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "libcephsqlite", 2024-05-20T16:25:42.338 INFO:teuthology.orchestra.run.smithi082.stdout: "addrvec": [ 2024-05-20T16:25:42.338 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-05-20T16:25:42.338 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "v2", 2024-05-20T16:25:42.338 INFO:teuthology.orchestra.run.smithi082.stdout: "addr": "172.21.15.82:0", 2024-05-20T16:25:42.338 INFO:teuthology.orchestra.run.smithi082.stdout: "nonce": 2353029080 2024-05-20T16:25:42.338 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-20T16:25:42.338 INFO:teuthology.orchestra.run.smithi082.stdout: ] 2024-05-20T16:25:42.338 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.338 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-05-20T16:25:42.338 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "rbd_support", 2024-05-20T16:25:42.338 INFO:teuthology.orchestra.run.smithi082.stdout: "addrvec": [ 2024-05-20T16:25:42.338 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-05-20T16:25:42.338 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "v2", 2024-05-20T16:25:42.338 INFO:teuthology.orchestra.run.smithi082.stdout: "addr": "172.21.15.82:0", 2024-05-20T16:25:42.339 INFO:teuthology.orchestra.run.smithi082.stdout: "nonce": 3379158184 2024-05-20T16:25:42.339 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-20T16:25:42.339 INFO:teuthology.orchestra.run.smithi082.stdout: ] 2024-05-20T16:25:42.339 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:42.339 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-05-20T16:25:42.339 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "volumes", 2024-05-20T16:25:42.339 INFO:teuthology.orchestra.run.smithi082.stdout: "addrvec": [ 2024-05-20T16:25:42.339 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-05-20T16:25:42.339 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "v2", 2024-05-20T16:25:42.339 INFO:teuthology.orchestra.run.smithi082.stdout: "addr": "172.21.15.82:0", 2024-05-20T16:25:42.339 INFO:teuthology.orchestra.run.smithi082.stdout: "nonce": 1953851727 2024-05-20T16:25:42.339 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-20T16:25:42.339 INFO:teuthology.orchestra.run.smithi082.stdout: ] 2024-05-20T16:25:42.339 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-20T16:25:42.339 INFO:teuthology.orchestra.run.smithi082.stdout: ] 2024-05-20T16:25:42.340 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-05-20T16:25:42.341 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:25:41 smithi184 ceph-mon[36209]: from='client.? 172.21.15.82:0/3463500826' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-20T16:25:42.341 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:25:41 smithi184 ceph-mon[36209]: from='client.? 172.21.15.82:0/4036978409' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-05-20T16:25:42.341 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:25:41 smithi160 ceph-mon[36347]: from='client.? 172.21.15.82:0/3463500826' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-20T16:25:42.341 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:25:41 smithi160 ceph-mon[36347]: from='client.? 172.21.15.82:0/4036978409' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-05-20T16:25:42.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:41.053+0000 7f47e0eeb640 1 -- 172.21.15.82:0/4036978409 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f47c80a2b90 msgr2=0x7f47c80a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:25:42.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:41.053+0000 7f47e0eeb640 1 --2- 172.21.15.82:0/4036978409 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f47c80a2b90 0x7f47c80a50a0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f47cc009230 tx=0x7f47cc000bc0 comp rx=0 tx=0).stop 2024-05-20T16:25:42.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:41.053+0000 7f47e0eeb640 1 -- 172.21.15.82:0/4036978409 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f47dc104050 msgr2=0x7f47dc1a0740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:25:42.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:41.053+0000 7f47e0eeb640 1 --2- 172.21.15.82:0/4036978409 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f47dc104050 0x7f47dc1a0740 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f47d4009fd0 tx=0x7f47d400b260 comp rx=0 tx=0).stop 2024-05-20T16:25:42.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:41.053+0000 7f47e0eeb640 1 -- 172.21.15.82:0/4036978409 shutdown_connections 2024-05-20T16:25:42.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:41.053+0000 7f47e0eeb640 1 --2- 172.21.15.82:0/4036978409 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f47c80a2b90 0x7f47c80a50a0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:25:42.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:41.053+0000 7f47e0eeb640 1 --2- 172.21.15.82:0/4036978409 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f47dc104050 0x7f47dc1a0740 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:25:42.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:41.053+0000 7f47e0eeb640 1 --2- 172.21.15.82:0/4036978409 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f47dc101200 0x7f47dc1997b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:25:42.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:41.053+0000 7f47e0eeb640 1 --2- 172.21.15.82:0/4036978409 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f47dc100700 0x7f47dc199270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:25:42.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:41.053+0000 7f47e0eeb640 1 -- 172.21.15.82:0/4036978409 >> 172.21.15.82:0/4036978409 conn(0x7f47dc0fc320 msgr2=0x7f47dc104bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:25:42.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:41.053+0000 7f47e0eeb640 1 -- 172.21.15.82:0/4036978409 shutdown_connections 2024-05-20T16:25:42.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:25:41.053+0000 7f47e0eeb640 1 -- 172.21.15.82:0/4036978409 wait complete. 2024-05-20T16:25:42.348 DEBUG:tasks.check_counter:Getting stats from a 2024-05-20T16:25:42.349 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell --fsid 59d1600a-16c2-11ef-bc9a-c7b262605968 -- ceph daemon mds.a perf dump 2024-05-20T16:25:42.349 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:25:41 smithi082 ceph-mon[32238]: from='client.? 172.21.15.82:0/3463500826' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-20T16:25:42.349 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:25:41 smithi082 ceph-mon[32238]: from='client.? 172.21.15.82:0/4036978409' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-05-20T16:25:42.599 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T16:25:42.721 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:25:42 smithi082 ceph-mon[32238]: pgmap v716: 129 pgs: 129 active+clean; 10 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 641 MiB/s wr, 1.24k op/s 2024-05-20T16:25:42.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:25:42 smithi160 ceph-mon[36347]: pgmap v716: 129 pgs: 129 active+clean; 10 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 641 MiB/s wr, 1.24k op/s 2024-05-20T16:25:42.885 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:25:42 smithi184 ceph-mon[36209]: pgmap v716: 129 pgs: 129 active+clean; 10 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 641 MiB/s wr, 1.24k op/s 2024-05-20T16:25:45.253 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:25:44 smithi082 ceph-mon[32238]: pgmap v717: 129 pgs: 129 active+clean; 10 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 575 MiB/s wr, 1.17k op/s 2024-05-20T16:25:45.312 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:25:44 smithi160 ceph-mon[36347]: pgmap v717: 129 pgs: 129 active+clean; 10 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 575 MiB/s wr, 1.17k op/s 2024-05-20T16:25:45.385 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:25:44 smithi184 ceph-mon[36209]: pgmap v717: 129 pgs: 129 active+clean; 10 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 575 MiB/s wr, 1.17k op/s 2024-05-20T16:25:45.415 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-05-20T16:25:45.415 INFO:teuthology.orchestra.run.smithi082.stdout: "AsyncMessenger::Worker-0": { 2024-05-20T16:25:45.416 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_messages": 232, 2024-05-20T16:25:45.416 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages": 227, 2024-05-20T16:25:45.416 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_bytes": 757248, 2024-05-20T16:25:45.416 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_bytes": 139584, 2024-05-20T16:25:45.416 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_created_connections": 4, 2024-05-20T16:25:45.416 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_active_connections": 0, 2024-05-20T16:25:45.416 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_total_time": 0.045917106, 2024-05-20T16:25:45.416 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_send_time": 0.016279629, 2024-05-20T16:25:45.416 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_recv_time": 0.024495290, 2024-05-20T16:25:45.416 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_fast_dispatch_time": 0.006925787, 2024-05-20T16:25:45.416 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages_queue_lat": { 2024-05-20T16:25:45.416 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 226, 2024-05-20T16:25:45.416 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.036758067, 2024-05-20T16:25:45.417 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000162646 2024-05-20T16:25:45.417 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:45.417 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_handle_ack_lat": { 2024-05-20T16:25:45.417 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-05-20T16:25:45.417 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-05-20T16:25:45.417 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-05-20T16:25:45.417 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:45.417 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_encrypted_bytes": 757248, 2024-05-20T16:25:45.417 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_encrypted_bytes": 139584 2024-05-20T16:25:45.417 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:45.417 INFO:teuthology.orchestra.run.smithi082.stdout: "AsyncMessenger::Worker-1": { 2024-05-20T16:25:45.417 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_messages": 5, 2024-05-20T16:25:45.417 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages": 174, 2024-05-20T16:25:45.418 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_bytes": 3184, 2024-05-20T16:25:45.418 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_bytes": 108416, 2024-05-20T16:25:45.418 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_created_connections": 3, 2024-05-20T16:25:45.418 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_active_connections": 2, 2024-05-20T16:25:45.418 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_total_time": 0.018203886, 2024-05-20T16:25:45.418 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_send_time": 0.011884247, 2024-05-20T16:25:45.418 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_recv_time": 0.001806649, 2024-05-20T16:25:45.418 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-20T16:25:45.418 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages_queue_lat": { 2024-05-20T16:25:45.418 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 173, 2024-05-20T16:25:45.418 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.017506962, 2024-05-20T16:25:45.418 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000101196 2024-05-20T16:25:45.418 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:45.418 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_handle_ack_lat": { 2024-05-20T16:25:45.419 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-05-20T16:25:45.419 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-05-20T16:25:45.419 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-05-20T16:25:45.419 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:45.419 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_encrypted_bytes": 3184, 2024-05-20T16:25:45.419 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_encrypted_bytes": 108416 2024-05-20T16:25:45.419 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:45.419 INFO:teuthology.orchestra.run.smithi082.stdout: "AsyncMessenger::Worker-2": { 2024-05-20T16:25:45.419 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_messages": 0, 2024-05-20T16:25:45.419 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages": 2, 2024-05-20T16:25:45.419 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_bytes": 0, 2024-05-20T16:25:45.419 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_bytes": 0, 2024-05-20T16:25:45.419 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_created_connections": 2, 2024-05-20T16:25:45.419 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_active_connections": 1, 2024-05-20T16:25:45.420 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_total_time": 0.000313706, 2024-05-20T16:25:45.420 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_send_time": 0.000000000, 2024-05-20T16:25:45.420 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_recv_time": 0.000245253, 2024-05-20T16:25:45.420 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-20T16:25:45.420 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages_queue_lat": { 2024-05-20T16:25:45.420 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-05-20T16:25:45.420 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-05-20T16:25:45.420 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-05-20T16:25:45.420 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:45.420 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_handle_ack_lat": { 2024-05-20T16:25:45.420 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-05-20T16:25:45.420 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-05-20T16:25:45.420 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-05-20T16:25:45.420 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:45.421 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-20T16:25:45.421 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_encrypted_bytes": 0 2024-05-20T16:25:45.421 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:45.421 INFO:teuthology.orchestra.run.smithi082.stdout: "cct": { 2024-05-20T16:25:45.421 INFO:teuthology.orchestra.run.smithi082.stdout: "total_workers": 0, 2024-05-20T16:25:45.421 INFO:teuthology.orchestra.run.smithi082.stdout: "unhealthy_workers": 0 2024-05-20T16:25:45.421 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:45.421 INFO:teuthology.orchestra.run.smithi082.stdout: "mempool": { 2024-05-20T16:25:45.421 INFO:teuthology.orchestra.run.smithi082.stdout: "bloom_filter_bytes": 0, 2024-05-20T16:25:45.421 INFO:teuthology.orchestra.run.smithi082.stdout: "bloom_filter_items": 0, 2024-05-20T16:25:45.422 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_alloc_bytes": 0, 2024-05-20T16:25:45.422 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_alloc_items": 0, 2024-05-20T16:25:45.422 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_data_bytes": 0, 2024-05-20T16:25:45.422 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_data_items": 0, 2024-05-20T16:25:45.422 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-20T16:25:45.422 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_onode_items": 0, 2024-05-20T16:25:45.422 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-20T16:25:45.422 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_meta_items": 0, 2024-05-20T16:25:45.422 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_other_bytes": 0, 2024-05-20T16:25:45.423 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_other_items": 0, 2024-05-20T16:25:45.423 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-20T16:25:45.423 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_buffer_items": 0, 2024-05-20T16:25:45.423 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_extent_bytes": 0, 2024-05-20T16:25:45.423 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_extent_items": 0, 2024-05-20T16:25:45.423 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_blob_bytes": 0, 2024-05-20T16:25:45.423 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_blob_items": 0, 2024-05-20T16:25:45.423 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-20T16:25:45.423 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_shared_blob_items": 0, 2024-05-20T16:25:45.423 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-20T16:25:45.423 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_inline_bl_items": 0, 2024-05-20T16:25:45.423 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_fsck_bytes": 0, 2024-05-20T16:25:45.423 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_fsck_items": 0, 2024-05-20T16:25:45.424 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_txc_bytes": 0, 2024-05-20T16:25:45.424 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_txc_items": 0, 2024-05-20T16:25:45.424 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-20T16:25:45.424 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_writing_deferred_items": 0, 2024-05-20T16:25:45.424 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_writing_bytes": 0, 2024-05-20T16:25:45.424 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_writing_items": 0, 2024-05-20T16:25:45.424 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_bytes": 0, 2024-05-20T16:25:45.424 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_items": 0, 2024-05-20T16:25:45.424 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_file_reader_bytes": 0, 2024-05-20T16:25:45.424 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_file_reader_items": 0, 2024-05-20T16:25:45.424 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_file_writer_bytes": 0, 2024-05-20T16:25:45.424 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_file_writer_items": 0, 2024-05-20T16:25:45.424 INFO:teuthology.orchestra.run.smithi082.stdout: "buffer_anon_bytes": 148984, 2024-05-20T16:25:45.424 INFO:teuthology.orchestra.run.smithi082.stdout: "buffer_anon_items": 30, 2024-05-20T16:25:45.425 INFO:teuthology.orchestra.run.smithi082.stdout: "buffer_meta_bytes": 0, 2024-05-20T16:25:45.425 INFO:teuthology.orchestra.run.smithi082.stdout: "buffer_meta_items": 0, 2024-05-20T16:25:45.425 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_bytes": 0, 2024-05-20T16:25:45.425 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_items": 0, 2024-05-20T16:25:45.425 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_mapbl_bytes": 0, 2024-05-20T16:25:45.425 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_mapbl_items": 0, 2024-05-20T16:25:45.425 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_pglog_bytes": 0, 2024-05-20T16:25:45.425 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_pglog_items": 0, 2024-05-20T16:25:45.425 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_bytes": 0, 2024-05-20T16:25:45.425 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_items": 0, 2024-05-20T16:25:45.425 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_mapping_bytes": 0, 2024-05-20T16:25:45.425 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_mapping_items": 0, 2024-05-20T16:25:45.425 INFO:teuthology.orchestra.run.smithi082.stdout: "pgmap_bytes": 0, 2024-05-20T16:25:45.425 INFO:teuthology.orchestra.run.smithi082.stdout: "pgmap_items": 0, 2024-05-20T16:25:45.426 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_co_bytes": 928, 2024-05-20T16:25:45.426 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_co_items": 2, 2024-05-20T16:25:45.426 INFO:teuthology.orchestra.run.smithi082.stdout: "unittest_1_bytes": 0, 2024-05-20T16:25:45.426 INFO:teuthology.orchestra.run.smithi082.stdout: "unittest_1_items": 0, 2024-05-20T16:25:45.426 INFO:teuthology.orchestra.run.smithi082.stdout: "unittest_2_bytes": 0, 2024-05-20T16:25:45.426 INFO:teuthology.orchestra.run.smithi082.stdout: "unittest_2_items": 0 2024-05-20T16:25:45.426 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:45.426 INFO:teuthology.orchestra.run.smithi082.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-20T16:25:45.426 INFO:teuthology.orchestra.run.smithi082.stdout: "val": 0, 2024-05-20T16:25:45.426 INFO:teuthology.orchestra.run.smithi082.stdout: "max": 104857600, 2024-05-20T16:25:45.426 INFO:teuthology.orchestra.run.smithi082.stdout: "get_started": 0, 2024-05-20T16:25:45.426 INFO:teuthology.orchestra.run.smithi082.stdout: "get": 233, 2024-05-20T16:25:45.427 INFO:teuthology.orchestra.run.smithi082.stdout: "get_sum": 724482, 2024-05-20T16:25:45.427 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_fail": 0, 2024-05-20T16:25:45.427 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_success": 233, 2024-05-20T16:25:45.427 INFO:teuthology.orchestra.run.smithi082.stdout: "take": 0, 2024-05-20T16:25:45.427 INFO:teuthology.orchestra.run.smithi082.stdout: "take_sum": 0, 2024-05-20T16:25:45.427 INFO:teuthology.orchestra.run.smithi082.stdout: "put": 233, 2024-05-20T16:25:45.427 INFO:teuthology.orchestra.run.smithi082.stdout: "put_sum": 724482, 2024-05-20T16:25:45.427 INFO:teuthology.orchestra.run.smithi082.stdout: "wait": { 2024-05-20T16:25:45.427 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-05-20T16:25:45.427 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-05-20T16:25:45.427 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-05-20T16:25:45.427 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-20T16:25:45.427 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-20T16:25:45.427 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-05-20T16:25:46.069 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.a 2024-05-20T16:25:46.069 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.a 2024-05-20T16:25:46.069 DEBUG:tasks.check_counter:Getting stats from d 2024-05-20T16:25:46.069 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell --fsid 59d1600a-16c2-11ef-bc9a-c7b262605968 -- ceph daemon mds.d perf dump 2024-05-20T16:25:46.317 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T16:25:47.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:25:46 smithi082 ceph-mon[32238]: pgmap v718: 129 pgs: 129 active+clean; 10 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 942 KiB/s rd, 397 MiB/s wr, 863 op/s 2024-05-20T16:25:47.312 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:25:46 smithi160 ceph-mon[36347]: pgmap v718: 129 pgs: 129 active+clean; 10 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 942 KiB/s rd, 397 MiB/s wr, 863 op/s 2024-05-20T16:25:47.385 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:25:46 smithi184 ceph-mon[36209]: pgmap v718: 129 pgs: 129 active+clean; 10 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 942 KiB/s rd, 397 MiB/s wr, 863 op/s 2024-05-20T16:25:48.312 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:25:48 smithi160 ceph-mon[36347]: pgmap v719: 129 pgs: 129 active+clean; 10 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 942 KiB/s rd, 315 MiB/s wr, 709 op/s 2024-05-20T16:25:48.321 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:25:48 smithi082 ceph-mon[32238]: pgmap v719: 129 pgs: 129 active+clean; 10 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 942 KiB/s rd, 315 MiB/s wr, 709 op/s 2024-05-20T16:25:48.385 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:25:48 smithi184 ceph-mon[36209]: pgmap v719: 129 pgs: 129 active+clean; 10 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 942 KiB/s rd, 315 MiB/s wr, 709 op/s 2024-05-20T16:25:48.655 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-05-20T16:25:48.656 INFO:teuthology.orchestra.run.smithi082.stdout: "AsyncMessenger::Worker-0": { 2024-05-20T16:25:48.656 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_messages": 237, 2024-05-20T16:25:48.656 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages": 232, 2024-05-20T16:25:48.656 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_bytes": 758608, 2024-05-20T16:25:48.656 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_bytes": 142784, 2024-05-20T16:25:48.656 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_created_connections": 4, 2024-05-20T16:25:48.656 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_active_connections": 0, 2024-05-20T16:25:48.656 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_total_time": 0.046189570, 2024-05-20T16:25:48.656 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_send_time": 0.015646624, 2024-05-20T16:25:48.656 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_recv_time": 0.024808547, 2024-05-20T16:25:48.656 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_fast_dispatch_time": 0.007239775, 2024-05-20T16:25:48.656 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages_queue_lat": { 2024-05-20T16:25:48.656 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 231, 2024-05-20T16:25:48.657 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.026608785, 2024-05-20T16:25:48.657 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000115189 2024-05-20T16:25:48.657 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:48.657 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_handle_ack_lat": { 2024-05-20T16:25:48.657 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-05-20T16:25:48.657 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-05-20T16:25:48.657 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-05-20T16:25:48.657 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:48.657 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_encrypted_bytes": 758608, 2024-05-20T16:25:48.657 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_encrypted_bytes": 142784 2024-05-20T16:25:48.657 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:48.657 INFO:teuthology.orchestra.run.smithi082.stdout: "AsyncMessenger::Worker-1": { 2024-05-20T16:25:48.657 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_messages": 1, 2024-05-20T16:25:48.657 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages": 177, 2024-05-20T16:25:48.658 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_bytes": 160, 2024-05-20T16:25:48.658 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_bytes": 108848, 2024-05-20T16:25:48.658 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_created_connections": 3, 2024-05-20T16:25:48.658 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_active_connections": 2, 2024-05-20T16:25:48.658 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_total_time": 0.015622537, 2024-05-20T16:25:48.658 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_send_time": 0.011574054, 2024-05-20T16:25:48.658 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_recv_time": 0.001327106, 2024-05-20T16:25:48.658 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-20T16:25:48.658 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages_queue_lat": { 2024-05-20T16:25:48.658 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 175, 2024-05-20T16:25:48.658 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.009942163, 2024-05-20T16:25:48.658 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000056812 2024-05-20T16:25:48.658 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:48.659 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_handle_ack_lat": { 2024-05-20T16:25:48.659 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-05-20T16:25:48.659 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-05-20T16:25:48.659 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-05-20T16:25:48.659 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:48.659 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_encrypted_bytes": 160, 2024-05-20T16:25:48.659 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_encrypted_bytes": 108848 2024-05-20T16:25:48.659 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:48.659 INFO:teuthology.orchestra.run.smithi082.stdout: "AsyncMessenger::Worker-2": { 2024-05-20T16:25:48.659 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_messages": 4, 2024-05-20T16:25:48.659 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages": 4, 2024-05-20T16:25:48.659 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_bytes": 3024, 2024-05-20T16:25:48.659 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_bytes": 448, 2024-05-20T16:25:48.660 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_created_connections": 2, 2024-05-20T16:25:48.660 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_active_connections": 1, 2024-05-20T16:25:48.660 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_total_time": 0.000699521, 2024-05-20T16:25:48.660 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_send_time": 0.000032895, 2024-05-20T16:25:48.660 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_recv_time": 0.000647877, 2024-05-20T16:25:48.660 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-20T16:25:48.660 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages_queue_lat": { 2024-05-20T16:25:48.660 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 3, 2024-05-20T16:25:48.660 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000790036, 2024-05-20T16:25:48.660 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000263345 2024-05-20T16:25:48.660 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:48.660 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_handle_ack_lat": { 2024-05-20T16:25:48.660 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-05-20T16:25:48.660 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-05-20T16:25:48.661 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-05-20T16:25:48.661 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:48.661 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_encrypted_bytes": 3024, 2024-05-20T16:25:48.661 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_encrypted_bytes": 448 2024-05-20T16:25:48.661 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:48.661 INFO:teuthology.orchestra.run.smithi082.stdout: "cct": { 2024-05-20T16:25:48.661 INFO:teuthology.orchestra.run.smithi082.stdout: "total_workers": 0, 2024-05-20T16:25:48.661 INFO:teuthology.orchestra.run.smithi082.stdout: "unhealthy_workers": 0 2024-05-20T16:25:48.661 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:48.661 INFO:teuthology.orchestra.run.smithi082.stdout: "mempool": { 2024-05-20T16:25:48.661 INFO:teuthology.orchestra.run.smithi082.stdout: "bloom_filter_bytes": 0, 2024-05-20T16:25:48.661 INFO:teuthology.orchestra.run.smithi082.stdout: "bloom_filter_items": 0, 2024-05-20T16:25:48.661 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_alloc_bytes": 0, 2024-05-20T16:25:48.661 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_alloc_items": 0, 2024-05-20T16:25:48.662 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_data_bytes": 0, 2024-05-20T16:25:48.662 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_data_items": 0, 2024-05-20T16:25:48.662 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-20T16:25:48.662 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_onode_items": 0, 2024-05-20T16:25:48.662 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-20T16:25:48.662 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_meta_items": 0, 2024-05-20T16:25:48.662 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_other_bytes": 0, 2024-05-20T16:25:48.662 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_other_items": 0, 2024-05-20T16:25:48.662 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-20T16:25:48.662 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_buffer_items": 0, 2024-05-20T16:25:48.662 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_extent_bytes": 0, 2024-05-20T16:25:48.662 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_extent_items": 0, 2024-05-20T16:25:48.662 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_blob_bytes": 0, 2024-05-20T16:25:48.663 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_blob_items": 0, 2024-05-20T16:25:48.663 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-20T16:25:48.663 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_shared_blob_items": 0, 2024-05-20T16:25:48.663 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-20T16:25:48.663 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_inline_bl_items": 0, 2024-05-20T16:25:48.663 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_fsck_bytes": 0, 2024-05-20T16:25:48.663 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_fsck_items": 0, 2024-05-20T16:25:48.663 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_txc_bytes": 0, 2024-05-20T16:25:48.663 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_txc_items": 0, 2024-05-20T16:25:48.663 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-20T16:25:48.663 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_writing_deferred_items": 0, 2024-05-20T16:25:48.663 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_writing_bytes": 0, 2024-05-20T16:25:48.663 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_writing_items": 0, 2024-05-20T16:25:48.663 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_bytes": 0, 2024-05-20T16:25:48.663 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_items": 0, 2024-05-20T16:25:48.664 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_file_reader_bytes": 0, 2024-05-20T16:25:48.664 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_file_reader_items": 0, 2024-05-20T16:25:48.664 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_file_writer_bytes": 0, 2024-05-20T16:25:48.664 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_file_writer_items": 0, 2024-05-20T16:25:48.664 INFO:teuthology.orchestra.run.smithi082.stdout: "buffer_anon_bytes": 148945, 2024-05-20T16:25:48.664 INFO:teuthology.orchestra.run.smithi082.stdout: "buffer_anon_items": 29, 2024-05-20T16:25:48.664 INFO:teuthology.orchestra.run.smithi082.stdout: "buffer_meta_bytes": 0, 2024-05-20T16:25:48.664 INFO:teuthology.orchestra.run.smithi082.stdout: "buffer_meta_items": 0, 2024-05-20T16:25:48.664 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_bytes": 0, 2024-05-20T16:25:48.664 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_items": 0, 2024-05-20T16:25:48.665 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_mapbl_bytes": 0, 2024-05-20T16:25:48.665 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_mapbl_items": 0, 2024-05-20T16:25:48.665 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_pglog_bytes": 0, 2024-05-20T16:25:48.665 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_pglog_items": 0, 2024-05-20T16:25:48.665 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_bytes": 0, 2024-05-20T16:25:48.665 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_items": 0, 2024-05-20T16:25:48.665 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_mapping_bytes": 0, 2024-05-20T16:25:48.665 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_mapping_items": 0, 2024-05-20T16:25:48.665 INFO:teuthology.orchestra.run.smithi082.stdout: "pgmap_bytes": 0, 2024-05-20T16:25:48.665 INFO:teuthology.orchestra.run.smithi082.stdout: "pgmap_items": 0, 2024-05-20T16:25:48.665 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_co_bytes": 928, 2024-05-20T16:25:48.666 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_co_items": 2, 2024-05-20T16:25:48.666 INFO:teuthology.orchestra.run.smithi082.stdout: "unittest_1_bytes": 0, 2024-05-20T16:25:48.666 INFO:teuthology.orchestra.run.smithi082.stdout: "unittest_1_items": 0, 2024-05-20T16:25:48.666 INFO:teuthology.orchestra.run.smithi082.stdout: "unittest_2_bytes": 0, 2024-05-20T16:25:48.666 INFO:teuthology.orchestra.run.smithi082.stdout: "unittest_2_items": 0 2024-05-20T16:25:48.666 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:48.666 INFO:teuthology.orchestra.run.smithi082.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-20T16:25:48.666 INFO:teuthology.orchestra.run.smithi082.stdout: "val": 0, 2024-05-20T16:25:48.666 INFO:teuthology.orchestra.run.smithi082.stdout: "max": 104857600, 2024-05-20T16:25:48.666 INFO:teuthology.orchestra.run.smithi082.stdout: "get_started": 0, 2024-05-20T16:25:48.666 INFO:teuthology.orchestra.run.smithi082.stdout: "get": 238, 2024-05-20T16:25:48.666 INFO:teuthology.orchestra.run.smithi082.stdout: "get_sum": 725132, 2024-05-20T16:25:48.667 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_fail": 0, 2024-05-20T16:25:48.667 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_success": 238, 2024-05-20T16:25:48.667 INFO:teuthology.orchestra.run.smithi082.stdout: "take": 0, 2024-05-20T16:25:48.667 INFO:teuthology.orchestra.run.smithi082.stdout: "take_sum": 0, 2024-05-20T16:25:48.667 INFO:teuthology.orchestra.run.smithi082.stdout: "put": 238, 2024-05-20T16:25:48.667 INFO:teuthology.orchestra.run.smithi082.stdout: "put_sum": 725132, 2024-05-20T16:25:48.667 INFO:teuthology.orchestra.run.smithi082.stdout: "wait": { 2024-05-20T16:25:48.667 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-05-20T16:25:48.667 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-05-20T16:25:48.667 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-05-20T16:25:48.667 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-20T16:25:48.667 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-20T16:25:48.667 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-05-20T16:25:49.485 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-05-20T16:25:49.485 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-05-20T16:25:49.485 DEBUG:tasks.check_counter:Getting stats from g 2024-05-20T16:25:49.486 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell --fsid 59d1600a-16c2-11ef-bc9a-c7b262605968 -- ceph daemon mds.g perf dump 2024-05-20T16:25:49.732 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T16:25:51.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:25:50 smithi082 ceph-mon[32238]: pgmap v720: 129 pgs: 129 active+clean; 10 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 336 KiB/s rd, 287 MiB/s wr, 642 op/s 2024-05-20T16:25:51.311 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:25:50 smithi160 ceph-mon[36347]: pgmap v720: 129 pgs: 129 active+clean; 10 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 336 KiB/s rd, 287 MiB/s wr, 642 op/s 2024-05-20T16:25:51.385 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:25:50 smithi184 ceph-mon[36209]: pgmap v720: 129 pgs: 129 active+clean; 10 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 336 KiB/s rd, 287 MiB/s wr, 642 op/s 2024-05-20T16:25:51.633 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-05-20T16:25:51.634 INFO:teuthology.orchestra.run.smithi082.stdout: "AsyncMessenger::Worker-0": { 2024-05-20T16:25:51.634 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_messages": 4, 2024-05-20T16:25:51.634 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages": 4, 2024-05-20T16:25:51.634 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_bytes": 3024, 2024-05-20T16:25:51.634 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_bytes": 448, 2024-05-20T16:25:51.634 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_created_connections": 4, 2024-05-20T16:25:51.634 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_active_connections": 0, 2024-05-20T16:25:51.634 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_total_time": 0.001747049, 2024-05-20T16:25:51.634 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_send_time": 0.000041544, 2024-05-20T16:25:51.634 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_recv_time": 0.000749839, 2024-05-20T16:25:51.635 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-20T16:25:51.635 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages_queue_lat": { 2024-05-20T16:25:51.635 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 3, 2024-05-20T16:25:51.635 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000882907, 2024-05-20T16:25:51.635 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000294302 2024-05-20T16:25:51.635 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:51.635 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_handle_ack_lat": { 2024-05-20T16:25:51.635 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-05-20T16:25:51.635 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-05-20T16:25:51.635 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-05-20T16:25:51.635 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:51.635 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_encrypted_bytes": 3024, 2024-05-20T16:25:51.635 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_encrypted_bytes": 448 2024-05-20T16:25:51.636 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:51.636 INFO:teuthology.orchestra.run.smithi082.stdout: "AsyncMessenger::Worker-1": { 2024-05-20T16:25:51.636 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_messages": 234, 2024-05-20T16:25:51.636 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages": 229, 2024-05-20T16:25:51.636 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_bytes": 757792, 2024-05-20T16:25:51.636 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_bytes": 140864, 2024-05-20T16:25:51.636 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_created_connections": 2, 2024-05-20T16:25:51.636 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_active_connections": 1, 2024-05-20T16:25:51.636 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_total_time": 0.047149548, 2024-05-20T16:25:51.636 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_send_time": 0.017042638, 2024-05-20T16:25:51.636 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_recv_time": 0.023143756, 2024-05-20T16:25:51.636 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_fast_dispatch_time": 0.007503844, 2024-05-20T16:25:51.636 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages_queue_lat": { 2024-05-20T16:25:51.637 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 228, 2024-05-20T16:25:51.637 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.045509463, 2024-05-20T16:25:51.637 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000199602 2024-05-20T16:25:51.637 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:51.637 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_handle_ack_lat": { 2024-05-20T16:25:51.637 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-05-20T16:25:51.637 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-05-20T16:25:51.637 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-05-20T16:25:51.637 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:51.637 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_encrypted_bytes": 757792, 2024-05-20T16:25:51.637 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_encrypted_bytes": 140864 2024-05-20T16:25:51.637 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:51.637 INFO:teuthology.orchestra.run.smithi082.stdout: "AsyncMessenger::Worker-2": { 2024-05-20T16:25:51.637 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_messages": 1, 2024-05-20T16:25:51.637 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages": 174, 2024-05-20T16:25:51.638 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_bytes": 160, 2024-05-20T16:25:51.638 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_bytes": 108320, 2024-05-20T16:25:51.638 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_created_connections": 3, 2024-05-20T16:25:51.638 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_active_connections": 2, 2024-05-20T16:25:51.638 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_total_time": 0.021890772, 2024-05-20T16:25:51.638 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_send_time": 0.017599753, 2024-05-20T16:25:51.638 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_recv_time": 0.001421458, 2024-05-20T16:25:51.638 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-20T16:25:51.638 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages_queue_lat": { 2024-05-20T16:25:51.638 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 172, 2024-05-20T16:25:51.638 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.014333444, 2024-05-20T16:25:51.638 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000083333 2024-05-20T16:25:51.638 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:51.638 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_handle_ack_lat": { 2024-05-20T16:25:51.638 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-05-20T16:25:51.639 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-05-20T16:25:51.639 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-05-20T16:25:51.639 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:51.639 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_encrypted_bytes": 160, 2024-05-20T16:25:51.639 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_encrypted_bytes": 108320 2024-05-20T16:25:51.639 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:51.639 INFO:teuthology.orchestra.run.smithi082.stdout: "cct": { 2024-05-20T16:25:51.639 INFO:teuthology.orchestra.run.smithi082.stdout: "total_workers": 0, 2024-05-20T16:25:51.639 INFO:teuthology.orchestra.run.smithi082.stdout: "unhealthy_workers": 0 2024-05-20T16:25:51.639 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:51.639 INFO:teuthology.orchestra.run.smithi082.stdout: "mempool": { 2024-05-20T16:25:51.639 INFO:teuthology.orchestra.run.smithi082.stdout: "bloom_filter_bytes": 0, 2024-05-20T16:25:51.639 INFO:teuthology.orchestra.run.smithi082.stdout: "bloom_filter_items": 0, 2024-05-20T16:25:51.639 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_alloc_bytes": 0, 2024-05-20T16:25:51.639 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_alloc_items": 0, 2024-05-20T16:25:51.640 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_data_bytes": 0, 2024-05-20T16:25:51.640 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_data_items": 0, 2024-05-20T16:25:51.640 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-20T16:25:51.640 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_onode_items": 0, 2024-05-20T16:25:51.640 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-20T16:25:51.640 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_meta_items": 0, 2024-05-20T16:25:51.640 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_other_bytes": 0, 2024-05-20T16:25:51.640 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_other_items": 0, 2024-05-20T16:25:51.640 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-20T16:25:51.640 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_buffer_items": 0, 2024-05-20T16:25:51.640 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_extent_bytes": 0, 2024-05-20T16:25:51.640 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_extent_items": 0, 2024-05-20T16:25:51.640 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_blob_bytes": 0, 2024-05-20T16:25:51.640 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_blob_items": 0, 2024-05-20T16:25:51.641 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-20T16:25:51.641 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_shared_blob_items": 0, 2024-05-20T16:25:51.641 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-20T16:25:51.641 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_inline_bl_items": 0, 2024-05-20T16:25:51.641 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_fsck_bytes": 0, 2024-05-20T16:25:51.641 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_fsck_items": 0, 2024-05-20T16:25:51.641 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_txc_bytes": 0, 2024-05-20T16:25:51.641 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_txc_items": 0, 2024-05-20T16:25:51.641 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-20T16:25:51.641 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_writing_deferred_items": 0, 2024-05-20T16:25:51.641 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_writing_bytes": 0, 2024-05-20T16:25:51.641 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_writing_items": 0, 2024-05-20T16:25:51.641 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_bytes": 0, 2024-05-20T16:25:51.641 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_items": 0, 2024-05-20T16:25:51.641 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_file_reader_bytes": 0, 2024-05-20T16:25:51.642 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_file_reader_items": 0, 2024-05-20T16:25:51.642 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_file_writer_bytes": 0, 2024-05-20T16:25:51.642 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_file_writer_items": 0, 2024-05-20T16:25:51.642 INFO:teuthology.orchestra.run.smithi082.stdout: "buffer_anon_bytes": 148984, 2024-05-20T16:25:51.642 INFO:teuthology.orchestra.run.smithi082.stdout: "buffer_anon_items": 30, 2024-05-20T16:25:51.642 INFO:teuthology.orchestra.run.smithi082.stdout: "buffer_meta_bytes": 0, 2024-05-20T16:25:51.642 INFO:teuthology.orchestra.run.smithi082.stdout: "buffer_meta_items": 0, 2024-05-20T16:25:51.642 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_bytes": 0, 2024-05-20T16:25:51.642 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_items": 0, 2024-05-20T16:25:51.642 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_mapbl_bytes": 0, 2024-05-20T16:25:51.642 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_mapbl_items": 0, 2024-05-20T16:25:51.642 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_pglog_bytes": 0, 2024-05-20T16:25:51.643 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_pglog_items": 0, 2024-05-20T16:25:51.643 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_bytes": 0, 2024-05-20T16:25:51.643 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_items": 0, 2024-05-20T16:25:51.643 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_mapping_bytes": 0, 2024-05-20T16:25:51.643 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_mapping_items": 0, 2024-05-20T16:25:51.643 INFO:teuthology.orchestra.run.smithi082.stdout: "pgmap_bytes": 0, 2024-05-20T16:25:51.643 INFO:teuthology.orchestra.run.smithi082.stdout: "pgmap_items": 0, 2024-05-20T16:25:51.643 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_co_bytes": 928, 2024-05-20T16:25:51.643 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_co_items": 2, 2024-05-20T16:25:51.643 INFO:teuthology.orchestra.run.smithi082.stdout: "unittest_1_bytes": 0, 2024-05-20T16:25:51.643 INFO:teuthology.orchestra.run.smithi082.stdout: "unittest_1_items": 0, 2024-05-20T16:25:51.643 INFO:teuthology.orchestra.run.smithi082.stdout: "unittest_2_bytes": 0, 2024-05-20T16:25:51.643 INFO:teuthology.orchestra.run.smithi082.stdout: "unittest_2_items": 0 2024-05-20T16:25:51.644 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:51.644 INFO:teuthology.orchestra.run.smithi082.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-20T16:25:51.644 INFO:teuthology.orchestra.run.smithi082.stdout: "val": 0, 2024-05-20T16:25:51.644 INFO:teuthology.orchestra.run.smithi082.stdout: "max": 104857600, 2024-05-20T16:25:51.644 INFO:teuthology.orchestra.run.smithi082.stdout: "get_started": 0, 2024-05-20T16:25:51.644 INFO:teuthology.orchestra.run.smithi082.stdout: "get": 235, 2024-05-20T16:25:51.644 INFO:teuthology.orchestra.run.smithi082.stdout: "get_sum": 724742, 2024-05-20T16:25:51.644 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_fail": 0, 2024-05-20T16:25:51.644 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_success": 235, 2024-05-20T16:25:51.644 INFO:teuthology.orchestra.run.smithi082.stdout: "take": 0, 2024-05-20T16:25:51.644 INFO:teuthology.orchestra.run.smithi082.stdout: "take_sum": 0, 2024-05-20T16:25:51.644 INFO:teuthology.orchestra.run.smithi082.stdout: "put": 235, 2024-05-20T16:25:51.644 INFO:teuthology.orchestra.run.smithi082.stdout: "put_sum": 724742, 2024-05-20T16:25:51.644 INFO:teuthology.orchestra.run.smithi082.stdout: "wait": { 2024-05-20T16:25:51.644 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-05-20T16:25:51.645 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-05-20T16:25:51.645 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-05-20T16:25:51.645 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-20T16:25:51.645 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-20T16:25:51.645 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-05-20T16:25:52.333 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.g 2024-05-20T16:25:52.334 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.g 2024-05-20T16:25:52.334 DEBUG:tasks.check_counter:Getting stats from j 2024-05-20T16:25:52.334 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell --fsid 59d1600a-16c2-11ef-bc9a-c7b262605968 -- ceph daemon mds.j perf dump 2024-05-20T16:25:52.578 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/mon.a/config 2024-05-20T16:25:53.062 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:25:52 smithi160 ceph-mon[36347]: pgmap v721: 129 pgs: 129 active+clean; 10 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 291 KiB/s rd, 101 MiB/s wr, 214 op/s 2024-05-20T16:25:53.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:25:52 smithi082 ceph-mon[32238]: pgmap v721: 129 pgs: 129 active+clean; 10 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 291 KiB/s rd, 101 MiB/s wr, 214 op/s 2024-05-20T16:25:53.385 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:25:52 smithi184 ceph-mon[36209]: pgmap v721: 129 pgs: 129 active+clean; 10 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 291 KiB/s rd, 101 MiB/s wr, 214 op/s 2024-05-20T16:25:54.940 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-05-20T16:25:54.940 INFO:teuthology.orchestra.run.smithi082.stdout: "AsyncMessenger::Worker-0": { 2024-05-20T16:25:54.940 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_messages": 0, 2024-05-20T16:25:54.940 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages": 2, 2024-05-20T16:25:54.941 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_bytes": 0, 2024-05-20T16:25:54.941 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_bytes": 0, 2024-05-20T16:25:54.941 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_created_connections": 4, 2024-05-20T16:25:54.941 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_active_connections": 0, 2024-05-20T16:25:54.941 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_total_time": 0.001211299, 2024-05-20T16:25:54.941 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_send_time": 0.000000000, 2024-05-20T16:25:54.941 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_recv_time": 0.000270200, 2024-05-20T16:25:54.941 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-20T16:25:54.941 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages_queue_lat": { 2024-05-20T16:25:54.941 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-05-20T16:25:54.941 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-05-20T16:25:54.941 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-05-20T16:25:54.941 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:54.941 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_handle_ack_lat": { 2024-05-20T16:25:54.942 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-05-20T16:25:54.942 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-05-20T16:25:54.942 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-05-20T16:25:54.942 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:54.942 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-20T16:25:54.942 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_encrypted_bytes": 0 2024-05-20T16:25:54.942 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:54.942 INFO:teuthology.orchestra.run.smithi082.stdout: "AsyncMessenger::Worker-1": { 2024-05-20T16:25:54.942 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_messages": 240, 2024-05-20T16:25:54.942 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages": 235, 2024-05-20T16:25:54.942 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_bytes": 759424, 2024-05-20T16:25:54.942 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_bytes": 144704, 2024-05-20T16:25:54.942 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_created_connections": 2, 2024-05-20T16:25:54.943 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_active_connections": 1, 2024-05-20T16:25:54.943 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_total_time": 0.046437727, 2024-05-20T16:25:54.943 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_send_time": 0.015970798, 2024-05-20T16:25:54.943 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_recv_time": 0.025789886, 2024-05-20T16:25:54.943 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_fast_dispatch_time": 0.007436589, 2024-05-20T16:25:54.943 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages_queue_lat": { 2024-05-20T16:25:54.943 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 234, 2024-05-20T16:25:54.943 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.022946480, 2024-05-20T16:25:54.943 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000098061 2024-05-20T16:25:54.943 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:54.943 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_handle_ack_lat": { 2024-05-20T16:25:54.943 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-05-20T16:25:54.943 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-05-20T16:25:54.943 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-05-20T16:25:54.944 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:54.944 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_encrypted_bytes": 759424, 2024-05-20T16:25:54.944 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_encrypted_bytes": 144704 2024-05-20T16:25:54.944 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:54.944 INFO:teuthology.orchestra.run.smithi082.stdout: "AsyncMessenger::Worker-2": { 2024-05-20T16:25:54.944 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_messages": 5, 2024-05-20T16:25:54.944 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages": 181, 2024-05-20T16:25:54.944 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_bytes": 3184, 2024-05-20T16:25:54.944 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_bytes": 109648, 2024-05-20T16:25:54.944 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_created_connections": 3, 2024-05-20T16:25:54.944 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_active_connections": 2, 2024-05-20T16:25:54.944 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_total_time": 0.016151284, 2024-05-20T16:25:54.944 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_send_time": 0.012326743, 2024-05-20T16:25:54.944 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_recv_time": 0.001773903, 2024-05-20T16:25:54.944 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-20T16:25:54.945 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages_queue_lat": { 2024-05-20T16:25:54.945 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 180, 2024-05-20T16:25:54.945 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.030229910, 2024-05-20T16:25:54.945 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000167943 2024-05-20T16:25:54.945 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:54.945 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_handle_ack_lat": { 2024-05-20T16:25:54.945 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-05-20T16:25:54.945 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-05-20T16:25:54.945 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-05-20T16:25:54.945 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:54.945 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_encrypted_bytes": 3184, 2024-05-20T16:25:54.945 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_encrypted_bytes": 109648 2024-05-20T16:25:54.945 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:54.945 INFO:teuthology.orchestra.run.smithi082.stdout: "cct": { 2024-05-20T16:25:54.946 INFO:teuthology.orchestra.run.smithi082.stdout: "total_workers": 0, 2024-05-20T16:25:54.946 INFO:teuthology.orchestra.run.smithi082.stdout: "unhealthy_workers": 0 2024-05-20T16:25:54.946 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:54.946 INFO:teuthology.orchestra.run.smithi082.stdout: "mempool": { 2024-05-20T16:25:54.946 INFO:teuthology.orchestra.run.smithi082.stdout: "bloom_filter_bytes": 0, 2024-05-20T16:25:54.946 INFO:teuthology.orchestra.run.smithi082.stdout: "bloom_filter_items": 0, 2024-05-20T16:25:54.946 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_alloc_bytes": 0, 2024-05-20T16:25:54.946 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_alloc_items": 0, 2024-05-20T16:25:54.946 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_data_bytes": 0, 2024-05-20T16:25:54.946 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_data_items": 0, 2024-05-20T16:25:54.946 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-20T16:25:54.946 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_onode_items": 0, 2024-05-20T16:25:54.946 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-20T16:25:54.947 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_meta_items": 0, 2024-05-20T16:25:54.947 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_other_bytes": 0, 2024-05-20T16:25:54.947 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_other_items": 0, 2024-05-20T16:25:54.947 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-20T16:25:54.947 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_buffer_items": 0, 2024-05-20T16:25:54.947 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_extent_bytes": 0, 2024-05-20T16:25:54.947 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_extent_items": 0, 2024-05-20T16:25:54.948 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_blob_bytes": 0, 2024-05-20T16:25:54.948 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_blob_items": 0, 2024-05-20T16:25:54.948 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-20T16:25:54.948 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_shared_blob_items": 0, 2024-05-20T16:25:54.948 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-20T16:25:54.948 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_inline_bl_items": 0, 2024-05-20T16:25:54.948 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_fsck_bytes": 0, 2024-05-20T16:25:54.948 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_fsck_items": 0, 2024-05-20T16:25:54.948 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_txc_bytes": 0, 2024-05-20T16:25:54.948 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_txc_items": 0, 2024-05-20T16:25:54.948 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-20T16:25:54.948 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_writing_deferred_items": 0, 2024-05-20T16:25:54.948 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_writing_bytes": 0, 2024-05-20T16:25:54.949 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_writing_items": 0, 2024-05-20T16:25:54.949 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_bytes": 0, 2024-05-20T16:25:54.949 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_items": 0, 2024-05-20T16:25:54.949 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_file_reader_bytes": 0, 2024-05-20T16:25:54.949 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_file_reader_items": 0, 2024-05-20T16:25:54.949 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_file_writer_bytes": 0, 2024-05-20T16:25:54.949 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_file_writer_items": 0, 2024-05-20T16:25:54.949 INFO:teuthology.orchestra.run.smithi082.stdout: "buffer_anon_bytes": 148967, 2024-05-20T16:25:54.949 INFO:teuthology.orchestra.run.smithi082.stdout: "buffer_anon_items": 29, 2024-05-20T16:25:54.949 INFO:teuthology.orchestra.run.smithi082.stdout: "buffer_meta_bytes": 0, 2024-05-20T16:25:54.949 INFO:teuthology.orchestra.run.smithi082.stdout: "buffer_meta_items": 0, 2024-05-20T16:25:54.949 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_bytes": 0, 2024-05-20T16:25:54.949 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_items": 0, 2024-05-20T16:25:54.949 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_mapbl_bytes": 0, 2024-05-20T16:25:54.949 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_mapbl_items": 0, 2024-05-20T16:25:54.950 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_pglog_bytes": 0, 2024-05-20T16:25:54.950 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_pglog_items": 0, 2024-05-20T16:25:54.950 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_bytes": 0, 2024-05-20T16:25:54.950 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_items": 0, 2024-05-20T16:25:54.950 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_mapping_bytes": 0, 2024-05-20T16:25:54.950 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_mapping_items": 0, 2024-05-20T16:25:54.950 INFO:teuthology.orchestra.run.smithi082.stdout: "pgmap_bytes": 0, 2024-05-20T16:25:54.950 INFO:teuthology.orchestra.run.smithi082.stdout: "pgmap_items": 0, 2024-05-20T16:25:54.951 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_co_bytes": 928, 2024-05-20T16:25:54.951 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_co_items": 2, 2024-05-20T16:25:54.951 INFO:teuthology.orchestra.run.smithi082.stdout: "unittest_1_bytes": 0, 2024-05-20T16:25:54.951 INFO:teuthology.orchestra.run.smithi082.stdout: "unittest_1_items": 0, 2024-05-20T16:25:54.951 INFO:teuthology.orchestra.run.smithi082.stdout: "unittest_2_bytes": 0, 2024-05-20T16:25:54.951 INFO:teuthology.orchestra.run.smithi082.stdout: "unittest_2_items": 0 2024-05-20T16:25:54.951 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-20T16:25:54.951 INFO:teuthology.orchestra.run.smithi082.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-20T16:25:54.952 INFO:teuthology.orchestra.run.smithi082.stdout: "val": 0, 2024-05-20T16:25:54.952 INFO:teuthology.orchestra.run.smithi082.stdout: "max": 104857600, 2024-05-20T16:25:54.952 INFO:teuthology.orchestra.run.smithi082.stdout: "get_started": 0, 2024-05-20T16:25:54.952 INFO:teuthology.orchestra.run.smithi082.stdout: "get": 241, 2024-05-20T16:25:54.952 INFO:teuthology.orchestra.run.smithi082.stdout: "get_sum": 725522, 2024-05-20T16:25:54.952 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_fail": 0, 2024-05-20T16:25:54.952 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_success": 241, 2024-05-20T16:25:54.952 INFO:teuthology.orchestra.run.smithi082.stdout: "take": 0, 2024-05-20T16:25:54.953 INFO:teuthology.orchestra.run.smithi082.stdout: "take_sum": 0, 2024-05-20T16:25:54.953 INFO:teuthology.orchestra.run.smithi082.stdout: "put": 241, 2024-05-20T16:25:54.953 INFO:teuthology.orchestra.run.smithi082.stdout: "put_sum": 725522, 2024-05-20T16:25:54.953 INFO:teuthology.orchestra.run.smithi082.stdout: "wait": { 2024-05-20T16:25:54.953 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-05-20T16:25:54.953 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-05-20T16:25:54.953 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-05-20T16:25:54.953 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-20T16:25:54.953 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-20T16:25:54.953 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-05-20T16:25:55.312 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:25:54 smithi160 ceph-mon[36347]: pgmap v722: 129 pgs: 129 active+clean; 10 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 292 KiB/s rd, 78 MiB/s wr, 150 op/s 2024-05-20T16:25:55.358 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:25:54 smithi082 ceph-mon[32238]: pgmap v722: 129 pgs: 129 active+clean; 10 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 292 KiB/s rd, 78 MiB/s wr, 150 op/s 2024-05-20T16:25:55.386 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:25:54 smithi184 ceph-mon[36209]: pgmap v722: 129 pgs: 129 active+clean; 10 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 292 KiB/s rd, 78 MiB/s wr, 150 op/s 2024-05-20T16:25:55.830 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-05-20T16:25:55.831 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-05-20T16:25:55.831 DEBUG:tasks.check_counter:Getting stats from b 2024-05-20T16:25:55.832 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell --fsid 59d1600a-16c2-11ef-bc9a-c7b262605968 -- ceph daemon mds.b perf dump 2024-05-20T16:25:56.143 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/mon.b/config 2024-05-20T16:25:57.308 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:25:56 smithi160 ceph-mon[36347]: pgmap v723: 129 pgs: 129 active+clean; 10 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 71 MiB/s wr, 132 op/s 2024-05-20T16:25:57.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:25:56 smithi082 ceph-mon[32238]: pgmap v723: 129 pgs: 129 active+clean; 10 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 71 MiB/s wr, 132 op/s 2024-05-20T16:25:57.385 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:25:56 smithi184 ceph-mon[36209]: pgmap v723: 129 pgs: 129 active+clean; 10 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 71 MiB/s wr, 132 op/s 2024-05-20T16:25:57.980 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:25:57 smithi160 ceph-mon[36347]: pgmap v724: 129 pgs: 129 active+clean; 10 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 68 MiB/s wr, 126 op/s 2024-05-20T16:25:58.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:25:57 smithi082 ceph-mon[32238]: pgmap v724: 129 pgs: 129 active+clean; 10 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 68 MiB/s wr, 126 op/s 2024-05-20T16:25:58.385 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:25:57 smithi184 ceph-mon[36209]: pgmap v724: 129 pgs: 129 active+clean; 10 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 68 MiB/s wr, 126 op/s 2024-05-20T16:25:58.393 INFO:teuthology.orchestra.run.smithi160.stdout:{ 2024-05-20T16:25:58.393 INFO:teuthology.orchestra.run.smithi160.stdout: "AsyncMessenger::Worker-0": { 2024-05-20T16:25:58.393 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_messages": 4782, 2024-05-20T16:25:58.393 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_messages": 5002, 2024-05-20T16:25:58.393 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_bytes": 1282069, 2024-05-20T16:25:58.393 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_bytes": 488466877, 2024-05-20T16:25:58.393 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_created_connections": 75, 2024-05-20T16:25:58.393 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_active_connections": 5, 2024-05-20T16:25:58.393 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_total_time": 3.877156991, 2024-05-20T16:25:58.393 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_send_time": 2.228008711, 2024-05-20T16:25:58.393 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_recv_time": 0.173979295, 2024-05-20T16:25:58.394 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_fast_dispatch_time": 1.431737561, 2024-05-20T16:25:58.394 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_messages_queue_lat": { 2024-05-20T16:25:58.394 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 5000, 2024-05-20T16:25:58.394 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 2.321875671, 2024-05-20T16:25:58.394 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000464375 2024-05-20T16:25:58.394 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-20T16:25:58.394 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_handle_ack_lat": { 2024-05-20T16:25:58.394 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 1667, 2024-05-20T16:25:58.394 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000445008, 2024-05-20T16:25:58.394 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000266 2024-05-20T16:25:58.394 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-20T16:25:58.394 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_encrypted_bytes": 32816, 2024-05-20T16:25:58.395 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_encrypted_bytes": 1456 2024-05-20T16:25:58.395 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-20T16:25:58.395 INFO:teuthology.orchestra.run.smithi160.stdout: "AsyncMessenger::Worker-1": { 2024-05-20T16:25:58.395 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_messages": 516875, 2024-05-20T16:25:58.395 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_messages": 519313, 2024-05-20T16:25:58.395 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_bytes": 181485258, 2024-05-20T16:25:58.395 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_bytes": 476670674, 2024-05-20T16:25:58.395 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_created_connections": 23, 2024-05-20T16:25:58.395 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_active_connections": 5, 2024-05-20T16:25:58.395 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_total_time": 27.581463295, 2024-05-20T16:25:58.395 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_send_time": 10.976762351, 2024-05-20T16:25:58.395 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_recv_time": 39.579130015, 2024-05-20T16:25:58.395 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_fast_dispatch_time": 1.382340582, 2024-05-20T16:25:58.396 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_messages_queue_lat": { 2024-05-20T16:25:58.396 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 519312, 2024-05-20T16:25:58.396 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 147.009819470, 2024-05-20T16:25:58.396 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000283085 2024-05-20T16:25:58.396 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-20T16:25:58.396 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_handle_ack_lat": { 2024-05-20T16:25:58.396 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 798603, 2024-05-20T16:25:58.396 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.242716449, 2024-05-20T16:25:58.396 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000303 2024-05-20T16:25:58.396 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-20T16:25:58.396 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_encrypted_bytes": 875504, 2024-05-20T16:25:58.396 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_encrypted_bytes": 179808 2024-05-20T16:25:58.396 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-20T16:25:58.397 INFO:teuthology.orchestra.run.smithi160.stdout: "AsyncMessenger::Worker-2": { 2024-05-20T16:25:58.397 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_messages": 1764, 2024-05-20T16:25:58.397 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_messages": 1966, 2024-05-20T16:25:58.397 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_bytes": 425964, 2024-05-20T16:25:58.397 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_bytes": 178560759, 2024-05-20T16:25:58.397 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_created_connections": 11, 2024-05-20T16:25:58.397 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_active_connections": 6, 2024-05-20T16:25:58.397 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_total_time": 1.738320396, 2024-05-20T16:25:58.397 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_send_time": 0.713971554, 2024-05-20T16:25:58.397 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_recv_time": 0.059452165, 2024-05-20T16:25:58.397 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_fast_dispatch_time": 0.953944160, 2024-05-20T16:25:58.397 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_messages_queue_lat": { 2024-05-20T16:25:58.397 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 1965, 2024-05-20T16:25:58.397 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 1.174476984, 2024-05-20T16:25:58.398 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000597698 2024-05-20T16:25:58.398 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-20T16:25:58.398 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_handle_ack_lat": { 2024-05-20T16:25:58.398 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 1188, 2024-05-20T16:25:58.398 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000380103, 2024-05-20T16:25:58.398 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000319 2024-05-20T16:25:58.398 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-20T16:25:58.398 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_encrypted_bytes": 3184, 2024-05-20T16:25:58.398 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_encrypted_bytes": 395472 2024-05-20T16:25:58.398 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-20T16:25:58.398 INFO:teuthology.orchestra.run.smithi160.stdout: "cct": { 2024-05-20T16:25:58.398 INFO:teuthology.orchestra.run.smithi160.stdout: "total_workers": 1, 2024-05-20T16:25:58.399 INFO:teuthology.orchestra.run.smithi160.stdout: "unhealthy_workers": 0 2024-05-20T16:25:58.399 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-20T16:25:58.399 INFO:teuthology.orchestra.run.smithi160.stdout: "finisher-MDSRank": { 2024-05-20T16:25:58.399 INFO:teuthology.orchestra.run.smithi160.stdout: "queue_len": 0, 2024-05-20T16:25:58.399 INFO:teuthology.orchestra.run.smithi160.stdout: "complete_latency": { 2024-05-20T16:25:58.399 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 3376, 2024-05-20T16:25:58.399 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 383.544552925, 2024-05-20T16:25:58.399 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.113609168 2024-05-20T16:25:58.399 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-05-20T16:25:58.399 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-20T16:25:58.399 INFO:teuthology.orchestra.run.smithi160.stdout: "finisher-PurgeQueue": { 2024-05-20T16:25:58.399 INFO:teuthology.orchestra.run.smithi160.stdout: "queue_len": 0, 2024-05-20T16:25:58.399 INFO:teuthology.orchestra.run.smithi160.stdout: "complete_latency": { 2024-05-20T16:25:58.400 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 37, 2024-05-20T16:25:58.400 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.003474204, 2024-05-20T16:25:58.400 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000093897 2024-05-20T16:25:58.400 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-05-20T16:25:58.400 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-20T16:25:58.400 INFO:teuthology.orchestra.run.smithi160.stdout: "mds": { 2024-05-20T16:25:58.400 INFO:teuthology.orchestra.run.smithi160.stdout: "request": 102, 2024-05-20T16:25:58.400 INFO:teuthology.orchestra.run.smithi160.stdout: "reply": 102, 2024-05-20T16:25:58.400 INFO:teuthology.orchestra.run.smithi160.stdout: "reply_latency": { 2024-05-20T16:25:58.400 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 102, 2024-05-20T16:25:58.400 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.441070522, 2024-05-20T16:25:58.400 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.004324220 2024-05-20T16:25:58.400 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-20T16:25:58.400 INFO:teuthology.orchestra.run.smithi160.stdout: "slow_reply": 0, 2024-05-20T16:25:58.401 INFO:teuthology.orchestra.run.smithi160.stdout: "forward": 0, 2024-05-20T16:25:58.401 INFO:teuthology.orchestra.run.smithi160.stdout: "dir_fetch_complete": 0, 2024-05-20T16:25:58.401 INFO:teuthology.orchestra.run.smithi160.stdout: "dir_fetch_keys": 0, 2024-05-20T16:25:58.401 INFO:teuthology.orchestra.run.smithi160.stdout: "dir_commit": 123, 2024-05-20T16:25:58.401 INFO:teuthology.orchestra.run.smithi160.stdout: "dir_split": 1, 2024-05-20T16:25:58.401 INFO:teuthology.orchestra.run.smithi160.stdout: "dir_merge": 1, 2024-05-20T16:25:58.401 INFO:teuthology.orchestra.run.smithi160.stdout: "inodes": 32, 2024-05-20T16:25:58.401 INFO:teuthology.orchestra.run.smithi160.stdout: "inodes_top": 5, 2024-05-20T16:25:58.401 INFO:teuthology.orchestra.run.smithi160.stdout: "inodes_bottom": 27, 2024-05-20T16:25:58.401 INFO:teuthology.orchestra.run.smithi160.stdout: "inodes_pin_tail": 0, 2024-05-20T16:25:58.401 INFO:teuthology.orchestra.run.smithi160.stdout: "inodes_pinned": 24, 2024-05-20T16:25:58.401 INFO:teuthology.orchestra.run.smithi160.stdout: "inodes_expired": 0, 2024-05-20T16:25:58.401 INFO:teuthology.orchestra.run.smithi160.stdout: "inodes_with_caps": 10, 2024-05-20T16:25:58.401 INFO:teuthology.orchestra.run.smithi160.stdout: "caps": 10, 2024-05-20T16:25:58.402 INFO:teuthology.orchestra.run.smithi160.stdout: "subtrees": 2, 2024-05-20T16:25:58.402 INFO:teuthology.orchestra.run.smithi160.stdout: "traverse": 129, 2024-05-20T16:25:58.402 INFO:teuthology.orchestra.run.smithi160.stdout: "traverse_hit": 121, 2024-05-20T16:25:58.402 INFO:teuthology.orchestra.run.smithi160.stdout: "traverse_forward": 0, 2024-05-20T16:25:58.402 INFO:teuthology.orchestra.run.smithi160.stdout: "traverse_discover": 0, 2024-05-20T16:25:58.402 INFO:teuthology.orchestra.run.smithi160.stdout: "traverse_dir_fetch": 0, 2024-05-20T16:25:58.402 INFO:teuthology.orchestra.run.smithi160.stdout: "traverse_remote_ino": 0, 2024-05-20T16:25:58.402 INFO:teuthology.orchestra.run.smithi160.stdout: "traverse_lock": 0, 2024-05-20T16:25:58.402 INFO:teuthology.orchestra.run.smithi160.stdout: "load_cent": 0, 2024-05-20T16:25:58.402 INFO:teuthology.orchestra.run.smithi160.stdout: "q": 0, 2024-05-20T16:25:58.402 INFO:teuthology.orchestra.run.smithi160.stdout: "exported": 0, 2024-05-20T16:25:58.402 INFO:teuthology.orchestra.run.smithi160.stdout: "exported_inodes": 0, 2024-05-20T16:25:58.403 INFO:teuthology.orchestra.run.smithi160.stdout: "imported": 0, 2024-05-20T16:25:58.403 INFO:teuthology.orchestra.run.smithi160.stdout: "imported_inodes": 0, 2024-05-20T16:25:58.403 INFO:teuthology.orchestra.run.smithi160.stdout: "openino_dir_fetch": 0, 2024-05-20T16:25:58.403 INFO:teuthology.orchestra.run.smithi160.stdout: "openino_backtrace_fetch": 0, 2024-05-20T16:25:58.403 INFO:teuthology.orchestra.run.smithi160.stdout: "openino_peer_discover": 0, 2024-05-20T16:25:58.403 INFO:teuthology.orchestra.run.smithi160.stdout: "root_rfiles": 4, 2024-05-20T16:25:58.403 INFO:teuthology.orchestra.run.smithi160.stdout: "root_rbytes": 234, 2024-05-20T16:25:58.403 INFO:teuthology.orchestra.run.smithi160.stdout: "root_rsnaps": 0, 2024-05-20T16:25:58.403 INFO:teuthology.orchestra.run.smithi160.stdout: "scrub_backtrace_fetch": 0, 2024-05-20T16:25:58.403 INFO:teuthology.orchestra.run.smithi160.stdout: "scrub_set_tag": 0, 2024-05-20T16:25:58.403 INFO:teuthology.orchestra.run.smithi160.stdout: "scrub_backtrace_repaired": 0, 2024-05-20T16:25:58.403 INFO:teuthology.orchestra.run.smithi160.stdout: "scrub_inotable_repaired": 0, 2024-05-20T16:25:58.403 INFO:teuthology.orchestra.run.smithi160.stdout: "scrub_dir_inodes": 0, 2024-05-20T16:25:58.403 INFO:teuthology.orchestra.run.smithi160.stdout: "scrub_dir_base_inodes": 0, 2024-05-20T16:25:58.404 INFO:teuthology.orchestra.run.smithi160.stdout: "scrub_dirfrag_rstats": 0, 2024-05-20T16:25:58.404 INFO:teuthology.orchestra.run.smithi160.stdout: "scrub_file_inodes": 0, 2024-05-20T16:25:58.404 INFO:teuthology.orchestra.run.smithi160.stdout: "handle_inode_file_caps": 1, 2024-05-20T16:25:58.404 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph_cap_op_revoke": 17, 2024-05-20T16:25:58.404 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph_cap_op_grant": 3192, 2024-05-20T16:25:58.404 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph_cap_op_trunc": 0, 2024-05-20T16:25:58.404 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-20T16:25:58.404 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-20T16:25:58.404 INFO:teuthology.orchestra.run.smithi160.stdout: "handle_client_caps": 543676, 2024-05-20T16:25:58.404 INFO:teuthology.orchestra.run.smithi160.stdout: "handle_client_caps_dirty": 543646, 2024-05-20T16:25:58.404 INFO:teuthology.orchestra.run.smithi160.stdout: "handle_client_cap_release": 2, 2024-05-20T16:25:58.404 INFO:teuthology.orchestra.run.smithi160.stdout: "process_request_cap_release": 49 2024-05-20T16:25:58.404 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-20T16:25:58.405 INFO:teuthology.orchestra.run.smithi160.stdout: "mds_cache": { 2024-05-20T16:25:58.405 INFO:teuthology.orchestra.run.smithi160.stdout: "dir_update": 0, 2024-05-20T16:25:58.405 INFO:teuthology.orchestra.run.smithi160.stdout: "dir_update_receipt": 0, 2024-05-20T16:25:58.405 INFO:teuthology.orchestra.run.smithi160.stdout: "dir_try_discover": 0, 2024-05-20T16:25:58.405 INFO:teuthology.orchestra.run.smithi160.stdout: "dir_send_discover": 0, 2024-05-20T16:25:58.405 INFO:teuthology.orchestra.run.smithi160.stdout: "dir_handle_discover": 5, 2024-05-20T16:25:58.405 INFO:teuthology.orchestra.run.smithi160.stdout: "num_strays": 8, 2024-05-20T16:25:58.405 INFO:teuthology.orchestra.run.smithi160.stdout: "num_strays_delayed": 0, 2024-05-20T16:25:58.405 INFO:teuthology.orchestra.run.smithi160.stdout: "num_strays_enqueuing": 0, 2024-05-20T16:25:58.405 INFO:teuthology.orchestra.run.smithi160.stdout: "strays_created": 22, 2024-05-20T16:25:58.405 INFO:teuthology.orchestra.run.smithi160.stdout: "strays_enqueued": 14, 2024-05-20T16:25:58.405 INFO:teuthology.orchestra.run.smithi160.stdout: "strays_reintegrated": 0, 2024-05-20T16:25:58.405 INFO:teuthology.orchestra.run.smithi160.stdout: "strays_migrated": 0, 2024-05-20T16:25:58.406 INFO:teuthology.orchestra.run.smithi160.stdout: "num_recovering_processing": 0, 2024-05-20T16:25:58.406 INFO:teuthology.orchestra.run.smithi160.stdout: "num_recovering_enqueued": 0, 2024-05-20T16:25:58.406 INFO:teuthology.orchestra.run.smithi160.stdout: "num_recovering_prioritized": 0, 2024-05-20T16:25:58.406 INFO:teuthology.orchestra.run.smithi160.stdout: "recovery_started": 0, 2024-05-20T16:25:58.406 INFO:teuthology.orchestra.run.smithi160.stdout: "recovery_completed": 0, 2024-05-20T16:25:58.406 INFO:teuthology.orchestra.run.smithi160.stdout: "ireq_quiesce_path": 0, 2024-05-20T16:25:58.406 INFO:teuthology.orchestra.run.smithi160.stdout: "ireq_quiesce_inode": 0, 2024-05-20T16:25:58.406 INFO:teuthology.orchestra.run.smithi160.stdout: "ireq_enqueue_scrub": 0, 2024-05-20T16:25:58.406 INFO:teuthology.orchestra.run.smithi160.stdout: "ireq_exportdir": 0, 2024-05-20T16:25:58.406 INFO:teuthology.orchestra.run.smithi160.stdout: "ireq_flush": 0, 2024-05-20T16:25:58.406 INFO:teuthology.orchestra.run.smithi160.stdout: "ireq_fragmentdir": 2, 2024-05-20T16:25:58.406 INFO:teuthology.orchestra.run.smithi160.stdout: "ireq_fragstats": 0, 2024-05-20T16:25:58.407 INFO:teuthology.orchestra.run.smithi160.stdout: "ireq_inodestats": 0 2024-05-20T16:25:58.407 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-20T16:25:58.407 INFO:teuthology.orchestra.run.smithi160.stdout: "mds_log": { 2024-05-20T16:25:58.407 INFO:teuthology.orchestra.run.smithi160.stdout: "evlrg": 0, 2024-05-20T16:25:58.407 INFO:teuthology.orchestra.run.smithi160.stdout: "evadd": 514943, 2024-05-20T16:25:58.407 INFO:teuthology.orchestra.run.smithi160.stdout: "evex": 507046, 2024-05-20T16:25:58.407 INFO:teuthology.orchestra.run.smithi160.stdout: "evtrm": 506307, 2024-05-20T16:25:58.407 INFO:teuthology.orchestra.run.smithi160.stdout: "ev": 8636, 2024-05-20T16:25:58.407 INFO:teuthology.orchestra.run.smithi160.stdout: "evexg": 0, 2024-05-20T16:25:58.407 INFO:teuthology.orchestra.run.smithi160.stdout: "evexd": 739, 2024-05-20T16:25:58.407 INFO:teuthology.orchestra.run.smithi160.stdout: "segadd": 593, 2024-05-20T16:25:58.407 INFO:teuthology.orchestra.run.smithi160.stdout: "segex": 585, 2024-05-20T16:25:58.407 INFO:teuthology.orchestra.run.smithi160.stdout: "segtrm": 584, 2024-05-20T16:25:58.407 INFO:teuthology.orchestra.run.smithi160.stdout: "seg": 9, 2024-05-20T16:25:58.407 INFO:teuthology.orchestra.run.smithi160.stdout: "segmjr": 1, 2024-05-20T16:25:58.408 INFO:teuthology.orchestra.run.smithi160.stdout: "segexg": 0, 2024-05-20T16:25:58.408 INFO:teuthology.orchestra.run.smithi160.stdout: "segexd": 1, 2024-05-20T16:25:58.408 INFO:teuthology.orchestra.run.smithi160.stdout: "expos": 1051205280, 2024-05-20T16:25:58.408 INFO:teuthology.orchestra.run.smithi160.stdout: "wrpos": 1069457242, 2024-05-20T16:25:58.408 INFO:teuthology.orchestra.run.smithi160.stdout: "rdpos": 0, 2024-05-20T16:25:58.408 INFO:teuthology.orchestra.run.smithi160.stdout: "jlat": { 2024-05-20T16:25:58.408 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 5130, 2024-05-20T16:25:58.408 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 1851.979038339, 2024-05-20T16:25:58.408 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.361009559 2024-05-20T16:25:58.408 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-20T16:25:58.408 INFO:teuthology.orchestra.run.smithi160.stdout: "replayed": 0 2024-05-20T16:25:58.408 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-20T16:25:58.408 INFO:teuthology.orchestra.run.smithi160.stdout: "mds_mem": { 2024-05-20T16:25:58.409 INFO:teuthology.orchestra.run.smithi160.stdout: "ino": 31, 2024-05-20T16:25:58.409 INFO:teuthology.orchestra.run.smithi160.stdout: "ino+": 45, 2024-05-20T16:25:58.409 INFO:teuthology.orchestra.run.smithi160.stdout: "ino-": 14, 2024-05-20T16:25:58.409 INFO:teuthology.orchestra.run.smithi160.stdout: "dir": 18, 2024-05-20T16:25:58.409 INFO:teuthology.orchestra.run.smithi160.stdout: "dir+": 31, 2024-05-20T16:25:58.409 INFO:teuthology.orchestra.run.smithi160.stdout: "dir-": 13, 2024-05-20T16:25:58.409 INFO:teuthology.orchestra.run.smithi160.stdout: "dn": 32, 2024-05-20T16:25:58.409 INFO:teuthology.orchestra.run.smithi160.stdout: "dn+": 67, 2024-05-20T16:25:58.409 INFO:teuthology.orchestra.run.smithi160.stdout: "dn-": 35, 2024-05-20T16:25:58.409 INFO:teuthology.orchestra.run.smithi160.stdout: "cap": 10, 2024-05-20T16:25:58.409 INFO:teuthology.orchestra.run.smithi160.stdout: "cap+": 37, 2024-05-20T16:25:58.409 INFO:teuthology.orchestra.run.smithi160.stdout: "cap-": 27, 2024-05-20T16:25:58.409 INFO:teuthology.orchestra.run.smithi160.stdout: "rss": 150776, 2024-05-20T16:25:58.409 INFO:teuthology.orchestra.run.smithi160.stdout: "heap": 231828 2024-05-20T16:25:58.410 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-20T16:25:58.410 INFO:teuthology.orchestra.run.smithi160.stdout: "mds_server": { 2024-05-20T16:25:58.410 INFO:teuthology.orchestra.run.smithi160.stdout: "dispatch_client_request": 129, 2024-05-20T16:25:58.410 INFO:teuthology.orchestra.run.smithi160.stdout: "dispatch_server_request": 1, 2024-05-20T16:25:58.410 INFO:teuthology.orchestra.run.smithi160.stdout: "handle_client_request": 102, 2024-05-20T16:25:58.410 INFO:teuthology.orchestra.run.smithi160.stdout: "handle_client_session": 24, 2024-05-20T16:25:58.410 INFO:teuthology.orchestra.run.smithi160.stdout: "handle_peer_request": 2, 2024-05-20T16:25:58.410 INFO:teuthology.orchestra.run.smithi160.stdout: "req_create_latency": { 2024-05-20T16:25:58.410 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 22, 2024-05-20T16:25:58.410 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.028257245, 2024-05-20T16:25:58.411 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.001284420 2024-05-20T16:25:58.411 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-20T16:25:58.411 INFO:teuthology.orchestra.run.smithi160.stdout: "req_getattr_latency": { 2024-05-20T16:25:58.411 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 27, 2024-05-20T16:25:58.411 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.005289985, 2024-05-20T16:25:58.411 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000195925 2024-05-20T16:25:58.411 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-20T16:25:58.411 INFO:teuthology.orchestra.run.smithi160.stdout: "req_getfilelock_latency": { 2024-05-20T16:25:58.411 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-20T16:25:58.411 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-20T16:25:58.411 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-20T16:25:58.411 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-20T16:25:58.411 INFO:teuthology.orchestra.run.smithi160.stdout: "req_link_latency": { 2024-05-20T16:25:58.412 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-20T16:25:58.412 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-20T16:25:58.412 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-20T16:25:58.412 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-20T16:25:58.412 INFO:teuthology.orchestra.run.smithi160.stdout: "req_lookup_latency": { 2024-05-20T16:25:58.412 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 1, 2024-05-20T16:25:58.412 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000329711, 2024-05-20T16:25:58.412 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000329711 2024-05-20T16:25:58.412 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-20T16:25:58.412 INFO:teuthology.orchestra.run.smithi160.stdout: "req_lookuphash_latency": { 2024-05-20T16:25:58.412 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-20T16:25:58.412 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-20T16:25:58.413 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-20T16:25:58.413 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-20T16:25:58.413 INFO:teuthology.orchestra.run.smithi160.stdout: "req_lookupino_latency": { 2024-05-20T16:25:58.413 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 1, 2024-05-20T16:25:58.413 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000215900, 2024-05-20T16:25:58.413 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000215900 2024-05-20T16:25:58.413 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-20T16:25:58.413 INFO:teuthology.orchestra.run.smithi160.stdout: "req_lookupname_latency": { 2024-05-20T16:25:58.413 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-20T16:25:58.413 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-20T16:25:58.413 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-20T16:25:58.413 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-20T16:25:58.413 INFO:teuthology.orchestra.run.smithi160.stdout: "req_lookupparent_latency": { 2024-05-20T16:25:58.414 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-20T16:25:58.414 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-20T16:25:58.414 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-20T16:25:58.414 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-20T16:25:58.414 INFO:teuthology.orchestra.run.smithi160.stdout: "req_lookupsnap_latency": { 2024-05-20T16:25:58.414 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-20T16:25:58.414 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-20T16:25:58.414 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-20T16:25:58.414 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-20T16:25:58.414 INFO:teuthology.orchestra.run.smithi160.stdout: "req_lssnap_latency": { 2024-05-20T16:25:58.414 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-20T16:25:58.415 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-20T16:25:58.415 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-20T16:25:58.415 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-20T16:25:58.415 INFO:teuthology.orchestra.run.smithi160.stdout: "req_mkdir_latency": { 2024-05-20T16:25:58.415 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 10, 2024-05-20T16:25:58.415 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.015387030, 2024-05-20T16:25:58.415 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.001538703 2024-05-20T16:25:58.415 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-20T16:25:58.415 INFO:teuthology.orchestra.run.smithi160.stdout: "req_mknod_latency": { 2024-05-20T16:25:58.416 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-20T16:25:58.416 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-20T16:25:58.416 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-20T16:25:58.416 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-20T16:25:58.416 INFO:teuthology.orchestra.run.smithi160.stdout: "req_mksnap_latency": { 2024-05-20T16:25:58.416 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-20T16:25:58.416 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-20T16:25:58.416 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-20T16:25:58.416 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-20T16:25:58.416 INFO:teuthology.orchestra.run.smithi160.stdout: "req_open_latency": { 2024-05-20T16:25:58.416 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 1, 2024-05-20T16:25:58.416 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000202399, 2024-05-20T16:25:58.416 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000202399 2024-05-20T16:25:58.417 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-20T16:25:58.417 INFO:teuthology.orchestra.run.smithi160.stdout: "req_readdir_latency": { 2024-05-20T16:25:58.417 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 5, 2024-05-20T16:25:58.417 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.012008060, 2024-05-20T16:25:58.417 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.002401612 2024-05-20T16:25:58.417 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-20T16:25:58.417 INFO:teuthology.orchestra.run.smithi160.stdout: "req_rename_latency": { 2024-05-20T16:25:58.417 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 2, 2024-05-20T16:25:58.417 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.002073748, 2024-05-20T16:25:58.417 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.001036874 2024-05-20T16:25:58.417 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-20T16:25:58.417 INFO:teuthology.orchestra.run.smithi160.stdout: "req_renamesnap_latency": { 2024-05-20T16:25:58.418 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-20T16:25:58.418 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-20T16:25:58.418 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-20T16:25:58.418 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-20T16:25:58.418 INFO:teuthology.orchestra.run.smithi160.stdout: "req_snapdiff_latency": { 2024-05-20T16:25:58.418 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-20T16:25:58.418 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-20T16:25:58.418 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-20T16:25:58.418 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-20T16:25:58.418 INFO:teuthology.orchestra.run.smithi160.stdout: "req_rmdir_latency": { 2024-05-20T16:25:58.418 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 4, 2024-05-20T16:25:58.418 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.328449432, 2024-05-20T16:25:58.418 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.082112358 2024-05-20T16:25:58.419 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-20T16:25:58.419 INFO:teuthology.orchestra.run.smithi160.stdout: "req_rmsnap_latency": { 2024-05-20T16:25:58.419 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-20T16:25:58.419 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-20T16:25:58.419 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-20T16:25:58.419 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-20T16:25:58.419 INFO:teuthology.orchestra.run.smithi160.stdout: "req_rmxattr_latency": { 2024-05-20T16:25:58.419 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-20T16:25:58.419 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-20T16:25:58.419 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-20T16:25:58.419 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-20T16:25:58.419 INFO:teuthology.orchestra.run.smithi160.stdout: "req_setattr_latency": { 2024-05-20T16:25:58.419 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 2, 2024-05-20T16:25:58.419 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.001521926, 2024-05-20T16:25:58.420 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000760963 2024-05-20T16:25:58.420 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-20T16:25:58.420 INFO:teuthology.orchestra.run.smithi160.stdout: "req_setdirlayout_latency": { 2024-05-20T16:25:58.420 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-20T16:25:58.420 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-20T16:25:58.420 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-20T16:25:58.420 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-20T16:25:58.420 INFO:teuthology.orchestra.run.smithi160.stdout: "req_setfilelock_latency": { 2024-05-20T16:25:58.420 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-20T16:25:58.420 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-20T16:25:58.420 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-20T16:25:58.420 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-20T16:25:58.420 INFO:teuthology.orchestra.run.smithi160.stdout: "req_setlayout_latency": { 2024-05-20T16:25:58.420 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-20T16:25:58.421 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-20T16:25:58.421 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-20T16:25:58.421 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-20T16:25:58.421 INFO:teuthology.orchestra.run.smithi160.stdout: "req_setxattr_latency": { 2024-05-20T16:25:58.421 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 9, 2024-05-20T16:25:58.421 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.023340825, 2024-05-20T16:25:58.421 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.002593425 2024-05-20T16:25:58.421 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-20T16:25:58.421 INFO:teuthology.orchestra.run.smithi160.stdout: "req_symlink_latency": { 2024-05-20T16:25:58.421 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-20T16:25:58.421 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-20T16:25:58.421 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-20T16:25:58.421 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-20T16:25:58.421 INFO:teuthology.orchestra.run.smithi160.stdout: "req_unlink_latency": { 2024-05-20T16:25:58.422 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 18, 2024-05-20T16:25:58.422 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.023994261, 2024-05-20T16:25:58.422 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.001333014 2024-05-20T16:25:58.422 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-20T16:25:58.422 INFO:teuthology.orchestra.run.smithi160.stdout: "cap_revoke_eviction": 0, 2024-05-20T16:25:58.422 INFO:teuthology.orchestra.run.smithi160.stdout: "cap_acquisition_throttle": 0, 2024-05-20T16:25:58.422 INFO:teuthology.orchestra.run.smithi160.stdout: "req_getvxattr_latency": { 2024-05-20T16:25:58.422 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-20T16:25:58.422 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-20T16:25:58.422 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-20T16:25:58.422 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-05-20T16:25:58.422 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-20T16:25:58.422 INFO:teuthology.orchestra.run.smithi160.stdout: "mds_sessions": { 2024-05-20T16:25:58.423 INFO:teuthology.orchestra.run.smithi160.stdout: "session_count": 1, 2024-05-20T16:25:58.423 INFO:teuthology.orchestra.run.smithi160.stdout: "session_add": 4, 2024-05-20T16:25:58.423 INFO:teuthology.orchestra.run.smithi160.stdout: "session_remove": 3, 2024-05-20T16:25:58.423 INFO:teuthology.orchestra.run.smithi160.stdout: "sessions_open": 1, 2024-05-20T16:25:58.423 INFO:teuthology.orchestra.run.smithi160.stdout: "sessions_stale": 0, 2024-05-20T16:25:58.423 INFO:teuthology.orchestra.run.smithi160.stdout: "total_load": 26, 2024-05-20T16:25:58.423 INFO:teuthology.orchestra.run.smithi160.stdout: "average_load": 26, 2024-05-20T16:25:58.423 INFO:teuthology.orchestra.run.smithi160.stdout: "avg_session_uptime": 708, 2024-05-20T16:25:58.423 INFO:teuthology.orchestra.run.smithi160.stdout: "mdthresh_evicted": 0 2024-05-20T16:25:58.423 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-20T16:25:58.423 INFO:teuthology.orchestra.run.smithi160.stdout: "mempool": { 2024-05-20T16:25:58.423 INFO:teuthology.orchestra.run.smithi160.stdout: "bloom_filter_bytes": 0, 2024-05-20T16:25:58.423 INFO:teuthology.orchestra.run.smithi160.stdout: "bloom_filter_items": 0, 2024-05-20T16:25:58.424 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_alloc_bytes": 0, 2024-05-20T16:25:58.424 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_alloc_items": 0, 2024-05-20T16:25:58.424 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_data_bytes": 0, 2024-05-20T16:25:58.424 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_data_items": 0, 2024-05-20T16:25:58.424 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-20T16:25:58.424 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_onode_items": 0, 2024-05-20T16:25:58.424 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-20T16:25:58.424 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_meta_items": 0, 2024-05-20T16:25:58.424 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_other_bytes": 0, 2024-05-20T16:25:58.424 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_other_items": 0, 2024-05-20T16:25:58.424 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-20T16:25:58.424 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_buffer_items": 0, 2024-05-20T16:25:58.424 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_extent_bytes": 0, 2024-05-20T16:25:58.424 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_extent_items": 0, 2024-05-20T16:25:58.424 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_blob_bytes": 0, 2024-05-20T16:25:58.425 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_blob_items": 0, 2024-05-20T16:25:58.425 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-20T16:25:58.425 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_shared_blob_items": 0, 2024-05-20T16:25:58.425 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-20T16:25:58.425 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_inline_bl_items": 0, 2024-05-20T16:25:58.425 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_fsck_bytes": 0, 2024-05-20T16:25:58.425 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_fsck_items": 0, 2024-05-20T16:25:58.425 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_txc_bytes": 0, 2024-05-20T16:25:58.425 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_txc_items": 0, 2024-05-20T16:25:58.425 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-20T16:25:58.425 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_writing_deferred_items": 0, 2024-05-20T16:25:58.425 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_writing_bytes": 0, 2024-05-20T16:25:58.425 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_writing_items": 0, 2024-05-20T16:25:58.425 INFO:teuthology.orchestra.run.smithi160.stdout: "bluefs_bytes": 0, 2024-05-20T16:25:58.425 INFO:teuthology.orchestra.run.smithi160.stdout: "bluefs_items": 0, 2024-05-20T16:25:58.426 INFO:teuthology.orchestra.run.smithi160.stdout: "bluefs_file_reader_bytes": 0, 2024-05-20T16:25:58.426 INFO:teuthology.orchestra.run.smithi160.stdout: "bluefs_file_reader_items": 0, 2024-05-20T16:25:58.426 INFO:teuthology.orchestra.run.smithi160.stdout: "bluefs_file_writer_bytes": 0, 2024-05-20T16:25:58.426 INFO:teuthology.orchestra.run.smithi160.stdout: "bluefs_file_writer_items": 0, 2024-05-20T16:25:58.426 INFO:teuthology.orchestra.run.smithi160.stdout: "buffer_anon_bytes": 306758, 2024-05-20T16:25:58.426 INFO:teuthology.orchestra.run.smithi160.stdout: "buffer_anon_items": 128, 2024-05-20T16:25:58.426 INFO:teuthology.orchestra.run.smithi160.stdout: "buffer_meta_bytes": 0, 2024-05-20T16:25:58.426 INFO:teuthology.orchestra.run.smithi160.stdout: "buffer_meta_items": 0, 2024-05-20T16:25:58.426 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_bytes": 0, 2024-05-20T16:25:58.426 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_items": 0, 2024-05-20T16:25:58.426 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_mapbl_bytes": 0, 2024-05-20T16:25:58.426 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_mapbl_items": 0, 2024-05-20T16:25:58.426 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_pglog_bytes": 0, 2024-05-20T16:25:58.426 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_pglog_items": 0, 2024-05-20T16:25:58.427 INFO:teuthology.orchestra.run.smithi160.stdout: "osdmap_bytes": 8888, 2024-05-20T16:25:58.427 INFO:teuthology.orchestra.run.smithi160.stdout: "osdmap_items": 170, 2024-05-20T16:25:58.427 INFO:teuthology.orchestra.run.smithi160.stdout: "osdmap_mapping_bytes": 0, 2024-05-20T16:25:58.427 INFO:teuthology.orchestra.run.smithi160.stdout: "osdmap_mapping_items": 0, 2024-05-20T16:25:58.427 INFO:teuthology.orchestra.run.smithi160.stdout: "pgmap_bytes": 0, 2024-05-20T16:25:58.427 INFO:teuthology.orchestra.run.smithi160.stdout: "pgmap_items": 0, 2024-05-20T16:25:58.427 INFO:teuthology.orchestra.run.smithi160.stdout: "mds_co_bytes": 123133, 2024-05-20T16:25:58.427 INFO:teuthology.orchestra.run.smithi160.stdout: "mds_co_items": 1686, 2024-05-20T16:25:58.427 INFO:teuthology.orchestra.run.smithi160.stdout: "unittest_1_bytes": 0, 2024-05-20T16:25:58.427 INFO:teuthology.orchestra.run.smithi160.stdout: "unittest_1_items": 0, 2024-05-20T16:25:58.427 INFO:teuthology.orchestra.run.smithi160.stdout: "unittest_2_bytes": 0, 2024-05-20T16:25:58.427 INFO:teuthology.orchestra.run.smithi160.stdout: "unittest_2_items": 0 2024-05-20T16:25:58.427 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-20T16:25:58.427 INFO:teuthology.orchestra.run.smithi160.stdout: "objecter": { 2024-05-20T16:25:58.428 INFO:teuthology.orchestra.run.smithi160.stdout: "op_active": 0, 2024-05-20T16:25:58.428 INFO:teuthology.orchestra.run.smithi160.stdout: "op_laggy": 0, 2024-05-20T16:25:58.428 INFO:teuthology.orchestra.run.smithi160.stdout: "op_send": 6963, 2024-05-20T16:25:58.428 INFO:teuthology.orchestra.run.smithi160.stdout: "op_send_bytes": 1065803022, 2024-05-20T16:25:58.428 INFO:teuthology.orchestra.run.smithi160.stdout: "op_resend": 37, 2024-05-20T16:25:58.428 INFO:teuthology.orchestra.run.smithi160.stdout: "op_reply": 6963, 2024-05-20T16:25:58.428 INFO:teuthology.orchestra.run.smithi160.stdout: "op_latency": { 2024-05-20T16:25:58.428 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 6963, 2024-05-20T16:25:58.428 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 640.699767505, 2024-05-20T16:25:58.428 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.092014902 2024-05-20T16:25:58.428 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-20T16:25:58.428 INFO:teuthology.orchestra.run.smithi160.stdout: "op_inflight": 0, 2024-05-20T16:25:58.428 INFO:teuthology.orchestra.run.smithi160.stdout: "oplen_avg": { 2024-05-20T16:25:58.428 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 6963, 2024-05-20T16:25:58.428 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 7302 2024-05-20T16:25:58.429 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-20T16:25:58.429 INFO:teuthology.orchestra.run.smithi160.stdout: "op": 6963, 2024-05-20T16:25:58.429 INFO:teuthology.orchestra.run.smithi160.stdout: "op_r": 2, 2024-05-20T16:25:58.429 INFO:teuthology.orchestra.run.smithi160.stdout: "op_w": 6961, 2024-05-20T16:25:58.429 INFO:teuthology.orchestra.run.smithi160.stdout: "op_rmw": 0, 2024-05-20T16:25:58.429 INFO:teuthology.orchestra.run.smithi160.stdout: "op_pg": 0, 2024-05-20T16:25:58.429 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_stat": 92, 2024-05-20T16:25:58.429 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_create": 66, 2024-05-20T16:25:58.429 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_read": 2, 2024-05-20T16:25:58.429 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_write": 5132, 2024-05-20T16:25:58.429 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_writefull": 142, 2024-05-20T16:25:58.429 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_writesame": 0, 2024-05-20T16:25:58.429 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_append": 0, 2024-05-20T16:25:58.429 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_zero": 0, 2024-05-20T16:25:58.430 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_truncate": 1, 2024-05-20T16:25:58.430 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_delete": 1046, 2024-05-20T16:25:58.430 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_mapext": 0, 2024-05-20T16:25:58.430 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_sparse_read": 0, 2024-05-20T16:25:58.430 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_clonerange": 0, 2024-05-20T16:25:58.430 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_getxattr": 0, 2024-05-20T16:25:58.430 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_setxattr": 110, 2024-05-20T16:25:58.430 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_cmpxattr": 0, 2024-05-20T16:25:58.430 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_rmxattr": 0, 2024-05-20T16:25:58.430 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_resetxattrs": 0, 2024-05-20T16:25:58.430 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_call": 0, 2024-05-20T16:25:58.430 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_watch": 0, 2024-05-20T16:25:58.430 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_notify": 0, 2024-05-20T16:25:58.430 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_src_cmpxattr": 0, 2024-05-20T16:25:58.431 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_pgls": 0, 2024-05-20T16:25:58.431 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_pgls_filter": 0, 2024-05-20T16:25:58.431 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_other": 0, 2024-05-20T16:25:58.431 INFO:teuthology.orchestra.run.smithi160.stdout: "linger_active": 0, 2024-05-20T16:25:58.431 INFO:teuthology.orchestra.run.smithi160.stdout: "linger_send": 0, 2024-05-20T16:25:58.431 INFO:teuthology.orchestra.run.smithi160.stdout: "linger_resend": 0, 2024-05-20T16:25:58.431 INFO:teuthology.orchestra.run.smithi160.stdout: "linger_ping": 0, 2024-05-20T16:25:58.431 INFO:teuthology.orchestra.run.smithi160.stdout: "poolop_active": 0, 2024-05-20T16:25:58.431 INFO:teuthology.orchestra.run.smithi160.stdout: "poolop_send": 0, 2024-05-20T16:25:58.431 INFO:teuthology.orchestra.run.smithi160.stdout: "poolop_resend": 0, 2024-05-20T16:25:58.431 INFO:teuthology.orchestra.run.smithi160.stdout: "poolstat_active": 0, 2024-05-20T16:25:58.431 INFO:teuthology.orchestra.run.smithi160.stdout: "poolstat_send": 0, 2024-05-20T16:25:58.431 INFO:teuthology.orchestra.run.smithi160.stdout: "poolstat_resend": 0, 2024-05-20T16:25:58.431 INFO:teuthology.orchestra.run.smithi160.stdout: "statfs_active": 0, 2024-05-20T16:25:58.432 INFO:teuthology.orchestra.run.smithi160.stdout: "statfs_send": 0, 2024-05-20T16:25:58.432 INFO:teuthology.orchestra.run.smithi160.stdout: "statfs_resend": 0, 2024-05-20T16:25:58.432 INFO:teuthology.orchestra.run.smithi160.stdout: "command_active": 0, 2024-05-20T16:25:58.432 INFO:teuthology.orchestra.run.smithi160.stdout: "command_send": 0, 2024-05-20T16:25:58.432 INFO:teuthology.orchestra.run.smithi160.stdout: "command_resend": 0, 2024-05-20T16:25:58.432 INFO:teuthology.orchestra.run.smithi160.stdout: "map_epoch": 98, 2024-05-20T16:25:58.432 INFO:teuthology.orchestra.run.smithi160.stdout: "map_full": 0, 2024-05-20T16:25:58.432 INFO:teuthology.orchestra.run.smithi160.stdout: "map_inc": 13, 2024-05-20T16:25:58.432 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_sessions": 12, 2024-05-20T16:25:58.432 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_session_open": 12, 2024-05-20T16:25:58.432 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_session_close": 0, 2024-05-20T16:25:58.432 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_laggy": 0, 2024-05-20T16:25:58.433 INFO:teuthology.orchestra.run.smithi160.stdout: "omap_wr": 698, 2024-05-20T16:25:58.433 INFO:teuthology.orchestra.run.smithi160.stdout: "omap_rd": 0, 2024-05-20T16:25:58.433 INFO:teuthology.orchestra.run.smithi160.stdout: "omap_del": 13 2024-05-20T16:25:58.433 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-20T16:25:58.433 INFO:teuthology.orchestra.run.smithi160.stdout: "oft": { 2024-05-20T16:25:58.433 INFO:teuthology.orchestra.run.smithi160.stdout: "omap_total_objs": 1, 2024-05-20T16:25:58.433 INFO:teuthology.orchestra.run.smithi160.stdout: "omap_total_kv_pairs": 0, 2024-05-20T16:25:58.433 INFO:teuthology.orchestra.run.smithi160.stdout: "omap_total_updates": 0, 2024-05-20T16:25:58.433 INFO:teuthology.orchestra.run.smithi160.stdout: "omap_total_removes": 0 2024-05-20T16:25:58.433 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-20T16:25:58.433 INFO:teuthology.orchestra.run.smithi160.stdout: "osd-slow-ops": { 2024-05-20T16:25:58.433 INFO:teuthology.orchestra.run.smithi160.stdout: "slow_ops_count": 2 2024-05-20T16:25:58.433 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-20T16:25:58.433 INFO:teuthology.orchestra.run.smithi160.stdout: "purge_queue": { 2024-05-20T16:25:58.433 INFO:teuthology.orchestra.run.smithi160.stdout: "pq_executing_ops": 0, 2024-05-20T16:25:58.434 INFO:teuthology.orchestra.run.smithi160.stdout: "pq_executing_ops_high_water": 17, 2024-05-20T16:25:58.434 INFO:teuthology.orchestra.run.smithi160.stdout: "pq_executing": 0, 2024-05-20T16:25:58.434 INFO:teuthology.orchestra.run.smithi160.stdout: "pq_executing_high_water": 9, 2024-05-20T16:25:58.434 INFO:teuthology.orchestra.run.smithi160.stdout: "pq_executed_ops": 24, 2024-05-20T16:25:58.434 INFO:teuthology.orchestra.run.smithi160.stdout: "pq_executed": 14, 2024-05-20T16:25:58.434 INFO:teuthology.orchestra.run.smithi160.stdout: "pq_item_in_journal": 0 2024-05-20T16:25:58.434 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-20T16:25:58.434 INFO:teuthology.orchestra.run.smithi160.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-20T16:25:58.434 INFO:teuthology.orchestra.run.smithi160.stdout: "val": 0, 2024-05-20T16:25:58.434 INFO:teuthology.orchestra.run.smithi160.stdout: "max": 104857600, 2024-05-20T16:25:58.434 INFO:teuthology.orchestra.run.smithi160.stdout: "get_started": 0, 2024-05-20T16:25:58.434 INFO:teuthology.orchestra.run.smithi160.stdout: "get": 523417, 2024-05-20T16:25:58.434 INFO:teuthology.orchestra.run.smithi160.stdout: "get_sum": 136059107, 2024-05-20T16:25:58.434 INFO:teuthology.orchestra.run.smithi160.stdout: "get_or_fail_fail": 0, 2024-05-20T16:25:58.435 INFO:teuthology.orchestra.run.smithi160.stdout: "get_or_fail_success": 523417, 2024-05-20T16:25:58.435 INFO:teuthology.orchestra.run.smithi160.stdout: "take": 0, 2024-05-20T16:25:58.435 INFO:teuthology.orchestra.run.smithi160.stdout: "take_sum": 0, 2024-05-20T16:25:58.435 INFO:teuthology.orchestra.run.smithi160.stdout: "put": 523417, 2024-05-20T16:25:58.435 INFO:teuthology.orchestra.run.smithi160.stdout: "put_sum": 136059107, 2024-05-20T16:25:58.435 INFO:teuthology.orchestra.run.smithi160.stdout: "wait": { 2024-05-20T16:25:58.435 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-20T16:25:58.435 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-20T16:25:58.435 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-20T16:25:58.435 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-05-20T16:25:58.435 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-20T16:25:58.435 INFO:teuthology.orchestra.run.smithi160.stdout: "throttle-objecter_bytes": { 2024-05-20T16:25:58.435 INFO:teuthology.orchestra.run.smithi160.stdout: "val": 0, 2024-05-20T16:25:58.435 INFO:teuthology.orchestra.run.smithi160.stdout: "max": 104857600, 2024-05-20T16:25:58.436 INFO:teuthology.orchestra.run.smithi160.stdout: "get_started": 0, 2024-05-20T16:25:58.436 INFO:teuthology.orchestra.run.smithi160.stdout: "get": 0, 2024-05-20T16:25:58.436 INFO:teuthology.orchestra.run.smithi160.stdout: "get_sum": 0, 2024-05-20T16:25:58.436 INFO:teuthology.orchestra.run.smithi160.stdout: "get_or_fail_fail": 0, 2024-05-20T16:25:58.436 INFO:teuthology.orchestra.run.smithi160.stdout: "get_or_fail_success": 0, 2024-05-20T16:25:58.436 INFO:teuthology.orchestra.run.smithi160.stdout: "take": 6963, 2024-05-20T16:25:58.436 INFO:teuthology.orchestra.run.smithi160.stdout: "take_sum": 1065804516, 2024-05-20T16:25:58.436 INFO:teuthology.orchestra.run.smithi160.stdout: "put": 6963, 2024-05-20T16:25:58.436 INFO:teuthology.orchestra.run.smithi160.stdout: "put_sum": 1065804516, 2024-05-20T16:25:58.436 INFO:teuthology.orchestra.run.smithi160.stdout: "wait": { 2024-05-20T16:25:58.436 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-20T16:25:58.436 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-20T16:25:58.436 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-20T16:25:58.437 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-05-20T16:25:58.437 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-20T16:25:58.437 INFO:teuthology.orchestra.run.smithi160.stdout: "throttle-objecter_ops": { 2024-05-20T16:25:58.437 INFO:teuthology.orchestra.run.smithi160.stdout: "val": 0, 2024-05-20T16:25:58.437 INFO:teuthology.orchestra.run.smithi160.stdout: "max": 1024, 2024-05-20T16:25:58.437 INFO:teuthology.orchestra.run.smithi160.stdout: "get_started": 0, 2024-05-20T16:25:58.437 INFO:teuthology.orchestra.run.smithi160.stdout: "get": 0, 2024-05-20T16:25:58.437 INFO:teuthology.orchestra.run.smithi160.stdout: "get_sum": 0, 2024-05-20T16:25:58.437 INFO:teuthology.orchestra.run.smithi160.stdout: "get_or_fail_fail": 0, 2024-05-20T16:25:58.437 INFO:teuthology.orchestra.run.smithi160.stdout: "get_or_fail_success": 0, 2024-05-20T16:25:58.437 INFO:teuthology.orchestra.run.smithi160.stdout: "take": 6963, 2024-05-20T16:25:58.437 INFO:teuthology.orchestra.run.smithi160.stdout: "take_sum": 6963, 2024-05-20T16:25:58.437 INFO:teuthology.orchestra.run.smithi160.stdout: "put": 6963, 2024-05-20T16:25:58.437 INFO:teuthology.orchestra.run.smithi160.stdout: "put_sum": 6963, 2024-05-20T16:25:58.438 INFO:teuthology.orchestra.run.smithi160.stdout: "wait": { 2024-05-20T16:25:58.438 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-20T16:25:58.438 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-20T16:25:58.438 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-20T16:25:58.438 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-05-20T16:25:58.438 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-20T16:25:58.438 INFO:teuthology.orchestra.run.smithi160.stdout: "throttle-write_buf_throttle": { 2024-05-20T16:25:58.438 INFO:teuthology.orchestra.run.smithi160.stdout: "val": 0, 2024-05-20T16:25:58.438 INFO:teuthology.orchestra.run.smithi160.stdout: "max": 3758096384, 2024-05-20T16:25:58.438 INFO:teuthology.orchestra.run.smithi160.stdout: "get_started": 0, 2024-05-20T16:25:58.438 INFO:teuthology.orchestra.run.smithi160.stdout: "get": 14, 2024-05-20T16:25:58.439 INFO:teuthology.orchestra.run.smithi160.stdout: "get_sum": 1494, 2024-05-20T16:25:58.439 INFO:teuthology.orchestra.run.smithi160.stdout: "get_or_fail_fail": 0, 2024-05-20T16:25:58.439 INFO:teuthology.orchestra.run.smithi160.stdout: "get_or_fail_success": 14, 2024-05-20T16:25:58.439 INFO:teuthology.orchestra.run.smithi160.stdout: "take": 0, 2024-05-20T16:25:58.439 INFO:teuthology.orchestra.run.smithi160.stdout: "take_sum": 0, 2024-05-20T16:25:58.439 INFO:teuthology.orchestra.run.smithi160.stdout: "put": 2, 2024-05-20T16:25:58.439 INFO:teuthology.orchestra.run.smithi160.stdout: "put_sum": 1494, 2024-05-20T16:25:58.439 INFO:teuthology.orchestra.run.smithi160.stdout: "wait": { 2024-05-20T16:25:58.439 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-20T16:25:58.439 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-20T16:25:58.439 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-20T16:25:58.439 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-05-20T16:25:58.440 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-20T16:25:58.440 INFO:teuthology.orchestra.run.smithi160.stdout: "throttle-write_buf_throttle-0x563b45c54b60": { 2024-05-20T16:25:58.440 INFO:teuthology.orchestra.run.smithi160.stdout: "val": 0, 2024-05-20T16:25:58.440 INFO:teuthology.orchestra.run.smithi160.stdout: "max": 3758096384, 2024-05-20T16:25:58.440 INFO:teuthology.orchestra.run.smithi160.stdout: "get_started": 0, 2024-05-20T16:25:58.440 INFO:teuthology.orchestra.run.smithi160.stdout: "get": 514943, 2024-05-20T16:25:58.440 INFO:teuthology.orchestra.run.smithi160.stdout: "get_sum": 1065262938, 2024-05-20T16:25:58.440 INFO:teuthology.orchestra.run.smithi160.stdout: "get_or_fail_fail": 0, 2024-05-20T16:25:58.440 INFO:teuthology.orchestra.run.smithi160.stdout: "get_or_fail_success": 514943, 2024-05-20T16:25:58.440 INFO:teuthology.orchestra.run.smithi160.stdout: "take": 0, 2024-05-20T16:25:58.440 INFO:teuthology.orchestra.run.smithi160.stdout: "take_sum": 0, 2024-05-20T16:25:58.441 INFO:teuthology.orchestra.run.smithi160.stdout: "put": 5130, 2024-05-20T16:25:58.441 INFO:teuthology.orchestra.run.smithi160.stdout: "put_sum": 1065262938, 2024-05-20T16:25:58.441 INFO:teuthology.orchestra.run.smithi160.stdout: "wait": { 2024-05-20T16:25:58.441 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-20T16:25:58.441 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-20T16:25:58.441 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-20T16:25:58.441 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-05-20T16:25:58.441 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-05-20T16:25:58.441 INFO:teuthology.orchestra.run.smithi160.stdout:} 2024-05-20T16:25:59.315 INFO:tasks.check_counter:Daemon mds.b mds.exported=0 2024-05-20T16:25:59.315 INFO:tasks.check_counter:Daemon mds.b mds.imported=0 2024-05-20T16:25:59.315 DEBUG:tasks.check_counter:Getting stats from e 2024-05-20T16:25:59.316 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell --fsid 59d1600a-16c2-11ef-bc9a-c7b262605968 -- ceph daemon mds.e perf dump 2024-05-20T16:25:59.562 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/mon.b/config 2024-05-20T16:26:01.207 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:26:00 smithi160 ceph-mon[36347]: pgmap v725: 129 pgs: 129 active+clean; 10 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 81 MiB/s wr, 149 op/s 2024-05-20T16:26:01.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:26:00 smithi082 ceph-mon[32238]: pgmap v725: 129 pgs: 129 active+clean; 10 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 81 MiB/s wr, 149 op/s 2024-05-20T16:26:01.370 INFO:teuthology.orchestra.run.smithi160.stdout:{ 2024-05-20T16:26:01.370 INFO:teuthology.orchestra.run.smithi160.stdout: "AsyncMessenger::Worker-0": { 2024-05-20T16:26:01.371 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_messages": 260, 2024-05-20T16:26:01.371 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_messages": 248, 2024-05-20T16:26:01.371 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_bytes": 806114, 2024-05-20T16:26:01.371 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_bytes": 147092, 2024-05-20T16:26:01.371 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_created_connections": 5, 2024-05-20T16:26:01.371 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_active_connections": 1, 2024-05-20T16:26:01.371 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_total_time": 0.045033918, 2024-05-20T16:26:01.371 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_send_time": 0.014337264, 2024-05-20T16:26:01.372 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_recv_time": 0.026245925, 2024-05-20T16:26:01.372 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_fast_dispatch_time": 0.007480961, 2024-05-20T16:26:01.372 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_messages_queue_lat": { 2024-05-20T16:26:01.372 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 247, 2024-05-20T16:26:01.372 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.018434166, 2024-05-20T16:26:01.372 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000074632 2024-05-20T16:26:01.372 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-20T16:26:01.372 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_handle_ack_lat": { 2024-05-20T16:26:01.372 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-20T16:26:01.373 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-20T16:26:01.373 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-20T16:26:01.373 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-20T16:26:01.373 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_encrypted_bytes": 805280, 2024-05-20T16:26:01.373 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_encrypted_bytes": 146768 2024-05-20T16:26:01.373 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-20T16:26:01.374 INFO:teuthology.orchestra.run.smithi160.stdout: "AsyncMessenger::Worker-1": { 2024-05-20T16:26:01.374 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_messages": 744, 2024-05-20T16:26:01.374 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_messages": 920, 2024-05-20T16:26:01.374 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_bytes": 185000, 2024-05-20T16:26:01.374 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_bytes": 622191, 2024-05-20T16:26:01.374 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_created_connections": 5, 2024-05-20T16:26:01.374 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_active_connections": 4, 2024-05-20T16:26:01.374 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_total_time": 0.098392220, 2024-05-20T16:26:01.374 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_send_time": 0.026157454, 2024-05-20T16:26:01.374 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_recv_time": 0.021923186, 2024-05-20T16:26:01.374 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_fast_dispatch_time": 0.038025996, 2024-05-20T16:26:01.374 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_messages_queue_lat": { 2024-05-20T16:26:01.375 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 919, 2024-05-20T16:26:01.375 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.051080399, 2024-05-20T16:26:01.375 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000055582 2024-05-20T16:26:01.375 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-20T16:26:01.375 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_handle_ack_lat": { 2024-05-20T16:26:01.375 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-20T16:26:01.375 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-20T16:26:01.375 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-20T16:26:01.375 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-20T16:26:01.375 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_encrypted_bytes": 3184, 2024-05-20T16:26:01.375 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_encrypted_bytes": 387184 2024-05-20T16:26:01.375 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-20T16:26:01.375 INFO:teuthology.orchestra.run.smithi160.stdout: "AsyncMessenger::Worker-2": { 2024-05-20T16:26:01.375 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_messages": 1604, 2024-05-20T16:26:01.375 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_messages": 1606, 2024-05-20T16:26:01.376 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_bytes": 480805, 2024-05-20T16:26:01.376 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_bytes": 510130, 2024-05-20T16:26:01.376 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_created_connections": 4, 2024-05-20T16:26:01.376 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_active_connections": 3, 2024-05-20T16:26:01.376 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_total_time": 0.174453885, 2024-05-20T16:26:01.376 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_send_time": 0.051858403, 2024-05-20T16:26:01.376 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_recv_time": 0.046856032, 2024-05-20T16:26:01.376 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_fast_dispatch_time": 0.063040612, 2024-05-20T16:26:01.376 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_messages_queue_lat": { 2024-05-20T16:26:01.376 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 1604, 2024-05-20T16:26:01.376 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.087097588, 2024-05-20T16:26:01.376 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000054300 2024-05-20T16:26:01.376 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-20T16:26:01.376 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_handle_ack_lat": { 2024-05-20T16:26:01.376 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-20T16:26:01.377 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-20T16:26:01.377 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-20T16:26:01.377 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-20T16:26:01.377 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-20T16:26:01.377 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_encrypted_bytes": 0 2024-05-20T16:26:01.377 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-20T16:26:01.377 INFO:teuthology.orchestra.run.smithi160.stdout: "cct": { 2024-05-20T16:26:01.377 INFO:teuthology.orchestra.run.smithi160.stdout: "total_workers": 1, 2024-05-20T16:26:01.377 INFO:teuthology.orchestra.run.smithi160.stdout: "unhealthy_workers": 0 2024-05-20T16:26:01.377 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-20T16:26:01.377 INFO:teuthology.orchestra.run.smithi160.stdout: "finisher-MDSRank": { 2024-05-20T16:26:01.377 INFO:teuthology.orchestra.run.smithi160.stdout: "queue_len": 0, 2024-05-20T16:26:01.377 INFO:teuthology.orchestra.run.smithi160.stdout: "complete_latency": { 2024-05-20T16:26:01.377 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 3920, 2024-05-20T16:26:01.378 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.143334829, 2024-05-20T16:26:01.378 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000036565 2024-05-20T16:26:01.378 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-05-20T16:26:01.378 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-20T16:26:01.378 INFO:teuthology.orchestra.run.smithi160.stdout: "finisher-PurgeQueue": { 2024-05-20T16:26:01.378 INFO:teuthology.orchestra.run.smithi160.stdout: "queue_len": 0, 2024-05-20T16:26:01.378 INFO:teuthology.orchestra.run.smithi160.stdout: "complete_latency": { 2024-05-20T16:26:01.378 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-20T16:26:01.378 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-20T16:26:01.378 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-20T16:26:01.378 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-05-20T16:26:01.378 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-20T16:26:01.379 INFO:teuthology.orchestra.run.smithi160.stdout: "mds": { 2024-05-20T16:26:01.379 INFO:teuthology.orchestra.run.smithi160.stdout: "request": 0, 2024-05-20T16:26:01.379 INFO:teuthology.orchestra.run.smithi160.stdout: "reply": 0, 2024-05-20T16:26:01.379 INFO:teuthology.orchestra.run.smithi160.stdout: "reply_latency": { 2024-05-20T16:26:01.379 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-20T16:26:01.379 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-20T16:26:01.379 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-20T16:26:01.379 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-20T16:26:01.379 INFO:teuthology.orchestra.run.smithi160.stdout: "slow_reply": 0, 2024-05-20T16:26:01.379 INFO:teuthology.orchestra.run.smithi160.stdout: "forward": 0, 2024-05-20T16:26:01.379 INFO:teuthology.orchestra.run.smithi160.stdout: "dir_fetch_complete": 0, 2024-05-20T16:26:01.379 INFO:teuthology.orchestra.run.smithi160.stdout: "dir_fetch_keys": 0, 2024-05-20T16:26:01.379 INFO:teuthology.orchestra.run.smithi160.stdout: "dir_commit": 0, 2024-05-20T16:26:01.379 INFO:teuthology.orchestra.run.smithi160.stdout: "dir_split": 0, 2024-05-20T16:26:01.379 INFO:teuthology.orchestra.run.smithi160.stdout: "dir_merge": 0, 2024-05-20T16:26:01.380 INFO:teuthology.orchestra.run.smithi160.stdout: "inodes": 0, 2024-05-20T16:26:01.380 INFO:teuthology.orchestra.run.smithi160.stdout: "inodes_top": 0, 2024-05-20T16:26:01.380 INFO:teuthology.orchestra.run.smithi160.stdout: "inodes_bottom": 0, 2024-05-20T16:26:01.380 INFO:teuthology.orchestra.run.smithi160.stdout: "inodes_pin_tail": 0, 2024-05-20T16:26:01.380 INFO:teuthology.orchestra.run.smithi160.stdout: "inodes_pinned": 0, 2024-05-20T16:26:01.380 INFO:teuthology.orchestra.run.smithi160.stdout: "inodes_expired": 0, 2024-05-20T16:26:01.380 INFO:teuthology.orchestra.run.smithi160.stdout: "inodes_with_caps": 0, 2024-05-20T16:26:01.380 INFO:teuthology.orchestra.run.smithi160.stdout: "caps": 0, 2024-05-20T16:26:01.380 INFO:teuthology.orchestra.run.smithi160.stdout: "subtrees": 1, 2024-05-20T16:26:01.380 INFO:teuthology.orchestra.run.smithi160.stdout: "traverse": 0, 2024-05-20T16:26:01.380 INFO:teuthology.orchestra.run.smithi160.stdout: "traverse_hit": 0, 2024-05-20T16:26:01.380 INFO:teuthology.orchestra.run.smithi160.stdout: "traverse_forward": 0, 2024-05-20T16:26:01.381 INFO:teuthology.orchestra.run.smithi160.stdout: "traverse_discover": 0, 2024-05-20T16:26:01.381 INFO:teuthology.orchestra.run.smithi160.stdout: "traverse_dir_fetch": 0, 2024-05-20T16:26:01.381 INFO:teuthology.orchestra.run.smithi160.stdout: "traverse_remote_ino": 0, 2024-05-20T16:26:01.381 INFO:teuthology.orchestra.run.smithi160.stdout: "traverse_lock": 0, 2024-05-20T16:26:01.381 INFO:teuthology.orchestra.run.smithi160.stdout: "load_cent": 0, 2024-05-20T16:26:01.381 INFO:teuthology.orchestra.run.smithi160.stdout: "q": 0, 2024-05-20T16:26:01.381 INFO:teuthology.orchestra.run.smithi160.stdout: "exported": 0, 2024-05-20T16:26:01.381 INFO:teuthology.orchestra.run.smithi160.stdout: "exported_inodes": 0, 2024-05-20T16:26:01.381 INFO:teuthology.orchestra.run.smithi160.stdout: "imported": 0, 2024-05-20T16:26:01.381 INFO:teuthology.orchestra.run.smithi160.stdout: "imported_inodes": 0, 2024-05-20T16:26:01.381 INFO:teuthology.orchestra.run.smithi160.stdout: "openino_dir_fetch": 0, 2024-05-20T16:26:01.381 INFO:teuthology.orchestra.run.smithi160.stdout: "openino_backtrace_fetch": 0, 2024-05-20T16:26:01.381 INFO:teuthology.orchestra.run.smithi160.stdout: "openino_peer_discover": 0, 2024-05-20T16:26:01.381 INFO:teuthology.orchestra.run.smithi160.stdout: "root_rfiles": 0, 2024-05-20T16:26:01.381 INFO:teuthology.orchestra.run.smithi160.stdout: "root_rbytes": 0, 2024-05-20T16:26:01.382 INFO:teuthology.orchestra.run.smithi160.stdout: "root_rsnaps": 0, 2024-05-20T16:26:01.382 INFO:teuthology.orchestra.run.smithi160.stdout: "scrub_backtrace_fetch": 0, 2024-05-20T16:26:01.382 INFO:teuthology.orchestra.run.smithi160.stdout: "scrub_set_tag": 0, 2024-05-20T16:26:01.382 INFO:teuthology.orchestra.run.smithi160.stdout: "scrub_backtrace_repaired": 0, 2024-05-20T16:26:01.382 INFO:teuthology.orchestra.run.smithi160.stdout: "scrub_inotable_repaired": 0, 2024-05-20T16:26:01.382 INFO:teuthology.orchestra.run.smithi160.stdout: "scrub_dir_inodes": 0, 2024-05-20T16:26:01.382 INFO:teuthology.orchestra.run.smithi160.stdout: "scrub_dir_base_inodes": 0, 2024-05-20T16:26:01.382 INFO:teuthology.orchestra.run.smithi160.stdout: "scrub_dirfrag_rstats": 0, 2024-05-20T16:26:01.382 INFO:teuthology.orchestra.run.smithi160.stdout: "scrub_file_inodes": 0, 2024-05-20T16:26:01.382 INFO:teuthology.orchestra.run.smithi160.stdout: "handle_inode_file_caps": 0, 2024-05-20T16:26:01.383 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph_cap_op_revoke": 0, 2024-05-20T16:26:01.383 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph_cap_op_grant": 0, 2024-05-20T16:26:01.383 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph_cap_op_trunc": 0, 2024-05-20T16:26:01.383 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-20T16:26:01.383 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-20T16:26:01.383 INFO:teuthology.orchestra.run.smithi160.stdout: "handle_client_caps": 0, 2024-05-20T16:26:01.383 INFO:teuthology.orchestra.run.smithi160.stdout: "handle_client_caps_dirty": 0, 2024-05-20T16:26:01.383 INFO:teuthology.orchestra.run.smithi160.stdout: "handle_client_cap_release": 0, 2024-05-20T16:26:01.383 INFO:teuthology.orchestra.run.smithi160.stdout: "process_request_cap_release": 0 2024-05-20T16:26:01.383 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-20T16:26:01.383 INFO:teuthology.orchestra.run.smithi160.stdout: "mds_cache": { 2024-05-20T16:26:01.383 INFO:teuthology.orchestra.run.smithi160.stdout: "dir_update": 0, 2024-05-20T16:26:01.383 INFO:teuthology.orchestra.run.smithi160.stdout: "dir_update_receipt": 0, 2024-05-20T16:26:01.384 INFO:teuthology.orchestra.run.smithi160.stdout: "dir_try_discover": 0, 2024-05-20T16:26:01.384 INFO:teuthology.orchestra.run.smithi160.stdout: "dir_send_discover": 0, 2024-05-20T16:26:01.384 INFO:teuthology.orchestra.run.smithi160.stdout: "dir_handle_discover": 0, 2024-05-20T16:26:01.384 INFO:teuthology.orchestra.run.smithi160.stdout: "num_strays": 0, 2024-05-20T16:26:01.384 INFO:teuthology.orchestra.run.smithi160.stdout: "num_strays_delayed": 0, 2024-05-20T16:26:01.384 INFO:teuthology.orchestra.run.smithi160.stdout: "num_strays_enqueuing": 0, 2024-05-20T16:26:01.384 INFO:teuthology.orchestra.run.smithi160.stdout: "strays_created": 0, 2024-05-20T16:26:01.384 INFO:teuthology.orchestra.run.smithi160.stdout: "strays_enqueued": 0, 2024-05-20T16:26:01.384 INFO:teuthology.orchestra.run.smithi160.stdout: "strays_reintegrated": 0, 2024-05-20T16:26:01.384 INFO:teuthology.orchestra.run.smithi160.stdout: "strays_migrated": 0, 2024-05-20T16:26:01.384 INFO:teuthology.orchestra.run.smithi160.stdout: "num_recovering_processing": 0, 2024-05-20T16:26:01.384 INFO:teuthology.orchestra.run.smithi160.stdout: "num_recovering_enqueued": 0, 2024-05-20T16:26:01.384 INFO:teuthology.orchestra.run.smithi160.stdout: "num_recovering_prioritized": 0, 2024-05-20T16:26:01.384 INFO:teuthology.orchestra.run.smithi160.stdout: "recovery_started": 0, 2024-05-20T16:26:01.384 INFO:teuthology.orchestra.run.smithi160.stdout: "recovery_completed": 0, 2024-05-20T16:26:01.385 INFO:teuthology.orchestra.run.smithi160.stdout: "ireq_quiesce_path": 0, 2024-05-20T16:26:01.385 INFO:teuthology.orchestra.run.smithi160.stdout: "ireq_quiesce_inode": 0, 2024-05-20T16:26:01.385 INFO:teuthology.orchestra.run.smithi160.stdout: "ireq_enqueue_scrub": 0, 2024-05-20T16:26:01.385 INFO:teuthology.orchestra.run.smithi160.stdout: "ireq_exportdir": 0, 2024-05-20T16:26:01.385 INFO:teuthology.orchestra.run.smithi160.stdout: "ireq_flush": 0, 2024-05-20T16:26:01.385 INFO:teuthology.orchestra.run.smithi160.stdout: "ireq_fragmentdir": 0, 2024-05-20T16:26:01.385 INFO:teuthology.orchestra.run.smithi160.stdout: "ireq_fragstats": 0, 2024-05-20T16:26:01.385 INFO:teuthology.orchestra.run.smithi160.stdout: "ireq_inodestats": 0 2024-05-20T16:26:01.385 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-20T16:26:01.385 INFO:teuthology.orchestra.run.smithi160.stdout: "mds_log": { 2024-05-20T16:26:01.385 INFO:teuthology.orchestra.run.smithi160.stdout: "evlrg": 0, 2024-05-20T16:26:01.385 INFO:teuthology.orchestra.run.smithi160.stdout: "evadd": 0, 2024-05-20T16:26:01.385 INFO:teuthology.orchestra.run.smithi160.stdout: "evex": 0, 2024-05-20T16:26:01.385 INFO:teuthology.orchestra.run.smithi160.stdout: "evtrm": 0, 2024-05-20T16:26:01.385 INFO:teuthology.orchestra.run.smithi160.stdout: "ev": 5, 2024-05-20T16:26:01.386 INFO:teuthology.orchestra.run.smithi160.stdout: "evexg": 0, 2024-05-20T16:26:01.386 INFO:teuthology.orchestra.run.smithi160.stdout: "evexd": 0, 2024-05-20T16:26:01.386 INFO:teuthology.orchestra.run.smithi160.stdout: "segadd": 0, 2024-05-20T16:26:01.386 INFO:teuthology.orchestra.run.smithi160.stdout: "segex": 0, 2024-05-20T16:26:01.386 INFO:teuthology.orchestra.run.smithi160.stdout: "segtrm": 0, 2024-05-20T16:26:01.386 INFO:teuthology.orchestra.run.smithi160.stdout: "seg": 2, 2024-05-20T16:26:01.386 INFO:teuthology.orchestra.run.smithi160.stdout: "segmjr": 2, 2024-05-20T16:26:01.386 INFO:teuthology.orchestra.run.smithi160.stdout: "segexg": 0, 2024-05-20T16:26:01.386 INFO:teuthology.orchestra.run.smithi160.stdout: "segexd": 0, 2024-05-20T16:26:01.386 INFO:teuthology.orchestra.run.smithi160.stdout: "expos": 4194304, 2024-05-20T16:26:01.386 INFO:teuthology.orchestra.run.smithi160.stdout: "wrpos": 4195732, 2024-05-20T16:26:01.386 INFO:teuthology.orchestra.run.smithi160.stdout: "rdpos": 4195424, 2024-05-20T16:26:01.386 INFO:teuthology.orchestra.run.smithi160.stdout: "jlat": { 2024-05-20T16:26:01.386 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-20T16:26:01.387 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-20T16:26:01.387 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-20T16:26:01.387 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-20T16:26:01.387 INFO:teuthology.orchestra.run.smithi160.stdout: "replayed": 5 2024-05-20T16:26:01.387 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-20T16:26:01.387 INFO:teuthology.orchestra.run.smithi160.stdout: "mds_mem": { 2024-05-20T16:26:01.387 INFO:teuthology.orchestra.run.smithi160.stdout: "ino": 3, 2024-05-20T16:26:01.387 INFO:teuthology.orchestra.run.smithi160.stdout: "ino+": 3, 2024-05-20T16:26:01.387 INFO:teuthology.orchestra.run.smithi160.stdout: "ino-": 0, 2024-05-20T16:26:01.387 INFO:teuthology.orchestra.run.smithi160.stdout: "dir": 1, 2024-05-20T16:26:01.387 INFO:teuthology.orchestra.run.smithi160.stdout: "dir+": 1, 2024-05-20T16:26:01.387 INFO:teuthology.orchestra.run.smithi160.stdout: "dir-": 0, 2024-05-20T16:26:01.387 INFO:teuthology.orchestra.run.smithi160.stdout: "dn": 0, 2024-05-20T16:26:01.388 INFO:teuthology.orchestra.run.smithi160.stdout: "dn+": 0, 2024-05-20T16:26:01.388 INFO:teuthology.orchestra.run.smithi160.stdout: "dn-": 0, 2024-05-20T16:26:01.388 INFO:teuthology.orchestra.run.smithi160.stdout: "cap": 0, 2024-05-20T16:26:01.388 INFO:teuthology.orchestra.run.smithi160.stdout: "cap+": 0, 2024-05-20T16:26:01.388 INFO:teuthology.orchestra.run.smithi160.stdout: "cap-": 0, 2024-05-20T16:26:01.388 INFO:teuthology.orchestra.run.smithi160.stdout: "rss": 120900, 2024-05-20T16:26:01.388 INFO:teuthology.orchestra.run.smithi160.stdout: "heap": 215444 2024-05-20T16:26:01.388 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-20T16:26:01.388 INFO:teuthology.orchestra.run.smithi160.stdout: "mds_server": { 2024-05-20T16:26:01.388 INFO:teuthology.orchestra.run.smithi160.stdout: "dispatch_client_request": 0, 2024-05-20T16:26:01.388 INFO:teuthology.orchestra.run.smithi160.stdout: "dispatch_server_request": 0, 2024-05-20T16:26:01.388 INFO:teuthology.orchestra.run.smithi160.stdout: "handle_client_request": 0, 2024-05-20T16:26:01.388 INFO:teuthology.orchestra.run.smithi160.stdout: "handle_client_session": 0, 2024-05-20T16:26:01.388 INFO:teuthology.orchestra.run.smithi160.stdout: "handle_peer_request": 0, 2024-05-20T16:26:01.389 INFO:teuthology.orchestra.run.smithi160.stdout: "req_create_latency": { 2024-05-20T16:26:01.389 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-20T16:26:01.389 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-20T16:26:01.389 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-20T16:26:01.389 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-20T16:26:01.389 INFO:teuthology.orchestra.run.smithi160.stdout: "req_getattr_latency": { 2024-05-20T16:26:01.389 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-20T16:26:01.389 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-20T16:26:01.389 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-20T16:26:01.389 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-20T16:26:01.389 INFO:teuthology.orchestra.run.smithi160.stdout: "req_getfilelock_latency": { 2024-05-20T16:26:01.390 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-20T16:26:01.390 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-20T16:26:01.390 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-20T16:26:01.390 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-20T16:26:01.390 INFO:teuthology.orchestra.run.smithi160.stdout: "req_link_latency": { 2024-05-20T16:26:01.391 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-20T16:26:01.391 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-20T16:26:01.391 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-20T16:26:01.391 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-20T16:26:01.391 INFO:teuthology.orchestra.run.smithi160.stdout: "req_lookup_latency": { 2024-05-20T16:26:01.392 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-20T16:26:01.392 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-20T16:26:01.392 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-20T16:26:01.392 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-20T16:26:01.392 INFO:teuthology.orchestra.run.smithi160.stdout: "req_lookuphash_latency": { 2024-05-20T16:26:01.392 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-20T16:26:01.392 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-20T16:26:01.392 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-20T16:26:01.392 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-20T16:26:01.392 INFO:teuthology.orchestra.run.smithi160.stdout: "req_lookupino_latency": { 2024-05-20T16:26:01.392 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-20T16:26:01.392 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-20T16:26:01.393 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-20T16:26:01.393 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-20T16:26:01.393 INFO:teuthology.orchestra.run.smithi160.stdout: "req_lookupname_latency": { 2024-05-20T16:26:01.393 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-20T16:26:01.393 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-20T16:26:01.393 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-20T16:26:01.393 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-20T16:26:01.393 INFO:teuthology.orchestra.run.smithi160.stdout: "req_lookupparent_latency": { 2024-05-20T16:26:01.393 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-20T16:26:01.393 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-20T16:26:01.394 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-20T16:26:01.394 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-20T16:26:01.394 INFO:teuthology.orchestra.run.smithi160.stdout: "req_lookupsnap_latency": { 2024-05-20T16:26:01.394 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-20T16:26:01.394 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-20T16:26:01.394 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-20T16:26:01.394 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-20T16:26:01.394 INFO:teuthology.orchestra.run.smithi160.stdout: "req_lssnap_latency": { 2024-05-20T16:26:01.394 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-20T16:26:01.394 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-20T16:26:01.394 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-20T16:26:01.394 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-20T16:26:01.394 INFO:teuthology.orchestra.run.smithi160.stdout: "req_mkdir_latency": { 2024-05-20T16:26:01.394 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-20T16:26:01.395 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:26:00 smithi184 ceph-mon[36209]: pgmap v725: 129 pgs: 129 active+clean; 10 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 81 MiB/s wr, 149 op/s 2024-05-20T16:26:01.395 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-20T16:26:01.395 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-20T16:26:01.395 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-20T16:26:01.396 INFO:teuthology.orchestra.run.smithi160.stdout: "req_mknod_latency": { 2024-05-20T16:26:01.396 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-20T16:26:01.396 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-20T16:26:01.396 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-20T16:26:01.396 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-20T16:26:01.396 INFO:teuthology.orchestra.run.smithi160.stdout: "req_mksnap_latency": { 2024-05-20T16:26:01.396 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-20T16:26:01.396 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-20T16:26:01.396 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-20T16:26:01.396 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-20T16:26:01.396 INFO:teuthology.orchestra.run.smithi160.stdout: "req_open_latency": { 2024-05-20T16:26:01.396 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-20T16:26:01.396 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-20T16:26:01.396 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-20T16:26:01.396 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-20T16:26:01.397 INFO:teuthology.orchestra.run.smithi160.stdout: "req_readdir_latency": { 2024-05-20T16:26:01.397 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-20T16:26:01.397 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-20T16:26:01.397 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-20T16:26:01.397 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-20T16:26:01.397 INFO:teuthology.orchestra.run.smithi160.stdout: "req_rename_latency": { 2024-05-20T16:26:01.397 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-20T16:26:01.397 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-20T16:26:01.397 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-20T16:26:01.397 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-20T16:26:01.397 INFO:teuthology.orchestra.run.smithi160.stdout: "req_renamesnap_latency": { 2024-05-20T16:26:01.397 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-20T16:26:01.397 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-20T16:26:01.397 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-20T16:26:01.397 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-20T16:26:01.398 INFO:teuthology.orchestra.run.smithi160.stdout: "req_snapdiff_latency": { 2024-05-20T16:26:01.398 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-20T16:26:01.398 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-20T16:26:01.398 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-20T16:26:01.398 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-20T16:26:01.398 INFO:teuthology.orchestra.run.smithi160.stdout: "req_rmdir_latency": { 2024-05-20T16:26:01.398 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-20T16:26:01.398 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-20T16:26:01.398 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-20T16:26:01.398 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-20T16:26:01.398 INFO:teuthology.orchestra.run.smithi160.stdout: "req_rmsnap_latency": { 2024-05-20T16:26:01.398 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-20T16:26:01.398 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-20T16:26:01.399 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-20T16:26:01.399 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-20T16:26:01.399 INFO:teuthology.orchestra.run.smithi160.stdout: "req_rmxattr_latency": { 2024-05-20T16:26:01.399 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-20T16:26:01.399 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-20T16:26:01.399 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-20T16:26:01.399 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-20T16:26:01.399 INFO:teuthology.orchestra.run.smithi160.stdout: "req_setattr_latency": { 2024-05-20T16:26:01.399 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-20T16:26:01.399 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-20T16:26:01.399 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-20T16:26:01.399 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-20T16:26:01.399 INFO:teuthology.orchestra.run.smithi160.stdout: "req_setdirlayout_latency": { 2024-05-20T16:26:01.399 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-20T16:26:01.399 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-20T16:26:01.400 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-20T16:26:01.400 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-20T16:26:01.400 INFO:teuthology.orchestra.run.smithi160.stdout: "req_setfilelock_latency": { 2024-05-20T16:26:01.400 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-20T16:26:01.400 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-20T16:26:01.400 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-20T16:26:01.400 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-20T16:26:01.400 INFO:teuthology.orchestra.run.smithi160.stdout: "req_setlayout_latency": { 2024-05-20T16:26:01.400 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-20T16:26:01.400 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-20T16:26:01.400 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-20T16:26:01.400 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-20T16:26:01.401 INFO:teuthology.orchestra.run.smithi160.stdout: "req_setxattr_latency": { 2024-05-20T16:26:01.401 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-20T16:26:01.401 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-20T16:26:01.401 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-20T16:26:01.401 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-20T16:26:01.401 INFO:teuthology.orchestra.run.smithi160.stdout: "req_symlink_latency": { 2024-05-20T16:26:01.401 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-20T16:26:01.401 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-20T16:26:01.401 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-20T16:26:01.401 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-20T16:26:01.401 INFO:teuthology.orchestra.run.smithi160.stdout: "req_unlink_latency": { 2024-05-20T16:26:01.401 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-20T16:26:01.401 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-20T16:26:01.402 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-20T16:26:01.402 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-20T16:26:01.402 INFO:teuthology.orchestra.run.smithi160.stdout: "cap_revoke_eviction": 0, 2024-05-20T16:26:01.402 INFO:teuthology.orchestra.run.smithi160.stdout: "cap_acquisition_throttle": 0, 2024-05-20T16:26:01.402 INFO:teuthology.orchestra.run.smithi160.stdout: "req_getvxattr_latency": { 2024-05-20T16:26:01.402 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-20T16:26:01.402 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-20T16:26:01.402 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-20T16:26:01.402 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-05-20T16:26:01.402 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-20T16:26:01.402 INFO:teuthology.orchestra.run.smithi160.stdout: "mds_sessions": { 2024-05-20T16:26:01.403 INFO:teuthology.orchestra.run.smithi160.stdout: "session_count": 1, 2024-05-20T16:26:01.403 INFO:teuthology.orchestra.run.smithi160.stdout: "session_add": 2, 2024-05-20T16:26:01.403 INFO:teuthology.orchestra.run.smithi160.stdout: "session_remove": 1, 2024-05-20T16:26:01.403 INFO:teuthology.orchestra.run.smithi160.stdout: "sessions_open": 1, 2024-05-20T16:26:01.403 INFO:teuthology.orchestra.run.smithi160.stdout: "sessions_stale": 0, 2024-05-20T16:26:01.403 INFO:teuthology.orchestra.run.smithi160.stdout: "total_load": 0, 2024-05-20T16:26:01.403 INFO:teuthology.orchestra.run.smithi160.stdout: "average_load": 0, 2024-05-20T16:26:01.403 INFO:teuthology.orchestra.run.smithi160.stdout: "avg_session_uptime": 2058, 2024-05-20T16:26:01.403 INFO:teuthology.orchestra.run.smithi160.stdout: "mdthresh_evicted": 0 2024-05-20T16:26:01.403 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-20T16:26:01.404 INFO:teuthology.orchestra.run.smithi160.stdout: "mempool": { 2024-05-20T16:26:01.404 INFO:teuthology.orchestra.run.smithi160.stdout: "bloom_filter_bytes": 0, 2024-05-20T16:26:01.404 INFO:teuthology.orchestra.run.smithi160.stdout: "bloom_filter_items": 0, 2024-05-20T16:26:01.404 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_alloc_bytes": 0, 2024-05-20T16:26:01.404 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_alloc_items": 0, 2024-05-20T16:26:01.404 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_data_bytes": 0, 2024-05-20T16:26:01.404 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_data_items": 0, 2024-05-20T16:26:01.404 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-20T16:26:01.404 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_onode_items": 0, 2024-05-20T16:26:01.404 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-20T16:26:01.404 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_meta_items": 0, 2024-05-20T16:26:01.404 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_other_bytes": 0, 2024-05-20T16:26:01.404 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_other_items": 0, 2024-05-20T16:26:01.405 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-20T16:26:01.405 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_buffer_items": 0, 2024-05-20T16:26:01.405 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_extent_bytes": 0, 2024-05-20T16:26:01.405 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_extent_items": 0, 2024-05-20T16:26:01.405 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_blob_bytes": 0, 2024-05-20T16:26:01.405 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_blob_items": 0, 2024-05-20T16:26:01.405 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-20T16:26:01.405 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_shared_blob_items": 0, 2024-05-20T16:26:01.405 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-20T16:26:01.405 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_inline_bl_items": 0, 2024-05-20T16:26:01.405 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_fsck_bytes": 0, 2024-05-20T16:26:01.405 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_fsck_items": 0, 2024-05-20T16:26:01.405 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_txc_bytes": 0, 2024-05-20T16:26:01.405 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_txc_items": 0, 2024-05-20T16:26:01.405 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-20T16:26:01.406 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_writing_deferred_items": 0, 2024-05-20T16:26:01.406 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_writing_bytes": 0, 2024-05-20T16:26:01.406 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_writing_items": 0, 2024-05-20T16:26:01.406 INFO:teuthology.orchestra.run.smithi160.stdout: "bluefs_bytes": 0, 2024-05-20T16:26:01.406 INFO:teuthology.orchestra.run.smithi160.stdout: "bluefs_items": 0, 2024-05-20T16:26:01.406 INFO:teuthology.orchestra.run.smithi160.stdout: "bluefs_file_reader_bytes": 0, 2024-05-20T16:26:01.406 INFO:teuthology.orchestra.run.smithi160.stdout: "bluefs_file_reader_items": 0, 2024-05-20T16:26:01.406 INFO:teuthology.orchestra.run.smithi160.stdout: "bluefs_file_writer_bytes": 0, 2024-05-20T16:26:01.406 INFO:teuthology.orchestra.run.smithi160.stdout: "bluefs_file_writer_items": 0, 2024-05-20T16:26:01.406 INFO:teuthology.orchestra.run.smithi160.stdout: "buffer_anon_bytes": 189463, 2024-05-20T16:26:01.406 INFO:teuthology.orchestra.run.smithi160.stdout: "buffer_anon_items": 52, 2024-05-20T16:26:01.406 INFO:teuthology.orchestra.run.smithi160.stdout: "buffer_meta_bytes": 0, 2024-05-20T16:26:01.406 INFO:teuthology.orchestra.run.smithi160.stdout: "buffer_meta_items": 0, 2024-05-20T16:26:01.406 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_bytes": 0, 2024-05-20T16:26:01.406 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_items": 0, 2024-05-20T16:26:01.407 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_mapbl_bytes": 0, 2024-05-20T16:26:01.407 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_mapbl_items": 0, 2024-05-20T16:26:01.407 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_pglog_bytes": 0, 2024-05-20T16:26:01.407 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_pglog_items": 0, 2024-05-20T16:26:01.407 INFO:teuthology.orchestra.run.smithi160.stdout: "osdmap_bytes": 8888, 2024-05-20T16:26:01.407 INFO:teuthology.orchestra.run.smithi160.stdout: "osdmap_items": 170, 2024-05-20T16:26:01.407 INFO:teuthology.orchestra.run.smithi160.stdout: "osdmap_mapping_bytes": 0, 2024-05-20T16:26:01.407 INFO:teuthology.orchestra.run.smithi160.stdout: "osdmap_mapping_items": 0, 2024-05-20T16:26:01.407 INFO:teuthology.orchestra.run.smithi160.stdout: "pgmap_bytes": 0, 2024-05-20T16:26:01.407 INFO:teuthology.orchestra.run.smithi160.stdout: "pgmap_items": 0, 2024-05-20T16:26:01.407 INFO:teuthology.orchestra.run.smithi160.stdout: "mds_co_bytes": 8464, 2024-05-20T16:26:01.407 INFO:teuthology.orchestra.run.smithi160.stdout: "mds_co_items": 13, 2024-05-20T16:26:01.407 INFO:teuthology.orchestra.run.smithi160.stdout: "unittest_1_bytes": 0, 2024-05-20T16:26:01.407 INFO:teuthology.orchestra.run.smithi160.stdout: "unittest_1_items": 0, 2024-05-20T16:26:01.407 INFO:teuthology.orchestra.run.smithi160.stdout: "unittest_2_bytes": 0, 2024-05-20T16:26:01.407 INFO:teuthology.orchestra.run.smithi160.stdout: "unittest_2_items": 0 2024-05-20T16:26:01.408 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-20T16:26:01.408 INFO:teuthology.orchestra.run.smithi160.stdout: "objecter": { 2024-05-20T16:26:01.408 INFO:teuthology.orchestra.run.smithi160.stdout: "op_active": 0, 2024-05-20T16:26:01.408 INFO:teuthology.orchestra.run.smithi160.stdout: "op_laggy": 0, 2024-05-20T16:26:01.408 INFO:teuthology.orchestra.run.smithi160.stdout: "op_send": 2344, 2024-05-20T16:26:01.408 INFO:teuthology.orchestra.run.smithi160.stdout: "op_send_bytes": 21, 2024-05-20T16:26:01.408 INFO:teuthology.orchestra.run.smithi160.stdout: "op_resend": 3, 2024-05-20T16:26:01.408 INFO:teuthology.orchestra.run.smithi160.stdout: "op_reply": 2344, 2024-05-20T16:26:01.408 INFO:teuthology.orchestra.run.smithi160.stdout: "op_latency": { 2024-05-20T16:26:01.408 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 2344, 2024-05-20T16:26:01.408 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 29.486576093, 2024-05-20T16:26:01.408 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.012579597 2024-05-20T16:26:01.408 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-20T16:26:01.408 INFO:teuthology.orchestra.run.smithi160.stdout: "op_inflight": 0, 2024-05-20T16:26:01.408 INFO:teuthology.orchestra.run.smithi160.stdout: "oplen_avg": { 2024-05-20T16:26:01.409 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 2344, 2024-05-20T16:26:01.409 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 2345 2024-05-20T16:26:01.409 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-20T16:26:01.409 INFO:teuthology.orchestra.run.smithi160.stdout: "op": 2344, 2024-05-20T16:26:01.409 INFO:teuthology.orchestra.run.smithi160.stdout: "op_r": 2344, 2024-05-20T16:26:01.409 INFO:teuthology.orchestra.run.smithi160.stdout: "op_w": 0, 2024-05-20T16:26:01.409 INFO:teuthology.orchestra.run.smithi160.stdout: "op_rmw": 0, 2024-05-20T16:26:01.409 INFO:teuthology.orchestra.run.smithi160.stdout: "op_pg": 0, 2024-05-20T16:26:01.409 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_stat": 1536, 2024-05-20T16:26:01.409 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_create": 0, 2024-05-20T16:26:01.409 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_read": 806, 2024-05-20T16:26:01.409 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_write": 0, 2024-05-20T16:26:01.409 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_writefull": 0, 2024-05-20T16:26:01.409 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_writesame": 0, 2024-05-20T16:26:01.410 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_append": 0, 2024-05-20T16:26:01.410 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_zero": 0, 2024-05-20T16:26:01.410 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_truncate": 0, 2024-05-20T16:26:01.410 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_delete": 0, 2024-05-20T16:26:01.410 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_mapext": 0, 2024-05-20T16:26:01.410 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_sparse_read": 0, 2024-05-20T16:26:01.410 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_clonerange": 0, 2024-05-20T16:26:01.410 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_getxattr": 1, 2024-05-20T16:26:01.410 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_setxattr": 0, 2024-05-20T16:26:01.410 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_cmpxattr": 0, 2024-05-20T16:26:01.410 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_rmxattr": 0, 2024-05-20T16:26:01.411 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_resetxattrs": 0, 2024-05-20T16:26:01.411 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_call": 0, 2024-05-20T16:26:01.411 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_watch": 0, 2024-05-20T16:26:01.411 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_notify": 0, 2024-05-20T16:26:01.411 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_src_cmpxattr": 0, 2024-05-20T16:26:01.411 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_pgls": 0, 2024-05-20T16:26:01.411 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_pgls_filter": 0, 2024-05-20T16:26:01.411 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_other": 0, 2024-05-20T16:26:01.411 INFO:teuthology.orchestra.run.smithi160.stdout: "linger_active": 0, 2024-05-20T16:26:01.411 INFO:teuthology.orchestra.run.smithi160.stdout: "linger_send": 0, 2024-05-20T16:26:01.411 INFO:teuthology.orchestra.run.smithi160.stdout: "linger_resend": 0, 2024-05-20T16:26:01.411 INFO:teuthology.orchestra.run.smithi160.stdout: "linger_ping": 0, 2024-05-20T16:26:01.411 INFO:teuthology.orchestra.run.smithi160.stdout: "poolop_active": 0, 2024-05-20T16:26:01.412 INFO:teuthology.orchestra.run.smithi160.stdout: "poolop_send": 0, 2024-05-20T16:26:01.412 INFO:teuthology.orchestra.run.smithi160.stdout: "poolop_resend": 0, 2024-05-20T16:26:01.412 INFO:teuthology.orchestra.run.smithi160.stdout: "poolstat_active": 0, 2024-05-20T16:26:01.412 INFO:teuthology.orchestra.run.smithi160.stdout: "poolstat_send": 0, 2024-05-20T16:26:01.412 INFO:teuthology.orchestra.run.smithi160.stdout: "poolstat_resend": 0, 2024-05-20T16:26:01.412 INFO:teuthology.orchestra.run.smithi160.stdout: "statfs_active": 0, 2024-05-20T16:26:01.412 INFO:teuthology.orchestra.run.smithi160.stdout: "statfs_send": 0, 2024-05-20T16:26:01.412 INFO:teuthology.orchestra.run.smithi160.stdout: "statfs_resend": 0, 2024-05-20T16:26:01.412 INFO:teuthology.orchestra.run.smithi160.stdout: "command_active": 0, 2024-05-20T16:26:01.412 INFO:teuthology.orchestra.run.smithi160.stdout: "command_send": 0, 2024-05-20T16:26:01.412 INFO:teuthology.orchestra.run.smithi160.stdout: "command_resend": 0, 2024-05-20T16:26:01.412 INFO:teuthology.orchestra.run.smithi160.stdout: "map_epoch": 98, 2024-05-20T16:26:01.412 INFO:teuthology.orchestra.run.smithi160.stdout: "map_full": 0, 2024-05-20T16:26:01.412 INFO:teuthology.orchestra.run.smithi160.stdout: "map_inc": 7, 2024-05-20T16:26:01.413 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_sessions": 5, 2024-05-20T16:26:01.413 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_session_open": 5, 2024-05-20T16:26:01.413 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_session_close": 0, 2024-05-20T16:26:01.413 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_laggy": 0, 2024-05-20T16:26:01.413 INFO:teuthology.orchestra.run.smithi160.stdout: "omap_wr": 0, 2024-05-20T16:26:01.413 INFO:teuthology.orchestra.run.smithi160.stdout: "omap_rd": 2, 2024-05-20T16:26:01.413 INFO:teuthology.orchestra.run.smithi160.stdout: "omap_del": 0 2024-05-20T16:26:01.413 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-20T16:26:01.413 INFO:teuthology.orchestra.run.smithi160.stdout: "oft": { 2024-05-20T16:26:01.413 INFO:teuthology.orchestra.run.smithi160.stdout: "omap_total_objs": 0, 2024-05-20T16:26:01.414 INFO:teuthology.orchestra.run.smithi160.stdout: "omap_total_kv_pairs": 0, 2024-05-20T16:26:01.414 INFO:teuthology.orchestra.run.smithi160.stdout: "omap_total_updates": 0, 2024-05-20T16:26:01.414 INFO:teuthology.orchestra.run.smithi160.stdout: "omap_total_removes": 0 2024-05-20T16:26:01.414 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-20T16:26:01.414 INFO:teuthology.orchestra.run.smithi160.stdout: "osd-slow-ops": { 2024-05-20T16:26:01.414 INFO:teuthology.orchestra.run.smithi160.stdout: "slow_ops_count": 0 2024-05-20T16:26:01.414 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-20T16:26:01.414 INFO:teuthology.orchestra.run.smithi160.stdout: "purge_queue": { 2024-05-20T16:26:01.415 INFO:teuthology.orchestra.run.smithi160.stdout: "pq_executing_ops": 0, 2024-05-20T16:26:01.415 INFO:teuthology.orchestra.run.smithi160.stdout: "pq_executing_ops_high_water": 0, 2024-05-20T16:26:01.415 INFO:teuthology.orchestra.run.smithi160.stdout: "pq_executing": 0, 2024-05-20T16:26:01.415 INFO:teuthology.orchestra.run.smithi160.stdout: "pq_executing_high_water": 0, 2024-05-20T16:26:01.415 INFO:teuthology.orchestra.run.smithi160.stdout: "pq_executed_ops": 0, 2024-05-20T16:26:01.415 INFO:teuthology.orchestra.run.smithi160.stdout: "pq_executed": 0, 2024-05-20T16:26:01.415 INFO:teuthology.orchestra.run.smithi160.stdout: "pq_item_in_journal": 0 2024-05-20T16:26:01.415 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-20T16:26:01.415 INFO:teuthology.orchestra.run.smithi160.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-20T16:26:01.416 INFO:teuthology.orchestra.run.smithi160.stdout: "val": 0, 2024-05-20T16:26:01.416 INFO:teuthology.orchestra.run.smithi160.stdout: "max": 104857600, 2024-05-20T16:26:01.416 INFO:teuthology.orchestra.run.smithi160.stdout: "get_started": 0, 2024-05-20T16:26:01.416 INFO:teuthology.orchestra.run.smithi160.stdout: "get": 2604, 2024-05-20T16:26:01.416 INFO:teuthology.orchestra.run.smithi160.stdout: "get_sum": 1221312, 2024-05-20T16:26:01.416 INFO:teuthology.orchestra.run.smithi160.stdout: "get_or_fail_fail": 0, 2024-05-20T16:26:01.416 INFO:teuthology.orchestra.run.smithi160.stdout: "get_or_fail_success": 2604, 2024-05-20T16:26:01.416 INFO:teuthology.orchestra.run.smithi160.stdout: "take": 0, 2024-05-20T16:26:01.416 INFO:teuthology.orchestra.run.smithi160.stdout: "take_sum": 0, 2024-05-20T16:26:01.416 INFO:teuthology.orchestra.run.smithi160.stdout: "put": 2604, 2024-05-20T16:26:01.416 INFO:teuthology.orchestra.run.smithi160.stdout: "put_sum": 1221312, 2024-05-20T16:26:01.417 INFO:teuthology.orchestra.run.smithi160.stdout: "wait": { 2024-05-20T16:26:01.417 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-20T16:26:01.417 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-20T16:26:01.417 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-20T16:26:01.417 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-05-20T16:26:01.417 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-20T16:26:01.417 INFO:teuthology.orchestra.run.smithi160.stdout: "throttle-objecter_bytes": { 2024-05-20T16:26:01.417 INFO:teuthology.orchestra.run.smithi160.stdout: "val": 0, 2024-05-20T16:26:01.417 INFO:teuthology.orchestra.run.smithi160.stdout: "max": 104857600, 2024-05-20T16:26:01.417 INFO:teuthology.orchestra.run.smithi160.stdout: "get_started": 0, 2024-05-20T16:26:01.417 INFO:teuthology.orchestra.run.smithi160.stdout: "get": 0, 2024-05-20T16:26:01.417 INFO:teuthology.orchestra.run.smithi160.stdout: "get_sum": 0, 2024-05-20T16:26:01.417 INFO:teuthology.orchestra.run.smithi160.stdout: "get_or_fail_fail": 0, 2024-05-20T16:26:01.417 INFO:teuthology.orchestra.run.smithi160.stdout: "get_or_fail_success": 0, 2024-05-20T16:26:01.417 INFO:teuthology.orchestra.run.smithi160.stdout: "take": 2344, 2024-05-20T16:26:01.418 INFO:teuthology.orchestra.run.smithi160.stdout: "take_sum": 1433, 2024-05-20T16:26:01.418 INFO:teuthology.orchestra.run.smithi160.stdout: "put": 2344, 2024-05-20T16:26:01.418 INFO:teuthology.orchestra.run.smithi160.stdout: "put_sum": 1433, 2024-05-20T16:26:01.418 INFO:teuthology.orchestra.run.smithi160.stdout: "wait": { 2024-05-20T16:26:01.418 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-20T16:26:01.418 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-20T16:26:01.418 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-20T16:26:01.418 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-05-20T16:26:01.418 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-20T16:26:01.418 INFO:teuthology.orchestra.run.smithi160.stdout: "throttle-objecter_ops": { 2024-05-20T16:26:01.418 INFO:teuthology.orchestra.run.smithi160.stdout: "val": 0, 2024-05-20T16:26:01.418 INFO:teuthology.orchestra.run.smithi160.stdout: "max": 1024, 2024-05-20T16:26:01.419 INFO:teuthology.orchestra.run.smithi160.stdout: "get_started": 0, 2024-05-20T16:26:01.419 INFO:teuthology.orchestra.run.smithi160.stdout: "get": 0, 2024-05-20T16:26:01.419 INFO:teuthology.orchestra.run.smithi160.stdout: "get_sum": 0, 2024-05-20T16:26:01.419 INFO:teuthology.orchestra.run.smithi160.stdout: "get_or_fail_fail": 0, 2024-05-20T16:26:01.419 INFO:teuthology.orchestra.run.smithi160.stdout: "get_or_fail_success": 0, 2024-05-20T16:26:01.419 INFO:teuthology.orchestra.run.smithi160.stdout: "take": 2344, 2024-05-20T16:26:01.419 INFO:teuthology.orchestra.run.smithi160.stdout: "take_sum": 2344, 2024-05-20T16:26:01.419 INFO:teuthology.orchestra.run.smithi160.stdout: "put": 2344, 2024-05-20T16:26:01.419 INFO:teuthology.orchestra.run.smithi160.stdout: "put_sum": 2344, 2024-05-20T16:26:01.419 INFO:teuthology.orchestra.run.smithi160.stdout: "wait": { 2024-05-20T16:26:01.419 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-20T16:26:01.419 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-20T16:26:01.420 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-20T16:26:01.420 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-05-20T16:26:01.420 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-20T16:26:01.420 INFO:teuthology.orchestra.run.smithi160.stdout: "throttle-write_buf_throttle": { 2024-05-20T16:26:01.420 INFO:teuthology.orchestra.run.smithi160.stdout: "val": 0, 2024-05-20T16:26:01.420 INFO:teuthology.orchestra.run.smithi160.stdout: "max": 3758096384, 2024-05-20T16:26:01.420 INFO:teuthology.orchestra.run.smithi160.stdout: "get_started": 0, 2024-05-20T16:26:01.420 INFO:teuthology.orchestra.run.smithi160.stdout: "get": 0, 2024-05-20T16:26:01.420 INFO:teuthology.orchestra.run.smithi160.stdout: "get_sum": 0, 2024-05-20T16:26:01.420 INFO:teuthology.orchestra.run.smithi160.stdout: "get_or_fail_fail": 0, 2024-05-20T16:26:01.420 INFO:teuthology.orchestra.run.smithi160.stdout: "get_or_fail_success": 0, 2024-05-20T16:26:01.420 INFO:teuthology.orchestra.run.smithi160.stdout: "take": 0, 2024-05-20T16:26:01.420 INFO:teuthology.orchestra.run.smithi160.stdout: "take_sum": 0, 2024-05-20T16:26:01.420 INFO:teuthology.orchestra.run.smithi160.stdout: "put": 0, 2024-05-20T16:26:01.420 INFO:teuthology.orchestra.run.smithi160.stdout: "put_sum": 0, 2024-05-20T16:26:01.421 INFO:teuthology.orchestra.run.smithi160.stdout: "wait": { 2024-05-20T16:26:01.421 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-20T16:26:01.421 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-20T16:26:01.421 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-20T16:26:01.421 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-05-20T16:26:01.421 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-20T16:26:01.421 INFO:teuthology.orchestra.run.smithi160.stdout: "throttle-write_buf_throttle-0x555c746dc9c0": { 2024-05-20T16:26:01.421 INFO:teuthology.orchestra.run.smithi160.stdout: "val": 0, 2024-05-20T16:26:01.421 INFO:teuthology.orchestra.run.smithi160.stdout: "max": 3758096384, 2024-05-20T16:26:01.421 INFO:teuthology.orchestra.run.smithi160.stdout: "get_started": 0, 2024-05-20T16:26:01.421 INFO:teuthology.orchestra.run.smithi160.stdout: "get": 0, 2024-05-20T16:26:01.422 INFO:teuthology.orchestra.run.smithi160.stdout: "get_sum": 0, 2024-05-20T16:26:01.422 INFO:teuthology.orchestra.run.smithi160.stdout: "get_or_fail_fail": 0, 2024-05-20T16:26:01.422 INFO:teuthology.orchestra.run.smithi160.stdout: "get_or_fail_success": 0, 2024-05-20T16:26:01.422 INFO:teuthology.orchestra.run.smithi160.stdout: "take": 0, 2024-05-20T16:26:01.422 INFO:teuthology.orchestra.run.smithi160.stdout: "take_sum": 0, 2024-05-20T16:26:01.422 INFO:teuthology.orchestra.run.smithi160.stdout: "put": 0, 2024-05-20T16:26:01.422 INFO:teuthology.orchestra.run.smithi160.stdout: "put_sum": 0, 2024-05-20T16:26:01.422 INFO:teuthology.orchestra.run.smithi160.stdout: "wait": { 2024-05-20T16:26:01.422 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-20T16:26:01.422 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-20T16:26:01.422 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-20T16:26:01.422 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-05-20T16:26:01.423 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-05-20T16:26:01.423 INFO:teuthology.orchestra.run.smithi160.stdout:} 2024-05-20T16:26:02.105 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-05-20T16:26:02.106 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2024-05-20T16:26:02.106 DEBUG:tasks.check_counter:Getting stats from h 2024-05-20T16:26:02.106 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell --fsid 59d1600a-16c2-11ef-bc9a-c7b262605968 -- ceph daemon mds.h perf dump 2024-05-20T16:26:02.346 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/mon.b/config 2024-05-20T16:26:03.062 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:26:02 smithi160 ceph-mon[36347]: pgmap v726: 129 pgs: 129 active+clean; 10 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 66 MiB/s wr, 122 op/s 2024-05-20T16:26:03.360 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:26:02 smithi082 ceph-mon[32238]: pgmap v726: 129 pgs: 129 active+clean; 10 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 66 MiB/s wr, 122 op/s 2024-05-20T16:26:03.385 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:26:02 smithi184 ceph-mon[36209]: pgmap v726: 129 pgs: 129 active+clean; 10 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 66 MiB/s wr, 122 op/s 2024-05-20T16:26:04.223 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:26:04 smithi160 ceph-mon[36347]: pgmap v727: 129 pgs: 129 active+clean; 10 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 74 MiB/s wr, 139 op/s 2024-05-20T16:26:04.223 INFO:teuthology.orchestra.run.smithi160.stdout:{ 2024-05-20T16:26:04.223 INFO:teuthology.orchestra.run.smithi160.stdout: "AsyncMessenger::Worker-0": { 2024-05-20T16:26:04.224 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_messages": 1037, 2024-05-20T16:26:04.224 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_messages": 751, 2024-05-20T16:26:04.224 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_bytes": 252009, 2024-05-20T16:26:04.224 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_bytes": 284402, 2024-05-20T16:26:04.224 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_created_connections": 8, 2024-05-20T16:26:04.224 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_active_connections": 2, 2024-05-20T16:26:04.224 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_total_time": 0.086925112, 2024-05-20T16:26:04.224 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_send_time": 0.036339469, 2024-05-20T16:26:04.224 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_recv_time": 0.060712294, 2024-05-20T16:26:04.224 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_fast_dispatch_time": 0.004683596, 2024-05-20T16:26:04.224 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_messages_queue_lat": { 2024-05-20T16:26:04.224 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 753, 2024-05-20T16:26:04.224 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.064382557, 2024-05-20T16:26:04.224 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000085501 2024-05-20T16:26:04.224 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-20T16:26:04.225 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_handle_ack_lat": { 2024-05-20T16:26:04.225 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 1637, 2024-05-20T16:26:04.225 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000389723, 2024-05-20T16:26:04.225 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000238 2024-05-20T16:26:04.225 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-20T16:26:04.225 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-20T16:26:04.225 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_encrypted_bytes": 0 2024-05-20T16:26:04.225 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-20T16:26:04.225 INFO:teuthology.orchestra.run.smithi160.stdout: "AsyncMessenger::Worker-1": { 2024-05-20T16:26:04.225 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_messages": 306, 2024-05-20T16:26:04.225 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_messages": 292, 2024-05-20T16:26:04.225 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_bytes": 819398, 2024-05-20T16:26:04.226 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_bytes": 164157, 2024-05-20T16:26:04.226 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_created_connections": 5, 2024-05-20T16:26:04.226 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_active_connections": 4, 2024-05-20T16:26:04.226 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_total_time": 0.045410268, 2024-05-20T16:26:04.226 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_send_time": 0.014954474, 2024-05-20T16:26:04.226 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_recv_time": 0.027440265, 2024-05-20T16:26:04.226 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_fast_dispatch_time": 0.007088384, 2024-05-20T16:26:04.226 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_messages_queue_lat": { 2024-05-20T16:26:04.226 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 291, 2024-05-20T16:26:04.226 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.028380129, 2024-05-20T16:26:04.226 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000097526 2024-05-20T16:26:04.227 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-20T16:26:04.227 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_handle_ack_lat": { 2024-05-20T16:26:04.227 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-20T16:26:04.227 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-20T16:26:04.227 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-20T16:26:04.227 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-20T16:26:04.227 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_encrypted_bytes": 815024, 2024-05-20T16:26:04.227 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_encrypted_bytes": 150928 2024-05-20T16:26:04.227 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-20T16:26:04.227 INFO:teuthology.orchestra.run.smithi160.stdout: "AsyncMessenger::Worker-2": { 2024-05-20T16:26:04.227 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_messages": 36, 2024-05-20T16:26:04.227 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_messages": 218, 2024-05-20T16:26:04.227 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_bytes": 9989, 2024-05-20T16:26:04.228 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_bytes": 443596, 2024-05-20T16:26:04.228 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_created_connections": 12, 2024-05-20T16:26:04.228 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_active_connections": 8, 2024-05-20T16:26:04.228 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_total_time": 0.022246456, 2024-05-20T16:26:04.228 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_send_time": 0.010266413, 2024-05-20T16:26:04.228 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_recv_time": 0.009859603, 2024-05-20T16:26:04.228 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_fast_dispatch_time": 0.000211363, 2024-05-20T16:26:04.228 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_messages_queue_lat": { 2024-05-20T16:26:04.228 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 213, 2024-05-20T16:26:04.228 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.020577285, 2024-05-20T16:26:04.228 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000096606 2024-05-20T16:26:04.228 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-20T16:26:04.228 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_handle_ack_lat": { 2024-05-20T16:26:04.229 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 47, 2024-05-20T16:26:04.229 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000011626, 2024-05-20T16:26:04.229 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000247 2024-05-20T16:26:04.229 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-20T16:26:04.229 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_encrypted_bytes": 5120, 2024-05-20T16:26:04.229 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_encrypted_bytes": 388592 2024-05-20T16:26:04.229 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-20T16:26:04.229 INFO:teuthology.orchestra.run.smithi160.stdout: "cct": { 2024-05-20T16:26:04.229 INFO:teuthology.orchestra.run.smithi160.stdout: "total_workers": 1, 2024-05-20T16:26:04.229 INFO:teuthology.orchestra.run.smithi160.stdout: "unhealthy_workers": 0 2024-05-20T16:26:04.229 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-20T16:26:04.229 INFO:teuthology.orchestra.run.smithi160.stdout: "finisher-MDSRank": { 2024-05-20T16:26:04.229 INFO:teuthology.orchestra.run.smithi160.stdout: "queue_len": 0, 2024-05-20T16:26:04.230 INFO:teuthology.orchestra.run.smithi160.stdout: "complete_latency": { 2024-05-20T16:26:04.230 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 49, 2024-05-20T16:26:04.230 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.001548297, 2024-05-20T16:26:04.230 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000031597 2024-05-20T16:26:04.230 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-05-20T16:26:04.230 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-20T16:26:04.230 INFO:teuthology.orchestra.run.smithi160.stdout: "finisher-PurgeQueue": { 2024-05-20T16:26:04.230 INFO:teuthology.orchestra.run.smithi160.stdout: "queue_len": 0, 2024-05-20T16:26:04.230 INFO:teuthology.orchestra.run.smithi160.stdout: "complete_latency": { 2024-05-20T16:26:04.230 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 2, 2024-05-20T16:26:04.230 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000008469, 2024-05-20T16:26:04.230 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000004234 2024-05-20T16:26:04.231 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-05-20T16:26:04.231 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-20T16:26:04.231 INFO:teuthology.orchestra.run.smithi160.stdout: "mds": { 2024-05-20T16:26:04.231 INFO:teuthology.orchestra.run.smithi160.stdout: "request": 6, 2024-05-20T16:26:04.231 INFO:teuthology.orchestra.run.smithi160.stdout: "reply": 3, 2024-05-20T16:26:04.231 INFO:teuthology.orchestra.run.smithi160.stdout: "reply_latency": { 2024-05-20T16:26:04.231 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 3, 2024-05-20T16:26:04.231 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.001302147, 2024-05-20T16:26:04.231 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000434049 2024-05-20T16:26:04.231 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-20T16:26:04.231 INFO:teuthology.orchestra.run.smithi160.stdout: "slow_reply": 0, 2024-05-20T16:26:04.231 INFO:teuthology.orchestra.run.smithi160.stdout: "forward": 3, 2024-05-20T16:26:04.231 INFO:teuthology.orchestra.run.smithi160.stdout: "dir_fetch_complete": 0, 2024-05-20T16:26:04.231 INFO:teuthology.orchestra.run.smithi160.stdout: "dir_fetch_keys": 0, 2024-05-20T16:26:04.232 INFO:teuthology.orchestra.run.smithi160.stdout: "dir_commit": 11, 2024-05-20T16:26:04.232 INFO:teuthology.orchestra.run.smithi160.stdout: "dir_split": 0, 2024-05-20T16:26:04.232 INFO:teuthology.orchestra.run.smithi160.stdout: "dir_merge": 0, 2024-05-20T16:26:04.232 INFO:teuthology.orchestra.run.smithi160.stdout: "inodes": 14, 2024-05-20T16:26:04.232 INFO:teuthology.orchestra.run.smithi160.stdout: "inodes_top": 0, 2024-05-20T16:26:04.232 INFO:teuthology.orchestra.run.smithi160.stdout: "inodes_bottom": 14, 2024-05-20T16:26:04.232 INFO:teuthology.orchestra.run.smithi160.stdout: "inodes_pin_tail": 0, 2024-05-20T16:26:04.232 INFO:teuthology.orchestra.run.smithi160.stdout: "inodes_pinned": 14, 2024-05-20T16:26:04.232 INFO:teuthology.orchestra.run.smithi160.stdout: "inodes_expired": 0, 2024-05-20T16:26:04.232 INFO:teuthology.orchestra.run.smithi160.stdout: "inodes_with_caps": 2, 2024-05-20T16:26:04.232 INFO:teuthology.orchestra.run.smithi160.stdout: "caps": 2, 2024-05-20T16:26:04.232 INFO:teuthology.orchestra.run.smithi160.stdout: "subtrees": 2, 2024-05-20T16:26:04.232 INFO:teuthology.orchestra.run.smithi160.stdout: "traverse": 7, 2024-05-20T16:26:04.233 INFO:teuthology.orchestra.run.smithi160.stdout: "traverse_hit": 3, 2024-05-20T16:26:04.233 INFO:teuthology.orchestra.run.smithi160.stdout: "traverse_forward": 0, 2024-05-20T16:26:04.233 INFO:teuthology.orchestra.run.smithi160.stdout: "traverse_discover": 1, 2024-05-20T16:26:04.233 INFO:teuthology.orchestra.run.smithi160.stdout: "traverse_dir_fetch": 0, 2024-05-20T16:26:04.233 INFO:teuthology.orchestra.run.smithi160.stdout: "traverse_remote_ino": 0, 2024-05-20T16:26:04.233 INFO:teuthology.orchestra.run.smithi160.stdout: "traverse_lock": 0, 2024-05-20T16:26:04.233 INFO:teuthology.orchestra.run.smithi160.stdout: "load_cent": 0, 2024-05-20T16:26:04.233 INFO:teuthology.orchestra.run.smithi160.stdout: "q": 0, 2024-05-20T16:26:04.233 INFO:teuthology.orchestra.run.smithi160.stdout: "exported": 0, 2024-05-20T16:26:04.233 INFO:teuthology.orchestra.run.smithi160.stdout: "exported_inodes": 0, 2024-05-20T16:26:04.233 INFO:teuthology.orchestra.run.smithi160.stdout: "imported": 0, 2024-05-20T16:26:04.233 INFO:teuthology.orchestra.run.smithi160.stdout: "imported_inodes": 0, 2024-05-20T16:26:04.233 INFO:teuthology.orchestra.run.smithi160.stdout: "openino_dir_fetch": 0, 2024-05-20T16:26:04.233 INFO:teuthology.orchestra.run.smithi160.stdout: "openino_backtrace_fetch": 0, 2024-05-20T16:26:04.234 INFO:teuthology.orchestra.run.smithi160.stdout: "openino_peer_discover": 0, 2024-05-20T16:26:04.234 INFO:teuthology.orchestra.run.smithi160.stdout: "root_rfiles": 22, 2024-05-20T16:26:04.234 INFO:teuthology.orchestra.run.smithi160.stdout: "root_rbytes": 10738144091, 2024-05-20T16:26:04.234 INFO:teuthology.orchestra.run.smithi160.stdout: "root_rsnaps": 0, 2024-05-20T16:26:04.234 INFO:teuthology.orchestra.run.smithi160.stdout: "scrub_backtrace_fetch": 0, 2024-05-20T16:26:04.234 INFO:teuthology.orchestra.run.smithi160.stdout: "scrub_set_tag": 0, 2024-05-20T16:26:04.234 INFO:teuthology.orchestra.run.smithi160.stdout: "scrub_backtrace_repaired": 0, 2024-05-20T16:26:04.234 INFO:teuthology.orchestra.run.smithi160.stdout: "scrub_inotable_repaired": 0, 2024-05-20T16:26:04.234 INFO:teuthology.orchestra.run.smithi160.stdout: "scrub_dir_inodes": 0, 2024-05-20T16:26:04.234 INFO:teuthology.orchestra.run.smithi160.stdout: "scrub_dir_base_inodes": 0, 2024-05-20T16:26:04.234 INFO:teuthology.orchestra.run.smithi160.stdout: "scrub_dirfrag_rstats": 0, 2024-05-20T16:26:04.234 INFO:teuthology.orchestra.run.smithi160.stdout: "scrub_file_inodes": 0, 2024-05-20T16:26:04.235 INFO:teuthology.orchestra.run.smithi160.stdout: "handle_inode_file_caps": 0, 2024-05-20T16:26:04.235 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph_cap_op_revoke": 2, 2024-05-20T16:26:04.235 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph_cap_op_grant": 3, 2024-05-20T16:26:04.235 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph_cap_op_trunc": 0, 2024-05-20T16:26:04.235 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-20T16:26:04.235 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-20T16:26:04.235 INFO:teuthology.orchestra.run.smithi160.stdout: "handle_client_caps": 2, 2024-05-20T16:26:04.235 INFO:teuthology.orchestra.run.smithi160.stdout: "handle_client_caps_dirty": 0, 2024-05-20T16:26:04.235 INFO:teuthology.orchestra.run.smithi160.stdout: "handle_client_cap_release": 0, 2024-05-20T16:26:04.235 INFO:teuthology.orchestra.run.smithi160.stdout: "process_request_cap_release": 3 2024-05-20T16:26:04.235 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-20T16:26:04.235 INFO:teuthology.orchestra.run.smithi160.stdout: "mds_cache": { 2024-05-20T16:26:04.235 INFO:teuthology.orchestra.run.smithi160.stdout: "dir_update": 0, 2024-05-20T16:26:04.236 INFO:teuthology.orchestra.run.smithi160.stdout: "dir_update_receipt": 0, 2024-05-20T16:26:04.236 INFO:teuthology.orchestra.run.smithi160.stdout: "dir_try_discover": 0, 2024-05-20T16:26:04.236 INFO:teuthology.orchestra.run.smithi160.stdout: "dir_send_discover": 2, 2024-05-20T16:26:04.236 INFO:teuthology.orchestra.run.smithi160.stdout: "dir_handle_discover": 0, 2024-05-20T16:26:04.236 INFO:teuthology.orchestra.run.smithi160.stdout: "num_strays": 0, 2024-05-20T16:26:04.236 INFO:teuthology.orchestra.run.smithi160.stdout: "num_strays_delayed": 0, 2024-05-20T16:26:04.236 INFO:teuthology.orchestra.run.smithi160.stdout: "num_strays_enqueuing": 0, 2024-05-20T16:26:04.236 INFO:teuthology.orchestra.run.smithi160.stdout: "strays_created": 0, 2024-05-20T16:26:04.236 INFO:teuthology.orchestra.run.smithi160.stdout: "strays_enqueued": 0, 2024-05-20T16:26:04.236 INFO:teuthology.orchestra.run.smithi160.stdout: "strays_reintegrated": 0, 2024-05-20T16:26:04.236 INFO:teuthology.orchestra.run.smithi160.stdout: "strays_migrated": 0, 2024-05-20T16:26:04.236 INFO:teuthology.orchestra.run.smithi160.stdout: "num_recovering_processing": 0, 2024-05-20T16:26:04.236 INFO:teuthology.orchestra.run.smithi160.stdout: "num_recovering_enqueued": 0, 2024-05-20T16:26:04.236 INFO:teuthology.orchestra.run.smithi160.stdout: "num_recovering_prioritized": 0, 2024-05-20T16:26:04.237 INFO:teuthology.orchestra.run.smithi160.stdout: "recovery_started": 0, 2024-05-20T16:26:04.237 INFO:teuthology.orchestra.run.smithi160.stdout: "recovery_completed": 0, 2024-05-20T16:26:04.237 INFO:teuthology.orchestra.run.smithi160.stdout: "ireq_quiesce_path": 0, 2024-05-20T16:26:04.237 INFO:teuthology.orchestra.run.smithi160.stdout: "ireq_quiesce_inode": 0, 2024-05-20T16:26:04.237 INFO:teuthology.orchestra.run.smithi160.stdout: "ireq_enqueue_scrub": 0, 2024-05-20T16:26:04.237 INFO:teuthology.orchestra.run.smithi160.stdout: "ireq_exportdir": 0, 2024-05-20T16:26:04.237 INFO:teuthology.orchestra.run.smithi160.stdout: "ireq_flush": 0, 2024-05-20T16:26:04.237 INFO:teuthology.orchestra.run.smithi160.stdout: "ireq_fragmentdir": 0, 2024-05-20T16:26:04.237 INFO:teuthology.orchestra.run.smithi160.stdout: "ireq_fragstats": 0, 2024-05-20T16:26:04.237 INFO:teuthology.orchestra.run.smithi160.stdout: "ireq_inodestats": 0 2024-05-20T16:26:04.237 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-20T16:26:04.237 INFO:teuthology.orchestra.run.smithi160.stdout: "mds_log": { 2024-05-20T16:26:04.237 INFO:teuthology.orchestra.run.smithi160.stdout: "evlrg": 0, 2024-05-20T16:26:04.237 INFO:teuthology.orchestra.run.smithi160.stdout: "evadd": 5, 2024-05-20T16:26:04.238 INFO:teuthology.orchestra.run.smithi160.stdout: "evex": 0, 2024-05-20T16:26:04.238 INFO:teuthology.orchestra.run.smithi160.stdout: "evtrm": 0, 2024-05-20T16:26:04.238 INFO:teuthology.orchestra.run.smithi160.stdout: "ev": 5, 2024-05-20T16:26:04.238 INFO:teuthology.orchestra.run.smithi160.stdout: "evexg": 0, 2024-05-20T16:26:04.238 INFO:teuthology.orchestra.run.smithi160.stdout: "evexd": 0, 2024-05-20T16:26:04.238 INFO:teuthology.orchestra.run.smithi160.stdout: "segadd": 2, 2024-05-20T16:26:04.238 INFO:teuthology.orchestra.run.smithi160.stdout: "segex": 0, 2024-05-20T16:26:04.238 INFO:teuthology.orchestra.run.smithi160.stdout: "segtrm": 0, 2024-05-20T16:26:04.238 INFO:teuthology.orchestra.run.smithi160.stdout: "seg": 2, 2024-05-20T16:26:04.238 INFO:teuthology.orchestra.run.smithi160.stdout: "segmjr": 2, 2024-05-20T16:26:04.238 INFO:teuthology.orchestra.run.smithi160.stdout: "segexg": 0, 2024-05-20T16:26:04.238 INFO:teuthology.orchestra.run.smithi160.stdout: "segexd": 0, 2024-05-20T16:26:04.238 INFO:teuthology.orchestra.run.smithi160.stdout: "expos": 4194304, 2024-05-20T16:26:04.239 INFO:teuthology.orchestra.run.smithi160.stdout: "wrpos": 4195732, 2024-05-20T16:26:04.239 INFO:teuthology.orchestra.run.smithi160.stdout: "rdpos": 0, 2024-05-20T16:26:04.239 INFO:teuthology.orchestra.run.smithi160.stdout: "jlat": { 2024-05-20T16:26:04.239 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 4, 2024-05-20T16:26:04.239 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.007938114, 2024-05-20T16:26:04.239 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.001984528 2024-05-20T16:26:04.239 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-20T16:26:04.239 INFO:teuthology.orchestra.run.smithi160.stdout: "replayed": 0 2024-05-20T16:26:04.239 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-20T16:26:04.239 INFO:teuthology.orchestra.run.smithi160.stdout: "mds_mem": { 2024-05-20T16:26:04.239 INFO:teuthology.orchestra.run.smithi160.stdout: "ino": 17, 2024-05-20T16:26:04.239 INFO:teuthology.orchestra.run.smithi160.stdout: "ino+": 17, 2024-05-20T16:26:04.239 INFO:teuthology.orchestra.run.smithi160.stdout: "ino-": 0, 2024-05-20T16:26:04.239 INFO:teuthology.orchestra.run.smithi160.stdout: "dir": 15, 2024-05-20T16:26:04.240 INFO:teuthology.orchestra.run.smithi160.stdout: "dir+": 16, 2024-05-20T16:26:04.240 INFO:teuthology.orchestra.run.smithi160.stdout: "dir-": 1, 2024-05-20T16:26:04.240 INFO:teuthology.orchestra.run.smithi160.stdout: "dn": 14, 2024-05-20T16:26:04.240 INFO:teuthology.orchestra.run.smithi160.stdout: "dn+": 14, 2024-05-20T16:26:04.240 INFO:teuthology.orchestra.run.smithi160.stdout: "dn-": 0, 2024-05-20T16:26:04.240 INFO:teuthology.orchestra.run.smithi160.stdout: "cap": 2, 2024-05-20T16:26:04.240 INFO:teuthology.orchestra.run.smithi160.stdout: "cap+": 3, 2024-05-20T16:26:04.240 INFO:teuthology.orchestra.run.smithi160.stdout: "cap-": 1, 2024-05-20T16:26:04.240 INFO:teuthology.orchestra.run.smithi160.stdout: "rss": 120988, 2024-05-20T16:26:04.240 INFO:teuthology.orchestra.run.smithi160.stdout: "heap": 223636 2024-05-20T16:26:04.240 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-20T16:26:04.240 INFO:teuthology.orchestra.run.smithi160.stdout: "mds_server": { 2024-05-20T16:26:04.240 INFO:teuthology.orchestra.run.smithi160.stdout: "dispatch_client_request": 7, 2024-05-20T16:26:04.241 INFO:teuthology.orchestra.run.smithi160.stdout: "dispatch_server_request": 0, 2024-05-20T16:26:04.241 INFO:teuthology.orchestra.run.smithi160.stdout: "handle_client_request": 6, 2024-05-20T16:26:04.241 INFO:teuthology.orchestra.run.smithi160.stdout: "handle_client_session": 14, 2024-05-20T16:26:04.241 INFO:teuthology.orchestra.run.smithi160.stdout: "handle_peer_request": 0, 2024-05-20T16:26:04.241 INFO:teuthology.orchestra.run.smithi160.stdout: "req_create_latency": { 2024-05-20T16:26:04.241 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-20T16:26:04.241 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-20T16:26:04.241 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-20T16:26:04.241 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-20T16:26:04.241 INFO:teuthology.orchestra.run.smithi160.stdout: "req_getattr_latency": { 2024-05-20T16:26:04.241 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 2, 2024-05-20T16:26:04.241 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.001187591, 2024-05-20T16:26:04.241 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000593795 2024-05-20T16:26:04.242 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-20T16:26:04.242 INFO:teuthology.orchestra.run.smithi160.stdout: "req_getfilelock_latency": { 2024-05-20T16:26:04.242 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-20T16:26:04.242 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-20T16:26:04.242 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-20T16:26:04.242 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-20T16:26:04.242 INFO:teuthology.orchestra.run.smithi160.stdout: "req_link_latency": { 2024-05-20T16:26:04.242 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-20T16:26:04.242 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-20T16:26:04.242 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-20T16:26:04.242 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-20T16:26:04.243 INFO:teuthology.orchestra.run.smithi160.stdout: "req_lookup_latency": { 2024-05-20T16:26:04.243 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-20T16:26:04.243 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-20T16:26:04.243 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-20T16:26:04.243 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-20T16:26:04.243 INFO:teuthology.orchestra.run.smithi160.stdout: "req_lookuphash_latency": { 2024-05-20T16:26:04.243 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-20T16:26:04.243 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-20T16:26:04.243 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-20T16:26:04.243 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-20T16:26:04.243 INFO:teuthology.orchestra.run.smithi160.stdout: "req_lookupino_latency": { 2024-05-20T16:26:04.243 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 1, 2024-05-20T16:26:04.243 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000114556, 2024-05-20T16:26:04.243 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000114556 2024-05-20T16:26:04.243 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-20T16:26:04.244 INFO:teuthology.orchestra.run.smithi160.stdout: "req_lookupname_latency": { 2024-05-20T16:26:04.244 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-20T16:26:04.244 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-20T16:26:04.244 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-20T16:26:04.244 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-20T16:26:04.244 INFO:teuthology.orchestra.run.smithi160.stdout: "req_lookupparent_latency": { 2024-05-20T16:26:04.244 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-20T16:26:04.244 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-20T16:26:04.244 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-20T16:26:04.244 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-20T16:26:04.244 INFO:teuthology.orchestra.run.smithi160.stdout: "req_lookupsnap_latency": { 2024-05-20T16:26:04.244 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-20T16:26:04.244 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-20T16:26:04.245 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-20T16:26:04.245 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-20T16:26:04.245 INFO:teuthology.orchestra.run.smithi160.stdout: "req_lssnap_latency": { 2024-05-20T16:26:04.245 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-20T16:26:04.245 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-20T16:26:04.245 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-20T16:26:04.245 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-20T16:26:04.245 INFO:teuthology.orchestra.run.smithi160.stdout: "req_mkdir_latency": { 2024-05-20T16:26:04.245 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-20T16:26:04.245 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-20T16:26:04.246 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-20T16:26:04.246 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-20T16:26:04.246 INFO:teuthology.orchestra.run.smithi160.stdout: "req_mknod_latency": { 2024-05-20T16:26:04.246 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-20T16:26:04.246 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-20T16:26:04.246 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-20T16:26:04.246 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-20T16:26:04.246 INFO:teuthology.orchestra.run.smithi160.stdout: "req_mksnap_latency": { 2024-05-20T16:26:04.246 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-20T16:26:04.246 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-20T16:26:04.246 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-20T16:26:04.246 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-20T16:26:04.247 INFO:teuthology.orchestra.run.smithi160.stdout: "req_open_latency": { 2024-05-20T16:26:04.247 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-20T16:26:04.247 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-20T16:26:04.247 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-20T16:26:04.247 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-20T16:26:04.247 INFO:teuthology.orchestra.run.smithi160.stdout: "req_readdir_latency": { 2024-05-20T16:26:04.247 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-20T16:26:04.247 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-20T16:26:04.247 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-20T16:26:04.247 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-20T16:26:04.247 INFO:teuthology.orchestra.run.smithi160.stdout: "req_rename_latency": { 2024-05-20T16:26:04.247 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-20T16:26:04.247 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-20T16:26:04.247 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-20T16:26:04.248 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-20T16:26:04.248 INFO:teuthology.orchestra.run.smithi160.stdout: "req_renamesnap_latency": { 2024-05-20T16:26:04.248 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-20T16:26:04.248 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-20T16:26:04.248 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-20T16:26:04.248 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-20T16:26:04.248 INFO:teuthology.orchestra.run.smithi160.stdout: "req_snapdiff_latency": { 2024-05-20T16:26:04.248 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-20T16:26:04.248 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-20T16:26:04.248 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-20T16:26:04.248 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-20T16:26:04.248 INFO:teuthology.orchestra.run.smithi160.stdout: "req_rmdir_latency": { 2024-05-20T16:26:04.248 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-20T16:26:04.249 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-20T16:26:04.249 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-20T16:26:04.249 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-20T16:26:04.249 INFO:teuthology.orchestra.run.smithi160.stdout: "req_rmsnap_latency": { 2024-05-20T16:26:04.249 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-20T16:26:04.249 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-20T16:26:04.249 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-20T16:26:04.249 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-20T16:26:04.249 INFO:teuthology.orchestra.run.smithi160.stdout: "req_rmxattr_latency": { 2024-05-20T16:26:04.249 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-20T16:26:04.249 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-20T16:26:04.249 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-20T16:26:04.249 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-20T16:26:04.249 INFO:teuthology.orchestra.run.smithi160.stdout: "req_setattr_latency": { 2024-05-20T16:26:04.249 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-20T16:26:04.250 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-20T16:26:04.250 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-20T16:26:04.250 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-20T16:26:04.250 INFO:teuthology.orchestra.run.smithi160.stdout: "req_setdirlayout_latency": { 2024-05-20T16:26:04.250 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-20T16:26:04.250 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-20T16:26:04.250 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-20T16:26:04.250 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-20T16:26:04.250 INFO:teuthology.orchestra.run.smithi160.stdout: "req_setfilelock_latency": { 2024-05-20T16:26:04.250 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-20T16:26:04.250 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-20T16:26:04.250 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-20T16:26:04.250 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-20T16:26:04.250 INFO:teuthology.orchestra.run.smithi160.stdout: "req_setlayout_latency": { 2024-05-20T16:26:04.251 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-20T16:26:04.251 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-20T16:26:04.251 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-20T16:26:04.251 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-20T16:26:04.251 INFO:teuthology.orchestra.run.smithi160.stdout: "req_setxattr_latency": { 2024-05-20T16:26:04.251 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-20T16:26:04.251 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-20T16:26:04.251 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-20T16:26:04.251 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-20T16:26:04.251 INFO:teuthology.orchestra.run.smithi160.stdout: "req_symlink_latency": { 2024-05-20T16:26:04.251 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-20T16:26:04.251 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-20T16:26:04.251 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-20T16:26:04.251 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-20T16:26:04.252 INFO:teuthology.orchestra.run.smithi160.stdout: "req_unlink_latency": { 2024-05-20T16:26:04.252 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-20T16:26:04.252 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-20T16:26:04.252 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-20T16:26:04.252 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-20T16:26:04.252 INFO:teuthology.orchestra.run.smithi160.stdout: "cap_revoke_eviction": 0, 2024-05-20T16:26:04.252 INFO:teuthology.orchestra.run.smithi160.stdout: "cap_acquisition_throttle": 0, 2024-05-20T16:26:04.252 INFO:teuthology.orchestra.run.smithi160.stdout: "req_getvxattr_latency": { 2024-05-20T16:26:04.252 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-20T16:26:04.252 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-20T16:26:04.252 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-20T16:26:04.252 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-05-20T16:26:04.252 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-20T16:26:04.252 INFO:teuthology.orchestra.run.smithi160.stdout: "mds_sessions": { 2024-05-20T16:26:04.253 INFO:teuthology.orchestra.run.smithi160.stdout: "session_count": 1, 2024-05-20T16:26:04.253 INFO:teuthology.orchestra.run.smithi160.stdout: "session_add": 2, 2024-05-20T16:26:04.253 INFO:teuthology.orchestra.run.smithi160.stdout: "session_remove": 1, 2024-05-20T16:26:04.253 INFO:teuthology.orchestra.run.smithi160.stdout: "sessions_open": 1, 2024-05-20T16:26:04.253 INFO:teuthology.orchestra.run.smithi160.stdout: "sessions_stale": 0, 2024-05-20T16:26:04.253 INFO:teuthology.orchestra.run.smithi160.stdout: "total_load": 2, 2024-05-20T16:26:04.253 INFO:teuthology.orchestra.run.smithi160.stdout: "average_load": 2, 2024-05-20T16:26:04.253 INFO:teuthology.orchestra.run.smithi160.stdout: "avg_session_uptime": 716, 2024-05-20T16:26:04.253 INFO:teuthology.orchestra.run.smithi160.stdout: "mdthresh_evicted": 0 2024-05-20T16:26:04.253 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-20T16:26:04.253 INFO:teuthology.orchestra.run.smithi160.stdout: "mempool": { 2024-05-20T16:26:04.253 INFO:teuthology.orchestra.run.smithi160.stdout: "bloom_filter_bytes": 0, 2024-05-20T16:26:04.253 INFO:teuthology.orchestra.run.smithi160.stdout: "bloom_filter_items": 0, 2024-05-20T16:26:04.253 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_alloc_bytes": 0, 2024-05-20T16:26:04.253 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_alloc_items": 0, 2024-05-20T16:26:04.254 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_data_bytes": 0, 2024-05-20T16:26:04.254 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_data_items": 0, 2024-05-20T16:26:04.254 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-20T16:26:04.254 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_onode_items": 0, 2024-05-20T16:26:04.254 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-20T16:26:04.254 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_meta_items": 0, 2024-05-20T16:26:04.254 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_other_bytes": 0, 2024-05-20T16:26:04.254 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_other_items": 0, 2024-05-20T16:26:04.254 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-20T16:26:04.254 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_buffer_items": 0, 2024-05-20T16:26:04.254 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_extent_bytes": 0, 2024-05-20T16:26:04.254 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_extent_items": 0, 2024-05-20T16:26:04.255 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_blob_bytes": 0, 2024-05-20T16:26:04.255 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_blob_items": 0, 2024-05-20T16:26:04.255 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-20T16:26:04.255 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_shared_blob_items": 0, 2024-05-20T16:26:04.255 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-20T16:26:04.255 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_inline_bl_items": 0, 2024-05-20T16:26:04.255 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_fsck_bytes": 0, 2024-05-20T16:26:04.255 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_fsck_items": 0, 2024-05-20T16:26:04.255 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_txc_bytes": 0, 2024-05-20T16:26:04.255 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_txc_items": 0, 2024-05-20T16:26:04.255 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-20T16:26:04.255 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_writing_deferred_items": 0, 2024-05-20T16:26:04.255 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_writing_bytes": 0, 2024-05-20T16:26:04.255 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_writing_items": 0, 2024-05-20T16:26:04.255 INFO:teuthology.orchestra.run.smithi160.stdout: "bluefs_bytes": 0, 2024-05-20T16:26:04.255 INFO:teuthology.orchestra.run.smithi160.stdout: "bluefs_items": 0, 2024-05-20T16:26:04.256 INFO:teuthology.orchestra.run.smithi160.stdout: "bluefs_file_reader_bytes": 0, 2024-05-20T16:26:04.256 INFO:teuthology.orchestra.run.smithi160.stdout: "bluefs_file_reader_items": 0, 2024-05-20T16:26:04.256 INFO:teuthology.orchestra.run.smithi160.stdout: "bluefs_file_writer_bytes": 0, 2024-05-20T16:26:04.256 INFO:teuthology.orchestra.run.smithi160.stdout: "bluefs_file_writer_items": 0, 2024-05-20T16:26:04.256 INFO:teuthology.orchestra.run.smithi160.stdout: "buffer_anon_bytes": 239627, 2024-05-20T16:26:04.256 INFO:teuthology.orchestra.run.smithi160.stdout: "buffer_anon_items": 84, 2024-05-20T16:26:04.256 INFO:teuthology.orchestra.run.smithi160.stdout: "buffer_meta_bytes": 0, 2024-05-20T16:26:04.256 INFO:teuthology.orchestra.run.smithi160.stdout: "buffer_meta_items": 0, 2024-05-20T16:26:04.256 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_bytes": 0, 2024-05-20T16:26:04.256 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_items": 0, 2024-05-20T16:26:04.256 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_mapbl_bytes": 0, 2024-05-20T16:26:04.256 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_mapbl_items": 0, 2024-05-20T16:26:04.256 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_pglog_bytes": 0, 2024-05-20T16:26:04.256 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_pglog_items": 0, 2024-05-20T16:26:04.256 INFO:teuthology.orchestra.run.smithi160.stdout: "osdmap_bytes": 8888, 2024-05-20T16:26:04.257 INFO:teuthology.orchestra.run.smithi160.stdout: "osdmap_items": 170, 2024-05-20T16:26:04.257 INFO:teuthology.orchestra.run.smithi160.stdout: "osdmap_mapping_bytes": 0, 2024-05-20T16:26:04.257 INFO:teuthology.orchestra.run.smithi160.stdout: "osdmap_mapping_items": 0, 2024-05-20T16:26:04.257 INFO:teuthology.orchestra.run.smithi160.stdout: "pgmap_bytes": 0, 2024-05-20T16:26:04.257 INFO:teuthology.orchestra.run.smithi160.stdout: "pgmap_items": 0, 2024-05-20T16:26:04.257 INFO:teuthology.orchestra.run.smithi160.stdout: "mds_co_bytes": 68141, 2024-05-20T16:26:04.257 INFO:teuthology.orchestra.run.smithi160.stdout: "mds_co_items": 291, 2024-05-20T16:26:04.257 INFO:teuthology.orchestra.run.smithi160.stdout: "unittest_1_bytes": 0, 2024-05-20T16:26:04.257 INFO:teuthology.orchestra.run.smithi160.stdout: "unittest_1_items": 0, 2024-05-20T16:26:04.257 INFO:teuthology.orchestra.run.smithi160.stdout: "unittest_2_bytes": 0, 2024-05-20T16:26:04.257 INFO:teuthology.orchestra.run.smithi160.stdout: "unittest_2_items": 0 2024-05-20T16:26:04.257 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-20T16:26:04.257 INFO:teuthology.orchestra.run.smithi160.stdout: "objecter": { 2024-05-20T16:26:04.257 INFO:teuthology.orchestra.run.smithi160.stdout: "op_active": 0, 2024-05-20T16:26:04.257 INFO:teuthology.orchestra.run.smithi160.stdout: "op_laggy": 0, 2024-05-20T16:26:04.258 INFO:teuthology.orchestra.run.smithi160.stdout: "op_send": 42, 2024-05-20T16:26:04.258 INFO:teuthology.orchestra.run.smithi160.stdout: "op_send_bytes": 12135, 2024-05-20T16:26:04.258 INFO:teuthology.orchestra.run.smithi160.stdout: "op_resend": 33, 2024-05-20T16:26:04.258 INFO:teuthology.orchestra.run.smithi160.stdout: "op_reply": 42, 2024-05-20T16:26:04.258 INFO:teuthology.orchestra.run.smithi160.stdout: "op_latency": { 2024-05-20T16:26:04.258 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 42, 2024-05-20T16:26:04.258 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.247996387, 2024-05-20T16:26:04.258 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.005904675 2024-05-20T16:26:04.258 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-20T16:26:04.258 INFO:teuthology.orchestra.run.smithi160.stdout: "op_inflight": 0, 2024-05-20T16:26:04.258 INFO:teuthology.orchestra.run.smithi160.stdout: "oplen_avg": { 2024-05-20T16:26:04.258 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 42, 2024-05-20T16:26:04.258 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 66 2024-05-20T16:26:04.259 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-20T16:26:04.259 INFO:teuthology.orchestra.run.smithi160.stdout: "op": 42, 2024-05-20T16:26:04.259 INFO:teuthology.orchestra.run.smithi160.stdout: "op_r": 0, 2024-05-20T16:26:04.259 INFO:teuthology.orchestra.run.smithi160.stdout: "op_w": 42, 2024-05-20T16:26:04.259 INFO:teuthology.orchestra.run.smithi160.stdout: "op_rmw": 0, 2024-05-20T16:26:04.259 INFO:teuthology.orchestra.run.smithi160.stdout: "op_pg": 0, 2024-05-20T16:26:04.259 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_stat": 0, 2024-05-20T16:26:04.259 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_create": 10, 2024-05-20T16:26:04.259 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_read": 0, 2024-05-20T16:26:04.259 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_write": 4, 2024-05-20T16:26:04.259 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_writefull": 7, 2024-05-20T16:26:04.259 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_writesame": 0, 2024-05-20T16:26:04.259 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_append": 0, 2024-05-20T16:26:04.260 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_zero": 0, 2024-05-20T16:26:04.260 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_truncate": 0, 2024-05-20T16:26:04.260 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_delete": 6, 2024-05-20T16:26:04.260 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_mapext": 0, 2024-05-20T16:26:04.260 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_sparse_read": 0, 2024-05-20T16:26:04.260 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_clonerange": 0, 2024-05-20T16:26:04.260 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_getxattr": 0, 2024-05-20T16:26:04.260 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_setxattr": 20, 2024-05-20T16:26:04.260 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_cmpxattr": 0, 2024-05-20T16:26:04.260 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_rmxattr": 0, 2024-05-20T16:26:04.260 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_resetxattrs": 0, 2024-05-20T16:26:04.260 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_call": 0, 2024-05-20T16:26:04.261 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_watch": 0, 2024-05-20T16:26:04.261 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_notify": 0, 2024-05-20T16:26:04.261 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_src_cmpxattr": 0, 2024-05-20T16:26:04.261 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_pgls": 0, 2024-05-20T16:26:04.261 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_pgls_filter": 0, 2024-05-20T16:26:04.261 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_other": 0, 2024-05-20T16:26:04.261 INFO:teuthology.orchestra.run.smithi160.stdout: "linger_active": 0, 2024-05-20T16:26:04.261 INFO:teuthology.orchestra.run.smithi160.stdout: "linger_send": 0, 2024-05-20T16:26:04.261 INFO:teuthology.orchestra.run.smithi160.stdout: "linger_resend": 0, 2024-05-20T16:26:04.261 INFO:teuthology.orchestra.run.smithi160.stdout: "linger_ping": 0, 2024-05-20T16:26:04.261 INFO:teuthology.orchestra.run.smithi160.stdout: "poolop_active": 0, 2024-05-20T16:26:04.261 INFO:teuthology.orchestra.run.smithi160.stdout: "poolop_send": 0, 2024-05-20T16:26:04.261 INFO:teuthology.orchestra.run.smithi160.stdout: "poolop_resend": 0, 2024-05-20T16:26:04.262 INFO:teuthology.orchestra.run.smithi160.stdout: "poolstat_active": 0, 2024-05-20T16:26:04.262 INFO:teuthology.orchestra.run.smithi160.stdout: "poolstat_send": 0, 2024-05-20T16:26:04.262 INFO:teuthology.orchestra.run.smithi160.stdout: "poolstat_resend": 0, 2024-05-20T16:26:04.262 INFO:teuthology.orchestra.run.smithi160.stdout: "statfs_active": 0, 2024-05-20T16:26:04.262 INFO:teuthology.orchestra.run.smithi160.stdout: "statfs_send": 0, 2024-05-20T16:26:04.262 INFO:teuthology.orchestra.run.smithi160.stdout: "statfs_resend": 0, 2024-05-20T16:26:04.262 INFO:teuthology.orchestra.run.smithi160.stdout: "command_active": 0, 2024-05-20T16:26:04.262 INFO:teuthology.orchestra.run.smithi160.stdout: "command_send": 0, 2024-05-20T16:26:04.262 INFO:teuthology.orchestra.run.smithi160.stdout: "command_resend": 0, 2024-05-20T16:26:04.262 INFO:teuthology.orchestra.run.smithi160.stdout: "map_epoch": 98, 2024-05-20T16:26:04.262 INFO:teuthology.orchestra.run.smithi160.stdout: "map_full": 0, 2024-05-20T16:26:04.262 INFO:teuthology.orchestra.run.smithi160.stdout: "map_inc": 7, 2024-05-20T16:26:04.263 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_sessions": 8, 2024-05-20T16:26:04.263 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_session_open": 8, 2024-05-20T16:26:04.263 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_session_close": 0, 2024-05-20T16:26:04.263 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_laggy": 0, 2024-05-20T16:26:04.263 INFO:teuthology.orchestra.run.smithi160.stdout: "omap_wr": 18, 2024-05-20T16:26:04.263 INFO:teuthology.orchestra.run.smithi160.stdout: "omap_rd": 0, 2024-05-20T16:26:04.263 INFO:teuthology.orchestra.run.smithi160.stdout: "omap_del": 1 2024-05-20T16:26:04.263 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-20T16:26:04.263 INFO:teuthology.orchestra.run.smithi160.stdout: "oft": { 2024-05-20T16:26:04.263 INFO:teuthology.orchestra.run.smithi160.stdout: "omap_total_objs": 1, 2024-05-20T16:26:04.263 INFO:teuthology.orchestra.run.smithi160.stdout: "omap_total_kv_pairs": 0, 2024-05-20T16:26:04.263 INFO:teuthology.orchestra.run.smithi160.stdout: "omap_total_updates": 0, 2024-05-20T16:26:04.263 INFO:teuthology.orchestra.run.smithi160.stdout: "omap_total_removes": 0 2024-05-20T16:26:04.264 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-20T16:26:04.264 INFO:teuthology.orchestra.run.smithi160.stdout: "osd-slow-ops": { 2024-05-20T16:26:04.264 INFO:teuthology.orchestra.run.smithi160.stdout: "slow_ops_count": 0 2024-05-20T16:26:04.264 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-20T16:26:04.264 INFO:teuthology.orchestra.run.smithi160.stdout: "purge_queue": { 2024-05-20T16:26:04.264 INFO:teuthology.orchestra.run.smithi160.stdout: "pq_executing_ops": 0, 2024-05-20T16:26:04.264 INFO:teuthology.orchestra.run.smithi160.stdout: "pq_executing_ops_high_water": 0, 2024-05-20T16:26:04.264 INFO:teuthology.orchestra.run.smithi160.stdout: "pq_executing": 0, 2024-05-20T16:26:04.264 INFO:teuthology.orchestra.run.smithi160.stdout: "pq_executing_high_water": 0, 2024-05-20T16:26:04.264 INFO:teuthology.orchestra.run.smithi160.stdout: "pq_executed_ops": 0, 2024-05-20T16:26:04.264 INFO:teuthology.orchestra.run.smithi160.stdout: "pq_executed": 0, 2024-05-20T16:26:04.264 INFO:teuthology.orchestra.run.smithi160.stdout: "pq_item_in_journal": 0 2024-05-20T16:26:04.264 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-20T16:26:04.264 INFO:teuthology.orchestra.run.smithi160.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-20T16:26:04.264 INFO:teuthology.orchestra.run.smithi160.stdout: "val": 0, 2024-05-20T16:26:04.264 INFO:teuthology.orchestra.run.smithi160.stdout: "max": 104857600, 2024-05-20T16:26:04.265 INFO:teuthology.orchestra.run.smithi160.stdout: "get_started": 0, 2024-05-20T16:26:04.265 INFO:teuthology.orchestra.run.smithi160.stdout: "get": 1375, 2024-05-20T16:26:04.265 INFO:teuthology.orchestra.run.smithi160.stdout: "get_sum": 939671, 2024-05-20T16:26:04.265 INFO:teuthology.orchestra.run.smithi160.stdout: "get_or_fail_fail": 0, 2024-05-20T16:26:04.265 INFO:teuthology.orchestra.run.smithi160.stdout: "get_or_fail_success": 1375, 2024-05-20T16:26:04.265 INFO:teuthology.orchestra.run.smithi160.stdout: "take": 0, 2024-05-20T16:26:04.265 INFO:teuthology.orchestra.run.smithi160.stdout: "take_sum": 0, 2024-05-20T16:26:04.265 INFO:teuthology.orchestra.run.smithi160.stdout: "put": 1375, 2024-05-20T16:26:04.265 INFO:teuthology.orchestra.run.smithi160.stdout: "put_sum": 939671, 2024-05-20T16:26:04.265 INFO:teuthology.orchestra.run.smithi160.stdout: "wait": { 2024-05-20T16:26:04.265 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-20T16:26:04.265 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-20T16:26:04.265 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-20T16:26:04.265 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-05-20T16:26:04.265 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-20T16:26:04.266 INFO:teuthology.orchestra.run.smithi160.stdout: "throttle-objecter_bytes": { 2024-05-20T16:26:04.266 INFO:teuthology.orchestra.run.smithi160.stdout: "val": 0, 2024-05-20T16:26:04.266 INFO:teuthology.orchestra.run.smithi160.stdout: "max": 104857600, 2024-05-20T16:26:04.266 INFO:teuthology.orchestra.run.smithi160.stdout: "get_started": 0, 2024-05-20T16:26:04.266 INFO:teuthology.orchestra.run.smithi160.stdout: "get": 0, 2024-05-20T16:26:04.266 INFO:teuthology.orchestra.run.smithi160.stdout: "get_sum": 0, 2024-05-20T16:26:04.266 INFO:teuthology.orchestra.run.smithi160.stdout: "get_or_fail_fail": 0, 2024-05-20T16:26:04.266 INFO:teuthology.orchestra.run.smithi160.stdout: "get_or_fail_success": 0, 2024-05-20T16:26:04.266 INFO:teuthology.orchestra.run.smithi160.stdout: "take": 42, 2024-05-20T16:26:04.266 INFO:teuthology.orchestra.run.smithi160.stdout: "take_sum": 12135, 2024-05-20T16:26:04.266 INFO:teuthology.orchestra.run.smithi160.stdout: "put": 42, 2024-05-20T16:26:04.266 INFO:teuthology.orchestra.run.smithi160.stdout: "put_sum": 12135, 2024-05-20T16:26:04.266 INFO:teuthology.orchestra.run.smithi160.stdout: "wait": { 2024-05-20T16:26:04.266 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-20T16:26:04.266 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-20T16:26:04.267 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-20T16:26:04.267 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-05-20T16:26:04.267 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-20T16:26:04.267 INFO:teuthology.orchestra.run.smithi160.stdout: "throttle-objecter_ops": { 2024-05-20T16:26:04.267 INFO:teuthology.orchestra.run.smithi160.stdout: "val": 0, 2024-05-20T16:26:04.267 INFO:teuthology.orchestra.run.smithi160.stdout: "max": 1024, 2024-05-20T16:26:04.267 INFO:teuthology.orchestra.run.smithi160.stdout: "get_started": 0, 2024-05-20T16:26:04.267 INFO:teuthology.orchestra.run.smithi160.stdout: "get": 0, 2024-05-20T16:26:04.267 INFO:teuthology.orchestra.run.smithi160.stdout: "get_sum": 0, 2024-05-20T16:26:04.267 INFO:teuthology.orchestra.run.smithi160.stdout: "get_or_fail_fail": 0, 2024-05-20T16:26:04.267 INFO:teuthology.orchestra.run.smithi160.stdout: "get_or_fail_success": 0, 2024-05-20T16:26:04.267 INFO:teuthology.orchestra.run.smithi160.stdout: "take": 42, 2024-05-20T16:26:04.267 INFO:teuthology.orchestra.run.smithi160.stdout: "take_sum": 42, 2024-05-20T16:26:04.267 INFO:teuthology.orchestra.run.smithi160.stdout: "put": 42, 2024-05-20T16:26:04.268 INFO:teuthology.orchestra.run.smithi160.stdout: "put_sum": 42, 2024-05-20T16:26:04.268 INFO:teuthology.orchestra.run.smithi160.stdout: "wait": { 2024-05-20T16:26:04.268 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-20T16:26:04.268 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-20T16:26:04.268 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-20T16:26:04.268 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-05-20T16:26:04.268 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-20T16:26:04.268 INFO:teuthology.orchestra.run.smithi160.stdout: "throttle-write_buf_throttle": { 2024-05-20T16:26:04.268 INFO:teuthology.orchestra.run.smithi160.stdout: "val": 0, 2024-05-20T16:26:04.268 INFO:teuthology.orchestra.run.smithi160.stdout: "max": 3758096384, 2024-05-20T16:26:04.268 INFO:teuthology.orchestra.run.smithi160.stdout: "get_started": 0, 2024-05-20T16:26:04.268 INFO:teuthology.orchestra.run.smithi160.stdout: "get": 0, 2024-05-20T16:26:04.269 INFO:teuthology.orchestra.run.smithi160.stdout: "get_sum": 0, 2024-05-20T16:26:04.269 INFO:teuthology.orchestra.run.smithi160.stdout: "get_or_fail_fail": 0, 2024-05-20T16:26:04.269 INFO:teuthology.orchestra.run.smithi160.stdout: "get_or_fail_success": 0, 2024-05-20T16:26:04.269 INFO:teuthology.orchestra.run.smithi160.stdout: "take": 0, 2024-05-20T16:26:04.269 INFO:teuthology.orchestra.run.smithi160.stdout: "take_sum": 0, 2024-05-20T16:26:04.269 INFO:teuthology.orchestra.run.smithi160.stdout: "put": 0, 2024-05-20T16:26:04.269 INFO:teuthology.orchestra.run.smithi160.stdout: "put_sum": 0, 2024-05-20T16:26:04.269 INFO:teuthology.orchestra.run.smithi160.stdout: "wait": { 2024-05-20T16:26:04.269 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-20T16:26:04.269 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-20T16:26:04.269 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-20T16:26:04.269 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-05-20T16:26:04.269 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-20T16:26:04.269 INFO:teuthology.orchestra.run.smithi160.stdout: "throttle-write_buf_throttle-0x55db42eecb60": { 2024-05-20T16:26:04.269 INFO:teuthology.orchestra.run.smithi160.stdout: "val": 0, 2024-05-20T16:26:04.270 INFO:teuthology.orchestra.run.smithi160.stdout: "max": 3758096384, 2024-05-20T16:26:04.270 INFO:teuthology.orchestra.run.smithi160.stdout: "get_started": 0, 2024-05-20T16:26:04.270 INFO:teuthology.orchestra.run.smithi160.stdout: "get": 5, 2024-05-20T16:26:04.270 INFO:teuthology.orchestra.run.smithi160.stdout: "get_sum": 1428, 2024-05-20T16:26:04.270 INFO:teuthology.orchestra.run.smithi160.stdout: "get_or_fail_fail": 0, 2024-05-20T16:26:04.270 INFO:teuthology.orchestra.run.smithi160.stdout: "get_or_fail_success": 5, 2024-05-20T16:26:04.270 INFO:teuthology.orchestra.run.smithi160.stdout: "take": 0, 2024-05-20T16:26:04.270 INFO:teuthology.orchestra.run.smithi160.stdout: "take_sum": 0, 2024-05-20T16:26:04.270 INFO:teuthology.orchestra.run.smithi160.stdout: "put": 4, 2024-05-20T16:26:04.270 INFO:teuthology.orchestra.run.smithi160.stdout: "put_sum": 1428, 2024-05-20T16:26:04.270 INFO:teuthology.orchestra.run.smithi160.stdout: "wait": { 2024-05-20T16:26:04.270 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-20T16:26:04.270 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-20T16:26:04.270 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-20T16:26:04.270 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-05-20T16:26:04.271 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-05-20T16:26:04.271 INFO:teuthology.orchestra.run.smithi160.stdout:} 2024-05-20T16:26:04.358 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:26:04 smithi082 ceph-mon[32238]: pgmap v727: 129 pgs: 129 active+clean; 10 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 74 MiB/s wr, 139 op/s 2024-05-20T16:26:04.385 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:26:04 smithi184 ceph-mon[36209]: pgmap v727: 129 pgs: 129 active+clean; 10 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 74 MiB/s wr, 139 op/s 2024-05-20T16:26:05.033 INFO:tasks.check_counter:Daemon mds.h mds.exported=0 2024-05-20T16:26:05.034 INFO:tasks.check_counter:Daemon mds.h mds.imported=0 2024-05-20T16:26:05.034 DEBUG:tasks.check_counter:Getting stats from k 2024-05-20T16:26:05.034 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell --fsid 59d1600a-16c2-11ef-bc9a-c7b262605968 -- ceph daemon mds.k perf dump 2024-05-20T16:26:05.273 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/mon.b/config 2024-05-20T16:26:07.062 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:26:06 smithi160 ceph-mon[36347]: pgmap v728: 129 pgs: 129 active+clean; 10 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 68 MiB/s wr, 125 op/s 2024-05-20T16:26:07.082 INFO:teuthology.orchestra.run.smithi160.stdout:{ 2024-05-20T16:26:07.082 INFO:teuthology.orchestra.run.smithi160.stdout: "AsyncMessenger::Worker-0": { 2024-05-20T16:26:07.082 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_messages": 244, 2024-05-20T16:26:07.082 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_messages": 239, 2024-05-20T16:26:07.082 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_bytes": 760512, 2024-05-20T16:26:07.082 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_bytes": 147264, 2024-05-20T16:26:07.082 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_created_connections": 4, 2024-05-20T16:26:07.083 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_active_connections": 0, 2024-05-20T16:26:07.083 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_total_time": 0.043845039, 2024-05-20T16:26:07.083 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_send_time": 0.014544797, 2024-05-20T16:26:07.083 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_recv_time": 0.024268828, 2024-05-20T16:26:07.083 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_fast_dispatch_time": 0.007287393, 2024-05-20T16:26:07.083 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_messages_queue_lat": { 2024-05-20T16:26:07.083 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 238, 2024-05-20T16:26:07.083 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.014683663, 2024-05-20T16:26:07.083 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000061696 2024-05-20T16:26:07.083 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-20T16:26:07.083 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_handle_ack_lat": { 2024-05-20T16:26:07.083 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-20T16:26:07.083 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-20T16:26:07.083 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-20T16:26:07.083 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-20T16:26:07.084 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_encrypted_bytes": 760512, 2024-05-20T16:26:07.084 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_encrypted_bytes": 147264 2024-05-20T16:26:07.084 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-20T16:26:07.084 INFO:teuthology.orchestra.run.smithi160.stdout: "AsyncMessenger::Worker-1": { 2024-05-20T16:26:07.084 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_messages": 1, 2024-05-20T16:26:07.084 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_messages": 182, 2024-05-20T16:26:07.084 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_bytes": 160, 2024-05-20T16:26:07.084 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_bytes": 109728, 2024-05-20T16:26:07.084 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_created_connections": 3, 2024-05-20T16:26:07.084 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_active_connections": 2, 2024-05-20T16:26:07.084 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_total_time": 0.010558622, 2024-05-20T16:26:07.084 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_send_time": 0.007599367, 2024-05-20T16:26:07.084 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_recv_time": 0.001038229, 2024-05-20T16:26:07.084 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-20T16:26:07.084 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_messages_queue_lat": { 2024-05-20T16:26:07.085 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 180, 2024-05-20T16:26:07.085 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.016612478, 2024-05-20T16:26:07.085 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000092291 2024-05-20T16:26:07.085 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-20T16:26:07.085 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_handle_ack_lat": { 2024-05-20T16:26:07.085 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-20T16:26:07.085 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-20T16:26:07.085 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-20T16:26:07.085 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-20T16:26:07.085 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_encrypted_bytes": 160, 2024-05-20T16:26:07.085 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_encrypted_bytes": 109728 2024-05-20T16:26:07.085 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-20T16:26:07.085 INFO:teuthology.orchestra.run.smithi160.stdout: "AsyncMessenger::Worker-2": { 2024-05-20T16:26:07.086 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_messages": 4, 2024-05-20T16:26:07.086 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_messages": 4, 2024-05-20T16:26:07.086 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_bytes": 3024, 2024-05-20T16:26:07.086 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_bytes": 448, 2024-05-20T16:26:07.086 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_created_connections": 2, 2024-05-20T16:26:07.086 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_active_connections": 1, 2024-05-20T16:26:07.086 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_total_time": 0.000727508, 2024-05-20T16:26:07.086 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_send_time": 0.000031273, 2024-05-20T16:26:07.086 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_recv_time": 0.000657255, 2024-05-20T16:26:07.086 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-20T16:26:07.086 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_messages_queue_lat": { 2024-05-20T16:26:07.086 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 3, 2024-05-20T16:26:07.086 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000792292, 2024-05-20T16:26:07.086 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000264097 2024-05-20T16:26:07.087 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-20T16:26:07.087 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_handle_ack_lat": { 2024-05-20T16:26:07.087 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-20T16:26:07.087 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-20T16:26:07.087 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-20T16:26:07.087 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-20T16:26:07.087 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_encrypted_bytes": 3024, 2024-05-20T16:26:07.087 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_encrypted_bytes": 448 2024-05-20T16:26:07.087 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-20T16:26:07.087 INFO:teuthology.orchestra.run.smithi160.stdout: "cct": { 2024-05-20T16:26:07.087 INFO:teuthology.orchestra.run.smithi160.stdout: "total_workers": 0, 2024-05-20T16:26:07.087 INFO:teuthology.orchestra.run.smithi160.stdout: "unhealthy_workers": 0 2024-05-20T16:26:07.087 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-20T16:26:07.087 INFO:teuthology.orchestra.run.smithi160.stdout: "mempool": { 2024-05-20T16:26:07.088 INFO:teuthology.orchestra.run.smithi160.stdout: "bloom_filter_bytes": 0, 2024-05-20T16:26:07.088 INFO:teuthology.orchestra.run.smithi160.stdout: "bloom_filter_items": 0, 2024-05-20T16:26:07.088 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_alloc_bytes": 0, 2024-05-20T16:26:07.088 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_alloc_items": 0, 2024-05-20T16:26:07.088 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_data_bytes": 0, 2024-05-20T16:26:07.088 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_data_items": 0, 2024-05-20T16:26:07.088 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-20T16:26:07.088 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_onode_items": 0, 2024-05-20T16:26:07.088 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-20T16:26:07.088 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_meta_items": 0, 2024-05-20T16:26:07.088 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_other_bytes": 0, 2024-05-20T16:26:07.088 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_other_items": 0, 2024-05-20T16:26:07.088 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-20T16:26:07.088 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_buffer_items": 0, 2024-05-20T16:26:07.088 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_extent_bytes": 0, 2024-05-20T16:26:07.088 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_extent_items": 0, 2024-05-20T16:26:07.089 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_blob_bytes": 0, 2024-05-20T16:26:07.089 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_blob_items": 0, 2024-05-20T16:26:07.089 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-20T16:26:07.089 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_shared_blob_items": 0, 2024-05-20T16:26:07.089 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-20T16:26:07.089 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_inline_bl_items": 0, 2024-05-20T16:26:07.089 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_fsck_bytes": 0, 2024-05-20T16:26:07.089 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_fsck_items": 0, 2024-05-20T16:26:07.089 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_txc_bytes": 0, 2024-05-20T16:26:07.089 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_txc_items": 0, 2024-05-20T16:26:07.089 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-20T16:26:07.089 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_writing_deferred_items": 0, 2024-05-20T16:26:07.089 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_writing_bytes": 0, 2024-05-20T16:26:07.089 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_writing_items": 0, 2024-05-20T16:26:07.089 INFO:teuthology.orchestra.run.smithi160.stdout: "bluefs_bytes": 0, 2024-05-20T16:26:07.090 INFO:teuthology.orchestra.run.smithi160.stdout: "bluefs_items": 0, 2024-05-20T16:26:07.090 INFO:teuthology.orchestra.run.smithi160.stdout: "bluefs_file_reader_bytes": 0, 2024-05-20T16:26:07.090 INFO:teuthology.orchestra.run.smithi160.stdout: "bluefs_file_reader_items": 0, 2024-05-20T16:26:07.090 INFO:teuthology.orchestra.run.smithi160.stdout: "bluefs_file_writer_bytes": 0, 2024-05-20T16:26:07.090 INFO:teuthology.orchestra.run.smithi160.stdout: "bluefs_file_writer_items": 0, 2024-05-20T16:26:07.090 INFO:teuthology.orchestra.run.smithi160.stdout: "buffer_anon_bytes": 148967, 2024-05-20T16:26:07.090 INFO:teuthology.orchestra.run.smithi160.stdout: "buffer_anon_items": 29, 2024-05-20T16:26:07.090 INFO:teuthology.orchestra.run.smithi160.stdout: "buffer_meta_bytes": 0, 2024-05-20T16:26:07.090 INFO:teuthology.orchestra.run.smithi160.stdout: "buffer_meta_items": 0, 2024-05-20T16:26:07.090 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_bytes": 0, 2024-05-20T16:26:07.090 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_items": 0, 2024-05-20T16:26:07.090 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_mapbl_bytes": 0, 2024-05-20T16:26:07.090 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_mapbl_items": 0, 2024-05-20T16:26:07.091 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_pglog_bytes": 0, 2024-05-20T16:26:07.091 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_pglog_items": 0, 2024-05-20T16:26:07.091 INFO:teuthology.orchestra.run.smithi160.stdout: "osdmap_bytes": 0, 2024-05-20T16:26:07.091 INFO:teuthology.orchestra.run.smithi160.stdout: "osdmap_items": 0, 2024-05-20T16:26:07.091 INFO:teuthology.orchestra.run.smithi160.stdout: "osdmap_mapping_bytes": 0, 2024-05-20T16:26:07.091 INFO:teuthology.orchestra.run.smithi160.stdout: "osdmap_mapping_items": 0, 2024-05-20T16:26:07.091 INFO:teuthology.orchestra.run.smithi160.stdout: "pgmap_bytes": 0, 2024-05-20T16:26:07.091 INFO:teuthology.orchestra.run.smithi160.stdout: "pgmap_items": 0, 2024-05-20T16:26:07.091 INFO:teuthology.orchestra.run.smithi160.stdout: "mds_co_bytes": 928, 2024-05-20T16:26:07.091 INFO:teuthology.orchestra.run.smithi160.stdout: "mds_co_items": 2, 2024-05-20T16:26:07.091 INFO:teuthology.orchestra.run.smithi160.stdout: "unittest_1_bytes": 0, 2024-05-20T16:26:07.091 INFO:teuthology.orchestra.run.smithi160.stdout: "unittest_1_items": 0, 2024-05-20T16:26:07.091 INFO:teuthology.orchestra.run.smithi160.stdout: "unittest_2_bytes": 0, 2024-05-20T16:26:07.092 INFO:teuthology.orchestra.run.smithi160.stdout: "unittest_2_items": 0 2024-05-20T16:26:07.092 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-20T16:26:07.092 INFO:teuthology.orchestra.run.smithi160.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-20T16:26:07.092 INFO:teuthology.orchestra.run.smithi160.stdout: "val": 0, 2024-05-20T16:26:07.092 INFO:teuthology.orchestra.run.smithi160.stdout: "max": 104857600, 2024-05-20T16:26:07.092 INFO:teuthology.orchestra.run.smithi160.stdout: "get_started": 0, 2024-05-20T16:26:07.092 INFO:teuthology.orchestra.run.smithi160.stdout: "get": 245, 2024-05-20T16:26:07.092 INFO:teuthology.orchestra.run.smithi160.stdout: "get_sum": 726042, 2024-05-20T16:26:07.092 INFO:teuthology.orchestra.run.smithi160.stdout: "get_or_fail_fail": 0, 2024-05-20T16:26:07.092 INFO:teuthology.orchestra.run.smithi160.stdout: "get_or_fail_success": 245, 2024-05-20T16:26:07.092 INFO:teuthology.orchestra.run.smithi160.stdout: "take": 0, 2024-05-20T16:26:07.092 INFO:teuthology.orchestra.run.smithi160.stdout: "take_sum": 0, 2024-05-20T16:26:07.092 INFO:teuthology.orchestra.run.smithi160.stdout: "put": 245, 2024-05-20T16:26:07.092 INFO:teuthology.orchestra.run.smithi160.stdout: "put_sum": 726042, 2024-05-20T16:26:07.093 INFO:teuthology.orchestra.run.smithi160.stdout: "wait": { 2024-05-20T16:26:07.093 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-20T16:26:07.093 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-20T16:26:07.093 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-20T16:26:07.093 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-05-20T16:26:07.093 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-05-20T16:26:07.093 INFO:teuthology.orchestra.run.smithi160.stdout:} 2024-05-20T16:26:07.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:26:06 smithi082 ceph-mon[32238]: pgmap v728: 129 pgs: 129 active+clean; 10 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 68 MiB/s wr, 125 op/s 2024-05-20T16:26:07.385 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:26:06 smithi184 ceph-mon[36209]: pgmap v728: 129 pgs: 129 active+clean; 10 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 68 MiB/s wr, 125 op/s 2024-05-20T16:26:07.802 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.k 2024-05-20T16:26:07.802 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.k 2024-05-20T16:26:07.802 DEBUG:tasks.check_counter:Getting stats from c 2024-05-20T16:26:07.802 DEBUG:teuthology.orchestra.run.smithi184:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell --fsid 59d1600a-16c2-11ef-bc9a-c7b262605968 -- ceph daemon mds.c perf dump 2024-05-20T16:26:08.121 INFO:teuthology.orchestra.run.smithi184.stderr:Inferring config /var/lib/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/mon.c/config 2024-05-20T16:26:09.300 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:26:08 smithi184 ceph-mon[36209]: pgmap v729: 129 pgs: 129 active+clean; 10 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 65 MiB/s wr, 119 op/s 2024-05-20T16:26:09.312 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:26:08 smithi160 ceph-mon[36347]: pgmap v729: 129 pgs: 129 active+clean; 10 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 65 MiB/s wr, 119 op/s 2024-05-20T16:26:09.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:26:08 smithi082 ceph-mon[32238]: pgmap v729: 129 pgs: 129 active+clean; 10 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 65 MiB/s wr, 119 op/s 2024-05-20T16:26:10.076 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-05-20T16:26:10.076 INFO:teuthology.orchestra.run.smithi184.stdout: "AsyncMessenger::Worker-0": { 2024-05-20T16:26:10.077 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_recv_messages": 166, 2024-05-20T16:26:10.077 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_messages": 165, 2024-05-20T16:26:10.077 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_recv_bytes": 19236229, 2024-05-20T16:26:10.077 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_bytes": 56740, 2024-05-20T16:26:10.077 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_created_connections": 6, 2024-05-20T16:26:10.077 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_active_connections": 2, 2024-05-20T16:26:10.077 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_total_time": 0.038035579, 2024-05-20T16:26:10.077 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_send_time": 0.005449930, 2024-05-20T16:26:10.077 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_recv_time": 0.024121360, 2024-05-20T16:26:10.077 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_fast_dispatch_time": 0.005872331, 2024-05-20T16:26:10.077 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_messages_queue_lat": { 2024-05-20T16:26:10.077 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 164, 2024-05-20T16:26:10.077 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.064644054, 2024-05-20T16:26:10.078 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000394171 2024-05-20T16:26:10.078 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-20T16:26:10.078 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_handle_ack_lat": { 2024-05-20T16:26:10.078 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-20T16:26:10.078 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-20T16:26:10.078 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-20T16:26:10.078 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-20T16:26:10.078 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_recv_encrypted_bytes": 161216, 2024-05-20T16:26:10.078 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_encrypted_bytes": 13488 2024-05-20T16:26:10.078 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-20T16:26:10.078 INFO:teuthology.orchestra.run.smithi184.stdout: "AsyncMessenger::Worker-1": { 2024-05-20T16:26:10.078 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_recv_messages": 55, 2024-05-20T16:26:10.078 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_messages": 69, 2024-05-20T16:26:10.079 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_recv_bytes": 20993154, 2024-05-20T16:26:10.079 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_bytes": 127381, 2024-05-20T16:26:10.079 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_created_connections": 5, 2024-05-20T16:26:10.079 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_active_connections": 4, 2024-05-20T16:26:10.079 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_total_time": 0.019092664, 2024-05-20T16:26:10.079 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_send_time": 0.002073322, 2024-05-20T16:26:10.079 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_recv_time": 0.014244305, 2024-05-20T16:26:10.079 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_fast_dispatch_time": 0.002125614, 2024-05-20T16:26:10.079 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_messages_queue_lat": { 2024-05-20T16:26:10.079 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 68, 2024-05-20T16:26:10.079 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.036838556, 2024-05-20T16:26:10.079 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000541743 2024-05-20T16:26:10.079 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-20T16:26:10.079 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_handle_ack_lat": { 2024-05-20T16:26:10.079 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-20T16:26:10.080 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-20T16:26:10.080 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-20T16:26:10.080 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-20T16:26:10.080 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_recv_encrypted_bytes": 3184, 2024-05-20T16:26:10.080 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_encrypted_bytes": 111424 2024-05-20T16:26:10.080 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-20T16:26:10.080 INFO:teuthology.orchestra.run.smithi184.stdout: "AsyncMessenger::Worker-2": { 2024-05-20T16:26:10.080 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_recv_messages": 118, 2024-05-20T16:26:10.080 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_messages": 120, 2024-05-20T16:26:10.080 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_recv_bytes": 29303718, 2024-05-20T16:26:10.080 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_bytes": 37540, 2024-05-20T16:26:10.080 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_created_connections": 5, 2024-05-20T16:26:10.080 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_active_connections": 4, 2024-05-20T16:26:10.080 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_total_time": 0.027075202, 2024-05-20T16:26:10.081 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_send_time": 0.003521491, 2024-05-20T16:26:10.081 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_recv_time": 0.018795073, 2024-05-20T16:26:10.081 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_fast_dispatch_time": 0.003772212, 2024-05-20T16:26:10.081 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_messages_queue_lat": { 2024-05-20T16:26:10.081 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 118, 2024-05-20T16:26:10.081 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.072806198, 2024-05-20T16:26:10.081 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000617001 2024-05-20T16:26:10.081 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-20T16:26:10.081 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_handle_ack_lat": { 2024-05-20T16:26:10.081 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-20T16:26:10.081 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-20T16:26:10.081 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-20T16:26:10.081 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-20T16:26:10.081 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-20T16:26:10.081 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_encrypted_bytes": 0 2024-05-20T16:26:10.082 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-20T16:26:10.082 INFO:teuthology.orchestra.run.smithi184.stdout: "cct": { 2024-05-20T16:26:10.082 INFO:teuthology.orchestra.run.smithi184.stdout: "total_workers": 1, 2024-05-20T16:26:10.082 INFO:teuthology.orchestra.run.smithi184.stdout: "unhealthy_workers": 0 2024-05-20T16:26:10.082 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-20T16:26:10.082 INFO:teuthology.orchestra.run.smithi184.stdout: "finisher-MDSRank": { 2024-05-20T16:26:10.082 INFO:teuthology.orchestra.run.smithi184.stdout: "queue_len": 0, 2024-05-20T16:26:10.082 INFO:teuthology.orchestra.run.smithi184.stdout: "complete_latency": { 2024-05-20T16:26:10.082 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 444, 2024-05-20T16:26:10.082 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.014004265, 2024-05-20T16:26:10.082 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000031541 2024-05-20T16:26:10.082 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-05-20T16:26:10.083 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-20T16:26:10.083 INFO:teuthology.orchestra.run.smithi184.stdout: "finisher-PurgeQueue": { 2024-05-20T16:26:10.083 INFO:teuthology.orchestra.run.smithi184.stdout: "queue_len": 0, 2024-05-20T16:26:10.083 INFO:teuthology.orchestra.run.smithi184.stdout: "complete_latency": { 2024-05-20T16:26:10.083 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-20T16:26:10.083 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-20T16:26:10.083 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-20T16:26:10.083 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-05-20T16:26:10.083 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-20T16:26:10.083 INFO:teuthology.orchestra.run.smithi184.stdout: "mds": { 2024-05-20T16:26:10.083 INFO:teuthology.orchestra.run.smithi184.stdout: "request": 0, 2024-05-20T16:26:10.083 INFO:teuthology.orchestra.run.smithi184.stdout: "reply": 0, 2024-05-20T16:26:10.083 INFO:teuthology.orchestra.run.smithi184.stdout: "reply_latency": { 2024-05-20T16:26:10.084 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-20T16:26:10.084 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-20T16:26:10.084 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-20T16:26:10.084 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-20T16:26:10.084 INFO:teuthology.orchestra.run.smithi184.stdout: "slow_reply": 0, 2024-05-20T16:26:10.084 INFO:teuthology.orchestra.run.smithi184.stdout: "forward": 0, 2024-05-20T16:26:10.084 INFO:teuthology.orchestra.run.smithi184.stdout: "dir_fetch_complete": 0, 2024-05-20T16:26:10.084 INFO:teuthology.orchestra.run.smithi184.stdout: "dir_fetch_keys": 0, 2024-05-20T16:26:10.084 INFO:teuthology.orchestra.run.smithi184.stdout: "dir_commit": 0, 2024-05-20T16:26:10.084 INFO:teuthology.orchestra.run.smithi184.stdout: "dir_split": 0, 2024-05-20T16:26:10.084 INFO:teuthology.orchestra.run.smithi184.stdout: "dir_merge": 0, 2024-05-20T16:26:10.084 INFO:teuthology.orchestra.run.smithi184.stdout: "inodes": 28, 2024-05-20T16:26:10.084 INFO:teuthology.orchestra.run.smithi184.stdout: "inodes_top": 0, 2024-05-20T16:26:10.084 INFO:teuthology.orchestra.run.smithi184.stdout: "inodes_bottom": 28, 2024-05-20T16:26:10.085 INFO:teuthology.orchestra.run.smithi184.stdout: "inodes_pin_tail": 0, 2024-05-20T16:26:10.085 INFO:teuthology.orchestra.run.smithi184.stdout: "inodes_pinned": 28, 2024-05-20T16:26:10.085 INFO:teuthology.orchestra.run.smithi184.stdout: "inodes_expired": 0, 2024-05-20T16:26:10.085 INFO:teuthology.orchestra.run.smithi184.stdout: "inodes_with_caps": 0, 2024-05-20T16:26:10.085 INFO:teuthology.orchestra.run.smithi184.stdout: "caps": 0, 2024-05-20T16:26:10.085 INFO:teuthology.orchestra.run.smithi184.stdout: "subtrees": 2, 2024-05-20T16:26:10.085 INFO:teuthology.orchestra.run.smithi184.stdout: "traverse": 0, 2024-05-20T16:26:10.085 INFO:teuthology.orchestra.run.smithi184.stdout: "traverse_hit": 0, 2024-05-20T16:26:10.085 INFO:teuthology.orchestra.run.smithi184.stdout: "traverse_forward": 0, 2024-05-20T16:26:10.085 INFO:teuthology.orchestra.run.smithi184.stdout: "traverse_discover": 0, 2024-05-20T16:26:10.085 INFO:teuthology.orchestra.run.smithi184.stdout: "traverse_dir_fetch": 0, 2024-05-20T16:26:10.085 INFO:teuthology.orchestra.run.smithi184.stdout: "traverse_remote_ino": 0, 2024-05-20T16:26:10.085 INFO:teuthology.orchestra.run.smithi184.stdout: "traverse_lock": 0, 2024-05-20T16:26:10.085 INFO:teuthology.orchestra.run.smithi184.stdout: "load_cent": 0, 2024-05-20T16:26:10.086 INFO:teuthology.orchestra.run.smithi184.stdout: "q": 0, 2024-05-20T16:26:10.086 INFO:teuthology.orchestra.run.smithi184.stdout: "exported": 0, 2024-05-20T16:26:10.086 INFO:teuthology.orchestra.run.smithi184.stdout: "exported_inodes": 0, 2024-05-20T16:26:10.086 INFO:teuthology.orchestra.run.smithi184.stdout: "imported": 0, 2024-05-20T16:26:10.086 INFO:teuthology.orchestra.run.smithi184.stdout: "imported_inodes": 0, 2024-05-20T16:26:10.087 INFO:teuthology.orchestra.run.smithi184.stdout: "openino_dir_fetch": 0, 2024-05-20T16:26:10.087 INFO:teuthology.orchestra.run.smithi184.stdout: "openino_backtrace_fetch": 0, 2024-05-20T16:26:10.087 INFO:teuthology.orchestra.run.smithi184.stdout: "openino_peer_discover": 0, 2024-05-20T16:26:10.087 INFO:teuthology.orchestra.run.smithi184.stdout: "root_rfiles": 4, 2024-05-20T16:26:10.087 INFO:teuthology.orchestra.run.smithi184.stdout: "root_rbytes": 234, 2024-05-20T16:26:10.088 INFO:teuthology.orchestra.run.smithi184.stdout: "root_rsnaps": 0, 2024-05-20T16:26:10.088 INFO:teuthology.orchestra.run.smithi184.stdout: "scrub_backtrace_fetch": 0, 2024-05-20T16:26:10.088 INFO:teuthology.orchestra.run.smithi184.stdout: "scrub_set_tag": 0, 2024-05-20T16:26:10.088 INFO:teuthology.orchestra.run.smithi184.stdout: "scrub_backtrace_repaired": 0, 2024-05-20T16:26:10.088 INFO:teuthology.orchestra.run.smithi184.stdout: "scrub_inotable_repaired": 0, 2024-05-20T16:26:10.088 INFO:teuthology.orchestra.run.smithi184.stdout: "scrub_dir_inodes": 0, 2024-05-20T16:26:10.088 INFO:teuthology.orchestra.run.smithi184.stdout: "scrub_dir_base_inodes": 0, 2024-05-20T16:26:10.088 INFO:teuthology.orchestra.run.smithi184.stdout: "scrub_dirfrag_rstats": 0, 2024-05-20T16:26:10.088 INFO:teuthology.orchestra.run.smithi184.stdout: "scrub_file_inodes": 0, 2024-05-20T16:26:10.088 INFO:teuthology.orchestra.run.smithi184.stdout: "handle_inode_file_caps": 0, 2024-05-20T16:26:10.088 INFO:teuthology.orchestra.run.smithi184.stdout: "ceph_cap_op_revoke": 0, 2024-05-20T16:26:10.088 INFO:teuthology.orchestra.run.smithi184.stdout: "ceph_cap_op_grant": 0, 2024-05-20T16:26:10.089 INFO:teuthology.orchestra.run.smithi184.stdout: "ceph_cap_op_trunc": 0, 2024-05-20T16:26:10.089 INFO:teuthology.orchestra.run.smithi184.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-20T16:26:10.089 INFO:teuthology.orchestra.run.smithi184.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-20T16:26:10.089 INFO:teuthology.orchestra.run.smithi184.stdout: "handle_client_caps": 0, 2024-05-20T16:26:10.089 INFO:teuthology.orchestra.run.smithi184.stdout: "handle_client_caps_dirty": 0, 2024-05-20T16:26:10.089 INFO:teuthology.orchestra.run.smithi184.stdout: "handle_client_cap_release": 0, 2024-05-20T16:26:10.089 INFO:teuthology.orchestra.run.smithi184.stdout: "process_request_cap_release": 0 2024-05-20T16:26:10.089 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-20T16:26:10.089 INFO:teuthology.orchestra.run.smithi184.stdout: "mds_cache": { 2024-05-20T16:26:10.089 INFO:teuthology.orchestra.run.smithi184.stdout: "dir_update": 0, 2024-05-20T16:26:10.089 INFO:teuthology.orchestra.run.smithi184.stdout: "dir_update_receipt": 0, 2024-05-20T16:26:10.089 INFO:teuthology.orchestra.run.smithi184.stdout: "dir_try_discover": 0, 2024-05-20T16:26:10.089 INFO:teuthology.orchestra.run.smithi184.stdout: "dir_send_discover": 0, 2024-05-20T16:26:10.089 INFO:teuthology.orchestra.run.smithi184.stdout: "dir_handle_discover": 0, 2024-05-20T16:26:10.090 INFO:teuthology.orchestra.run.smithi184.stdout: "num_strays": 0, 2024-05-20T16:26:10.090 INFO:teuthology.orchestra.run.smithi184.stdout: "num_strays_delayed": 0, 2024-05-20T16:26:10.090 INFO:teuthology.orchestra.run.smithi184.stdout: "num_strays_enqueuing": 0, 2024-05-20T16:26:10.090 INFO:teuthology.orchestra.run.smithi184.stdout: "strays_created": 0, 2024-05-20T16:26:10.090 INFO:teuthology.orchestra.run.smithi184.stdout: "strays_enqueued": 0, 2024-05-20T16:26:10.090 INFO:teuthology.orchestra.run.smithi184.stdout: "strays_reintegrated": 0, 2024-05-20T16:26:10.090 INFO:teuthology.orchestra.run.smithi184.stdout: "strays_migrated": 0, 2024-05-20T16:26:10.090 INFO:teuthology.orchestra.run.smithi184.stdout: "num_recovering_processing": 0, 2024-05-20T16:26:10.090 INFO:teuthology.orchestra.run.smithi184.stdout: "num_recovering_enqueued": 0, 2024-05-20T16:26:10.090 INFO:teuthology.orchestra.run.smithi184.stdout: "num_recovering_prioritized": 0, 2024-05-20T16:26:10.090 INFO:teuthology.orchestra.run.smithi184.stdout: "recovery_started": 0, 2024-05-20T16:26:10.090 INFO:teuthology.orchestra.run.smithi184.stdout: "recovery_completed": 0, 2024-05-20T16:26:10.091 INFO:teuthology.orchestra.run.smithi184.stdout: "ireq_quiesce_path": 0, 2024-05-20T16:26:10.091 INFO:teuthology.orchestra.run.smithi184.stdout: "ireq_quiesce_inode": 0, 2024-05-20T16:26:10.091 INFO:teuthology.orchestra.run.smithi184.stdout: "ireq_enqueue_scrub": 0, 2024-05-20T16:26:10.091 INFO:teuthology.orchestra.run.smithi184.stdout: "ireq_exportdir": 0, 2024-05-20T16:26:10.091 INFO:teuthology.orchestra.run.smithi184.stdout: "ireq_flush": 0, 2024-05-20T16:26:10.091 INFO:teuthology.orchestra.run.smithi184.stdout: "ireq_fragmentdir": 0, 2024-05-20T16:26:10.091 INFO:teuthology.orchestra.run.smithi184.stdout: "ireq_fragstats": 0, 2024-05-20T16:26:10.091 INFO:teuthology.orchestra.run.smithi184.stdout: "ireq_inodestats": 0 2024-05-20T16:26:10.091 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-20T16:26:10.091 INFO:teuthology.orchestra.run.smithi184.stdout: "mds_log": { 2024-05-20T16:26:10.091 INFO:teuthology.orchestra.run.smithi184.stdout: "evlrg": 0, 2024-05-20T16:26:10.091 INFO:teuthology.orchestra.run.smithi184.stdout: "evadd": 0, 2024-05-20T16:26:10.091 INFO:teuthology.orchestra.run.smithi184.stdout: "evex": 0, 2024-05-20T16:26:10.092 INFO:teuthology.orchestra.run.smithi184.stdout: "evtrm": 0, 2024-05-20T16:26:10.092 INFO:teuthology.orchestra.run.smithi184.stdout: "ev": 32793, 2024-05-20T16:26:10.092 INFO:teuthology.orchestra.run.smithi184.stdout: "evexg": 0, 2024-05-20T16:26:10.092 INFO:teuthology.orchestra.run.smithi184.stdout: "evexd": 0, 2024-05-20T16:26:10.092 INFO:teuthology.orchestra.run.smithi184.stdout: "segadd": 0, 2024-05-20T16:26:10.092 INFO:teuthology.orchestra.run.smithi184.stdout: "segex": 0, 2024-05-20T16:26:10.092 INFO:teuthology.orchestra.run.smithi184.stdout: "segtrm": 0, 2024-05-20T16:26:10.092 INFO:teuthology.orchestra.run.smithi184.stdout: "seg": 9, 2024-05-20T16:26:10.092 INFO:teuthology.orchestra.run.smithi184.stdout: "segmjr": 4, 2024-05-20T16:26:10.092 INFO:teuthology.orchestra.run.smithi184.stdout: "segexg": 0, 2024-05-20T16:26:10.092 INFO:teuthology.orchestra.run.smithi184.stdout: "segexd": 0, 2024-05-20T16:26:10.092 INFO:teuthology.orchestra.run.smithi184.stdout: "expos": 1051205280, 2024-05-20T16:26:10.092 INFO:teuthology.orchestra.run.smithi184.stdout: "wrpos": 1069459896, 2024-05-20T16:26:10.092 INFO:teuthology.orchestra.run.smithi184.stdout: "rdpos": 1069457242, 2024-05-20T16:26:10.092 INFO:teuthology.orchestra.run.smithi184.stdout: "jlat": { 2024-05-20T16:26:10.093 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-20T16:26:10.093 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-20T16:26:10.093 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-20T16:26:10.093 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-20T16:26:10.093 INFO:teuthology.orchestra.run.smithi184.stdout: "replayed": 32793 2024-05-20T16:26:10.093 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-20T16:26:10.093 INFO:teuthology.orchestra.run.smithi184.stdout: "mds_mem": { 2024-05-20T16:26:10.093 INFO:teuthology.orchestra.run.smithi184.stdout: "ino": 16, 2024-05-20T16:26:10.093 INFO:teuthology.orchestra.run.smithi184.stdout: "ino+": 30, 2024-05-20T16:26:10.093 INFO:teuthology.orchestra.run.smithi184.stdout: "ino-": 14, 2024-05-20T16:26:10.093 INFO:teuthology.orchestra.run.smithi184.stdout: "dir": 7, 2024-05-20T16:26:10.093 INFO:teuthology.orchestra.run.smithi184.stdout: "dir+": 11, 2024-05-20T16:26:10.093 INFO:teuthology.orchestra.run.smithi184.stdout: "dir-": 4, 2024-05-20T16:26:10.093 INFO:teuthology.orchestra.run.smithi184.stdout: "dn": 28, 2024-05-20T16:26:10.094 INFO:teuthology.orchestra.run.smithi184.stdout: "dn+": 49, 2024-05-20T16:26:10.094 INFO:teuthology.orchestra.run.smithi184.stdout: "dn-": 21, 2024-05-20T16:26:10.094 INFO:teuthology.orchestra.run.smithi184.stdout: "cap": 0, 2024-05-20T16:26:10.094 INFO:teuthology.orchestra.run.smithi184.stdout: "cap+": 0, 2024-05-20T16:26:10.094 INFO:teuthology.orchestra.run.smithi184.stdout: "cap-": 0, 2024-05-20T16:26:10.094 INFO:teuthology.orchestra.run.smithi184.stdout: "rss": 118976, 2024-05-20T16:26:10.094 INFO:teuthology.orchestra.run.smithi184.stdout: "heap": 215420 2024-05-20T16:26:10.094 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-20T16:26:10.094 INFO:teuthology.orchestra.run.smithi184.stdout: "mds_server": { 2024-05-20T16:26:10.094 INFO:teuthology.orchestra.run.smithi184.stdout: "dispatch_client_request": 0, 2024-05-20T16:26:10.094 INFO:teuthology.orchestra.run.smithi184.stdout: "dispatch_server_request": 0, 2024-05-20T16:26:10.094 INFO:teuthology.orchestra.run.smithi184.stdout: "handle_client_request": 0, 2024-05-20T16:26:10.094 INFO:teuthology.orchestra.run.smithi184.stdout: "handle_client_session": 0, 2024-05-20T16:26:10.095 INFO:teuthology.orchestra.run.smithi184.stdout: "handle_peer_request": 0, 2024-05-20T16:26:10.095 INFO:teuthology.orchestra.run.smithi184.stdout: "req_create_latency": { 2024-05-20T16:26:10.095 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-20T16:26:10.095 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-20T16:26:10.095 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-20T16:26:10.095 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-20T16:26:10.095 INFO:teuthology.orchestra.run.smithi184.stdout: "req_getattr_latency": { 2024-05-20T16:26:10.095 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-20T16:26:10.095 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-20T16:26:10.095 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-20T16:26:10.095 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-20T16:26:10.095 INFO:teuthology.orchestra.run.smithi184.stdout: "req_getfilelock_latency": { 2024-05-20T16:26:10.095 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-20T16:26:10.096 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-20T16:26:10.096 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-20T16:26:10.096 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-20T16:26:10.096 INFO:teuthology.orchestra.run.smithi184.stdout: "req_link_latency": { 2024-05-20T16:26:10.096 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-20T16:26:10.096 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-20T16:26:10.096 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-20T16:26:10.096 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-20T16:26:10.096 INFO:teuthology.orchestra.run.smithi184.stdout: "req_lookup_latency": { 2024-05-20T16:26:10.096 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-20T16:26:10.096 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-20T16:26:10.096 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-20T16:26:10.096 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-20T16:26:10.097 INFO:teuthology.orchestra.run.smithi184.stdout: "req_lookuphash_latency": { 2024-05-20T16:26:10.097 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-20T16:26:10.097 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-20T16:26:10.097 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-20T16:26:10.097 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-20T16:26:10.097 INFO:teuthology.orchestra.run.smithi184.stdout: "req_lookupino_latency": { 2024-05-20T16:26:10.097 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-20T16:26:10.097 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-20T16:26:10.097 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-20T16:26:10.097 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-20T16:26:10.097 INFO:teuthology.orchestra.run.smithi184.stdout: "req_lookupname_latency": { 2024-05-20T16:26:10.097 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-20T16:26:10.097 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-20T16:26:10.097 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-20T16:26:10.098 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-20T16:26:10.098 INFO:teuthology.orchestra.run.smithi184.stdout: "req_lookupparent_latency": { 2024-05-20T16:26:10.098 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-20T16:26:10.098 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-20T16:26:10.098 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-20T16:26:10.098 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-20T16:26:10.098 INFO:teuthology.orchestra.run.smithi184.stdout: "req_lookupsnap_latency": { 2024-05-20T16:26:10.098 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-20T16:26:10.098 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-20T16:26:10.098 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-20T16:26:10.098 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-20T16:26:10.098 INFO:teuthology.orchestra.run.smithi184.stdout: "req_lssnap_latency": { 2024-05-20T16:26:10.098 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-20T16:26:10.098 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-20T16:26:10.099 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-20T16:26:10.099 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-20T16:26:10.099 INFO:teuthology.orchestra.run.smithi184.stdout: "req_mkdir_latency": { 2024-05-20T16:26:10.099 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-20T16:26:10.099 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-20T16:26:10.099 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-20T16:26:10.099 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-20T16:26:10.100 INFO:teuthology.orchestra.run.smithi184.stdout: "req_mknod_latency": { 2024-05-20T16:26:10.100 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-20T16:26:10.100 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-20T16:26:10.100 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-20T16:26:10.100 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-20T16:26:10.100 INFO:teuthology.orchestra.run.smithi184.stdout: "req_mksnap_latency": { 2024-05-20T16:26:10.100 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-20T16:26:10.100 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-20T16:26:10.100 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-20T16:26:10.100 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-20T16:26:10.100 INFO:teuthology.orchestra.run.smithi184.stdout: "req_open_latency": { 2024-05-20T16:26:10.100 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-20T16:26:10.100 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-20T16:26:10.100 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-20T16:26:10.101 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-20T16:26:10.101 INFO:teuthology.orchestra.run.smithi184.stdout: "req_readdir_latency": { 2024-05-20T16:26:10.101 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-20T16:26:10.101 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-20T16:26:10.101 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-20T16:26:10.101 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-20T16:26:10.101 INFO:teuthology.orchestra.run.smithi184.stdout: "req_rename_latency": { 2024-05-20T16:26:10.101 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-20T16:26:10.101 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-20T16:26:10.101 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-20T16:26:10.101 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-20T16:26:10.101 INFO:teuthology.orchestra.run.smithi184.stdout: "req_renamesnap_latency": { 2024-05-20T16:26:10.101 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-20T16:26:10.101 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-20T16:26:10.102 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-20T16:26:10.102 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-20T16:26:10.102 INFO:teuthology.orchestra.run.smithi184.stdout: "req_snapdiff_latency": { 2024-05-20T16:26:10.102 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-20T16:26:10.102 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-20T16:26:10.102 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-20T16:26:10.102 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-20T16:26:10.102 INFO:teuthology.orchestra.run.smithi184.stdout: "req_rmdir_latency": { 2024-05-20T16:26:10.102 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-20T16:26:10.102 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-20T16:26:10.102 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-20T16:26:10.102 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-20T16:26:10.102 INFO:teuthology.orchestra.run.smithi184.stdout: "req_rmsnap_latency": { 2024-05-20T16:26:10.103 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-20T16:26:10.103 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-20T16:26:10.103 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-20T16:26:10.103 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-20T16:26:10.103 INFO:teuthology.orchestra.run.smithi184.stdout: "req_rmxattr_latency": { 2024-05-20T16:26:10.103 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-20T16:26:10.103 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-20T16:26:10.103 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-20T16:26:10.103 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-20T16:26:10.103 INFO:teuthology.orchestra.run.smithi184.stdout: "req_setattr_latency": { 2024-05-20T16:26:10.103 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-20T16:26:10.103 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-20T16:26:10.103 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-20T16:26:10.104 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-20T16:26:10.104 INFO:teuthology.orchestra.run.smithi184.stdout: "req_setdirlayout_latency": { 2024-05-20T16:26:10.104 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-20T16:26:10.104 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-20T16:26:10.104 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-20T16:26:10.104 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-20T16:26:10.104 INFO:teuthology.orchestra.run.smithi184.stdout: "req_setfilelock_latency": { 2024-05-20T16:26:10.104 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-20T16:26:10.104 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-20T16:26:10.104 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-20T16:26:10.104 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-20T16:26:10.104 INFO:teuthology.orchestra.run.smithi184.stdout: "req_setlayout_latency": { 2024-05-20T16:26:10.104 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-20T16:26:10.105 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-20T16:26:10.105 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-20T16:26:10.105 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-20T16:26:10.105 INFO:teuthology.orchestra.run.smithi184.stdout: "req_setxattr_latency": { 2024-05-20T16:26:10.105 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-20T16:26:10.105 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-20T16:26:10.105 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-20T16:26:10.105 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-20T16:26:10.105 INFO:teuthology.orchestra.run.smithi184.stdout: "req_symlink_latency": { 2024-05-20T16:26:10.105 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-20T16:26:10.105 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-20T16:26:10.105 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-20T16:26:10.105 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-20T16:26:10.106 INFO:teuthology.orchestra.run.smithi184.stdout: "req_unlink_latency": { 2024-05-20T16:26:10.106 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-20T16:26:10.106 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-20T16:26:10.106 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-20T16:26:10.106 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-20T16:26:10.106 INFO:teuthology.orchestra.run.smithi184.stdout: "cap_revoke_eviction": 0, 2024-05-20T16:26:10.106 INFO:teuthology.orchestra.run.smithi184.stdout: "cap_acquisition_throttle": 0, 2024-05-20T16:26:10.106 INFO:teuthology.orchestra.run.smithi184.stdout: "req_getvxattr_latency": { 2024-05-20T16:26:10.106 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-20T16:26:10.106 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-20T16:26:10.106 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-20T16:26:10.106 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-05-20T16:26:10.106 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-20T16:26:10.107 INFO:teuthology.orchestra.run.smithi184.stdout: "mds_sessions": { 2024-05-20T16:26:10.107 INFO:teuthology.orchestra.run.smithi184.stdout: "session_count": 1, 2024-05-20T16:26:10.107 INFO:teuthology.orchestra.run.smithi184.stdout: "session_add": 1, 2024-05-20T16:26:10.107 INFO:teuthology.orchestra.run.smithi184.stdout: "session_remove": 0, 2024-05-20T16:26:10.107 INFO:teuthology.orchestra.run.smithi184.stdout: "sessions_open": 0, 2024-05-20T16:26:10.107 INFO:teuthology.orchestra.run.smithi184.stdout: "sessions_stale": 0, 2024-05-20T16:26:10.107 INFO:teuthology.orchestra.run.smithi184.stdout: "total_load": 0, 2024-05-20T16:26:10.107 INFO:teuthology.orchestra.run.smithi184.stdout: "average_load": 0, 2024-05-20T16:26:10.107 INFO:teuthology.orchestra.run.smithi184.stdout: "avg_session_uptime": 2099, 2024-05-20T16:26:10.107 INFO:teuthology.orchestra.run.smithi184.stdout: "mdthresh_evicted": 0 2024-05-20T16:26:10.107 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-20T16:26:10.107 INFO:teuthology.orchestra.run.smithi184.stdout: "mempool": { 2024-05-20T16:26:10.107 INFO:teuthology.orchestra.run.smithi184.stdout: "bloom_filter_bytes": 0, 2024-05-20T16:26:10.107 INFO:teuthology.orchestra.run.smithi184.stdout: "bloom_filter_items": 0, 2024-05-20T16:26:10.107 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_alloc_bytes": 0, 2024-05-20T16:26:10.108 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_alloc_items": 0, 2024-05-20T16:26:10.108 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_data_bytes": 0, 2024-05-20T16:26:10.108 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_data_items": 0, 2024-05-20T16:26:10.108 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-20T16:26:10.108 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_onode_items": 0, 2024-05-20T16:26:10.108 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-20T16:26:10.108 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_meta_items": 0, 2024-05-20T16:26:10.108 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_other_bytes": 0, 2024-05-20T16:26:10.108 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_other_items": 0, 2024-05-20T16:26:10.108 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-20T16:26:10.108 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_buffer_items": 0, 2024-05-20T16:26:10.108 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_extent_bytes": 0, 2024-05-20T16:26:10.108 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_extent_items": 0, 2024-05-20T16:26:10.108 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_blob_bytes": 0, 2024-05-20T16:26:10.108 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_blob_items": 0, 2024-05-20T16:26:10.109 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-20T16:26:10.109 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_shared_blob_items": 0, 2024-05-20T16:26:10.109 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-20T16:26:10.109 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_inline_bl_items": 0, 2024-05-20T16:26:10.109 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_fsck_bytes": 0, 2024-05-20T16:26:10.109 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_fsck_items": 0, 2024-05-20T16:26:10.109 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_txc_bytes": 0, 2024-05-20T16:26:10.109 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_txc_items": 0, 2024-05-20T16:26:10.109 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-20T16:26:10.109 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_writing_deferred_items": 0, 2024-05-20T16:26:10.109 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_writing_bytes": 0, 2024-05-20T16:26:10.109 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_writing_items": 0, 2024-05-20T16:26:10.110 INFO:teuthology.orchestra.run.smithi184.stdout: "bluefs_bytes": 0, 2024-05-20T16:26:10.110 INFO:teuthology.orchestra.run.smithi184.stdout: "bluefs_items": 0, 2024-05-20T16:26:10.110 INFO:teuthology.orchestra.run.smithi184.stdout: "bluefs_file_reader_bytes": 0, 2024-05-20T16:26:10.110 INFO:teuthology.orchestra.run.smithi184.stdout: "bluefs_file_reader_items": 0, 2024-05-20T16:26:10.110 INFO:teuthology.orchestra.run.smithi184.stdout: "bluefs_file_writer_bytes": 0, 2024-05-20T16:26:10.110 INFO:teuthology.orchestra.run.smithi184.stdout: "bluefs_file_writer_items": 0, 2024-05-20T16:26:10.110 INFO:teuthology.orchestra.run.smithi184.stdout: "buffer_anon_bytes": 204895, 2024-05-20T16:26:10.110 INFO:teuthology.orchestra.run.smithi184.stdout: "buffer_anon_items": 61, 2024-05-20T16:26:10.110 INFO:teuthology.orchestra.run.smithi184.stdout: "buffer_meta_bytes": 0, 2024-05-20T16:26:10.110 INFO:teuthology.orchestra.run.smithi184.stdout: "buffer_meta_items": 0, 2024-05-20T16:26:10.110 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_bytes": 0, 2024-05-20T16:26:10.110 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_items": 0, 2024-05-20T16:26:10.110 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_mapbl_bytes": 0, 2024-05-20T16:26:10.110 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_mapbl_items": 0, 2024-05-20T16:26:10.111 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_pglog_bytes": 0, 2024-05-20T16:26:10.111 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_pglog_items": 0, 2024-05-20T16:26:10.111 INFO:teuthology.orchestra.run.smithi184.stdout: "osdmap_bytes": 8888, 2024-05-20T16:26:10.111 INFO:teuthology.orchestra.run.smithi184.stdout: "osdmap_items": 170, 2024-05-20T16:26:10.111 INFO:teuthology.orchestra.run.smithi184.stdout: "osdmap_mapping_bytes": 0, 2024-05-20T16:26:10.111 INFO:teuthology.orchestra.run.smithi184.stdout: "osdmap_mapping_items": 0, 2024-05-20T16:26:10.111 INFO:teuthology.orchestra.run.smithi184.stdout: "pgmap_bytes": 0, 2024-05-20T16:26:10.111 INFO:teuthology.orchestra.run.smithi184.stdout: "pgmap_items": 0, 2024-05-20T16:26:10.111 INFO:teuthology.orchestra.run.smithi184.stdout: "mds_co_bytes": 66776, 2024-05-20T16:26:10.111 INFO:teuthology.orchestra.run.smithi184.stdout: "mds_co_items": 1177, 2024-05-20T16:26:10.111 INFO:teuthology.orchestra.run.smithi184.stdout: "unittest_1_bytes": 0, 2024-05-20T16:26:10.111 INFO:teuthology.orchestra.run.smithi184.stdout: "unittest_1_items": 0, 2024-05-20T16:26:10.111 INFO:teuthology.orchestra.run.smithi184.stdout: "unittest_2_bytes": 0, 2024-05-20T16:26:10.111 INFO:teuthology.orchestra.run.smithi184.stdout: "unittest_2_items": 0 2024-05-20T16:26:10.111 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-20T16:26:10.111 INFO:teuthology.orchestra.run.smithi184.stdout: "objecter": { 2024-05-20T16:26:10.112 INFO:teuthology.orchestra.run.smithi184.stdout: "op_active": 0, 2024-05-20T16:26:10.112 INFO:teuthology.orchestra.run.smithi184.stdout: "op_laggy": 0, 2024-05-20T16:26:10.112 INFO:teuthology.orchestra.run.smithi184.stdout: "op_send": 304, 2024-05-20T16:26:10.112 INFO:teuthology.orchestra.run.smithi184.stdout: "op_send_bytes": 26, 2024-05-20T16:26:10.112 INFO:teuthology.orchestra.run.smithi184.stdout: "op_resend": 4, 2024-05-20T16:26:10.112 INFO:teuthology.orchestra.run.smithi184.stdout: "op_reply": 304, 2024-05-20T16:26:10.112 INFO:teuthology.orchestra.run.smithi184.stdout: "op_latency": { 2024-05-20T16:26:10.112 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 304, 2024-05-20T16:26:10.112 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 1.758967249, 2024-05-20T16:26:10.112 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.005786076 2024-05-20T16:26:10.112 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-20T16:26:10.112 INFO:teuthology.orchestra.run.smithi184.stdout: "op_inflight": 0, 2024-05-20T16:26:10.112 INFO:teuthology.orchestra.run.smithi184.stdout: "oplen_avg": { 2024-05-20T16:26:10.112 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 304, 2024-05-20T16:26:10.112 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 305 2024-05-20T16:26:10.112 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-20T16:26:10.113 INFO:teuthology.orchestra.run.smithi184.stdout: "op": 304, 2024-05-20T16:26:10.113 INFO:teuthology.orchestra.run.smithi184.stdout: "op_r": 304, 2024-05-20T16:26:10.113 INFO:teuthology.orchestra.run.smithi184.stdout: "op_w": 0, 2024-05-20T16:26:10.113 INFO:teuthology.orchestra.run.smithi184.stdout: "op_rmw": 0, 2024-05-20T16:26:10.113 INFO:teuthology.orchestra.run.smithi184.stdout: "op_pg": 0, 2024-05-20T16:26:10.113 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_stat": 188, 2024-05-20T16:26:10.113 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_create": 0, 2024-05-20T16:26:10.113 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_read": 113, 2024-05-20T16:26:10.113 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_write": 0, 2024-05-20T16:26:10.113 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_writefull": 0, 2024-05-20T16:26:10.113 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_writesame": 0, 2024-05-20T16:26:10.113 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_append": 0, 2024-05-20T16:26:10.113 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_zero": 0, 2024-05-20T16:26:10.113 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_truncate": 0, 2024-05-20T16:26:10.113 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_delete": 0, 2024-05-20T16:26:10.113 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_mapext": 0, 2024-05-20T16:26:10.114 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_sparse_read": 0, 2024-05-20T16:26:10.114 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_clonerange": 0, 2024-05-20T16:26:10.114 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_getxattr": 2, 2024-05-20T16:26:10.114 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_setxattr": 0, 2024-05-20T16:26:10.114 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_cmpxattr": 0, 2024-05-20T16:26:10.114 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_rmxattr": 0, 2024-05-20T16:26:10.114 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_resetxattrs": 0, 2024-05-20T16:26:10.114 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_call": 0, 2024-05-20T16:26:10.114 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_watch": 0, 2024-05-20T16:26:10.114 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_notify": 0, 2024-05-20T16:26:10.114 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_src_cmpxattr": 0, 2024-05-20T16:26:10.114 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_pgls": 0, 2024-05-20T16:26:10.114 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_pgls_filter": 0, 2024-05-20T16:26:10.114 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_other": 0, 2024-05-20T16:26:10.114 INFO:teuthology.orchestra.run.smithi184.stdout: "linger_active": 0, 2024-05-20T16:26:10.115 INFO:teuthology.orchestra.run.smithi184.stdout: "linger_send": 0, 2024-05-20T16:26:10.115 INFO:teuthology.orchestra.run.smithi184.stdout: "linger_resend": 0, 2024-05-20T16:26:10.115 INFO:teuthology.orchestra.run.smithi184.stdout: "linger_ping": 0, 2024-05-20T16:26:10.115 INFO:teuthology.orchestra.run.smithi184.stdout: "poolop_active": 0, 2024-05-20T16:26:10.115 INFO:teuthology.orchestra.run.smithi184.stdout: "poolop_send": 0, 2024-05-20T16:26:10.115 INFO:teuthology.orchestra.run.smithi184.stdout: "poolop_resend": 0, 2024-05-20T16:26:10.115 INFO:teuthology.orchestra.run.smithi184.stdout: "poolstat_active": 0, 2024-05-20T16:26:10.115 INFO:teuthology.orchestra.run.smithi184.stdout: "poolstat_send": 0, 2024-05-20T16:26:10.115 INFO:teuthology.orchestra.run.smithi184.stdout: "poolstat_resend": 0, 2024-05-20T16:26:10.115 INFO:teuthology.orchestra.run.smithi184.stdout: "statfs_active": 0, 2024-05-20T16:26:10.115 INFO:teuthology.orchestra.run.smithi184.stdout: "statfs_send": 0, 2024-05-20T16:26:10.115 INFO:teuthology.orchestra.run.smithi184.stdout: "statfs_resend": 0, 2024-05-20T16:26:10.115 INFO:teuthology.orchestra.run.smithi184.stdout: "command_active": 0, 2024-05-20T16:26:10.115 INFO:teuthology.orchestra.run.smithi184.stdout: "command_send": 0, 2024-05-20T16:26:10.115 INFO:teuthology.orchestra.run.smithi184.stdout: "command_resend": 0, 2024-05-20T16:26:10.116 INFO:teuthology.orchestra.run.smithi184.stdout: "map_epoch": 0, 2024-05-20T16:26:10.116 INFO:teuthology.orchestra.run.smithi184.stdout: "map_full": 0, 2024-05-20T16:26:10.116 INFO:teuthology.orchestra.run.smithi184.stdout: "map_inc": 0, 2024-05-20T16:26:10.116 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_sessions": 7, 2024-05-20T16:26:10.116 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_session_open": 7, 2024-05-20T16:26:10.116 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_session_close": 0, 2024-05-20T16:26:10.116 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_laggy": 0, 2024-05-20T16:26:10.116 INFO:teuthology.orchestra.run.smithi184.stdout: "omap_wr": 0, 2024-05-20T16:26:10.116 INFO:teuthology.orchestra.run.smithi184.stdout: "omap_rd": 2, 2024-05-20T16:26:10.116 INFO:teuthology.orchestra.run.smithi184.stdout: "omap_del": 0 2024-05-20T16:26:10.116 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-20T16:26:10.116 INFO:teuthology.orchestra.run.smithi184.stdout: "oft": { 2024-05-20T16:26:10.116 INFO:teuthology.orchestra.run.smithi184.stdout: "omap_total_objs": 0, 2024-05-20T16:26:10.116 INFO:teuthology.orchestra.run.smithi184.stdout: "omap_total_kv_pairs": 0, 2024-05-20T16:26:10.116 INFO:teuthology.orchestra.run.smithi184.stdout: "omap_total_updates": 0, 2024-05-20T16:26:10.117 INFO:teuthology.orchestra.run.smithi184.stdout: "omap_total_removes": 0 2024-05-20T16:26:10.117 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-20T16:26:10.117 INFO:teuthology.orchestra.run.smithi184.stdout: "osd-slow-ops": { 2024-05-20T16:26:10.117 INFO:teuthology.orchestra.run.smithi184.stdout: "slow_ops_count": 0 2024-05-20T16:26:10.117 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-20T16:26:10.117 INFO:teuthology.orchestra.run.smithi184.stdout: "purge_queue": { 2024-05-20T16:26:10.117 INFO:teuthology.orchestra.run.smithi184.stdout: "pq_executing_ops": 0, 2024-05-20T16:26:10.117 INFO:teuthology.orchestra.run.smithi184.stdout: "pq_executing_ops_high_water": 0, 2024-05-20T16:26:10.117 INFO:teuthology.orchestra.run.smithi184.stdout: "pq_executing": 0, 2024-05-20T16:26:10.117 INFO:teuthology.orchestra.run.smithi184.stdout: "pq_executing_high_water": 0, 2024-05-20T16:26:10.117 INFO:teuthology.orchestra.run.smithi184.stdout: "pq_executed_ops": 0, 2024-05-20T16:26:10.117 INFO:teuthology.orchestra.run.smithi184.stdout: "pq_executed": 0, 2024-05-20T16:26:10.117 INFO:teuthology.orchestra.run.smithi184.stdout: "pq_item_in_journal": 0 2024-05-20T16:26:10.117 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-20T16:26:10.117 INFO:teuthology.orchestra.run.smithi184.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-20T16:26:10.117 INFO:teuthology.orchestra.run.smithi184.stdout: "val": 0, 2024-05-20T16:26:10.118 INFO:teuthology.orchestra.run.smithi184.stdout: "max": 104857600, 2024-05-20T16:26:10.118 INFO:teuthology.orchestra.run.smithi184.stdout: "get_started": 0, 2024-05-20T16:26:10.118 INFO:teuthology.orchestra.run.smithi184.stdout: "get": 335, 2024-05-20T16:26:10.118 INFO:teuthology.orchestra.run.smithi184.stdout: "get_sum": 69498425, 2024-05-20T16:26:10.118 INFO:teuthology.orchestra.run.smithi184.stdout: "get_or_fail_fail": 0, 2024-05-20T16:26:10.118 INFO:teuthology.orchestra.run.smithi184.stdout: "get_or_fail_success": 335, 2024-05-20T16:26:10.118 INFO:teuthology.orchestra.run.smithi184.stdout: "take": 0, 2024-05-20T16:26:10.118 INFO:teuthology.orchestra.run.smithi184.stdout: "take_sum": 0, 2024-05-20T16:26:10.118 INFO:teuthology.orchestra.run.smithi184.stdout: "put": 335, 2024-05-20T16:26:10.118 INFO:teuthology.orchestra.run.smithi184.stdout: "put_sum": 69498425, 2024-05-20T16:26:10.118 INFO:teuthology.orchestra.run.smithi184.stdout: "wait": { 2024-05-20T16:26:10.118 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-20T16:26:10.118 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-20T16:26:10.119 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-20T16:26:10.119 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-05-20T16:26:10.119 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-20T16:26:10.119 INFO:teuthology.orchestra.run.smithi184.stdout: "throttle-objecter_bytes": { 2024-05-20T16:26:10.119 INFO:teuthology.orchestra.run.smithi184.stdout: "val": 0, 2024-05-20T16:26:10.119 INFO:teuthology.orchestra.run.smithi184.stdout: "max": 104857600, 2024-05-20T16:26:10.119 INFO:teuthology.orchestra.run.smithi184.stdout: "get_started": 0, 2024-05-20T16:26:10.119 INFO:teuthology.orchestra.run.smithi184.stdout: "get": 0, 2024-05-20T16:26:10.119 INFO:teuthology.orchestra.run.smithi184.stdout: "get_sum": 0, 2024-05-20T16:26:10.119 INFO:teuthology.orchestra.run.smithi184.stdout: "get_or_fail_fail": 0, 2024-05-20T16:26:10.119 INFO:teuthology.orchestra.run.smithi184.stdout: "get_or_fail_success": 0, 2024-05-20T16:26:10.119 INFO:teuthology.orchestra.run.smithi184.stdout: "take": 304, 2024-05-20T16:26:10.119 INFO:teuthology.orchestra.run.smithi184.stdout: "take_sum": 69283483, 2024-05-20T16:26:10.119 INFO:teuthology.orchestra.run.smithi184.stdout: "put": 304, 2024-05-20T16:26:10.120 INFO:teuthology.orchestra.run.smithi184.stdout: "put_sum": 69283483, 2024-05-20T16:26:10.120 INFO:teuthology.orchestra.run.smithi184.stdout: "wait": { 2024-05-20T16:26:10.120 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-20T16:26:10.120 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-20T16:26:10.120 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-20T16:26:10.120 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-05-20T16:26:10.120 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-20T16:26:10.120 INFO:teuthology.orchestra.run.smithi184.stdout: "throttle-objecter_ops": { 2024-05-20T16:26:10.120 INFO:teuthology.orchestra.run.smithi184.stdout: "val": 0, 2024-05-20T16:26:10.120 INFO:teuthology.orchestra.run.smithi184.stdout: "max": 1024, 2024-05-20T16:26:10.120 INFO:teuthology.orchestra.run.smithi184.stdout: "get_started": 0, 2024-05-20T16:26:10.120 INFO:teuthology.orchestra.run.smithi184.stdout: "get": 0, 2024-05-20T16:26:10.121 INFO:teuthology.orchestra.run.smithi184.stdout: "get_sum": 0, 2024-05-20T16:26:10.121 INFO:teuthology.orchestra.run.smithi184.stdout: "get_or_fail_fail": 0, 2024-05-20T16:26:10.121 INFO:teuthology.orchestra.run.smithi184.stdout: "get_or_fail_success": 0, 2024-05-20T16:26:10.121 INFO:teuthology.orchestra.run.smithi184.stdout: "take": 304, 2024-05-20T16:26:10.121 INFO:teuthology.orchestra.run.smithi184.stdout: "take_sum": 304, 2024-05-20T16:26:10.121 INFO:teuthology.orchestra.run.smithi184.stdout: "put": 304, 2024-05-20T16:26:10.121 INFO:teuthology.orchestra.run.smithi184.stdout: "put_sum": 304, 2024-05-20T16:26:10.121 INFO:teuthology.orchestra.run.smithi184.stdout: "wait": { 2024-05-20T16:26:10.121 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-20T16:26:10.121 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-20T16:26:10.121 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-20T16:26:10.122 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-05-20T16:26:10.122 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-20T16:26:10.122 INFO:teuthology.orchestra.run.smithi184.stdout: "throttle-write_buf_throttle": { 2024-05-20T16:26:10.122 INFO:teuthology.orchestra.run.smithi184.stdout: "val": 0, 2024-05-20T16:26:10.122 INFO:teuthology.orchestra.run.smithi184.stdout: "max": 3758096384, 2024-05-20T16:26:10.122 INFO:teuthology.orchestra.run.smithi184.stdout: "get_started": 0, 2024-05-20T16:26:10.122 INFO:teuthology.orchestra.run.smithi184.stdout: "get": 0, 2024-05-20T16:26:10.122 INFO:teuthology.orchestra.run.smithi184.stdout: "get_sum": 0, 2024-05-20T16:26:10.122 INFO:teuthology.orchestra.run.smithi184.stdout: "get_or_fail_fail": 0, 2024-05-20T16:26:10.122 INFO:teuthology.orchestra.run.smithi184.stdout: "get_or_fail_success": 0, 2024-05-20T16:26:10.122 INFO:teuthology.orchestra.run.smithi184.stdout: "take": 0, 2024-05-20T16:26:10.122 INFO:teuthology.orchestra.run.smithi184.stdout: "take_sum": 0, 2024-05-20T16:26:10.122 INFO:teuthology.orchestra.run.smithi184.stdout: "put": 0, 2024-05-20T16:26:10.122 INFO:teuthology.orchestra.run.smithi184.stdout: "put_sum": 0, 2024-05-20T16:26:10.122 INFO:teuthology.orchestra.run.smithi184.stdout: "wait": { 2024-05-20T16:26:10.123 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-20T16:26:10.123 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-20T16:26:10.123 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-20T16:26:10.123 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-05-20T16:26:10.123 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-20T16:26:10.123 INFO:teuthology.orchestra.run.smithi184.stdout: "throttle-write_buf_throttle-0x55f4c95369c0": { 2024-05-20T16:26:10.123 INFO:teuthology.orchestra.run.smithi184.stdout: "val": 0, 2024-05-20T16:26:10.123 INFO:teuthology.orchestra.run.smithi184.stdout: "max": 3758096384, 2024-05-20T16:26:10.123 INFO:teuthology.orchestra.run.smithi184.stdout: "get_started": 0, 2024-05-20T16:26:10.123 INFO:teuthology.orchestra.run.smithi184.stdout: "get": 0, 2024-05-20T16:26:10.123 INFO:teuthology.orchestra.run.smithi184.stdout: "get_sum": 0, 2024-05-20T16:26:10.123 INFO:teuthology.orchestra.run.smithi184.stdout: "get_or_fail_fail": 0, 2024-05-20T16:26:10.123 INFO:teuthology.orchestra.run.smithi184.stdout: "get_or_fail_success": 0, 2024-05-20T16:26:10.123 INFO:teuthology.orchestra.run.smithi184.stdout: "take": 0, 2024-05-20T16:26:10.123 INFO:teuthology.orchestra.run.smithi184.stdout: "take_sum": 0, 2024-05-20T16:26:10.123 INFO:teuthology.orchestra.run.smithi184.stdout: "put": 0, 2024-05-20T16:26:10.124 INFO:teuthology.orchestra.run.smithi184.stdout: "put_sum": 0, 2024-05-20T16:26:10.124 INFO:teuthology.orchestra.run.smithi184.stdout: "wait": { 2024-05-20T16:26:10.124 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-20T16:26:10.124 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-20T16:26:10.124 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-20T16:26:10.124 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-05-20T16:26:10.124 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-05-20T16:26:10.124 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-05-20T16:26:11.010 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2024-05-20T16:26:11.010 INFO:tasks.check_counter:Daemon mds.c mds.imported=0 2024-05-20T16:26:11.010 DEBUG:tasks.check_counter:Getting stats from f 2024-05-20T16:26:11.011 DEBUG:teuthology.orchestra.run.smithi184:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell --fsid 59d1600a-16c2-11ef-bc9a-c7b262605968 -- ceph daemon mds.f perf dump 2024-05-20T16:26:11.045 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:26:10 smithi184 ceph-mon[36209]: pgmap v730: 129 pgs: 129 active+clean; 10 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 77 MiB/s wr, 186 op/s 2024-05-20T16:26:11.269 INFO:teuthology.orchestra.run.smithi184.stderr:Inferring config /var/lib/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/mon.c/config 2024-05-20T16:26:11.311 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:26:10 smithi160 ceph-mon[36347]: pgmap v730: 129 pgs: 129 active+clean; 10 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 77 MiB/s wr, 186 op/s 2024-05-20T16:26:11.358 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:26:10 smithi082 ceph-mon[32238]: pgmap v730: 129 pgs: 129 active+clean; 10 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 77 MiB/s wr, 186 op/s 2024-05-20T16:26:12.312 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:26:11 smithi160 ceph-mon[36347]: pgmap v731: 129 pgs: 129 active+clean; 9.8 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 65 MiB/s wr, 205 op/s 2024-05-20T16:26:12.358 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:26:11 smithi082 ceph-mon[32238]: pgmap v731: 129 pgs: 129 active+clean; 9.8 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 65 MiB/s wr, 205 op/s 2024-05-20T16:26:12.385 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:26:11 smithi184 ceph-mon[36209]: pgmap v731: 129 pgs: 129 active+clean; 9.8 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 65 MiB/s wr, 205 op/s 2024-05-20T16:26:13.311 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:26:12 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T16:26:13.358 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:26:12 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T16:26:13.385 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:26:12 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-20T16:26:14.562 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:26:14 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:26:14.562 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:26:14 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:26:14.562 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:26:14 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:26:14.562 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:26:14 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:26:14.562 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:26:14 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:26:14.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:26:14 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:26:14.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:26:14 smithi160 ceph-mon[36347]: pgmap v732: 129 pgs: 129 active+clean; 8.9 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 69 MiB/s wr, 289 op/s 2024-05-20T16:26:14.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:26:14 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:26:14.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:26:14 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:26:14.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:26:14 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:26:14.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:26:14 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:26:14.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:26:14 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:26:14.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:26:14 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:26:14.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:26:14 smithi082 ceph-mon[32238]: pgmap v732: 129 pgs: 129 active+clean; 8.9 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 69 MiB/s wr, 289 op/s 2024-05-20T16:26:14.635 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:26:14 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:26:14.635 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:26:14 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:26:14.635 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:26:14 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:26:14.635 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:26:14 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:26:14.635 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:26:14 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:26:14.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:26:14 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:26:14.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:26:14 smithi184 ceph-mon[36209]: pgmap v732: 129 pgs: 129 active+clean; 8.9 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 69 MiB/s wr, 289 op/s 2024-05-20T16:26:14.660 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-05-20T16:26:14.661 INFO:teuthology.orchestra.run.smithi184.stdout: "AsyncMessenger::Worker-0": { 2024-05-20T16:26:14.661 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_recv_messages": 2402, 2024-05-20T16:26:14.661 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_messages": 2404, 2024-05-20T16:26:14.661 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_recv_bytes": 681357, 2024-05-20T16:26:14.661 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_bytes": 763843, 2024-05-20T16:26:14.661 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_created_connections": 6, 2024-05-20T16:26:14.661 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_active_connections": 2, 2024-05-20T16:26:14.661 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_total_time": 0.234814485, 2024-05-20T16:26:14.661 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_send_time": 0.066179165, 2024-05-20T16:26:14.661 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_recv_time": 0.061886995, 2024-05-20T16:26:14.661 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_fast_dispatch_time": 0.090885498, 2024-05-20T16:26:14.661 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_messages_queue_lat": { 2024-05-20T16:26:14.662 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 2402, 2024-05-20T16:26:14.662 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.104769016, 2024-05-20T16:26:14.662 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000043617 2024-05-20T16:26:14.662 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-20T16:26:14.662 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_handle_ack_lat": { 2024-05-20T16:26:14.662 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-20T16:26:14.662 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-20T16:26:14.662 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-20T16:26:14.662 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-20T16:26:14.662 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-20T16:26:14.662 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_encrypted_bytes": 0 2024-05-20T16:26:14.662 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-20T16:26:14.662 INFO:teuthology.orchestra.run.smithi184.stdout: "AsyncMessenger::Worker-1": { 2024-05-20T16:26:14.663 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_recv_messages": 3, 2024-05-20T16:26:14.663 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_messages": 183, 2024-05-20T16:26:14.663 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_recv_bytes": 716, 2024-05-20T16:26:14.663 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_bytes": 391018, 2024-05-20T16:26:14.663 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_created_connections": 4, 2024-05-20T16:26:14.663 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_active_connections": 3, 2024-05-20T16:26:14.663 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_total_time": 0.011940275, 2024-05-20T16:26:14.663 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_send_time": 0.008639930, 2024-05-20T16:26:14.663 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_recv_time": 0.001399670, 2024-05-20T16:26:14.663 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_fast_dispatch_time": 0.000085948, 2024-05-20T16:26:14.663 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_messages_queue_lat": { 2024-05-20T16:26:14.663 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 181, 2024-05-20T16:26:14.663 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.013433146, 2024-05-20T16:26:14.663 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000074216 2024-05-20T16:26:14.664 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-20T16:26:14.664 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_handle_ack_lat": { 2024-05-20T16:26:14.664 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-20T16:26:14.664 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-20T16:26:14.664 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-20T16:26:14.664 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-20T16:26:14.664 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_recv_encrypted_bytes": 160, 2024-05-20T16:26:14.664 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_encrypted_bytes": 390320 2024-05-20T16:26:14.664 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-20T16:26:14.664 INFO:teuthology.orchestra.run.smithi184.stdout: "AsyncMessenger::Worker-2": { 2024-05-20T16:26:14.664 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_recv_messages": 267, 2024-05-20T16:26:14.664 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_messages": 253, 2024-05-20T16:26:14.664 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_recv_bytes": 809388, 2024-05-20T16:26:14.664 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_bytes": 149454, 2024-05-20T16:26:14.665 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_created_connections": 3, 2024-05-20T16:26:14.665 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_active_connections": 2, 2024-05-20T16:26:14.665 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_total_time": 0.046538785, 2024-05-20T16:26:14.665 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_send_time": 0.015571066, 2024-05-20T16:26:14.665 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_recv_time": 0.024934882, 2024-05-20T16:26:14.665 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_fast_dispatch_time": 0.008190039, 2024-05-20T16:26:14.665 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_messages_queue_lat": { 2024-05-20T16:26:14.665 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 253, 2024-05-20T16:26:14.665 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.018362672, 2024-05-20T16:26:14.665 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000072579 2024-05-20T16:26:14.665 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-20T16:26:14.665 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_handle_ack_lat": { 2024-05-20T16:26:14.665 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-20T16:26:14.666 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-20T16:26:14.666 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-20T16:26:14.666 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-20T16:26:14.666 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_recv_encrypted_bytes": 809120, 2024-05-20T16:26:14.666 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_encrypted_bytes": 149136 2024-05-20T16:26:14.666 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-20T16:26:14.666 INFO:teuthology.orchestra.run.smithi184.stdout: "cct": { 2024-05-20T16:26:14.666 INFO:teuthology.orchestra.run.smithi184.stdout: "total_workers": 1, 2024-05-20T16:26:14.666 INFO:teuthology.orchestra.run.smithi184.stdout: "unhealthy_workers": 0 2024-05-20T16:26:14.666 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-20T16:26:14.666 INFO:teuthology.orchestra.run.smithi184.stdout: "finisher-MDSRank": { 2024-05-20T16:26:14.666 INFO:teuthology.orchestra.run.smithi184.stdout: "queue_len": 0, 2024-05-20T16:26:14.666 INFO:teuthology.orchestra.run.smithi184.stdout: "complete_latency": { 2024-05-20T16:26:14.667 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 4002, 2024-05-20T16:26:14.667 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.130722940, 2024-05-20T16:26:14.667 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000032664 2024-05-20T16:26:14.667 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-05-20T16:26:14.667 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-20T16:26:14.667 INFO:teuthology.orchestra.run.smithi184.stdout: "finisher-PurgeQueue": { 2024-05-20T16:26:14.667 INFO:teuthology.orchestra.run.smithi184.stdout: "queue_len": 0, 2024-05-20T16:26:14.667 INFO:teuthology.orchestra.run.smithi184.stdout: "complete_latency": { 2024-05-20T16:26:14.667 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-20T16:26:14.667 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-20T16:26:14.667 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-20T16:26:14.667 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-05-20T16:26:14.667 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-20T16:26:14.667 INFO:teuthology.orchestra.run.smithi184.stdout: "mds": { 2024-05-20T16:26:14.668 INFO:teuthology.orchestra.run.smithi184.stdout: "request": 0, 2024-05-20T16:26:14.668 INFO:teuthology.orchestra.run.smithi184.stdout: "reply": 0, 2024-05-20T16:26:14.668 INFO:teuthology.orchestra.run.smithi184.stdout: "reply_latency": { 2024-05-20T16:26:14.668 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-20T16:26:14.668 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-20T16:26:14.668 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-20T16:26:14.668 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-20T16:26:14.668 INFO:teuthology.orchestra.run.smithi184.stdout: "slow_reply": 0, 2024-05-20T16:26:14.668 INFO:teuthology.orchestra.run.smithi184.stdout: "forward": 0, 2024-05-20T16:26:14.668 INFO:teuthology.orchestra.run.smithi184.stdout: "dir_fetch_complete": 0, 2024-05-20T16:26:14.668 INFO:teuthology.orchestra.run.smithi184.stdout: "dir_fetch_keys": 0, 2024-05-20T16:26:14.668 INFO:teuthology.orchestra.run.smithi184.stdout: "dir_commit": 0, 2024-05-20T16:26:14.669 INFO:teuthology.orchestra.run.smithi184.stdout: "dir_split": 0, 2024-05-20T16:26:14.669 INFO:teuthology.orchestra.run.smithi184.stdout: "dir_merge": 0, 2024-05-20T16:26:14.669 INFO:teuthology.orchestra.run.smithi184.stdout: "inodes": 0, 2024-05-20T16:26:14.669 INFO:teuthology.orchestra.run.smithi184.stdout: "inodes_top": 0, 2024-05-20T16:26:14.669 INFO:teuthology.orchestra.run.smithi184.stdout: "inodes_bottom": 0, 2024-05-20T16:26:14.669 INFO:teuthology.orchestra.run.smithi184.stdout: "inodes_pin_tail": 0, 2024-05-20T16:26:14.669 INFO:teuthology.orchestra.run.smithi184.stdout: "inodes_pinned": 0, 2024-05-20T16:26:14.669 INFO:teuthology.orchestra.run.smithi184.stdout: "inodes_expired": 0, 2024-05-20T16:26:14.669 INFO:teuthology.orchestra.run.smithi184.stdout: "inodes_with_caps": 0, 2024-05-20T16:26:14.669 INFO:teuthology.orchestra.run.smithi184.stdout: "caps": 0, 2024-05-20T16:26:14.669 INFO:teuthology.orchestra.run.smithi184.stdout: "subtrees": 1, 2024-05-20T16:26:14.669 INFO:teuthology.orchestra.run.smithi184.stdout: "traverse": 0, 2024-05-20T16:26:14.669 INFO:teuthology.orchestra.run.smithi184.stdout: "traverse_hit": 0, 2024-05-20T16:26:14.669 INFO:teuthology.orchestra.run.smithi184.stdout: "traverse_forward": 0, 2024-05-20T16:26:14.669 INFO:teuthology.orchestra.run.smithi184.stdout: "traverse_discover": 0, 2024-05-20T16:26:14.669 INFO:teuthology.orchestra.run.smithi184.stdout: "traverse_dir_fetch": 0, 2024-05-20T16:26:14.670 INFO:teuthology.orchestra.run.smithi184.stdout: "traverse_remote_ino": 0, 2024-05-20T16:26:14.670 INFO:teuthology.orchestra.run.smithi184.stdout: "traverse_lock": 0, 2024-05-20T16:26:14.670 INFO:teuthology.orchestra.run.smithi184.stdout: "load_cent": 0, 2024-05-20T16:26:14.670 INFO:teuthology.orchestra.run.smithi184.stdout: "q": 0, 2024-05-20T16:26:14.670 INFO:teuthology.orchestra.run.smithi184.stdout: "exported": 0, 2024-05-20T16:26:14.670 INFO:teuthology.orchestra.run.smithi184.stdout: "exported_inodes": 0, 2024-05-20T16:26:14.670 INFO:teuthology.orchestra.run.smithi184.stdout: "imported": 0, 2024-05-20T16:26:14.670 INFO:teuthology.orchestra.run.smithi184.stdout: "imported_inodes": 0, 2024-05-20T16:26:14.670 INFO:teuthology.orchestra.run.smithi184.stdout: "openino_dir_fetch": 0, 2024-05-20T16:26:14.670 INFO:teuthology.orchestra.run.smithi184.stdout: "openino_backtrace_fetch": 0, 2024-05-20T16:26:14.670 INFO:teuthology.orchestra.run.smithi184.stdout: "openino_peer_discover": 0, 2024-05-20T16:26:14.670 INFO:teuthology.orchestra.run.smithi184.stdout: "root_rfiles": 0, 2024-05-20T16:26:14.671 INFO:teuthology.orchestra.run.smithi184.stdout: "root_rbytes": 0, 2024-05-20T16:26:14.671 INFO:teuthology.orchestra.run.smithi184.stdout: "root_rsnaps": 0, 2024-05-20T16:26:14.671 INFO:teuthology.orchestra.run.smithi184.stdout: "scrub_backtrace_fetch": 0, 2024-05-20T16:26:14.671 INFO:teuthology.orchestra.run.smithi184.stdout: "scrub_set_tag": 0, 2024-05-20T16:26:14.671 INFO:teuthology.orchestra.run.smithi184.stdout: "scrub_backtrace_repaired": 0, 2024-05-20T16:26:14.671 INFO:teuthology.orchestra.run.smithi184.stdout: "scrub_inotable_repaired": 0, 2024-05-20T16:26:14.671 INFO:teuthology.orchestra.run.smithi184.stdout: "scrub_dir_inodes": 0, 2024-05-20T16:26:14.671 INFO:teuthology.orchestra.run.smithi184.stdout: "scrub_dir_base_inodes": 0, 2024-05-20T16:26:14.671 INFO:teuthology.orchestra.run.smithi184.stdout: "scrub_dirfrag_rstats": 0, 2024-05-20T16:26:14.671 INFO:teuthology.orchestra.run.smithi184.stdout: "scrub_file_inodes": 0, 2024-05-20T16:26:14.671 INFO:teuthology.orchestra.run.smithi184.stdout: "handle_inode_file_caps": 0, 2024-05-20T16:26:14.671 INFO:teuthology.orchestra.run.smithi184.stdout: "ceph_cap_op_revoke": 0, 2024-05-20T16:26:14.671 INFO:teuthology.orchestra.run.smithi184.stdout: "ceph_cap_op_grant": 0, 2024-05-20T16:26:14.672 INFO:teuthology.orchestra.run.smithi184.stdout: "ceph_cap_op_trunc": 0, 2024-05-20T16:26:14.672 INFO:teuthology.orchestra.run.smithi184.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-20T16:26:14.672 INFO:teuthology.orchestra.run.smithi184.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-20T16:26:14.672 INFO:teuthology.orchestra.run.smithi184.stdout: "handle_client_caps": 0, 2024-05-20T16:26:14.672 INFO:teuthology.orchestra.run.smithi184.stdout: "handle_client_caps_dirty": 0, 2024-05-20T16:26:14.672 INFO:teuthology.orchestra.run.smithi184.stdout: "handle_client_cap_release": 0, 2024-05-20T16:26:14.672 INFO:teuthology.orchestra.run.smithi184.stdout: "process_request_cap_release": 0 2024-05-20T16:26:14.672 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-20T16:26:14.672 INFO:teuthology.orchestra.run.smithi184.stdout: "mds_cache": { 2024-05-20T16:26:14.672 INFO:teuthology.orchestra.run.smithi184.stdout: "dir_update": 0, 2024-05-20T16:26:14.672 INFO:teuthology.orchestra.run.smithi184.stdout: "dir_update_receipt": 0, 2024-05-20T16:26:14.672 INFO:teuthology.orchestra.run.smithi184.stdout: "dir_try_discover": 0, 2024-05-20T16:26:14.672 INFO:teuthology.orchestra.run.smithi184.stdout: "dir_send_discover": 0, 2024-05-20T16:26:14.672 INFO:teuthology.orchestra.run.smithi184.stdout: "dir_handle_discover": 0, 2024-05-20T16:26:14.673 INFO:teuthology.orchestra.run.smithi184.stdout: "num_strays": 0, 2024-05-20T16:26:14.673 INFO:teuthology.orchestra.run.smithi184.stdout: "num_strays_delayed": 0, 2024-05-20T16:26:14.673 INFO:teuthology.orchestra.run.smithi184.stdout: "num_strays_enqueuing": 0, 2024-05-20T16:26:14.673 INFO:teuthology.orchestra.run.smithi184.stdout: "strays_created": 0, 2024-05-20T16:26:14.673 INFO:teuthology.orchestra.run.smithi184.stdout: "strays_enqueued": 0, 2024-05-20T16:26:14.673 INFO:teuthology.orchestra.run.smithi184.stdout: "strays_reintegrated": 0, 2024-05-20T16:26:14.673 INFO:teuthology.orchestra.run.smithi184.stdout: "strays_migrated": 0, 2024-05-20T16:26:14.673 INFO:teuthology.orchestra.run.smithi184.stdout: "num_recovering_processing": 0, 2024-05-20T16:26:14.673 INFO:teuthology.orchestra.run.smithi184.stdout: "num_recovering_enqueued": 0, 2024-05-20T16:26:14.673 INFO:teuthology.orchestra.run.smithi184.stdout: "num_recovering_prioritized": 0, 2024-05-20T16:26:14.673 INFO:teuthology.orchestra.run.smithi184.stdout: "recovery_started": 0, 2024-05-20T16:26:14.673 INFO:teuthology.orchestra.run.smithi184.stdout: "recovery_completed": 0, 2024-05-20T16:26:14.673 INFO:teuthology.orchestra.run.smithi184.stdout: "ireq_quiesce_path": 0, 2024-05-20T16:26:14.673 INFO:teuthology.orchestra.run.smithi184.stdout: "ireq_quiesce_inode": 0, 2024-05-20T16:26:14.674 INFO:teuthology.orchestra.run.smithi184.stdout: "ireq_enqueue_scrub": 0, 2024-05-20T16:26:14.674 INFO:teuthology.orchestra.run.smithi184.stdout: "ireq_exportdir": 0, 2024-05-20T16:26:14.674 INFO:teuthology.orchestra.run.smithi184.stdout: "ireq_flush": 0, 2024-05-20T16:26:14.674 INFO:teuthology.orchestra.run.smithi184.stdout: "ireq_fragmentdir": 0, 2024-05-20T16:26:14.674 INFO:teuthology.orchestra.run.smithi184.stdout: "ireq_fragstats": 0, 2024-05-20T16:26:14.674 INFO:teuthology.orchestra.run.smithi184.stdout: "ireq_inodestats": 0 2024-05-20T16:26:14.674 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-20T16:26:14.674 INFO:teuthology.orchestra.run.smithi184.stdout: "mds_log": { 2024-05-20T16:26:14.674 INFO:teuthology.orchestra.run.smithi184.stdout: "evlrg": 0, 2024-05-20T16:26:14.674 INFO:teuthology.orchestra.run.smithi184.stdout: "evadd": 0, 2024-05-20T16:26:14.674 INFO:teuthology.orchestra.run.smithi184.stdout: "evex": 0, 2024-05-20T16:26:14.674 INFO:teuthology.orchestra.run.smithi184.stdout: "evtrm": 0, 2024-05-20T16:26:14.674 INFO:teuthology.orchestra.run.smithi184.stdout: "ev": 4, 2024-05-20T16:26:14.675 INFO:teuthology.orchestra.run.smithi184.stdout: "evexg": 0, 2024-05-20T16:26:14.675 INFO:teuthology.orchestra.run.smithi184.stdout: "evexd": 0, 2024-05-20T16:26:14.675 INFO:teuthology.orchestra.run.smithi184.stdout: "segadd": 0, 2024-05-20T16:26:14.675 INFO:teuthology.orchestra.run.smithi184.stdout: "segex": 0, 2024-05-20T16:26:14.675 INFO:teuthology.orchestra.run.smithi184.stdout: "segtrm": 0, 2024-05-20T16:26:14.675 INFO:teuthology.orchestra.run.smithi184.stdout: "seg": 2, 2024-05-20T16:26:14.675 INFO:teuthology.orchestra.run.smithi184.stdout: "segmjr": 2, 2024-05-20T16:26:14.675 INFO:teuthology.orchestra.run.smithi184.stdout: "segexg": 0, 2024-05-20T16:26:14.675 INFO:teuthology.orchestra.run.smithi184.stdout: "segexd": 0, 2024-05-20T16:26:14.675 INFO:teuthology.orchestra.run.smithi184.stdout: "expos": 4194304, 2024-05-20T16:26:14.675 INFO:teuthology.orchestra.run.smithi184.stdout: "wrpos": 4195424, 2024-05-20T16:26:14.675 INFO:teuthology.orchestra.run.smithi184.stdout: "rdpos": 4195283, 2024-05-20T16:26:14.675 INFO:teuthology.orchestra.run.smithi184.stdout: "jlat": { 2024-05-20T16:26:14.675 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-20T16:26:14.676 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-20T16:26:14.676 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-20T16:26:14.676 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-20T16:26:14.676 INFO:teuthology.orchestra.run.smithi184.stdout: "replayed": 4 2024-05-20T16:26:14.676 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-20T16:26:14.676 INFO:teuthology.orchestra.run.smithi184.stdout: "mds_mem": { 2024-05-20T16:26:14.676 INFO:teuthology.orchestra.run.smithi184.stdout: "ino": 3, 2024-05-20T16:26:14.676 INFO:teuthology.orchestra.run.smithi184.stdout: "ino+": 3, 2024-05-20T16:26:14.676 INFO:teuthology.orchestra.run.smithi184.stdout: "ino-": 0, 2024-05-20T16:26:14.676 INFO:teuthology.orchestra.run.smithi184.stdout: "dir": 1, 2024-05-20T16:26:14.676 INFO:teuthology.orchestra.run.smithi184.stdout: "dir+": 1, 2024-05-20T16:26:14.676 INFO:teuthology.orchestra.run.smithi184.stdout: "dir-": 0, 2024-05-20T16:26:14.676 INFO:teuthology.orchestra.run.smithi184.stdout: "dn": 0, 2024-05-20T16:26:14.676 INFO:teuthology.orchestra.run.smithi184.stdout: "dn+": 0, 2024-05-20T16:26:14.677 INFO:teuthology.orchestra.run.smithi184.stdout: "dn-": 0, 2024-05-20T16:26:14.677 INFO:teuthology.orchestra.run.smithi184.stdout: "cap": 0, 2024-05-20T16:26:14.677 INFO:teuthology.orchestra.run.smithi184.stdout: "cap+": 0, 2024-05-20T16:26:14.677 INFO:teuthology.orchestra.run.smithi184.stdout: "cap-": 0, 2024-05-20T16:26:14.677 INFO:teuthology.orchestra.run.smithi184.stdout: "rss": 118580, 2024-05-20T16:26:14.677 INFO:teuthology.orchestra.run.smithi184.stdout: "heap": 215444 2024-05-20T16:26:14.677 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-20T16:26:14.677 INFO:teuthology.orchestra.run.smithi184.stdout: "mds_server": { 2024-05-20T16:26:14.677 INFO:teuthology.orchestra.run.smithi184.stdout: "dispatch_client_request": 0, 2024-05-20T16:26:14.677 INFO:teuthology.orchestra.run.smithi184.stdout: "dispatch_server_request": 0, 2024-05-20T16:26:14.677 INFO:teuthology.orchestra.run.smithi184.stdout: "handle_client_request": 0, 2024-05-20T16:26:14.677 INFO:teuthology.orchestra.run.smithi184.stdout: "handle_client_session": 0, 2024-05-20T16:26:14.677 INFO:teuthology.orchestra.run.smithi184.stdout: "handle_peer_request": 0, 2024-05-20T16:26:14.677 INFO:teuthology.orchestra.run.smithi184.stdout: "req_create_latency": { 2024-05-20T16:26:14.677 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-20T16:26:14.678 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-20T16:26:14.678 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-20T16:26:14.678 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-20T16:26:14.678 INFO:teuthology.orchestra.run.smithi184.stdout: "req_getattr_latency": { 2024-05-20T16:26:14.678 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-20T16:26:14.678 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-20T16:26:14.678 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-20T16:26:14.678 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-20T16:26:14.678 INFO:teuthology.orchestra.run.smithi184.stdout: "req_getfilelock_latency": { 2024-05-20T16:26:14.678 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-20T16:26:14.678 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-20T16:26:14.678 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-20T16:26:14.678 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-20T16:26:14.678 INFO:teuthology.orchestra.run.smithi184.stdout: "req_link_latency": { 2024-05-20T16:26:14.679 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-20T16:26:14.679 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-20T16:26:14.679 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-20T16:26:14.679 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-20T16:26:14.679 INFO:teuthology.orchestra.run.smithi184.stdout: "req_lookup_latency": { 2024-05-20T16:26:14.679 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-20T16:26:14.679 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-20T16:26:14.679 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-20T16:26:14.679 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-20T16:26:14.679 INFO:teuthology.orchestra.run.smithi184.stdout: "req_lookuphash_latency": { 2024-05-20T16:26:14.679 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-20T16:26:14.679 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-20T16:26:14.679 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-20T16:26:14.679 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-20T16:26:14.679 INFO:teuthology.orchestra.run.smithi184.stdout: "req_lookupino_latency": { 2024-05-20T16:26:14.680 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-20T16:26:14.680 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-20T16:26:14.680 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-20T16:26:14.680 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-20T16:26:14.680 INFO:teuthology.orchestra.run.smithi184.stdout: "req_lookupname_latency": { 2024-05-20T16:26:14.680 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-20T16:26:14.680 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-20T16:26:14.680 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-20T16:26:14.680 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-20T16:26:14.680 INFO:teuthology.orchestra.run.smithi184.stdout: "req_lookupparent_latency": { 2024-05-20T16:26:14.680 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-20T16:26:14.680 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-20T16:26:14.680 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-20T16:26:14.680 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-20T16:26:14.680 INFO:teuthology.orchestra.run.smithi184.stdout: "req_lookupsnap_latency": { 2024-05-20T16:26:14.681 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-20T16:26:14.681 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-20T16:26:14.681 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-20T16:26:14.681 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-20T16:26:14.681 INFO:teuthology.orchestra.run.smithi184.stdout: "req_lssnap_latency": { 2024-05-20T16:26:14.681 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-20T16:26:14.681 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-20T16:26:14.681 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-20T16:26:14.681 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-20T16:26:14.681 INFO:teuthology.orchestra.run.smithi184.stdout: "req_mkdir_latency": { 2024-05-20T16:26:14.681 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-20T16:26:14.682 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-20T16:26:14.682 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-20T16:26:14.682 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-20T16:26:14.682 INFO:teuthology.orchestra.run.smithi184.stdout: "req_mknod_latency": { 2024-05-20T16:26:14.682 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-20T16:26:14.682 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-20T16:26:14.682 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-20T16:26:14.682 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-20T16:26:14.682 INFO:teuthology.orchestra.run.smithi184.stdout: "req_mksnap_latency": { 2024-05-20T16:26:14.683 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-20T16:26:14.683 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-20T16:26:14.683 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-20T16:26:14.683 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-20T16:26:14.683 INFO:teuthology.orchestra.run.smithi184.stdout: "req_open_latency": { 2024-05-20T16:26:14.683 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-20T16:26:14.683 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-20T16:26:14.683 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-20T16:26:14.683 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-20T16:26:14.683 INFO:teuthology.orchestra.run.smithi184.stdout: "req_readdir_latency": { 2024-05-20T16:26:14.683 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-20T16:26:14.683 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-20T16:26:14.683 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-20T16:26:14.683 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-20T16:26:14.683 INFO:teuthology.orchestra.run.smithi184.stdout: "req_rename_latency": { 2024-05-20T16:26:14.684 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-20T16:26:14.684 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-20T16:26:14.684 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-20T16:26:14.684 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-20T16:26:14.684 INFO:teuthology.orchestra.run.smithi184.stdout: "req_renamesnap_latency": { 2024-05-20T16:26:14.684 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-20T16:26:14.684 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-20T16:26:14.684 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-20T16:26:14.684 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-20T16:26:14.684 INFO:teuthology.orchestra.run.smithi184.stdout: "req_snapdiff_latency": { 2024-05-20T16:26:14.684 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-20T16:26:14.684 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-20T16:26:14.684 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-20T16:26:14.684 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-20T16:26:14.685 INFO:teuthology.orchestra.run.smithi184.stdout: "req_rmdir_latency": { 2024-05-20T16:26:14.685 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-20T16:26:14.685 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-20T16:26:14.685 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-20T16:26:14.685 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-20T16:26:14.685 INFO:teuthology.orchestra.run.smithi184.stdout: "req_rmsnap_latency": { 2024-05-20T16:26:14.685 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-20T16:26:14.685 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-20T16:26:14.685 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-20T16:26:14.685 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-20T16:26:14.685 INFO:teuthology.orchestra.run.smithi184.stdout: "req_rmxattr_latency": { 2024-05-20T16:26:14.685 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-20T16:26:14.685 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-20T16:26:14.686 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-20T16:26:14.686 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-20T16:26:14.686 INFO:teuthology.orchestra.run.smithi184.stdout: "req_setattr_latency": { 2024-05-20T16:26:14.686 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-20T16:26:14.686 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-20T16:26:14.686 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-20T16:26:14.686 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-20T16:26:14.686 INFO:teuthology.orchestra.run.smithi184.stdout: "req_setdirlayout_latency": { 2024-05-20T16:26:14.686 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-20T16:26:14.686 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-20T16:26:14.686 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-20T16:26:14.686 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-20T16:26:14.686 INFO:teuthology.orchestra.run.smithi184.stdout: "req_setfilelock_latency": { 2024-05-20T16:26:14.686 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-20T16:26:14.687 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-20T16:26:14.687 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-20T16:26:14.687 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-20T16:26:14.687 INFO:teuthology.orchestra.run.smithi184.stdout: "req_setlayout_latency": { 2024-05-20T16:26:14.687 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-20T16:26:14.687 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-20T16:26:14.687 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-20T16:26:14.687 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-20T16:26:14.687 INFO:teuthology.orchestra.run.smithi184.stdout: "req_setxattr_latency": { 2024-05-20T16:26:14.687 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-20T16:26:14.687 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-20T16:26:14.687 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-20T16:26:14.687 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-20T16:26:14.687 INFO:teuthology.orchestra.run.smithi184.stdout: "req_symlink_latency": { 2024-05-20T16:26:14.688 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-20T16:26:14.688 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-20T16:26:14.688 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-20T16:26:14.688 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-20T16:26:14.688 INFO:teuthology.orchestra.run.smithi184.stdout: "req_unlink_latency": { 2024-05-20T16:26:14.688 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-20T16:26:14.688 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-20T16:26:14.688 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-20T16:26:14.688 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-20T16:26:14.688 INFO:teuthology.orchestra.run.smithi184.stdout: "cap_revoke_eviction": 0, 2024-05-20T16:26:14.688 INFO:teuthology.orchestra.run.smithi184.stdout: "cap_acquisition_throttle": 0, 2024-05-20T16:26:14.688 INFO:teuthology.orchestra.run.smithi184.stdout: "req_getvxattr_latency": { 2024-05-20T16:26:14.688 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-20T16:26:14.688 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-20T16:26:14.688 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-20T16:26:14.689 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-05-20T16:26:14.689 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-20T16:26:14.689 INFO:teuthology.orchestra.run.smithi184.stdout: "mds_sessions": { 2024-05-20T16:26:14.689 INFO:teuthology.orchestra.run.smithi184.stdout: "session_count": 0, 2024-05-20T16:26:14.689 INFO:teuthology.orchestra.run.smithi184.stdout: "session_add": 1, 2024-05-20T16:26:14.689 INFO:teuthology.orchestra.run.smithi184.stdout: "session_remove": 1, 2024-05-20T16:26:14.689 INFO:teuthology.orchestra.run.smithi184.stdout: "sessions_open": 1, 2024-05-20T16:26:14.689 INFO:teuthology.orchestra.run.smithi184.stdout: "sessions_stale": 0, 2024-05-20T16:26:14.689 INFO:teuthology.orchestra.run.smithi184.stdout: "total_load": 0, 2024-05-20T16:26:14.689 INFO:teuthology.orchestra.run.smithi184.stdout: "average_load": 0, 2024-05-20T16:26:14.689 INFO:teuthology.orchestra.run.smithi184.stdout: "avg_session_uptime": 1442, 2024-05-20T16:26:14.689 INFO:teuthology.orchestra.run.smithi184.stdout: "mdthresh_evicted": 0 2024-05-20T16:26:14.689 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-20T16:26:14.689 INFO:teuthology.orchestra.run.smithi184.stdout: "mempool": { 2024-05-20T16:26:14.689 INFO:teuthology.orchestra.run.smithi184.stdout: "bloom_filter_bytes": 0, 2024-05-20T16:26:14.690 INFO:teuthology.orchestra.run.smithi184.stdout: "bloom_filter_items": 0, 2024-05-20T16:26:14.690 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_alloc_bytes": 0, 2024-05-20T16:26:14.690 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_alloc_items": 0, 2024-05-20T16:26:14.690 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_data_bytes": 0, 2024-05-20T16:26:14.690 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_data_items": 0, 2024-05-20T16:26:14.690 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-20T16:26:14.690 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_onode_items": 0, 2024-05-20T16:26:14.690 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-20T16:26:14.690 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_meta_items": 0, 2024-05-20T16:26:14.690 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_other_bytes": 0, 2024-05-20T16:26:14.690 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_other_items": 0, 2024-05-20T16:26:14.690 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-20T16:26:14.690 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_buffer_items": 0, 2024-05-20T16:26:14.690 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_extent_bytes": 0, 2024-05-20T16:26:14.691 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_extent_items": 0, 2024-05-20T16:26:14.691 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_blob_bytes": 0, 2024-05-20T16:26:14.691 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_blob_items": 0, 2024-05-20T16:26:14.691 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-20T16:26:14.691 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_shared_blob_items": 0, 2024-05-20T16:26:14.691 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-20T16:26:14.691 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_inline_bl_items": 0, 2024-05-20T16:26:14.691 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_fsck_bytes": 0, 2024-05-20T16:26:14.691 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_fsck_items": 0, 2024-05-20T16:26:14.691 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_txc_bytes": 0, 2024-05-20T16:26:14.691 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_txc_items": 0, 2024-05-20T16:26:14.691 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-20T16:26:14.691 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_writing_deferred_items": 0, 2024-05-20T16:26:14.691 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_writing_bytes": 0, 2024-05-20T16:26:14.691 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_writing_items": 0, 2024-05-20T16:26:14.692 INFO:teuthology.orchestra.run.smithi184.stdout: "bluefs_bytes": 0, 2024-05-20T16:26:14.692 INFO:teuthology.orchestra.run.smithi184.stdout: "bluefs_items": 0, 2024-05-20T16:26:14.692 INFO:teuthology.orchestra.run.smithi184.stdout: "bluefs_file_reader_bytes": 0, 2024-05-20T16:26:14.692 INFO:teuthology.orchestra.run.smithi184.stdout: "bluefs_file_reader_items": 0, 2024-05-20T16:26:14.692 INFO:teuthology.orchestra.run.smithi184.stdout: "bluefs_file_writer_bytes": 0, 2024-05-20T16:26:14.692 INFO:teuthology.orchestra.run.smithi184.stdout: "bluefs_file_writer_items": 0, 2024-05-20T16:26:14.692 INFO:teuthology.orchestra.run.smithi184.stdout: "buffer_anon_bytes": 181303, 2024-05-20T16:26:14.692 INFO:teuthology.orchestra.run.smithi184.stdout: "buffer_anon_items": 46, 2024-05-20T16:26:14.692 INFO:teuthology.orchestra.run.smithi184.stdout: "buffer_meta_bytes": 0, 2024-05-20T16:26:14.692 INFO:teuthology.orchestra.run.smithi184.stdout: "buffer_meta_items": 0, 2024-05-20T16:26:14.692 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_bytes": 0, 2024-05-20T16:26:14.692 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_items": 0, 2024-05-20T16:26:14.692 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_mapbl_bytes": 0, 2024-05-20T16:26:14.692 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_mapbl_items": 0, 2024-05-20T16:26:14.693 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_pglog_bytes": 0, 2024-05-20T16:26:14.693 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_pglog_items": 0, 2024-05-20T16:26:14.693 INFO:teuthology.orchestra.run.smithi184.stdout: "osdmap_bytes": 8888, 2024-05-20T16:26:14.693 INFO:teuthology.orchestra.run.smithi184.stdout: "osdmap_items": 170, 2024-05-20T16:26:14.693 INFO:teuthology.orchestra.run.smithi184.stdout: "osdmap_mapping_bytes": 0, 2024-05-20T16:26:14.693 INFO:teuthology.orchestra.run.smithi184.stdout: "osdmap_mapping_items": 0, 2024-05-20T16:26:14.693 INFO:teuthology.orchestra.run.smithi184.stdout: "pgmap_bytes": 0, 2024-05-20T16:26:14.693 INFO:teuthology.orchestra.run.smithi184.stdout: "pgmap_items": 0, 2024-05-20T16:26:14.693 INFO:teuthology.orchestra.run.smithi184.stdout: "mds_co_bytes": 8464, 2024-05-20T16:26:14.693 INFO:teuthology.orchestra.run.smithi184.stdout: "mds_co_items": 13, 2024-05-20T16:26:14.693 INFO:teuthology.orchestra.run.smithi184.stdout: "unittest_1_bytes": 0, 2024-05-20T16:26:14.693 INFO:teuthology.orchestra.run.smithi184.stdout: "unittest_1_items": 0, 2024-05-20T16:26:14.693 INFO:teuthology.orchestra.run.smithi184.stdout: "unittest_2_bytes": 0, 2024-05-20T16:26:14.693 INFO:teuthology.orchestra.run.smithi184.stdout: "unittest_2_items": 0 2024-05-20T16:26:14.693 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-20T16:26:14.694 INFO:teuthology.orchestra.run.smithi184.stdout: "objecter": { 2024-05-20T16:26:14.694 INFO:teuthology.orchestra.run.smithi184.stdout: "op_active": 0, 2024-05-20T16:26:14.694 INFO:teuthology.orchestra.run.smithi184.stdout: "op_laggy": 0, 2024-05-20T16:26:14.694 INFO:teuthology.orchestra.run.smithi184.stdout: "op_send": 2405, 2024-05-20T16:26:14.694 INFO:teuthology.orchestra.run.smithi184.stdout: "op_send_bytes": 21, 2024-05-20T16:26:14.694 INFO:teuthology.orchestra.run.smithi184.stdout: "op_resend": 3, 2024-05-20T16:26:14.694 INFO:teuthology.orchestra.run.smithi184.stdout: "op_reply": 2405, 2024-05-20T16:26:14.694 INFO:teuthology.orchestra.run.smithi184.stdout: "op_latency": { 2024-05-20T16:26:14.694 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 2405, 2024-05-20T16:26:14.694 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 6.069884173, 2024-05-20T16:26:14.694 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.002523860 2024-05-20T16:26:14.694 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-20T16:26:14.694 INFO:teuthology.orchestra.run.smithi184.stdout: "op_inflight": 0, 2024-05-20T16:26:14.694 INFO:teuthology.orchestra.run.smithi184.stdout: "oplen_avg": { 2024-05-20T16:26:14.695 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 2405, 2024-05-20T16:26:14.695 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 2406 2024-05-20T16:26:14.695 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-20T16:26:14.695 INFO:teuthology.orchestra.run.smithi184.stdout: "op": 2405, 2024-05-20T16:26:14.695 INFO:teuthology.orchestra.run.smithi184.stdout: "op_r": 2405, 2024-05-20T16:26:14.695 INFO:teuthology.orchestra.run.smithi184.stdout: "op_w": 0, 2024-05-20T16:26:14.695 INFO:teuthology.orchestra.run.smithi184.stdout: "op_rmw": 0, 2024-05-20T16:26:14.695 INFO:teuthology.orchestra.run.smithi184.stdout: "op_pg": 0, 2024-05-20T16:26:14.695 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_stat": 1560, 2024-05-20T16:26:14.695 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_create": 0, 2024-05-20T16:26:14.695 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_read": 843, 2024-05-20T16:26:14.695 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_write": 0, 2024-05-20T16:26:14.695 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_writefull": 0, 2024-05-20T16:26:14.695 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_writesame": 0, 2024-05-20T16:26:14.695 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_append": 0, 2024-05-20T16:26:14.695 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_zero": 0, 2024-05-20T16:26:14.696 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_truncate": 0, 2024-05-20T16:26:14.696 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_delete": 0, 2024-05-20T16:26:14.696 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_mapext": 0, 2024-05-20T16:26:14.696 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_sparse_read": 0, 2024-05-20T16:26:14.696 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_clonerange": 0, 2024-05-20T16:26:14.696 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_getxattr": 1, 2024-05-20T16:26:14.696 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_setxattr": 0, 2024-05-20T16:26:14.696 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_cmpxattr": 0, 2024-05-20T16:26:14.696 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_rmxattr": 0, 2024-05-20T16:26:14.696 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_resetxattrs": 0, 2024-05-20T16:26:14.696 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_call": 0, 2024-05-20T16:26:14.696 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_watch": 0, 2024-05-20T16:26:14.696 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_notify": 0, 2024-05-20T16:26:14.696 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_src_cmpxattr": 0, 2024-05-20T16:26:14.696 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_pgls": 0, 2024-05-20T16:26:14.696 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_pgls_filter": 0, 2024-05-20T16:26:14.697 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_other": 0, 2024-05-20T16:26:14.697 INFO:teuthology.orchestra.run.smithi184.stdout: "linger_active": 0, 2024-05-20T16:26:14.697 INFO:teuthology.orchestra.run.smithi184.stdout: "linger_send": 0, 2024-05-20T16:26:14.697 INFO:teuthology.orchestra.run.smithi184.stdout: "linger_resend": 0, 2024-05-20T16:26:14.697 INFO:teuthology.orchestra.run.smithi184.stdout: "linger_ping": 0, 2024-05-20T16:26:14.697 INFO:teuthology.orchestra.run.smithi184.stdout: "poolop_active": 0, 2024-05-20T16:26:14.697 INFO:teuthology.orchestra.run.smithi184.stdout: "poolop_send": 0, 2024-05-20T16:26:14.697 INFO:teuthology.orchestra.run.smithi184.stdout: "poolop_resend": 0, 2024-05-20T16:26:14.697 INFO:teuthology.orchestra.run.smithi184.stdout: "poolstat_active": 0, 2024-05-20T16:26:14.697 INFO:teuthology.orchestra.run.smithi184.stdout: "poolstat_send": 0, 2024-05-20T16:26:14.697 INFO:teuthology.orchestra.run.smithi184.stdout: "poolstat_resend": 0, 2024-05-20T16:26:14.697 INFO:teuthology.orchestra.run.smithi184.stdout: "statfs_active": 0, 2024-05-20T16:26:14.697 INFO:teuthology.orchestra.run.smithi184.stdout: "statfs_send": 0, 2024-05-20T16:26:14.697 INFO:teuthology.orchestra.run.smithi184.stdout: "statfs_resend": 0, 2024-05-20T16:26:14.697 INFO:teuthology.orchestra.run.smithi184.stdout: "command_active": 0, 2024-05-20T16:26:14.698 INFO:teuthology.orchestra.run.smithi184.stdout: "command_send": 0, 2024-05-20T16:26:14.698 INFO:teuthology.orchestra.run.smithi184.stdout: "command_resend": 0, 2024-05-20T16:26:14.698 INFO:teuthology.orchestra.run.smithi184.stdout: "map_epoch": 98, 2024-05-20T16:26:14.698 INFO:teuthology.orchestra.run.smithi184.stdout: "map_full": 0, 2024-05-20T16:26:14.698 INFO:teuthology.orchestra.run.smithi184.stdout: "map_inc": 7, 2024-05-20T16:26:14.698 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_sessions": 4, 2024-05-20T16:26:14.698 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_session_open": 4, 2024-05-20T16:26:14.698 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_session_close": 0, 2024-05-20T16:26:14.698 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_laggy": 0, 2024-05-20T16:26:14.698 INFO:teuthology.orchestra.run.smithi184.stdout: "omap_wr": 0, 2024-05-20T16:26:14.698 INFO:teuthology.orchestra.run.smithi184.stdout: "omap_rd": 2, 2024-05-20T16:26:14.698 INFO:teuthology.orchestra.run.smithi184.stdout: "omap_del": 0 2024-05-20T16:26:14.698 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-20T16:26:14.698 INFO:teuthology.orchestra.run.smithi184.stdout: "oft": { 2024-05-20T16:26:14.699 INFO:teuthology.orchestra.run.smithi184.stdout: "omap_total_objs": 0, 2024-05-20T16:26:14.699 INFO:teuthology.orchestra.run.smithi184.stdout: "omap_total_kv_pairs": 0, 2024-05-20T16:26:14.699 INFO:teuthology.orchestra.run.smithi184.stdout: "omap_total_updates": 0, 2024-05-20T16:26:14.699 INFO:teuthology.orchestra.run.smithi184.stdout: "omap_total_removes": 0 2024-05-20T16:26:14.699 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-20T16:26:14.699 INFO:teuthology.orchestra.run.smithi184.stdout: "osd-slow-ops": { 2024-05-20T16:26:14.699 INFO:teuthology.orchestra.run.smithi184.stdout: "slow_ops_count": 0 2024-05-20T16:26:14.699 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-20T16:26:14.699 INFO:teuthology.orchestra.run.smithi184.stdout: "purge_queue": { 2024-05-20T16:26:14.699 INFO:teuthology.orchestra.run.smithi184.stdout: "pq_executing_ops": 0, 2024-05-20T16:26:14.699 INFO:teuthology.orchestra.run.smithi184.stdout: "pq_executing_ops_high_water": 0, 2024-05-20T16:26:14.699 INFO:teuthology.orchestra.run.smithi184.stdout: "pq_executing": 0, 2024-05-20T16:26:14.699 INFO:teuthology.orchestra.run.smithi184.stdout: "pq_executing_high_water": 0, 2024-05-20T16:26:14.699 INFO:teuthology.orchestra.run.smithi184.stdout: "pq_executed_ops": 0, 2024-05-20T16:26:14.699 INFO:teuthology.orchestra.run.smithi184.stdout: "pq_executed": 0, 2024-05-20T16:26:14.699 INFO:teuthology.orchestra.run.smithi184.stdout: "pq_item_in_journal": 0 2024-05-20T16:26:14.700 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-20T16:26:14.700 INFO:teuthology.orchestra.run.smithi184.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-20T16:26:14.700 INFO:teuthology.orchestra.run.smithi184.stdout: "val": 0, 2024-05-20T16:26:14.700 INFO:teuthology.orchestra.run.smithi184.stdout: "max": 104857600, 2024-05-20T16:26:14.700 INFO:teuthology.orchestra.run.smithi184.stdout: "get_started": 0, 2024-05-20T16:26:14.700 INFO:teuthology.orchestra.run.smithi184.stdout: "get": 2668, 2024-05-20T16:26:14.700 INFO:teuthology.orchestra.run.smithi184.stdout: "get_sum": 1234938, 2024-05-20T16:26:14.700 INFO:teuthology.orchestra.run.smithi184.stdout: "get_or_fail_fail": 0, 2024-05-20T16:26:14.700 INFO:teuthology.orchestra.run.smithi184.stdout: "get_or_fail_success": 2668, 2024-05-20T16:26:14.700 INFO:teuthology.orchestra.run.smithi184.stdout: "take": 0, 2024-05-20T16:26:14.700 INFO:teuthology.orchestra.run.smithi184.stdout: "take_sum": 0, 2024-05-20T16:26:14.700 INFO:teuthology.orchestra.run.smithi184.stdout: "put": 2668, 2024-05-20T16:26:14.700 INFO:teuthology.orchestra.run.smithi184.stdout: "put_sum": 1234938, 2024-05-20T16:26:14.700 INFO:teuthology.orchestra.run.smithi184.stdout: "wait": { 2024-05-20T16:26:14.701 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-20T16:26:14.701 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-20T16:26:14.701 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-20T16:26:14.701 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-05-20T16:26:14.701 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-20T16:26:14.701 INFO:teuthology.orchestra.run.smithi184.stdout: "throttle-objecter_bytes": { 2024-05-20T16:26:14.701 INFO:teuthology.orchestra.run.smithi184.stdout: "val": 0, 2024-05-20T16:26:14.701 INFO:teuthology.orchestra.run.smithi184.stdout: "max": 104857600, 2024-05-20T16:26:14.701 INFO:teuthology.orchestra.run.smithi184.stdout: "get_started": 0, 2024-05-20T16:26:14.701 INFO:teuthology.orchestra.run.smithi184.stdout: "get": 0, 2024-05-20T16:26:14.701 INFO:teuthology.orchestra.run.smithi184.stdout: "get_sum": 0, 2024-05-20T16:26:14.701 INFO:teuthology.orchestra.run.smithi184.stdout: "get_or_fail_fail": 0, 2024-05-20T16:26:14.701 INFO:teuthology.orchestra.run.smithi184.stdout: "get_or_fail_success": 0, 2024-05-20T16:26:14.701 INFO:teuthology.orchestra.run.smithi184.stdout: "take": 2405, 2024-05-20T16:26:14.701 INFO:teuthology.orchestra.run.smithi184.stdout: "take_sum": 1125, 2024-05-20T16:26:14.702 INFO:teuthology.orchestra.run.smithi184.stdout: "put": 2405, 2024-05-20T16:26:14.702 INFO:teuthology.orchestra.run.smithi184.stdout: "put_sum": 1125, 2024-05-20T16:26:14.702 INFO:teuthology.orchestra.run.smithi184.stdout: "wait": { 2024-05-20T16:26:14.702 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-20T16:26:14.702 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-20T16:26:14.702 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-20T16:26:14.702 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-05-20T16:26:14.702 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-20T16:26:14.702 INFO:teuthology.orchestra.run.smithi184.stdout: "throttle-objecter_ops": { 2024-05-20T16:26:14.702 INFO:teuthology.orchestra.run.smithi184.stdout: "val": 0, 2024-05-20T16:26:14.702 INFO:teuthology.orchestra.run.smithi184.stdout: "max": 1024, 2024-05-20T16:26:14.702 INFO:teuthology.orchestra.run.smithi184.stdout: "get_started": 0, 2024-05-20T16:26:14.702 INFO:teuthology.orchestra.run.smithi184.stdout: "get": 0, 2024-05-20T16:26:14.702 INFO:teuthology.orchestra.run.smithi184.stdout: "get_sum": 0, 2024-05-20T16:26:14.703 INFO:teuthology.orchestra.run.smithi184.stdout: "get_or_fail_fail": 0, 2024-05-20T16:26:14.703 INFO:teuthology.orchestra.run.smithi184.stdout: "get_or_fail_success": 0, 2024-05-20T16:26:14.703 INFO:teuthology.orchestra.run.smithi184.stdout: "take": 2405, 2024-05-20T16:26:14.703 INFO:teuthology.orchestra.run.smithi184.stdout: "take_sum": 2405, 2024-05-20T16:26:14.703 INFO:teuthology.orchestra.run.smithi184.stdout: "put": 2405, 2024-05-20T16:26:14.703 INFO:teuthology.orchestra.run.smithi184.stdout: "put_sum": 2405, 2024-05-20T16:26:14.703 INFO:teuthology.orchestra.run.smithi184.stdout: "wait": { 2024-05-20T16:26:14.703 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-20T16:26:14.703 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-20T16:26:14.703 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-20T16:26:14.703 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-05-20T16:26:14.703 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-20T16:26:14.703 INFO:teuthology.orchestra.run.smithi184.stdout: "throttle-write_buf_throttle": { 2024-05-20T16:26:14.703 INFO:teuthology.orchestra.run.smithi184.stdout: "val": 0, 2024-05-20T16:26:14.704 INFO:teuthology.orchestra.run.smithi184.stdout: "max": 3758096384, 2024-05-20T16:26:14.704 INFO:teuthology.orchestra.run.smithi184.stdout: "get_started": 0, 2024-05-20T16:26:14.704 INFO:teuthology.orchestra.run.smithi184.stdout: "get": 0, 2024-05-20T16:26:14.704 INFO:teuthology.orchestra.run.smithi184.stdout: "get_sum": 0, 2024-05-20T16:26:14.704 INFO:teuthology.orchestra.run.smithi184.stdout: "get_or_fail_fail": 0, 2024-05-20T16:26:14.704 INFO:teuthology.orchestra.run.smithi184.stdout: "get_or_fail_success": 0, 2024-05-20T16:26:14.704 INFO:teuthology.orchestra.run.smithi184.stdout: "take": 0, 2024-05-20T16:26:14.704 INFO:teuthology.orchestra.run.smithi184.stdout: "take_sum": 0, 2024-05-20T16:26:14.704 INFO:teuthology.orchestra.run.smithi184.stdout: "put": 0, 2024-05-20T16:26:14.704 INFO:teuthology.orchestra.run.smithi184.stdout: "put_sum": 0, 2024-05-20T16:26:14.704 INFO:teuthology.orchestra.run.smithi184.stdout: "wait": { 2024-05-20T16:26:14.704 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-20T16:26:14.704 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-20T16:26:14.704 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-20T16:26:14.704 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-05-20T16:26:14.705 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-20T16:26:14.705 INFO:teuthology.orchestra.run.smithi184.stdout: "throttle-write_buf_throttle-0x55ced3fee9c0": { 2024-05-20T16:26:14.705 INFO:teuthology.orchestra.run.smithi184.stdout: "val": 0, 2024-05-20T16:26:14.705 INFO:teuthology.orchestra.run.smithi184.stdout: "max": 3758096384, 2024-05-20T16:26:14.705 INFO:teuthology.orchestra.run.smithi184.stdout: "get_started": 0, 2024-05-20T16:26:14.705 INFO:teuthology.orchestra.run.smithi184.stdout: "get": 0, 2024-05-20T16:26:14.705 INFO:teuthology.orchestra.run.smithi184.stdout: "get_sum": 0, 2024-05-20T16:26:14.705 INFO:teuthology.orchestra.run.smithi184.stdout: "get_or_fail_fail": 0, 2024-05-20T16:26:14.705 INFO:teuthology.orchestra.run.smithi184.stdout: "get_or_fail_success": 0, 2024-05-20T16:26:14.705 INFO:teuthology.orchestra.run.smithi184.stdout: "take": 0, 2024-05-20T16:26:14.705 INFO:teuthology.orchestra.run.smithi184.stdout: "take_sum": 0, 2024-05-20T16:26:14.705 INFO:teuthology.orchestra.run.smithi184.stdout: "put": 0, 2024-05-20T16:26:14.705 INFO:teuthology.orchestra.run.smithi184.stdout: "put_sum": 0, 2024-05-20T16:26:14.705 INFO:teuthology.orchestra.run.smithi184.stdout: "wait": { 2024-05-20T16:26:14.706 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-20T16:26:14.706 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-20T16:26:14.706 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-20T16:26:14.706 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-05-20T16:26:14.706 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-05-20T16:26:14.706 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-05-20T16:26:15.562 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:26:15 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:26:15.562 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:26:15 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T16:26:15.562 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:26:15 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:26:15.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:26:15 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:26:15.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:26:15 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T16:26:15.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:26:15 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:26:15.635 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:26:15 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-20T16:26:15.635 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:26:15 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-20T16:26:15.635 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:26:15 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' 2024-05-20T16:26:16.009 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-05-20T16:26:16.009 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-05-20T16:26:16.009 DEBUG:tasks.check_counter:Getting stats from i 2024-05-20T16:26:16.009 DEBUG:teuthology.orchestra.run.smithi184:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell --fsid 59d1600a-16c2-11ef-bc9a-c7b262605968 -- ceph daemon mds.i perf dump 2024-05-20T16:26:16.295 INFO:teuthology.orchestra.run.smithi184.stderr:Inferring config /var/lib/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/mon.c/config 2024-05-20T16:26:16.562 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:26:16 smithi160 ceph-mon[36347]: pgmap v733: 129 pgs: 129 active+clean; 8.0 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 67 MiB/s wr, 351 op/s 2024-05-20T16:26:16.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:26:16 smithi082 ceph-mon[32238]: pgmap v733: 129 pgs: 129 active+clean; 8.0 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 67 MiB/s wr, 351 op/s 2024-05-20T16:26:16.609 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:26:16 smithi184 ceph-mon[36209]: pgmap v733: 129 pgs: 129 active+clean; 8.0 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 67 MiB/s wr, 351 op/s 2024-05-20T16:26:17.855 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-05-20T16:26:17.856 INFO:teuthology.orchestra.run.smithi184.stdout: "AsyncMessenger::Worker-0": { 2024-05-20T16:26:17.856 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_recv_messages": 37, 2024-05-20T16:26:17.856 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_messages": 39, 2024-05-20T16:26:17.856 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_recv_bytes": 11281, 2024-05-20T16:26:17.856 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_bytes": 62715, 2024-05-20T16:26:17.856 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_created_connections": 13, 2024-05-20T16:26:17.856 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_active_connections": 6, 2024-05-20T16:26:17.857 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_total_time": 0.007476694, 2024-05-20T16:26:17.857 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_send_time": 0.000718717, 2024-05-20T16:26:17.857 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_recv_time": 0.004926648, 2024-05-20T16:26:17.857 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_fast_dispatch_time": 0.000419470, 2024-05-20T16:26:17.857 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_messages_queue_lat": { 2024-05-20T16:26:17.857 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 36, 2024-05-20T16:26:17.857 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.019795350, 2024-05-20T16:26:17.857 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000549870 2024-05-20T16:26:17.857 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-20T16:26:17.857 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_handle_ack_lat": { 2024-05-20T16:26:17.857 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 19, 2024-05-20T16:26:17.858 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000003287, 2024-05-20T16:26:17.858 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000173 2024-05-20T16:26:17.858 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-20T16:26:17.858 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_recv_encrypted_bytes": 5200, 2024-05-20T16:26:17.858 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_encrypted_bytes": 3776 2024-05-20T16:26:17.858 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-20T16:26:17.858 INFO:teuthology.orchestra.run.smithi184.stdout: "AsyncMessenger::Worker-1": { 2024-05-20T16:26:17.858 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_recv_messages": 660, 2024-05-20T16:26:17.858 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_messages": 811, 2024-05-20T16:26:17.858 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_recv_bytes": 157576, 2024-05-20T16:26:17.858 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_bytes": 525664, 2024-05-20T16:26:17.859 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_created_connections": 7, 2024-05-20T16:26:17.859 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_active_connections": 4, 2024-05-20T16:26:17.859 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_total_time": 0.089471451, 2024-05-20T16:26:17.859 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_send_time": 0.033498257, 2024-05-20T16:26:17.859 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_recv_time": 0.059560314, 2024-05-20T16:26:17.859 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_fast_dispatch_time": 0.005551948, 2024-05-20T16:26:17.859 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_messages_queue_lat": { 2024-05-20T16:26:17.859 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 811, 2024-05-20T16:26:17.859 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.054583881, 2024-05-20T16:26:17.859 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000067304 2024-05-20T16:26:17.859 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-20T16:26:17.860 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_handle_ack_lat": { 2024-05-20T16:26:17.860 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 1222, 2024-05-20T16:26:17.860 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000481986, 2024-05-20T16:26:17.860 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000394 2024-05-20T16:26:17.860 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-20T16:26:17.860 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_recv_encrypted_bytes": 160, 2024-05-20T16:26:17.860 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_encrypted_bytes": 391760 2024-05-20T16:26:17.860 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-20T16:26:17.860 INFO:teuthology.orchestra.run.smithi184.stdout: "AsyncMessenger::Worker-2": { 2024-05-20T16:26:17.860 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_recv_messages": 300, 2024-05-20T16:26:17.860 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_messages": 284, 2024-05-20T16:26:17.861 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_recv_bytes": 821835, 2024-05-20T16:26:17.861 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_bytes": 162052, 2024-05-20T16:26:17.861 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_created_connections": 5, 2024-05-20T16:26:17.861 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_active_connections": 4, 2024-05-20T16:26:17.861 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_total_time": 0.049366602, 2024-05-20T16:26:17.861 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_send_time": 0.016500369, 2024-05-20T16:26:17.861 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_recv_time": 0.027280390, 2024-05-20T16:26:17.861 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_fast_dispatch_time": 0.008498274, 2024-05-20T16:26:17.861 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_messages_queue_lat": { 2024-05-20T16:26:17.861 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 283, 2024-05-20T16:26:17.861 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.023712826, 2024-05-20T16:26:17.862 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000083790 2024-05-20T16:26:17.862 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-20T16:26:17.862 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_handle_ack_lat": { 2024-05-20T16:26:17.862 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-20T16:26:17.862 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-20T16:26:17.862 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-20T16:26:17.862 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-20T16:26:17.862 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_recv_encrypted_bytes": 819024, 2024-05-20T16:26:17.862 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_encrypted_bytes": 152048 2024-05-20T16:26:17.863 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-20T16:26:17.863 INFO:teuthology.orchestra.run.smithi184.stdout: "cct": { 2024-05-20T16:26:17.863 INFO:teuthology.orchestra.run.smithi184.stdout: "total_workers": 1, 2024-05-20T16:26:17.863 INFO:teuthology.orchestra.run.smithi184.stdout: "unhealthy_workers": 0 2024-05-20T16:26:17.863 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-20T16:26:17.863 INFO:teuthology.orchestra.run.smithi184.stdout: "finisher-MDSRank": { 2024-05-20T16:26:17.863 INFO:teuthology.orchestra.run.smithi184.stdout: "queue_len": 0, 2024-05-20T16:26:17.863 INFO:teuthology.orchestra.run.smithi184.stdout: "complete_latency": { 2024-05-20T16:26:17.864 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 49, 2024-05-20T16:26:17.864 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.001804835, 2024-05-20T16:26:17.864 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000036833 2024-05-20T16:26:17.864 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-05-20T16:26:17.864 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-20T16:26:17.864 INFO:teuthology.orchestra.run.smithi184.stdout: "finisher-PurgeQueue": { 2024-05-20T16:26:17.864 INFO:teuthology.orchestra.run.smithi184.stdout: "queue_len": 0, 2024-05-20T16:26:17.864 INFO:teuthology.orchestra.run.smithi184.stdout: "complete_latency": { 2024-05-20T16:26:17.864 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 2, 2024-05-20T16:26:17.864 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000014928, 2024-05-20T16:26:17.864 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000007464 2024-05-20T16:26:17.864 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-05-20T16:26:17.864 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-20T16:26:17.864 INFO:teuthology.orchestra.run.smithi184.stdout: "mds": { 2024-05-20T16:26:17.865 INFO:teuthology.orchestra.run.smithi184.stdout: "request": 4, 2024-05-20T16:26:17.865 INFO:teuthology.orchestra.run.smithi184.stdout: "reply": 3, 2024-05-20T16:26:17.865 INFO:teuthology.orchestra.run.smithi184.stdout: "reply_latency": { 2024-05-20T16:26:17.865 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 3, 2024-05-20T16:26:17.865 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.002987117, 2024-05-20T16:26:17.865 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000995705 2024-05-20T16:26:17.865 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-20T16:26:17.865 INFO:teuthology.orchestra.run.smithi184.stdout: "slow_reply": 0, 2024-05-20T16:26:17.865 INFO:teuthology.orchestra.run.smithi184.stdout: "forward": 1, 2024-05-20T16:26:17.865 INFO:teuthology.orchestra.run.smithi184.stdout: "dir_fetch_complete": 0, 2024-05-20T16:26:17.865 INFO:teuthology.orchestra.run.smithi184.stdout: "dir_fetch_keys": 0, 2024-05-20T16:26:17.865 INFO:teuthology.orchestra.run.smithi184.stdout: "dir_commit": 11, 2024-05-20T16:26:17.865 INFO:teuthology.orchestra.run.smithi184.stdout: "dir_split": 0, 2024-05-20T16:26:17.865 INFO:teuthology.orchestra.run.smithi184.stdout: "dir_merge": 0, 2024-05-20T16:26:17.866 INFO:teuthology.orchestra.run.smithi184.stdout: "inodes": 12, 2024-05-20T16:26:17.866 INFO:teuthology.orchestra.run.smithi184.stdout: "inodes_top": 0, 2024-05-20T16:26:17.866 INFO:teuthology.orchestra.run.smithi184.stdout: "inodes_bottom": 12, 2024-05-20T16:26:17.866 INFO:teuthology.orchestra.run.smithi184.stdout: "inodes_pin_tail": 0, 2024-05-20T16:26:17.866 INFO:teuthology.orchestra.run.smithi184.stdout: "inodes_pinned": 11, 2024-05-20T16:26:17.866 INFO:teuthology.orchestra.run.smithi184.stdout: "inodes_expired": 0, 2024-05-20T16:26:17.866 INFO:teuthology.orchestra.run.smithi184.stdout: "inodes_with_caps": 0, 2024-05-20T16:26:17.866 INFO:teuthology.orchestra.run.smithi184.stdout: "caps": 0, 2024-05-20T16:26:17.866 INFO:teuthology.orchestra.run.smithi184.stdout: "subtrees": 2, 2024-05-20T16:26:17.866 INFO:teuthology.orchestra.run.smithi184.stdout: "traverse": 8, 2024-05-20T16:26:17.866 INFO:teuthology.orchestra.run.smithi184.stdout: "traverse_hit": 6, 2024-05-20T16:26:17.866 INFO:teuthology.orchestra.run.smithi184.stdout: "traverse_forward": 0, 2024-05-20T16:26:17.867 INFO:teuthology.orchestra.run.smithi184.stdout: "traverse_discover": 2, 2024-05-20T16:26:17.867 INFO:teuthology.orchestra.run.smithi184.stdout: "traverse_dir_fetch": 0, 2024-05-20T16:26:17.867 INFO:teuthology.orchestra.run.smithi184.stdout: "traverse_remote_ino": 0, 2024-05-20T16:26:17.867 INFO:teuthology.orchestra.run.smithi184.stdout: "traverse_lock": 0, 2024-05-20T16:26:17.867 INFO:teuthology.orchestra.run.smithi184.stdout: "load_cent": 0, 2024-05-20T16:26:17.867 INFO:teuthology.orchestra.run.smithi184.stdout: "q": 0, 2024-05-20T16:26:17.867 INFO:teuthology.orchestra.run.smithi184.stdout: "exported": 0, 2024-05-20T16:26:17.867 INFO:teuthology.orchestra.run.smithi184.stdout: "exported_inodes": 0, 2024-05-20T16:26:17.867 INFO:teuthology.orchestra.run.smithi184.stdout: "imported": 0, 2024-05-20T16:26:17.867 INFO:teuthology.orchestra.run.smithi184.stdout: "imported_inodes": 0, 2024-05-20T16:26:17.867 INFO:teuthology.orchestra.run.smithi184.stdout: "openino_dir_fetch": 0, 2024-05-20T16:26:17.867 INFO:teuthology.orchestra.run.smithi184.stdout: "openino_backtrace_fetch": 0, 2024-05-20T16:26:17.868 INFO:teuthology.orchestra.run.smithi184.stdout: "openino_peer_discover": 0, 2024-05-20T16:26:17.868 INFO:teuthology.orchestra.run.smithi184.stdout: "root_rfiles": 22, 2024-05-20T16:26:17.868 INFO:teuthology.orchestra.run.smithi184.stdout: "root_rbytes": 10738144091, 2024-05-20T16:26:17.868 INFO:teuthology.orchestra.run.smithi184.stdout: "root_rsnaps": 0, 2024-05-20T16:26:17.868 INFO:teuthology.orchestra.run.smithi184.stdout: "scrub_backtrace_fetch": 0, 2024-05-20T16:26:17.868 INFO:teuthology.orchestra.run.smithi184.stdout: "scrub_set_tag": 0, 2024-05-20T16:26:17.868 INFO:teuthology.orchestra.run.smithi184.stdout: "scrub_backtrace_repaired": 0, 2024-05-20T16:26:17.868 INFO:teuthology.orchestra.run.smithi184.stdout: "scrub_inotable_repaired": 0, 2024-05-20T16:26:17.868 INFO:teuthology.orchestra.run.smithi184.stdout: "scrub_dir_inodes": 0, 2024-05-20T16:26:17.868 INFO:teuthology.orchestra.run.smithi184.stdout: "scrub_dir_base_inodes": 0, 2024-05-20T16:26:17.868 INFO:teuthology.orchestra.run.smithi184.stdout: "scrub_dirfrag_rstats": 0, 2024-05-20T16:26:17.868 INFO:teuthology.orchestra.run.smithi184.stdout: "scrub_file_inodes": 0, 2024-05-20T16:26:17.868 INFO:teuthology.orchestra.run.smithi184.stdout: "handle_inode_file_caps": 0, 2024-05-20T16:26:17.868 INFO:teuthology.orchestra.run.smithi184.stdout: "ceph_cap_op_revoke": 0, 2024-05-20T16:26:17.869 INFO:teuthology.orchestra.run.smithi184.stdout: "ceph_cap_op_grant": 0, 2024-05-20T16:26:17.869 INFO:teuthology.orchestra.run.smithi184.stdout: "ceph_cap_op_trunc": 0, 2024-05-20T16:26:17.869 INFO:teuthology.orchestra.run.smithi184.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-20T16:26:17.869 INFO:teuthology.orchestra.run.smithi184.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-20T16:26:17.869 INFO:teuthology.orchestra.run.smithi184.stdout: "handle_client_caps": 0, 2024-05-20T16:26:17.869 INFO:teuthology.orchestra.run.smithi184.stdout: "handle_client_caps_dirty": 0, 2024-05-20T16:26:17.869 INFO:teuthology.orchestra.run.smithi184.stdout: "handle_client_cap_release": 0, 2024-05-20T16:26:17.869 INFO:teuthology.orchestra.run.smithi184.stdout: "process_request_cap_release": 0 2024-05-20T16:26:17.869 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-20T16:26:17.869 INFO:teuthology.orchestra.run.smithi184.stdout: "mds_cache": { 2024-05-20T16:26:17.869 INFO:teuthology.orchestra.run.smithi184.stdout: "dir_update": 0, 2024-05-20T16:26:17.869 INFO:teuthology.orchestra.run.smithi184.stdout: "dir_update_receipt": 0, 2024-05-20T16:26:17.869 INFO:teuthology.orchestra.run.smithi184.stdout: "dir_try_discover": 0, 2024-05-20T16:26:17.870 INFO:teuthology.orchestra.run.smithi184.stdout: "dir_send_discover": 3, 2024-05-20T16:26:17.870 INFO:teuthology.orchestra.run.smithi184.stdout: "dir_handle_discover": 0, 2024-05-20T16:26:17.870 INFO:teuthology.orchestra.run.smithi184.stdout: "num_strays": 0, 2024-05-20T16:26:17.870 INFO:teuthology.orchestra.run.smithi184.stdout: "num_strays_delayed": 0, 2024-05-20T16:26:17.870 INFO:teuthology.orchestra.run.smithi184.stdout: "num_strays_enqueuing": 0, 2024-05-20T16:26:17.870 INFO:teuthology.orchestra.run.smithi184.stdout: "strays_created": 0, 2024-05-20T16:26:17.870 INFO:teuthology.orchestra.run.smithi184.stdout: "strays_enqueued": 0, 2024-05-20T16:26:17.870 INFO:teuthology.orchestra.run.smithi184.stdout: "strays_reintegrated": 0, 2024-05-20T16:26:17.870 INFO:teuthology.orchestra.run.smithi184.stdout: "strays_migrated": 0, 2024-05-20T16:26:17.870 INFO:teuthology.orchestra.run.smithi184.stdout: "num_recovering_processing": 0, 2024-05-20T16:26:17.870 INFO:teuthology.orchestra.run.smithi184.stdout: "num_recovering_enqueued": 0, 2024-05-20T16:26:17.870 INFO:teuthology.orchestra.run.smithi184.stdout: "num_recovering_prioritized": 0, 2024-05-20T16:26:17.871 INFO:teuthology.orchestra.run.smithi184.stdout: "recovery_started": 0, 2024-05-20T16:26:17.871 INFO:teuthology.orchestra.run.smithi184.stdout: "recovery_completed": 0, 2024-05-20T16:26:17.871 INFO:teuthology.orchestra.run.smithi184.stdout: "ireq_quiesce_path": 0, 2024-05-20T16:26:17.871 INFO:teuthology.orchestra.run.smithi184.stdout: "ireq_quiesce_inode": 0, 2024-05-20T16:26:17.871 INFO:teuthology.orchestra.run.smithi184.stdout: "ireq_enqueue_scrub": 0, 2024-05-20T16:26:17.871 INFO:teuthology.orchestra.run.smithi184.stdout: "ireq_exportdir": 0, 2024-05-20T16:26:17.871 INFO:teuthology.orchestra.run.smithi184.stdout: "ireq_flush": 0, 2024-05-20T16:26:17.871 INFO:teuthology.orchestra.run.smithi184.stdout: "ireq_fragmentdir": 0, 2024-05-20T16:26:17.871 INFO:teuthology.orchestra.run.smithi184.stdout: "ireq_fragstats": 0, 2024-05-20T16:26:17.871 INFO:teuthology.orchestra.run.smithi184.stdout: "ireq_inodestats": 0 2024-05-20T16:26:17.871 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-20T16:26:17.871 INFO:teuthology.orchestra.run.smithi184.stdout: "mds_log": { 2024-05-20T16:26:17.872 INFO:teuthology.orchestra.run.smithi184.stdout: "evlrg": 0, 2024-05-20T16:26:17.872 INFO:teuthology.orchestra.run.smithi184.stdout: "evadd": 4, 2024-05-20T16:26:17.872 INFO:teuthology.orchestra.run.smithi184.stdout: "evex": 0, 2024-05-20T16:26:17.872 INFO:teuthology.orchestra.run.smithi184.stdout: "evtrm": 0, 2024-05-20T16:26:17.872 INFO:teuthology.orchestra.run.smithi184.stdout: "ev": 4, 2024-05-20T16:26:17.872 INFO:teuthology.orchestra.run.smithi184.stdout: "evexg": 0, 2024-05-20T16:26:17.872 INFO:teuthology.orchestra.run.smithi184.stdout: "evexd": 0, 2024-05-20T16:26:17.872 INFO:teuthology.orchestra.run.smithi184.stdout: "segadd": 2, 2024-05-20T16:26:17.872 INFO:teuthology.orchestra.run.smithi184.stdout: "segex": 0, 2024-05-20T16:26:17.872 INFO:teuthology.orchestra.run.smithi184.stdout: "segtrm": 0, 2024-05-20T16:26:17.872 INFO:teuthology.orchestra.run.smithi184.stdout: "seg": 2, 2024-05-20T16:26:17.872 INFO:teuthology.orchestra.run.smithi184.stdout: "segmjr": 2, 2024-05-20T16:26:17.872 INFO:teuthology.orchestra.run.smithi184.stdout: "segexg": 0, 2024-05-20T16:26:17.872 INFO:teuthology.orchestra.run.smithi184.stdout: "segexd": 0, 2024-05-20T16:26:17.873 INFO:teuthology.orchestra.run.smithi184.stdout: "expos": 4194304, 2024-05-20T16:26:17.873 INFO:teuthology.orchestra.run.smithi184.stdout: "wrpos": 4195424, 2024-05-20T16:26:17.873 INFO:teuthology.orchestra.run.smithi184.stdout: "rdpos": 0, 2024-05-20T16:26:17.873 INFO:teuthology.orchestra.run.smithi184.stdout: "jlat": { 2024-05-20T16:26:17.873 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 3, 2024-05-20T16:26:17.873 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.005138073, 2024-05-20T16:26:17.873 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.001712691 2024-05-20T16:26:17.873 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-20T16:26:17.873 INFO:teuthology.orchestra.run.smithi184.stdout: "replayed": 0 2024-05-20T16:26:17.873 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-20T16:26:17.873 INFO:teuthology.orchestra.run.smithi184.stdout: "mds_mem": { 2024-05-20T16:26:17.873 INFO:teuthology.orchestra.run.smithi184.stdout: "ino": 15, 2024-05-20T16:26:17.873 INFO:teuthology.orchestra.run.smithi184.stdout: "ino+": 15, 2024-05-20T16:26:17.873 INFO:teuthology.orchestra.run.smithi184.stdout: "ino-": 0, 2024-05-20T16:26:17.874 INFO:teuthology.orchestra.run.smithi184.stdout: "dir": 13, 2024-05-20T16:26:17.874 INFO:teuthology.orchestra.run.smithi184.stdout: "dir+": 14, 2024-05-20T16:26:17.874 INFO:teuthology.orchestra.run.smithi184.stdout: "dir-": 1, 2024-05-20T16:26:17.874 INFO:teuthology.orchestra.run.smithi184.stdout: "dn": 12, 2024-05-20T16:26:17.874 INFO:teuthology.orchestra.run.smithi184.stdout: "dn+": 12, 2024-05-20T16:26:17.874 INFO:teuthology.orchestra.run.smithi184.stdout: "dn-": 0, 2024-05-20T16:26:17.874 INFO:teuthology.orchestra.run.smithi184.stdout: "cap": 0, 2024-05-20T16:26:17.874 INFO:teuthology.orchestra.run.smithi184.stdout: "cap+": 3, 2024-05-20T16:26:17.874 INFO:teuthology.orchestra.run.smithi184.stdout: "cap-": 3, 2024-05-20T16:26:17.874 INFO:teuthology.orchestra.run.smithi184.stdout: "rss": 120844, 2024-05-20T16:26:17.874 INFO:teuthology.orchestra.run.smithi184.stdout: "heap": 223636 2024-05-20T16:26:17.874 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-20T16:26:17.875 INFO:teuthology.orchestra.run.smithi184.stdout: "mds_server": { 2024-05-20T16:26:17.875 INFO:teuthology.orchestra.run.smithi184.stdout: "dispatch_client_request": 9, 2024-05-20T16:26:17.875 INFO:teuthology.orchestra.run.smithi184.stdout: "dispatch_server_request": 0, 2024-05-20T16:26:17.875 INFO:teuthology.orchestra.run.smithi184.stdout: "handle_client_request": 4, 2024-05-20T16:26:17.875 INFO:teuthology.orchestra.run.smithi184.stdout: "handle_client_session": 4, 2024-05-20T16:26:17.875 INFO:teuthology.orchestra.run.smithi184.stdout: "handle_peer_request": 1, 2024-05-20T16:26:17.875 INFO:teuthology.orchestra.run.smithi184.stdout: "req_create_latency": { 2024-05-20T16:26:17.875 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-20T16:26:17.875 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-20T16:26:17.875 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-20T16:26:17.875 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-20T16:26:17.875 INFO:teuthology.orchestra.run.smithi184.stdout: "req_getattr_latency": { 2024-05-20T16:26:17.875 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 1, 2024-05-20T16:26:17.876 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.001518871, 2024-05-20T16:26:17.876 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.001518871 2024-05-20T16:26:17.876 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-20T16:26:17.876 INFO:teuthology.orchestra.run.smithi184.stdout: "req_getfilelock_latency": { 2024-05-20T16:26:17.876 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-20T16:26:17.876 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-20T16:26:17.876 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-20T16:26:17.876 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-20T16:26:17.876 INFO:teuthology.orchestra.run.smithi184.stdout: "req_link_latency": { 2024-05-20T16:26:17.876 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-20T16:26:17.876 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-20T16:26:17.876 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-20T16:26:17.876 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-20T16:26:17.876 INFO:teuthology.orchestra.run.smithi184.stdout: "req_lookup_latency": { 2024-05-20T16:26:17.877 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 2, 2024-05-20T16:26:17.877 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.001468246, 2024-05-20T16:26:17.877 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000734123 2024-05-20T16:26:17.877 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-20T16:26:17.877 INFO:teuthology.orchestra.run.smithi184.stdout: "req_lookuphash_latency": { 2024-05-20T16:26:17.877 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-20T16:26:17.877 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-20T16:26:17.877 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-20T16:26:17.877 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-20T16:26:17.877 INFO:teuthology.orchestra.run.smithi184.stdout: "req_lookupino_latency": { 2024-05-20T16:26:17.877 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-20T16:26:17.877 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-20T16:26:17.877 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-20T16:26:17.877 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-20T16:26:17.878 INFO:teuthology.orchestra.run.smithi184.stdout: "req_lookupname_latency": { 2024-05-20T16:26:17.878 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-20T16:26:17.878 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-20T16:26:17.878 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-20T16:26:17.878 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-20T16:26:17.878 INFO:teuthology.orchestra.run.smithi184.stdout: "req_lookupparent_latency": { 2024-05-20T16:26:17.878 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-20T16:26:17.878 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-20T16:26:17.878 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-20T16:26:17.878 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-20T16:26:17.878 INFO:teuthology.orchestra.run.smithi184.stdout: "req_lookupsnap_latency": { 2024-05-20T16:26:17.878 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-20T16:26:17.878 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-20T16:26:17.879 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-20T16:26:17.879 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-20T16:26:17.879 INFO:teuthology.orchestra.run.smithi184.stdout: "req_lssnap_latency": { 2024-05-20T16:26:17.879 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-20T16:26:17.879 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-20T16:26:17.879 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-20T16:26:17.879 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-20T16:26:17.879 INFO:teuthology.orchestra.run.smithi184.stdout: "req_mkdir_latency": { 2024-05-20T16:26:17.879 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-20T16:26:17.880 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-20T16:26:17.880 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-20T16:26:17.880 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-20T16:26:17.880 INFO:teuthology.orchestra.run.smithi184.stdout: "req_mknod_latency": { 2024-05-20T16:26:17.880 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-20T16:26:17.880 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-20T16:26:17.880 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-20T16:26:17.880 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-20T16:26:17.881 INFO:teuthology.orchestra.run.smithi184.stdout: "req_mksnap_latency": { 2024-05-20T16:26:17.881 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-20T16:26:17.881 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-20T16:26:17.881 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-20T16:26:17.881 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-20T16:26:17.881 INFO:teuthology.orchestra.run.smithi184.stdout: "req_open_latency": { 2024-05-20T16:26:17.881 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-20T16:26:17.881 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-20T16:26:17.881 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-20T16:26:17.881 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-20T16:26:17.881 INFO:teuthology.orchestra.run.smithi184.stdout: "req_readdir_latency": { 2024-05-20T16:26:17.881 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-20T16:26:17.881 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-20T16:26:17.881 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-20T16:26:17.881 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-20T16:26:17.882 INFO:teuthology.orchestra.run.smithi184.stdout: "req_rename_latency": { 2024-05-20T16:26:17.882 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-20T16:26:17.882 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-20T16:26:17.882 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-20T16:26:17.882 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-20T16:26:17.882 INFO:teuthology.orchestra.run.smithi184.stdout: "req_renamesnap_latency": { 2024-05-20T16:26:17.882 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-20T16:26:17.882 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-20T16:26:17.882 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-20T16:26:17.882 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-20T16:26:17.882 INFO:teuthology.orchestra.run.smithi184.stdout: "req_snapdiff_latency": { 2024-05-20T16:26:17.883 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-20T16:26:17.883 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-20T16:26:17.883 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-20T16:26:17.883 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-20T16:26:17.883 INFO:teuthology.orchestra.run.smithi184.stdout: "req_rmdir_latency": { 2024-05-20T16:26:17.883 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-20T16:26:17.883 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-20T16:26:17.883 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-20T16:26:17.883 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-20T16:26:17.883 INFO:teuthology.orchestra.run.smithi184.stdout: "req_rmsnap_latency": { 2024-05-20T16:26:17.883 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-20T16:26:17.883 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-20T16:26:17.884 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-20T16:26:17.884 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-20T16:26:17.884 INFO:teuthology.orchestra.run.smithi184.stdout: "req_rmxattr_latency": { 2024-05-20T16:26:17.884 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-20T16:26:17.884 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-20T16:26:17.884 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-20T16:26:17.884 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-20T16:26:17.884 INFO:teuthology.orchestra.run.smithi184.stdout: "req_setattr_latency": { 2024-05-20T16:26:17.884 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-20T16:26:17.884 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-20T16:26:17.884 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-20T16:26:17.884 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-20T16:26:17.884 INFO:teuthology.orchestra.run.smithi184.stdout: "req_setdirlayout_latency": { 2024-05-20T16:26:17.884 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-20T16:26:17.884 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-20T16:26:17.885 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-20T16:26:17.885 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-20T16:26:17.885 INFO:teuthology.orchestra.run.smithi184.stdout: "req_setfilelock_latency": { 2024-05-20T16:26:17.885 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-20T16:26:17.885 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-20T16:26:17.885 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-20T16:26:17.885 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-20T16:26:17.885 INFO:teuthology.orchestra.run.smithi184.stdout: "req_setlayout_latency": { 2024-05-20T16:26:17.885 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-20T16:26:17.885 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-20T16:26:17.885 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-20T16:26:17.885 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-20T16:26:17.885 INFO:teuthology.orchestra.run.smithi184.stdout: "req_setxattr_latency": { 2024-05-20T16:26:17.885 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-20T16:26:17.885 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-20T16:26:17.886 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-20T16:26:17.886 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-20T16:26:17.886 INFO:teuthology.orchestra.run.smithi184.stdout: "req_symlink_latency": { 2024-05-20T16:26:17.886 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-20T16:26:17.886 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-20T16:26:17.886 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-20T16:26:17.886 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-20T16:26:17.886 INFO:teuthology.orchestra.run.smithi184.stdout: "req_unlink_latency": { 2024-05-20T16:26:17.886 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-20T16:26:17.886 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-20T16:26:17.886 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-20T16:26:17.886 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-20T16:26:17.886 INFO:teuthology.orchestra.run.smithi184.stdout: "cap_revoke_eviction": 0, 2024-05-20T16:26:17.886 INFO:teuthology.orchestra.run.smithi184.stdout: "cap_acquisition_throttle": 0, 2024-05-20T16:26:17.887 INFO:teuthology.orchestra.run.smithi184.stdout: "req_getvxattr_latency": { 2024-05-20T16:26:17.887 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-20T16:26:17.887 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-20T16:26:17.887 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-20T16:26:17.887 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-05-20T16:26:17.887 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-20T16:26:17.887 INFO:teuthology.orchestra.run.smithi184.stdout: "mds_sessions": { 2024-05-20T16:26:17.887 INFO:teuthology.orchestra.run.smithi184.stdout: "session_count": 0, 2024-05-20T16:26:17.887 INFO:teuthology.orchestra.run.smithi184.stdout: "session_add": 1, 2024-05-20T16:26:17.887 INFO:teuthology.orchestra.run.smithi184.stdout: "session_remove": 1, 2024-05-20T16:26:17.887 INFO:teuthology.orchestra.run.smithi184.stdout: "sessions_open": 0, 2024-05-20T16:26:17.887 INFO:teuthology.orchestra.run.smithi184.stdout: "sessions_stale": 0, 2024-05-20T16:26:17.887 INFO:teuthology.orchestra.run.smithi184.stdout: "total_load": 2, 2024-05-20T16:26:17.888 INFO:teuthology.orchestra.run.smithi184.stdout: "average_load": 2, 2024-05-20T16:26:17.888 INFO:teuthology.orchestra.run.smithi184.stdout: "avg_session_uptime": 67, 2024-05-20T16:26:17.888 INFO:teuthology.orchestra.run.smithi184.stdout: "mdthresh_evicted": 0 2024-05-20T16:26:17.888 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-20T16:26:17.888 INFO:teuthology.orchestra.run.smithi184.stdout: "mempool": { 2024-05-20T16:26:17.888 INFO:teuthology.orchestra.run.smithi184.stdout: "bloom_filter_bytes": 0, 2024-05-20T16:26:17.888 INFO:teuthology.orchestra.run.smithi184.stdout: "bloom_filter_items": 0, 2024-05-20T16:26:17.888 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_alloc_bytes": 0, 2024-05-20T16:26:17.888 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_alloc_items": 0, 2024-05-20T16:26:17.888 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_data_bytes": 0, 2024-05-20T16:26:17.888 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_data_items": 0, 2024-05-20T16:26:17.888 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-20T16:26:17.889 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_onode_items": 0, 2024-05-20T16:26:17.889 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-20T16:26:17.889 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_meta_items": 0, 2024-05-20T16:26:17.889 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_other_bytes": 0, 2024-05-20T16:26:17.889 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_other_items": 0, 2024-05-20T16:26:17.889 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-20T16:26:17.889 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_buffer_items": 0, 2024-05-20T16:26:17.889 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_extent_bytes": 0, 2024-05-20T16:26:17.889 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_extent_items": 0, 2024-05-20T16:26:17.889 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_blob_bytes": 0, 2024-05-20T16:26:17.889 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_blob_items": 0, 2024-05-20T16:26:17.889 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-20T16:26:17.889 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_shared_blob_items": 0, 2024-05-20T16:26:17.889 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-20T16:26:17.890 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_inline_bl_items": 0, 2024-05-20T16:26:17.890 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_fsck_bytes": 0, 2024-05-20T16:26:17.890 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_fsck_items": 0, 2024-05-20T16:26:17.890 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_txc_bytes": 0, 2024-05-20T16:26:17.890 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_txc_items": 0, 2024-05-20T16:26:17.890 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-20T16:26:17.890 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_writing_deferred_items": 0, 2024-05-20T16:26:17.890 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_writing_bytes": 0, 2024-05-20T16:26:17.890 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_writing_items": 0, 2024-05-20T16:26:17.891 INFO:teuthology.orchestra.run.smithi184.stdout: "bluefs_bytes": 0, 2024-05-20T16:26:17.891 INFO:teuthology.orchestra.run.smithi184.stdout: "bluefs_items": 0, 2024-05-20T16:26:17.891 INFO:teuthology.orchestra.run.smithi184.stdout: "bluefs_file_reader_bytes": 0, 2024-05-20T16:26:17.891 INFO:teuthology.orchestra.run.smithi184.stdout: "bluefs_file_reader_items": 0, 2024-05-20T16:26:17.891 INFO:teuthology.orchestra.run.smithi184.stdout: "bluefs_file_writer_bytes": 0, 2024-05-20T16:26:17.891 INFO:teuthology.orchestra.run.smithi184.stdout: "bluefs_file_writer_items": 0, 2024-05-20T16:26:17.891 INFO:teuthology.orchestra.run.smithi184.stdout: "buffer_anon_bytes": 238935, 2024-05-20T16:26:17.891 INFO:teuthology.orchestra.run.smithi184.stdout: "buffer_anon_items": 80, 2024-05-20T16:26:17.891 INFO:teuthology.orchestra.run.smithi184.stdout: "buffer_meta_bytes": 88, 2024-05-20T16:26:17.891 INFO:teuthology.orchestra.run.smithi184.stdout: "buffer_meta_items": 1, 2024-05-20T16:26:17.891 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_bytes": 0, 2024-05-20T16:26:17.891 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_items": 0, 2024-05-20T16:26:17.891 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_mapbl_bytes": 0, 2024-05-20T16:26:17.891 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_mapbl_items": 0, 2024-05-20T16:26:17.891 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_pglog_bytes": 0, 2024-05-20T16:26:17.891 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_pglog_items": 0, 2024-05-20T16:26:17.892 INFO:teuthology.orchestra.run.smithi184.stdout: "osdmap_bytes": 8888, 2024-05-20T16:26:17.892 INFO:teuthology.orchestra.run.smithi184.stdout: "osdmap_items": 170, 2024-05-20T16:26:17.892 INFO:teuthology.orchestra.run.smithi184.stdout: "osdmap_mapping_bytes": 0, 2024-05-20T16:26:17.892 INFO:teuthology.orchestra.run.smithi184.stdout: "osdmap_mapping_items": 0, 2024-05-20T16:26:17.892 INFO:teuthology.orchestra.run.smithi184.stdout: "pgmap_bytes": 0, 2024-05-20T16:26:17.892 INFO:teuthology.orchestra.run.smithi184.stdout: "pgmap_items": 0, 2024-05-20T16:26:17.892 INFO:teuthology.orchestra.run.smithi184.stdout: "mds_co_bytes": 59120, 2024-05-20T16:26:17.892 INFO:teuthology.orchestra.run.smithi184.stdout: "mds_co_items": 237, 2024-05-20T16:26:17.892 INFO:teuthology.orchestra.run.smithi184.stdout: "unittest_1_bytes": 0, 2024-05-20T16:26:17.892 INFO:teuthology.orchestra.run.smithi184.stdout: "unittest_1_items": 0, 2024-05-20T16:26:17.892 INFO:teuthology.orchestra.run.smithi184.stdout: "unittest_2_bytes": 0, 2024-05-20T16:26:17.892 INFO:teuthology.orchestra.run.smithi184.stdout: "unittest_2_items": 0 2024-05-20T16:26:17.892 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-20T16:26:17.893 INFO:teuthology.orchestra.run.smithi184.stdout: "objecter": { 2024-05-20T16:26:17.893 INFO:teuthology.orchestra.run.smithi184.stdout: "op_active": 0, 2024-05-20T16:26:17.893 INFO:teuthology.orchestra.run.smithi184.stdout: "op_laggy": 0, 2024-05-20T16:26:17.893 INFO:teuthology.orchestra.run.smithi184.stdout: "op_send": 41, 2024-05-20T16:26:17.893 INFO:teuthology.orchestra.run.smithi184.stdout: "op_send_bytes": 11647, 2024-05-20T16:26:17.893 INFO:teuthology.orchestra.run.smithi184.stdout: "op_resend": 28, 2024-05-20T16:26:17.893 INFO:teuthology.orchestra.run.smithi184.stdout: "op_reply": 41, 2024-05-20T16:26:17.893 INFO:teuthology.orchestra.run.smithi184.stdout: "op_latency": { 2024-05-20T16:26:17.893 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 41, 2024-05-20T16:26:17.893 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.242995252, 2024-05-20T16:26:17.893 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.005926713 2024-05-20T16:26:17.893 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-20T16:26:17.894 INFO:teuthology.orchestra.run.smithi184.stdout: "op_inflight": 0, 2024-05-20T16:26:17.894 INFO:teuthology.orchestra.run.smithi184.stdout: "oplen_avg": { 2024-05-20T16:26:17.894 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 41, 2024-05-20T16:26:17.894 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 65 2024-05-20T16:26:17.894 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-20T16:26:17.894 INFO:teuthology.orchestra.run.smithi184.stdout: "op": 41, 2024-05-20T16:26:17.894 INFO:teuthology.orchestra.run.smithi184.stdout: "op_r": 0, 2024-05-20T16:26:17.894 INFO:teuthology.orchestra.run.smithi184.stdout: "op_w": 41, 2024-05-20T16:26:17.894 INFO:teuthology.orchestra.run.smithi184.stdout: "op_rmw": 0, 2024-05-20T16:26:17.894 INFO:teuthology.orchestra.run.smithi184.stdout: "op_pg": 0, 2024-05-20T16:26:17.894 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_stat": 0, 2024-05-20T16:26:17.894 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_create": 10, 2024-05-20T16:26:17.895 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_read": 0, 2024-05-20T16:26:17.895 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_write": 3, 2024-05-20T16:26:17.895 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_writefull": 7, 2024-05-20T16:26:17.895 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_writesame": 0, 2024-05-20T16:26:17.895 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_append": 0, 2024-05-20T16:26:17.895 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_zero": 0, 2024-05-20T16:26:17.895 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_truncate": 0, 2024-05-20T16:26:17.895 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_delete": 6, 2024-05-20T16:26:17.895 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_mapext": 0, 2024-05-20T16:26:17.895 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_sparse_read": 0, 2024-05-20T16:26:17.895 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_clonerange": 0, 2024-05-20T16:26:17.895 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_getxattr": 0, 2024-05-20T16:26:17.896 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_setxattr": 20, 2024-05-20T16:26:17.896 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_cmpxattr": 0, 2024-05-20T16:26:17.896 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_rmxattr": 0, 2024-05-20T16:26:17.896 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_resetxattrs": 0, 2024-05-20T16:26:17.896 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_call": 0, 2024-05-20T16:26:17.896 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_watch": 0, 2024-05-20T16:26:17.896 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_notify": 0, 2024-05-20T16:26:17.896 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_src_cmpxattr": 0, 2024-05-20T16:26:17.896 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_pgls": 0, 2024-05-20T16:26:17.896 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_pgls_filter": 0, 2024-05-20T16:26:17.896 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_other": 0, 2024-05-20T16:26:17.897 INFO:teuthology.orchestra.run.smithi184.stdout: "linger_active": 0, 2024-05-20T16:26:17.897 INFO:teuthology.orchestra.run.smithi184.stdout: "linger_send": 0, 2024-05-20T16:26:17.897 INFO:teuthology.orchestra.run.smithi184.stdout: "linger_resend": 0, 2024-05-20T16:26:17.897 INFO:teuthology.orchestra.run.smithi184.stdout: "linger_ping": 0, 2024-05-20T16:26:17.897 INFO:teuthology.orchestra.run.smithi184.stdout: "poolop_active": 0, 2024-05-20T16:26:17.897 INFO:teuthology.orchestra.run.smithi184.stdout: "poolop_send": 0, 2024-05-20T16:26:17.897 INFO:teuthology.orchestra.run.smithi184.stdout: "poolop_resend": 0, 2024-05-20T16:26:17.897 INFO:teuthology.orchestra.run.smithi184.stdout: "poolstat_active": 0, 2024-05-20T16:26:17.897 INFO:teuthology.orchestra.run.smithi184.stdout: "poolstat_send": 0, 2024-05-20T16:26:17.897 INFO:teuthology.orchestra.run.smithi184.stdout: "poolstat_resend": 0, 2024-05-20T16:26:17.897 INFO:teuthology.orchestra.run.smithi184.stdout: "statfs_active": 0, 2024-05-20T16:26:17.897 INFO:teuthology.orchestra.run.smithi184.stdout: "statfs_send": 0, 2024-05-20T16:26:17.898 INFO:teuthology.orchestra.run.smithi184.stdout: "statfs_resend": 0, 2024-05-20T16:26:17.898 INFO:teuthology.orchestra.run.smithi184.stdout: "command_active": 0, 2024-05-20T16:26:17.898 INFO:teuthology.orchestra.run.smithi184.stdout: "command_send": 0, 2024-05-20T16:26:17.898 INFO:teuthology.orchestra.run.smithi184.stdout: "command_resend": 0, 2024-05-20T16:26:17.898 INFO:teuthology.orchestra.run.smithi184.stdout: "map_epoch": 98, 2024-05-20T16:26:17.898 INFO:teuthology.orchestra.run.smithi184.stdout: "map_full": 0, 2024-05-20T16:26:17.898 INFO:teuthology.orchestra.run.smithi184.stdout: "map_inc": 7, 2024-05-20T16:26:17.898 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_sessions": 9, 2024-05-20T16:26:17.898 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_session_open": 9, 2024-05-20T16:26:17.898 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_session_close": 0, 2024-05-20T16:26:17.898 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_laggy": 0, 2024-05-20T16:26:17.898 INFO:teuthology.orchestra.run.smithi184.stdout: "omap_wr": 17, 2024-05-20T16:26:17.898 INFO:teuthology.orchestra.run.smithi184.stdout: "omap_rd": 0, 2024-05-20T16:26:17.899 INFO:teuthology.orchestra.run.smithi184.stdout: "omap_del": 2 2024-05-20T16:26:17.899 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-20T16:26:17.899 INFO:teuthology.orchestra.run.smithi184.stdout: "oft": { 2024-05-20T16:26:17.899 INFO:teuthology.orchestra.run.smithi184.stdout: "omap_total_objs": 1, 2024-05-20T16:26:17.899 INFO:teuthology.orchestra.run.smithi184.stdout: "omap_total_kv_pairs": 0, 2024-05-20T16:26:17.899 INFO:teuthology.orchestra.run.smithi184.stdout: "omap_total_updates": 0, 2024-05-20T16:26:17.899 INFO:teuthology.orchestra.run.smithi184.stdout: "omap_total_removes": 0 2024-05-20T16:26:17.899 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-20T16:26:17.899 INFO:teuthology.orchestra.run.smithi184.stdout: "osd-slow-ops": { 2024-05-20T16:26:17.899 INFO:teuthology.orchestra.run.smithi184.stdout: "slow_ops_count": 0 2024-05-20T16:26:17.899 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-20T16:26:17.899 INFO:teuthology.orchestra.run.smithi184.stdout: "purge_queue": { 2024-05-20T16:26:17.899 INFO:teuthology.orchestra.run.smithi184.stdout: "pq_executing_ops": 0, 2024-05-20T16:26:17.900 INFO:teuthology.orchestra.run.smithi184.stdout: "pq_executing_ops_high_water": 0, 2024-05-20T16:26:17.900 INFO:teuthology.orchestra.run.smithi184.stdout: "pq_executing": 0, 2024-05-20T16:26:17.900 INFO:teuthology.orchestra.run.smithi184.stdout: "pq_executing_high_water": 0, 2024-05-20T16:26:17.900 INFO:teuthology.orchestra.run.smithi184.stdout: "pq_executed_ops": 0, 2024-05-20T16:26:17.900 INFO:teuthology.orchestra.run.smithi184.stdout: "pq_executed": 0, 2024-05-20T16:26:17.900 INFO:teuthology.orchestra.run.smithi184.stdout: "pq_item_in_journal": 0 2024-05-20T16:26:17.900 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-20T16:26:17.900 INFO:teuthology.orchestra.run.smithi184.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-20T16:26:17.900 INFO:teuthology.orchestra.run.smithi184.stdout: "val": 0, 2024-05-20T16:26:17.900 INFO:teuthology.orchestra.run.smithi184.stdout: "max": 104857600, 2024-05-20T16:26:17.900 INFO:teuthology.orchestra.run.smithi184.stdout: "get_started": 0, 2024-05-20T16:26:17.900 INFO:teuthology.orchestra.run.smithi184.stdout: "get": 993, 2024-05-20T16:26:17.900 INFO:teuthology.orchestra.run.smithi184.stdout: "get_sum": 883278, 2024-05-20T16:26:17.901 INFO:teuthology.orchestra.run.smithi184.stdout: "get_or_fail_fail": 0, 2024-05-20T16:26:17.901 INFO:teuthology.orchestra.run.smithi184.stdout: "get_or_fail_success": 993, 2024-05-20T16:26:17.901 INFO:teuthology.orchestra.run.smithi184.stdout: "take": 0, 2024-05-20T16:26:17.901 INFO:teuthology.orchestra.run.smithi184.stdout: "take_sum": 0, 2024-05-20T16:26:17.901 INFO:teuthology.orchestra.run.smithi184.stdout: "put": 993, 2024-05-20T16:26:17.901 INFO:teuthology.orchestra.run.smithi184.stdout: "put_sum": 883278, 2024-05-20T16:26:17.901 INFO:teuthology.orchestra.run.smithi184.stdout: "wait": { 2024-05-20T16:26:17.901 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-20T16:26:17.901 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-20T16:26:17.901 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-20T16:26:17.901 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-05-20T16:26:17.901 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-20T16:26:17.901 INFO:teuthology.orchestra.run.smithi184.stdout: "throttle-objecter_bytes": { 2024-05-20T16:26:17.902 INFO:teuthology.orchestra.run.smithi184.stdout: "val": 0, 2024-05-20T16:26:17.902 INFO:teuthology.orchestra.run.smithi184.stdout: "max": 104857600, 2024-05-20T16:26:17.902 INFO:teuthology.orchestra.run.smithi184.stdout: "get_started": 0, 2024-05-20T16:26:17.902 INFO:teuthology.orchestra.run.smithi184.stdout: "get": 0, 2024-05-20T16:26:17.902 INFO:teuthology.orchestra.run.smithi184.stdout: "get_sum": 0, 2024-05-20T16:26:17.902 INFO:teuthology.orchestra.run.smithi184.stdout: "get_or_fail_fail": 0, 2024-05-20T16:26:17.902 INFO:teuthology.orchestra.run.smithi184.stdout: "get_or_fail_success": 0, 2024-05-20T16:26:17.902 INFO:teuthology.orchestra.run.smithi184.stdout: "take": 41, 2024-05-20T16:26:17.902 INFO:teuthology.orchestra.run.smithi184.stdout: "take_sum": 11647, 2024-05-20T16:26:17.902 INFO:teuthology.orchestra.run.smithi184.stdout: "put": 41, 2024-05-20T16:26:17.902 INFO:teuthology.orchestra.run.smithi184.stdout: "put_sum": 11647, 2024-05-20T16:26:17.902 INFO:teuthology.orchestra.run.smithi184.stdout: "wait": { 2024-05-20T16:26:17.902 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-20T16:26:17.903 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-20T16:26:17.903 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-20T16:26:17.903 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-05-20T16:26:17.903 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-20T16:26:17.903 INFO:teuthology.orchestra.run.smithi184.stdout: "throttle-objecter_ops": { 2024-05-20T16:26:17.903 INFO:teuthology.orchestra.run.smithi184.stdout: "val": 0, 2024-05-20T16:26:17.903 INFO:teuthology.orchestra.run.smithi184.stdout: "max": 1024, 2024-05-20T16:26:17.903 INFO:teuthology.orchestra.run.smithi184.stdout: "get_started": 0, 2024-05-20T16:26:17.903 INFO:teuthology.orchestra.run.smithi184.stdout: "get": 0, 2024-05-20T16:26:17.903 INFO:teuthology.orchestra.run.smithi184.stdout: "get_sum": 0, 2024-05-20T16:26:17.903 INFO:teuthology.orchestra.run.smithi184.stdout: "get_or_fail_fail": 0, 2024-05-20T16:26:17.903 INFO:teuthology.orchestra.run.smithi184.stdout: "get_or_fail_success": 0, 2024-05-20T16:26:17.903 INFO:teuthology.orchestra.run.smithi184.stdout: "take": 41, 2024-05-20T16:26:17.904 INFO:teuthology.orchestra.run.smithi184.stdout: "take_sum": 41, 2024-05-20T16:26:17.904 INFO:teuthology.orchestra.run.smithi184.stdout: "put": 41, 2024-05-20T16:26:17.904 INFO:teuthology.orchestra.run.smithi184.stdout: "put_sum": 41, 2024-05-20T16:26:17.904 INFO:teuthology.orchestra.run.smithi184.stdout: "wait": { 2024-05-20T16:26:17.904 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-20T16:26:17.904 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-20T16:26:17.904 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-20T16:26:17.904 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-05-20T16:26:17.904 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-20T16:26:17.904 INFO:teuthology.orchestra.run.smithi184.stdout: "throttle-write_buf_throttle": { 2024-05-20T16:26:17.904 INFO:teuthology.orchestra.run.smithi184.stdout: "val": 0, 2024-05-20T16:26:17.904 INFO:teuthology.orchestra.run.smithi184.stdout: "max": 3758096384, 2024-05-20T16:26:17.904 INFO:teuthology.orchestra.run.smithi184.stdout: "get_started": 0, 2024-05-20T16:26:17.904 INFO:teuthology.orchestra.run.smithi184.stdout: "get": 0, 2024-05-20T16:26:17.904 INFO:teuthology.orchestra.run.smithi184.stdout: "get_sum": 0, 2024-05-20T16:26:17.905 INFO:teuthology.orchestra.run.smithi184.stdout: "get_or_fail_fail": 0, 2024-05-20T16:26:17.905 INFO:teuthology.orchestra.run.smithi184.stdout: "get_or_fail_success": 0, 2024-05-20T16:26:17.905 INFO:teuthology.orchestra.run.smithi184.stdout: "take": 0, 2024-05-20T16:26:17.905 INFO:teuthology.orchestra.run.smithi184.stdout: "take_sum": 0, 2024-05-20T16:26:17.905 INFO:teuthology.orchestra.run.smithi184.stdout: "put": 0, 2024-05-20T16:26:17.905 INFO:teuthology.orchestra.run.smithi184.stdout: "put_sum": 0, 2024-05-20T16:26:17.905 INFO:teuthology.orchestra.run.smithi184.stdout: "wait": { 2024-05-20T16:26:17.905 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-20T16:26:17.905 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-20T16:26:17.905 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-20T16:26:17.905 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-05-20T16:26:17.905 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-20T16:26:17.905 INFO:teuthology.orchestra.run.smithi184.stdout: "throttle-write_buf_throttle-0x55e061bd6b60": { 2024-05-20T16:26:17.905 INFO:teuthology.orchestra.run.smithi184.stdout: "val": 0, 2024-05-20T16:26:17.905 INFO:teuthology.orchestra.run.smithi184.stdout: "max": 3758096384, 2024-05-20T16:26:17.906 INFO:teuthology.orchestra.run.smithi184.stdout: "get_started": 0, 2024-05-20T16:26:17.906 INFO:teuthology.orchestra.run.smithi184.stdout: "get": 4, 2024-05-20T16:26:17.906 INFO:teuthology.orchestra.run.smithi184.stdout: "get_sum": 1120, 2024-05-20T16:26:17.906 INFO:teuthology.orchestra.run.smithi184.stdout: "get_or_fail_fail": 0, 2024-05-20T16:26:17.906 INFO:teuthology.orchestra.run.smithi184.stdout: "get_or_fail_success": 4, 2024-05-20T16:26:17.906 INFO:teuthology.orchestra.run.smithi184.stdout: "take": 0, 2024-05-20T16:26:17.906 INFO:teuthology.orchestra.run.smithi184.stdout: "take_sum": 0, 2024-05-20T16:26:17.906 INFO:teuthology.orchestra.run.smithi184.stdout: "put": 3, 2024-05-20T16:26:17.906 INFO:teuthology.orchestra.run.smithi184.stdout: "put_sum": 1120, 2024-05-20T16:26:17.906 INFO:teuthology.orchestra.run.smithi184.stdout: "wait": { 2024-05-20T16:26:17.906 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-20T16:26:17.906 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-20T16:26:17.906 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-20T16:26:17.907 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-05-20T16:26:17.907 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-05-20T16:26:17.907 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-05-20T16:26:18.690 INFO:tasks.check_counter:Daemon mds.i mds.exported=0 2024-05-20T16:26:18.690 INFO:tasks.check_counter:Daemon mds.i mds.imported=0 2024-05-20T16:26:18.690 DEBUG:tasks.check_counter:Getting stats from l 2024-05-20T16:26:18.691 DEBUG:teuthology.orchestra.run.smithi184:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell --fsid 59d1600a-16c2-11ef-bc9a-c7b262605968 -- ceph daemon mds.l perf dump 2024-05-20T16:26:19.117 INFO:teuthology.orchestra.run.smithi184.stderr:Inferring config /var/lib/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/mon.c/config 2024-05-20T16:26:19.241 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:26:18 smithi184 ceph-mon[36209]: pgmap v734: 129 pgs: 129 active+clean; 8.0 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 55 MiB/s wr, 336 op/s 2024-05-20T16:26:19.311 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:26:18 smithi082 ceph-mon[32238]: pgmap v734: 129 pgs: 129 active+clean; 8.0 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 55 MiB/s wr, 336 op/s 2024-05-20T16:26:19.312 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:26:18 smithi160 ceph-mon[36347]: pgmap v734: 129 pgs: 129 active+clean; 8.0 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 55 MiB/s wr, 336 op/s 2024-05-20T16:26:21.132 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:26:20 smithi184 ceph-mon[36209]: pgmap v735: 129 pgs: 129 active+clean; 8.0 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 72 MiB/s wr, 445 op/s 2024-05-20T16:26:21.312 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:26:20 smithi160 ceph-mon[36347]: pgmap v735: 129 pgs: 129 active+clean; 8.0 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 72 MiB/s wr, 445 op/s 2024-05-20T16:26:21.319 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:26:20 smithi082 ceph-mon[32238]: pgmap v735: 129 pgs: 129 active+clean; 8.0 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 72 MiB/s wr, 445 op/s 2024-05-20T16:26:21.522 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-05-20T16:26:21.522 INFO:teuthology.orchestra.run.smithi184.stdout: "AsyncMessenger::Worker-0": { 2024-05-20T16:26:21.522 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_recv_messages": 4, 2024-05-20T16:26:21.522 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_messages": 4, 2024-05-20T16:26:21.523 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_recv_bytes": 3024, 2024-05-20T16:26:21.523 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_bytes": 448, 2024-05-20T16:26:21.523 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_created_connections": 4, 2024-05-20T16:26:21.523 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_active_connections": 0, 2024-05-20T16:26:21.523 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_total_time": 0.001462804, 2024-05-20T16:26:21.523 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_send_time": 0.000044752, 2024-05-20T16:26:21.523 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_recv_time": 0.000691559, 2024-05-20T16:26:21.523 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-20T16:26:21.523 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_messages_queue_lat": { 2024-05-20T16:26:21.523 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 3, 2024-05-20T16:26:21.523 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000816480, 2024-05-20T16:26:21.523 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000272160 2024-05-20T16:26:21.523 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-20T16:26:21.524 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_handle_ack_lat": { 2024-05-20T16:26:21.524 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-20T16:26:21.524 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-20T16:26:21.524 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-20T16:26:21.524 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-20T16:26:21.524 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_recv_encrypted_bytes": 3024, 2024-05-20T16:26:21.524 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_encrypted_bytes": 448 2024-05-20T16:26:21.524 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-20T16:26:21.524 INFO:teuthology.orchestra.run.smithi184.stdout: "AsyncMessenger::Worker-1": { 2024-05-20T16:26:21.524 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_recv_messages": 249, 2024-05-20T16:26:21.524 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_messages": 244, 2024-05-20T16:26:21.524 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_recv_bytes": 762496, 2024-05-20T16:26:21.524 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_bytes": 150160, 2024-05-20T16:26:21.524 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_created_connections": 2, 2024-05-20T16:26:21.524 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_active_connections": 1, 2024-05-20T16:26:21.524 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_total_time": 0.045683692, 2024-05-20T16:26:21.525 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_send_time": 0.014669835, 2024-05-20T16:26:21.525 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_recv_time": 0.024621369, 2024-05-20T16:26:21.525 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_fast_dispatch_time": 0.008584197, 2024-05-20T16:26:21.525 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_messages_queue_lat": { 2024-05-20T16:26:21.525 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 243, 2024-05-20T16:26:21.525 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.015140321, 2024-05-20T16:26:21.525 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000062305 2024-05-20T16:26:21.525 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-20T16:26:21.525 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_handle_ack_lat": { 2024-05-20T16:26:21.525 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-20T16:26:21.525 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-20T16:26:21.525 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-20T16:26:21.525 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-20T16:26:21.525 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_recv_encrypted_bytes": 762496, 2024-05-20T16:26:21.525 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_encrypted_bytes": 150160 2024-05-20T16:26:21.526 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-20T16:26:21.526 INFO:teuthology.orchestra.run.smithi184.stdout: "AsyncMessenger::Worker-2": { 2024-05-20T16:26:21.526 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_recv_messages": 2, 2024-05-20T16:26:21.526 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_messages": 186, 2024-05-20T16:26:21.526 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_recv_bytes": 320, 2024-05-20T16:26:21.526 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_bytes": 188480, 2024-05-20T16:26:21.526 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_created_connections": 4, 2024-05-20T16:26:21.526 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_active_connections": 2, 2024-05-20T16:26:21.526 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_total_time": 0.011851329, 2024-05-20T16:26:21.526 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_send_time": 0.008345233, 2024-05-20T16:26:21.526 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_recv_time": 0.001224474, 2024-05-20T16:26:21.526 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-20T16:26:21.527 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_messages_queue_lat": { 2024-05-20T16:26:21.527 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 184, 2024-05-20T16:26:21.527 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.013457467, 2024-05-20T16:26:21.527 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000073138 2024-05-20T16:26:21.527 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-20T16:26:21.527 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_handle_ack_lat": { 2024-05-20T16:26:21.527 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-20T16:26:21.527 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-20T16:26:21.527 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-20T16:26:21.527 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-20T16:26:21.527 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_recv_encrypted_bytes": 320, 2024-05-20T16:26:21.527 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_encrypted_bytes": 188480 2024-05-20T16:26:21.527 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-20T16:26:21.527 INFO:teuthology.orchestra.run.smithi184.stdout: "cct": { 2024-05-20T16:26:21.528 INFO:teuthology.orchestra.run.smithi184.stdout: "total_workers": 0, 2024-05-20T16:26:21.528 INFO:teuthology.orchestra.run.smithi184.stdout: "unhealthy_workers": 0 2024-05-20T16:26:21.528 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-20T16:26:21.528 INFO:teuthology.orchestra.run.smithi184.stdout: "mempool": { 2024-05-20T16:26:21.528 INFO:teuthology.orchestra.run.smithi184.stdout: "bloom_filter_bytes": 0, 2024-05-20T16:26:21.528 INFO:teuthology.orchestra.run.smithi184.stdout: "bloom_filter_items": 0, 2024-05-20T16:26:21.528 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_alloc_bytes": 0, 2024-05-20T16:26:21.528 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_alloc_items": 0, 2024-05-20T16:26:21.528 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_data_bytes": 0, 2024-05-20T16:26:21.528 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_data_items": 0, 2024-05-20T16:26:21.528 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-20T16:26:21.528 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_onode_items": 0, 2024-05-20T16:26:21.528 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-20T16:26:21.528 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_meta_items": 0, 2024-05-20T16:26:21.528 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_other_bytes": 0, 2024-05-20T16:26:21.529 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_other_items": 0, 2024-05-20T16:26:21.529 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-20T16:26:21.529 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_buffer_items": 0, 2024-05-20T16:26:21.529 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_extent_bytes": 0, 2024-05-20T16:26:21.529 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_extent_items": 0, 2024-05-20T16:26:21.529 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_blob_bytes": 0, 2024-05-20T16:26:21.529 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_blob_items": 0, 2024-05-20T16:26:21.529 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-20T16:26:21.529 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_shared_blob_items": 0, 2024-05-20T16:26:21.529 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-20T16:26:21.529 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_inline_bl_items": 0, 2024-05-20T16:26:21.529 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_fsck_bytes": 0, 2024-05-20T16:26:21.529 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_fsck_items": 0, 2024-05-20T16:26:21.529 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_txc_bytes": 0, 2024-05-20T16:26:21.529 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_txc_items": 0, 2024-05-20T16:26:21.530 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-20T16:26:21.530 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_writing_deferred_items": 0, 2024-05-20T16:26:21.530 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_writing_bytes": 0, 2024-05-20T16:26:21.530 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_writing_items": 0, 2024-05-20T16:26:21.530 INFO:teuthology.orchestra.run.smithi184.stdout: "bluefs_bytes": 0, 2024-05-20T16:26:21.530 INFO:teuthology.orchestra.run.smithi184.stdout: "bluefs_items": 0, 2024-05-20T16:26:21.530 INFO:teuthology.orchestra.run.smithi184.stdout: "bluefs_file_reader_bytes": 0, 2024-05-20T16:26:21.530 INFO:teuthology.orchestra.run.smithi184.stdout: "bluefs_file_reader_items": 0, 2024-05-20T16:26:21.530 INFO:teuthology.orchestra.run.smithi184.stdout: "bluefs_file_writer_bytes": 0, 2024-05-20T16:26:21.530 INFO:teuthology.orchestra.run.smithi184.stdout: "bluefs_file_writer_items": 0, 2024-05-20T16:26:21.530 INFO:teuthology.orchestra.run.smithi184.stdout: "buffer_anon_bytes": 148738, 2024-05-20T16:26:21.530 INFO:teuthology.orchestra.run.smithi184.stdout: "buffer_anon_items": 31, 2024-05-20T16:26:21.530 INFO:teuthology.orchestra.run.smithi184.stdout: "buffer_meta_bytes": 0, 2024-05-20T16:26:21.530 INFO:teuthology.orchestra.run.smithi184.stdout: "buffer_meta_items": 0, 2024-05-20T16:26:21.530 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_bytes": 0, 2024-05-20T16:26:21.531 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_items": 0, 2024-05-20T16:26:21.531 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_mapbl_bytes": 0, 2024-05-20T16:26:21.531 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_mapbl_items": 0, 2024-05-20T16:26:21.531 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_pglog_bytes": 0, 2024-05-20T16:26:21.531 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_pglog_items": 0, 2024-05-20T16:26:21.531 INFO:teuthology.orchestra.run.smithi184.stdout: "osdmap_bytes": 0, 2024-05-20T16:26:21.531 INFO:teuthology.orchestra.run.smithi184.stdout: "osdmap_items": 0, 2024-05-20T16:26:21.531 INFO:teuthology.orchestra.run.smithi184.stdout: "osdmap_mapping_bytes": 0, 2024-05-20T16:26:21.531 INFO:teuthology.orchestra.run.smithi184.stdout: "osdmap_mapping_items": 0, 2024-05-20T16:26:21.531 INFO:teuthology.orchestra.run.smithi184.stdout: "pgmap_bytes": 0, 2024-05-20T16:26:21.531 INFO:teuthology.orchestra.run.smithi184.stdout: "pgmap_items": 0, 2024-05-20T16:26:21.531 INFO:teuthology.orchestra.run.smithi184.stdout: "mds_co_bytes": 928, 2024-05-20T16:26:21.531 INFO:teuthology.orchestra.run.smithi184.stdout: "mds_co_items": 2, 2024-05-20T16:26:21.531 INFO:teuthology.orchestra.run.smithi184.stdout: "unittest_1_bytes": 0, 2024-05-20T16:26:21.531 INFO:teuthology.orchestra.run.smithi184.stdout: "unittest_1_items": 0, 2024-05-20T16:26:21.532 INFO:teuthology.orchestra.run.smithi184.stdout: "unittest_2_bytes": 0, 2024-05-20T16:26:21.532 INFO:teuthology.orchestra.run.smithi184.stdout: "unittest_2_items": 0 2024-05-20T16:26:21.532 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-20T16:26:21.532 INFO:teuthology.orchestra.run.smithi184.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-20T16:26:21.532 INFO:teuthology.orchestra.run.smithi184.stdout: "val": 0, 2024-05-20T16:26:21.532 INFO:teuthology.orchestra.run.smithi184.stdout: "max": 104857600, 2024-05-20T16:26:21.532 INFO:teuthology.orchestra.run.smithi184.stdout: "get_started": 0, 2024-05-20T16:26:21.532 INFO:teuthology.orchestra.run.smithi184.stdout: "get": 251, 2024-05-20T16:26:21.532 INFO:teuthology.orchestra.run.smithi184.stdout: "get_sum": 727356, 2024-05-20T16:26:21.532 INFO:teuthology.orchestra.run.smithi184.stdout: "get_or_fail_fail": 0, 2024-05-20T16:26:21.532 INFO:teuthology.orchestra.run.smithi184.stdout: "get_or_fail_success": 251, 2024-05-20T16:26:21.532 INFO:teuthology.orchestra.run.smithi184.stdout: "take": 0, 2024-05-20T16:26:21.532 INFO:teuthology.orchestra.run.smithi184.stdout: "take_sum": 0, 2024-05-20T16:26:21.532 INFO:teuthology.orchestra.run.smithi184.stdout: "put": 251, 2024-05-20T16:26:21.532 INFO:teuthology.orchestra.run.smithi184.stdout: "put_sum": 727356, 2024-05-20T16:26:21.532 INFO:teuthology.orchestra.run.smithi184.stdout: "wait": { 2024-05-20T16:26:21.532 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-20T16:26:21.533 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-20T16:26:21.533 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-20T16:26:21.533 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-05-20T16:26:21.533 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-05-20T16:26:21.533 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-05-20T16:26:22.135 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:26:21 smithi184 ceph-mon[36209]: pgmap v736: 129 pgs: 129 active+clean; 8.0 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 64 MiB/s wr, 423 op/s 2024-05-20T16:26:22.237 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.l 2024-05-20T16:26:22.237 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.l 2024-05-20T16:26:22.237 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-05-20T16:26:22.251 INFO:tasks.kclient:Unmounting kernel clients... 2024-05-20T16:26:22.251 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-05-20T16:26:22.251 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-20T16:26:22.279 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-05-20T16:26:22.279 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-20T16:26:22.311 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:26:21 smithi160 ceph-mon[36347]: pgmap v736: 129 pgs: 129 active+clean; 8.0 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 64 MiB/s wr, 423 op/s 2024-05-20T16:26:22.336 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-20T16:26:22.358 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:26:21 smithi082 ceph-mon[32238]: pgmap v736: 129 pgs: 129 active+clean; 8.0 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 64 MiB/s wr, 423 op/s 2024-05-20T16:26:22.471 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:22.470+0000 7fd5f6a6c640 1 -- 172.21.15.82:0/2925073635 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd5f0102bb0 msgr2=0x7fd5f0102fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:26:22.471 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:22.470+0000 7fd5f6a6c640 1 --2- 172.21.15.82:0/2925073635 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd5f0102bb0 0x7fd5f0102fc0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fd5dc009900 tx=0x7fd5dc02a410 comp rx=0 tx=0).stop 2024-05-20T16:26:22.471 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:22.470+0000 7fd5f6a6c640 1 -- 172.21.15.82:0/2925073635 shutdown_connections 2024-05-20T16:26:22.471 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:22.470+0000 7fd5f6a6c640 1 --2- 172.21.15.82:0/2925073635 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd5f0102bb0 0x7fd5f0102fc0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:26:22.471 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:22.470+0000 7fd5f6a6c640 1 --2- 172.21.15.82:0/2925073635 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd5f01078b0 0x7fd5f0109cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:26:22.471 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:22.470+0000 7fd5f6a6c640 1 --2- 172.21.15.82:0/2925073635 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd5f0101970 0x7fd5f0102670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:26:22.471 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:22.470+0000 7fd5f6a6c640 1 -- 172.21.15.82:0/2925073635 >> 172.21.15.82:0/2925073635 conn(0x7fd5f00fcb80 msgr2=0x7fd5f00ff010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:26:22.472 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:22.470+0000 7fd5f6a6c640 1 -- 172.21.15.82:0/2925073635 shutdown_connections 2024-05-20T16:26:22.472 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:22.470+0000 7fd5f6a6c640 1 -- 172.21.15.82:0/2925073635 wait complete. 2024-05-20T16:26:22.474 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:22.471+0000 7fd5f6a6c640 1 Processor -- start 2024-05-20T16:26:22.474 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:22.471+0000 7fd5f6a6c640 1 -- start start 2024-05-20T16:26:22.474 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:22.471+0000 7fd5f6a6c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd5f0101970 0x7fd5f0069910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:26:22.474 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:22.471+0000 7fd5f6a6c640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd5f0102bb0 0x7fd5f019e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:26:22.474 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:22.471+0000 7fd5f6a6c640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd5f01078b0 0x7fd5f01a05c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:26:22.474 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:22.471+0000 7fd5f6a6c640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fd5f010d5e0 con 0x7fd5f0101970 2024-05-20T16:26:22.474 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:22.471+0000 7fd5f6a6c640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fd5f010d460 con 0x7fd5f0102bb0 2024-05-20T16:26:22.474 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:22.471+0000 7fd5f6a6c640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd5f010d760 con 0x7fd5f01078b0 2024-05-20T16:26:22.474 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:22.471+0000 7fd5f4808640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd5f0101970 0x7fd5f0069910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:26:22.474 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:22.471+0000 7fd5f4808640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd5f0101970 0x7fd5f0069910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:45752/0 (socket says 172.21.15.82:45752) 2024-05-20T16:26:22.474 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:22.471+0000 7fd5f4808640 1 -- 172.21.15.82:0/3468299622 learned_addr learned my addr 172.21.15.82:0/3468299622 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:26:22.475 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:22.472+0000 7fd5f5009640 1 --2- 172.21.15.82:0/3468299622 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd5f01078b0 0x7fd5f01a05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:26:22.475 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:22.472+0000 7fd5effff640 1 --2- 172.21.15.82:0/3468299622 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd5f0102bb0 0x7fd5f019e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:26:22.475 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:22.472+0000 7fd5f4808640 1 -- 172.21.15.82:0/3468299622 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd5f01078b0 msgr2=0x7fd5f01a05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:26:22.475 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:22.472+0000 7fd5f4808640 1 --2- 172.21.15.82:0/3468299622 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd5f01078b0 0x7fd5f01a05c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:26:22.475 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:22.472+0000 7fd5f4808640 1 -- 172.21.15.82:0/3468299622 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd5f0102bb0 msgr2=0x7fd5f019e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:26:22.475 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:22.472+0000 7fd5f4808640 1 --2- 172.21.15.82:0/3468299622 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd5f0102bb0 0x7fd5f019e010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:26:22.475 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:22.472+0000 7fd5f4808640 1 -- 172.21.15.82:0/3468299622 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5f01a0de0 con 0x7fd5f0101970 2024-05-20T16:26:22.475 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:22.472+0000 7fd5f4808640 1 --2- 172.21.15.82:0/3468299622 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd5f0101970 0x7fd5f0069910 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fd5e400e9f0 tx=0x7fd5e400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:26:22.475 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:22.472+0000 7fd5edffb640 1 -- 172.21.15.82:0/3468299622 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5e400ccc0 con 0x7fd5f0101970 2024-05-20T16:26:22.475 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:22.472+0000 7fd5f6a6c640 1 -- 172.21.15.82:0/3468299622 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5f01a10d0 con 0x7fd5f0101970 2024-05-20T16:26:22.475 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:22.472+0000 7fd5edffb640 1 -- 172.21.15.82:0/3468299622 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd5e40035f0 con 0x7fd5f0101970 2024-05-20T16:26:22.475 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:22.472+0000 7fd5edffb640 1 -- 172.21.15.82:0/3468299622 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5e400ce60 con 0x7fd5f0101970 2024-05-20T16:26:22.475 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:22.472+0000 7fd5f6a6c640 1 -- 172.21.15.82:0/3468299622 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5f01a1620 con 0x7fd5f0101970 2024-05-20T16:26:22.475 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:22.473+0000 7fd5f6a6c640 1 -- 172.21.15.82:0/3468299622 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5f01066c0 con 0x7fd5f0101970 2024-05-20T16:26:22.476 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:22.474+0000 7fd5edffb640 1 -- 172.21.15.82:0/3468299622 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fd5e40037a0 con 0x7fd5f0101970 2024-05-20T16:26:22.476 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:22.475+0000 7fd5edffb640 1 --2- 172.21.15.82:0/3468299622 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fd5e00a2b90 0x7fd5e00a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:26:22.476 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:22.475+0000 7fd5effff640 1 --2- 172.21.15.82:0/3468299622 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fd5e00a2b90 0x7fd5e00a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:26:22.476 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:22.475+0000 7fd5edffb640 1 -- 172.21.15.82:0/3468299622 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9150+0+0 (secure 0 0 0) 0x7fd5e40cfa40 con 0x7fd5f0101970 2024-05-20T16:26:22.477 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:22.476+0000 7fd5effff640 1 --2- 172.21.15.82:0/3468299622 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fd5e00a2b90 0x7fd5e00a50a0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fd5dc005f70 tx=0x7fd5dc000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:26:22.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:22.481+0000 7fd5edffb640 1 -- 172.21.15.82:0/3468299622 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fd5e409c610 con 0x7fd5f0101970 2024-05-20T16:26:22.657 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:22.657+0000 7fd5f6a6c640 1 -- 172.21.15.82:0/3468299622 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd5f0062750 con 0x7fd5f0101970 2024-05-20T16:26:22.659 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:22.658+0000 7fd5edffb640 1 -- 172.21.15.82:0/3468299622 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v98) ==== 74+0+24771 (secure 0 0 0) 0x7fd5e409bf10 con 0x7fd5f0101970 2024-05-20T16:26:22.659 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-05-20T16:26:22.659 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":98,"fsid":"59d1600a-16c2-11ef-bc9a-c7b262605968","created":"2024-05-20T16:03:46.611556+0000","modified":"2024-05-20T16:25:03.536196+0000","last_up_change":"2024-05-20T16:10:59.303479+0000","last_in_change":"2024-05-20T16:10:39.063561+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-20T16:07:09.493727+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-20T16:11:45.058035+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-20T16:11:45.943862+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-20T16:11:49.637579+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"bae842a9-03ec-490e-b56d-f7957235cdc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":1484330167},{"type":"v1","addr":"172.21.15.82:6803","nonce":1484330167}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":1484330167},{"type":"v1","addr":"172.21.15.82:6805","nonce":1484330167}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1484330167},{"type":"v1","addr":"172.21.15.82:6809","nonce":1484330167}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":1484330167},{"type":"v1","addr":"172.21.15.82:6807","nonce":1484330167}]},"public_addr":"172.21.15.82:6803/1484330167","cluster_addr":"172.21.15.82:6805/1484330167","heartbeat_back_addr":"172.21.15.82:6809/1484330167","heartbeat_front_addr":"172.21.15.82:6807/1484330167","state":["exists","up"]},{"osd":1,"uuid":"59f6a2ba-b5de-4161-81c5-92a417abacdf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":1457003568},{"type":"v1","addr":"172.21.15.160:6801","nonce":1457003568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":1457003568},{"type":"v1","addr":"172.21.15.160:6803","nonce":1457003568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6806","nonce":1457003568},{"type":"v1","addr":"172.21.15.160:6807","nonce":1457003568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":1457003568},{"type":"v1","addr":"172.21.15.160:6805","nonce":1457003568}]},"public_addr":"172.21.15.160:6801/1457003568","cluster_addr":"172.21.15.160:6803/1457003568","heartbeat_back_addr":"172.21.15.160:6807/1457003568","heartbeat_front_addr":"172.21.15.160:6805/1457003568","state":["exists","up"]},{"osd":2,"uuid":"3bbb5587-b449-41f3-bd26-67ec3d6afdfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":4051293934},{"type":"v1","addr":"172.21.15.184:6801","nonce":4051293934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":4051293934},{"type":"v1","addr":"172.21.15.184:6803","nonce":4051293934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":4051293934},{"type":"v1","addr":"172.21.15.184:6807","nonce":4051293934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":4051293934},{"type":"v1","addr":"172.21.15.184:6805","nonce":4051293934}]},"public_addr":"172.21.15.184:6801/4051293934","cluster_addr":"172.21.15.184:6803/4051293934","heartbeat_back_addr":"172.21.15.184:6807/4051293934","heartbeat_front_addr":"172.21.15.184:6805/4051293934","state":["exists","up"]},{"osd":3,"uuid":"ca651f90-09b8-4081-a6f9-40b31268a837","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":3947485599},{"type":"v1","addr":"172.21.15.82:6811","nonce":3947485599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":3947485599},{"type":"v1","addr":"172.21.15.82:6813","nonce":3947485599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":3947485599},{"type":"v1","addr":"172.21.15.82:6817","nonce":3947485599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":3947485599},{"type":"v1","addr":"172.21.15.82:6815","nonce":3947485599}]},"public_addr":"172.21.15.82:6811/3947485599","cluster_addr":"172.21.15.82:6813/3947485599","heartbeat_back_addr":"172.21.15.82:6817/3947485599","heartbeat_front_addr":"172.21.15.82:6815/3947485599","state":["exists","up"]},{"osd":4,"uuid":"cb3dfd56-f677-40cb-9c96-8319ea5ac788","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":4160830014},{"type":"v1","addr":"172.21.15.160:6809","nonce":4160830014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":4160830014},{"type":"v1","addr":"172.21.15.160:6811","nonce":4160830014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":4160830014},{"type":"v1","addr":"172.21.15.160:6815","nonce":4160830014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6812","nonce":4160830014},{"type":"v1","addr":"172.21.15.160:6813","nonce":4160830014}]},"public_addr":"172.21.15.160:6809/4160830014","cluster_addr":"172.21.15.160:6811/4160830014","heartbeat_back_addr":"172.21.15.160:6815/4160830014","heartbeat_front_addr":"172.21.15.160:6813/4160830014","state":["exists","up"]},{"osd":5,"uuid":"c70cfcd1-a4ff-461f-8c73-479b061200fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":1046282672},{"type":"v1","addr":"172.21.15.184:6809","nonce":1046282672}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":1046282672},{"type":"v1","addr":"172.21.15.184:6811","nonce":1046282672}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":1046282672},{"type":"v1","addr":"172.21.15.184:6815","nonce":1046282672}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":1046282672},{"type":"v1","addr":"172.21.15.184:6813","nonce":1046282672}]},"public_addr":"172.21.15.184:6809/1046282672","cluster_addr":"172.21.15.184:6811/1046282672","heartbeat_back_addr":"172.21.15.184:6815/1046282672","heartbeat_front_addr":"172.21.15.184:6813/1046282672","state":["exists","up"]},{"osd":6,"uuid":"d47ce001-9dd1-4830-bcc9-888601187c6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":2789736499},{"type":"v1","addr":"172.21.15.82:6819","nonce":2789736499}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":2789736499},{"type":"v1","addr":"172.21.15.82:6821","nonce":2789736499}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":2789736499},{"type":"v1","addr":"172.21.15.82:6825","nonce":2789736499}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":2789736499},{"type":"v1","addr":"172.21.15.82:6823","nonce":2789736499}]},"public_addr":"172.21.15.82:6819/2789736499","cluster_addr":"172.21.15.82:6821/2789736499","heartbeat_back_addr":"172.21.15.82:6825/2789736499","heartbeat_front_addr":"172.21.15.82:6823/2789736499","state":["exists","up"]},{"osd":7,"uuid":"25c54f43-18cb-41e4-83b4-2732b5a93889","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6816","nonce":2340683378},{"type":"v1","addr":"172.21.15.160:6817","nonce":2340683378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":2340683378},{"type":"v1","addr":"172.21.15.160:6819","nonce":2340683378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":2340683378},{"type":"v1","addr":"172.21.15.160:6823","nonce":2340683378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6820","nonce":2340683378},{"type":"v1","addr":"172.21.15.160:6821","nonce":2340683378}]},"public_addr":"172.21.15.160:6817/2340683378","cluster_addr":"172.21.15.160:6819/2340683378","heartbeat_back_addr":"172.21.15.160:6823/2340683378","heartbeat_front_addr":"172.21.15.160:6821/2340683378","state":["exists","up"]},{"osd":8,"uuid":"1f8b478d-287f-45f5-8526-d1a58c5bc07c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":3831576372},{"type":"v1","addr":"172.21.15.184:6817","nonce":3831576372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":3831576372},{"type":"v1","addr":"172.21.15.184:6819","nonce":3831576372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":3831576372},{"type":"v1","addr":"172.21.15.184:6823","nonce":3831576372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":3831576372},{"type":"v1","addr":"172.21.15.184:6821","nonce":3831576372}]},"public_addr":"172.21.15.184:6817/3831576372","cluster_addr":"172.21.15.184:6819/3831576372","heartbeat_back_addr":"172.21.15.184:6823/3831576372","heartbeat_front_addr":"172.21.15.184:6821/3831576372","state":["exists","up"]},{"osd":9,"uuid":"0d69e319-954f-4b18-95ea-31794f18dc3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":716330782},{"type":"v1","addr":"172.21.15.82:6827","nonce":716330782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":716330782},{"type":"v1","addr":"172.21.15.82:6829","nonce":716330782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":716330782},{"type":"v1","addr":"172.21.15.82:6833","nonce":716330782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":716330782},{"type":"v1","addr":"172.21.15.82:6831","nonce":716330782}]},"public_addr":"172.21.15.82:6827/716330782","cluster_addr":"172.21.15.82:6829/716330782","heartbeat_back_addr":"172.21.15.82:6833/716330782","heartbeat_front_addr":"172.21.15.82:6831/716330782","state":["exists","up"]},{"osd":10,"uuid":"39dc1b78-61cb-42ca-b5b6-1e44ac2b6500","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":2327428014},{"type":"v1","addr":"172.21.15.160:6825","nonce":2327428014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":2327428014},{"type":"v1","addr":"172.21.15.160:6827","nonce":2327428014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":2327428014},{"type":"v1","addr":"172.21.15.160:6831","nonce":2327428014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":2327428014},{"type":"v1","addr":"172.21.15.160:6829","nonce":2327428014}]},"public_addr":"172.21.15.160:6825/2327428014","cluster_addr":"172.21.15.160:6827/2327428014","heartbeat_back_addr":"172.21.15.160:6831/2327428014","heartbeat_front_addr":"172.21.15.160:6829/2327428014","state":["exists","up"]},{"osd":11,"uuid":"4306b3c1-707a-4010-987c-209ea0e948e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":3594364650},{"type":"v1","addr":"172.21.15.184:6825","nonce":3594364650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":3594364650},{"type":"v1","addr":"172.21.15.184:6827","nonce":3594364650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3594364650},{"type":"v1","addr":"172.21.15.184:6831","nonce":3594364650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3594364650},{"type":"v1","addr":"172.21.15.184:6829","nonce":3594364650}]},"public_addr":"172.21.15.184:6825/3594364650","cluster_addr":"172.21.15.184:6827/3594364650","heartbeat_back_addr":"172.21.15.184:6831/3594364650","heartbeat_front_addr":"172.21.15.184:6829/3594364650","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:06:17.855998+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:06:41.473270+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:07:05.304108+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:07:29.887472+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:07:55.387286+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:08:20.579669+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:08:46.333051+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:09:12.289668+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:09:38.180696+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:10:04.471459+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:10:31.041912+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:10:57.477266+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.1d","mappings":[{"from":5,"to":9}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.5","mappings":[{"from":9,"to":10}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.82:0/2005424509":"2024-05-21T16:04:45.161949+0000","172.21.15.82:6801/1763049308":"2024-05-21T16:04:45.161949+0000","172.21.15.82:6801/1959459348":"2024-05-21T16:04:19.208291+0000","172.21.15.82:0/3131904556":"2024-05-21T16:04:19.208291+0000","172.21.15.82:6800/1959459348":"2024-05-21T16:04:19.208291+0000","172.21.15.82:0/2456966342":"2024-05-21T16:04:45.161949+0000","172.21.15.82:0/4179978833":"2024-05-21T16:04:19.208291+0000","172.21.15.82:0/1602862225":"2024-05-21T16:04:45.161949+0000","172.21.15.82:0/106101180":"2024-05-21T16:04:19.208291+0000","172.21.15.82:6800/1763049308":"2024-05-21T16:04:45.161949+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-20T16:26:22.661 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:22.660+0000 7fd5f6a6c640 1 -- 172.21.15.82:0/3468299622 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fd5e00a2b90 msgr2=0x7fd5e00a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:26:22.661 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:22.660+0000 7fd5f6a6c640 1 --2- 172.21.15.82:0/3468299622 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fd5e00a2b90 0x7fd5e00a50a0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fd5dc005f70 tx=0x7fd5dc000bc0 comp rx=0 tx=0).stop 2024-05-20T16:26:22.661 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:22.660+0000 7fd5f6a6c640 1 -- 172.21.15.82:0/3468299622 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd5f0101970 msgr2=0x7fd5f0069910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:26:22.662 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:22.660+0000 7fd5f6a6c640 1 --2- 172.21.15.82:0/3468299622 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd5f0101970 0x7fd5f0069910 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fd5e400e9f0 tx=0x7fd5e400eeb0 comp rx=0 tx=0).stop 2024-05-20T16:26:22.662 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:22.661+0000 7fd5f6a6c640 1 -- 172.21.15.82:0/3468299622 shutdown_connections 2024-05-20T16:26:22.662 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:22.661+0000 7fd5f6a6c640 1 --2- 172.21.15.82:0/3468299622 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fd5e00a2b90 0x7fd5e00a50a0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:26:22.662 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:22.661+0000 7fd5f6a6c640 1 --2- 172.21.15.82:0/3468299622 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd5f01078b0 0x7fd5f01a05c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:26:22.662 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:22.661+0000 7fd5f6a6c640 1 --2- 172.21.15.82:0/3468299622 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd5f0102bb0 0x7fd5f019e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:26:22.662 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:22.661+0000 7fd5f6a6c640 1 --2- 172.21.15.82:0/3468299622 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd5f0101970 0x7fd5f0069910 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:26:22.662 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:22.661+0000 7fd5f6a6c640 1 -- 172.21.15.82:0/3468299622 >> 172.21.15.82:0/3468299622 conn(0x7fd5f00fcb80 msgr2=0x7fd5f0103890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:26:22.662 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:22.661+0000 7fd5f6a6c640 1 -- 172.21.15.82:0/3468299622 shutdown_connections 2024-05-20T16:26:22.662 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:22.661+0000 7fd5f6a6c640 1 -- 172.21.15.82:0/3468299622 wait complete. 2024-05-20T16:26:22.674 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-20T16:07:09.493727+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-20T16:11:45.058035+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-20T16:11:45.943862+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-20T16:11:49.637579+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '91', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-20T16:26:22.674 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-20T16:26:22.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:22.788+0000 7f9a6465a640 1 -- 172.21.15.82:0/826015272 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9a5c103d00 msgr2=0x7f9a5c104110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:26:22.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:22.788+0000 7f9a6465a640 1 --2- 172.21.15.82:0/826015272 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9a5c103d00 0x7f9a5c104110 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f9a4c009900 tx=0x7f9a4c02a410 comp rx=0 tx=0).stop 2024-05-20T16:26:22.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:22.788+0000 7f9a6465a640 1 -- 172.21.15.82:0/826015272 shutdown_connections 2024-05-20T16:26:22.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:22.788+0000 7f9a6465a640 1 --2- 172.21.15.82:0/826015272 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9a5c103d00 0x7f9a5c104110 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:26:22.790 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:22.788+0000 7f9a6465a640 1 --2- 172.21.15.82:0/826015272 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9a5c108a00 0x7f9a5c10ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:26:22.790 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:22.788+0000 7f9a6465a640 1 --2- 172.21.15.82:0/826015272 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9a5c102ac0 0x7f9a5c1037c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:26:22.790 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:22.789+0000 7f9a6465a640 1 -- 172.21.15.82:0/826015272 >> 172.21.15.82:0/826015272 conn(0x7f9a5c0fdcf0 msgr2=0x7f9a5c100160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:26:22.790 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:22.789+0000 7f9a6465a640 1 -- 172.21.15.82:0/826015272 shutdown_connections 2024-05-20T16:26:22.790 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:22.789+0000 7f9a6465a640 1 -- 172.21.15.82:0/826015272 wait complete. 2024-05-20T16:26:22.792 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:22.789+0000 7f9a6465a640 1 Processor -- start 2024-05-20T16:26:22.792 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:22.789+0000 7f9a6465a640 1 -- start start 2024-05-20T16:26:22.792 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:22.789+0000 7f9a6465a640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9a5c102ac0 0x7f9a5c1993d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:26:22.792 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:22.789+0000 7f9a6465a640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9a5c103d00 0x7f9a5c199910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:26:22.792 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:22.789+0000 7f9a6465a640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9a5c108a00 0x7f9a5c1a0830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:26:22.792 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:22.789+0000 7f9a6465a640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f9a5c10e730 con 0x7f9a5c102ac0 2024-05-20T16:26:22.792 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:22.789+0000 7f9a6465a640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f9a5c10e5b0 con 0x7f9a5c103d00 2024-05-20T16:26:22.792 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:22.789+0000 7f9a6465a640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f9a5c10e8b0 con 0x7f9a5c108a00 2024-05-20T16:26:22.792 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:22.790+0000 7f9a623f6640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9a5c102ac0 0x7f9a5c1993d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:26:22.792 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:22.790+0000 7f9a623f6640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9a5c102ac0 0x7f9a5c1993d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:45772/0 (socket says 172.21.15.82:45772) 2024-05-20T16:26:22.792 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:22.790+0000 7f9a623f6640 1 -- 172.21.15.82:0/4208104592 learned_addr learned my addr 172.21.15.82:0/4208104592 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:26:22.792 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:22.790+0000 7f9a623f6640 1 -- 172.21.15.82:0/4208104592 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9a5c108a00 msgr2=0x7f9a5c1a0830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:26:22.792 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:22.790+0000 7f9a623f6640 1 --2- 172.21.15.82:0/4208104592 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9a5c108a00 0x7f9a5c1a0830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:26:22.792 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:22.790+0000 7f9a623f6640 1 -- 172.21.15.82:0/4208104592 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9a5c103d00 msgr2=0x7f9a5c199910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:26:22.793 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:22.790+0000 7f9a61bf5640 1 --2- 172.21.15.82:0/4208104592 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9a5c103d00 0x7f9a5c199910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:26:22.793 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:22.790+0000 7f9a623f6640 1 --2- 172.21.15.82:0/4208104592 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9a5c103d00 0x7f9a5c199910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:26:22.793 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:22.790+0000 7f9a623f6640 1 -- 172.21.15.82:0/4208104592 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a5c1a0df0 con 0x7f9a5c102ac0 2024-05-20T16:26:22.793 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:22.790+0000 7f9a623f6640 1 --2- 172.21.15.82:0/4208104592 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9a5c102ac0 0x7f9a5c1993d0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f9a5800c970 tx=0x7f9a5800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:26:22.793 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:22.790+0000 7f9a537fe640 1 -- 172.21.15.82:0/4208104592 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a58004220 con 0x7f9a5c102ac0 2024-05-20T16:26:22.793 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:22.790+0000 7f9a537fe640 1 -- 172.21.15.82:0/4208104592 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9a580043c0 con 0x7f9a5c102ac0 2024-05-20T16:26:22.793 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:22.790+0000 7f9a6465a640 1 -- 172.21.15.82:0/4208104592 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a5c1a10e0 con 0x7f9a5c102ac0 2024-05-20T16:26:22.793 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:22.790+0000 7f9a537fe640 1 -- 172.21.15.82:0/4208104592 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a58004a00 con 0x7f9a5c102ac0 2024-05-20T16:26:22.793 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:22.790+0000 7f9a6465a640 1 -- 172.21.15.82:0/4208104592 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a5c1a15e0 con 0x7f9a5c102ac0 2024-05-20T16:26:22.793 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:22.791+0000 7f9a6465a640 1 -- 172.21.15.82:0/4208104592 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a5c107810 con 0x7f9a5c102ac0 2024-05-20T16:26:22.794 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:22.792+0000 7f9a537fe640 1 -- 172.21.15.82:0/4208104592 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f9a580079b0 con 0x7f9a5c102ac0 2024-05-20T16:26:22.794 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:22.793+0000 7f9a537fe640 1 --2- 172.21.15.82:0/4208104592 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f9a480a2b90 0x7f9a480a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:26:22.794 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:22.793+0000 7f9a61bf5640 1 --2- 172.21.15.82:0/4208104592 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f9a480a2b90 0x7f9a480a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:26:22.794 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:22.793+0000 7f9a537fe640 1 -- 172.21.15.82:0/4208104592 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9150+0+0 (secure 0 0 0) 0x7f9a580cf690 con 0x7f9a5c102ac0 2024-05-20T16:26:22.794 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:22.794+0000 7f9a61bf5640 1 --2- 172.21.15.82:0/4208104592 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f9a480a2b90 0x7f9a480a50a0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f9a4c031610 tx=0x7f9a4c000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:26:22.797 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:22.796+0000 7f9a537fe640 1 -- 172.21.15.82:0/4208104592 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f9a58014030 con 0x7f9a5c102ac0 2024-05-20T16:26:22.972 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:22.971+0000 7f9a6465a640 1 -- 172.21.15.82:0/4208104592 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f9a5c02d550 con 0x7f9a5c102ac0 2024-05-20T16:26:22.972 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:22.971+0000 7f9a537fe640 1 -- 172.21.15.82:0/4208104592 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v98) ==== 93+0+10 (secure 0 0 0) 0x7f9a5809c260 con 0x7f9a5c102ac0 2024-05-20T16:26:22.972 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 1 2024-05-20T16:26:22.974 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:22.973+0000 7f9a6465a640 1 -- 172.21.15.82:0/4208104592 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f9a480a2b90 msgr2=0x7f9a480a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:26:22.974 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:22.973+0000 7f9a6465a640 1 --2- 172.21.15.82:0/4208104592 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f9a480a2b90 0x7f9a480a50a0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f9a4c031610 tx=0x7f9a4c000bc0 comp rx=0 tx=0).stop 2024-05-20T16:26:22.974 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:22.973+0000 7f9a6465a640 1 -- 172.21.15.82:0/4208104592 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9a5c102ac0 msgr2=0x7f9a5c1993d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:26:22.974 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:22.973+0000 7f9a6465a640 1 --2- 172.21.15.82:0/4208104592 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9a5c102ac0 0x7f9a5c1993d0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f9a5800c970 tx=0x7f9a5800ce30 comp rx=0 tx=0).stop 2024-05-20T16:26:22.975 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:22.973+0000 7f9a6465a640 1 -- 172.21.15.82:0/4208104592 shutdown_connections 2024-05-20T16:26:22.975 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:22.973+0000 7f9a6465a640 1 --2- 172.21.15.82:0/4208104592 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f9a480a2b90 0x7f9a480a50a0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:26:22.975 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:22.973+0000 7f9a6465a640 1 --2- 172.21.15.82:0/4208104592 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9a5c108a00 0x7f9a5c1a0830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:26:22.975 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:22.973+0000 7f9a6465a640 1 --2- 172.21.15.82:0/4208104592 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9a5c103d00 0x7f9a5c199910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:26:22.975 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:22.973+0000 7f9a6465a640 1 --2- 172.21.15.82:0/4208104592 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9a5c102ac0 0x7f9a5c1993d0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:26:22.975 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:22.974+0000 7f9a6465a640 1 -- 172.21.15.82:0/4208104592 >> 172.21.15.82:0/4208104592 conn(0x7f9a5c0fdcf0 msgr2=0x7f9a5c1049e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:26:22.975 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:22.974+0000 7f9a6465a640 1 -- 172.21.15.82:0/4208104592 shutdown_connections 2024-05-20T16:26:22.975 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:22.974+0000 7f9a6465a640 1 -- 172.21.15.82:0/4208104592 wait complete. 2024-05-20T16:26:22.986 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-20T16:26:23.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:23.100+0000 7f3710f21640 1 -- 172.21.15.82:0/1116455326 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f370c102aa0 msgr2=0x7f370c1037a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:26:23.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:23.100+0000 7f3710f21640 1 --2- 172.21.15.82:0/1116455326 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f370c102aa0 0x7f370c1037a0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f36fc009900 tx=0x7f36fc02a410 comp rx=0 tx=0).stop 2024-05-20T16:26:23.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:23.100+0000 7f3710f21640 1 -- 172.21.15.82:0/1116455326 shutdown_connections 2024-05-20T16:26:23.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:23.100+0000 7f3710f21640 1 --2- 172.21.15.82:0/1116455326 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f370c1089e0 0x7f370c10ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:26:23.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:23.100+0000 7f3710f21640 1 --2- 172.21.15.82:0/1116455326 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f370c103ce0 0x7f370c1040f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:26:23.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:23.100+0000 7f3710f21640 1 --2- 172.21.15.82:0/1116455326 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f370c102aa0 0x7f370c1037a0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:26:23.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:23.100+0000 7f3710f21640 1 -- 172.21.15.82:0/1116455326 >> 172.21.15.82:0/1116455326 conn(0x7f370c0fdc90 msgr2=0x7f370c100140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:26:23.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:23.100+0000 7f3710f21640 1 -- 172.21.15.82:0/1116455326 shutdown_connections 2024-05-20T16:26:23.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:23.100+0000 7f3710f21640 1 -- 172.21.15.82:0/1116455326 wait complete. 2024-05-20T16:26:23.103 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:23.101+0000 7f3710f21640 1 Processor -- start 2024-05-20T16:26:23.103 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:23.101+0000 7f3710f21640 1 -- start start 2024-05-20T16:26:23.103 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:23.101+0000 7f3710f21640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f370c102aa0 0x7f370c199280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:26:23.103 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:23.101+0000 7f3710f21640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f370c103ce0 0x7f370c1997c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:26:23.103 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:23.101+0000 7f3710f21640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f370c1089e0 0x7f370c1a0750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:26:23.103 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:23.101+0000 7f3710f21640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f370c10e6d0 con 0x7f370c103ce0 2024-05-20T16:26:23.103 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:23.101+0000 7f3710f21640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f370c10e550 con 0x7f370c1089e0 2024-05-20T16:26:23.103 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:23.101+0000 7f3710f21640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f370c10e850 con 0x7f370c102aa0 2024-05-20T16:26:23.103 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:23.101+0000 7f3703fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f370c103ce0 0x7f370c1997c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:26:23.103 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:23.101+0000 7f3703fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f370c103ce0 0x7f370c1997c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:45794/0 (socket says 172.21.15.82:45794) 2024-05-20T16:26:23.103 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:23.101+0000 7f3703fff640 1 -- 172.21.15.82:0/4008671602 learned_addr learned my addr 172.21.15.82:0/4008671602 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:26:23.104 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:23.102+0000 7f370a59c640 1 --2- 172.21.15.82:0/4008671602 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f370c102aa0 0x7f370c199280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:26:23.104 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:23.102+0000 7f370ad9d640 1 --2- 172.21.15.82:0/4008671602 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f370c1089e0 0x7f370c1a0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:26:23.104 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:23.102+0000 7f3703fff640 1 -- 172.21.15.82:0/4008671602 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f370c102aa0 msgr2=0x7f370c199280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:26:23.104 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:23.102+0000 7f3703fff640 1 --2- 172.21.15.82:0/4008671602 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f370c102aa0 0x7f370c199280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:26:23.104 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:23.102+0000 7f3703fff640 1 -- 172.21.15.82:0/4008671602 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f370c1089e0 msgr2=0x7f370c1a0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:26:23.104 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:23.102+0000 7f3703fff640 1 --2- 172.21.15.82:0/4008671602 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f370c1089e0 0x7f370c1a0750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:26:23.104 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:23.102+0000 7f3703fff640 1 -- 172.21.15.82:0/4008671602 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f370c1a0d10 con 0x7f370c103ce0 2024-05-20T16:26:23.104 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:23.102+0000 7f3703fff640 1 --2- 172.21.15.82:0/4008671602 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f370c103ce0 0x7f370c1997c0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f36f400c920 tx=0x7f36f400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:26:23.104 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:23.102+0000 7f37037fe640 1 -- 172.21.15.82:0/4008671602 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36f4007c70 con 0x7f370c103ce0 2024-05-20T16:26:23.104 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:23.102+0000 7f37037fe640 1 -- 172.21.15.82:0/4008671602 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f36f4007e10 con 0x7f370c103ce0 2024-05-20T16:26:23.104 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:23.102+0000 7f3710f21640 1 -- 172.21.15.82:0/4008671602 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f370c1a0fa0 con 0x7f370c103ce0 2024-05-20T16:26:23.104 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:23.102+0000 7f37037fe640 1 -- 172.21.15.82:0/4008671602 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36f4018890 con 0x7f370c103ce0 2024-05-20T16:26:23.104 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:23.102+0000 7f3710f21640 1 -- 172.21.15.82:0/4008671602 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f370c1a1490 con 0x7f370c103ce0 2024-05-20T16:26:23.104 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:23.103+0000 7f3710f21640 1 -- 172.21.15.82:0/4008671602 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f370c1077f0 con 0x7f370c103ce0 2024-05-20T16:26:23.105 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:23.104+0000 7f37037fe640 1 -- 172.21.15.82:0/4008671602 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f36f4018a70 con 0x7f370c103ce0 2024-05-20T16:26:23.105 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:23.105+0000 7f37037fe640 1 --2- 172.21.15.82:0/4008671602 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f36f80a2b90 0x7f36f80a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:26:23.105 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:23.105+0000 7f370a59c640 1 --2- 172.21.15.82:0/4008671602 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f36f80a2b90 0x7f36f80a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:26:23.105 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:23.105+0000 7f37037fe640 1 -- 172.21.15.82:0/4008671602 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9150+0+0 (secure 0 0 0) 0x7f36f40cf5d0 con 0x7f370c103ce0 2024-05-20T16:26:23.105 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:23.106+0000 7f370a59c640 1 --2- 172.21.15.82:0/4008671602 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f36f80a2b90 0x7f36f80a50a0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f36fc031610 tx=0x7f36fc000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:26:23.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:23.110+0000 7f37037fe640 1 -- 172.21.15.82:0/4008671602 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f36f4014030 con 0x7f370c103ce0 2024-05-20T16:26:23.287 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:23.288+0000 7f3710f21640 1 -- 172.21.15.82:0/4008671602 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f370c062750 con 0x7f370c103ce0 2024-05-20T16:26:23.287 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:23.288+0000 7f37037fe640 1 -- 172.21.15.82:0/4008671602 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v98) ==== 104+0+11 (secure 0 0 0) 0x7f36f400de20 con 0x7f370c103ce0 2024-05-20T16:26:23.288 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 32 2024-05-20T16:26:23.290 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:23.290+0000 7f3710f21640 1 -- 172.21.15.82:0/4008671602 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f36f80a2b90 msgr2=0x7f36f80a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:26:23.290 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:23.290+0000 7f3710f21640 1 --2- 172.21.15.82:0/4008671602 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f36f80a2b90 0x7f36f80a50a0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f36fc031610 tx=0x7f36fc000bc0 comp rx=0 tx=0).stop 2024-05-20T16:26:23.290 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:23.290+0000 7f3710f21640 1 -- 172.21.15.82:0/4008671602 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f370c103ce0 msgr2=0x7f370c1997c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:26:23.290 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:23.290+0000 7f3710f21640 1 --2- 172.21.15.82:0/4008671602 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f370c103ce0 0x7f370c1997c0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f36f400c920 tx=0x7f36f400cde0 comp rx=0 tx=0).stop 2024-05-20T16:26:23.290 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:23.290+0000 7f3710f21640 1 -- 172.21.15.82:0/4008671602 shutdown_connections 2024-05-20T16:26:23.290 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:23.290+0000 7f3710f21640 1 --2- 172.21.15.82:0/4008671602 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f36f80a2b90 0x7f36f80a50a0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:26:23.290 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:23.290+0000 7f3710f21640 1 --2- 172.21.15.82:0/4008671602 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f370c103ce0 0x7f370c1997c0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:26:23.290 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:23.290+0000 7f3710f21640 1 --2- 172.21.15.82:0/4008671602 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f370c1089e0 0x7f370c1a0750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:26:23.290 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:23.290+0000 7f3710f21640 1 --2- 172.21.15.82:0/4008671602 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f370c102aa0 0x7f370c199280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:26:23.291 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:23.290+0000 7f3710f21640 1 -- 172.21.15.82:0/4008671602 >> 172.21.15.82:0/4008671602 conn(0x7f370c0fdc90 msgr2=0x7f370c1049c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:26:23.291 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:23.290+0000 7f3710f21640 1 -- 172.21.15.82:0/4008671602 shutdown_connections 2024-05-20T16:26:23.291 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:23.290+0000 7f3710f21640 1 -- 172.21.15.82:0/4008671602 wait complete. 2024-05-20T16:26:23.300 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:26:22 smithi082 ceph-mon[32238]: from='client.? 172.21.15.82:0/3468299622' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-20T16:26:23.301 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:26:22 smithi082 ceph-mon[32238]: from='client.? 172.21.15.82:0/4208104592' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-20T16:26:23.301 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-20T16:26:23.311 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:26:22 smithi160 ceph-mon[36347]: from='client.? 172.21.15.82:0/3468299622' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-20T16:26:23.312 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:26:22 smithi160 ceph-mon[36347]: from='client.? 172.21.15.82:0/4208104592' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-20T16:26:23.385 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:26:22 smithi184 ceph-mon[36209]: from='client.? 172.21.15.82:0/3468299622' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-20T16:26:23.385 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:26:22 smithi184 ceph-mon[36209]: from='client.? 172.21.15.82:0/4208104592' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-20T16:26:23.416 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:23.416+0000 7fbc415e0640 1 -- 172.21.15.82:0/433379102 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbc3c102380 msgr2=0x7fbc3c102790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:26:23.416 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:23.416+0000 7fbc415e0640 1 --2- 172.21.15.82:0/433379102 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbc3c102380 0x7fbc3c102790 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fbc30009900 tx=0x7fbc3002a410 comp rx=0 tx=0).stop 2024-05-20T16:26:23.416 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:23.416+0000 7fbc415e0640 1 -- 172.21.15.82:0/433379102 shutdown_connections 2024-05-20T16:26:23.416 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:23.416+0000 7fbc415e0640 1 --2- 172.21.15.82:0/433379102 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbc3c102380 0x7fbc3c102790 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:26:23.416 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:23.416+0000 7fbc415e0640 1 --2- 172.21.15.82:0/433379102 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbc3c107080 0x7fbc3c1094a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:26:23.416 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:23.416+0000 7fbc415e0640 1 --2- 172.21.15.82:0/433379102 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbc3c101140 0x7fbc3c101e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:26:23.416 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:23.416+0000 7fbc415e0640 1 -- 172.21.15.82:0/433379102 >> 172.21.15.82:0/433379102 conn(0x7fbc3c0fc330 msgr2=0x7fbc3c0fe7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:26:23.416 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:23.416+0000 7fbc415e0640 1 -- 172.21.15.82:0/433379102 shutdown_connections 2024-05-20T16:26:23.416 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:23.416+0000 7fbc415e0640 1 -- 172.21.15.82:0/433379102 wait complete. 2024-05-20T16:26:23.418 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:23.417+0000 7fbc415e0640 1 Processor -- start 2024-05-20T16:26:23.418 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:23.417+0000 7fbc415e0640 1 -- start start 2024-05-20T16:26:23.418 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:23.417+0000 7fbc415e0640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbc3c101140 0x7fbc3c1992c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:26:23.419 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:23.417+0000 7fbc415e0640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbc3c102380 0x7fbc3c199800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:26:23.419 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:23.417+0000 7fbc415e0640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbc3c107080 0x7fbc3c1a0790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:26:23.419 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:23.417+0000 7fbc415e0640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fbc3c10c520 con 0x7fbc3c101140 2024-05-20T16:26:23.419 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:23.417+0000 7fbc415e0640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fbc3c10c3a0 con 0x7fbc3c107080 2024-05-20T16:26:23.419 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:23.417+0000 7fbc415e0640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fbc3c10c6a0 con 0x7fbc3c102380 2024-05-20T16:26:23.419 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:23.417+0000 7fbc3affd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbc3c101140 0x7fbc3c1992c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:26:23.419 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:23.417+0000 7fbc3affd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbc3c101140 0x7fbc3c1992c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:45810/0 (socket says 172.21.15.82:45810) 2024-05-20T16:26:23.419 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:23.417+0000 7fbc3affd640 1 -- 172.21.15.82:0/3429605115 learned_addr learned my addr 172.21.15.82:0/3429605115 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:26:23.419 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:23.417+0000 7fbc3b7fe640 1 --2- 172.21.15.82:0/3429605115 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbc3c107080 0x7fbc3c1a0790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:26:23.419 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:23.417+0000 7fbc3affd640 1 -- 172.21.15.82:0/3429605115 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbc3c102380 msgr2=0x7fbc3c199800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:26:23.419 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:23.417+0000 7fbc3affd640 1 --2- 172.21.15.82:0/3429605115 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbc3c102380 0x7fbc3c199800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:26:23.419 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:23.417+0000 7fbc3affd640 1 -- 172.21.15.82:0/3429605115 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbc3c107080 msgr2=0x7fbc3c1a0790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:26:23.419 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:23.417+0000 7fbc3affd640 1 --2- 172.21.15.82:0/3429605115 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbc3c107080 0x7fbc3c1a0790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:26:23.419 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:23.417+0000 7fbc3affd640 1 -- 172.21.15.82:0/3429605115 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc3c1a0e00 con 0x7fbc3c101140 2024-05-20T16:26:23.419 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:23.417+0000 7fbc3b7fe640 1 --2- 172.21.15.82:0/3429605115 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbc3c107080 0x7fbc3c1a0790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T16:26:23.420 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:23.418+0000 7fbc3affd640 1 --2- 172.21.15.82:0/3429605115 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbc3c101140 0x7fbc3c1992c0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fbc2c00d740 tx=0x7fbc2c00dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:26:23.420 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:23.418+0000 7fbc23fff640 1 -- 172.21.15.82:0/3429605115 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc2c017070 con 0x7fbc3c101140 2024-05-20T16:26:23.420 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:23.418+0000 7fbc415e0640 1 -- 172.21.15.82:0/3429605115 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc3c1a10f0 con 0x7fbc3c101140 2024-05-20T16:26:23.420 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:23.418+0000 7fbc23fff640 1 -- 172.21.15.82:0/3429605115 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc2c00de50 con 0x7fbc3c101140 2024-05-20T16:26:23.420 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:23.418+0000 7fbc23fff640 1 -- 172.21.15.82:0/3429605115 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc2c003e00 con 0x7fbc3c101140 2024-05-20T16:26:23.420 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:23.418+0000 7fbc415e0640 1 -- 172.21.15.82:0/3429605115 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc3c1a1640 con 0x7fbc3c101140 2024-05-20T16:26:23.420 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:23.419+0000 7fbc415e0640 1 -- 172.21.15.82:0/3429605115 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc3c105e90 con 0x7fbc3c101140 2024-05-20T16:26:23.421 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:23.420+0000 7fbc23fff640 1 -- 172.21.15.82:0/3429605115 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fbc2c003fa0 con 0x7fbc3c101140 2024-05-20T16:26:23.421 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:23.421+0000 7fbc23fff640 1 --2- 172.21.15.82:0/3429605115 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fbc240a2b90 0x7fbc240a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:26:23.421 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:23.421+0000 7fbc3a7fc640 1 --2- 172.21.15.82:0/3429605115 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fbc240a2b90 0x7fbc240a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:26:23.421 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:23.421+0000 7fbc23fff640 1 -- 172.21.15.82:0/3429605115 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9150+0+0 (secure 0 0 0) 0x7fbc2c0cf270 con 0x7fbc3c101140 2024-05-20T16:26:23.421 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:23.421+0000 7fbc3a7fc640 1 --2- 172.21.15.82:0/3429605115 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fbc240a2b90 0x7fbc240a50a0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fbc300096c0 tx=0x7fbc30031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:26:23.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:23.424+0000 7fbc23fff640 1 -- 172.21.15.82:0/3429605115 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fbc2c09be40 con 0x7fbc3c101140 2024-05-20T16:26:23.603 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:23.604+0000 7fbc415e0640 1 -- 172.21.15.82:0/3429605115 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fbc3c062750 con 0x7fbc3c101140 2024-05-20T16:26:23.604 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:23.604+0000 7fbc23fff640 1 -- 172.21.15.82:0/3429605115 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v98) ==== 100+0+11 (secure 0 0 0) 0x7fbc2c09b740 con 0x7fbc3c101140 2024-05-20T16:26:23.604 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 64 2024-05-20T16:26:23.606 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:23.606+0000 7fbc415e0640 1 -- 172.21.15.82:0/3429605115 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fbc240a2b90 msgr2=0x7fbc240a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:26:23.606 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:23.606+0000 7fbc415e0640 1 --2- 172.21.15.82:0/3429605115 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fbc240a2b90 0x7fbc240a50a0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fbc300096c0 tx=0x7fbc30031400 comp rx=0 tx=0).stop 2024-05-20T16:26:23.606 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:23.606+0000 7fbc415e0640 1 -- 172.21.15.82:0/3429605115 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbc3c101140 msgr2=0x7fbc3c1992c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:26:23.606 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:23.606+0000 7fbc415e0640 1 --2- 172.21.15.82:0/3429605115 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbc3c101140 0x7fbc3c1992c0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fbc2c00d740 tx=0x7fbc2c00dc00 comp rx=0 tx=0).stop 2024-05-20T16:26:23.606 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:23.606+0000 7fbc415e0640 1 -- 172.21.15.82:0/3429605115 shutdown_connections 2024-05-20T16:26:23.607 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:23.606+0000 7fbc415e0640 1 --2- 172.21.15.82:0/3429605115 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fbc240a2b90 0x7fbc240a50a0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:26:23.607 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:23.606+0000 7fbc415e0640 1 --2- 172.21.15.82:0/3429605115 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbc3c107080 0x7fbc3c1a0790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:26:23.607 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:23.606+0000 7fbc415e0640 1 --2- 172.21.15.82:0/3429605115 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbc3c102380 0x7fbc3c199800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:26:23.607 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:23.606+0000 7fbc415e0640 1 --2- 172.21.15.82:0/3429605115 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbc3c101140 0x7fbc3c1992c0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:26:23.607 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:23.606+0000 7fbc415e0640 1 -- 172.21.15.82:0/3429605115 >> 172.21.15.82:0/3429605115 conn(0x7fbc3c0fc330 msgr2=0x7fbc3c103060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:26:23.607 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:23.606+0000 7fbc415e0640 1 -- 172.21.15.82:0/3429605115 shutdown_connections 2024-05-20T16:26:23.607 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:23.606+0000 7fbc415e0640 1 -- 172.21.15.82:0/3429605115 wait complete. 2024-05-20T16:26:23.617 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-20T16:26:23.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:23.733+0000 7f76a376b640 1 -- 172.21.15.82:0/1982787666 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f769c1089f0 msgr2=0x7f769c10ae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:26:23.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:23.733+0000 7f76a376b640 1 --2- 172.21.15.82:0/1982787666 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f769c1089f0 0x7f769c10ae10 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f7694009ca0 tx=0x7f769402a8c0 comp rx=0 tx=0).stop 2024-05-20T16:26:23.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:23.733+0000 7f76a376b640 1 -- 172.21.15.82:0/1982787666 shutdown_connections 2024-05-20T16:26:23.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:23.733+0000 7f76a376b640 1 --2- 172.21.15.82:0/1982787666 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f769c1089f0 0x7f769c10ae10 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:26:23.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:23.733+0000 7f76a376b640 1 --2- 172.21.15.82:0/1982787666 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f769c103cf0 0x7f769c104100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:26:23.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:23.733+0000 7f76a376b640 1 --2- 172.21.15.82:0/1982787666 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f769c102ab0 0x7f769c1037b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:26:23.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:23.733+0000 7f76a376b640 1 -- 172.21.15.82:0/1982787666 >> 172.21.15.82:0/1982787666 conn(0x7f769c0fdca0 msgr2=0x7f769c100150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:26:23.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:23.733+0000 7f76a376b640 1 -- 172.21.15.82:0/1982787666 shutdown_connections 2024-05-20T16:26:23.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:23.733+0000 7f76a376b640 1 -- 172.21.15.82:0/1982787666 wait complete. 2024-05-20T16:26:23.736 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:23.734+0000 7f76a376b640 1 Processor -- start 2024-05-20T16:26:23.736 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:23.734+0000 7f76a376b640 1 -- start start 2024-05-20T16:26:23.736 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:23.734+0000 7f76a376b640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f769c102ab0 0x7f769c199230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:26:23.736 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:23.734+0000 7f76a376b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f769c103cf0 0x7f769c199770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:26:23.736 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:23.734+0000 7f76a376b640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f769c1089f0 0x7f769c1a0700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:26:23.736 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:23.734+0000 7f76a376b640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f769c10e710 con 0x7f769c103cf0 2024-05-20T16:26:23.736 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:23.734+0000 7f76a376b640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f769c10e590 con 0x7f769c1089f0 2024-05-20T16:26:23.736 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:23.734+0000 7f76a376b640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f769c10e890 con 0x7f769c102ab0 2024-05-20T16:26:23.736 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:23.734+0000 7f76a0d06640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f769c103cf0 0x7f769c199770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:26:23.736 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:23.734+0000 7f76a0d06640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f769c103cf0 0x7f769c199770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:45828/0 (socket says 172.21.15.82:45828) 2024-05-20T16:26:23.736 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:23.734+0000 7f76a0d06640 1 -- 172.21.15.82:0/3556694004 learned_addr learned my addr 172.21.15.82:0/3556694004 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:26:23.736 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:23.734+0000 7f76a1507640 1 --2- 172.21.15.82:0/3556694004 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f769c102ab0 0x7f769c199230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:26:23.736 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:23.734+0000 7f76a1d08640 1 --2- 172.21.15.82:0/3556694004 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f769c1089f0 0x7f769c1a0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:26:23.736 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:23.734+0000 7f76a0d06640 1 -- 172.21.15.82:0/3556694004 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f769c102ab0 msgr2=0x7f769c199230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:26:23.737 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:23.734+0000 7f76a0d06640 1 --2- 172.21.15.82:0/3556694004 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f769c102ab0 0x7f769c199230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:26:23.737 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:23.734+0000 7f76a0d06640 1 -- 172.21.15.82:0/3556694004 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f769c1089f0 msgr2=0x7f769c1a0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:26:23.737 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:23.734+0000 7f76a0d06640 1 --2- 172.21.15.82:0/3556694004 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f769c1089f0 0x7f769c1a0700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:26:23.737 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:23.734+0000 7f76a0d06640 1 -- 172.21.15.82:0/3556694004 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f769c1a0d70 con 0x7f769c103cf0 2024-05-20T16:26:23.737 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:23.734+0000 7f76a1507640 1 --2- 172.21.15.82:0/3556694004 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f769c102ab0 0x7f769c199230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T16:26:23.737 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:23.735+0000 7f76a0d06640 1 --2- 172.21.15.82:0/3556694004 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f769c103cf0 0x7f769c199770 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f768c00e9f0 tx=0x7f768c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:26:23.737 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:23.735+0000 7f76927fc640 1 -- 172.21.15.82:0/3556694004 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f768c00cd80 con 0x7f769c103cf0 2024-05-20T16:26:23.737 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:23.735+0000 7f76927fc640 1 -- 172.21.15.82:0/3556694004 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f768c010910 con 0x7f769c103cf0 2024-05-20T16:26:23.737 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:23.735+0000 7f76a376b640 1 -- 172.21.15.82:0/3556694004 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f769c1a1060 con 0x7f769c103cf0 2024-05-20T16:26:23.737 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:23.735+0000 7f76927fc640 1 -- 172.21.15.82:0/3556694004 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f768c018990 con 0x7f769c103cf0 2024-05-20T16:26:23.737 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:23.735+0000 7f76a376b640 1 -- 172.21.15.82:0/3556694004 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f769c1a15b0 con 0x7f769c103cf0 2024-05-20T16:26:23.738 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:23.736+0000 7f76a376b640 1 -- 172.21.15.82:0/3556694004 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f769c107800 con 0x7f769c103cf0 2024-05-20T16:26:23.738 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:23.737+0000 7f76927fc640 1 -- 172.21.15.82:0/3556694004 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f768c010ac0 con 0x7f769c103cf0 2024-05-20T16:26:23.738 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:23.738+0000 7f76927fc640 1 --2- 172.21.15.82:0/3556694004 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f76880a2b90 0x7f76880a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:26:23.738 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:23.738+0000 7f76a1507640 1 --2- 172.21.15.82:0/3556694004 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f76880a2b90 0x7f76880a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:26:23.738 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:23.738+0000 7f76927fc640 1 -- 172.21.15.82:0/3556694004 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9150+0+0 (secure 0 0 0) 0x7f768c0cfc80 con 0x7f769c103cf0 2024-05-20T16:26:23.738 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:23.738+0000 7f76a1507640 1 --2- 172.21.15.82:0/3556694004 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f76880a2b90 0x7f76880a50a0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f76980058c0 tx=0x7f7698005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:26:23.741 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:23.741+0000 7f76927fc640 1 -- 172.21.15.82:0/3556694004 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f768c09c850 con 0x7f769c103cf0 2024-05-20T16:26:24.102 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:24.101+0000 7f76a376b640 1 -- 172.21.15.82:0/3556694004 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f769c062750 con 0x7f769c103cf0 2024-05-20T16:26:24.104 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:24.104+0000 7f76927fc640 1 -- 172.21.15.82:0/3556694004 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v98) ==== 103+0+11 (secure 0 0 0) 0x7f768c09c150 con 0x7f769c103cf0 2024-05-20T16:26:24.104 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 32 2024-05-20T16:26:24.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:24.108+0000 7f76a376b640 1 -- 172.21.15.82:0/3556694004 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f76880a2b90 msgr2=0x7f76880a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:26:24.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:24.108+0000 7f76a376b640 1 --2- 172.21.15.82:0/3556694004 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f76880a2b90 0x7f76880a50a0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f76980058c0 tx=0x7f7698005850 comp rx=0 tx=0).stop 2024-05-20T16:26:24.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:24.108+0000 7f76a376b640 1 -- 172.21.15.82:0/3556694004 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f769c103cf0 msgr2=0x7f769c199770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:26:24.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:24.108+0000 7f76a376b640 1 --2- 172.21.15.82:0/3556694004 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f769c103cf0 0x7f769c199770 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f768c00e9f0 tx=0x7f768c00eeb0 comp rx=0 tx=0).stop 2024-05-20T16:26:24.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:24.108+0000 7f76a376b640 1 -- 172.21.15.82:0/3556694004 shutdown_connections 2024-05-20T16:26:24.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:24.108+0000 7f76a376b640 1 --2- 172.21.15.82:0/3556694004 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f76880a2b90 0x7f76880a50a0 secure :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f76980058c0 tx=0x7f7698005850 comp rx=0 tx=0).stop 2024-05-20T16:26:24.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:24.108+0000 7f76a376b640 1 --2- 172.21.15.82:0/3556694004 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f769c103cf0 0x7f769c199770 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:26:24.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:24.108+0000 7f76a376b640 1 --2- 172.21.15.82:0/3556694004 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f769c1089f0 0x7f769c1a0700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:26:24.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:24.108+0000 7f76a376b640 1 --2- 172.21.15.82:0/3556694004 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f769c102ab0 0x7f769c199230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:26:24.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:24.108+0000 7f76a376b640 1 -- 172.21.15.82:0/3556694004 >> 172.21.15.82:0/3556694004 conn(0x7f769c0fdca0 msgr2=0x7f769c1049d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:26:24.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:24.108+0000 7f76a376b640 1 -- 172.21.15.82:0/3556694004 shutdown_connections 2024-05-20T16:26:24.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:24.108+0000 7f76a376b640 1 -- 172.21.15.82:0/3556694004 wait complete. 2024-05-20T16:26:24.130 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-20T16:26:24.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:24.271+0000 7f07c9502640 1 -- 172.21.15.82:0/3287323461 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f07c4104040 msgr2=0x7f07c4104450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:26:24.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:24.271+0000 7f07c9502640 1 --2- 172.21.15.82:0/3287323461 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f07c4104040 0x7f07c4104450 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f07bc009900 tx=0x7f07bc02a410 comp rx=0 tx=0).stop 2024-05-20T16:26:24.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:24.271+0000 7f07c9502640 1 -- 172.21.15.82:0/3287323461 shutdown_connections 2024-05-20T16:26:24.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:24.271+0000 7f07c9502640 1 --2- 172.21.15.82:0/3287323461 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f07c41011f0 0x7f07c410ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:26:24.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:24.271+0000 7f07c9502640 1 --2- 172.21.15.82:0/3287323461 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f07c41006f0 0x7f07c4100b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:26:24.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:24.271+0000 7f07c9502640 1 --2- 172.21.15.82:0/3287323461 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f07c4104040 0x7f07c4104450 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:26:24.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:24.271+0000 7f07c9502640 1 -- 172.21.15.82:0/3287323461 >> 172.21.15.82:0/3287323461 conn(0x7f07c40fc2f0 msgr2=0x7f07c40fe7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:26:24.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:24.271+0000 7f07c9502640 1 -- 172.21.15.82:0/3287323461 shutdown_connections 2024-05-20T16:26:24.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:24.271+0000 7f07c9502640 1 -- 172.21.15.82:0/3287323461 wait complete. 2024-05-20T16:26:24.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:24.271+0000 7f07c9502640 1 Processor -- start 2024-05-20T16:26:24.273 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:24.272+0000 7f07c9502640 1 -- start start 2024-05-20T16:26:24.274 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:24.272+0000 7f07c9502640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f07c41006f0 0x7f07c41992c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:26:24.274 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:24.272+0000 7f07c9502640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f07c41011f0 0x7f07c4199800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:26:24.274 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:24.272+0000 7f07c9502640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f07c4104040 0x7f07c41a0790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:26:24.274 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:24.272+0000 7f07c9502640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f07c410e750 con 0x7f07c41011f0 2024-05-20T16:26:24.274 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:24.272+0000 7f07c9502640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f07c410e5d0 con 0x7f07c4104040 2024-05-20T16:26:24.274 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:24.272+0000 7f07c9502640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f07c410e8d0 con 0x7f07c41006f0 2024-05-20T16:26:24.274 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:24.272+0000 7f07c27fc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f07c41011f0 0x7f07c4199800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:26:24.274 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:24.272+0000 7f07c27fc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f07c41011f0 0x7f07c4199800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:45850/0 (socket says 172.21.15.82:45850) 2024-05-20T16:26:24.274 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:24.272+0000 7f07c27fc640 1 -- 172.21.15.82:0/2855556769 learned_addr learned my addr 172.21.15.82:0/2855556769 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:26:24.274 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:24.272+0000 7f07c2ffd640 1 --2- 172.21.15.82:0/2855556769 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f07c41006f0 0x7f07c41992c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:26:24.274 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:24.272+0000 7f07c37fe640 1 --2- 172.21.15.82:0/2855556769 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f07c4104040 0x7f07c41a0790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:26:24.274 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:24.272+0000 7f07c27fc640 1 -- 172.21.15.82:0/2855556769 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f07c41006f0 msgr2=0x7f07c41992c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:26:24.274 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:24.272+0000 7f07c27fc640 1 --2- 172.21.15.82:0/2855556769 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f07c41006f0 0x7f07c41992c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:26:24.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:24.272+0000 7f07c27fc640 1 -- 172.21.15.82:0/2855556769 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f07c4104040 msgr2=0x7f07c41a0790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:26:24.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:24.272+0000 7f07c27fc640 1 --2- 172.21.15.82:0/2855556769 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f07c4104040 0x7f07c41a0790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:26:24.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:24.272+0000 7f07c27fc640 1 -- 172.21.15.82:0/2855556769 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07c41a0e00 con 0x7f07c41011f0 2024-05-20T16:26:24.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:24.272+0000 7f07c27fc640 1 --2- 172.21.15.82:0/2855556769 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f07c41011f0 0x7f07c4199800 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f07b800db80 tx=0x7f07b800b630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:26:24.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:24.273+0000 7f07abfff640 1 -- 172.21.15.82:0/2855556769 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07b8017070 con 0x7f07c41011f0 2024-05-20T16:26:24.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:24.273+0000 7f07abfff640 1 -- 172.21.15.82:0/2855556769 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f07b8003b70 con 0x7f07c41011f0 2024-05-20T16:26:24.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:24.273+0000 7f07c9502640 1 -- 172.21.15.82:0/2855556769 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07c41a10f0 con 0x7f07c41011f0 2024-05-20T16:26:24.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:24.273+0000 7f07abfff640 1 -- 172.21.15.82:0/2855556769 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07b80106c0 con 0x7f07c41011f0 2024-05-20T16:26:24.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:24.273+0000 7f07c9502640 1 -- 172.21.15.82:0/2855556769 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07c41a1640 con 0x7f07c41011f0 2024-05-20T16:26:24.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:24.274+0000 7f07c9502640 1 -- 172.21.15.82:0/2855556769 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07c41079d0 con 0x7f07c41011f0 2024-05-20T16:26:24.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:24.276+0000 7f07abfff640 1 -- 172.21.15.82:0/2855556769 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f07b8004020 con 0x7f07c41011f0 2024-05-20T16:26:24.277 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:24.277+0000 7f07abfff640 1 --2- 172.21.15.82:0/2855556769 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f07ac0a2b90 0x7f07ac0a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:26:24.277 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:24.277+0000 7f07c2ffd640 1 --2- 172.21.15.82:0/2855556769 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f07ac0a2b90 0x7f07ac0a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:26:24.277 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:24.277+0000 7f07abfff640 1 -- 172.21.15.82:0/2855556769 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9150+0+0 (secure 0 0 0) 0x7f07b80cf4e0 con 0x7f07c41011f0 2024-05-20T16:26:24.277 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:24.278+0000 7f07c2ffd640 1 --2- 172.21.15.82:0/2855556769 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f07ac0a2b90 0x7f07ac0a50a0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f07bc02aa00 tx=0x7f07bc0315b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:26:24.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:24.280+0000 7f07abfff640 1 -- 172.21.15.82:0/2855556769 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f07b8020070 con 0x7f07c41011f0 2024-05-20T16:26:24.311 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:26:24 smithi160 ceph-mon[36347]: from='client.? 172.21.15.82:0/4008671602' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-20T16:26:24.312 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:26:24 smithi160 ceph-mon[36347]: from='client.? 172.21.15.82:0/3429605115' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-20T16:26:24.312 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:26:24 smithi160 ceph-mon[36347]: pgmap v737: 129 pgs: 129 active+clean; 8.0 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 61 MiB/s wr, 402 op/s 2024-05-20T16:26:24.358 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:26:24 smithi082 ceph-mon[32238]: from='client.? 172.21.15.82:0/4008671602' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-20T16:26:24.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:26:24 smithi082 ceph-mon[32238]: from='client.? 172.21.15.82:0/3429605115' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-20T16:26:24.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:26:24 smithi082 ceph-mon[32238]: pgmap v737: 129 pgs: 129 active+clean; 8.0 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 61 MiB/s wr, 402 op/s 2024-05-20T16:26:24.385 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:26:24 smithi184 ceph-mon[36209]: from='client.? 172.21.15.82:0/4008671602' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-20T16:26:24.385 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:26:24 smithi184 ceph-mon[36209]: from='client.? 172.21.15.82:0/3429605115' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-20T16:26:24.385 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:26:24 smithi184 ceph-mon[36209]: pgmap v737: 129 pgs: 129 active+clean; 8.0 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 61 MiB/s wr, 402 op/s 2024-05-20T16:26:24.483 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:24.484+0000 7f07c9502640 1 -- 172.21.15.82:0/2855556769 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f07c402d2a0 con 0x7f07c41011f0 2024-05-20T16:26:24.484 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:24.484+0000 7f07abfff640 1 -- 172.21.15.82:0/2855556769 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v98) ==== 81+0+591 (secure 0 0 0) 0x7f07b809c090 con 0x7f07c41011f0 2024-05-20T16:26:24.484 INFO:teuthology.orchestra.run.smithi082.stdout:172.21.15.82:0/2005424509 2024-05-21T16:04:45.161949+0000 2024-05-20T16:26:24.484 INFO:teuthology.orchestra.run.smithi082.stdout:172.21.15.82:6801/1763049308 2024-05-21T16:04:45.161949+0000 2024-05-20T16:26:24.484 INFO:teuthology.orchestra.run.smithi082.stdout:172.21.15.82:6801/1959459348 2024-05-21T16:04:19.208291+0000 2024-05-20T16:26:24.484 INFO:teuthology.orchestra.run.smithi082.stdout:172.21.15.82:0/3131904556 2024-05-21T16:04:19.208291+0000 2024-05-20T16:26:24.484 INFO:teuthology.orchestra.run.smithi082.stdout:172.21.15.82:6800/1959459348 2024-05-21T16:04:19.208291+0000 2024-05-20T16:26:24.484 INFO:teuthology.orchestra.run.smithi082.stdout:172.21.15.82:0/2456966342 2024-05-21T16:04:45.161949+0000 2024-05-20T16:26:24.484 INFO:teuthology.orchestra.run.smithi082.stdout:172.21.15.82:0/4179978833 2024-05-21T16:04:19.208291+0000 2024-05-20T16:26:24.485 INFO:teuthology.orchestra.run.smithi082.stdout:172.21.15.82:0/1602862225 2024-05-21T16:04:45.161949+0000 2024-05-20T16:26:24.485 INFO:teuthology.orchestra.run.smithi082.stdout:172.21.15.82:0/106101180 2024-05-21T16:04:19.208291+0000 2024-05-20T16:26:24.485 INFO:teuthology.orchestra.run.smithi082.stdout:172.21.15.82:6800/1763049308 2024-05-21T16:04:45.161949+0000 2024-05-20T16:26:24.485 INFO:teuthology.orchestra.run.smithi082.stderr:listed 10 entries 2024-05-20T16:26:24.486 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:24.486+0000 7f07c9502640 1 -- 172.21.15.82:0/2855556769 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f07ac0a2b90 msgr2=0x7f07ac0a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:26:24.486 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:24.486+0000 7f07c9502640 1 --2- 172.21.15.82:0/2855556769 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f07ac0a2b90 0x7f07ac0a50a0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f07bc02aa00 tx=0x7f07bc0315b0 comp rx=0 tx=0).stop 2024-05-20T16:26:24.486 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:24.486+0000 7f07c9502640 1 -- 172.21.15.82:0/2855556769 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f07c41011f0 msgr2=0x7f07c4199800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:26:24.486 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:24.486+0000 7f07c9502640 1 --2- 172.21.15.82:0/2855556769 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f07c41011f0 0x7f07c4199800 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f07b800db80 tx=0x7f07b800b630 comp rx=0 tx=0).stop 2024-05-20T16:26:24.486 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:24.486+0000 7f07c9502640 1 -- 172.21.15.82:0/2855556769 shutdown_connections 2024-05-20T16:26:24.486 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:24.486+0000 7f07c9502640 1 --2- 172.21.15.82:0/2855556769 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f07ac0a2b90 0x7f07ac0a50a0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:26:24.486 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:24.486+0000 7f07c9502640 1 --2- 172.21.15.82:0/2855556769 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f07c41011f0 0x7f07c4199800 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:26:24.486 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:24.486+0000 7f07c9502640 1 --2- 172.21.15.82:0/2855556769 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f07c4104040 0x7f07c41a0790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:26:24.487 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:24.486+0000 7f07c9502640 1 --2- 172.21.15.82:0/2855556769 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f07c41006f0 0x7f07c41992c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:26:24.487 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:24.486+0000 7f07c9502640 1 -- 172.21.15.82:0/2855556769 >> 172.21.15.82:0/2855556769 conn(0x7f07c40fc2f0 msgr2=0x7f07c4104ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:26:24.487 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:24.486+0000 7f07c9502640 1 -- 172.21.15.82:0/2855556769 shutdown_connections 2024-05-20T16:26:24.487 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:24.486+0000 7f07c9502640 1 -- 172.21.15.82:0/2855556769 wait complete. 2024-05-20T16:26:24.498 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-05-20T16:26:24.498 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T16:26:24.498 DEBUG:teuthology.orchestra.run.smithi082:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-05-20T16:26:25.312 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:26:25 smithi160 ceph-mon[36347]: from='client.? 172.21.15.82:0/3556694004' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-20T16:26:25.312 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:26:25 smithi160 ceph-mon[36347]: from='client.? 172.21.15.82:0/2855556769' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-20T16:26:25.358 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:26:25 smithi082 ceph-mon[32238]: from='client.? 172.21.15.82:0/3556694004' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-20T16:26:25.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:26:25 smithi082 ceph-mon[32238]: from='client.? 172.21.15.82:0/2855556769' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-20T16:26:25.385 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:26:25 smithi184 ceph-mon[36209]: from='client.? 172.21.15.82:0/3556694004' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-20T16:26:25.385 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:26:25 smithi184 ceph-mon[36209]: from='client.? 172.21.15.82:0/2855556769' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-20T16:26:26.311 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:26:26 smithi160 ceph-mon[36347]: pgmap v738: 129 pgs: 129 active+clean; 8.0 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 84 MiB/s wr, 581 op/s 2024-05-20T16:26:26.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:26:26 smithi082 ceph-mon[32238]: pgmap v738: 129 pgs: 129 active+clean; 8.0 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 84 MiB/s wr, 581 op/s 2024-05-20T16:26:26.385 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:26:26 smithi184 ceph-mon[36209]: pgmap v738: 129 pgs: 129 active+clean; 8.0 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 84 MiB/s wr, 581 op/s 2024-05-20T16:26:27.455 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi082.front.sepia.ceph.com 2024-05-20T16:26:27.455 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T16:26:27.455 DEBUG:teuthology.orchestra.run.smithi082:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-20T16:26:27.492 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T16:26:27.492 DEBUG:teuthology.orchestra.run.smithi082:> ip netns list 2024-05-20T16:26:27.580 INFO:teuthology.orchestra.run.smithi082.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-20T16:26:27.580 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T16:26:27.580 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-20T16:26:27.622 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T16:26:27.622 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip link delete ceph-brx 2024-05-20T16:26:27.725 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-05-20T16:26:27.735 INFO:tasks.cephadm:Teardown begin 2024-05-20T16:26:27.736 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-05-20T16:26:27.736 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-20T16:26:27.893 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:27.890+0000 7fe9696e9640 1 -- 172.21.15.82:0/1032212769 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe9641031a0 msgr2=0x7fe9641055c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:26:27.893 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:27.890+0000 7fe9696e9640 1 --2- 172.21.15.82:0/1032212769 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe9641031a0 0x7fe9641055c0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fe954009900 tx=0x7fe95402a410 comp rx=0 tx=0).stop 2024-05-20T16:26:27.893 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:27.890+0000 7fe9696e9640 1 -- 172.21.15.82:0/1032212769 shutdown_connections 2024-05-20T16:26:27.893 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:27.890+0000 7fe9696e9640 1 --2- 172.21.15.82:0/1032212769 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe964108700 0x7fe96410ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:26:27.893 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:27.890+0000 7fe9696e9640 1 --2- 172.21.15.82:0/1032212769 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe964105b90 0x7fe964108070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:26:27.893 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:27.890+0000 7fe9696e9640 1 --2- 172.21.15.82:0/1032212769 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe9641031a0 0x7fe9641055c0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:26:27.893 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:27.890+0000 7fe9696e9640 1 -- 172.21.15.82:0/1032212769 >> 172.21.15.82:0/1032212769 conn(0x7fe9640fcb80 msgr2=0x7fe9640ff010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:26:27.893 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:27.890+0000 7fe9696e9640 1 -- 172.21.15.82:0/1032212769 shutdown_connections 2024-05-20T16:26:27.893 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:27.890+0000 7fe9696e9640 1 -- 172.21.15.82:0/1032212769 wait complete. 2024-05-20T16:26:27.894 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:27.891+0000 7fe9696e9640 1 Processor -- start 2024-05-20T16:26:27.894 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:27.891+0000 7fe9696e9640 1 -- start start 2024-05-20T16:26:27.894 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:27.891+0000 7fe9696e9640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe9641031a0 0x7fe964199290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:26:27.894 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:27.891+0000 7fe9696e9640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe964105b90 0x7fe9641997d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:26:27.894 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:27.891+0000 7fe9696e9640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe964108700 0x7fe9641a0760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:26:27.894 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:27.891+0000 7fe9696e9640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fe96410e6b0 con 0x7fe964108700 2024-05-20T16:26:27.894 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:27.891+0000 7fe9696e9640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fe96410e530 con 0x7fe9641031a0 2024-05-20T16:26:27.894 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:27.891+0000 7fe9696e9640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fe96410e830 con 0x7fe964105b90 2024-05-20T16:26:27.894 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:27.891+0000 7fe9637fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe964108700 0x7fe9641a0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:26:27.894 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:27.891+0000 7fe9637fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe964108700 0x7fe9641a0760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:45874/0 (socket says 172.21.15.82:45874) 2024-05-20T16:26:27.895 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:27.891+0000 7fe9637fe640 1 -- 172.21.15.82:0/2300391398 learned_addr learned my addr 172.21.15.82:0/2300391398 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:26:27.895 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:27.891+0000 7fe962ffd640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe9641031a0 0x7fe964199290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:26:27.895 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:27.891+0000 7fe95bfff640 1 --2- 172.21.15.82:0/2300391398 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe964105b90 0x7fe9641997d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:26:27.895 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:27.892+0000 7fe9637fe640 1 -- 172.21.15.82:0/2300391398 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe964105b90 msgr2=0x7fe9641997d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:26:27.895 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:27.892+0000 7fe9637fe640 1 --2- 172.21.15.82:0/2300391398 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe964105b90 0x7fe9641997d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:26:27.895 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:27.892+0000 7fe9637fe640 1 -- 172.21.15.82:0/2300391398 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe9641031a0 msgr2=0x7fe964199290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:26:27.895 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:27.892+0000 7fe9637fe640 1 --2- 172.21.15.82:0/2300391398 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe9641031a0 0x7fe964199290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:26:27.895 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:27.892+0000 7fe9637fe640 1 -- 172.21.15.82:0/2300391398 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9641a0dd0 con 0x7fe964108700 2024-05-20T16:26:27.895 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:27.892+0000 7fe9637fe640 1 --2- 172.21.15.82:0/2300391398 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe964108700 0x7fe9641a0760 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fe95c00eaa0 tx=0x7fe95c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:26:27.895 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:27.892+0000 7fe960ff9640 1 -- 172.21.15.82:0/2300391398 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe95c00cdb0 con 0x7fe964108700 2024-05-20T16:26:27.895 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:27.892+0000 7fe960ff9640 1 -- 172.21.15.82:0/2300391398 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe95c005020 con 0x7fe964108700 2024-05-20T16:26:27.895 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:27.892+0000 7fe9696e9640 1 -- 172.21.15.82:0/2300391398 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9641a10c0 con 0x7fe964108700 2024-05-20T16:26:27.895 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:27.892+0000 7fe960ff9640 1 -- 172.21.15.82:0/2300391398 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe95c0116b0 con 0x7fe964108700 2024-05-20T16:26:27.896 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:27.892+0000 7fe9696e9640 1 -- 172.21.15.82:0/2300391398 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9641a1610 con 0x7fe964108700 2024-05-20T16:26:27.896 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:27.893+0000 7fe9696e9640 1 -- 172.21.15.82:0/2300391398 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9640fe6a0 con 0x7fe964108700 2024-05-20T16:26:27.897 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:27.894+0000 7fe960ff9640 1 -- 172.21.15.82:0/2300391398 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fe95c005560 con 0x7fe964108700 2024-05-20T16:26:27.897 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:27.895+0000 7fe960ff9640 1 --2- 172.21.15.82:0/2300391398 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fe9500a2c10 0x7fe9500a5120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:26:27.897 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:27.895+0000 7fe962ffd640 1 --2- 172.21.15.82:0/2300391398 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fe9500a2c10 0x7fe9500a5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:26:27.897 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:27.895+0000 7fe960ff9640 1 -- 172.21.15.82:0/2300391398 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9150+0+0 (secure 0 0 0) 0x7fe95c0cf540 con 0x7fe964108700 2024-05-20T16:26:27.897 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:27.895+0000 7fe962ffd640 1 --2- 172.21.15.82:0/2300391398 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fe9500a2c10 0x7fe9500a5120 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fe954009230 tx=0x7fe954031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:26:27.899 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:27.900+0000 7fe960ff9640 1 -- 172.21.15.82:0/2300391398 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fe95c016030 con 0x7fe964108700 2024-05-20T16:26:28.116 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:28.116+0000 7fe9696e9640 1 -- 172.21.15.82:0/2300391398 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe96402d2a0 con 0x7fe964108700 2024-05-20T16:26:28.116 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:28.117+0000 7fe960ff9640 1 -- 172.21.15.82:0/2300391398 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+11879 (secure 0 0 0) 0x7fe95c09bac0 con 0x7fe964108700 2024-05-20T16:26:28.117 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-05-20T16:26:28.117 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":31,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/2044534217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":2044534217},{"type":"v1","addr":"172.21.15.82:6835","nonce":2044534217}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/2635635097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":2635635097},{"type":"v1","addr":"172.21.15.82:6837","nonce":2635635097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14625,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6839/2118233541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6838","nonce":2118233541},{"type":"v1","addr":"172.21.15.82:6839","nonce":2118233541}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14637,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6841/2119517973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6840","nonce":2119517973},{"type":"v1","addr":"172.21.15.82:6841","nonce":2119517973}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6833/4015144007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":4015144007},{"type":"v1","addr":"172.21.15.184:6833","nonce":4015144007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6833/975243372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6832","nonce":975243372},{"type":"v1","addr":"172.21.15.160:6833","nonce":975243372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":31,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-20T16:11:46.751632+0000","modified":"2024-05-20T16:25:03.964111+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24548,"mds_1":24511,"mds_2":24506},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24470":{"gid":24470,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.160:6835/664068363","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":664068363},{"type":"v1","addr":"172.21.15.160:6835","nonce":664068363}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6835/4242156090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":4242156090},{"type":"v1","addr":"172.21.15.184:6835","nonce":4242156090}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.160:6837/2846188680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6836","nonce":2846188680},{"type":"v1","addr":"172.21.15.160:6837","nonce":2846188680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.184:6839/2199955067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":2199955067},{"type":"v1","addr":"172.21.15.184:6839","nonce":2199955067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24548":{"gid":24548,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.160:6839/2765554482","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6838","nonce":2765554482},{"type":"v1","addr":"172.21.15.160:6839","nonce":2765554482}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25528":{"gid":25528,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6837/371714807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":371714807},{"type":"v1","addr":"172.21.15.184:6837","nonce":371714807}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24548,"qdb_cluster":[24511,24548,24506]},"id":1}]} 2024-05-20T16:26:28.117 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 31 2024-05-20T16:26:28.119 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:28.119+0000 7fe9696e9640 1 -- 172.21.15.82:0/2300391398 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fe9500a2c10 msgr2=0x7fe9500a5120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:26:28.119 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:28.119+0000 7fe9696e9640 1 --2- 172.21.15.82:0/2300391398 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fe9500a2c10 0x7fe9500a5120 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fe954009230 tx=0x7fe954031400 comp rx=0 tx=0).stop 2024-05-20T16:26:28.119 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:28.119+0000 7fe9696e9640 1 -- 172.21.15.82:0/2300391398 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe964108700 msgr2=0x7fe9641a0760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:26:28.119 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:28.119+0000 7fe9696e9640 1 --2- 172.21.15.82:0/2300391398 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe964108700 0x7fe9641a0760 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fe95c00eaa0 tx=0x7fe95c00ef60 comp rx=0 tx=0).stop 2024-05-20T16:26:28.119 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:28.119+0000 7fe9696e9640 1 -- 172.21.15.82:0/2300391398 shutdown_connections 2024-05-20T16:26:28.119 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:28.119+0000 7fe9696e9640 1 --2- 172.21.15.82:0/2300391398 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fe9500a2c10 0x7fe9500a5120 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:26:28.119 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:28.119+0000 7fe9696e9640 1 --2- 172.21.15.82:0/2300391398 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe964108700 0x7fe9641a0760 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:26:28.119 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:28.119+0000 7fe9696e9640 1 --2- 172.21.15.82:0/2300391398 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe964105b90 0x7fe9641997d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:26:28.119 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:28.119+0000 7fe9696e9640 1 --2- 172.21.15.82:0/2300391398 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe9641031a0 0x7fe964199290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:26:28.119 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:28.119+0000 7fe9696e9640 1 -- 172.21.15.82:0/2300391398 >> 172.21.15.82:0/2300391398 conn(0x7fe9640fcb80 msgr2=0x7fe964103e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:26:28.119 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:28.119+0000 7fe9696e9640 1 -- 172.21.15.82:0/2300391398 shutdown_connections 2024-05-20T16:26:28.119 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:28.119+0000 7fe9696e9640 1 -- 172.21.15.82:0/2300391398 wait complete. 2024-05-20T16:26:28.131 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-20T16:26:28.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:28.243+0000 7f27ea2b8640 1 -- 172.21.15.82:0/2721812457 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f27e4102ab0 msgr2=0x7f27e41037b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:26:28.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:28.243+0000 7f27ea2b8640 1 --2- 172.21.15.82:0/2721812457 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f27e4102ab0 0x7f27e41037b0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f27d4009900 tx=0x7f27d402a410 comp rx=0 tx=0).stop 2024-05-20T16:26:28.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:28.243+0000 7f27ea2b8640 1 -- 172.21.15.82:0/2721812457 shutdown_connections 2024-05-20T16:26:28.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:28.243+0000 7f27ea2b8640 1 --2- 172.21.15.82:0/2721812457 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f27e41089f0 0x7f27e410ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:26:28.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:28.243+0000 7f27ea2b8640 1 --2- 172.21.15.82:0/2721812457 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f27e4103cf0 0x7f27e4104100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:26:28.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:28.243+0000 7f27ea2b8640 1 --2- 172.21.15.82:0/2721812457 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f27e4102ab0 0x7f27e41037b0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:26:28.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:28.243+0000 7f27ea2b8640 1 -- 172.21.15.82:0/2721812457 >> 172.21.15.82:0/2721812457 conn(0x7f27e40fdcc0 msgr2=0x7f27e4100150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:26:28.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:28.243+0000 7f27ea2b8640 1 -- 172.21.15.82:0/2721812457 shutdown_connections 2024-05-20T16:26:28.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:28.243+0000 7f27ea2b8640 1 -- 172.21.15.82:0/2721812457 wait complete. 2024-05-20T16:26:28.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:28.244+0000 7f27ea2b8640 1 Processor -- start 2024-05-20T16:26:28.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:28.244+0000 7f27ea2b8640 1 -- start start 2024-05-20T16:26:28.246 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:28.244+0000 7f27ea2b8640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f27e4102ab0 0x7f27e41992f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:26:28.246 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:28.244+0000 7f27ea2b8640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f27e4103cf0 0x7f27e4199830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:26:28.246 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:28.244+0000 7f27ea2b8640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f27e41089f0 0x7f27e41a07c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:26:28.246 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:28.244+0000 7f27ea2b8640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f27e410e750 con 0x7f27e41089f0 2024-05-20T16:26:28.246 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:28.244+0000 7f27ea2b8640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f27e410e5d0 con 0x7f27e4102ab0 2024-05-20T16:26:28.246 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:28.244+0000 7f27ea2b8640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f27e410e8d0 con 0x7f27e4103cf0 2024-05-20T16:26:28.246 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:28.244+0000 7f27e8855640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f27e41089f0 0x7f27e41a07c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:26:28.246 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:28.244+0000 7f27e8855640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f27e41089f0 0x7f27e41a07c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:45898/0 (socket says 172.21.15.82:45898) 2024-05-20T16:26:28.246 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:28.244+0000 7f27e8855640 1 -- 172.21.15.82:0/4182150349 learned_addr learned my addr 172.21.15.82:0/4182150349 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:26:28.246 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:28.245+0000 7f27e3fff640 1 --2- 172.21.15.82:0/4182150349 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f27e4102ab0 0x7f27e41992f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:26:28.247 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:28.245+0000 7f27e37fe640 1 --2- 172.21.15.82:0/4182150349 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f27e4103cf0 0x7f27e4199830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:26:28.247 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:28.245+0000 7f27e8855640 1 -- 172.21.15.82:0/4182150349 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f27e4103cf0 msgr2=0x7f27e4199830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:26:28.247 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:28.245+0000 7f27e8855640 1 --2- 172.21.15.82:0/4182150349 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f27e4103cf0 0x7f27e4199830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:26:28.247 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:28.245+0000 7f27e8855640 1 -- 172.21.15.82:0/4182150349 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f27e4102ab0 msgr2=0x7f27e41992f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:26:28.247 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:28.245+0000 7f27e8855640 1 --2- 172.21.15.82:0/4182150349 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f27e4102ab0 0x7f27e41992f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:26:28.247 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:28.245+0000 7f27e8855640 1 -- 172.21.15.82:0/4182150349 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27e41a0e30 con 0x7f27e41089f0 2024-05-20T16:26:28.247 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:28.245+0000 7f27e8855640 1 --2- 172.21.15.82:0/4182150349 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f27e41089f0 0x7f27e41a07c0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f27dc00eb10 tx=0x7f27dc00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:26:28.247 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:28.245+0000 7f27e17fa640 1 -- 172.21.15.82:0/4182150349 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27dc004b20 con 0x7f27e41089f0 2024-05-20T16:26:28.247 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:28.245+0000 7f27e17fa640 1 -- 172.21.15.82:0/4182150349 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f27dc004cc0 con 0x7f27e41089f0 2024-05-20T16:26:28.247 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:28.245+0000 7f27ea2b8640 1 -- 172.21.15.82:0/4182150349 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27e41a1120 con 0x7f27e41089f0 2024-05-20T16:26:28.247 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:28.245+0000 7f27e17fa640 1 -- 172.21.15.82:0/4182150349 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27dc010760 con 0x7f27e41089f0 2024-05-20T16:26:28.247 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:28.245+0000 7f27ea2b8640 1 -- 172.21.15.82:0/4182150349 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27e41a1620 con 0x7f27e41089f0 2024-05-20T16:26:28.247 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:28.246+0000 7f27ea2b8640 1 -- 172.21.15.82:0/4182150349 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27e4107800 con 0x7f27e41089f0 2024-05-20T16:26:28.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:28.248+0000 7f27e17fa640 1 -- 172.21.15.82:0/4182150349 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f27dc005000 con 0x7f27e41089f0 2024-05-20T16:26:28.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:28.249+0000 7f27e17fa640 1 --2- 172.21.15.82:0/4182150349 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f27cc0a2b90 0x7f27cc0a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:26:28.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:28.249+0000 7f27e17fa640 1 -- 172.21.15.82:0/4182150349 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9150+0+0 (secure 0 0 0) 0x7f27dc0cfd60 con 0x7f27e41089f0 2024-05-20T16:26:28.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:28.249+0000 7f27e3fff640 1 --2- 172.21.15.82:0/4182150349 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f27cc0a2b90 0x7f27cc0a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:26:28.249 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:28.249+0000 7f27e3fff640 1 --2- 172.21.15.82:0/4182150349 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f27cc0a2b90 0x7f27cc0a50a0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f27d402aa00 tx=0x7f27d4031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:26:28.252 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:28.252+0000 7f27e17fa640 1 -- 172.21.15.82:0/4182150349 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f27dc053850 con 0x7f27e41089f0 2024-05-20T16:26:28.661 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:28.661+0000 7f27ea2b8640 1 -- 172.21.15.82:0/4182150349 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f27e402d2a0 con 0x7f27e41089f0 2024-05-20T16:26:28.665 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-05-20T16:26:28.665 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":31,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/2044534217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":2044534217},{"type":"v1","addr":"172.21.15.82:6835","nonce":2044534217}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/2635635097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":2635635097},{"type":"v1","addr":"172.21.15.82:6837","nonce":2635635097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14625,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6839/2118233541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6838","nonce":2118233541},{"type":"v1","addr":"172.21.15.82:6839","nonce":2118233541}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14637,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6841/2119517973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6840","nonce":2119517973},{"type":"v1","addr":"172.21.15.82:6841","nonce":2119517973}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6833/4015144007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":4015144007},{"type":"v1","addr":"172.21.15.184:6833","nonce":4015144007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6833/975243372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6832","nonce":975243372},{"type":"v1","addr":"172.21.15.160:6833","nonce":975243372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":31,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-20T16:11:46.751632+0000","modified":"2024-05-20T16:25:03.964111+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24548,"mds_1":24511,"mds_2":24506},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24470":{"gid":24470,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.160:6835/664068363","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":664068363},{"type":"v1","addr":"172.21.15.160:6835","nonce":664068363}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6835/4242156090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":4242156090},{"type":"v1","addr":"172.21.15.184:6835","nonce":4242156090}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.160:6837/2846188680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6836","nonce":2846188680},{"type":"v1","addr":"172.21.15.160:6837","nonce":2846188680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.184:6839/2199955067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":2199955067},{"type":"v1","addr":"172.21.15.184:6839","nonce":2199955067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24548":{"gid":24548,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.160:6839/2765554482","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6838","nonce":2765554482},{"type":"v1","addr":"172.21.15.160:6839","nonce":2765554482}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25528":{"gid":25528,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6837/371714807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":371714807},{"type":"v1","addr":"172.21.15.184:6837","nonce":371714807}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24548,"qdb_cluster":[24511,24548,24506]},"id":1}]} 2024-05-20T16:26:28.665 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:28.665+0000 7f27e17fa640 1 -- 172.21.15.82:0/4182150349 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+11879 (secure 0 0 0) 0x7f27dc09c8d0 con 0x7f27e41089f0 2024-05-20T16:26:28.665 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 31 2024-05-20T16:26:28.668 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:28.668+0000 7f27ea2b8640 1 -- 172.21.15.82:0/4182150349 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f27cc0a2b90 msgr2=0x7f27cc0a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:26:28.668 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:28.668+0000 7f27ea2b8640 1 --2- 172.21.15.82:0/4182150349 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f27cc0a2b90 0x7f27cc0a50a0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f27d402aa00 tx=0x7f27d4031400 comp rx=0 tx=0).stop 2024-05-20T16:26:28.668 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:28.668+0000 7f27ea2b8640 1 -- 172.21.15.82:0/4182150349 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f27e41089f0 msgr2=0x7f27e41a07c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:26:28.668 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:28.668+0000 7f27ea2b8640 1 --2- 172.21.15.82:0/4182150349 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f27e41089f0 0x7f27e41a07c0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f27dc00eb10 tx=0x7f27dc00c2d0 comp rx=0 tx=0).stop 2024-05-20T16:26:28.669 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:28.669+0000 7f27ea2b8640 1 -- 172.21.15.82:0/4182150349 shutdown_connections 2024-05-20T16:26:28.669 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:28.669+0000 7f27ea2b8640 1 --2- 172.21.15.82:0/4182150349 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f27cc0a2b90 0x7f27cc0a50a0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:26:28.669 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:28.669+0000 7f27ea2b8640 1 --2- 172.21.15.82:0/4182150349 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f27e41089f0 0x7f27e41a07c0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:26:28.669 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:28.669+0000 7f27ea2b8640 1 --2- 172.21.15.82:0/4182150349 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f27e4103cf0 0x7f27e4199830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:26:28.669 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:28.669+0000 7f27ea2b8640 1 --2- 172.21.15.82:0/4182150349 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f27e4102ab0 0x7f27e41992f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:26:28.669 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:28.669+0000 7f27ea2b8640 1 -- 172.21.15.82:0/4182150349 >> 172.21.15.82:0/4182150349 conn(0x7f27e40fdcc0 msgr2=0x7f27e41049d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:26:28.669 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:28.669+0000 7f27ea2b8640 1 -- 172.21.15.82:0/4182150349 shutdown_connections 2024-05-20T16:26:28.669 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:28.669+0000 7f27ea2b8640 1 -- 172.21.15.82:0/4182150349 wait complete. 2024-05-20T16:26:28.687 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-20T16:26:28.868 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:28.869+0000 7f995a268640 1 -- 172.21.15.82:0/3370801149 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f994c0a5280 msgr2=0x7f994c0a7250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:26:28.868 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:28.869+0000 7f995a268640 1 --2- 172.21.15.82:0/3370801149 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f994c0a5280 0x7f994c0a7250 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f994400fb60 tx=0x7f994402e4c0 comp rx=0 tx=0).stop 2024-05-20T16:26:28.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:28.870+0000 7f995a268640 1 -- 172.21.15.82:0/3370801149 shutdown_connections 2024-05-20T16:26:28.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:28.870+0000 7f995a268640 1 --2- 172.21.15.82:0/3370801149 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f994c0a5280 0x7f994c0a7250 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:26:28.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:28.870+0000 7f995a268640 1 --2- 172.21.15.82:0/3370801149 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f994c0a5690 0x7f994c0a77d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:26:28.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:28.870+0000 7f995a268640 1 --2- 172.21.15.82:0/3370801149 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f994c0a63e0 0x7f994c0a6d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:26:28.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:28.870+0000 7f995a268640 1 -- 172.21.15.82:0/3370801149 >> 172.21.15.82:0/3370801149 conn(0x7f994c0a0980 msgr2=0x7f994c0a2e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:26:28.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:28.875+0000 7f995a268640 1 -- 172.21.15.82:0/3370801149 shutdown_connections 2024-05-20T16:26:28.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:28.875+0000 7f995a268640 1 -- 172.21.15.82:0/3370801149 wait complete. 2024-05-20T16:26:28.877 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:28.876+0000 7f995a268640 1 Processor -- start 2024-05-20T16:26:28.878 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:28.879+0000 7f995a268640 1 -- start start 2024-05-20T16:26:28.879 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:28.879+0000 7f995a268640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f994c0a5280 0x7f994c13be90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:26:28.879 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:28.879+0000 7f995a268640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f994c0a5690 0x7f994c13c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:26:28.879 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:28.879+0000 7f995a268640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f994c0a63e0 0x7f994c143360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:26:28.879 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:28.879+0000 7f995a268640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f994c0b1550 con 0x7f994c0a5280 2024-05-20T16:26:28.879 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:28.879+0000 7f995a268640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f994c0b13d0 con 0x7f994c0a5690 2024-05-20T16:26:28.879 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:28.879+0000 7f995a268640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f994c0b16d0 con 0x7f994c0a63e0 2024-05-20T16:26:28.879 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:28.879+0000 7f9959266640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f994c0a5280 0x7f994c13be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:26:28.880 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:28.879+0000 7f9959266640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f994c0a5280 0x7f994c13be90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:45904/0 (socket says 172.21.15.82:45904) 2024-05-20T16:26:28.880 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:28.879+0000 7f9959266640 1 -- 172.21.15.82:0/4152964219 learned_addr learned my addr 172.21.15.82:0/4152964219 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:26:28.880 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:28.880+0000 7f9959266640 1 -- 172.21.15.82:0/4152964219 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f994c0a63e0 msgr2=0x7f994c143360 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-20T16:26:28.880 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:28.880+0000 7f9959266640 1 --2- 172.21.15.82:0/4152964219 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f994c0a63e0 0x7f994c143360 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:26:28.880 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:28.880+0000 7f9959266640 1 -- 172.21.15.82:0/4152964219 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f994c0a5690 msgr2=0x7f994c13c3d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-20T16:26:28.880 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:28.880+0000 7f9959266640 1 --2- 172.21.15.82:0/4152964219 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f994c0a5690 0x7f994c13c3d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:26:28.880 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:28.880+0000 7f9959266640 1 -- 172.21.15.82:0/4152964219 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f994c1439d0 con 0x7f994c0a5280 2024-05-20T16:26:28.881 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:28.880+0000 7f9959266640 1 --2- 172.21.15.82:0/4152964219 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f994c0a5280 0x7f994c13be90 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f995000e9f0 tx=0x7f995000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:26:28.881 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:28.881+0000 7f994a7fc640 1 -- 172.21.15.82:0/4152964219 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f995000ccc0 con 0x7f994c0a5280 2024-05-20T16:26:28.881 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:28.881+0000 7f994a7fc640 1 -- 172.21.15.82:0/4152964219 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9950010910 con 0x7f994c0a5280 2024-05-20T16:26:28.881 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:28.881+0000 7f994a7fc640 1 -- 172.21.15.82:0/4152964219 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99500188a0 con 0x7f994c0a5280 2024-05-20T16:26:28.881 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:28.881+0000 7f995a268640 1 -- 172.21.15.82:0/4152964219 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f994c143cc0 con 0x7f994c0a5280 2024-05-20T16:26:28.881 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:28.881+0000 7f995a268640 1 -- 172.21.15.82:0/4152964219 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f994c1441c0 con 0x7f994c0a5280 2024-05-20T16:26:28.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:28.886+0000 7f994a7fc640 1 -- 172.21.15.82:0/4152964219 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f9950010ac0 con 0x7f994c0a5280 2024-05-20T16:26:28.886 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:28.886+0000 7f995a268640 1 -- 172.21.15.82:0/4152964219 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9938005290 con 0x7f994c0a5280 2024-05-20T16:26:28.886 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:28.887+0000 7f994a7fc640 1 --2- 172.21.15.82:0/4152964219 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f99400a2d90 0x7f99400a52a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:26:28.887 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:28.888+0000 7f994a7fc640 1 -- 172.21.15.82:0/4152964219 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9150+0+0 (secure 0 0 0) 0x7f99500cf6e0 con 0x7f994c0a5280 2024-05-20T16:26:28.889 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:28.890+0000 7f9958a65640 1 --2- 172.21.15.82:0/4152964219 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f99400a2d90 0x7f99400a52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:26:28.894 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:28.892+0000 7f9958a65640 1 --2- 172.21.15.82:0/4152964219 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f99400a2d90 0x7f99400a52a0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f9944005eb0 tx=0x7f994400f7e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:26:28.900 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:28.901+0000 7f994a7fc640 1 -- 172.21.15.82:0/4152964219 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f995009c250 con 0x7f994c0a5280 2024-05-20T16:26:29.246 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:29.246+0000 7f995a268640 1 -- 172.21.15.82:0/4152964219 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9938005580 con 0x7f994c0a5280 2024-05-20T16:26:29.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:29.249+0000 7f994a7fc640 1 -- 172.21.15.82:0/4152964219 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v98) ==== 74+0+24771 (secure 0 0 0) 0x7f995009bb50 con 0x7f994c0a5280 2024-05-20T16:26:29.249 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-05-20T16:26:29.249 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":98,"fsid":"59d1600a-16c2-11ef-bc9a-c7b262605968","created":"2024-05-20T16:03:46.611556+0000","modified":"2024-05-20T16:25:03.536196+0000","last_up_change":"2024-05-20T16:10:59.303479+0000","last_in_change":"2024-05-20T16:10:39.063561+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-20T16:07:09.493727+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-20T16:11:45.058035+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-20T16:11:45.943862+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-20T16:11:49.637579+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"bae842a9-03ec-490e-b56d-f7957235cdc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":1484330167},{"type":"v1","addr":"172.21.15.82:6803","nonce":1484330167}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":1484330167},{"type":"v1","addr":"172.21.15.82:6805","nonce":1484330167}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1484330167},{"type":"v1","addr":"172.21.15.82:6809","nonce":1484330167}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":1484330167},{"type":"v1","addr":"172.21.15.82:6807","nonce":1484330167}]},"public_addr":"172.21.15.82:6803/1484330167","cluster_addr":"172.21.15.82:6805/1484330167","heartbeat_back_addr":"172.21.15.82:6809/1484330167","heartbeat_front_addr":"172.21.15.82:6807/1484330167","state":["exists","up"]},{"osd":1,"uuid":"59f6a2ba-b5de-4161-81c5-92a417abacdf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":1457003568},{"type":"v1","addr":"172.21.15.160:6801","nonce":1457003568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":1457003568},{"type":"v1","addr":"172.21.15.160:6803","nonce":1457003568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6806","nonce":1457003568},{"type":"v1","addr":"172.21.15.160:6807","nonce":1457003568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":1457003568},{"type":"v1","addr":"172.21.15.160:6805","nonce":1457003568}]},"public_addr":"172.21.15.160:6801/1457003568","cluster_addr":"172.21.15.160:6803/1457003568","heartbeat_back_addr":"172.21.15.160:6807/1457003568","heartbeat_front_addr":"172.21.15.160:6805/1457003568","state":["exists","up"]},{"osd":2,"uuid":"3bbb5587-b449-41f3-bd26-67ec3d6afdfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":4051293934},{"type":"v1","addr":"172.21.15.184:6801","nonce":4051293934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":4051293934},{"type":"v1","addr":"172.21.15.184:6803","nonce":4051293934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":4051293934},{"type":"v1","addr":"172.21.15.184:6807","nonce":4051293934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":4051293934},{"type":"v1","addr":"172.21.15.184:6805","nonce":4051293934}]},"public_addr":"172.21.15.184:6801/4051293934","cluster_addr":"172.21.15.184:6803/4051293934","heartbeat_back_addr":"172.21.15.184:6807/4051293934","heartbeat_front_addr":"172.21.15.184:6805/4051293934","state":["exists","up"]},{"osd":3,"uuid":"ca651f90-09b8-4081-a6f9-40b31268a837","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":3947485599},{"type":"v1","addr":"172.21.15.82:6811","nonce":3947485599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":3947485599},{"type":"v1","addr":"172.21.15.82:6813","nonce":3947485599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":3947485599},{"type":"v1","addr":"172.21.15.82:6817","nonce":3947485599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":3947485599},{"type":"v1","addr":"172.21.15.82:6815","nonce":3947485599}]},"public_addr":"172.21.15.82:6811/3947485599","cluster_addr":"172.21.15.82:6813/3947485599","heartbeat_back_addr":"172.21.15.82:6817/3947485599","heartbeat_front_addr":"172.21.15.82:6815/3947485599","state":["exists","up"]},{"osd":4,"uuid":"cb3dfd56-f677-40cb-9c96-8319ea5ac788","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":4160830014},{"type":"v1","addr":"172.21.15.160:6809","nonce":4160830014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":4160830014},{"type":"v1","addr":"172.21.15.160:6811","nonce":4160830014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":4160830014},{"type":"v1","addr":"172.21.15.160:6815","nonce":4160830014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6812","nonce":4160830014},{"type":"v1","addr":"172.21.15.160:6813","nonce":4160830014}]},"public_addr":"172.21.15.160:6809/4160830014","cluster_addr":"172.21.15.160:6811/4160830014","heartbeat_back_addr":"172.21.15.160:6815/4160830014","heartbeat_front_addr":"172.21.15.160:6813/4160830014","state":["exists","up"]},{"osd":5,"uuid":"c70cfcd1-a4ff-461f-8c73-479b061200fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":1046282672},{"type":"v1","addr":"172.21.15.184:6809","nonce":1046282672}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":1046282672},{"type":"v1","addr":"172.21.15.184:6811","nonce":1046282672}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":1046282672},{"type":"v1","addr":"172.21.15.184:6815","nonce":1046282672}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":1046282672},{"type":"v1","addr":"172.21.15.184:6813","nonce":1046282672}]},"public_addr":"172.21.15.184:6809/1046282672","cluster_addr":"172.21.15.184:6811/1046282672","heartbeat_back_addr":"172.21.15.184:6815/1046282672","heartbeat_front_addr":"172.21.15.184:6813/1046282672","state":["exists","up"]},{"osd":6,"uuid":"d47ce001-9dd1-4830-bcc9-888601187c6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":2789736499},{"type":"v1","addr":"172.21.15.82:6819","nonce":2789736499}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":2789736499},{"type":"v1","addr":"172.21.15.82:6821","nonce":2789736499}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":2789736499},{"type":"v1","addr":"172.21.15.82:6825","nonce":2789736499}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":2789736499},{"type":"v1","addr":"172.21.15.82:6823","nonce":2789736499}]},"public_addr":"172.21.15.82:6819/2789736499","cluster_addr":"172.21.15.82:6821/2789736499","heartbeat_back_addr":"172.21.15.82:6825/2789736499","heartbeat_front_addr":"172.21.15.82:6823/2789736499","state":["exists","up"]},{"osd":7,"uuid":"25c54f43-18cb-41e4-83b4-2732b5a93889","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6816","nonce":2340683378},{"type":"v1","addr":"172.21.15.160:6817","nonce":2340683378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":2340683378},{"type":"v1","addr":"172.21.15.160:6819","nonce":2340683378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":2340683378},{"type":"v1","addr":"172.21.15.160:6823","nonce":2340683378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6820","nonce":2340683378},{"type":"v1","addr":"172.21.15.160:6821","nonce":2340683378}]},"public_addr":"172.21.15.160:6817/2340683378","cluster_addr":"172.21.15.160:6819/2340683378","heartbeat_back_addr":"172.21.15.160:6823/2340683378","heartbeat_front_addr":"172.21.15.160:6821/2340683378","state":["exists","up"]},{"osd":8,"uuid":"1f8b478d-287f-45f5-8526-d1a58c5bc07c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":3831576372},{"type":"v1","addr":"172.21.15.184:6817","nonce":3831576372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":3831576372},{"type":"v1","addr":"172.21.15.184:6819","nonce":3831576372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":3831576372},{"type":"v1","addr":"172.21.15.184:6823","nonce":3831576372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":3831576372},{"type":"v1","addr":"172.21.15.184:6821","nonce":3831576372}]},"public_addr":"172.21.15.184:6817/3831576372","cluster_addr":"172.21.15.184:6819/3831576372","heartbeat_back_addr":"172.21.15.184:6823/3831576372","heartbeat_front_addr":"172.21.15.184:6821/3831576372","state":["exists","up"]},{"osd":9,"uuid":"0d69e319-954f-4b18-95ea-31794f18dc3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":716330782},{"type":"v1","addr":"172.21.15.82:6827","nonce":716330782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":716330782},{"type":"v1","addr":"172.21.15.82:6829","nonce":716330782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":716330782},{"type":"v1","addr":"172.21.15.82:6833","nonce":716330782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":716330782},{"type":"v1","addr":"172.21.15.82:6831","nonce":716330782}]},"public_addr":"172.21.15.82:6827/716330782","cluster_addr":"172.21.15.82:6829/716330782","heartbeat_back_addr":"172.21.15.82:6833/716330782","heartbeat_front_addr":"172.21.15.82:6831/716330782","state":["exists","up"]},{"osd":10,"uuid":"39dc1b78-61cb-42ca-b5b6-1e44ac2b6500","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":2327428014},{"type":"v1","addr":"172.21.15.160:6825","nonce":2327428014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":2327428014},{"type":"v1","addr":"172.21.15.160:6827","nonce":2327428014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":2327428014},{"type":"v1","addr":"172.21.15.160:6831","nonce":2327428014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":2327428014},{"type":"v1","addr":"172.21.15.160:6829","nonce":2327428014}]},"public_addr":"172.21.15.160:6825/2327428014","cluster_addr":"172.21.15.160:6827/2327428014","heartbeat_back_addr":"172.21.15.160:6831/2327428014","heartbeat_front_addr":"172.21.15.160:6829/2327428014","state":["exists","up"]},{"osd":11,"uuid":"4306b3c1-707a-4010-987c-209ea0e948e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":3594364650},{"type":"v1","addr":"172.21.15.184:6825","nonce":3594364650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":3594364650},{"type":"v1","addr":"172.21.15.184:6827","nonce":3594364650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3594364650},{"type":"v1","addr":"172.21.15.184:6831","nonce":3594364650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3594364650},{"type":"v1","addr":"172.21.15.184:6829","nonce":3594364650}]},"public_addr":"172.21.15.184:6825/3594364650","cluster_addr":"172.21.15.184:6827/3594364650","heartbeat_back_addr":"172.21.15.184:6831/3594364650","heartbeat_front_addr":"172.21.15.184:6829/3594364650","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:06:17.855998+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:06:41.473270+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:07:05.304108+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:07:29.887472+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:07:55.387286+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:08:20.579669+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:08:46.333051+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:09:12.289668+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:09:38.180696+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:10:04.471459+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:10:31.041912+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T16:10:57.477266+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.1d","mappings":[{"from":5,"to":9}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.5","mappings":[{"from":9,"to":10}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.82:0/2005424509":"2024-05-21T16:04:45.161949+0000","172.21.15.82:6801/1763049308":"2024-05-21T16:04:45.161949+0000","172.21.15.82:6801/1959459348":"2024-05-21T16:04:19.208291+0000","172.21.15.82:0/3131904556":"2024-05-21T16:04:19.208291+0000","172.21.15.82:6800/1959459348":"2024-05-21T16:04:19.208291+0000","172.21.15.82:0/2456966342":"2024-05-21T16:04:45.161949+0000","172.21.15.82:0/4179978833":"2024-05-21T16:04:19.208291+0000","172.21.15.82:0/1602862225":"2024-05-21T16:04:45.161949+0000","172.21.15.82:0/106101180":"2024-05-21T16:04:19.208291+0000","172.21.15.82:6800/1763049308":"2024-05-21T16:04:45.161949+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-20T16:26:29.252 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:29.253+0000 7f995a268640 1 -- 172.21.15.82:0/4152964219 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f99400a2d90 msgr2=0x7f99400a52a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:26:29.253 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:29.253+0000 7f995a268640 1 --2- 172.21.15.82:0/4152964219 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f99400a2d90 0x7f99400a52a0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f9944005eb0 tx=0x7f994400f7e0 comp rx=0 tx=0).stop 2024-05-20T16:26:29.253 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:29.253+0000 7f995a268640 1 -- 172.21.15.82:0/4152964219 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f994c0a5280 msgr2=0x7f994c13be90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:26:29.253 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:29.253+0000 7f995a268640 1 --2- 172.21.15.82:0/4152964219 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f994c0a5280 0x7f994c13be90 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f995000e9f0 tx=0x7f995000eeb0 comp rx=0 tx=0).stop 2024-05-20T16:26:29.253 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:29.253+0000 7f995a268640 1 -- 172.21.15.82:0/4152964219 shutdown_connections 2024-05-20T16:26:29.253 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:29.253+0000 7f995a268640 1 --2- 172.21.15.82:0/4152964219 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f99400a2d90 0x7f99400a52a0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:26:29.253 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:29.253+0000 7f995a268640 1 --2- 172.21.15.82:0/4152964219 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f994c0a63e0 0x7f994c143360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:26:29.253 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:29.253+0000 7f995a268640 1 --2- 172.21.15.82:0/4152964219 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f994c0a5690 0x7f994c13c3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:26:29.253 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:29.253+0000 7f995a268640 1 --2- 172.21.15.82:0/4152964219 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f994c0a5280 0x7f994c13be90 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:26:29.253 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:29.253+0000 7f995a268640 1 -- 172.21.15.82:0/4152964219 >> 172.21.15.82:0/4152964219 conn(0x7f994c0a0980 msgr2=0x7f994c0ab030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:26:29.253 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:29.253+0000 7f995a268640 1 -- 172.21.15.82:0/4152964219 shutdown_connections 2024-05-20T16:26:29.253 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:29.253+0000 7f995a268640 1 -- 172.21.15.82:0/4152964219 wait complete. 2024-05-20T16:26:29.269 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:26:28 smithi082 ceph-mon[32238]: pgmap v739: 129 pgs: 129 active+clean; 8.0 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 78 MiB/s wr, 562 op/s 2024-05-20T16:26:29.270 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:26:28 smithi082 ceph-mon[32238]: from='client.? 172.21.15.82:0/2300391398' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T16:26:29.270 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:26:28 smithi082 ceph-mon[32238]: from='client.? 172.21.15.82:0/4182150349' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T16:26:29.271 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-05-20T16:26:29.311 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:26:28 smithi160 ceph-mon[36347]: pgmap v739: 129 pgs: 129 active+clean; 8.0 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 78 MiB/s wr, 562 op/s 2024-05-20T16:26:29.312 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:26:28 smithi160 ceph-mon[36347]: from='client.? 172.21.15.82:0/2300391398' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T16:26:29.312 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:26:28 smithi160 ceph-mon[36347]: from='client.? 172.21.15.82:0/4182150349' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T16:26:29.385 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:26:28 smithi184 ceph-mon[36209]: pgmap v739: 129 pgs: 129 active+clean; 8.0 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 78 MiB/s wr, 562 op/s 2024-05-20T16:26:29.385 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:26:28 smithi184 ceph-mon[36209]: from='client.? 172.21.15.82:0/2300391398' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T16:26:29.385 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:26:28 smithi184 ceph-mon[36209]: from='client.? 172.21.15.82:0/4182150349' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-20T16:26:29.460 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:29.460+0000 7f827071a640 1 -- 172.21.15.82:0/2395984596 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8268106640 msgr2=0x7f8268106a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:26:29.461 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:29.460+0000 7f827071a640 1 --2- 172.21.15.82:0/2395984596 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8268106640 0x7f8268106a50 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f826001acb0 tx=0x7f826003b920 comp rx=0 tx=0).stop 2024-05-20T16:26:29.461 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:29.461+0000 7f827071a640 1 -- 172.21.15.82:0/2395984596 shutdown_connections 2024-05-20T16:26:29.461 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:29.461+0000 7f827071a640 1 --2- 172.21.15.82:0/2395984596 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8268106640 0x7f8268106a50 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:26:29.461 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:29.461+0000 7f827071a640 1 --2- 172.21.15.82:0/2395984596 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8268103ce0 0x7f8268106100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:26:29.461 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:29.461+0000 7f827071a640 1 --2- 172.21.15.82:0/2395984596 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8268101380 0x7f82681037a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:26:29.461 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:29.461+0000 7f827071a640 1 -- 172.21.15.82:0/2395984596 >> 172.21.15.82:0/2395984596 conn(0x7f82680fcbb0 msgr2=0x7f82680ff040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:26:29.465 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:29.465+0000 7f827071a640 1 -- 172.21.15.82:0/2395984596 shutdown_connections 2024-05-20T16:26:29.465 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:29.466+0000 7f827071a640 1 -- 172.21.15.82:0/2395984596 wait complete. 2024-05-20T16:26:29.466 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:29.467+0000 7f827071a640 1 Processor -- start 2024-05-20T16:26:29.467 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:29.468+0000 7f827071a640 1 -- start start 2024-05-20T16:26:29.467 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:29.468+0000 7f827071a640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8268101380 0x7f8268197120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:26:29.468 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:29.468+0000 7f827071a640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8268103ce0 0x7f8268197660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:26:29.468 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:29.468+0000 7f827071a640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8268106640 0x7f826819e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:26:29.468 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:29.468+0000 7f827071a640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f826810a140 con 0x7f8268106640 2024-05-20T16:26:29.468 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:29.468+0000 7f827071a640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f8268109fc0 con 0x7f8268103ce0 2024-05-20T16:26:29.468 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:29.468+0000 7f827071a640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f826810a2c0 con 0x7f8268101380 2024-05-20T16:26:29.468 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:29.468+0000 7f826e4b6640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8268101380 0x7f8268197120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:26:29.468 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:29.469+0000 7f826e4b6640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8268101380 0x7f8268197120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.82:47014/0 (socket says 172.21.15.82:47014) 2024-05-20T16:26:29.468 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:29.469+0000 7f826e4b6640 1 -- 172.21.15.82:0/3509613272 learned_addr learned my addr 172.21.15.82:0/3509613272 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:26:29.472 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:29.471+0000 7f826e4b6640 1 -- 172.21.15.82:0/3509613272 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8268103ce0 msgr2=0x7f8268197660 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-20T16:26:29.472 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:29.471+0000 7f826e4b6640 1 --2- 172.21.15.82:0/3509613272 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8268103ce0 0x7f8268197660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:26:29.472 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:29.471+0000 7f826e4b6640 1 -- 172.21.15.82:0/3509613272 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8268106640 msgr2=0x7f826819e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:26:29.472 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:29.471+0000 7f826e4b6640 1 --2- 172.21.15.82:0/3509613272 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8268106640 0x7f826819e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:26:29.472 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:29.471+0000 7f826e4b6640 1 -- 172.21.15.82:0/3509613272 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f826819eb90 con 0x7f8268101380 2024-05-20T16:26:29.474 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:29.475+0000 7f826e4b6640 1 --2- 172.21.15.82:0/3509613272 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8268101380 0x7f8268197120 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f826400b470 tx=0x7f826400b930 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:26:29.478 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:29.479+0000 7f825f7fe640 1 -- 172.21.15.82:0/3509613272 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8264015020 con 0x7f8268101380 2024-05-20T16:26:29.478 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:29.479+0000 7f825f7fe640 1 -- 172.21.15.82:0/3509613272 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8264011d70 con 0x7f8268101380 2024-05-20T16:26:29.479 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:29.479+0000 7f827071a640 1 -- 172.21.15.82:0/3509613272 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f826819ee80 con 0x7f8268101380 2024-05-20T16:26:29.479 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:29.479+0000 7f827071a640 1 -- 172.21.15.82:0/3509613272 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f826819f360 con 0x7f8268101380 2024-05-20T16:26:29.479 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:29.479+0000 7f825f7fe640 1 -- 172.21.15.82:0/3509613272 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f826400f340 con 0x7f8268101380 2024-05-20T16:26:29.481 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:29.481+0000 7f825d7fa640 1 -- 172.21.15.82:0/3509613272 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8268062750 con 0x7f8268101380 2024-05-20T16:26:29.483 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:29.484+0000 7f825f7fe640 1 -- 172.21.15.82:0/3509613272 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f8264018020 con 0x7f8268101380 2024-05-20T16:26:29.484 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:29.485+0000 7f825f7fe640 1 --2- 172.21.15.82:0/3509613272 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f82540a2d60 0x7f82540a5270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:26:29.484 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:29.485+0000 7f825f7fe640 1 -- 172.21.15.82:0/3509613272 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9150+0+0 (secure 0 0 0) 0x7f82640ceef0 con 0x7f8268101380 2024-05-20T16:26:29.491 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:29.492+0000 7f826dcb5640 1 --2- 172.21.15.82:0/3509613272 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f82540a2d60 0x7f82540a5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:26:29.493 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:29.493+0000 7f825f7fe640 1 -- 172.21.15.82:0/3509613272 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f826401f020 con 0x7f8268101380 2024-05-20T16:26:29.496 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:29.496+0000 7f826dcb5640 1 --2- 172.21.15.82:0/3509613272 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f82540a2d60 0x7f82540a5270 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f8268198190 tx=0x7f825800b370 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:26:29.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:29.915+0000 7f825d7fa640 1 -- 172.21.15.82:0/3509613272 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f8268062920 con 0x7f8268101380 2024-05-20T16:26:29.928 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:29.928+0000 7f825f7fe640 1 -- 172.21.15.82:0/3509613272 <== mon.1 v2:172.21.15.184:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v32) ==== 161+0+0 (secure 0 0 0) 0x7f826400d040 con 0x7f8268101380 2024-05-20T16:26:29.928 INFO:teuthology.orchestra.run.smithi082.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-20T16:26:29.931 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:29.931+0000 7f825d7fa640 1 -- 172.21.15.82:0/3509613272 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f82540a2d60 msgr2=0x7f82540a5270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:26:29.931 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:29.931+0000 7f825d7fa640 1 --2- 172.21.15.82:0/3509613272 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f82540a2d60 0x7f82540a5270 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f8268198190 tx=0x7f825800b370 comp rx=0 tx=0).stop 2024-05-20T16:26:29.931 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:29.932+0000 7f825d7fa640 1 -- 172.21.15.82:0/3509613272 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8268101380 msgr2=0x7f8268197120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:26:29.931 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:29.932+0000 7f825d7fa640 1 --2- 172.21.15.82:0/3509613272 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8268101380 0x7f8268197120 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f826400b470 tx=0x7f826400b930 comp rx=0 tx=0).stop 2024-05-20T16:26:29.932 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:29.932+0000 7f825d7fa640 1 -- 172.21.15.82:0/3509613272 shutdown_connections 2024-05-20T16:26:29.932 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:29.932+0000 7f825d7fa640 1 --2- 172.21.15.82:0/3509613272 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f82540a2d60 0x7f82540a5270 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:26:29.932 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:29.932+0000 7f825d7fa640 1 --2- 172.21.15.82:0/3509613272 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8268106640 0x7f826819e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:26:29.932 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:29.932+0000 7f825d7fa640 1 --2- 172.21.15.82:0/3509613272 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8268103ce0 0x7f8268197660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:26:29.932 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:29.932+0000 7f825d7fa640 1 --2- 172.21.15.82:0/3509613272 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8268101380 0x7f8268197120 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:26:29.932 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:29.932+0000 7f825d7fa640 1 -- 172.21.15.82:0/3509613272 >> 172.21.15.82:0/3509613272 conn(0x7f82680fcbb0 msgr2=0x7f82680fefd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:26:29.932 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:29.932+0000 7f825d7fa640 1 -- 172.21.15.82:0/3509613272 shutdown_connections 2024-05-20T16:26:29.932 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:29.932+0000 7f825d7fa640 1 -- 172.21.15.82:0/3509613272 wait complete. 2024-05-20T16:26:29.950 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-05-20T16:26:30.072 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:30.072+0000 7fb69d765640 1 -- 172.21.15.82:0/700379242 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb698103cd0 msgr2=0x7fb6981040e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:26:30.073 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:30.072+0000 7fb69d765640 1 --2- 172.21.15.82:0/700379242 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb698103cd0 0x7fb6981040e0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fb68c009ca0 tx=0x7fb68c02a8c0 comp rx=0 tx=0).stop 2024-05-20T16:26:30.073 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:30.073+0000 7fb69d765640 1 -- 172.21.15.82:0/700379242 shutdown_connections 2024-05-20T16:26:30.073 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:30.073+0000 7fb69d765640 1 --2- 172.21.15.82:0/700379242 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb698103cd0 0x7fb6981040e0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:26:30.073 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:30.073+0000 7fb69d765640 1 --2- 172.21.15.82:0/700379242 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb6981089d0 0x7fb69810adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:26:30.073 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:30.073+0000 7fb69d765640 1 --2- 172.21.15.82:0/700379242 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb698102a90 0x7fb698103790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:26:30.073 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:30.073+0000 7fb69d765640 1 -- 172.21.15.82:0/700379242 >> 172.21.15.82:0/700379242 conn(0x7fb6980fdcc0 msgr2=0x7fb698100130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:26:30.073 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:30.073+0000 7fb69d765640 1 -- 172.21.15.82:0/700379242 shutdown_connections 2024-05-20T16:26:30.073 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:30.073+0000 7fb69d765640 1 -- 172.21.15.82:0/700379242 wait complete. 2024-05-20T16:26:30.073 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:30.073+0000 7fb69d765640 1 Processor -- start 2024-05-20T16:26:30.075 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:30.073+0000 7fb69d765640 1 -- start start 2024-05-20T16:26:30.075 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:30.074+0000 7fb69d765640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb698102a90 0x7fb6981992a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:26:30.076 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:30.074+0000 7fb69d765640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb698103cd0 0x7fb6981997e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:26:30.076 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:30.074+0000 7fb69d765640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb6981089d0 0x7fb6981a0770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:26:30.076 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:30.074+0000 7fb69d765640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fb69810e730 con 0x7fb6981089d0 2024-05-20T16:26:30.076 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:30.074+0000 7fb69d765640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fb69810e5b0 con 0x7fb698102a90 2024-05-20T16:26:30.076 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:30.074+0000 7fb69d765640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fb69810e8b0 con 0x7fb698103cd0 2024-05-20T16:26:30.076 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:30.074+0000 7fb6977fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb6981089d0 0x7fb6981a0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:26:30.076 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:30.074+0000 7fb6977fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb6981089d0 0x7fb6981a0770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:49586/0 (socket says 172.21.15.82:49586) 2024-05-20T16:26:30.076 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:30.074+0000 7fb6977fe640 1 -- 172.21.15.82:0/2644945321 learned_addr learned my addr 172.21.15.82:0/2644945321 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:26:30.076 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:30.074+0000 7fb6967fc640 1 --2- 172.21.15.82:0/2644945321 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb698103cd0 0x7fb6981997e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:26:30.076 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:30.074+0000 7fb696ffd640 1 --2- 172.21.15.82:0/2644945321 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb698102a90 0x7fb6981992a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:26:30.076 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:30.074+0000 7fb6977fe640 1 -- 172.21.15.82:0/2644945321 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb698103cd0 msgr2=0x7fb6981997e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:26:30.076 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:30.074+0000 7fb6977fe640 1 --2- 172.21.15.82:0/2644945321 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb698103cd0 0x7fb6981997e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:26:30.076 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:30.074+0000 7fb6977fe640 1 -- 172.21.15.82:0/2644945321 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb698102a90 msgr2=0x7fb6981992a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:26:30.076 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:30.074+0000 7fb6977fe640 1 --2- 172.21.15.82:0/2644945321 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb698102a90 0x7fb6981992a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:26:30.077 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:30.074+0000 7fb6977fe640 1 -- 172.21.15.82:0/2644945321 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6981a0d30 con 0x7fb6981089d0 2024-05-20T16:26:30.077 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:30.074+0000 7fb6977fe640 1 --2- 172.21.15.82:0/2644945321 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb6981089d0 0x7fb6981a0770 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fb69000c970 tx=0x7fb69000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:26:30.077 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:30.074+0000 7fb67ffff640 1 -- 172.21.15.82:0/2644945321 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb690007bb0 con 0x7fb6981089d0 2024-05-20T16:26:30.077 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:30.074+0000 7fb67ffff640 1 -- 172.21.15.82:0/2644945321 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb690007d50 con 0x7fb6981089d0 2024-05-20T16:26:30.077 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:30.074+0000 7fb69d765640 1 -- 172.21.15.82:0/2644945321 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6981a0fc0 con 0x7fb6981089d0 2024-05-20T16:26:30.077 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:30.074+0000 7fb67ffff640 1 -- 172.21.15.82:0/2644945321 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb690018800 con 0x7fb6981089d0 2024-05-20T16:26:30.077 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:30.074+0000 7fb69d765640 1 -- 172.21.15.82:0/2644945321 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6981a1430 con 0x7fb6981089d0 2024-05-20T16:26:30.077 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:30.076+0000 7fb69d765640 1 -- 172.21.15.82:0/2644945321 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6981077e0 con 0x7fb6981089d0 2024-05-20T16:26:30.078 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:30.077+0000 7fb67ffff640 1 -- 172.21.15.82:0/2644945321 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fb69000fa50 con 0x7fb6981089d0 2024-05-20T16:26:30.078 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:30.077+0000 7fb67ffff640 1 --2- 172.21.15.82:0/2644945321 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fb6800a2b90 0x7fb6800a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:26:30.078 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:30.078+0000 7fb696ffd640 1 --2- 172.21.15.82:0/2644945321 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fb6800a2b90 0x7fb6800a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:26:30.078 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:30.078+0000 7fb67ffff640 1 -- 172.21.15.82:0/2644945321 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9408+0+0 (secure 0 0 0) 0x7fb6900cf470 con 0x7fb6981089d0 2024-05-20T16:26:30.078 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:30.078+0000 7fb696ffd640 1 --2- 172.21.15.82:0/2644945321 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fb6800a2b90 0x7fb6800a50a0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fb698103690 tx=0x7fb688005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:26:30.081 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:30.082+0000 7fb67ffff640 1 -- 172.21.15.82:0/2644945321 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fb690014030 con 0x7fb6981089d0 2024-05-20T16:26:30.296 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:30.296+0000 7fb69d765640 1 -- 172.21.15.82:0/2644945321 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fb698062750 con 0x7fb6981089d0 2024-05-20T16:26:30.312 INFO:journalctl@ceph.mds.e.smithi160.stdout:May 20 16:26:30 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-e[66323]: ignoring --setuser ceph since I am not root 2024-05-20T16:26:30.312 INFO:journalctl@ceph.mds.e.smithi160.stdout:May 20 16:26:30 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-e[66323]: ignoring --setgroup ceph since I am not root 2024-05-20T16:26:30.312 INFO:journalctl@ceph.mds.e.smithi160.stdout:May 20 16:26:30 smithi160 ceph-mds[66327]: -- 172.21.15.160:0/1730311710 <== mon.2 v2:172.21.15.160:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5557c0da9a40 con 0x5557c0e21180 2024-05-20T16:26:30.312 INFO:journalctl@ceph.mds.e.smithi160.stdout:May 20 16:26:30 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-e[66323]: starting mds.e at 2024-05-20T16:26:30.312 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:26:29 smithi160 ceph-mon[36347]: from='client.? 172.21.15.82:0/4152964219' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-20T16:26:30.312 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:26:29 smithi160 ceph-mon[36347]: from='client.? 172.21.15.82:0/3509613272' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-05-20T16:26:30.312 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:26:29 smithi160 ceph-mon[36347]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-05-20T16:26:30.312 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:26:29 smithi160 ceph-mon[36347]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-05-20T16:26:30.312 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:26:29 smithi160 ceph-mon[36347]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-20T16:26:30.313 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:26:29 smithi160 ceph-mon[36347]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-05-20T16:26:30.313 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:26:29 smithi160 ceph-mon[36347]: osdmap e99: 12 total, 12 up, 12 in 2024-05-20T16:26:30.313 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:26:29 smithi160 ceph-mon[36347]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-05-20T16:26:30.313 INFO:journalctl@ceph.mds.h.smithi160.stdout:May 20 16:26:30 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-h[66640]: ignoring --setuser ceph since I am not root 2024-05-20T16:26:30.313 INFO:journalctl@ceph.mds.h.smithi160.stdout:May 20 16:26:30 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-h[66640]: ignoring --setgroup ceph since I am not root 2024-05-20T16:26:30.313 INFO:journalctl@ceph.mds.h.smithi160.stdout:May 20 16:26:30 smithi160 ceph-mds[66644]: -- 172.21.15.160:0/477203889 <== mon.2 v2:172.21.15.160:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f34075fa40 con 0x55f3407d7180 2024-05-20T16:26:30.313 INFO:journalctl@ceph.mds.h.smithi160.stdout:May 20 16:26:30 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-h[66640]: starting mds.h at 2024-05-20T16:26:30.313 INFO:journalctl@ceph.mds.b.smithi160.stdout:May 20 16:26:30 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-b[66944]: ignoring --setuser ceph since I am not root 2024-05-20T16:26:30.314 INFO:journalctl@ceph.mds.b.smithi160.stdout:May 20 16:26:30 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-b[66944]: ignoring --setgroup ceph since I am not root 2024-05-20T16:26:30.314 INFO:journalctl@ceph.mds.b.smithi160.stdout:May 20 16:26:30 smithi160 ceph-mds[66948]: -- 172.21.15.160:0/1918477774 <== mon.2 v2:172.21.15.160:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x557be4a77a40 con 0x557be4aef600 2024-05-20T16:26:30.314 INFO:journalctl@ceph.mds.b.smithi160.stdout:May 20 16:26:30 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-b[66944]: starting mds.b at 2024-05-20T16:26:30.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:26:29 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mon-a[32234]: 2024-05-20T16:26:29.919+0000 7fbc2199c700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-20T16:26:30.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:26:29 smithi082 ceph-mon[32238]: from='client.? 172.21.15.82:0/4152964219' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-20T16:26:30.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:26:29 smithi082 ceph-mon[32238]: from='client.? 172.21.15.82:0/3509613272' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-05-20T16:26:30.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:26:29 smithi082 ceph-mon[32238]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-05-20T16:26:30.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:26:29 smithi082 ceph-mon[32238]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-05-20T16:26:30.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:26:29 smithi082 ceph-mon[32238]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-20T16:26:30.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:26:29 smithi082 ceph-mon[32238]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-05-20T16:26:30.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:26:29 smithi082 ceph-mon[32238]: osdmap e99: 12 total, 12 up, 12 in 2024-05-20T16:26:30.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:26:29 smithi082 ceph-mon[32238]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-05-20T16:26:30.385 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:26:29 smithi184 ceph-mon[36209]: from='client.? 172.21.15.82:0/4152964219' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-20T16:26:30.385 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:26:29 smithi184 ceph-mon[36209]: from='client.? 172.21.15.82:0/3509613272' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-05-20T16:26:30.385 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:26:29 smithi184 ceph-mon[36209]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-05-20T16:26:30.385 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:26:29 smithi184 ceph-mon[36209]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-05-20T16:26:30.385 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:26:29 smithi184 ceph-mon[36209]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-20T16:26:30.386 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:26:29 smithi184 ceph-mon[36209]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-05-20T16:26:30.386 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:26:29 smithi184 ceph-mon[36209]: osdmap e99: 12 total, 12 up, 12 in 2024-05-20T16:26:30.386 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:26:29 smithi184 ceph-mon[36209]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-05-20T16:26:30.386 INFO:journalctl@ceph.mds.c.smithi184.stdout:May 20 16:26:30 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-c[71243]: ignoring --setuser ceph since I am not root 2024-05-20T16:26:30.386 INFO:journalctl@ceph.mds.c.smithi184.stdout:May 20 16:26:30 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-c[71243]: ignoring --setgroup ceph since I am not root 2024-05-20T16:26:30.386 INFO:journalctl@ceph.mds.c.smithi184.stdout:May 20 16:26:30 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-c[71243]: starting mds.c at 2024-05-20T16:26:30.386 INFO:journalctl@ceph.mds.f.smithi184.stdout:May 20 16:26:30 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-f[67090]: ignoring --setuser ceph since I am not root 2024-05-20T16:26:30.387 INFO:journalctl@ceph.mds.f.smithi184.stdout:May 20 16:26:30 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-f[67090]: ignoring --setgroup ceph since I am not root 2024-05-20T16:26:30.387 INFO:journalctl@ceph.mds.f.smithi184.stdout:May 20 16:26:30 smithi184 ceph-mds[67094]: -- 172.21.15.184:0/3083462920 <== mon.1 v2:172.21.15.184:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x555f3b1e1a40 con 0x555f3b259a80 2024-05-20T16:26:30.387 INFO:journalctl@ceph.mds.f.smithi184.stdout:May 20 16:26:30 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-f[67090]: starting mds.f at 2024-05-20T16:26:30.387 INFO:journalctl@ceph.mds.i.smithi184.stdout:May 20 16:26:30 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-i[67705]: ignoring --setuser ceph since I am not root 2024-05-20T16:26:30.387 INFO:journalctl@ceph.mds.i.smithi184.stdout:May 20 16:26:30 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-i[67705]: ignoring --setgroup ceph since I am not root 2024-05-20T16:26:30.387 INFO:journalctl@ceph.mds.i.smithi184.stdout:May 20 16:26:30 smithi184 ceph-mds[67709]: -- 172.21.15.184:0/3269847835 <== mon.1 v2:172.21.15.184:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5581fc7d5a40 con 0x5581fc84da80 2024-05-20T16:26:30.387 INFO:journalctl@ceph.mds.i.smithi184.stdout:May 20 16:26:30 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-i[67705]: starting mds.i at 2024-05-20T16:26:30.981 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:30.982+0000 7fb67ffff640 1 -- 172.21.15.82:0/2644945321 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v33) ==== 104+0+0 (secure 0 0 0) 0x7fb69009bfe0 con 0x7fb6981089d0 2024-05-20T16:26:30.985 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:30.985+0000 7fb69d765640 1 -- 172.21.15.82:0/2644945321 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fb6800a2b90 msgr2=0x7fb6800a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:26:30.985 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:30.985+0000 7fb69d765640 1 --2- 172.21.15.82:0/2644945321 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fb6800a2b90 0x7fb6800a50a0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fb698103690 tx=0x7fb688005f90 comp rx=0 tx=0).stop 2024-05-20T16:26:30.985 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:30.985+0000 7fb69d765640 1 -- 172.21.15.82:0/2644945321 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb6981089d0 msgr2=0x7fb6981a0770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:26:30.985 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:30.985+0000 7fb69d765640 1 --2- 172.21.15.82:0/2644945321 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb6981089d0 0x7fb6981a0770 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fb69000c970 tx=0x7fb69000ce30 comp rx=0 tx=0).stop 2024-05-20T16:26:30.985 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:30.985+0000 7fb69d765640 1 -- 172.21.15.82:0/2644945321 shutdown_connections 2024-05-20T16:26:30.985 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:30.985+0000 7fb69d765640 1 --2- 172.21.15.82:0/2644945321 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fb6800a2b90 0x7fb6800a50a0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:26:30.985 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:30.985+0000 7fb69d765640 1 --2- 172.21.15.82:0/2644945321 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb6981089d0 0x7fb6981a0770 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:26:30.985 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:30.985+0000 7fb69d765640 1 --2- 172.21.15.82:0/2644945321 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb698103cd0 0x7fb6981997e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:26:30.985 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:30.985+0000 7fb69d765640 1 --2- 172.21.15.82:0/2644945321 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb698102a90 0x7fb6981992a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:26:30.985 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:30.985+0000 7fb69d765640 1 -- 172.21.15.82:0/2644945321 >> 172.21.15.82:0/2644945321 conn(0x7fb6980fdcc0 msgr2=0x7fb6981049b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:26:30.985 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:30.985+0000 7fb69d765640 1 -- 172.21.15.82:0/2644945321 shutdown_connections 2024-05-20T16:26:30.986 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:30.985+0000 7fb69d765640 1 -- 172.21.15.82:0/2644945321 wait complete. 2024-05-20T16:26:31.000 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-05-20T16:26:31.122 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:31.122+0000 7fbeac83d640 1 -- 172.21.15.82:0/422958366 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbea8101130 msgr2=0x7fbea8101e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:26:31.122 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:31.122+0000 7fbeac83d640 1 --2- 172.21.15.82:0/422958366 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbea8101130 0x7fbea8101e30 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fbe9800aa10 tx=0x7fbe9802b720 comp rx=0 tx=0).stop 2024-05-20T16:26:31.122 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:31.122+0000 7fbeac83d640 1 -- 172.21.15.82:0/422958366 shutdown_connections 2024-05-20T16:26:31.123 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:31.122+0000 7fbeac83d640 1 --2- 172.21.15.82:0/422958366 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbea8107070 0x7fbea8109490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:26:31.123 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:31.122+0000 7fbeac83d640 1 --2- 172.21.15.82:0/422958366 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbea8102370 0x7fbea8102780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:26:31.123 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:31.122+0000 7fbeac83d640 1 --2- 172.21.15.82:0/422958366 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbea8101130 0x7fbea8101e30 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:26:31.123 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:31.122+0000 7fbeac83d640 1 -- 172.21.15.82:0/422958366 >> 172.21.15.82:0/422958366 conn(0x7fbea80fc300 msgr2=0x7fbea80fe7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:26:31.123 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:31.122+0000 7fbeac83d640 1 -- 172.21.15.82:0/422958366 shutdown_connections 2024-05-20T16:26:31.123 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:31.122+0000 7fbeac83d640 1 -- 172.21.15.82:0/422958366 wait complete. 2024-05-20T16:26:31.125 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:31.123+0000 7fbeac83d640 1 Processor -- start 2024-05-20T16:26:31.125 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:31.123+0000 7fbeac83d640 1 -- start start 2024-05-20T16:26:31.125 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:31.123+0000 7fbeac83d640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbea8101130 0x7fbea8197270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:26:31.125 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:31.123+0000 7fbeac83d640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbea8102370 0x7fbea81977b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:26:31.125 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:31.123+0000 7fbeac83d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbea8107070 0x7fbea819e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:26:31.126 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:31.123+0000 7fbeac83d640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fbea810c5d0 con 0x7fbea8107070 2024-05-20T16:26:31.126 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:31.123+0000 7fbeac83d640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fbea810c450 con 0x7fbea8102370 2024-05-20T16:26:31.126 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:31.123+0000 7fbeac83d640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fbea810c750 con 0x7fbea8101130 2024-05-20T16:26:31.126 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:31.124+0000 7fbea6d9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbea8107070 0x7fbea819e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:26:31.126 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:31.124+0000 7fbea6d9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbea8107070 0x7fbea819e6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:49612/0 (socket says 172.21.15.82:49612) 2024-05-20T16:26:31.126 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:31.124+0000 7fbea6d9d640 1 -- 172.21.15.82:0/2324511370 learned_addr learned my addr 172.21.15.82:0/2324511370 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:26:31.126 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:31.124+0000 7fbea5d9b640 1 --2- 172.21.15.82:0/2324511370 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbea8102370 0x7fbea81977b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:26:31.126 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:31.124+0000 7fbea659c640 1 --2- 172.21.15.82:0/2324511370 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbea8101130 0x7fbea8197270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:26:31.126 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:31.124+0000 7fbea6d9d640 1 -- 172.21.15.82:0/2324511370 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbea8101130 msgr2=0x7fbea8197270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:26:31.126 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:31.124+0000 7fbea6d9d640 1 --2- 172.21.15.82:0/2324511370 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbea8101130 0x7fbea8197270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:26:31.126 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:31.124+0000 7fbea6d9d640 1 -- 172.21.15.82:0/2324511370 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbea8102370 msgr2=0x7fbea81977b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:26:31.126 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:31.124+0000 7fbea6d9d640 1 --2- 172.21.15.82:0/2324511370 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbea8102370 0x7fbea81977b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:26:31.126 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:31.124+0000 7fbea6d9d640 1 -- 172.21.15.82:0/2324511370 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbea819ed40 con 0x7fbea8107070 2024-05-20T16:26:31.127 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:31.124+0000 7fbea5d9b640 1 --2- 172.21.15.82:0/2324511370 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbea8102370 0x7fbea81977b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T16:26:31.127 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:31.124+0000 7fbea6d9d640 1 --2- 172.21.15.82:0/2324511370 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbea8107070 0x7fbea819e6d0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fbea000b970 tx=0x7fbea000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:26:31.127 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:31.124+0000 7fbe977fe640 1 -- 172.21.15.82:0/2324511370 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbea0019070 con 0x7fbea8107070 2024-05-20T16:26:31.127 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:31.124+0000 7fbe977fe640 1 -- 172.21.15.82:0/2324511370 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbea000cb40 con 0x7fbea8107070 2024-05-20T16:26:31.127 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:31.124+0000 7fbeac83d640 1 -- 172.21.15.82:0/2324511370 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbea819efd0 con 0x7fbea8107070 2024-05-20T16:26:31.127 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:31.124+0000 7fbe977fe640 1 -- 172.21.15.82:0/2324511370 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbea0007650 con 0x7fbea8107070 2024-05-20T16:26:31.127 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:31.124+0000 7fbeac83d640 1 -- 172.21.15.82:0/2324511370 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbea819f4c0 con 0x7fbea8107070 2024-05-20T16:26:31.127 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:31.126+0000 7fbeac83d640 1 -- 172.21.15.82:0/2324511370 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbea8105e80 con 0x7fbea8107070 2024-05-20T16:26:31.128 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:31.126+0000 7fbe977fe640 1 -- 172.21.15.82:0/2324511370 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fbea001c070 con 0x7fbea8107070 2024-05-20T16:26:31.128 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:31.127+0000 7fbe977fe640 1 --2- 172.21.15.82:0/2324511370 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fbe900a2b90 0x7fbe900a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:26:31.128 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:31.128+0000 7fbea659c640 1 --2- 172.21.15.82:0/2324511370 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fbe900a2b90 0x7fbe900a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:26:31.128 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:31.128+0000 7fbe977fe640 1 -- 172.21.15.82:0/2324511370 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9408+0+0 (secure 0 0 0) 0x7fbea00ceb40 con 0x7fbea8107070 2024-05-20T16:26:31.128 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:31.128+0000 7fbea659c640 1 --2- 172.21.15.82:0/2324511370 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fbe900a2b90 0x7fbe900a50a0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fbe9800a340 tx=0x7fbe98030510 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:26:31.132 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:31.132+0000 7fbe977fe640 1 -- 172.21.15.82:0/2324511370 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fbea009b610 con 0x7fbea8107070 2024-05-20T16:26:31.287 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:26:30 smithi184 ceph-mon[36209]: pgmap v741: 129 pgs: 129 active+clean; 5.8 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 161 MiB/s wr, 1.51k op/s 2024-05-20T16:26:31.287 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:26:30 smithi184 ceph-mon[36209]: from='client.? 172.21.15.82:0/2644945321' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-05-20T16:26:31.309 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:31.310+0000 7fbeac83d640 1 -- 172.21.15.82:0/2324511370 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fbea8062750 con 0x7fbea8107070 2024-05-20T16:26:31.311 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:26:30 smithi160 ceph-mon[36347]: pgmap v741: 129 pgs: 129 active+clean; 5.8 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 161 MiB/s wr, 1.51k op/s 2024-05-20T16:26:31.312 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:26:30 smithi160 ceph-mon[36347]: from='client.? 172.21.15.82:0/2644945321' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-05-20T16:26:31.358 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:26:30 smithi082 ceph-mon[32238]: pgmap v741: 129 pgs: 129 active+clean; 5.8 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 161 MiB/s wr, 1.51k op/s 2024-05-20T16:26:31.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:26:30 smithi082 ceph-mon[32238]: from='client.? 172.21.15.82:0/2644945321' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-05-20T16:26:31.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:31.990+0000 7fbe977fe640 1 -- 172.21.15.82:0/2324511370 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v100) ==== 181+0+0 (secure 0 0 0) 0x7fbea009af10 con 0x7fbea8107070 2024-05-20T16:26:31.990 INFO:teuthology.orchestra.run.smithi082.stderr:pool 'cephfs_metadata' removed 2024-05-20T16:26:31.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:31.993+0000 7fbeac83d640 1 -- 172.21.15.82:0/2324511370 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fbe900a2b90 msgr2=0x7fbe900a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:26:31.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:31.993+0000 7fbeac83d640 1 --2- 172.21.15.82:0/2324511370 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fbe900a2b90 0x7fbe900a50a0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fbe9800a340 tx=0x7fbe98030510 comp rx=0 tx=0).stop 2024-05-20T16:26:31.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:31.993+0000 7fbeac83d640 1 -- 172.21.15.82:0/2324511370 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbea8107070 msgr2=0x7fbea819e6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:26:31.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:31.993+0000 7fbeac83d640 1 --2- 172.21.15.82:0/2324511370 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbea8107070 0x7fbea819e6d0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fbea000b970 tx=0x7fbea000be30 comp rx=0 tx=0).stop 2024-05-20T16:26:31.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:31.994+0000 7fbeac83d640 1 -- 172.21.15.82:0/2324511370 shutdown_connections 2024-05-20T16:26:31.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:31.994+0000 7fbeac83d640 1 --2- 172.21.15.82:0/2324511370 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fbe900a2b90 0x7fbe900a50a0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:26:31.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:31.994+0000 7fbeac83d640 1 --2- 172.21.15.82:0/2324511370 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbea8107070 0x7fbea819e6d0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:26:31.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:31.994+0000 7fbeac83d640 1 --2- 172.21.15.82:0/2324511370 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbea8102370 0x7fbea81977b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:26:31.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:31.994+0000 7fbeac83d640 1 --2- 172.21.15.82:0/2324511370 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbea8101130 0x7fbea8197270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:26:31.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:31.994+0000 7fbeac83d640 1 -- 172.21.15.82:0/2324511370 >> 172.21.15.82:0/2324511370 conn(0x7fbea80fc300 msgr2=0x7fbea8103050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:26:31.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:31.994+0000 7fbeac83d640 1 -- 172.21.15.82:0/2324511370 shutdown_connections 2024-05-20T16:26:31.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:31.994+0000 7fbeac83d640 1 -- 172.21.15.82:0/2324511370 wait complete. 2024-05-20T16:26:32.010 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-05-20T16:26:32.131 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:32.131+0000 7fdb3f4b6640 1 -- 172.21.15.82:0/2947206800 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdb38101f30 msgr2=0x7fdb38102340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:26:32.131 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:32.131+0000 7fdb3f4b6640 1 --2- 172.21.15.82:0/2947206800 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdb38101f30 0x7fdb38102340 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fdb34009900 tx=0x7fdb3402a410 comp rx=0 tx=0).stop 2024-05-20T16:26:32.131 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:32.131+0000 7fdb3f4b6640 1 -- 172.21.15.82:0/2947206800 shutdown_connections 2024-05-20T16:26:32.131 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:32.131+0000 7fdb3f4b6640 1 --2- 172.21.15.82:0/2947206800 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdb38103430 0x7fdb38109b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:26:32.131 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:32.131+0000 7fdb3f4b6640 1 --2- 172.21.15.82:0/2947206800 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdb38102910 0x7fdb38102da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:26:32.131 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:32.131+0000 7fdb3f4b6640 1 --2- 172.21.15.82:0/2947206800 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdb38101f30 0x7fdb38102340 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:26:32.131 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:32.131+0000 7fdb3f4b6640 1 -- 172.21.15.82:0/2947206800 >> 172.21.15.82:0/2947206800 conn(0x7fdb380fc330 msgr2=0x7fdb380fe7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:26:32.131 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:32.131+0000 7fdb3f4b6640 1 -- 172.21.15.82:0/2947206800 shutdown_connections 2024-05-20T16:26:32.131 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:32.131+0000 7fdb3f4b6640 1 -- 172.21.15.82:0/2947206800 wait complete. 2024-05-20T16:26:32.131 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:32.131+0000 7fdb3f4b6640 1 Processor -- start 2024-05-20T16:26:32.131 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:32.131+0000 7fdb3f4b6640 1 -- start start 2024-05-20T16:26:32.133 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:32.132+0000 7fdb3f4b6640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdb38101f30 0x7fdb38198190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:26:32.133 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:32.132+0000 7fdb3f4b6640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdb38102910 0x7fdb381986d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:26:32.134 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:32.132+0000 7fdb3f4b6640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdb38103430 0x7fdb3819f660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:26:32.134 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:32.132+0000 7fdb3f4b6640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fdb3810d620 con 0x7fdb38102910 2024-05-20T16:26:32.134 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:32.132+0000 7fdb3f4b6640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fdb3810d4a0 con 0x7fdb38103430 2024-05-20T16:26:32.134 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:32.132+0000 7fdb3f4b6640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fdb3810d7a0 con 0x7fdb38101f30 2024-05-20T16:26:32.134 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:32.132+0000 7fdb3ca51640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdb38102910 0x7fdb381986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:26:32.134 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:32.132+0000 7fdb3ca51640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdb38102910 0x7fdb381986d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:49618/0 (socket says 172.21.15.82:49618) 2024-05-20T16:26:32.134 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:32.132+0000 7fdb3ca51640 1 -- 172.21.15.82:0/1287862478 learned_addr learned my addr 172.21.15.82:0/1287862478 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:26:32.134 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:32.132+0000 7fdb3d252640 1 --2- 172.21.15.82:0/1287862478 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdb38101f30 0x7fdb38198190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:26:32.134 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:32.132+0000 7fdb3da53640 1 --2- 172.21.15.82:0/1287862478 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdb38103430 0x7fdb3819f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:26:32.134 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:32.132+0000 7fdb3ca51640 1 -- 172.21.15.82:0/1287862478 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdb38101f30 msgr2=0x7fdb38198190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:26:32.135 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:32.132+0000 7fdb3ca51640 1 --2- 172.21.15.82:0/1287862478 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdb38101f30 0x7fdb38198190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:26:32.135 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:32.132+0000 7fdb3ca51640 1 -- 172.21.15.82:0/1287862478 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdb38103430 msgr2=0x7fdb3819f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:26:32.135 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:32.132+0000 7fdb3ca51640 1 --2- 172.21.15.82:0/1287862478 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdb38103430 0x7fdb3819f660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:26:32.135 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:32.132+0000 7fdb3ca51640 1 -- 172.21.15.82:0/1287862478 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb3819fcd0 con 0x7fdb38102910 2024-05-20T16:26:32.135 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:32.132+0000 7fdb3ca51640 1 --2- 172.21.15.82:0/1287862478 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdb38102910 0x7fdb381986d0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fdb2800d950 tx=0x7fdb2800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:26:32.135 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:32.132+0000 7fdb2e7fc640 1 -- 172.21.15.82:0/1287862478 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb28017070 con 0x7fdb38102910 2024-05-20T16:26:32.135 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:32.132+0000 7fdb2e7fc640 1 -- 172.21.15.82:0/1287862478 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb28009c30 con 0x7fdb38102910 2024-05-20T16:26:32.135 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:32.132+0000 7fdb3f4b6640 1 -- 172.21.15.82:0/1287862478 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb3819ffc0 con 0x7fdb38102910 2024-05-20T16:26:32.135 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:32.132+0000 7fdb2e7fc640 1 -- 172.21.15.82:0/1287862478 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb2800f6b0 con 0x7fdb38102910 2024-05-20T16:26:32.135 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:32.132+0000 7fdb3f4b6640 1 -- 172.21.15.82:0/1287862478 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb381a0430 con 0x7fdb38102910 2024-05-20T16:26:32.135 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:32.134+0000 7fdb3f4b6640 1 -- 172.21.15.82:0/1287862478 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb381068d0 con 0x7fdb38102910 2024-05-20T16:26:32.136 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:32.135+0000 7fdb2e7fc640 1 -- 172.21.15.82:0/1287862478 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fdb280109d0 con 0x7fdb38102910 2024-05-20T16:26:32.136 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:32.135+0000 7fdb2e7fc640 1 --2- 172.21.15.82:0/1287862478 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fdb240a2b90 0x7fdb240a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:26:32.136 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:32.136+0000 7fdb3d252640 1 --2- 172.21.15.82:0/1287862478 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fdb240a2b90 0x7fdb240a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:26:32.136 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:32.136+0000 7fdb2e7fc640 1 -- 172.21.15.82:0/1287862478 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8854+0+0 (secure 0 0 0) 0x7fdb280ceef0 con 0x7fdb38102910 2024-05-20T16:26:32.136 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:32.136+0000 7fdb3d252640 1 --2- 172.21.15.82:0/1287862478 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fdb240a2b90 0x7fdb240a50a0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fdb34009230 tx=0x7fdb34031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:26:32.138 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:32.139+0000 7fdb2e7fc640 1 -- 172.21.15.82:0/1287862478 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fdb28009de0 con 0x7fdb38102910 2024-05-20T16:26:32.312 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:26:31 smithi160 ceph-mon[36347]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-05-20T16:26:32.312 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:26:31 smithi160 ceph-mon[36347]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-05-20T16:26:32.312 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:26:31 smithi160 ceph-mon[36347]: Cluster is now healthy 2024-05-20T16:26:32.312 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:26:31 smithi160 ceph-mon[36347]: from='client.? 172.21.15.82:0/2644945321' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-05-20T16:26:32.313 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:26:31 smithi160 ceph-mon[36347]: mds.? [v2:172.21.15.184:6834/3884708021,v1:172.21.15.184:6835/3884708021] up:boot 2024-05-20T16:26:32.313 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:26:31 smithi160 ceph-mon[36347]: mds.? [v2:172.21.15.184:6836/4018984633,v1:172.21.15.184:6837/4018984633] up:boot 2024-05-20T16:26:32.313 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:26:31 smithi160 ceph-mon[36347]: mds.? [v2:172.21.15.184:6838/50112225,v1:172.21.15.184:6839/50112225] up:boot 2024-05-20T16:26:32.313 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:26:31 smithi160 ceph-mon[36347]: mds.? [v2:172.21.15.160:6834/1122148821,v1:172.21.15.160:6835/1122148821] up:boot 2024-05-20T16:26:32.313 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:26:31 smithi160 ceph-mon[36347]: mds.? [v2:172.21.15.160:6836/2926924767,v1:172.21.15.160:6838/2926924767] up:boot 2024-05-20T16:26:32.313 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:26:31 smithi160 ceph-mon[36347]: mds.? [v2:172.21.15.160:6837/1237671443,v1:172.21.15.160:6839/1237671443] up:boot 2024-05-20T16:26:32.313 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:26:31 smithi160 ceph-mon[36347]: fsmap 12 up:standby 2024-05-20T16:26:32.313 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:26:31 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-20T16:26:32.313 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:26:31 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-20T16:26:32.313 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:26:31 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-20T16:26:32.313 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:26:31 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-20T16:26:32.313 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:26:31 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-20T16:26:32.314 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:26:31 smithi160 ceph-mon[36347]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-20T16:26:32.314 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:26:31 smithi160 ceph-mon[36347]: from='client.? 172.21.15.82:0/2324511370' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-20T16:26:32.318 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:32.319+0000 7fdb3f4b6640 1 -- 172.21.15.82:0/1287862478 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fdb3802d2a0 con 0x7fdb38102910 2024-05-20T16:26:32.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:26:31 smithi082 ceph-mon[32238]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-05-20T16:26:32.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:26:31 smithi082 ceph-mon[32238]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-05-20T16:26:32.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:26:31 smithi082 ceph-mon[32238]: Cluster is now healthy 2024-05-20T16:26:32.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:26:31 smithi082 ceph-mon[32238]: from='client.? 172.21.15.82:0/2644945321' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-05-20T16:26:32.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:26:31 smithi082 ceph-mon[32238]: mds.? [v2:172.21.15.184:6834/3884708021,v1:172.21.15.184:6835/3884708021] up:boot 2024-05-20T16:26:32.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:26:31 smithi082 ceph-mon[32238]: mds.? [v2:172.21.15.184:6836/4018984633,v1:172.21.15.184:6837/4018984633] up:boot 2024-05-20T16:26:32.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:26:31 smithi082 ceph-mon[32238]: mds.? [v2:172.21.15.184:6838/50112225,v1:172.21.15.184:6839/50112225] up:boot 2024-05-20T16:26:32.360 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:26:31 smithi082 ceph-mon[32238]: mds.? [v2:172.21.15.160:6834/1122148821,v1:172.21.15.160:6835/1122148821] up:boot 2024-05-20T16:26:32.360 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:26:31 smithi082 ceph-mon[32238]: mds.? [v2:172.21.15.160:6836/2926924767,v1:172.21.15.160:6838/2926924767] up:boot 2024-05-20T16:26:32.360 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:26:31 smithi082 ceph-mon[32238]: mds.? [v2:172.21.15.160:6837/1237671443,v1:172.21.15.160:6839/1237671443] up:boot 2024-05-20T16:26:32.360 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:26:31 smithi082 ceph-mon[32238]: fsmap 12 up:standby 2024-05-20T16:26:32.360 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:26:31 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-20T16:26:32.360 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:26:31 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-20T16:26:32.360 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:26:31 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-20T16:26:32.360 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:26:31 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-20T16:26:32.360 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:26:31 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-20T16:26:32.360 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:26:31 smithi082 ceph-mon[32238]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-20T16:26:32.360 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:26:31 smithi082 ceph-mon[32238]: from='client.? 172.21.15.82:0/2324511370' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-20T16:26:32.385 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:26:31 smithi184 ceph-mon[36209]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-05-20T16:26:32.385 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:26:31 smithi184 ceph-mon[36209]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-05-20T16:26:32.385 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:26:31 smithi184 ceph-mon[36209]: Cluster is now healthy 2024-05-20T16:26:32.385 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:26:31 smithi184 ceph-mon[36209]: from='client.? 172.21.15.82:0/2644945321' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-05-20T16:26:32.385 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:26:31 smithi184 ceph-mon[36209]: mds.? [v2:172.21.15.184:6834/3884708021,v1:172.21.15.184:6835/3884708021] up:boot 2024-05-20T16:26:32.386 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:26:31 smithi184 ceph-mon[36209]: mds.? [v2:172.21.15.184:6836/4018984633,v1:172.21.15.184:6837/4018984633] up:boot 2024-05-20T16:26:32.386 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:26:31 smithi184 ceph-mon[36209]: mds.? [v2:172.21.15.184:6838/50112225,v1:172.21.15.184:6839/50112225] up:boot 2024-05-20T16:26:32.386 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:26:31 smithi184 ceph-mon[36209]: mds.? [v2:172.21.15.160:6834/1122148821,v1:172.21.15.160:6835/1122148821] up:boot 2024-05-20T16:26:32.386 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:26:31 smithi184 ceph-mon[36209]: mds.? [v2:172.21.15.160:6836/2926924767,v1:172.21.15.160:6838/2926924767] up:boot 2024-05-20T16:26:32.386 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:26:31 smithi184 ceph-mon[36209]: mds.? [v2:172.21.15.160:6837/1237671443,v1:172.21.15.160:6839/1237671443] up:boot 2024-05-20T16:26:32.386 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:26:31 smithi184 ceph-mon[36209]: fsmap 12 up:standby 2024-05-20T16:26:32.386 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:26:31 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-20T16:26:32.386 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:26:31 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-20T16:26:32.386 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:26:31 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-20T16:26:32.386 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:26:31 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-20T16:26:32.386 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:26:31 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-20T16:26:32.386 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:26:31 smithi184 ceph-mon[36209]: from='mgr.14150 172.21.15.82:0/1141261082' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-20T16:26:32.387 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:26:31 smithi184 ceph-mon[36209]: from='client.? 172.21.15.82:0/2324511370' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-20T16:26:32.991 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:32.991+0000 7fdb2e7fc640 1 -- 172.21.15.82:0/1287862478 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v101) ==== 169+0+0 (secure 0 0 0) 0x7fdb2809bbe0 con 0x7fdb38102910 2024-05-20T16:26:32.991 INFO:teuthology.orchestra.run.smithi082.stderr:pool 'cephfs_data' removed 2024-05-20T16:26:32.995 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:32.995+0000 7fdb3f4b6640 1 -- 172.21.15.82:0/1287862478 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fdb240a2b90 msgr2=0x7fdb240a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:26:32.995 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:32.995+0000 7fdb3f4b6640 1 --2- 172.21.15.82:0/1287862478 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fdb240a2b90 0x7fdb240a50a0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fdb34009230 tx=0x7fdb34031400 comp rx=0 tx=0).stop 2024-05-20T16:26:32.995 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:32.995+0000 7fdb3f4b6640 1 -- 172.21.15.82:0/1287862478 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdb38102910 msgr2=0x7fdb381986d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:26:32.995 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:32.995+0000 7fdb3f4b6640 1 --2- 172.21.15.82:0/1287862478 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdb38102910 0x7fdb381986d0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fdb2800d950 tx=0x7fdb2800de10 comp rx=0 tx=0).stop 2024-05-20T16:26:32.995 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:32.995+0000 7fdb3f4b6640 1 -- 172.21.15.82:0/1287862478 shutdown_connections 2024-05-20T16:26:32.995 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:32.995+0000 7fdb3f4b6640 1 --2- 172.21.15.82:0/1287862478 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7fdb240a2b90 0x7fdb240a50a0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:26:32.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:32.995+0000 7fdb3f4b6640 1 --2- 172.21.15.82:0/1287862478 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdb38102910 0x7fdb381986d0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:26:32.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:32.995+0000 7fdb3f4b6640 1 --2- 172.21.15.82:0/1287862478 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdb38103430 0x7fdb3819f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:26:32.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:32.995+0000 7fdb3f4b6640 1 --2- 172.21.15.82:0/1287862478 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdb38101f30 0x7fdb38198190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:26:32.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:32.995+0000 7fdb3f4b6640 1 -- 172.21.15.82:0/1287862478 >> 172.21.15.82:0/1287862478 conn(0x7fdb380fc330 msgr2=0x7fdb38103aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:26:32.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:32.995+0000 7fdb3f4b6640 1 -- 172.21.15.82:0/1287862478 shutdown_connections 2024-05-20T16:26:32.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:32.995+0000 7fdb3f4b6640 1 -- 172.21.15.82:0/1287862478 wait complete. 2024-05-20T16:26:33.009 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-05-20T16:26:33.128 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:33.127+0000 7f71c5492640 1 -- 172.21.15.82:0/3572132963 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f71c0101f30 msgr2=0x7f71c0102340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:26:33.128 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:33.127+0000 7f71c5492640 1 --2- 172.21.15.82:0/3572132963 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f71c0101f30 0x7f71c0102340 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f71b0009900 tx=0x7f71b002a410 comp rx=0 tx=0).stop 2024-05-20T16:26:33.128 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:33.128+0000 7f71c5492640 1 -- 172.21.15.82:0/3572132963 shutdown_connections 2024-05-20T16:26:33.128 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:33.128+0000 7f71c5492640 1 --2- 172.21.15.82:0/3572132963 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f71c0103410 0x7f71c0109b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:26:33.128 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:33.128+0000 7f71c5492640 1 --2- 172.21.15.82:0/3572132963 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f71c0102910 0x7f71c0102d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:26:33.128 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:33.128+0000 7f71c5492640 1 --2- 172.21.15.82:0/3572132963 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f71c0101f30 0x7f71c0102340 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:26:33.128 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:33.128+0000 7f71c5492640 1 -- 172.21.15.82:0/3572132963 >> 172.21.15.82:0/3572132963 conn(0x7f71c00fc330 msgr2=0x7f71c00fe7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:26:33.128 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:33.128+0000 7f71c5492640 1 -- 172.21.15.82:0/3572132963 shutdown_connections 2024-05-20T16:26:33.128 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:33.128+0000 7f71c5492640 1 -- 172.21.15.82:0/3572132963 wait complete. 2024-05-20T16:26:33.128 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:33.128+0000 7f71c5492640 1 Processor -- start 2024-05-20T16:26:33.128 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:33.128+0000 7f71c5492640 1 -- start start 2024-05-20T16:26:33.131 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:33.129+0000 7f71c5492640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f71c0101f30 0x7f71c01011d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:26:33.131 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:33.129+0000 7f71c5492640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f71c0102910 0x7f71c019e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:26:33.131 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:33.129+0000 7f71c5492640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f71c0103410 0x7f71c01a05b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:26:33.131 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:33.129+0000 7f71c5492640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f71c010d630 con 0x7f71c0102910 2024-05-20T16:26:33.131 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:33.129+0000 7f71c5492640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f71c010d4b0 con 0x7f71c0103410 2024-05-20T16:26:33.132 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:33.129+0000 7f71c5492640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f71c010d7b0 con 0x7f71c0101f30 2024-05-20T16:26:33.132 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:33.129+0000 7f71b67fc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f71c0102910 0x7f71c019e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:26:33.132 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:33.129+0000 7f71b67fc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f71c0102910 0x7f71c019e060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:49636/0 (socket says 172.21.15.82:49636) 2024-05-20T16:26:33.132 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:33.129+0000 7f71b67fc640 1 -- 172.21.15.82:0/304677363 learned_addr learned my addr 172.21.15.82:0/304677363 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-20T16:26:33.132 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:33.129+0000 7f71beffd640 1 --2- 172.21.15.82:0/304677363 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f71c0101f30 0x7f71c01011d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:26:33.132 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:33.129+0000 7f71bf7fe640 1 --2- 172.21.15.82:0/304677363 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f71c0103410 0x7f71c01a05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:26:33.132 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:33.129+0000 7f71b67fc640 1 -- 172.21.15.82:0/304677363 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f71c0101f30 msgr2=0x7f71c01011d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:26:33.132 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:33.129+0000 7f71b67fc640 1 --2- 172.21.15.82:0/304677363 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f71c0101f30 0x7f71c01011d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:26:33.132 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:33.129+0000 7f71b67fc640 1 -- 172.21.15.82:0/304677363 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f71c0103410 msgr2=0x7f71c01a05b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:26:33.132 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:33.129+0000 7f71b67fc640 1 --2- 172.21.15.82:0/304677363 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f71c0103410 0x7f71c01a05b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:26:33.132 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:33.129+0000 7f71b67fc640 1 -- 172.21.15.82:0/304677363 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71c01a0dd0 con 0x7f71c0102910 2024-05-20T16:26:33.132 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:33.129+0000 7f71bf7fe640 1 --2- 172.21.15.82:0/304677363 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f71c0103410 0x7f71c01a05b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T16:26:33.133 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:33.129+0000 7f71b67fc640 1 --2- 172.21.15.82:0/304677363 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f71c0102910 0x7f71c019e060 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f71a800c890 tx=0x7f71a800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:26:33.133 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:33.129+0000 7f71bcff9640 1 -- 172.21.15.82:0/304677363 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71a8004220 con 0x7f71c0102910 2024-05-20T16:26:33.133 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:33.129+0000 7f71c5492640 1 -- 172.21.15.82:0/304677363 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71c01a10c0 con 0x7f71c0102910 2024-05-20T16:26:33.133 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:33.129+0000 7f71bcff9640 1 -- 172.21.15.82:0/304677363 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f71a80043c0 con 0x7f71c0102910 2024-05-20T16:26:33.133 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:33.129+0000 7f71bcff9640 1 -- 172.21.15.82:0/304677363 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71a8004a00 con 0x7f71c0102910 2024-05-20T16:26:33.133 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:33.129+0000 7f71c5492640 1 -- 172.21.15.82:0/304677363 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71c01a1610 con 0x7f71c0102910 2024-05-20T16:26:33.133 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:33.130+0000 7f71c5492640 1 -- 172.21.15.82:0/304677363 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71c01068d0 con 0x7f71c0102910 2024-05-20T16:26:33.133 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:33.132+0000 7f71bcff9640 1 -- 172.21.15.82:0/304677363 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f71a80079b0 con 0x7f71c0102910 2024-05-20T16:26:33.134 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:33.132+0000 7f71bcff9640 1 --2- 172.21.15.82:0/304677363 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f71ac0a2b90 0x7f71ac0a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T16:26:33.134 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:33.132+0000 7f71beffd640 1 --2- 172.21.15.82:0/304677363 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f71ac0a2b90 0x7f71ac0a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T16:26:33.134 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:33.132+0000 7f71bcff9640 1 -- 172.21.15.82:0/304677363 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8324+0+0 (secure 0 0 0) 0x7f71a80cf250 con 0x7f71c0102910 2024-05-20T16:26:33.134 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:33.133+0000 7f71beffd640 1 --2- 172.21.15.82:0/304677363 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f71ac0a2b90 0x7f71ac0a50a0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f71b002aa00 tx=0x7f71b0031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T16:26:33.136 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:33.136+0000 7f71bcff9640 1 -- 172.21.15.82:0/304677363 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f71a8014030 con 0x7f71c0102910 2024-05-20T16:26:33.214 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:26:32 smithi160 ceph-mon[36347]: pgmap v742: 129 pgs: 129 active+clean; 3.7 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 168 MiB/s wr, 1.69k op/s 2024-05-20T16:26:33.215 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:26:32 smithi160 ceph-mon[36347]: from='client.? 172.21.15.82:0/2324511370' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-05-20T16:26:33.215 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:26:32 smithi160 ceph-mon[36347]: osdmap e100: 12 total, 12 up, 12 in 2024-05-20T16:26:33.215 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:26:32 smithi160 ceph-mon[36347]: from='client.? 172.21.15.82:0/1287862478' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-20T16:26:33.314 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:33.315+0000 7f71c5492640 1 -- 172.21.15.82:0/304677363 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) -- 0x7f71c002d2a0 con 0x7f71c0102910 2024-05-20T16:26:33.358 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:26:32 smithi082 ceph-mon[32238]: pgmap v742: 129 pgs: 129 active+clean; 3.7 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 168 MiB/s wr, 1.69k op/s 2024-05-20T16:26:33.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:26:32 smithi082 ceph-mon[32238]: from='client.? 172.21.15.82:0/2324511370' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-05-20T16:26:33.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:26:32 smithi082 ceph-mon[32238]: osdmap e100: 12 total, 12 up, 12 in 2024-05-20T16:26:33.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:26:32 smithi082 ceph-mon[32238]: from='client.? 172.21.15.82:0/1287862478' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-20T16:26:33.385 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:26:32 smithi184 ceph-mon[36209]: pgmap v742: 129 pgs: 129 active+clean; 3.7 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 168 MiB/s wr, 1.69k op/s 2024-05-20T16:26:33.385 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:26:32 smithi184 ceph-mon[36209]: from='client.? 172.21.15.82:0/2324511370' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-05-20T16:26:33.385 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:26:32 smithi184 ceph-mon[36209]: osdmap e100: 12 total, 12 up, 12 in 2024-05-20T16:26:33.385 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:26:32 smithi184 ceph-mon[36209]: from='client.? 172.21.15.82:0/1287862478' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-20T16:26:33.997 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:33.997+0000 7f71bcff9640 1 -- 172.21.15.82:0/304677363 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v102) ==== 178+0+0 (secure 0 0 0) 0x7f71a80ce0d0 con 0x7f71c0102910 2024-05-20T16:26:33.997 INFO:teuthology.orchestra.run.smithi082.stderr:pool 'cephfs_data_ec' removed 2024-05-20T16:26:34.001 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:34.001+0000 7f71c5492640 1 -- 172.21.15.82:0/304677363 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f71ac0a2b90 msgr2=0x7f71ac0a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:26:34.001 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:34.001+0000 7f71c5492640 1 --2- 172.21.15.82:0/304677363 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f71ac0a2b90 0x7f71ac0a50a0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f71b002aa00 tx=0x7f71b0031400 comp rx=0 tx=0).stop 2024-05-20T16:26:34.001 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:34.001+0000 7f71c5492640 1 -- 172.21.15.82:0/304677363 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f71c0102910 msgr2=0x7f71c019e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T16:26:34.001 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:34.001+0000 7f71c5492640 1 --2- 172.21.15.82:0/304677363 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f71c0102910 0x7f71c019e060 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f71a800c890 tx=0x7f71a800cd50 comp rx=0 tx=0).stop 2024-05-20T16:26:34.001 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:34.001+0000 7f71c5492640 1 -- 172.21.15.82:0/304677363 shutdown_connections 2024-05-20T16:26:34.001 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:34.001+0000 7f71c5492640 1 --2- 172.21.15.82:0/304677363 >> [v2:172.21.15.82:6800/379530823,v1:172.21.15.82:6801/379530823] conn(0x7f71ac0a2b90 0x7f71ac0a50a0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:26:34.001 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:34.001+0000 7f71c5492640 1 --2- 172.21.15.82:0/304677363 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f71c0102910 0x7f71c019e060 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:26:34.001 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:34.001+0000 7f71c5492640 1 --2- 172.21.15.82:0/304677363 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f71c0103410 0x7f71c01a05b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:26:34.001 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:34.001+0000 7f71c5492640 1 --2- 172.21.15.82:0/304677363 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f71c0101f30 0x7f71c01011d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T16:26:34.001 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:34.001+0000 7f71c5492640 1 -- 172.21.15.82:0/304677363 >> 172.21.15.82:0/304677363 conn(0x7f71c00fc330 msgr2=0x7f71c0103aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T16:26:34.002 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:34.001+0000 7f71c5492640 1 -- 172.21.15.82:0/304677363 shutdown_connections 2024-05-20T16:26:34.002 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-20T16:26:34.002+0000 7f71c5492640 1 -- 172.21.15.82:0/304677363 wait complete. 2024-05-20T16:26:34.020 DEBUG:teuthology.orchestra.run.smithi082:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-20T16:26:34.052 DEBUG:teuthology.orchestra.run.smithi160:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-20T16:26:34.061 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:26:33 smithi160 ceph-mon[36347]: from='client.? 172.21.15.82:0/1287862478' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-05-20T16:26:34.062 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:26:33 smithi160 ceph-mon[36347]: osdmap e101: 12 total, 12 up, 12 in 2024-05-20T16:26:34.062 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:26:33 smithi160 ceph-mon[36347]: from='client.? 172.21.15.82:0/304677363' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-20T16:26:34.062 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:26:33 smithi160 ceph-mon[36347]: pgmap v745: 33 pgs: 33 active+clean; 3.0 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s wr, 1k op/s 2024-05-20T16:26:34.080 DEBUG:teuthology.orchestra.run.smithi184:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-20T16:26:34.110 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-05-20T16:26:34.110 DEBUG:teuthology.orchestra.run.smithi082:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-05-20T16:26:34.126 DEBUG:teuthology.orchestra.run.smithi160:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-05-20T16:26:34.141 DEBUG:teuthology.orchestra.run.smithi184:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-05-20T16:26:34.167 INFO:tasks.cephadm:Stopping all daemons... 2024-05-20T16:26:34.167 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-05-20T16:26:34.167 DEBUG:teuthology.orchestra.run.smithi082:> sudo systemctl stop ceph-59d1600a-16c2-11ef-bc9a-c7b262605968@mon.a 2024-05-20T16:26:34.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:26:33 smithi082 ceph-mon[32238]: from='client.? 172.21.15.82:0/1287862478' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-05-20T16:26:34.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:26:33 smithi082 ceph-mon[32238]: osdmap e101: 12 total, 12 up, 12 in 2024-05-20T16:26:34.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:26:33 smithi082 ceph-mon[32238]: from='client.? 172.21.15.82:0/304677363' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-20T16:26:34.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:26:33 smithi082 ceph-mon[32238]: pgmap v745: 33 pgs: 33 active+clean; 3.0 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s wr, 1k op/s 2024-05-20T16:26:34.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:26:34 smithi082 systemd[1]: Stopping Ceph mon.a for 59d1600a-16c2-11ef-bc9a-c7b262605968... 2024-05-20T16:26:34.385 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:26:33 smithi184 ceph-mon[36209]: from='client.? 172.21.15.82:0/1287862478' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-05-20T16:26:34.385 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:26:33 smithi184 ceph-mon[36209]: osdmap e101: 12 total, 12 up, 12 in 2024-05-20T16:26:34.385 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:26:33 smithi184 ceph-mon[36209]: from='client.? 172.21.15.82:0/304677363' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-20T16:26:34.385 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:26:33 smithi184 ceph-mon[36209]: pgmap v745: 33 pgs: 33 active+clean; 3.0 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s wr, 1k op/s 2024-05-20T16:26:34.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:26:34 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mon-a[32234]: 2024-05-20T16:26:34.490+0000 7fbc271a7700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-20T16:26:34.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:26:34 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mon-a[32234]: 2024-05-20T16:26:34.490+0000 7fbc271a7700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-05-20T16:26:34.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:26:34 smithi082 podman[91503]: 2024-05-20 16:26:34.583023812 +0000 UTC m=+0.211193155 container died 1c5b1c87352a8fe3c4399e9c039e6ffd8962413e7ab6afaaa251874aa2fc7fe3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d, name=ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mon-a, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240514, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-05-20T16:26:35.259 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:26:35 smithi082 podman[91503]: 2024-05-20 16:26:34.999915186 +0000 UTC m=+0.628084527 container cleanup 1c5b1c87352a8fe3c4399e9c039e6ffd8962413e7ab6afaaa251874aa2fc7fe3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d, name=ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mon-a, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240514, GIT_BRANCH=HEAD) 2024-05-20T16:26:35.259 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:26:35 smithi082 bash[91503]: ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mon-a 2024-05-20T16:26:35.542 INFO:journalctl@ceph.mon.a.smithi082.stdout:May 20 16:26:35 smithi082 podman[91515]: 2024-05-20 16:26:35.260119183 +0000 UTC m=+0.672478698 container remove 1c5b1c87352a8fe3c4399e9c039e6ffd8962413e7ab6afaaa251874aa2fc7fe3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d, name=ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mon-a, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240514, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c) 2024-05-20T16:26:35.557 DEBUG:teuthology.orchestra.run.smithi082:> sudo pkill -f 'journalctl -f -n 0 -u ceph-59d1600a-16c2-11ef-bc9a-c7b262605968@mon.a.service' 2024-05-20T16:26:35.597 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-20T16:26:35.597 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-05-20T16:26:35.597 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-05-20T16:26:35.597 DEBUG:teuthology.orchestra.run.smithi160:> sudo systemctl stop ceph-59d1600a-16c2-11ef-bc9a-c7b262605968@mon.b 2024-05-20T16:26:36.008 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:26:35 smithi160 systemd[1]: Stopping Ceph mon.b for 59d1600a-16c2-11ef-bc9a-c7b262605968... 2024-05-20T16:26:36.312 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:26:36 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mon-b[36343]: 2024-05-20T16:26:36.011+0000 7f984e7b2700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-20T16:26:36.312 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:26:36 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mon-b[36343]: 2024-05-20T16:26:36.011+0000 7f984e7b2700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-05-20T16:26:36.312 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:26:36 smithi160 podman[71392]: 2024-05-20 16:26:36.116964674 +0000 UTC m=+0.245546209 container died 8de30f4d82e8fbfbceeda87df920c160c2b31856d9c59f76e06928e7f3a0e1fd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mon-b, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, GIT_CLEAN=True, org.label-schema.build-date=20240514, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-05-20T16:26:36.763 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:26:36 smithi160 podman[71392]: 2024-05-20 16:26:36.538089988 +0000 UTC m=+0.666671515 container cleanup 8de30f4d82e8fbfbceeda87df920c160c2b31856d9c59f76e06928e7f3a0e1fd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mon-b, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240514, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-05-20T16:26:36.764 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:26:36 smithi160 bash[71392]: ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mon-b 2024-05-20T16:26:37.008 DEBUG:teuthology.orchestra.run.smithi160:> sudo pkill -f 'journalctl -f -n 0 -u ceph-59d1600a-16c2-11ef-bc9a-c7b262605968@mon.b.service' 2024-05-20T16:26:37.028 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:26:36 smithi160 podman[71407]: 2024-05-20 16:26:36.764741104 +0000 UTC m=+0.647133472 container remove 8de30f4d82e8fbfbceeda87df920c160c2b31856d9c59f76e06928e7f3a0e1fd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mon-b, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240514, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-20T16:26:37.028 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:26:37 smithi160 systemd[1]: ceph-59d1600a-16c2-11ef-bc9a-c7b262605968@mon.b.service: Deactivated successfully. 2024-05-20T16:26:37.028 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:26:37 smithi160 systemd[1]: Stopped Ceph mon.b for 59d1600a-16c2-11ef-bc9a-c7b262605968. 2024-05-20T16:26:37.028 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 20 16:26:37 smithi160 systemd[1]: ceph-59d1600a-16c2-11ef-bc9a-c7b262605968@mon.b.service: Consumed 15.201s CPU time. 2024-05-20T16:26:37.086 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-20T16:26:37.087 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-05-20T16:26:37.087 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-05-20T16:26:37.087 DEBUG:teuthology.orchestra.run.smithi184:> sudo systemctl stop ceph-59d1600a-16c2-11ef-bc9a-c7b262605968@mon.c 2024-05-20T16:26:37.385 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:26:37 smithi184 systemd[1]: Stopping Ceph mon.c for 59d1600a-16c2-11ef-bc9a-c7b262605968... 2024-05-20T16:26:37.885 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:26:37 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mon-c[36194]: 2024-05-20T16:26:37.464+0000 7f1e1c9a3700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-20T16:26:37.886 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:26:37 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mon-c[36194]: 2024-05-20T16:26:37.464+0000 7f1e1c9a3700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-05-20T16:26:37.886 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:26:37 smithi184 podman[72738]: 2024-05-20 16:26:37.659603803 +0000 UTC m=+0.343446491 container died 6f3a6f7f153ed2e221e7100598a5b7f2dc633158aa424dec09f5b9f6ffd379f7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mon-c, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240514, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-05-20T16:26:38.333 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:26:38 smithi184 podman[72738]: 2024-05-20 16:26:38.066031244 +0000 UTC m=+0.749873924 container cleanup 6f3a6f7f153ed2e221e7100598a5b7f2dc633158aa424dec09f5b9f6ffd379f7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mon-c, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, org.label-schema.build-date=20240514, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-05-20T16:26:38.334 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 20 16:26:38 smithi184 bash[72738]: ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mon-c 2024-05-20T16:26:38.467 DEBUG:teuthology.orchestra.run.smithi184:> sudo pkill -f 'journalctl -f -n 0 -u ceph-59d1600a-16c2-11ef-bc9a-c7b262605968@mon.c.service' 2024-05-20T16:26:38.514 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-20T16:26:38.514 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-05-20T16:26:38.514 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-05-20T16:26:38.514 DEBUG:teuthology.orchestra.run.smithi082:> sudo systemctl stop ceph-59d1600a-16c2-11ef-bc9a-c7b262605968@mgr.x 2024-05-20T16:26:38.829 INFO:journalctl@ceph.mgr.x.smithi082.stdout:May 20 16:26:38 smithi082 systemd[1]: Stopping Ceph mgr.x for 59d1600a-16c2-11ef-bc9a-c7b262605968... 2024-05-20T16:26:39.080 INFO:journalctl@ceph.mgr.x.smithi082.stdout:May 20 16:26:38 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mgr-x[32467]: 2024-05-20T16:26:38.829+0000 7f1abfb88700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-20T16:26:39.081 INFO:journalctl@ceph.mgr.x.smithi082.stdout:May 20 16:26:38 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mgr-x[32467]: 2024-05-20T16:26:38.829+0000 7f1abfb88700 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-05-20T16:26:39.081 INFO:journalctl@ceph.mgr.x.smithi082.stdout:May 20 16:26:38 smithi082 podman[91622]: 2024-05-20 16:26:38.862211846 +0000 UTC m=+0.144424120 container died c80a162fbf8bfb0508a8b4414fd657ef0ab259d17f91d1f750cd84cb994ceb2c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d, name=ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mgr-x, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, org.label-schema.build-date=20240514, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-05-20T16:26:39.355 INFO:journalctl@ceph.mgr.x.smithi082.stdout:May 20 16:26:39 smithi082 podman[91622]: 2024-05-20 16:26:39.172563126 +0000 UTC m=+0.454775403 container cleanup c80a162fbf8bfb0508a8b4414fd657ef0ab259d17f91d1f750cd84cb994ceb2c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d, name=ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mgr-x, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, org.label-schema.build-date=20240514, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-05-20T16:26:39.356 INFO:journalctl@ceph.mgr.x.smithi082.stdout:May 20 16:26:39 smithi082 bash[91622]: ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mgr-x 2024-05-20T16:26:39.572 DEBUG:teuthology.orchestra.run.smithi082:> sudo pkill -f 'journalctl -f -n 0 -u ceph-59d1600a-16c2-11ef-bc9a-c7b262605968@mgr.x.service' 2024-05-20T16:26:39.608 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-20T16:26:39.609 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-05-20T16:26:39.609 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-05-20T16:26:39.609 DEBUG:teuthology.orchestra.run.smithi160:> sudo systemctl stop ceph-59d1600a-16c2-11ef-bc9a-c7b262605968@mgr.y 2024-05-20T16:26:39.954 INFO:journalctl@ceph.mgr.y.smithi160.stdout:May 20 16:26:39 smithi160 systemd[1]: Stopping Ceph mgr.y for 59d1600a-16c2-11ef-bc9a-c7b262605968... 2024-05-20T16:26:40.312 INFO:journalctl@ceph.mgr.y.smithi160.stdout:May 20 16:26:39 smithi160 podman[71510]: 2024-05-20 16:26:39.955549774 +0000 UTC m=+0.148302736 container died 83c00ac57e95195b99534a88fe54e5775f8ff54dd24962ca5b457e02c4dd89a7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mgr-y, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240514, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-05-20T16:26:41.241 INFO:journalctl@ceph.mgr.y.smithi160.stdout:May 20 16:26:40 smithi160 podman[71510]: 2024-05-20 16:26:40.927449732 +0000 UTC m=+1.120202686 container cleanup 83c00ac57e95195b99534a88fe54e5775f8ff54dd24962ca5b457e02c4dd89a7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mgr-y, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240514, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c) 2024-05-20T16:26:41.241 INFO:journalctl@ceph.mgr.y.smithi160.stdout:May 20 16:26:40 smithi160 bash[71510]: ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mgr-y 2024-05-20T16:26:41.414 DEBUG:teuthology.orchestra.run.smithi160:> sudo pkill -f 'journalctl -f -n 0 -u ceph-59d1600a-16c2-11ef-bc9a-c7b262605968@mgr.y.service' 2024-05-20T16:26:41.448 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-20T16:26:41.448 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-05-20T16:26:41.448 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-05-20T16:26:41.449 DEBUG:teuthology.orchestra.run.smithi184:> sudo systemctl stop ceph-59d1600a-16c2-11ef-bc9a-c7b262605968@mgr.z 2024-05-20T16:26:41.885 INFO:journalctl@ceph.mgr.z.smithi184.stdout:May 20 16:26:41 smithi184 systemd[1]: Stopping Ceph mgr.z for 59d1600a-16c2-11ef-bc9a-c7b262605968... 2024-05-20T16:26:42.385 INFO:journalctl@ceph.mgr.z.smithi184.stdout:May 20 16:26:42 smithi184 podman[72859]: 2024-05-20 16:26:42.255447793 +0000 UTC m=+0.612393210 container died f9abb8907d231d6494badea5a296b0b8ff52ee42226fb7ace2c02ac70fcd2b4b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mgr-z, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240514, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-20T16:26:43.424 INFO:journalctl@ceph.mgr.z.smithi184.stdout:May 20 16:26:43 smithi184 podman[72859]: 2024-05-20 16:26:43.148383288 +0000 UTC m=+1.505328714 container cleanup f9abb8907d231d6494badea5a296b0b8ff52ee42226fb7ace2c02ac70fcd2b4b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mgr-z, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, org.label-schema.build-date=20240514, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-20T16:26:43.424 INFO:journalctl@ceph.mgr.z.smithi184.stdout:May 20 16:26:43 smithi184 bash[72859]: ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mgr-z 2024-05-20T16:26:43.576 DEBUG:teuthology.orchestra.run.smithi184:> sudo pkill -f 'journalctl -f -n 0 -u ceph-59d1600a-16c2-11ef-bc9a-c7b262605968@mgr.z.service' 2024-05-20T16:26:43.609 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-20T16:26:43.610 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-05-20T16:26:43.610 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-05-20T16:26:43.610 DEBUG:teuthology.orchestra.run.smithi082:> sudo systemctl stop ceph-59d1600a-16c2-11ef-bc9a-c7b262605968@osd.0 2024-05-20T16:26:43.925 INFO:journalctl@ceph.osd.0.smithi082.stdout:May 20 16:26:43 smithi082 systemd[1]: Stopping Ceph osd.0 for 59d1600a-16c2-11ef-bc9a-c7b262605968... 2024-05-20T16:26:44.359 INFO:journalctl@ceph.osd.0.smithi082.stdout:May 20 16:26:43 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-0[46979]: 2024-05-20T16:26:43.925+0000 7f2b7dc31700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-20T16:26:44.359 INFO:journalctl@ceph.osd.0.smithi082.stdout:May 20 16:26:43 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-0[46979]: 2024-05-20T16:26:43.925+0000 7f2b7dc31700 -1 osd.0 102 *** Got signal Terminated *** 2024-05-20T16:26:44.359 INFO:journalctl@ceph.osd.0.smithi082.stdout:May 20 16:26:43 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-0[46979]: 2024-05-20T16:26:43.925+0000 7f2b7dc31700 -1 osd.0 102 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-20T16:26:49.777 INFO:journalctl@ceph.osd.0.smithi082.stdout:May 20 16:26:49 smithi082 podman[91741]: 2024-05-20 16:26:49.515397306 +0000 UTC m=+5.704200063 container died b6842cf5e4902c063ce3cafa52b80732b8d2e4ce6311ae52cfd52f8103ac73af (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, org.label-schema.build-date=20240514, org.label-schema.vendor=CentOS, ceph=True) 2024-05-20T16:26:50.069 INFO:journalctl@ceph.osd.0.smithi082.stdout:May 20 16:26:49 smithi082 podman[91741]: 2024-05-20 16:26:49.860179571 +0000 UTC m=+6.048982321 container cleanup b6842cf5e4902c063ce3cafa52b80732b8d2e4ce6311ae52cfd52f8103ac73af (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-0, org.label-schema.build-date=20240514, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-05-20T16:26:50.069 INFO:journalctl@ceph.osd.0.smithi082.stdout:May 20 16:26:49 smithi082 bash[91741]: ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-0 2024-05-20T16:26:50.358 INFO:journalctl@ceph.osd.0.smithi082.stdout:May 20 16:26:50 smithi082 podman[91763]: 2024-05-20 16:26:50.070103523 +0000 UTC m=+0.541611385 container remove b6842cf5e4902c063ce3cafa52b80732b8d2e4ce6311ae52cfd52f8103ac73af (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-0, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, org.label-schema.build-date=20240514, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-20T16:26:50.755 INFO:journalctl@ceph.osd.0.smithi082.stdout:May 20 16:26:50 smithi082 podman[91860]: 2024-05-20 16:26:50.382240921 +0000 UTC m=+0.017356574 image pull cfcc61b2d476f611479310eb042aa05f11dd5f60dbb65e3b0d7e0e11d024cdb3 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708 2024-05-20T16:26:50.755 INFO:journalctl@ceph.osd.0.smithi082.stdout:May 20 16:26:50 smithi082 podman[91860]: 2024-05-20 16:26:50.512748024 +0000 UTC m=+0.147863667 container create 1b7e1399ad6ea1d61dccda883f23eb2c93437d4d23e3090b80b3530857091e62 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-0-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240514, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, CEPH_POINT_RELEASE=) 2024-05-20T16:26:51.109 INFO:journalctl@ceph.osd.0.smithi082.stdout:May 20 16:26:50 smithi082 podman[91860]: 2024-05-20 16:26:50.807642717 +0000 UTC m=+0.442758377 container init 1b7e1399ad6ea1d61dccda883f23eb2c93437d4d23e3090b80b3530857091e62 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-0-deactivate, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240514) 2024-05-20T16:26:51.109 INFO:journalctl@ceph.osd.0.smithi082.stdout:May 20 16:26:50 smithi082 podman[91860]: 2024-05-20 16:26:50.811575122 +0000 UTC m=+0.446690773 container start 1b7e1399ad6ea1d61dccda883f23eb2c93437d4d23e3090b80b3530857091e62 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-0-deactivate, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240514, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-05-20T16:26:51.109 INFO:journalctl@ceph.osd.0.smithi082.stdout:May 20 16:26:50 smithi082 podman[91860]: 2024-05-20 16:26:50.89063163 +0000 UTC m=+0.525747292 container attach 1b7e1399ad6ea1d61dccda883f23eb2c93437d4d23e3090b80b3530857091e62 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-0-deactivate, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240514) 2024-05-20T16:26:51.609 INFO:journalctl@ceph.osd.0.smithi082.stdout:May 20 16:26:51 smithi082 podman[91860]: 2024-05-20 16:26:51.353602049 +0000 UTC m=+0.988717701 container died 1b7e1399ad6ea1d61dccda883f23eb2c93437d4d23e3090b80b3530857091e62 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-0-deactivate, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240514, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-05-20T16:26:51.941 DEBUG:teuthology.orchestra.run.smithi082:> sudo pkill -f 'journalctl -f -n 0 -u ceph-59d1600a-16c2-11ef-bc9a-c7b262605968@osd.0.service' 2024-05-20T16:26:51.981 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-20T16:26:51.981 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-05-20T16:26:51.981 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-05-20T16:26:51.981 DEBUG:teuthology.orchestra.run.smithi160:> sudo systemctl stop ceph-59d1600a-16c2-11ef-bc9a-c7b262605968@osd.1 2024-05-20T16:26:52.302 INFO:journalctl@ceph.osd.1.smithi160.stdout:May 20 16:26:52 smithi160 systemd[1]: Stopping Ceph osd.1 for 59d1600a-16c2-11ef-bc9a-c7b262605968... 2024-05-20T16:26:52.562 INFO:journalctl@ceph.osd.1.smithi160.stdout:May 20 16:26:52 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-1[40898]: 2024-05-20T16:26:52.302+0000 7f3553e59700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-20T16:26:52.562 INFO:journalctl@ceph.osd.1.smithi160.stdout:May 20 16:26:52 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-1[40898]: 2024-05-20T16:26:52.302+0000 7f3553e59700 -1 osd.1 102 *** Got signal Terminated *** 2024-05-20T16:26:52.562 INFO:journalctl@ceph.osd.1.smithi160.stdout:May 20 16:26:52 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-1[40898]: 2024-05-20T16:26:52.302+0000 7f3553e59700 -1 osd.1 102 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-20T16:26:58.286 INFO:journalctl@ceph.osd.1.smithi160.stdout:May 20 16:26:57 smithi160 podman[71635]: 2024-05-20 16:26:57.9797211 +0000 UTC m=+5.805996971 container died fb3a05234447243b0b3ee4d881cf97d958bbe207d45d9e57edabdacc86b2e6c7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-1, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240514) 2024-05-20T16:26:58.562 INFO:journalctl@ceph.osd.1.smithi160.stdout:May 20 16:26:58 smithi160 podman[71635]: 2024-05-20 16:26:58.382644403 +0000 UTC m=+6.208920259 container cleanup fb3a05234447243b0b3ee4d881cf97d958bbe207d45d9e57edabdacc86b2e6c7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-1, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240514, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-20T16:26:58.562 INFO:journalctl@ceph.osd.1.smithi160.stdout:May 20 16:26:58 smithi160 bash[71635]: ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-1 2024-05-20T16:26:59.001 INFO:journalctl@ceph.osd.1.smithi160.stdout:May 20 16:26:58 smithi160 podman[71653]: 2024-05-20 16:26:58.676384343 +0000 UTC m=+0.689538205 container remove fb3a05234447243b0b3ee4d881cf97d958bbe207d45d9e57edabdacc86b2e6c7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-1, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240514, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-05-20T16:26:59.303 INFO:journalctl@ceph.osd.1.smithi160.stdout:May 20 16:26:59 smithi160 podman[71744]: 2024-05-20 16:26:58.905056883 +0000 UTC m=+0.017270364 image pull cfcc61b2d476f611479310eb042aa05f11dd5f60dbb65e3b0d7e0e11d024cdb3 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708 2024-05-20T16:26:59.304 INFO:journalctl@ceph.osd.1.smithi160.stdout:May 20 16:26:59 smithi160 podman[71744]: 2024-05-20 16:26:59.060786096 +0000 UTC m=+0.172999582 container create 2056ab5848644842e9f1a434e970f0650bd56370df0015ee3eb037eff7a27da1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-1-deactivate, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240514, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True) 2024-05-20T16:26:59.304 INFO:journalctl@ceph.osd.1.smithi160.stdout:May 20 16:26:59 smithi160 podman[71744]: 2024-05-20 16:26:59.304848229 +0000 UTC m=+0.417061703 container init 2056ab5848644842e9f1a434e970f0650bd56370df0015ee3eb037eff7a27da1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-1-deactivate, org.label-schema.build-date=20240514, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0) 2024-05-20T16:26:59.562 INFO:journalctl@ceph.osd.1.smithi160.stdout:May 20 16:26:59 smithi160 podman[71744]: 2024-05-20 16:26:59.30830801 +0000 UTC m=+0.420521486 container start 2056ab5848644842e9f1a434e970f0650bd56370df0015ee3eb037eff7a27da1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-1-deactivate, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240514, org.label-schema.schema-version=1.0) 2024-05-20T16:26:59.562 INFO:journalctl@ceph.osd.1.smithi160.stdout:May 20 16:26:59 smithi160 podman[71744]: 2024-05-20 16:26:59.430078111 +0000 UTC m=+0.542291581 container attach 2056ab5848644842e9f1a434e970f0650bd56370df0015ee3eb037eff7a27da1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-1-deactivate, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, org.label-schema.build-date=20240514, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-05-20T16:27:00.062 INFO:journalctl@ceph.osd.1.smithi160.stdout:May 20 16:26:59 smithi160 podman[71744]: 2024-05-20 16:26:59.801982256 +0000 UTC m=+0.914195732 container died 2056ab5848644842e9f1a434e970f0650bd56370df0015ee3eb037eff7a27da1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-1-deactivate, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240514, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-05-20T16:27:00.475 DEBUG:teuthology.orchestra.run.smithi160:> sudo pkill -f 'journalctl -f -n 0 -u ceph-59d1600a-16c2-11ef-bc9a-c7b262605968@osd.1.service' 2024-05-20T16:27:00.515 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-20T16:27:00.516 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-05-20T16:27:00.516 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-05-20T16:27:00.516 DEBUG:teuthology.orchestra.run.smithi184:> sudo systemctl stop ceph-59d1600a-16c2-11ef-bc9a-c7b262605968@osd.2 2024-05-20T16:27:00.851 INFO:journalctl@ceph.osd.2.smithi184.stdout:May 20 16:27:00 smithi184 systemd[1]: Stopping Ceph osd.2 for 59d1600a-16c2-11ef-bc9a-c7b262605968... 2024-05-20T16:27:01.135 INFO:journalctl@ceph.osd.2.smithi184.stdout:May 20 16:27:00 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-2[41326]: 2024-05-20T16:27:00.851+0000 7f52a1ac7700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-20T16:27:01.135 INFO:journalctl@ceph.osd.2.smithi184.stdout:May 20 16:27:00 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-2[41326]: 2024-05-20T16:27:00.851+0000 7f52a1ac7700 -1 osd.2 102 *** Got signal Terminated *** 2024-05-20T16:27:01.136 INFO:journalctl@ceph.osd.2.smithi184.stdout:May 20 16:27:00 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-2[41326]: 2024-05-20T16:27:00.851+0000 7f52a1ac7700 -1 osd.2 102 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-20T16:27:06.805 INFO:journalctl@ceph.osd.2.smithi184.stdout:May 20 16:27:06 smithi184 podman[72982]: 2024-05-20 16:27:06.517170329 +0000 UTC m=+5.796742116 container died d38d48c4a4845662eab78093a865497bca7d2944e3f65adc9e0216099dec5380 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-2, org.label-schema.build-date=20240514, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-05-20T16:27:07.135 INFO:journalctl@ceph.osd.2.smithi184.stdout:May 20 16:27:06 smithi184 podman[72982]: 2024-05-20 16:27:06.921434206 +0000 UTC m=+6.201005987 container cleanup d38d48c4a4845662eab78093a865497bca7d2944e3f65adc9e0216099dec5380 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240514, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True) 2024-05-20T16:27:07.135 INFO:journalctl@ceph.osd.2.smithi184.stdout:May 20 16:27:06 smithi184 bash[72982]: ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-2 2024-05-20T16:27:07.505 INFO:journalctl@ceph.osd.2.smithi184.stdout:May 20 16:27:07 smithi184 podman[73010]: 2024-05-20 16:27:07.189802096 +0000 UTC m=+0.671368266 container remove d38d48c4a4845662eab78093a865497bca7d2944e3f65adc9e0216099dec5380 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240514, GIT_BRANCH=HEAD) 2024-05-20T16:27:07.784 INFO:journalctl@ceph.osd.2.smithi184.stdout:May 20 16:27:07 smithi184 podman[73097]: 2024-05-20 16:27:07.408703234 +0000 UTC m=+0.016847735 image pull cfcc61b2d476f611479310eb042aa05f11dd5f60dbb65e3b0d7e0e11d024cdb3 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708 2024-05-20T16:27:07.784 INFO:journalctl@ceph.osd.2.smithi184.stdout:May 20 16:27:07 smithi184 podman[73097]: 2024-05-20 16:27:07.532735359 +0000 UTC m=+0.140879837 container create 4c60e6d5d70406209a2f0938737609deb2f5c37880f4c47082cb7e4240302791 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-2-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240514, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c) 2024-05-20T16:27:07.784 INFO:journalctl@ceph.osd.2.smithi184.stdout:May 20 16:27:07 smithi184 podman[73097]: 2024-05-20 16:27:07.785340961 +0000 UTC m=+0.393485444 container init 4c60e6d5d70406209a2f0938737609deb2f5c37880f4c47082cb7e4240302791 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-2-deactivate, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240514, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-05-20T16:27:08.135 INFO:journalctl@ceph.osd.2.smithi184.stdout:May 20 16:27:07 smithi184 podman[73097]: 2024-05-20 16:27:07.788731458 +0000 UTC m=+0.396875942 container start 4c60e6d5d70406209a2f0938737609deb2f5c37880f4c47082cb7e4240302791 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-2-deactivate, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240514, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-20T16:27:08.135 INFO:journalctl@ceph.osd.2.smithi184.stdout:May 20 16:27:07 smithi184 podman[73097]: 2024-05-20 16:27:07.893989305 +0000 UTC m=+0.502133801 container attach 4c60e6d5d70406209a2f0938737609deb2f5c37880f4c47082cb7e4240302791 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-2-deactivate, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240514, io.buildah.version=1.33.5) 2024-05-20T16:27:08.635 INFO:journalctl@ceph.osd.2.smithi184.stdout:May 20 16:27:08 smithi184 podman[73097]: 2024-05-20 16:27:08.349735452 +0000 UTC m=+0.957879936 container died 4c60e6d5d70406209a2f0938737609deb2f5c37880f4c47082cb7e4240302791 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-2-deactivate, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240514, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-05-20T16:27:08.976 DEBUG:teuthology.orchestra.run.smithi184:> sudo pkill -f 'journalctl -f -n 0 -u ceph-59d1600a-16c2-11ef-bc9a-c7b262605968@osd.2.service' 2024-05-20T16:27:09.016 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-20T16:27:09.016 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-05-20T16:27:09.016 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-05-20T16:27:09.016 DEBUG:teuthology.orchestra.run.smithi082:> sudo systemctl stop ceph-59d1600a-16c2-11ef-bc9a-c7b262605968@osd.3 2024-05-20T16:27:09.312 INFO:journalctl@ceph.osd.3.smithi082.stdout:May 20 16:27:09 smithi082 systemd[1]: Stopping Ceph osd.3 for 59d1600a-16c2-11ef-bc9a-c7b262605968... 2024-05-20T16:27:09.609 INFO:journalctl@ceph.osd.3.smithi082.stdout:May 20 16:27:09 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-3[53439]: 2024-05-20T16:27:09.312+0000 7ff25d83a700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-20T16:27:09.609 INFO:journalctl@ceph.osd.3.smithi082.stdout:May 20 16:27:09 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-3[53439]: 2024-05-20T16:27:09.312+0000 7ff25d83a700 -1 osd.3 102 *** Got signal Terminated *** 2024-05-20T16:27:09.609 INFO:journalctl@ceph.osd.3.smithi082.stdout:May 20 16:27:09 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-3[53439]: 2024-05-20T16:27:09.312+0000 7ff25d83a700 -1 osd.3 102 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-20T16:27:10.859 INFO:journalctl@ceph.osd.9.smithi082.stdout:May 20 16:27:10 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-9[66777]: 2024-05-20T16:27:10.607+0000 7fa187fc1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-05-20T16:26:44.585296+0000 front 2024-05-20T16:26:44.585359+0000 (oldest deadline 2024-05-20T16:27:09.885493+0000) 2024-05-20T16:27:11.635 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:27:11 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:27:11.293+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-05-20T16:26:47.217074+0000 front 2024-05-20T16:26:47.216883+0000 (oldest deadline 2024-05-20T16:27:10.716827+0000) 2024-05-20T16:27:11.859 INFO:journalctl@ceph.osd.3.smithi082.stdout:May 20 16:27:11 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-3[53439]: 2024-05-20T16:27:11.686+0000 7ff259df0700 -1 osd.3 102 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-05-20T16:26:48.461778+0000 front 2024-05-20T16:26:48.461807+0000 (oldest deadline 2024-05-20T16:27:10.761786+0000) 2024-05-20T16:27:11.859 INFO:journalctl@ceph.osd.9.smithi082.stdout:May 20 16:27:11 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-9[66777]: 2024-05-20T16:27:11.603+0000 7fa187fc1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-05-20T16:26:44.585296+0000 front 2024-05-20T16:26:44.585359+0000 (oldest deadline 2024-05-20T16:27:09.885493+0000) 2024-05-20T16:27:12.635 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:27:12 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:27:12.290+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-05-20T16:26:47.217074+0000 front 2024-05-20T16:26:47.216883+0000 (oldest deadline 2024-05-20T16:27:10.716827+0000) 2024-05-20T16:27:12.635 INFO:journalctl@ceph.osd.5.smithi184.stdout:May 20 16:27:12 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-5[48238]: 2024-05-20T16:27:12.328+0000 7fc3d0134700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-05-20T16:26:48.037517+0000 front 2024-05-20T16:26:48.037701+0000 (oldest deadline 2024-05-20T16:27:11.537505+0000) 2024-05-20T16:27:12.858 INFO:journalctl@ceph.osd.9.smithi082.stdout:May 20 16:27:12 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-9[66777]: 2024-05-20T16:27:12.587+0000 7fa187fc1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-05-20T16:26:44.585296+0000 front 2024-05-20T16:26:44.585359+0000 (oldest deadline 2024-05-20T16:27:09.885493+0000) 2024-05-20T16:27:12.859 INFO:journalctl@ceph.osd.3.smithi082.stdout:May 20 16:27:12 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-3[53439]: 2024-05-20T16:27:12.656+0000 7ff259df0700 -1 osd.3 102 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-05-20T16:26:48.461778+0000 front 2024-05-20T16:26:48.461807+0000 (oldest deadline 2024-05-20T16:27:10.761786+0000) 2024-05-20T16:27:13.635 INFO:journalctl@ceph.osd.5.smithi184.stdout:May 20 16:27:13 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-5[48238]: 2024-05-20T16:27:13.297+0000 7fc3d0134700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-05-20T16:26:48.037517+0000 front 2024-05-20T16:26:48.037701+0000 (oldest deadline 2024-05-20T16:27:11.537505+0000) 2024-05-20T16:27:13.635 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:27:13 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:27:13.288+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-05-20T16:26:47.217074+0000 front 2024-05-20T16:26:47.216883+0000 (oldest deadline 2024-05-20T16:27:10.716827+0000) 2024-05-20T16:27:13.859 INFO:journalctl@ceph.osd.3.smithi082.stdout:May 20 16:27:13 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-3[53439]: 2024-05-20T16:27:13.663+0000 7ff259df0700 -1 osd.3 102 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-05-20T16:26:48.461778+0000 front 2024-05-20T16:26:48.461807+0000 (oldest deadline 2024-05-20T16:27:10.761786+0000) 2024-05-20T16:27:13.859 INFO:journalctl@ceph.osd.9.smithi082.stdout:May 20 16:27:13 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-9[66777]: 2024-05-20T16:27:13.603+0000 7fa187fc1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-05-20T16:26:44.585296+0000 front 2024-05-20T16:26:44.585359+0000 (oldest deadline 2024-05-20T16:27:09.885493+0000) 2024-05-20T16:27:14.635 INFO:journalctl@ceph.osd.5.smithi184.stdout:May 20 16:27:14 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-5[48238]: 2024-05-20T16:27:14.273+0000 7fc3d0134700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-05-20T16:26:48.037517+0000 front 2024-05-20T16:26:48.037701+0000 (oldest deadline 2024-05-20T16:27:11.537505+0000) 2024-05-20T16:27:14.636 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:27:14 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:27:14.298+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-05-20T16:26:47.217074+0000 front 2024-05-20T16:26:47.216883+0000 (oldest deadline 2024-05-20T16:27:10.716827+0000) 2024-05-20T16:27:14.859 INFO:journalctl@ceph.osd.9.smithi082.stdout:May 20 16:27:14 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-9[66777]: 2024-05-20T16:27:14.562+0000 7fa187fc1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-05-20T16:26:44.585296+0000 front 2024-05-20T16:26:44.585359+0000 (oldest deadline 2024-05-20T16:27:09.885493+0000) 2024-05-20T16:27:15.314 INFO:journalctl@ceph.osd.3.smithi082.stdout:May 20 16:27:15 smithi082 podman[91965]: 2024-05-20 16:27:15.02026899 +0000 UTC m=+5.806700679 container died 4c47bfe5902b8a6cda50cfe53c85071742bbd693ab9dd2691170f00a4a14bfcf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-3, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, ceph=True, org.label-schema.build-date=20240514, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-05-20T16:27:15.582 INFO:journalctl@ceph.osd.3.smithi082.stdout:May 20 16:27:15 smithi082 podman[91965]: 2024-05-20 16:27:15.40331143 +0000 UTC m=+6.189743115 container cleanup 4c47bfe5902b8a6cda50cfe53c85071742bbd693ab9dd2691170f00a4a14bfcf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-3, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240514) 2024-05-20T16:27:15.582 INFO:journalctl@ceph.osd.3.smithi082.stdout:May 20 16:27:15 smithi082 bash[91965]: ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-3 2024-05-20T16:27:15.635 INFO:journalctl@ceph.osd.5.smithi184.stdout:May 20 16:27:15 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-5[48238]: 2024-05-20T16:27:15.233+0000 7fc3d0134700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-05-20T16:26:48.037517+0000 front 2024-05-20T16:26:48.037701+0000 (oldest deadline 2024-05-20T16:27:11.537505+0000) 2024-05-20T16:27:15.635 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:27:15 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:27:15.301+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-05-20T16:26:47.217074+0000 front 2024-05-20T16:26:47.216883+0000 (oldest deadline 2024-05-20T16:27:10.716827+0000) 2024-05-20T16:27:15.858 INFO:journalctl@ceph.osd.3.smithi082.stdout:May 20 16:27:15 smithi082 podman[91980]: 2024-05-20 16:27:15.671852265 +0000 UTC m=+0.649146814 container remove 4c47bfe5902b8a6cda50cfe53c85071742bbd693ab9dd2691170f00a4a14bfcf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-3, org.label-schema.build-date=20240514, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-20T16:27:15.859 INFO:journalctl@ceph.osd.9.smithi082.stdout:May 20 16:27:15 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-9[66777]: 2024-05-20T16:27:15.581+0000 7fa187fc1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-05-20T16:26:44.585296+0000 front 2024-05-20T16:26:44.585359+0000 (oldest deadline 2024-05-20T16:27:09.885493+0000) 2024-05-20T16:27:16.062 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:27:15 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:27:15.742+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-05-20T16:26:47.437555+0000 front 2024-05-20T16:26:47.437468+0000 (oldest deadline 2024-05-20T16:27:15.038423+0000) 2024-05-20T16:27:16.307 INFO:journalctl@ceph.osd.3.smithi082.stdout:May 20 16:27:16 smithi082 podman[92070]: 2024-05-20 16:27:15.918014416 +0000 UTC m=+0.017239133 image pull cfcc61b2d476f611479310eb042aa05f11dd5f60dbb65e3b0d7e0e11d024cdb3 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708 2024-05-20T16:27:16.308 INFO:journalctl@ceph.osd.3.smithi082.stdout:May 20 16:27:16 smithi082 podman[92070]: 2024-05-20 16:27:16.03938175 +0000 UTC m=+0.138606445 container create 5bdff397f64139b6539150c1fc8e6672ce1f2413d2b1ad4cf40ec6da504bc5a2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-3-deactivate, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240514, GIT_BRANCH=HEAD) 2024-05-20T16:27:16.308 INFO:journalctl@ceph.osd.3.smithi082.stdout:May 20 16:27:16 smithi082 podman[92070]: 2024-05-20 16:27:16.208618226 +0000 UTC m=+0.307842936 container init 5bdff397f64139b6539150c1fc8e6672ce1f2413d2b1ad4cf40ec6da504bc5a2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-3-deactivate, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240514, ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, GIT_BRANCH=HEAD) 2024-05-20T16:27:16.308 INFO:journalctl@ceph.osd.3.smithi082.stdout:May 20 16:27:16 smithi082 podman[92070]: 2024-05-20 16:27:16.212067682 +0000 UTC m=+0.311292384 container start 5bdff397f64139b6539150c1fc8e6672ce1f2413d2b1ad4cf40ec6da504bc5a2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-3-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, org.label-schema.build-date=20240514, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-20T16:27:16.308 INFO:journalctl@ceph.osd.3.smithi082.stdout:May 20 16:27:16 smithi082 podman[92070]: 2024-05-20 16:27:16.308517691 +0000 UTC m=+0.407742391 container attach 5bdff397f64139b6539150c1fc8e6672ce1f2413d2b1ad4cf40ec6da504bc5a2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-3-deactivate, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240514, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-05-20T16:27:16.581 INFO:journalctl@ceph.osd.3.smithi082.stdout:May 20 16:27:16 smithi082 podman[92070]: 2024-05-20 16:27:16.501426086 +0000 UTC m=+0.600650787 container died 5bdff397f64139b6539150c1fc8e6672ce1f2413d2b1ad4cf40ec6da504bc5a2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-3-deactivate, org.label-schema.build-date=20240514, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-05-20T16:27:16.635 INFO:journalctl@ceph.osd.5.smithi184.stdout:May 20 16:27:16 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-5[48238]: 2024-05-20T16:27:16.273+0000 7fc3d0134700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-05-20T16:26:48.037517+0000 front 2024-05-20T16:26:48.037701+0000 (oldest deadline 2024-05-20T16:27:11.537505+0000) 2024-05-20T16:27:16.635 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:27:16 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:27:16.321+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-05-20T16:26:47.217074+0000 front 2024-05-20T16:26:47.216883+0000 (oldest deadline 2024-05-20T16:27:10.716827+0000) 2024-05-20T16:27:16.858 INFO:journalctl@ceph.osd.9.smithi082.stdout:May 20 16:27:16 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-9[66777]: 2024-05-20T16:27:16.581+0000 7fa187fc1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-05-20T16:26:44.585296+0000 front 2024-05-20T16:26:44.585359+0000 (oldest deadline 2024-05-20T16:27:09.885493+0000) 2024-05-20T16:27:17.062 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:27:16 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:27:16.702+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-05-20T16:26:47.437555+0000 front 2024-05-20T16:26:47.437468+0000 (oldest deadline 2024-05-20T16:27:15.038423+0000) 2024-05-20T16:27:17.131 DEBUG:teuthology.orchestra.run.smithi082:> sudo pkill -f 'journalctl -f -n 0 -u ceph-59d1600a-16c2-11ef-bc9a-c7b262605968@osd.3.service' 2024-05-20T16:27:17.171 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-20T16:27:17.171 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-05-20T16:27:17.171 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-05-20T16:27:17.172 DEBUG:teuthology.orchestra.run.smithi160:> sudo systemctl stop ceph-59d1600a-16c2-11ef-bc9a-c7b262605968@osd.4 2024-05-20T16:27:17.486 INFO:journalctl@ceph.osd.4.smithi160.stdout:May 20 16:27:17 smithi160 systemd[1]: Stopping Ceph osd.4 for 59d1600a-16c2-11ef-bc9a-c7b262605968... 2024-05-20T16:27:17.635 INFO:journalctl@ceph.osd.5.smithi184.stdout:May 20 16:27:17 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-5[48238]: 2024-05-20T16:27:17.322+0000 7fc3d0134700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-05-20T16:26:48.037517+0000 front 2024-05-20T16:26:48.037701+0000 (oldest deadline 2024-05-20T16:27:11.537505+0000) 2024-05-20T16:27:17.635 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:27:17 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:27:17.283+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-05-20T16:26:47.217074+0000 front 2024-05-20T16:26:47.216883+0000 (oldest deadline 2024-05-20T16:27:10.716827+0000) 2024-05-20T16:27:17.811 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:27:17 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:27:17.718+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-05-20T16:26:47.437555+0000 front 2024-05-20T16:26:47.437468+0000 (oldest deadline 2024-05-20T16:27:15.038423+0000) 2024-05-20T16:27:17.812 INFO:journalctl@ceph.osd.4.smithi160.stdout:May 20 16:27:17 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-4[47371]: 2024-05-20T16:27:17.485+0000 7f60f4a5e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-20T16:27:17.812 INFO:journalctl@ceph.osd.4.smithi160.stdout:May 20 16:27:17 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-4[47371]: 2024-05-20T16:27:17.485+0000 7f60f4a5e700 -1 osd.4 102 *** Got signal Terminated *** 2024-05-20T16:27:17.812 INFO:journalctl@ceph.osd.4.smithi160.stdout:May 20 16:27:17 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-4[47371]: 2024-05-20T16:27:17.486+0000 7f60f4a5e700 -1 osd.4 102 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-20T16:27:17.858 INFO:journalctl@ceph.osd.9.smithi082.stdout:May 20 16:27:17 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-9[66777]: 2024-05-20T16:27:17.555+0000 7fa187fc1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-05-20T16:26:44.585296+0000 front 2024-05-20T16:26:44.585359+0000 (oldest deadline 2024-05-20T16:27:09.885493+0000) 2024-05-20T16:27:18.635 INFO:journalctl@ceph.osd.5.smithi184.stdout:May 20 16:27:18 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-5[48238]: 2024-05-20T16:27:18.344+0000 7fc3d0134700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-05-20T16:26:48.037517+0000 front 2024-05-20T16:26:48.037701+0000 (oldest deadline 2024-05-20T16:27:11.537505+0000) 2024-05-20T16:27:18.635 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:27:18 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:27:18.333+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-05-20T16:26:47.217074+0000 front 2024-05-20T16:26:47.216883+0000 (oldest deadline 2024-05-20T16:27:10.716827+0000) 2024-05-20T16:27:18.636 INFO:journalctl@ceph.osd.8.smithi184.stdout:May 20 16:27:18 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-8[55074]: 2024-05-20T16:27:18.315+0000 7f4a3856b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:52.470105+0000 front 2024-05-20T16:26:52.470039+0000 (oldest deadline 2024-05-20T16:27:18.270525+0000) 2024-05-20T16:27:18.859 INFO:journalctl@ceph.osd.9.smithi082.stdout:May 20 16:27:18 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-9[66777]: 2024-05-20T16:27:18.567+0000 7fa187fc1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-05-20T16:26:44.585296+0000 front 2024-05-20T16:26:44.585359+0000 (oldest deadline 2024-05-20T16:27:09.885493+0000) 2024-05-20T16:27:18.986 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:27:18 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:27:18.696+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-05-20T16:26:47.437555+0000 front 2024-05-20T16:26:47.437468+0000 (oldest deadline 2024-05-20T16:27:15.038423+0000) 2024-05-20T16:27:18.987 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:27:18 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:27:18.696+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:55.038908+0000 front 2024-05-20T16:26:55.038825+0000 (oldest deadline 2024-05-20T16:27:18.439444+0000) 2024-05-20T16:27:19.312 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 20 16:27:18 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-7[54382]: 2024-05-20T16:27:18.986+0000 7f68a2750700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:56.630029+0000 front 2024-05-20T16:26:56.630059+0000 (oldest deadline 2024-05-20T16:27:18.930178+0000) 2024-05-20T16:27:19.635 INFO:journalctl@ceph.osd.8.smithi184.stdout:May 20 16:27:19 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-8[55074]: 2024-05-20T16:27:19.335+0000 7f4a3856b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:52.470105+0000 front 2024-05-20T16:26:52.470039+0000 (oldest deadline 2024-05-20T16:27:18.270525+0000) 2024-05-20T16:27:19.636 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:27:19 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:27:19.313+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-05-20T16:26:47.217074+0000 front 2024-05-20T16:26:47.216883+0000 (oldest deadline 2024-05-20T16:27:10.716827+0000) 2024-05-20T16:27:19.636 INFO:journalctl@ceph.osd.5.smithi184.stdout:May 20 16:27:19 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-5[48238]: 2024-05-20T16:27:19.356+0000 7fc3d0134700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-05-20T16:26:48.037517+0000 front 2024-05-20T16:26:48.037701+0000 (oldest deadline 2024-05-20T16:27:11.537505+0000) 2024-05-20T16:27:19.636 INFO:journalctl@ceph.osd.5.smithi184.stdout:May 20 16:27:19 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-5[48238]: 2024-05-20T16:27:19.356+0000 7fc3d0134700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:56.238799+0000 front 2024-05-20T16:26:56.238849+0000 (oldest deadline 2024-05-20T16:27:18.538808+0000) 2024-05-20T16:27:19.859 INFO:journalctl@ceph.osd.9.smithi082.stdout:May 20 16:27:19 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-9[66777]: 2024-05-20T16:27:19.568+0000 7fa187fc1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-05-20T16:26:44.585296+0000 front 2024-05-20T16:26:44.585359+0000 (oldest deadline 2024-05-20T16:27:09.885493+0000) 2024-05-20T16:27:19.859 INFO:journalctl@ceph.osd.9.smithi082.stdout:May 20 16:27:19 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-9[66777]: 2024-05-20T16:27:19.568+0000 7fa187fc1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:55.786620+0000 front 2024-05-20T16:26:55.786679+0000 (oldest deadline 2024-05-20T16:27:19.286485+0000) 2024-05-20T16:27:19.982 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:27:19 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:27:19.692+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-05-20T16:26:47.437555+0000 front 2024-05-20T16:26:47.437468+0000 (oldest deadline 2024-05-20T16:27:15.038423+0000) 2024-05-20T16:27:19.982 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:27:19 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:27:19.692+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:55.038908+0000 front 2024-05-20T16:26:55.038825+0000 (oldest deadline 2024-05-20T16:27:18.439444+0000) 2024-05-20T16:27:20.312 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 20 16:27:19 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-7[54382]: 2024-05-20T16:27:19.982+0000 7f68a2750700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:56.630029+0000 front 2024-05-20T16:26:56.630059+0000 (oldest deadline 2024-05-20T16:27:18.930178+0000) 2024-05-20T16:27:20.587 INFO:journalctl@ceph.osd.6.smithi082.stdout:May 20 16:27:20 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-6[60057]: 2024-05-20T16:27:20.176+0000 7f3f143a5700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:56.025271+0000 front 2024-05-20T16:26:56.025414+0000 (oldest deadline 2024-05-20T16:27:20.125270+0000) 2024-05-20T16:27:20.635 INFO:journalctl@ceph.osd.8.smithi184.stdout:May 20 16:27:20 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-8[55074]: 2024-05-20T16:27:20.335+0000 7f4a3856b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:52.470105+0000 front 2024-05-20T16:26:52.470039+0000 (oldest deadline 2024-05-20T16:27:18.270525+0000) 2024-05-20T16:27:20.636 INFO:journalctl@ceph.osd.5.smithi184.stdout:May 20 16:27:20 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-5[48238]: 2024-05-20T16:27:20.312+0000 7fc3d0134700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-05-20T16:26:48.037517+0000 front 2024-05-20T16:26:48.037701+0000 (oldest deadline 2024-05-20T16:27:11.537505+0000) 2024-05-20T16:27:20.636 INFO:journalctl@ceph.osd.5.smithi184.stdout:May 20 16:27:20 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-5[48238]: 2024-05-20T16:27:20.312+0000 7fc3d0134700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:56.238799+0000 front 2024-05-20T16:26:56.238849+0000 (oldest deadline 2024-05-20T16:27:18.538808+0000) 2024-05-20T16:27:20.636 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:27:20 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:27:20.354+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-05-20T16:26:47.217074+0000 front 2024-05-20T16:26:47.216883+0000 (oldest deadline 2024-05-20T16:27:10.716827+0000) 2024-05-20T16:27:20.636 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:27:20 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:27:20.354+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:54.817899+0000 front 2024-05-20T16:26:54.818008+0000 (oldest deadline 2024-05-20T16:27:20.018619+0000) 2024-05-20T16:27:20.812 INFO:journalctl@ceph.osd.4.smithi160.stdout:May 20 16:27:20 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-4[47371]: 2024-05-20T16:27:20.487+0000 7f60f1014700 -1 osd.4 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:56.923427+0000 front 2024-05-20T16:26:56.923508+0000 (oldest deadline 2024-05-20T16:27:19.823579+0000) 2024-05-20T16:27:20.812 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:27:20 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:27:20.713+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-05-20T16:26:47.437555+0000 front 2024-05-20T16:26:47.437468+0000 (oldest deadline 2024-05-20T16:27:15.038423+0000) 2024-05-20T16:27:20.812 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:27:20 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:27:20.713+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:55.038908+0000 front 2024-05-20T16:26:55.038825+0000 (oldest deadline 2024-05-20T16:27:18.439444+0000) 2024-05-20T16:27:20.859 INFO:journalctl@ceph.osd.9.smithi082.stdout:May 20 16:27:20 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-9[66777]: 2024-05-20T16:27:20.587+0000 7fa187fc1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-05-20T16:26:44.585296+0000 front 2024-05-20T16:26:44.585359+0000 (oldest deadline 2024-05-20T16:27:09.885493+0000) 2024-05-20T16:27:20.859 INFO:journalctl@ceph.osd.9.smithi082.stdout:May 20 16:27:20 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-9[66777]: 2024-05-20T16:27:20.587+0000 7fa187fc1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:55.786620+0000 front 2024-05-20T16:26:55.786679+0000 (oldest deadline 2024-05-20T16:27:19.286485+0000) 2024-05-20T16:27:21.312 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 20 16:27:21 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-7[54382]: 2024-05-20T16:27:21.000+0000 7f68a2750700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:56.630029+0000 front 2024-05-20T16:26:56.630059+0000 (oldest deadline 2024-05-20T16:27:18.930178+0000) 2024-05-20T16:27:21.602 INFO:journalctl@ceph.osd.6.smithi082.stdout:May 20 16:27:21 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-6[60057]: 2024-05-20T16:27:21.204+0000 7f3f143a5700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:56.025271+0000 front 2024-05-20T16:26:56.025414+0000 (oldest deadline 2024-05-20T16:27:20.125270+0000) 2024-05-20T16:27:21.635 INFO:journalctl@ceph.osd.8.smithi184.stdout:May 20 16:27:21 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-8[55074]: 2024-05-20T16:27:21.372+0000 7f4a3856b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:52.470105+0000 front 2024-05-20T16:26:52.470039+0000 (oldest deadline 2024-05-20T16:27:18.270525+0000) 2024-05-20T16:27:21.636 INFO:journalctl@ceph.osd.5.smithi184.stdout:May 20 16:27:21 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-5[48238]: 2024-05-20T16:27:21.277+0000 7fc3d0134700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-05-20T16:26:48.037517+0000 front 2024-05-20T16:26:48.037701+0000 (oldest deadline 2024-05-20T16:27:11.537505+0000) 2024-05-20T16:27:21.636 INFO:journalctl@ceph.osd.5.smithi184.stdout:May 20 16:27:21 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-5[48238]: 2024-05-20T16:27:21.277+0000 7fc3d0134700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:56.238799+0000 front 2024-05-20T16:26:56.238849+0000 (oldest deadline 2024-05-20T16:27:18.538808+0000) 2024-05-20T16:27:21.636 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:27:21 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:27:21.394+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-05-20T16:26:47.217074+0000 front 2024-05-20T16:26:47.216883+0000 (oldest deadline 2024-05-20T16:27:10.716827+0000) 2024-05-20T16:27:21.636 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:27:21 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:27:21.394+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:54.817899+0000 front 2024-05-20T16:26:54.818008+0000 (oldest deadline 2024-05-20T16:27:20.018619+0000) 2024-05-20T16:27:21.811 INFO:journalctl@ceph.osd.4.smithi160.stdout:May 20 16:27:21 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-4[47371]: 2024-05-20T16:27:21.508+0000 7f60f1014700 -1 osd.4 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:56.923427+0000 front 2024-05-20T16:26:56.923508+0000 (oldest deadline 2024-05-20T16:27:19.823579+0000) 2024-05-20T16:27:21.812 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:27:21 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:27:21.729+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-05-20T16:26:47.437555+0000 front 2024-05-20T16:26:47.437468+0000 (oldest deadline 2024-05-20T16:27:15.038423+0000) 2024-05-20T16:27:21.812 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:27:21 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:27:21.729+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:55.038908+0000 front 2024-05-20T16:26:55.038825+0000 (oldest deadline 2024-05-20T16:27:18.439444+0000) 2024-05-20T16:27:21.858 INFO:journalctl@ceph.osd.9.smithi082.stdout:May 20 16:27:21 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-9[66777]: 2024-05-20T16:27:21.602+0000 7fa187fc1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-05-20T16:26:44.585296+0000 front 2024-05-20T16:26:44.585359+0000 (oldest deadline 2024-05-20T16:27:09.885493+0000) 2024-05-20T16:27:21.859 INFO:journalctl@ceph.osd.9.smithi082.stdout:May 20 16:27:21 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-9[66777]: 2024-05-20T16:27:21.602+0000 7fa187fc1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:55.786620+0000 front 2024-05-20T16:26:55.786679+0000 (oldest deadline 2024-05-20T16:27:19.286485+0000) 2024-05-20T16:27:22.312 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 20 16:27:22 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-7[54382]: 2024-05-20T16:27:22.015+0000 7f68a2750700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:56.630029+0000 front 2024-05-20T16:26:56.630059+0000 (oldest deadline 2024-05-20T16:27:18.930178+0000) 2024-05-20T16:27:22.590 INFO:journalctl@ceph.osd.6.smithi082.stdout:May 20 16:27:22 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-6[60057]: 2024-05-20T16:27:22.216+0000 7f3f143a5700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:56.025271+0000 front 2024-05-20T16:26:56.025414+0000 (oldest deadline 2024-05-20T16:27:20.125270+0000) 2024-05-20T16:27:22.635 INFO:journalctl@ceph.osd.8.smithi184.stdout:May 20 16:27:22 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-8[55074]: 2024-05-20T16:27:22.345+0000 7f4a3856b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:52.470105+0000 front 2024-05-20T16:26:52.470039+0000 (oldest deadline 2024-05-20T16:27:18.270525+0000) 2024-05-20T16:27:22.636 INFO:journalctl@ceph.osd.5.smithi184.stdout:May 20 16:27:22 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-5[48238]: 2024-05-20T16:27:22.234+0000 7fc3d0134700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-05-20T16:26:48.037517+0000 front 2024-05-20T16:26:48.037701+0000 (oldest deadline 2024-05-20T16:27:11.537505+0000) 2024-05-20T16:27:22.636 INFO:journalctl@ceph.osd.5.smithi184.stdout:May 20 16:27:22 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-5[48238]: 2024-05-20T16:27:22.234+0000 7fc3d0134700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:56.238799+0000 front 2024-05-20T16:26:56.238849+0000 (oldest deadline 2024-05-20T16:27:18.538808+0000) 2024-05-20T16:27:22.636 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:27:22 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:27:22.354+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-05-20T16:26:47.217074+0000 front 2024-05-20T16:26:47.216883+0000 (oldest deadline 2024-05-20T16:27:10.716827+0000) 2024-05-20T16:27:22.636 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:27:22 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:27:22.354+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:54.817899+0000 front 2024-05-20T16:26:54.818008+0000 (oldest deadline 2024-05-20T16:27:20.018619+0000) 2024-05-20T16:27:22.858 INFO:journalctl@ceph.osd.9.smithi082.stdout:May 20 16:27:22 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-9[66777]: 2024-05-20T16:27:22.590+0000 7fa187fc1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-05-20T16:26:44.585296+0000 front 2024-05-20T16:26:44.585359+0000 (oldest deadline 2024-05-20T16:27:09.885493+0000) 2024-05-20T16:27:22.859 INFO:journalctl@ceph.osd.9.smithi082.stdout:May 20 16:27:22 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-9[66777]: 2024-05-20T16:27:22.590+0000 7fa187fc1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:55.786620+0000 front 2024-05-20T16:26:55.786679+0000 (oldest deadline 2024-05-20T16:27:19.286485+0000) 2024-05-20T16:27:22.984 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:27:22 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:27:22.715+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-05-20T16:26:47.437555+0000 front 2024-05-20T16:26:47.437468+0000 (oldest deadline 2024-05-20T16:27:15.038423+0000) 2024-05-20T16:27:22.984 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:27:22 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:27:22.715+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:55.038908+0000 front 2024-05-20T16:26:55.038825+0000 (oldest deadline 2024-05-20T16:27:18.439444+0000) 2024-05-20T16:27:23.312 INFO:journalctl@ceph.osd.4.smithi160.stdout:May 20 16:27:23 smithi160 podman[71867]: 2024-05-20 16:27:23.146431385 +0000 UTC m=+5.780581766 container died 2ff7d831d05da8b6b1cf4ce551b18d87735a955116f29af5a0a2e0aa88b2f03b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-4, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240514, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux ) 2024-05-20T16:27:23.312 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 20 16:27:22 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-7[54382]: 2024-05-20T16:27:22.984+0000 7f68a2750700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:56.630029+0000 front 2024-05-20T16:26:56.630059+0000 (oldest deadline 2024-05-20T16:27:18.930178+0000) 2024-05-20T16:27:23.550 INFO:journalctl@ceph.osd.6.smithi082.stdout:May 20 16:27:23 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-6[60057]: 2024-05-20T16:27:23.231+0000 7f3f143a5700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:56.025271+0000 front 2024-05-20T16:26:56.025414+0000 (oldest deadline 2024-05-20T16:27:20.125270+0000) 2024-05-20T16:27:23.635 INFO:journalctl@ceph.osd.8.smithi184.stdout:May 20 16:27:23 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-8[55074]: 2024-05-20T16:27:23.304+0000 7f4a3856b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:52.470105+0000 front 2024-05-20T16:26:52.470039+0000 (oldest deadline 2024-05-20T16:27:18.270525+0000) 2024-05-20T16:27:23.636 INFO:journalctl@ceph.osd.5.smithi184.stdout:May 20 16:27:23 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-5[48238]: 2024-05-20T16:27:23.209+0000 7fc3d0134700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-05-20T16:26:48.037517+0000 front 2024-05-20T16:26:48.037701+0000 (oldest deadline 2024-05-20T16:27:11.537505+0000) 2024-05-20T16:27:23.636 INFO:journalctl@ceph.osd.5.smithi184.stdout:May 20 16:27:23 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-5[48238]: 2024-05-20T16:27:23.209+0000 7fc3d0134700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:56.238799+0000 front 2024-05-20T16:26:56.238849+0000 (oldest deadline 2024-05-20T16:27:18.538808+0000) 2024-05-20T16:27:23.636 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:27:23 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:27:23.349+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-05-20T16:26:47.217074+0000 front 2024-05-20T16:26:47.216883+0000 (oldest deadline 2024-05-20T16:27:10.716827+0000) 2024-05-20T16:27:23.636 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:27:23 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:27:23.349+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:54.817899+0000 front 2024-05-20T16:26:54.818008+0000 (oldest deadline 2024-05-20T16:27:20.018619+0000) 2024-05-20T16:27:23.705 INFO:journalctl@ceph.osd.4.smithi160.stdout:May 20 16:27:23 smithi160 podman[71867]: 2024-05-20 16:27:23.562213581 +0000 UTC m=+6.196363955 container cleanup 2ff7d831d05da8b6b1cf4ce551b18d87735a955116f29af5a0a2e0aa88b2f03b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-4, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240514, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-05-20T16:27:23.706 INFO:journalctl@ceph.osd.4.smithi160.stdout:May 20 16:27:23 smithi160 bash[71867]: ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-4 2024-05-20T16:27:23.858 INFO:journalctl@ceph.osd.9.smithi082.stdout:May 20 16:27:23 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-9[66777]: 2024-05-20T16:27:23.550+0000 7fa187fc1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-05-20T16:26:44.585296+0000 front 2024-05-20T16:26:44.585359+0000 (oldest deadline 2024-05-20T16:27:09.885493+0000) 2024-05-20T16:27:23.859 INFO:journalctl@ceph.osd.9.smithi082.stdout:May 20 16:27:23 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-9[66777]: 2024-05-20T16:27:23.550+0000 7fa187fc1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:55.786620+0000 front 2024-05-20T16:26:55.786679+0000 (oldest deadline 2024-05-20T16:27:19.286485+0000) 2024-05-20T16:27:24.023 INFO:journalctl@ceph.osd.4.smithi160.stdout:May 20 16:27:23 smithi160 podman[71884]: 2024-05-20 16:27:23.830517521 +0000 UTC m=+0.685233666 container remove 2ff7d831d05da8b6b1cf4ce551b18d87735a955116f29af5a0a2e0aa88b2f03b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-4, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240514, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS) 2024-05-20T16:27:24.023 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:27:23 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:27:23.705+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-05-20T16:26:47.437555+0000 front 2024-05-20T16:26:47.437468+0000 (oldest deadline 2024-05-20T16:27:15.038423+0000) 2024-05-20T16:27:24.023 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:27:23 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:27:23.705+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:55.038908+0000 front 2024-05-20T16:26:55.038825+0000 (oldest deadline 2024-05-20T16:27:18.439444+0000) 2024-05-20T16:27:24.312 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 20 16:27:24 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-7[54382]: 2024-05-20T16:27:24.022+0000 7f68a2750700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:56.630029+0000 front 2024-05-20T16:26:56.630059+0000 (oldest deadline 2024-05-20T16:27:18.930178+0000) 2024-05-20T16:27:24.535 INFO:journalctl@ceph.osd.6.smithi082.stdout:May 20 16:27:24 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-6[60057]: 2024-05-20T16:27:24.224+0000 7f3f143a5700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:56.025271+0000 front 2024-05-20T16:26:56.025414+0000 (oldest deadline 2024-05-20T16:27:20.125270+0000) 2024-05-20T16:27:24.608 INFO:journalctl@ceph.osd.4.smithi160.stdout:May 20 16:27:24 smithi160 podman[71974]: 2024-05-20 16:27:24.229149777 +0000 UTC m=+0.017370395 image pull cfcc61b2d476f611479310eb042aa05f11dd5f60dbb65e3b0d7e0e11d024cdb3 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708 2024-05-20T16:27:24.608 INFO:journalctl@ceph.osd.4.smithi160.stdout:May 20 16:27:24 smithi160 podman[71974]: 2024-05-20 16:27:24.364664421 +0000 UTC m=+0.152885019 container create d1cdb79284866fb99c6151401e9d58ee59b9c7dad7c6ca228acc340a03eed397 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-4-deactivate, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, ceph=True, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240514) 2024-05-20T16:27:24.635 INFO:journalctl@ceph.osd.8.smithi184.stdout:May 20 16:27:24 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-8[55074]: 2024-05-20T16:27:24.348+0000 7f4a3856b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:52.470105+0000 front 2024-05-20T16:26:52.470039+0000 (oldest deadline 2024-05-20T16:27:18.270525+0000) 2024-05-20T16:27:24.636 INFO:journalctl@ceph.osd.5.smithi184.stdout:May 20 16:27:24 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-5[48238]: 2024-05-20T16:27:24.257+0000 7fc3d0134700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-05-20T16:26:48.037517+0000 front 2024-05-20T16:26:48.037701+0000 (oldest deadline 2024-05-20T16:27:11.537505+0000) 2024-05-20T16:27:24.636 INFO:journalctl@ceph.osd.5.smithi184.stdout:May 20 16:27:24 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-5[48238]: 2024-05-20T16:27:24.257+0000 7fc3d0134700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:56.238799+0000 front 2024-05-20T16:26:56.238849+0000 (oldest deadline 2024-05-20T16:27:18.538808+0000) 2024-05-20T16:27:24.636 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:27:24 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:27:24.371+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-05-20T16:26:47.217074+0000 front 2024-05-20T16:26:47.216883+0000 (oldest deadline 2024-05-20T16:27:10.716827+0000) 2024-05-20T16:27:24.636 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:27:24 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:27:24.371+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:54.817899+0000 front 2024-05-20T16:26:54.818008+0000 (oldest deadline 2024-05-20T16:27:20.018619+0000) 2024-05-20T16:27:24.858 INFO:journalctl@ceph.osd.9.smithi082.stdout:May 20 16:27:24 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-9[66777]: 2024-05-20T16:27:24.535+0000 7fa187fc1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-05-20T16:26:44.585296+0000 front 2024-05-20T16:26:44.585359+0000 (oldest deadline 2024-05-20T16:27:09.885493+0000) 2024-05-20T16:27:24.859 INFO:journalctl@ceph.osd.9.smithi082.stdout:May 20 16:27:24 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-9[66777]: 2024-05-20T16:27:24.535+0000 7fa187fc1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:55.786620+0000 front 2024-05-20T16:26:55.786679+0000 (oldest deadline 2024-05-20T16:27:19.286485+0000) 2024-05-20T16:27:24.905 INFO:journalctl@ceph.osd.4.smithi160.stdout:May 20 16:27:24 smithi160 podman[71974]: 2024-05-20 16:27:24.608865302 +0000 UTC m=+0.397085901 container init d1cdb79284866fb99c6151401e9d58ee59b9c7dad7c6ca228acc340a03eed397 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-4-deactivate, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240514, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-05-20T16:27:24.905 INFO:journalctl@ceph.osd.4.smithi160.stdout:May 20 16:27:24 smithi160 podman[71974]: 2024-05-20 16:27:24.612298935 +0000 UTC m=+0.400519550 container start d1cdb79284866fb99c6151401e9d58ee59b9c7dad7c6ca228acc340a03eed397 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-4-deactivate, io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240514, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c) 2024-05-20T16:27:24.905 INFO:journalctl@ceph.osd.4.smithi160.stdout:May 20 16:27:24 smithi160 podman[71974]: 2024-05-20 16:27:24.717286717 +0000 UTC m=+0.505507324 container attach d1cdb79284866fb99c6151401e9d58ee59b9c7dad7c6ca228acc340a03eed397 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-4-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240514, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-05-20T16:27:24.905 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:27:24 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:27:24.701+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-05-20T16:26:47.437555+0000 front 2024-05-20T16:26:47.437468+0000 (oldest deadline 2024-05-20T16:27:15.038423+0000) 2024-05-20T16:27:24.906 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:27:24 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:27:24.701+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:55.038908+0000 front 2024-05-20T16:26:55.038825+0000 (oldest deadline 2024-05-20T16:27:18.439444+0000) 2024-05-20T16:27:25.269 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 20 16:27:24 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-7[54382]: 2024-05-20T16:27:24.991+0000 7f68a2750700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:56.630029+0000 front 2024-05-20T16:26:56.630059+0000 (oldest deadline 2024-05-20T16:27:18.930178+0000) 2024-05-20T16:27:25.269 INFO:journalctl@ceph.osd.4.smithi160.stdout:May 20 16:27:24 smithi160 podman[71974]: 2024-05-20 16:27:24.9060968 +0000 UTC m=+0.694317413 container died d1cdb79284866fb99c6151401e9d58ee59b9c7dad7c6ca228acc340a03eed397 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-4-deactivate, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240514, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2) 2024-05-20T16:27:25.515 INFO:journalctl@ceph.osd.6.smithi082.stdout:May 20 16:27:25 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-6[60057]: 2024-05-20T16:27:25.217+0000 7f3f143a5700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:56.025271+0000 front 2024-05-20T16:26:56.025414+0000 (oldest deadline 2024-05-20T16:27:20.125270+0000) 2024-05-20T16:27:25.621 DEBUG:teuthology.orchestra.run.smithi160:> sudo pkill -f 'journalctl -f -n 0 -u ceph-59d1600a-16c2-11ef-bc9a-c7b262605968@osd.4.service' 2024-05-20T16:27:25.635 INFO:journalctl@ceph.osd.8.smithi184.stdout:May 20 16:27:25 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-8[55074]: 2024-05-20T16:27:25.386+0000 7f4a3856b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:52.470105+0000 front 2024-05-20T16:26:52.470039+0000 (oldest deadline 2024-05-20T16:27:18.270525+0000) 2024-05-20T16:27:25.635 INFO:journalctl@ceph.osd.5.smithi184.stdout:May 20 16:27:25 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-5[48238]: 2024-05-20T16:27:25.222+0000 7fc3d0134700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-05-20T16:26:48.037517+0000 front 2024-05-20T16:26:48.037701+0000 (oldest deadline 2024-05-20T16:27:11.537505+0000) 2024-05-20T16:27:25.635 INFO:journalctl@ceph.osd.5.smithi184.stdout:May 20 16:27:25 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-5[48238]: 2024-05-20T16:27:25.222+0000 7fc3d0134700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:56.238799+0000 front 2024-05-20T16:26:56.238849+0000 (oldest deadline 2024-05-20T16:27:18.538808+0000) 2024-05-20T16:27:25.636 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:27:25 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:27:25.420+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-05-20T16:26:47.217074+0000 front 2024-05-20T16:26:47.216883+0000 (oldest deadline 2024-05-20T16:27:10.716827+0000) 2024-05-20T16:27:25.636 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:27:25 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:27:25.420+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:54.817899+0000 front 2024-05-20T16:26:54.818008+0000 (oldest deadline 2024-05-20T16:27:20.018619+0000) 2024-05-20T16:27:25.659 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-20T16:27:25.659 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-05-20T16:27:25.659 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-05-20T16:27:25.659 DEBUG:teuthology.orchestra.run.smithi184:> sudo systemctl stop ceph-59d1600a-16c2-11ef-bc9a-c7b262605968@osd.5 2024-05-20T16:27:25.858 INFO:journalctl@ceph.osd.9.smithi082.stdout:May 20 16:27:25 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-9[66777]: 2024-05-20T16:27:25.516+0000 7fa187fc1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-05-20T16:26:44.585296+0000 front 2024-05-20T16:26:44.585359+0000 (oldest deadline 2024-05-20T16:27:09.885493+0000) 2024-05-20T16:27:25.859 INFO:journalctl@ceph.osd.9.smithi082.stdout:May 20 16:27:25 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-9[66777]: 2024-05-20T16:27:25.516+0000 7fa187fc1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:55.786620+0000 front 2024-05-20T16:26:55.786679+0000 (oldest deadline 2024-05-20T16:27:19.286485+0000) 2024-05-20T16:27:25.967 INFO:journalctl@ceph.osd.5.smithi184.stdout:May 20 16:27:25 smithi184 systemd[1]: Stopping Ceph osd.5 for 59d1600a-16c2-11ef-bc9a-c7b262605968... 2024-05-20T16:27:25.987 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:27:25 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:27:25.704+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-05-20T16:26:47.437555+0000 front 2024-05-20T16:26:47.437468+0000 (oldest deadline 2024-05-20T16:27:15.038423+0000) 2024-05-20T16:27:25.987 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:27:25 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:27:25.704+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:55.038908+0000 front 2024-05-20T16:26:55.038825+0000 (oldest deadline 2024-05-20T16:27:18.439444+0000) 2024-05-20T16:27:26.312 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 20 16:27:25 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-7[54382]: 2024-05-20T16:27:25.987+0000 7f68a2750700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:56.630029+0000 front 2024-05-20T16:26:56.630059+0000 (oldest deadline 2024-05-20T16:27:18.930178+0000) 2024-05-20T16:27:26.385 INFO:journalctl@ceph.osd.5.smithi184.stdout:May 20 16:27:25 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-5[48238]: 2024-05-20T16:27:25.967+0000 7fc3d3b7e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-20T16:27:26.385 INFO:journalctl@ceph.osd.5.smithi184.stdout:May 20 16:27:25 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-5[48238]: 2024-05-20T16:27:25.967+0000 7fc3d3b7e700 -1 osd.5 102 *** Got signal Terminated *** 2024-05-20T16:27:26.385 INFO:journalctl@ceph.osd.5.smithi184.stdout:May 20 16:27:25 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-5[48238]: 2024-05-20T16:27:25.967+0000 7fc3d3b7e700 -1 osd.5 102 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-20T16:27:26.385 INFO:journalctl@ceph.osd.5.smithi184.stdout:May 20 16:27:26 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-5[48238]: 2024-05-20T16:27:26.208+0000 7fc3d0134700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-05-20T16:26:48.037517+0000 front 2024-05-20T16:26:48.037701+0000 (oldest deadline 2024-05-20T16:27:11.537505+0000) 2024-05-20T16:27:26.385 INFO:journalctl@ceph.osd.5.smithi184.stdout:May 20 16:27:26 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-5[48238]: 2024-05-20T16:27:26.208+0000 7fc3d0134700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:56.238799+0000 front 2024-05-20T16:26:56.238849+0000 (oldest deadline 2024-05-20T16:27:18.538808+0000) 2024-05-20T16:27:26.472 INFO:journalctl@ceph.osd.6.smithi082.stdout:May 20 16:27:26 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-6[60057]: 2024-05-20T16:27:26.197+0000 7f3f143a5700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:56.025271+0000 front 2024-05-20T16:26:56.025414+0000 (oldest deadline 2024-05-20T16:27:20.125270+0000) 2024-05-20T16:27:26.858 INFO:journalctl@ceph.osd.9.smithi082.stdout:May 20 16:27:26 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-9[66777]: 2024-05-20T16:27:26.473+0000 7fa187fc1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-05-20T16:26:44.585296+0000 front 2024-05-20T16:26:44.585359+0000 (oldest deadline 2024-05-20T16:27:09.885493+0000) 2024-05-20T16:27:26.859 INFO:journalctl@ceph.osd.9.smithi082.stdout:May 20 16:27:26 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-9[66777]: 2024-05-20T16:27:26.473+0000 7fa187fc1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:55.786620+0000 front 2024-05-20T16:26:55.786679+0000 (oldest deadline 2024-05-20T16:27:19.286485+0000) 2024-05-20T16:27:26.885 INFO:journalctl@ceph.osd.8.smithi184.stdout:May 20 16:27:26 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-8[55074]: 2024-05-20T16:27:26.421+0000 7f4a3856b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:52.470105+0000 front 2024-05-20T16:26:52.470039+0000 (oldest deadline 2024-05-20T16:27:18.270525+0000) 2024-05-20T16:27:26.885 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:27:26 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:27:26.412+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-05-20T16:26:47.217074+0000 front 2024-05-20T16:26:47.216883+0000 (oldest deadline 2024-05-20T16:27:10.716827+0000) 2024-05-20T16:27:26.885 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:27:26 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:27:26.412+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:54.817899+0000 front 2024-05-20T16:26:54.818008+0000 (oldest deadline 2024-05-20T16:27:20.018619+0000) 2024-05-20T16:27:27.004 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:27:26 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:27:26.681+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-05-20T16:26:47.437555+0000 front 2024-05-20T16:26:47.437468+0000 (oldest deadline 2024-05-20T16:27:15.038423+0000) 2024-05-20T16:27:27.005 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:27:26 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:27:26.681+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:55.038908+0000 front 2024-05-20T16:26:55.038825+0000 (oldest deadline 2024-05-20T16:27:18.439444+0000) 2024-05-20T16:27:27.311 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 20 16:27:27 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-7[54382]: 2024-05-20T16:27:27.005+0000 7f68a2750700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:56.630029+0000 front 2024-05-20T16:26:56.630059+0000 (oldest deadline 2024-05-20T16:27:18.930178+0000) 2024-05-20T16:27:27.430 INFO:journalctl@ceph.osd.5.smithi184.stdout:May 20 16:27:27 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-5[48238]: 2024-05-20T16:27:27.167+0000 7fc3d0134700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-05-20T16:26:48.037517+0000 front 2024-05-20T16:26:48.037701+0000 (oldest deadline 2024-05-20T16:27:11.537505+0000) 2024-05-20T16:27:27.430 INFO:journalctl@ceph.osd.5.smithi184.stdout:May 20 16:27:27 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-5[48238]: 2024-05-20T16:27:27.167+0000 7fc3d0134700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:56.238799+0000 front 2024-05-20T16:26:56.238849+0000 (oldest deadline 2024-05-20T16:27:18.538808+0000) 2024-05-20T16:27:27.430 INFO:journalctl@ceph.osd.5.smithi184.stdout:May 20 16:27:27 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-5[48238]: 2024-05-20T16:27:27.167+0000 7fc3d0134700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-20T16:27:01.439694+0000 front 2024-05-20T16:27:01.439876+0000 (oldest deadline 2024-05-20T16:27:26.739985+0000) 2024-05-20T16:27:27.511 INFO:journalctl@ceph.osd.6.smithi082.stdout:May 20 16:27:27 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-6[60057]: 2024-05-20T16:27:27.165+0000 7f3f143a5700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:56.025271+0000 front 2024-05-20T16:26:56.025414+0000 (oldest deadline 2024-05-20T16:27:20.125270+0000) 2024-05-20T16:27:27.858 INFO:journalctl@ceph.osd.9.smithi082.stdout:May 20 16:27:27 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-9[66777]: 2024-05-20T16:27:27.511+0000 7fa187fc1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-05-20T16:26:44.585296+0000 front 2024-05-20T16:26:44.585359+0000 (oldest deadline 2024-05-20T16:27:09.885493+0000) 2024-05-20T16:27:27.859 INFO:journalctl@ceph.osd.9.smithi082.stdout:May 20 16:27:27 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-9[66777]: 2024-05-20T16:27:27.511+0000 7fa187fc1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:55.786620+0000 front 2024-05-20T16:26:55.786679+0000 (oldest deadline 2024-05-20T16:27:19.286485+0000) 2024-05-20T16:27:27.885 INFO:journalctl@ceph.osd.8.smithi184.stdout:May 20 16:27:27 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-8[55074]: 2024-05-20T16:27:27.441+0000 7f4a3856b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:52.470105+0000 front 2024-05-20T16:26:52.470039+0000 (oldest deadline 2024-05-20T16:27:18.270525+0000) 2024-05-20T16:27:27.885 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:27:27 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:27:27.430+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-05-20T16:26:47.217074+0000 front 2024-05-20T16:26:47.216883+0000 (oldest deadline 2024-05-20T16:27:10.716827+0000) 2024-05-20T16:27:27.885 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:27:27 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:27:27.430+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:54.817899+0000 front 2024-05-20T16:26:54.818008+0000 (oldest deadline 2024-05-20T16:27:20.018619+0000) 2024-05-20T16:27:27.885 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:27:27 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:27:27.430+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-20T16:27:01.719536+0000 front 2024-05-20T16:27:01.719675+0000 (oldest deadline 2024-05-20T16:27:27.019603+0000) 2024-05-20T16:27:28.014 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:27:27 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:27:27.704+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-05-20T16:26:47.437555+0000 front 2024-05-20T16:26:47.437468+0000 (oldest deadline 2024-05-20T16:27:15.038423+0000) 2024-05-20T16:27:28.014 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:27:27 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:27:27.704+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:55.038908+0000 front 2024-05-20T16:26:55.038825+0000 (oldest deadline 2024-05-20T16:27:18.439444+0000) 2024-05-20T16:27:28.014 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:27:27 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:27:27.704+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-20T16:27:04.241075+0000 front 2024-05-20T16:27:04.241297+0000 (oldest deadline 2024-05-20T16:27:27.141055+0000) 2024-05-20T16:27:28.311 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 20 16:27:28 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-7[54382]: 2024-05-20T16:27:28.014+0000 7f68a2750700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:56.630029+0000 front 2024-05-20T16:26:56.630059+0000 (oldest deadline 2024-05-20T16:27:18.930178+0000) 2024-05-20T16:27:28.411 INFO:journalctl@ceph.osd.5.smithi184.stdout:May 20 16:27:28 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-5[48238]: 2024-05-20T16:27:28.136+0000 7fc3d0134700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-05-20T16:26:48.037517+0000 front 2024-05-20T16:26:48.037701+0000 (oldest deadline 2024-05-20T16:27:11.537505+0000) 2024-05-20T16:27:28.411 INFO:journalctl@ceph.osd.5.smithi184.stdout:May 20 16:27:28 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-5[48238]: 2024-05-20T16:27:28.136+0000 7fc3d0134700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:56.238799+0000 front 2024-05-20T16:26:56.238849+0000 (oldest deadline 2024-05-20T16:27:18.538808+0000) 2024-05-20T16:27:28.412 INFO:journalctl@ceph.osd.5.smithi184.stdout:May 20 16:27:28 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-5[48238]: 2024-05-20T16:27:28.136+0000 7fc3d0134700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-20T16:27:01.439694+0000 front 2024-05-20T16:27:01.439876+0000 (oldest deadline 2024-05-20T16:27:26.739985+0000) 2024-05-20T16:27:28.470 INFO:journalctl@ceph.osd.6.smithi082.stdout:May 20 16:27:28 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-6[60057]: 2024-05-20T16:27:28.128+0000 7f3f143a5700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:56.025271+0000 front 2024-05-20T16:26:56.025414+0000 (oldest deadline 2024-05-20T16:27:20.125270+0000) 2024-05-20T16:27:28.859 INFO:journalctl@ceph.osd.9.smithi082.stdout:May 20 16:27:28 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-9[66777]: 2024-05-20T16:27:28.471+0000 7fa187fc1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-05-20T16:26:44.585296+0000 front 2024-05-20T16:26:44.585359+0000 (oldest deadline 2024-05-20T16:27:09.885493+0000) 2024-05-20T16:27:28.859 INFO:journalctl@ceph.osd.9.smithi082.stdout:May 20 16:27:28 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-9[66777]: 2024-05-20T16:27:28.471+0000 7fa187fc1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:55.786620+0000 front 2024-05-20T16:26:55.786679+0000 (oldest deadline 2024-05-20T16:27:19.286485+0000) 2024-05-20T16:27:28.885 INFO:journalctl@ceph.osd.8.smithi184.stdout:May 20 16:27:28 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-8[55074]: 2024-05-20T16:27:28.451+0000 7f4a3856b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:52.470105+0000 front 2024-05-20T16:26:52.470039+0000 (oldest deadline 2024-05-20T16:27:18.270525+0000) 2024-05-20T16:27:28.885 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:27:28 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:27:28.412+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-05-20T16:26:47.217074+0000 front 2024-05-20T16:26:47.216883+0000 (oldest deadline 2024-05-20T16:27:10.716827+0000) 2024-05-20T16:27:28.886 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:27:28 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:27:28.412+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:54.817899+0000 front 2024-05-20T16:26:54.818008+0000 (oldest deadline 2024-05-20T16:27:20.018619+0000) 2024-05-20T16:27:28.886 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:27:28 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:27:28.412+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-20T16:27:01.719536+0000 front 2024-05-20T16:27:01.719675+0000 (oldest deadline 2024-05-20T16:27:27.019603+0000) 2024-05-20T16:27:28.976 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:27:28 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:27:28.675+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-05-20T16:26:47.437555+0000 front 2024-05-20T16:26:47.437468+0000 (oldest deadline 2024-05-20T16:27:15.038423+0000) 2024-05-20T16:27:28.976 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:27:28 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:27:28.675+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:55.038908+0000 front 2024-05-20T16:26:55.038825+0000 (oldest deadline 2024-05-20T16:27:18.439444+0000) 2024-05-20T16:27:28.976 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:27:28 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:27:28.675+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-20T16:27:04.241075+0000 front 2024-05-20T16:27:04.241297+0000 (oldest deadline 2024-05-20T16:27:27.141055+0000) 2024-05-20T16:27:29.311 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 20 16:27:28 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-7[54382]: 2024-05-20T16:27:28.976+0000 7f68a2750700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:56.630029+0000 front 2024-05-20T16:26:56.630059+0000 (oldest deadline 2024-05-20T16:27:18.930178+0000) 2024-05-20T16:27:29.430 INFO:journalctl@ceph.osd.5.smithi184.stdout:May 20 16:27:29 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-5[48238]: 2024-05-20T16:27:29.156+0000 7fc3d0134700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-05-20T16:26:48.037517+0000 front 2024-05-20T16:26:48.037701+0000 (oldest deadline 2024-05-20T16:27:11.537505+0000) 2024-05-20T16:27:29.430 INFO:journalctl@ceph.osd.5.smithi184.stdout:May 20 16:27:29 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-5[48238]: 2024-05-20T16:27:29.156+0000 7fc3d0134700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:56.238799+0000 front 2024-05-20T16:26:56.238849+0000 (oldest deadline 2024-05-20T16:27:18.538808+0000) 2024-05-20T16:27:29.430 INFO:journalctl@ceph.osd.5.smithi184.stdout:May 20 16:27:29 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-5[48238]: 2024-05-20T16:27:29.156+0000 7fc3d0134700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-20T16:27:01.439694+0000 front 2024-05-20T16:27:01.439876+0000 (oldest deadline 2024-05-20T16:27:26.739985+0000) 2024-05-20T16:27:29.448 INFO:journalctl@ceph.osd.6.smithi082.stdout:May 20 16:27:29 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-6[60057]: 2024-05-20T16:27:29.164+0000 7f3f143a5700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:56.025271+0000 front 2024-05-20T16:26:56.025414+0000 (oldest deadline 2024-05-20T16:27:20.125270+0000) 2024-05-20T16:27:29.448 INFO:journalctl@ceph.osd.6.smithi082.stdout:May 20 16:27:29 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-6[60057]: 2024-05-20T16:27:29.164+0000 7f3f143a5700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-20T16:27:04.826835+0000 front 2024-05-20T16:27:04.826693+0000 (oldest deadline 2024-05-20T16:27:28.326771+0000) 2024-05-20T16:27:29.816 INFO:journalctl@ceph.osd.9.smithi082.stdout:May 20 16:27:29 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-9[66777]: 2024-05-20T16:27:29.448+0000 7fa187fc1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-05-20T16:26:44.585296+0000 front 2024-05-20T16:26:44.585359+0000 (oldest deadline 2024-05-20T16:27:09.885493+0000) 2024-05-20T16:27:29.816 INFO:journalctl@ceph.osd.9.smithi082.stdout:May 20 16:27:29 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-9[66777]: 2024-05-20T16:27:29.448+0000 7fa187fc1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:55.786620+0000 front 2024-05-20T16:26:55.786679+0000 (oldest deadline 2024-05-20T16:27:19.286485+0000) 2024-05-20T16:27:29.816 INFO:journalctl@ceph.osd.9.smithi082.stdout:May 20 16:27:29 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-9[66777]: 2024-05-20T16:27:29.448+0000 7fa187fc1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-20T16:27:03.987755+0000 front 2024-05-20T16:27:03.987552+0000 (oldest deadline 2024-05-20T16:27:28.687562+0000) 2024-05-20T16:27:29.885 INFO:journalctl@ceph.osd.8.smithi184.stdout:May 20 16:27:29 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-8[55074]: 2024-05-20T16:27:29.489+0000 7f4a3856b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:52.470105+0000 front 2024-05-20T16:26:52.470039+0000 (oldest deadline 2024-05-20T16:27:18.270525+0000) 2024-05-20T16:27:29.885 INFO:journalctl@ceph.osd.8.smithi184.stdout:May 20 16:27:29 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-8[55074]: 2024-05-20T16:27:29.489+0000 7f4a3856b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-20T16:27:05.271959+0000 front 2024-05-20T16:27:05.272026+0000 (oldest deadline 2024-05-20T16:27:28.772153+0000) 2024-05-20T16:27:29.886 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:27:29 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:27:29.430+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-05-20T16:26:47.217074+0000 front 2024-05-20T16:26:47.216883+0000 (oldest deadline 2024-05-20T16:27:10.716827+0000) 2024-05-20T16:27:29.886 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:27:29 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:27:29.430+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:54.817899+0000 front 2024-05-20T16:26:54.818008+0000 (oldest deadline 2024-05-20T16:27:20.018619+0000) 2024-05-20T16:27:29.886 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:27:29 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:27:29.430+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-20T16:27:01.719536+0000 front 2024-05-20T16:27:01.719675+0000 (oldest deadline 2024-05-20T16:27:27.019603+0000) 2024-05-20T16:27:29.983 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:27:29 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:27:29.683+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-05-20T16:26:47.437555+0000 front 2024-05-20T16:26:47.437468+0000 (oldest deadline 2024-05-20T16:27:15.038423+0000) 2024-05-20T16:27:29.983 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:27:29 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:27:29.683+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:55.038908+0000 front 2024-05-20T16:26:55.038825+0000 (oldest deadline 2024-05-20T16:27:18.439444+0000) 2024-05-20T16:27:29.983 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:27:29 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:27:29.683+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-20T16:27:04.241075+0000 front 2024-05-20T16:27:04.241297+0000 (oldest deadline 2024-05-20T16:27:27.141055+0000) 2024-05-20T16:27:30.312 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 20 16:27:29 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-7[54382]: 2024-05-20T16:27:29.983+0000 7f68a2750700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:56.630029+0000 front 2024-05-20T16:26:56.630059+0000 (oldest deadline 2024-05-20T16:27:18.930178+0000) 2024-05-20T16:27:30.312 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 20 16:27:29 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-7[54382]: 2024-05-20T16:27:29.983+0000 7f68a2750700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-20T16:27:04.032669+0000 front 2024-05-20T16:27:04.032493+0000 (oldest deadline 2024-05-20T16:27:29.332344+0000) 2024-05-20T16:27:30.428 INFO:journalctl@ceph.osd.5.smithi184.stdout:May 20 16:27:30 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-5[48238]: 2024-05-20T16:27:30.172+0000 7fc3d0134700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-05-20T16:26:48.037517+0000 front 2024-05-20T16:26:48.037701+0000 (oldest deadline 2024-05-20T16:27:11.537505+0000) 2024-05-20T16:27:30.428 INFO:journalctl@ceph.osd.5.smithi184.stdout:May 20 16:27:30 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-5[48238]: 2024-05-20T16:27:30.172+0000 7fc3d0134700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:56.238799+0000 front 2024-05-20T16:26:56.238849+0000 (oldest deadline 2024-05-20T16:27:18.538808+0000) 2024-05-20T16:27:30.429 INFO:journalctl@ceph.osd.5.smithi184.stdout:May 20 16:27:30 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-5[48238]: 2024-05-20T16:27:30.172+0000 7fc3d0134700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-20T16:27:01.439694+0000 front 2024-05-20T16:27:01.439876+0000 (oldest deadline 2024-05-20T16:27:26.739985+0000) 2024-05-20T16:27:30.474 INFO:journalctl@ceph.osd.6.smithi082.stdout:May 20 16:27:30 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-6[60057]: 2024-05-20T16:27:30.203+0000 7f3f143a5700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:56.025271+0000 front 2024-05-20T16:26:56.025414+0000 (oldest deadline 2024-05-20T16:27:20.125270+0000) 2024-05-20T16:27:30.474 INFO:journalctl@ceph.osd.6.smithi082.stdout:May 20 16:27:30 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-6[60057]: 2024-05-20T16:27:30.203+0000 7f3f143a5700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-20T16:27:04.826835+0000 front 2024-05-20T16:27:04.826693+0000 (oldest deadline 2024-05-20T16:27:28.326771+0000) 2024-05-20T16:27:30.859 INFO:journalctl@ceph.osd.9.smithi082.stdout:May 20 16:27:30 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-9[66777]: 2024-05-20T16:27:30.474+0000 7fa187fc1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-05-20T16:26:44.585296+0000 front 2024-05-20T16:26:44.585359+0000 (oldest deadline 2024-05-20T16:27:09.885493+0000) 2024-05-20T16:27:30.859 INFO:journalctl@ceph.osd.9.smithi082.stdout:May 20 16:27:30 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-9[66777]: 2024-05-20T16:27:30.474+0000 7fa187fc1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:55.786620+0000 front 2024-05-20T16:26:55.786679+0000 (oldest deadline 2024-05-20T16:27:19.286485+0000) 2024-05-20T16:27:30.859 INFO:journalctl@ceph.osd.9.smithi082.stdout:May 20 16:27:30 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-9[66777]: 2024-05-20T16:27:30.474+0000 7fa187fc1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-20T16:27:03.987755+0000 front 2024-05-20T16:27:03.987552+0000 (oldest deadline 2024-05-20T16:27:28.687562+0000) 2024-05-20T16:27:30.885 INFO:journalctl@ceph.osd.8.smithi184.stdout:May 20 16:27:30 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-8[55074]: 2024-05-20T16:27:30.538+0000 7f4a3856b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:52.470105+0000 front 2024-05-20T16:26:52.470039+0000 (oldest deadline 2024-05-20T16:27:18.270525+0000) 2024-05-20T16:27:30.885 INFO:journalctl@ceph.osd.8.smithi184.stdout:May 20 16:27:30 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-8[55074]: 2024-05-20T16:27:30.538+0000 7f4a3856b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-20T16:27:05.271959+0000 front 2024-05-20T16:27:05.272026+0000 (oldest deadline 2024-05-20T16:27:28.772153+0000) 2024-05-20T16:27:30.886 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:27:30 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:27:30.428+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-05-20T16:26:47.217074+0000 front 2024-05-20T16:26:47.216883+0000 (oldest deadline 2024-05-20T16:27:10.716827+0000) 2024-05-20T16:27:30.886 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:27:30 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:27:30.428+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:54.817899+0000 front 2024-05-20T16:26:54.818008+0000 (oldest deadline 2024-05-20T16:27:20.018619+0000) 2024-05-20T16:27:30.886 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:27:30 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:27:30.428+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-20T16:27:01.719536+0000 front 2024-05-20T16:27:01.719675+0000 (oldest deadline 2024-05-20T16:27:27.019603+0000) 2024-05-20T16:27:30.944 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:27:30 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:27:30.676+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-05-20T16:26:47.437555+0000 front 2024-05-20T16:26:47.437468+0000 (oldest deadline 2024-05-20T16:27:15.038423+0000) 2024-05-20T16:27:30.945 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:27:30 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:27:30.676+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:55.038908+0000 front 2024-05-20T16:26:55.038825+0000 (oldest deadline 2024-05-20T16:27:18.439444+0000) 2024-05-20T16:27:30.945 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:27:30 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:27:30.676+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-20T16:27:04.241075+0000 front 2024-05-20T16:27:04.241297+0000 (oldest deadline 2024-05-20T16:27:27.141055+0000) 2024-05-20T16:27:31.312 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 20 16:27:30 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-7[54382]: 2024-05-20T16:27:30.945+0000 7f68a2750700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:56.630029+0000 front 2024-05-20T16:26:56.630059+0000 (oldest deadline 2024-05-20T16:27:18.930178+0000) 2024-05-20T16:27:31.312 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 20 16:27:30 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-7[54382]: 2024-05-20T16:27:30.945+0000 7f68a2750700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-20T16:27:04.032669+0000 front 2024-05-20T16:27:04.032493+0000 (oldest deadline 2024-05-20T16:27:29.332344+0000) 2024-05-20T16:27:31.609 INFO:journalctl@ceph.osd.6.smithi082.stdout:May 20 16:27:31 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-6[60057]: 2024-05-20T16:27:31.206+0000 7f3f143a5700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:56.025271+0000 front 2024-05-20T16:26:56.025414+0000 (oldest deadline 2024-05-20T16:27:20.125270+0000) 2024-05-20T16:27:31.609 INFO:journalctl@ceph.osd.6.smithi082.stdout:May 20 16:27:31 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-6[60057]: 2024-05-20T16:27:31.206+0000 7f3f143a5700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-20T16:27:04.826835+0000 front 2024-05-20T16:27:04.826693+0000 (oldest deadline 2024-05-20T16:27:28.326771+0000) 2024-05-20T16:27:31.609 INFO:journalctl@ceph.osd.9.smithi082.stdout:May 20 16:27:31 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-9[66777]: 2024-05-20T16:27:31.455+0000 7fa187fc1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-05-20T16:26:44.585296+0000 front 2024-05-20T16:26:44.585359+0000 (oldest deadline 2024-05-20T16:27:09.885493+0000) 2024-05-20T16:27:31.609 INFO:journalctl@ceph.osd.9.smithi082.stdout:May 20 16:27:31 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-9[66777]: 2024-05-20T16:27:31.455+0000 7fa187fc1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:55.786620+0000 front 2024-05-20T16:26:55.786679+0000 (oldest deadline 2024-05-20T16:27:19.286485+0000) 2024-05-20T16:27:31.609 INFO:journalctl@ceph.osd.9.smithi082.stdout:May 20 16:27:31 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-9[66777]: 2024-05-20T16:27:31.455+0000 7fa187fc1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-20T16:27:03.987755+0000 front 2024-05-20T16:27:03.987552+0000 (oldest deadline 2024-05-20T16:27:28.687562+0000) 2024-05-20T16:27:31.787 INFO:journalctl@ceph.osd.8.smithi184.stdout:May 20 16:27:31 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-8[55074]: 2024-05-20T16:27:31.517+0000 7f4a3856b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:52.470105+0000 front 2024-05-20T16:26:52.470039+0000 (oldest deadline 2024-05-20T16:27:18.270525+0000) 2024-05-20T16:27:31.787 INFO:journalctl@ceph.osd.8.smithi184.stdout:May 20 16:27:31 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-8[55074]: 2024-05-20T16:27:31.517+0000 7f4a3856b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-20T16:27:05.271959+0000 front 2024-05-20T16:27:05.272026+0000 (oldest deadline 2024-05-20T16:27:28.772153+0000) 2024-05-20T16:27:31.787 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:27:31 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:27:31.392+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-05-20T16:26:47.217074+0000 front 2024-05-20T16:26:47.216883+0000 (oldest deadline 2024-05-20T16:27:10.716827+0000) 2024-05-20T16:27:31.787 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:27:31 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:27:31.392+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:54.817899+0000 front 2024-05-20T16:26:54.818008+0000 (oldest deadline 2024-05-20T16:27:20.018619+0000) 2024-05-20T16:27:31.787 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:27:31 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:27:31.392+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-20T16:27:01.719536+0000 front 2024-05-20T16:27:01.719675+0000 (oldest deadline 2024-05-20T16:27:27.019603+0000) 2024-05-20T16:27:31.933 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:27:31 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:27:31.679+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-05-20T16:26:47.437555+0000 front 2024-05-20T16:26:47.437468+0000 (oldest deadline 2024-05-20T16:27:15.038423+0000) 2024-05-20T16:27:31.933 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:27:31 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:27:31.679+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:55.038908+0000 front 2024-05-20T16:26:55.038825+0000 (oldest deadline 2024-05-20T16:27:18.439444+0000) 2024-05-20T16:27:31.933 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:27:31 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:27:31.679+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-20T16:27:04.241075+0000 front 2024-05-20T16:27:04.241297+0000 (oldest deadline 2024-05-20T16:27:27.141055+0000) 2024-05-20T16:27:32.085 INFO:journalctl@ceph.osd.5.smithi184.stdout:May 20 16:27:31 smithi184 podman[73204]: 2024-05-20 16:27:31.803936728 +0000 UTC m=+5.953137254 container died 70cb5e21cdb7f65066ac5acadec755b19c8e91706c874856971dd8279d0d9058 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-5, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240514, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-20T16:27:32.312 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 20 16:27:31 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-7[54382]: 2024-05-20T16:27:31.933+0000 7f68a2750700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:56.630029+0000 front 2024-05-20T16:26:56.630059+0000 (oldest deadline 2024-05-20T16:27:18.930178+0000) 2024-05-20T16:27:32.312 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 20 16:27:31 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-7[54382]: 2024-05-20T16:27:31.933+0000 7f68a2750700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-20T16:27:04.032669+0000 front 2024-05-20T16:27:04.032493+0000 (oldest deadline 2024-05-20T16:27:29.332344+0000) 2024-05-20T16:27:32.354 INFO:journalctl@ceph.osd.5.smithi184.stdout:May 20 16:27:32 smithi184 podman[73204]: 2024-05-20 16:27:32.176399823 +0000 UTC m=+6.325600351 container cleanup 70cb5e21cdb7f65066ac5acadec755b19c8e91706c874856971dd8279d0d9058 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, org.label-schema.build-date=20240514, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-05-20T16:27:32.354 INFO:journalctl@ceph.osd.5.smithi184.stdout:May 20 16:27:32 smithi184 bash[73204]: ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-5 2024-05-20T16:27:32.609 INFO:journalctl@ceph.osd.6.smithi082.stdout:May 20 16:27:32 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-6[60057]: 2024-05-20T16:27:32.229+0000 7f3f143a5700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:56.025271+0000 front 2024-05-20T16:26:56.025414+0000 (oldest deadline 2024-05-20T16:27:20.125270+0000) 2024-05-20T16:27:32.609 INFO:journalctl@ceph.osd.6.smithi082.stdout:May 20 16:27:32 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-6[60057]: 2024-05-20T16:27:32.229+0000 7f3f143a5700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-20T16:27:04.826835+0000 front 2024-05-20T16:27:04.826693+0000 (oldest deadline 2024-05-20T16:27:28.326771+0000) 2024-05-20T16:27:32.609 INFO:journalctl@ceph.osd.9.smithi082.stdout:May 20 16:27:32 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-9[66777]: 2024-05-20T16:27:32.467+0000 7fa187fc1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-05-20T16:26:44.585296+0000 front 2024-05-20T16:26:44.585359+0000 (oldest deadline 2024-05-20T16:27:09.885493+0000) 2024-05-20T16:27:32.609 INFO:journalctl@ceph.osd.9.smithi082.stdout:May 20 16:27:32 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-9[66777]: 2024-05-20T16:27:32.467+0000 7fa187fc1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:55.786620+0000 front 2024-05-20T16:26:55.786679+0000 (oldest deadline 2024-05-20T16:27:19.286485+0000) 2024-05-20T16:27:32.609 INFO:journalctl@ceph.osd.9.smithi082.stdout:May 20 16:27:32 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-9[66777]: 2024-05-20T16:27:32.467+0000 7fa187fc1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-20T16:27:03.987755+0000 front 2024-05-20T16:27:03.987552+0000 (oldest deadline 2024-05-20T16:27:28.687562+0000) 2024-05-20T16:27:32.635 INFO:journalctl@ceph.osd.5.smithi184.stdout:May 20 16:27:32 smithi184 podman[73219]: 2024-05-20 16:27:32.444797841 +0000 UTC m=+0.643928550 container remove 70cb5e21cdb7f65066ac5acadec755b19c8e91706c874856971dd8279d0d9058 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-5, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240514, io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-05-20T16:27:32.635 INFO:journalctl@ceph.osd.8.smithi184.stdout:May 20 16:27:32 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-8[55074]: 2024-05-20T16:27:32.524+0000 7f4a3856b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:52.470105+0000 front 2024-05-20T16:26:52.470039+0000 (oldest deadline 2024-05-20T16:27:18.270525+0000) 2024-05-20T16:27:32.636 INFO:journalctl@ceph.osd.8.smithi184.stdout:May 20 16:27:32 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-8[55074]: 2024-05-20T16:27:32.524+0000 7f4a3856b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-20T16:27:05.271959+0000 front 2024-05-20T16:27:05.272026+0000 (oldest deadline 2024-05-20T16:27:28.772153+0000) 2024-05-20T16:27:32.636 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:27:32 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:27:32.354+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-05-20T16:26:47.217074+0000 front 2024-05-20T16:26:47.216883+0000 (oldest deadline 2024-05-20T16:27:10.716827+0000) 2024-05-20T16:27:32.636 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:27:32 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:27:32.354+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:54.817899+0000 front 2024-05-20T16:26:54.818008+0000 (oldest deadline 2024-05-20T16:27:20.018619+0000) 2024-05-20T16:27:32.636 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:27:32 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:27:32.354+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-20T16:27:01.719536+0000 front 2024-05-20T16:27:01.719675+0000 (oldest deadline 2024-05-20T16:27:27.019603+0000) 2024-05-20T16:27:32.914 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:27:32 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:27:32.639+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-05-20T16:26:47.437555+0000 front 2024-05-20T16:26:47.437468+0000 (oldest deadline 2024-05-20T16:27:15.038423+0000) 2024-05-20T16:27:32.915 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:27:32 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:27:32.639+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:55.038908+0000 front 2024-05-20T16:26:55.038825+0000 (oldest deadline 2024-05-20T16:27:18.439444+0000) 2024-05-20T16:27:32.915 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:27:32 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:27:32.639+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-20T16:27:04.241075+0000 front 2024-05-20T16:27:04.241297+0000 (oldest deadline 2024-05-20T16:27:27.141055+0000) 2024-05-20T16:27:33.081 INFO:journalctl@ceph.osd.5.smithi184.stdout:May 20 16:27:32 smithi184 podman[73313]: 2024-05-20 16:27:32.695924962 +0000 UTC m=+0.016674338 image pull cfcc61b2d476f611479310eb042aa05f11dd5f60dbb65e3b0d7e0e11d024cdb3 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708 2024-05-20T16:27:33.081 INFO:journalctl@ceph.osd.5.smithi184.stdout:May 20 16:27:32 smithi184 podman[73313]: 2024-05-20 16:27:32.812516979 +0000 UTC m=+0.133266337 container create 68ea45811d183eeffcfee4e40726a76b2c75726274cf32314084c1ba13ee1bca (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-5-deactivate, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240514, CEPH_POINT_RELEASE=) 2024-05-20T16:27:33.082 INFO:journalctl@ceph.osd.5.smithi184.stdout:May 20 16:27:33 smithi184 podman[73313]: 2024-05-20 16:27:33.082616927 +0000 UTC m=+0.403366291 container init 68ea45811d183eeffcfee4e40726a76b2c75726274cf32314084c1ba13ee1bca (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-5-deactivate, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240514, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-20T16:27:33.312 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 20 16:27:32 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-7[54382]: 2024-05-20T16:27:32.914+0000 7f68a2750700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:56.630029+0000 front 2024-05-20T16:26:56.630059+0000 (oldest deadline 2024-05-20T16:27:18.930178+0000) 2024-05-20T16:27:33.312 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 20 16:27:32 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-7[54382]: 2024-05-20T16:27:32.914+0000 7f68a2750700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-20T16:27:04.032669+0000 front 2024-05-20T16:27:04.032493+0000 (oldest deadline 2024-05-20T16:27:29.332344+0000) 2024-05-20T16:27:33.373 INFO:journalctl@ceph.osd.5.smithi184.stdout:May 20 16:27:33 smithi184 podman[73313]: 2024-05-20 16:27:33.085982857 +0000 UTC m=+0.406732225 container start 68ea45811d183eeffcfee4e40726a76b2c75726274cf32314084c1ba13ee1bca (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-5-deactivate, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240514, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-05-20T16:27:33.373 INFO:journalctl@ceph.osd.5.smithi184.stdout:May 20 16:27:33 smithi184 podman[73313]: 2024-05-20 16:27:33.157303542 +0000 UTC m=+0.478052910 container attach 68ea45811d183eeffcfee4e40726a76b2c75726274cf32314084c1ba13ee1bca (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-5-deactivate, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240514, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-05-20T16:27:33.609 INFO:journalctl@ceph.osd.6.smithi082.stdout:May 20 16:27:33 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-6[60057]: 2024-05-20T16:27:33.274+0000 7f3f143a5700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:56.025271+0000 front 2024-05-20T16:26:56.025414+0000 (oldest deadline 2024-05-20T16:27:20.125270+0000) 2024-05-20T16:27:33.609 INFO:journalctl@ceph.osd.6.smithi082.stdout:May 20 16:27:33 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-6[60057]: 2024-05-20T16:27:33.274+0000 7f3f143a5700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-20T16:27:04.826835+0000 front 2024-05-20T16:27:04.826693+0000 (oldest deadline 2024-05-20T16:27:28.326771+0000) 2024-05-20T16:27:33.609 INFO:journalctl@ceph.osd.9.smithi082.stdout:May 20 16:27:33 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-9[66777]: 2024-05-20T16:27:33.504+0000 7fa187fc1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-05-20T16:26:44.585296+0000 front 2024-05-20T16:26:44.585359+0000 (oldest deadline 2024-05-20T16:27:09.885493+0000) 2024-05-20T16:27:33.609 INFO:journalctl@ceph.osd.9.smithi082.stdout:May 20 16:27:33 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-9[66777]: 2024-05-20T16:27:33.504+0000 7fa187fc1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:55.786620+0000 front 2024-05-20T16:26:55.786679+0000 (oldest deadline 2024-05-20T16:27:19.286485+0000) 2024-05-20T16:27:33.609 INFO:journalctl@ceph.osd.9.smithi082.stdout:May 20 16:27:33 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-9[66777]: 2024-05-20T16:27:33.504+0000 7fa187fc1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-20T16:27:03.987755+0000 front 2024-05-20T16:27:03.987552+0000 (oldest deadline 2024-05-20T16:27:28.687562+0000) 2024-05-20T16:27:33.635 INFO:journalctl@ceph.osd.5.smithi184.stdout:May 20 16:27:33 smithi184 podman[73313]: 2024-05-20 16:27:33.377820027 +0000 UTC m=+0.698569385 container died 68ea45811d183eeffcfee4e40726a76b2c75726274cf32314084c1ba13ee1bca (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-5-deactivate, org.label-schema.build-date=20240514, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-20T16:27:33.635 INFO:journalctl@ceph.osd.8.smithi184.stdout:May 20 16:27:33 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-8[55074]: 2024-05-20T16:27:33.540+0000 7f4a3856b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:52.470105+0000 front 2024-05-20T16:26:52.470039+0000 (oldest deadline 2024-05-20T16:27:18.270525+0000) 2024-05-20T16:27:33.636 INFO:journalctl@ceph.osd.8.smithi184.stdout:May 20 16:27:33 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-8[55074]: 2024-05-20T16:27:33.540+0000 7f4a3856b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-20T16:27:05.271959+0000 front 2024-05-20T16:27:05.272026+0000 (oldest deadline 2024-05-20T16:27:28.772153+0000) 2024-05-20T16:27:33.636 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:27:33 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:27:33.372+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-05-20T16:26:47.217074+0000 front 2024-05-20T16:26:47.216883+0000 (oldest deadline 2024-05-20T16:27:10.716827+0000) 2024-05-20T16:27:33.636 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:27:33 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:27:33.372+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:54.817899+0000 front 2024-05-20T16:26:54.818008+0000 (oldest deadline 2024-05-20T16:27:20.018619+0000) 2024-05-20T16:27:33.636 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:27:33 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:27:33.372+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-20T16:27:01.719536+0000 front 2024-05-20T16:27:01.719675+0000 (oldest deadline 2024-05-20T16:27:27.019603+0000) 2024-05-20T16:27:34.014 DEBUG:teuthology.orchestra.run.smithi184:> sudo pkill -f 'journalctl -f -n 0 -u ceph-59d1600a-16c2-11ef-bc9a-c7b262605968@osd.5.service' 2024-05-20T16:27:34.052 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-20T16:27:34.053 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-05-20T16:27:34.053 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-05-20T16:27:34.053 DEBUG:teuthology.orchestra.run.smithi082:> sudo systemctl stop ceph-59d1600a-16c2-11ef-bc9a-c7b262605968@osd.6 2024-05-20T16:27:34.061 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 20 16:27:33 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-7[54382]: 2024-05-20T16:27:33.885+0000 7f68a2750700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:56.630029+0000 front 2024-05-20T16:26:56.630059+0000 (oldest deadline 2024-05-20T16:27:18.930178+0000) 2024-05-20T16:27:34.062 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 20 16:27:33 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-7[54382]: 2024-05-20T16:27:33.885+0000 7f68a2750700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-20T16:27:04.032669+0000 front 2024-05-20T16:27:04.032493+0000 (oldest deadline 2024-05-20T16:27:29.332344+0000) 2024-05-20T16:27:34.062 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:27:33 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:27:33.665+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-05-20T16:26:47.437555+0000 front 2024-05-20T16:26:47.437468+0000 (oldest deadline 2024-05-20T16:27:15.038423+0000) 2024-05-20T16:27:34.062 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:27:33 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:27:33.665+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:55.038908+0000 front 2024-05-20T16:26:55.038825+0000 (oldest deadline 2024-05-20T16:27:18.439444+0000) 2024-05-20T16:27:34.062 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:27:33 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:27:33.665+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-20T16:27:04.241075+0000 front 2024-05-20T16:27:04.241297+0000 (oldest deadline 2024-05-20T16:27:27.141055+0000) 2024-05-20T16:27:34.358 INFO:journalctl@ceph.osd.6.smithi082.stdout:May 20 16:27:34 smithi082 systemd[1]: Stopping Ceph osd.6 for 59d1600a-16c2-11ef-bc9a-c7b262605968... 2024-05-20T16:27:34.359 INFO:journalctl@ceph.osd.6.smithi082.stdout:May 20 16:27:34 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-6[60057]: 2024-05-20T16:27:34.245+0000 7f3f143a5700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:56.025271+0000 front 2024-05-20T16:26:56.025414+0000 (oldest deadline 2024-05-20T16:27:20.125270+0000) 2024-05-20T16:27:34.359 INFO:journalctl@ceph.osd.6.smithi082.stdout:May 20 16:27:34 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-6[60057]: 2024-05-20T16:27:34.245+0000 7f3f143a5700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-20T16:27:04.826835+0000 front 2024-05-20T16:27:04.826693+0000 (oldest deadline 2024-05-20T16:27:28.326771+0000) 2024-05-20T16:27:34.635 INFO:journalctl@ceph.osd.8.smithi184.stdout:May 20 16:27:34 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-8[55074]: 2024-05-20T16:27:34.495+0000 7f4a3856b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:52.470105+0000 front 2024-05-20T16:26:52.470039+0000 (oldest deadline 2024-05-20T16:27:18.270525+0000) 2024-05-20T16:27:34.635 INFO:journalctl@ceph.osd.8.smithi184.stdout:May 20 16:27:34 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-8[55074]: 2024-05-20T16:27:34.495+0000 7f4a3856b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-20T16:27:05.271959+0000 front 2024-05-20T16:27:05.272026+0000 (oldest deadline 2024-05-20T16:27:28.772153+0000) 2024-05-20T16:27:34.636 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:27:34 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:27:34.362+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-05-20T16:26:47.217074+0000 front 2024-05-20T16:26:47.216883+0000 (oldest deadline 2024-05-20T16:27:10.716827+0000) 2024-05-20T16:27:34.636 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:27:34 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:27:34.362+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:54.817899+0000 front 2024-05-20T16:26:54.818008+0000 (oldest deadline 2024-05-20T16:27:20.018619+0000) 2024-05-20T16:27:34.636 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:27:34 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:27:34.362+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-20T16:27:01.719536+0000 front 2024-05-20T16:27:01.719675+0000 (oldest deadline 2024-05-20T16:27:27.019603+0000) 2024-05-20T16:27:34.859 INFO:journalctl@ceph.osd.6.smithi082.stdout:May 20 16:27:34 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-6[60057]: 2024-05-20T16:27:34.388+0000 7f3f1bdf7700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-20T16:27:34.859 INFO:journalctl@ceph.osd.6.smithi082.stdout:May 20 16:27:34 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-6[60057]: 2024-05-20T16:27:34.388+0000 7f3f1bdf7700 -1 osd.6 102 *** Got signal Terminated *** 2024-05-20T16:27:34.859 INFO:journalctl@ceph.osd.6.smithi082.stdout:May 20 16:27:34 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-6[60057]: 2024-05-20T16:27:34.388+0000 7f3f1bdf7700 -1 osd.6 102 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-20T16:27:34.859 INFO:journalctl@ceph.osd.9.smithi082.stdout:May 20 16:27:34 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-9[66777]: 2024-05-20T16:27:34.462+0000 7fa187fc1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-05-20T16:26:44.585296+0000 front 2024-05-20T16:26:44.585359+0000 (oldest deadline 2024-05-20T16:27:09.885493+0000) 2024-05-20T16:27:34.859 INFO:journalctl@ceph.osd.9.smithi082.stdout:May 20 16:27:34 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-9[66777]: 2024-05-20T16:27:34.462+0000 7fa187fc1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:55.786620+0000 front 2024-05-20T16:26:55.786679+0000 (oldest deadline 2024-05-20T16:27:19.286485+0000) 2024-05-20T16:27:34.859 INFO:journalctl@ceph.osd.9.smithi082.stdout:May 20 16:27:34 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-9[66777]: 2024-05-20T16:27:34.462+0000 7fa187fc1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-20T16:27:03.987755+0000 front 2024-05-20T16:27:03.987552+0000 (oldest deadline 2024-05-20T16:27:28.687562+0000) 2024-05-20T16:27:35.062 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 20 16:27:34 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-7[54382]: 2024-05-20T16:27:34.913+0000 7f68a2750700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:56.630029+0000 front 2024-05-20T16:26:56.630059+0000 (oldest deadline 2024-05-20T16:27:18.930178+0000) 2024-05-20T16:27:35.062 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 20 16:27:34 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-7[54382]: 2024-05-20T16:27:34.913+0000 7f68a2750700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-20T16:27:04.032669+0000 front 2024-05-20T16:27:04.032493+0000 (oldest deadline 2024-05-20T16:27:29.332344+0000) 2024-05-20T16:27:35.062 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:27:34 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:27:34.679+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-05-20T16:26:47.437555+0000 front 2024-05-20T16:26:47.437468+0000 (oldest deadline 2024-05-20T16:27:15.038423+0000) 2024-05-20T16:27:35.062 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:27:34 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:27:34.679+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:55.038908+0000 front 2024-05-20T16:26:55.038825+0000 (oldest deadline 2024-05-20T16:27:18.439444+0000) 2024-05-20T16:27:35.062 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:27:34 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:27:34.679+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-20T16:27:04.241075+0000 front 2024-05-20T16:27:04.241297+0000 (oldest deadline 2024-05-20T16:27:27.141055+0000) 2024-05-20T16:27:35.609 INFO:journalctl@ceph.osd.6.smithi082.stdout:May 20 16:27:35 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-6[60057]: 2024-05-20T16:27:35.293+0000 7f3f143a5700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:56.025271+0000 front 2024-05-20T16:26:56.025414+0000 (oldest deadline 2024-05-20T16:27:20.125270+0000) 2024-05-20T16:27:35.609 INFO:journalctl@ceph.osd.6.smithi082.stdout:May 20 16:27:35 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-6[60057]: 2024-05-20T16:27:35.293+0000 7f3f143a5700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-20T16:27:04.826835+0000 front 2024-05-20T16:27:04.826693+0000 (oldest deadline 2024-05-20T16:27:28.326771+0000) 2024-05-20T16:27:35.609 INFO:journalctl@ceph.osd.9.smithi082.stdout:May 20 16:27:35 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-9[66777]: 2024-05-20T16:27:35.440+0000 7fa187fc1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-05-20T16:26:44.585296+0000 front 2024-05-20T16:26:44.585359+0000 (oldest deadline 2024-05-20T16:27:09.885493+0000) 2024-05-20T16:27:35.609 INFO:journalctl@ceph.osd.9.smithi082.stdout:May 20 16:27:35 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-9[66777]: 2024-05-20T16:27:35.440+0000 7fa187fc1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:55.786620+0000 front 2024-05-20T16:26:55.786679+0000 (oldest deadline 2024-05-20T16:27:19.286485+0000) 2024-05-20T16:27:35.609 INFO:journalctl@ceph.osd.9.smithi082.stdout:May 20 16:27:35 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-9[66777]: 2024-05-20T16:27:35.440+0000 7fa187fc1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-20T16:27:03.987755+0000 front 2024-05-20T16:27:03.987552+0000 (oldest deadline 2024-05-20T16:27:28.687562+0000) 2024-05-20T16:27:35.635 INFO:journalctl@ceph.osd.8.smithi184.stdout:May 20 16:27:35 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-8[55074]: 2024-05-20T16:27:35.520+0000 7f4a3856b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:52.470105+0000 front 2024-05-20T16:26:52.470039+0000 (oldest deadline 2024-05-20T16:27:18.270525+0000) 2024-05-20T16:27:35.635 INFO:journalctl@ceph.osd.8.smithi184.stdout:May 20 16:27:35 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-8[55074]: 2024-05-20T16:27:35.520+0000 7f4a3856b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-20T16:27:05.271959+0000 front 2024-05-20T16:27:05.272026+0000 (oldest deadline 2024-05-20T16:27:28.772153+0000) 2024-05-20T16:27:35.636 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:27:35 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:27:35.372+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-05-20T16:26:47.217074+0000 front 2024-05-20T16:26:47.216883+0000 (oldest deadline 2024-05-20T16:27:10.716827+0000) 2024-05-20T16:27:35.636 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:27:35 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:27:35.372+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:54.817899+0000 front 2024-05-20T16:26:54.818008+0000 (oldest deadline 2024-05-20T16:27:20.018619+0000) 2024-05-20T16:27:35.636 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:27:35 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:27:35.372+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-20T16:27:01.719536+0000 front 2024-05-20T16:27:01.719675+0000 (oldest deadline 2024-05-20T16:27:27.019603+0000) 2024-05-20T16:27:36.062 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 20 16:27:35 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-7[54382]: 2024-05-20T16:27:35.945+0000 7f68a2750700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:56.630029+0000 front 2024-05-20T16:26:56.630059+0000 (oldest deadline 2024-05-20T16:27:18.930178+0000) 2024-05-20T16:27:36.062 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 20 16:27:35 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-7[54382]: 2024-05-20T16:27:35.945+0000 7f68a2750700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-20T16:27:04.032669+0000 front 2024-05-20T16:27:04.032493+0000 (oldest deadline 2024-05-20T16:27:29.332344+0000) 2024-05-20T16:27:36.062 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:27:35 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:27:35.726+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-05-20T16:26:47.437555+0000 front 2024-05-20T16:26:47.437468+0000 (oldest deadline 2024-05-20T16:27:15.038423+0000) 2024-05-20T16:27:36.062 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:27:35 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:27:35.726+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:55.038908+0000 front 2024-05-20T16:26:55.038825+0000 (oldest deadline 2024-05-20T16:27:18.439444+0000) 2024-05-20T16:27:36.062 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:27:35 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:27:35.726+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-20T16:27:04.241075+0000 front 2024-05-20T16:27:04.241297+0000 (oldest deadline 2024-05-20T16:27:27.141055+0000) 2024-05-20T16:27:36.609 INFO:journalctl@ceph.osd.6.smithi082.stdout:May 20 16:27:36 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-6[60057]: 2024-05-20T16:27:36.304+0000 7f3f143a5700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:56.025271+0000 front 2024-05-20T16:26:56.025414+0000 (oldest deadline 2024-05-20T16:27:20.125270+0000) 2024-05-20T16:27:36.609 INFO:journalctl@ceph.osd.6.smithi082.stdout:May 20 16:27:36 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-6[60057]: 2024-05-20T16:27:36.304+0000 7f3f143a5700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-20T16:27:04.826835+0000 front 2024-05-20T16:27:04.826693+0000 (oldest deadline 2024-05-20T16:27:28.326771+0000) 2024-05-20T16:27:36.609 INFO:journalctl@ceph.osd.9.smithi082.stdout:May 20 16:27:36 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-9[66777]: 2024-05-20T16:27:36.427+0000 7fa187fc1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-05-20T16:26:44.585296+0000 front 2024-05-20T16:26:44.585359+0000 (oldest deadline 2024-05-20T16:27:09.885493+0000) 2024-05-20T16:27:36.609 INFO:journalctl@ceph.osd.9.smithi082.stdout:May 20 16:27:36 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-9[66777]: 2024-05-20T16:27:36.427+0000 7fa187fc1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:55.786620+0000 front 2024-05-20T16:26:55.786679+0000 (oldest deadline 2024-05-20T16:27:19.286485+0000) 2024-05-20T16:27:36.609 INFO:journalctl@ceph.osd.9.smithi082.stdout:May 20 16:27:36 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-9[66777]: 2024-05-20T16:27:36.427+0000 7fa187fc1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-20T16:27:03.987755+0000 front 2024-05-20T16:27:03.987552+0000 (oldest deadline 2024-05-20T16:27:28.687562+0000) 2024-05-20T16:27:36.635 INFO:journalctl@ceph.osd.8.smithi184.stdout:May 20 16:27:36 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-8[55074]: 2024-05-20T16:27:36.570+0000 7f4a3856b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:52.470105+0000 front 2024-05-20T16:26:52.470039+0000 (oldest deadline 2024-05-20T16:27:18.270525+0000) 2024-05-20T16:27:36.635 INFO:journalctl@ceph.osd.8.smithi184.stdout:May 20 16:27:36 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-8[55074]: 2024-05-20T16:27:36.570+0000 7f4a3856b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-20T16:27:05.271959+0000 front 2024-05-20T16:27:05.272026+0000 (oldest deadline 2024-05-20T16:27:28.772153+0000) 2024-05-20T16:27:36.636 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:27:36 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:27:36.332+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-05-20T16:26:47.217074+0000 front 2024-05-20T16:26:47.216883+0000 (oldest deadline 2024-05-20T16:27:10.716827+0000) 2024-05-20T16:27:36.636 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:27:36 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:27:36.332+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:54.817899+0000 front 2024-05-20T16:26:54.818008+0000 (oldest deadline 2024-05-20T16:27:20.018619+0000) 2024-05-20T16:27:36.636 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:27:36 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:27:36.332+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-20T16:27:01.719536+0000 front 2024-05-20T16:27:01.719675+0000 (oldest deadline 2024-05-20T16:27:27.019603+0000) 2024-05-20T16:27:37.062 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 20 16:27:36 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-7[54382]: 2024-05-20T16:27:36.914+0000 7f68a2750700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:56.630029+0000 front 2024-05-20T16:26:56.630059+0000 (oldest deadline 2024-05-20T16:27:18.930178+0000) 2024-05-20T16:27:37.062 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 20 16:27:36 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-7[54382]: 2024-05-20T16:27:36.914+0000 7f68a2750700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-20T16:27:04.032669+0000 front 2024-05-20T16:27:04.032493+0000 (oldest deadline 2024-05-20T16:27:29.332344+0000) 2024-05-20T16:27:37.062 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:27:36 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:27:36.732+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-05-20T16:26:47.437555+0000 front 2024-05-20T16:26:47.437468+0000 (oldest deadline 2024-05-20T16:27:15.038423+0000) 2024-05-20T16:27:37.062 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:27:36 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:27:36.732+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:55.038908+0000 front 2024-05-20T16:26:55.038825+0000 (oldest deadline 2024-05-20T16:27:18.439444+0000) 2024-05-20T16:27:37.062 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:27:36 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:27:36.732+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-20T16:27:04.241075+0000 front 2024-05-20T16:27:04.241297+0000 (oldest deadline 2024-05-20T16:27:27.141055+0000) 2024-05-20T16:27:37.062 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:27:36 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:27:36.732+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-05-20T16:27:12.943012+0000 front 2024-05-20T16:27:12.942981+0000 (oldest deadline 2024-05-20T16:27:36.442783+0000) 2024-05-20T16:27:37.609 INFO:journalctl@ceph.osd.6.smithi082.stdout:May 20 16:27:37 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-6[60057]: 2024-05-20T16:27:37.304+0000 7f3f143a5700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:56.025271+0000 front 2024-05-20T16:26:56.025414+0000 (oldest deadline 2024-05-20T16:27:20.125270+0000) 2024-05-20T16:27:37.609 INFO:journalctl@ceph.osd.6.smithi082.stdout:May 20 16:27:37 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-6[60057]: 2024-05-20T16:27:37.304+0000 7f3f143a5700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-20T16:27:04.826835+0000 front 2024-05-20T16:27:04.826693+0000 (oldest deadline 2024-05-20T16:27:28.326771+0000) 2024-05-20T16:27:37.609 INFO:journalctl@ceph.osd.6.smithi082.stdout:May 20 16:27:37 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-6[60057]: 2024-05-20T16:27:37.304+0000 7f3f143a5700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-05-20T16:27:11.227649+0000 front 2024-05-20T16:27:11.227574+0000 (oldest deadline 2024-05-20T16:27:36.527757+0000) 2024-05-20T16:27:37.609 INFO:journalctl@ceph.osd.9.smithi082.stdout:May 20 16:27:37 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-9[66777]: 2024-05-20T16:27:37.443+0000 7fa187fc1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-05-20T16:26:44.585296+0000 front 2024-05-20T16:26:44.585359+0000 (oldest deadline 2024-05-20T16:27:09.885493+0000) 2024-05-20T16:27:37.609 INFO:journalctl@ceph.osd.9.smithi082.stdout:May 20 16:27:37 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-9[66777]: 2024-05-20T16:27:37.443+0000 7fa187fc1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:55.786620+0000 front 2024-05-20T16:26:55.786679+0000 (oldest deadline 2024-05-20T16:27:19.286485+0000) 2024-05-20T16:27:37.609 INFO:journalctl@ceph.osd.9.smithi082.stdout:May 20 16:27:37 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-9[66777]: 2024-05-20T16:27:37.443+0000 7fa187fc1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-20T16:27:03.987755+0000 front 2024-05-20T16:27:03.987552+0000 (oldest deadline 2024-05-20T16:27:28.687562+0000) 2024-05-20T16:27:37.635 INFO:journalctl@ceph.osd.8.smithi184.stdout:May 20 16:27:37 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-8[55074]: 2024-05-20T16:27:37.530+0000 7f4a3856b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:52.470105+0000 front 2024-05-20T16:26:52.470039+0000 (oldest deadline 2024-05-20T16:27:18.270525+0000) 2024-05-20T16:27:37.635 INFO:journalctl@ceph.osd.8.smithi184.stdout:May 20 16:27:37 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-8[55074]: 2024-05-20T16:27:37.530+0000 7f4a3856b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-20T16:27:05.271959+0000 front 2024-05-20T16:27:05.272026+0000 (oldest deadline 2024-05-20T16:27:28.772153+0000) 2024-05-20T16:27:37.635 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:27:37 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:27:37.355+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-05-20T16:26:47.217074+0000 front 2024-05-20T16:26:47.216883+0000 (oldest deadline 2024-05-20T16:27:10.716827+0000) 2024-05-20T16:27:37.635 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:27:37 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:27:37.355+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:54.817899+0000 front 2024-05-20T16:26:54.818008+0000 (oldest deadline 2024-05-20T16:27:20.018619+0000) 2024-05-20T16:27:37.635 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:27:37 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:27:37.355+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-20T16:27:01.719536+0000 front 2024-05-20T16:27:01.719675+0000 (oldest deadline 2024-05-20T16:27:27.019603+0000) 2024-05-20T16:27:38.062 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 20 16:27:37 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-7[54382]: 2024-05-20T16:27:37.902+0000 7f68a2750700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:56.630029+0000 front 2024-05-20T16:26:56.630059+0000 (oldest deadline 2024-05-20T16:27:18.930178+0000) 2024-05-20T16:27:38.062 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 20 16:27:37 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-7[54382]: 2024-05-20T16:27:37.902+0000 7f68a2750700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-20T16:27:04.032669+0000 front 2024-05-20T16:27:04.032493+0000 (oldest deadline 2024-05-20T16:27:29.332344+0000) 2024-05-20T16:27:38.062 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:27:37 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:27:37.686+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-05-20T16:26:47.437555+0000 front 2024-05-20T16:26:47.437468+0000 (oldest deadline 2024-05-20T16:27:15.038423+0000) 2024-05-20T16:27:38.062 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:27:37 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:27:37.686+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:55.038908+0000 front 2024-05-20T16:26:55.038825+0000 (oldest deadline 2024-05-20T16:27:18.439444+0000) 2024-05-20T16:27:38.062 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:27:37 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:27:37.686+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-20T16:27:04.241075+0000 front 2024-05-20T16:27:04.241297+0000 (oldest deadline 2024-05-20T16:27:27.141055+0000) 2024-05-20T16:27:38.062 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:27:37 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:27:37.686+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-05-20T16:27:12.943012+0000 front 2024-05-20T16:27:12.942981+0000 (oldest deadline 2024-05-20T16:27:36.442783+0000) 2024-05-20T16:27:38.609 INFO:journalctl@ceph.osd.9.smithi082.stdout:May 20 16:27:38 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-9[66777]: 2024-05-20T16:27:38.441+0000 7fa187fc1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-05-20T16:26:44.585296+0000 front 2024-05-20T16:26:44.585359+0000 (oldest deadline 2024-05-20T16:27:09.885493+0000) 2024-05-20T16:27:38.609 INFO:journalctl@ceph.osd.9.smithi082.stdout:May 20 16:27:38 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-9[66777]: 2024-05-20T16:27:38.441+0000 7fa187fc1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:55.786620+0000 front 2024-05-20T16:26:55.786679+0000 (oldest deadline 2024-05-20T16:27:19.286485+0000) 2024-05-20T16:27:38.609 INFO:journalctl@ceph.osd.9.smithi082.stdout:May 20 16:27:38 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-9[66777]: 2024-05-20T16:27:38.441+0000 7fa187fc1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-20T16:27:03.987755+0000 front 2024-05-20T16:27:03.987552+0000 (oldest deadline 2024-05-20T16:27:28.687562+0000) 2024-05-20T16:27:38.609 INFO:journalctl@ceph.osd.6.smithi082.stdout:May 20 16:27:38 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-6[60057]: 2024-05-20T16:27:38.333+0000 7f3f143a5700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:56.025271+0000 front 2024-05-20T16:26:56.025414+0000 (oldest deadline 2024-05-20T16:27:20.125270+0000) 2024-05-20T16:27:38.609 INFO:journalctl@ceph.osd.6.smithi082.stdout:May 20 16:27:38 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-6[60057]: 2024-05-20T16:27:38.333+0000 7f3f143a5700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-20T16:27:04.826835+0000 front 2024-05-20T16:27:04.826693+0000 (oldest deadline 2024-05-20T16:27:28.326771+0000) 2024-05-20T16:27:38.609 INFO:journalctl@ceph.osd.6.smithi082.stdout:May 20 16:27:38 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-6[60057]: 2024-05-20T16:27:38.334+0000 7f3f143a5700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-05-20T16:27:11.227649+0000 front 2024-05-20T16:27:11.227574+0000 (oldest deadline 2024-05-20T16:27:36.527757+0000) 2024-05-20T16:27:38.635 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:27:38 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:27:38.346+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-05-20T16:26:47.217074+0000 front 2024-05-20T16:26:47.216883+0000 (oldest deadline 2024-05-20T16:27:10.716827+0000) 2024-05-20T16:27:38.635 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:27:38 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:27:38.346+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:54.817899+0000 front 2024-05-20T16:26:54.818008+0000 (oldest deadline 2024-05-20T16:27:20.018619+0000) 2024-05-20T16:27:38.635 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:27:38 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:27:38.346+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-20T16:27:01.719536+0000 front 2024-05-20T16:27:01.719675+0000 (oldest deadline 2024-05-20T16:27:27.019603+0000) 2024-05-20T16:27:38.636 INFO:journalctl@ceph.osd.8.smithi184.stdout:May 20 16:27:38 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-8[55074]: 2024-05-20T16:27:38.539+0000 7f4a3856b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:52.470105+0000 front 2024-05-20T16:26:52.470039+0000 (oldest deadline 2024-05-20T16:27:18.270525+0000) 2024-05-20T16:27:38.636 INFO:journalctl@ceph.osd.8.smithi184.stdout:May 20 16:27:38 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-8[55074]: 2024-05-20T16:27:38.539+0000 7f4a3856b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-20T16:27:05.271959+0000 front 2024-05-20T16:27:05.272026+0000 (oldest deadline 2024-05-20T16:27:28.772153+0000) 2024-05-20T16:27:39.062 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 20 16:27:38 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-7[54382]: 2024-05-20T16:27:38.890+0000 7f68a2750700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:56.630029+0000 front 2024-05-20T16:26:56.630059+0000 (oldest deadline 2024-05-20T16:27:18.930178+0000) 2024-05-20T16:27:39.062 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 20 16:27:38 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-7[54382]: 2024-05-20T16:27:38.890+0000 7f68a2750700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-20T16:27:04.032669+0000 front 2024-05-20T16:27:04.032493+0000 (oldest deadline 2024-05-20T16:27:29.332344+0000) 2024-05-20T16:27:39.062 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:27:38 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:27:38.727+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-05-20T16:26:47.437555+0000 front 2024-05-20T16:26:47.437468+0000 (oldest deadline 2024-05-20T16:27:15.038423+0000) 2024-05-20T16:27:39.062 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:27:38 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:27:38.727+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:55.038908+0000 front 2024-05-20T16:26:55.038825+0000 (oldest deadline 2024-05-20T16:27:18.439444+0000) 2024-05-20T16:27:39.062 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:27:38 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:27:38.727+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-20T16:27:04.241075+0000 front 2024-05-20T16:27:04.241297+0000 (oldest deadline 2024-05-20T16:27:27.141055+0000) 2024-05-20T16:27:39.062 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:27:38 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:27:38.727+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-05-20T16:27:12.943012+0000 front 2024-05-20T16:27:12.942981+0000 (oldest deadline 2024-05-20T16:27:36.442783+0000) 2024-05-20T16:27:39.635 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:27:39 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:27:39.379+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-05-20T16:26:47.217074+0000 front 2024-05-20T16:26:47.216883+0000 (oldest deadline 2024-05-20T16:27:10.716827+0000) 2024-05-20T16:27:39.635 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:27:39 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:27:39.379+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:54.817899+0000 front 2024-05-20T16:26:54.818008+0000 (oldest deadline 2024-05-20T16:27:20.018619+0000) 2024-05-20T16:27:39.636 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:27:39 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:27:39.379+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-20T16:27:01.719536+0000 front 2024-05-20T16:27:01.719675+0000 (oldest deadline 2024-05-20T16:27:27.019603+0000) 2024-05-20T16:27:39.636 INFO:journalctl@ceph.osd.8.smithi184.stdout:May 20 16:27:39 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-8[55074]: 2024-05-20T16:27:39.514+0000 7f4a3856b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:52.470105+0000 front 2024-05-20T16:26:52.470039+0000 (oldest deadline 2024-05-20T16:27:18.270525+0000) 2024-05-20T16:27:39.636 INFO:journalctl@ceph.osd.8.smithi184.stdout:May 20 16:27:39 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-8[55074]: 2024-05-20T16:27:39.514+0000 7f4a3856b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-20T16:27:05.271959+0000 front 2024-05-20T16:27:05.272026+0000 (oldest deadline 2024-05-20T16:27:28.772153+0000) 2024-05-20T16:27:39.636 INFO:journalctl@ceph.osd.8.smithi184.stdout:May 20 16:27:39 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-8[55074]: 2024-05-20T16:27:39.514+0000 7f4a3856b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-05-20T16:27:13.473547+0000 front 2024-05-20T16:27:13.473521+0000 (oldest deadline 2024-05-20T16:27:38.773369+0000) 2024-05-20T16:27:39.859 INFO:journalctl@ceph.osd.6.smithi082.stdout:May 20 16:27:39 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-6[60057]: 2024-05-20T16:27:39.381+0000 7f3f143a5700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:56.025271+0000 front 2024-05-20T16:26:56.025414+0000 (oldest deadline 2024-05-20T16:27:20.125270+0000) 2024-05-20T16:27:39.859 INFO:journalctl@ceph.osd.6.smithi082.stdout:May 20 16:27:39 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-6[60057]: 2024-05-20T16:27:39.381+0000 7f3f143a5700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-20T16:27:04.826835+0000 front 2024-05-20T16:27:04.826693+0000 (oldest deadline 2024-05-20T16:27:28.326771+0000) 2024-05-20T16:27:39.859 INFO:journalctl@ceph.osd.6.smithi082.stdout:May 20 16:27:39 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-6[60057]: 2024-05-20T16:27:39.381+0000 7f3f143a5700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-05-20T16:27:11.227649+0000 front 2024-05-20T16:27:11.227574+0000 (oldest deadline 2024-05-20T16:27:36.527757+0000) 2024-05-20T16:27:39.859 INFO:journalctl@ceph.osd.9.smithi082.stdout:May 20 16:27:39 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-9[66777]: 2024-05-20T16:27:39.407+0000 7fa187fc1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-05-20T16:26:44.585296+0000 front 2024-05-20T16:26:44.585359+0000 (oldest deadline 2024-05-20T16:27:09.885493+0000) 2024-05-20T16:27:39.860 INFO:journalctl@ceph.osd.9.smithi082.stdout:May 20 16:27:39 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-9[66777]: 2024-05-20T16:27:39.407+0000 7fa187fc1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:55.786620+0000 front 2024-05-20T16:26:55.786679+0000 (oldest deadline 2024-05-20T16:27:19.286485+0000) 2024-05-20T16:27:39.860 INFO:journalctl@ceph.osd.9.smithi082.stdout:May 20 16:27:39 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-9[66777]: 2024-05-20T16:27:39.407+0000 7fa187fc1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-20T16:27:03.987755+0000 front 2024-05-20T16:27:03.987552+0000 (oldest deadline 2024-05-20T16:27:28.687562+0000) 2024-05-20T16:27:39.860 INFO:journalctl@ceph.osd.9.smithi082.stdout:May 20 16:27:39 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-9[66777]: 2024-05-20T16:27:39.407+0000 7fa187fc1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-05-20T16:27:13.388520+0000 front 2024-05-20T16:27:13.388549+0000 (oldest deadline 2024-05-20T16:27:38.688716+0000) 2024-05-20T16:27:40.062 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 20 16:27:39 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-7[54382]: 2024-05-20T16:27:39.900+0000 7f68a2750700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:56.630029+0000 front 2024-05-20T16:26:56.630059+0000 (oldest deadline 2024-05-20T16:27:18.930178+0000) 2024-05-20T16:27:40.062 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 20 16:27:39 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-7[54382]: 2024-05-20T16:27:39.900+0000 7f68a2750700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-20T16:27:04.032669+0000 front 2024-05-20T16:27:04.032493+0000 (oldest deadline 2024-05-20T16:27:29.332344+0000) 2024-05-20T16:27:40.063 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 20 16:27:39 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-7[54382]: 2024-05-20T16:27:39.900+0000 7f68a2750700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-05-20T16:27:13.234707+0000 front 2024-05-20T16:27:13.234674+0000 (oldest deadline 2024-05-20T16:27:39.134461+0000) 2024-05-20T16:27:40.063 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:27:39 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:27:39.724+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-05-20T16:26:47.437555+0000 front 2024-05-20T16:26:47.437468+0000 (oldest deadline 2024-05-20T16:27:15.038423+0000) 2024-05-20T16:27:40.063 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:27:39 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:27:39.724+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:55.038908+0000 front 2024-05-20T16:26:55.038825+0000 (oldest deadline 2024-05-20T16:27:18.439444+0000) 2024-05-20T16:27:40.063 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:27:39 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:27:39.724+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-20T16:27:04.241075+0000 front 2024-05-20T16:27:04.241297+0000 (oldest deadline 2024-05-20T16:27:27.141055+0000) 2024-05-20T16:27:40.064 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:27:39 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:27:39.724+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-05-20T16:27:12.943012+0000 front 2024-05-20T16:27:12.942981+0000 (oldest deadline 2024-05-20T16:27:36.442783+0000) 2024-05-20T16:27:40.513 INFO:journalctl@ceph.osd.6.smithi082.stdout:May 20 16:27:40 smithi082 podman[92191]: 2024-05-20 16:27:40.211011375 +0000 UTC m=+5.951438863 container died c86c6eea4a11ba79c2d92f50defbdef602b7e5e12885198ee5936ffed04ec779 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-6, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240514, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c) 2024-05-20T16:27:40.514 INFO:journalctl@ceph.osd.9.smithi082.stdout:May 20 16:27:40 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-9[66777]: 2024-05-20T16:27:40.372+0000 7fa187fc1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-05-20T16:26:44.585296+0000 front 2024-05-20T16:26:44.585359+0000 (oldest deadline 2024-05-20T16:27:09.885493+0000) 2024-05-20T16:27:40.514 INFO:journalctl@ceph.osd.9.smithi082.stdout:May 20 16:27:40 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-9[66777]: 2024-05-20T16:27:40.372+0000 7fa187fc1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:55.786620+0000 front 2024-05-20T16:26:55.786679+0000 (oldest deadline 2024-05-20T16:27:19.286485+0000) 2024-05-20T16:27:40.514 INFO:journalctl@ceph.osd.9.smithi082.stdout:May 20 16:27:40 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-9[66777]: 2024-05-20T16:27:40.372+0000 7fa187fc1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-20T16:27:03.987755+0000 front 2024-05-20T16:27:03.987552+0000 (oldest deadline 2024-05-20T16:27:28.687562+0000) 2024-05-20T16:27:40.514 INFO:journalctl@ceph.osd.9.smithi082.stdout:May 20 16:27:40 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-9[66777]: 2024-05-20T16:27:40.372+0000 7fa187fc1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-05-20T16:27:13.388520+0000 front 2024-05-20T16:27:13.388549+0000 (oldest deadline 2024-05-20T16:27:38.688716+0000) 2024-05-20T16:27:40.788 INFO:journalctl@ceph.osd.6.smithi082.stdout:May 20 16:27:40 smithi082 podman[92191]: 2024-05-20 16:27:40.621286554 +0000 UTC m=+6.361714034 container cleanup c86c6eea4a11ba79c2d92f50defbdef602b7e5e12885198ee5936ffed04ec779 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-6, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240514, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-05-20T16:27:40.788 INFO:journalctl@ceph.osd.6.smithi082.stdout:May 20 16:27:40 smithi082 bash[92191]: ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-6 2024-05-20T16:27:40.885 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:27:40 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:27:40.394+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-05-20T16:26:47.217074+0000 front 2024-05-20T16:26:47.216883+0000 (oldest deadline 2024-05-20T16:27:10.716827+0000) 2024-05-20T16:27:40.886 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:27:40 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:27:40.394+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:54.817899+0000 front 2024-05-20T16:26:54.818008+0000 (oldest deadline 2024-05-20T16:27:20.018619+0000) 2024-05-20T16:27:40.886 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:27:40 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:27:40.394+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-20T16:27:01.719536+0000 front 2024-05-20T16:27:01.719675+0000 (oldest deadline 2024-05-20T16:27:27.019603+0000) 2024-05-20T16:27:40.886 INFO:journalctl@ceph.osd.8.smithi184.stdout:May 20 16:27:40 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-8[55074]: 2024-05-20T16:27:40.495+0000 7f4a3856b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:52.470105+0000 front 2024-05-20T16:26:52.470039+0000 (oldest deadline 2024-05-20T16:27:18.270525+0000) 2024-05-20T16:27:40.886 INFO:journalctl@ceph.osd.8.smithi184.stdout:May 20 16:27:40 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-8[55074]: 2024-05-20T16:27:40.495+0000 7f4a3856b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-20T16:27:05.271959+0000 front 2024-05-20T16:27:05.272026+0000 (oldest deadline 2024-05-20T16:27:28.772153+0000) 2024-05-20T16:27:40.886 INFO:journalctl@ceph.osd.8.smithi184.stdout:May 20 16:27:40 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-8[55074]: 2024-05-20T16:27:40.495+0000 7f4a3856b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-05-20T16:27:13.473547+0000 front 2024-05-20T16:27:13.473521+0000 (oldest deadline 2024-05-20T16:27:38.773369+0000) 2024-05-20T16:27:41.062 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 20 16:27:40 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-7[54382]: 2024-05-20T16:27:40.881+0000 7f68a2750700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:56.630029+0000 front 2024-05-20T16:26:56.630059+0000 (oldest deadline 2024-05-20T16:27:18.930178+0000) 2024-05-20T16:27:41.062 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 20 16:27:40 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-7[54382]: 2024-05-20T16:27:40.881+0000 7f68a2750700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-20T16:27:04.032669+0000 front 2024-05-20T16:27:04.032493+0000 (oldest deadline 2024-05-20T16:27:29.332344+0000) 2024-05-20T16:27:41.062 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 20 16:27:40 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-7[54382]: 2024-05-20T16:27:40.881+0000 7f68a2750700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-05-20T16:27:13.234707+0000 front 2024-05-20T16:27:13.234674+0000 (oldest deadline 2024-05-20T16:27:39.134461+0000) 2024-05-20T16:27:41.062 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:27:40 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:27:40.761+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-05-20T16:26:47.437555+0000 front 2024-05-20T16:26:47.437468+0000 (oldest deadline 2024-05-20T16:27:15.038423+0000) 2024-05-20T16:27:41.062 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:27:40 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:27:40.761+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:55.038908+0000 front 2024-05-20T16:26:55.038825+0000 (oldest deadline 2024-05-20T16:27:18.439444+0000) 2024-05-20T16:27:41.063 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:27:40 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:27:40.761+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-20T16:27:04.241075+0000 front 2024-05-20T16:27:04.241297+0000 (oldest deadline 2024-05-20T16:27:27.141055+0000) 2024-05-20T16:27:41.063 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:27:40 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:27:40.761+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-05-20T16:27:12.943012+0000 front 2024-05-20T16:27:12.942981+0000 (oldest deadline 2024-05-20T16:27:36.442783+0000) 2024-05-20T16:27:41.109 INFO:journalctl@ceph.osd.6.smithi082.stdout:May 20 16:27:40 smithi082 podman[92205]: 2024-05-20 16:27:40.789129863 +0000 UTC m=+0.579917851 container remove c86c6eea4a11ba79c2d92f50defbdef602b7e5e12885198ee5936ffed04ec779 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-6, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240514, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-20T16:27:41.371 INFO:journalctl@ceph.osd.6.smithi082.stdout:May 20 16:27:41 smithi082 podman[92296]: 2024-05-20 16:27:41.11484925 +0000 UTC m=+0.118159165 container create e4f993ae9bd75120ab81ef51c0aee6c4d9094fb39e7c3ed807a61f795d5dc5f4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-6-deactivate, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240514, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c) 2024-05-20T16:27:41.371 INFO:journalctl@ceph.osd.6.smithi082.stdout:May 20 16:27:41 smithi082 podman[92296]: 2024-05-20 16:27:41.019749894 +0000 UTC m=+0.023059792 image pull cfcc61b2d476f611479310eb042aa05f11dd5f60dbb65e3b0d7e0e11d024cdb3 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708 2024-05-20T16:27:41.371 INFO:journalctl@ceph.osd.6.smithi082.stdout:May 20 16:27:41 smithi082 podman[92296]: 2024-05-20 16:27:41.342857774 +0000 UTC m=+0.346167683 container init e4f993ae9bd75120ab81ef51c0aee6c4d9094fb39e7c3ed807a61f795d5dc5f4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-6-deactivate, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240514, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-05-20T16:27:41.371 INFO:journalctl@ceph.osd.6.smithi082.stdout:May 20 16:27:41 smithi082 podman[92296]: 2024-05-20 16:27:41.346653453 +0000 UTC m=+0.349963350 container start e4f993ae9bd75120ab81ef51c0aee6c4d9094fb39e7c3ed807a61f795d5dc5f4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-6-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240514, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-20T16:27:41.635 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:27:41 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:27:41.353+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-05-20T16:26:47.217074+0000 front 2024-05-20T16:26:47.216883+0000 (oldest deadline 2024-05-20T16:27:10.716827+0000) 2024-05-20T16:27:41.635 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:27:41 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:27:41.353+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:54.817899+0000 front 2024-05-20T16:26:54.818008+0000 (oldest deadline 2024-05-20T16:27:20.018619+0000) 2024-05-20T16:27:41.635 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:27:41 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:27:41.353+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-20T16:27:01.719536+0000 front 2024-05-20T16:27:01.719675+0000 (oldest deadline 2024-05-20T16:27:27.019603+0000) 2024-05-20T16:27:41.636 INFO:journalctl@ceph.osd.8.smithi184.stdout:May 20 16:27:41 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-8[55074]: 2024-05-20T16:27:41.491+0000 7f4a3856b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:52.470105+0000 front 2024-05-20T16:26:52.470039+0000 (oldest deadline 2024-05-20T16:27:18.270525+0000) 2024-05-20T16:27:41.636 INFO:journalctl@ceph.osd.8.smithi184.stdout:May 20 16:27:41 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-8[55074]: 2024-05-20T16:27:41.491+0000 7f4a3856b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-20T16:27:05.271959+0000 front 2024-05-20T16:27:05.272026+0000 (oldest deadline 2024-05-20T16:27:28.772153+0000) 2024-05-20T16:27:41.636 INFO:journalctl@ceph.osd.8.smithi184.stdout:May 20 16:27:41 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-8[55074]: 2024-05-20T16:27:41.491+0000 7f4a3856b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-05-20T16:27:13.473547+0000 front 2024-05-20T16:27:13.473521+0000 (oldest deadline 2024-05-20T16:27:38.773369+0000) 2024-05-20T16:27:41.639 INFO:journalctl@ceph.osd.6.smithi082.stdout:May 20 16:27:41 smithi082 podman[92296]: 2024-05-20 16:27:41.425912906 +0000 UTC m=+0.429222815 container attach e4f993ae9bd75120ab81ef51c0aee6c4d9094fb39e7c3ed807a61f795d5dc5f4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-6-deactivate, GIT_BRANCH=HEAD, org.label-schema.build-date=20240514, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-05-20T16:27:41.639 INFO:journalctl@ceph.osd.9.smithi082.stdout:May 20 16:27:41 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-9[66777]: 2024-05-20T16:27:41.371+0000 7fa187fc1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-05-20T16:26:44.585296+0000 front 2024-05-20T16:26:44.585359+0000 (oldest deadline 2024-05-20T16:27:09.885493+0000) 2024-05-20T16:27:41.639 INFO:journalctl@ceph.osd.9.smithi082.stdout:May 20 16:27:41 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-9[66777]: 2024-05-20T16:27:41.371+0000 7fa187fc1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:55.786620+0000 front 2024-05-20T16:26:55.786679+0000 (oldest deadline 2024-05-20T16:27:19.286485+0000) 2024-05-20T16:27:41.639 INFO:journalctl@ceph.osd.9.smithi082.stdout:May 20 16:27:41 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-9[66777]: 2024-05-20T16:27:41.371+0000 7fa187fc1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-20T16:27:03.987755+0000 front 2024-05-20T16:27:03.987552+0000 (oldest deadline 2024-05-20T16:27:28.687562+0000) 2024-05-20T16:27:41.639 INFO:journalctl@ceph.osd.9.smithi082.stdout:May 20 16:27:41 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-9[66777]: 2024-05-20T16:27:41.371+0000 7fa187fc1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-05-20T16:27:13.388520+0000 front 2024-05-20T16:27:13.388549+0000 (oldest deadline 2024-05-20T16:27:38.688716+0000) 2024-05-20T16:27:41.935 INFO:journalctl@ceph.osd.6.smithi082.stdout:May 20 16:27:41 smithi082 podman[92296]: 2024-05-20 16:27:41.640614831 +0000 UTC m=+0.643924731 container died e4f993ae9bd75120ab81ef51c0aee6c4d9094fb39e7c3ed807a61f795d5dc5f4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-6-deactivate, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240514, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-05-20T16:27:42.062 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 20 16:27:41 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-7[54382]: 2024-05-20T16:27:41.834+0000 7f68a2750700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:56.630029+0000 front 2024-05-20T16:26:56.630059+0000 (oldest deadline 2024-05-20T16:27:18.930178+0000) 2024-05-20T16:27:42.062 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 20 16:27:41 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-7[54382]: 2024-05-20T16:27:41.834+0000 7f68a2750700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-20T16:27:04.032669+0000 front 2024-05-20T16:27:04.032493+0000 (oldest deadline 2024-05-20T16:27:29.332344+0000) 2024-05-20T16:27:42.062 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 20 16:27:41 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-7[54382]: 2024-05-20T16:27:41.834+0000 7f68a2750700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-05-20T16:27:13.234707+0000 front 2024-05-20T16:27:13.234674+0000 (oldest deadline 2024-05-20T16:27:39.134461+0000) 2024-05-20T16:27:42.062 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:27:41 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:27:41.733+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-05-20T16:26:47.437555+0000 front 2024-05-20T16:26:47.437468+0000 (oldest deadline 2024-05-20T16:27:15.038423+0000) 2024-05-20T16:27:42.062 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:27:41 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:27:41.733+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:55.038908+0000 front 2024-05-20T16:26:55.038825+0000 (oldest deadline 2024-05-20T16:27:18.439444+0000) 2024-05-20T16:27:42.063 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:27:41 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:27:41.733+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-20T16:27:04.241075+0000 front 2024-05-20T16:27:04.241297+0000 (oldest deadline 2024-05-20T16:27:27.141055+0000) 2024-05-20T16:27:42.063 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:27:41 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:27:41.733+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-05-20T16:27:12.943012+0000 front 2024-05-20T16:27:12.942981+0000 (oldest deadline 2024-05-20T16:27:36.442783+0000) 2024-05-20T16:27:42.196 INFO:journalctl@ceph.osd.6.smithi082.stdout:May 20 16:27:42 smithi082 podman[92322]: 2024-05-20 16:27:42.179317289 +0000 UTC m=+0.528222496 container remove e4f993ae9bd75120ab81ef51c0aee6c4d9094fb39e7c3ed807a61f795d5dc5f4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-6-deactivate, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240514, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-20T16:27:42.201 DEBUG:teuthology.orchestra.run.smithi082:> sudo pkill -f 'journalctl -f -n 0 -u ceph-59d1600a-16c2-11ef-bc9a-c7b262605968@osd.6.service' 2024-05-20T16:27:42.239 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-20T16:27:42.239 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-05-20T16:27:42.239 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-05-20T16:27:42.240 DEBUG:teuthology.orchestra.run.smithi160:> sudo systemctl stop ceph-59d1600a-16c2-11ef-bc9a-c7b262605968@osd.7 2024-05-20T16:27:42.562 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 20 16:27:42 smithi160 systemd[1]: Stopping Ceph osd.7 for 59d1600a-16c2-11ef-bc9a-c7b262605968... 2024-05-20T16:27:42.609 INFO:journalctl@ceph.osd.9.smithi082.stdout:May 20 16:27:42 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-9[66777]: 2024-05-20T16:27:42.363+0000 7fa187fc1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-05-20T16:26:44.585296+0000 front 2024-05-20T16:26:44.585359+0000 (oldest deadline 2024-05-20T16:27:09.885493+0000) 2024-05-20T16:27:42.609 INFO:journalctl@ceph.osd.9.smithi082.stdout:May 20 16:27:42 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-9[66777]: 2024-05-20T16:27:42.363+0000 7fa187fc1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:55.786620+0000 front 2024-05-20T16:26:55.786679+0000 (oldest deadline 2024-05-20T16:27:19.286485+0000) 2024-05-20T16:27:42.609 INFO:journalctl@ceph.osd.9.smithi082.stdout:May 20 16:27:42 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-9[66777]: 2024-05-20T16:27:42.363+0000 7fa187fc1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-20T16:27:03.987755+0000 front 2024-05-20T16:27:03.987552+0000 (oldest deadline 2024-05-20T16:27:28.687562+0000) 2024-05-20T16:27:42.609 INFO:journalctl@ceph.osd.9.smithi082.stdout:May 20 16:27:42 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-9[66777]: 2024-05-20T16:27:42.363+0000 7fa187fc1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-05-20T16:27:13.388520+0000 front 2024-05-20T16:27:13.388549+0000 (oldest deadline 2024-05-20T16:27:38.688716+0000) 2024-05-20T16:27:42.635 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:27:42 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:27:42.308+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-05-20T16:26:47.217074+0000 front 2024-05-20T16:26:47.216883+0000 (oldest deadline 2024-05-20T16:27:10.716827+0000) 2024-05-20T16:27:42.635 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:27:42 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:27:42.308+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:54.817899+0000 front 2024-05-20T16:26:54.818008+0000 (oldest deadline 2024-05-20T16:27:20.018619+0000) 2024-05-20T16:27:42.635 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:27:42 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:27:42.308+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-20T16:27:01.719536+0000 front 2024-05-20T16:27:01.719675+0000 (oldest deadline 2024-05-20T16:27:27.019603+0000) 2024-05-20T16:27:42.635 INFO:journalctl@ceph.osd.8.smithi184.stdout:May 20 16:27:42 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-8[55074]: 2024-05-20T16:27:42.528+0000 7f4a3856b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:52.470105+0000 front 2024-05-20T16:26:52.470039+0000 (oldest deadline 2024-05-20T16:27:18.270525+0000) 2024-05-20T16:27:42.636 INFO:journalctl@ceph.osd.8.smithi184.stdout:May 20 16:27:42 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-8[55074]: 2024-05-20T16:27:42.528+0000 7f4a3856b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-20T16:27:05.271959+0000 front 2024-05-20T16:27:05.272026+0000 (oldest deadline 2024-05-20T16:27:28.772153+0000) 2024-05-20T16:27:42.636 INFO:journalctl@ceph.osd.8.smithi184.stdout:May 20 16:27:42 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-8[55074]: 2024-05-20T16:27:42.528+0000 7f4a3856b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-05-20T16:27:13.473547+0000 front 2024-05-20T16:27:13.473521+0000 (oldest deadline 2024-05-20T16:27:38.773369+0000) 2024-05-20T16:27:43.062 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 20 16:27:42 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-7[54382]: 2024-05-20T16:27:42.580+0000 7f68aa1a2700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-20T16:27:43.062 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 20 16:27:42 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-7[54382]: 2024-05-20T16:27:42.580+0000 7f68aa1a2700 -1 osd.7 102 *** Got signal Terminated *** 2024-05-20T16:27:43.062 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 20 16:27:42 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-7[54382]: 2024-05-20T16:27:42.580+0000 7f68aa1a2700 -1 osd.7 102 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-20T16:27:43.062 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 20 16:27:42 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-7[54382]: 2024-05-20T16:27:42.824+0000 7f68a2750700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:56.630029+0000 front 2024-05-20T16:26:56.630059+0000 (oldest deadline 2024-05-20T16:27:18.930178+0000) 2024-05-20T16:27:43.062 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 20 16:27:42 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-7[54382]: 2024-05-20T16:27:42.824+0000 7f68a2750700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-20T16:27:04.032669+0000 front 2024-05-20T16:27:04.032493+0000 (oldest deadline 2024-05-20T16:27:29.332344+0000) 2024-05-20T16:27:43.062 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 20 16:27:42 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-7[54382]: 2024-05-20T16:27:42.824+0000 7f68a2750700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-05-20T16:27:13.234707+0000 front 2024-05-20T16:27:13.234674+0000 (oldest deadline 2024-05-20T16:27:39.134461+0000) 2024-05-20T16:27:43.062 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:27:42 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:27:42.741+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-05-20T16:26:47.437555+0000 front 2024-05-20T16:26:47.437468+0000 (oldest deadline 2024-05-20T16:27:15.038423+0000) 2024-05-20T16:27:43.063 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:27:42 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:27:42.741+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:55.038908+0000 front 2024-05-20T16:26:55.038825+0000 (oldest deadline 2024-05-20T16:27:18.439444+0000) 2024-05-20T16:27:43.063 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:27:42 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:27:42.741+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-20T16:27:04.241075+0000 front 2024-05-20T16:27:04.241297+0000 (oldest deadline 2024-05-20T16:27:27.141055+0000) 2024-05-20T16:27:43.063 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:27:42 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:27:42.741+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-05-20T16:27:12.943012+0000 front 2024-05-20T16:27:12.942981+0000 (oldest deadline 2024-05-20T16:27:36.442783+0000) 2024-05-20T16:27:43.550 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:27:43 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:27:43.290+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-05-20T16:26:47.217074+0000 front 2024-05-20T16:26:47.216883+0000 (oldest deadline 2024-05-20T16:27:10.716827+0000) 2024-05-20T16:27:43.550 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:27:43 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:27:43.290+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:54.817899+0000 front 2024-05-20T16:26:54.818008+0000 (oldest deadline 2024-05-20T16:27:20.018619+0000) 2024-05-20T16:27:43.550 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:27:43 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:27:43.290+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-20T16:27:01.719536+0000 front 2024-05-20T16:27:01.719675+0000 (oldest deadline 2024-05-20T16:27:27.019603+0000) 2024-05-20T16:27:43.859 INFO:journalctl@ceph.osd.9.smithi082.stdout:May 20 16:27:43 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-9[66777]: 2024-05-20T16:27:43.400+0000 7fa187fc1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-05-20T16:26:44.585296+0000 front 2024-05-20T16:26:44.585359+0000 (oldest deadline 2024-05-20T16:27:09.885493+0000) 2024-05-20T16:27:43.859 INFO:journalctl@ceph.osd.9.smithi082.stdout:May 20 16:27:43 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-9[66777]: 2024-05-20T16:27:43.400+0000 7fa187fc1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:55.786620+0000 front 2024-05-20T16:26:55.786679+0000 (oldest deadline 2024-05-20T16:27:19.286485+0000) 2024-05-20T16:27:43.859 INFO:journalctl@ceph.osd.9.smithi082.stdout:May 20 16:27:43 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-9[66777]: 2024-05-20T16:27:43.400+0000 7fa187fc1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-20T16:27:03.987755+0000 front 2024-05-20T16:27:03.987552+0000 (oldest deadline 2024-05-20T16:27:28.687562+0000) 2024-05-20T16:27:43.859 INFO:journalctl@ceph.osd.9.smithi082.stdout:May 20 16:27:43 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-9[66777]: 2024-05-20T16:27:43.400+0000 7fa187fc1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-05-20T16:27:13.388520+0000 front 2024-05-20T16:27:13.388549+0000 (oldest deadline 2024-05-20T16:27:38.688716+0000) 2024-05-20T16:27:43.885 INFO:journalctl@ceph.osd.8.smithi184.stdout:May 20 16:27:43 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-8[55074]: 2024-05-20T16:27:43.551+0000 7f4a3856b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:52.470105+0000 front 2024-05-20T16:26:52.470039+0000 (oldest deadline 2024-05-20T16:27:18.270525+0000) 2024-05-20T16:27:43.885 INFO:journalctl@ceph.osd.8.smithi184.stdout:May 20 16:27:43 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-8[55074]: 2024-05-20T16:27:43.551+0000 7f4a3856b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-20T16:27:05.271959+0000 front 2024-05-20T16:27:05.272026+0000 (oldest deadline 2024-05-20T16:27:28.772153+0000) 2024-05-20T16:27:43.885 INFO:journalctl@ceph.osd.8.smithi184.stdout:May 20 16:27:43 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-8[55074]: 2024-05-20T16:27:43.551+0000 7f4a3856b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-05-20T16:27:13.473547+0000 front 2024-05-20T16:27:13.473521+0000 (oldest deadline 2024-05-20T16:27:38.773369+0000) 2024-05-20T16:27:44.062 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 20 16:27:43 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-7[54382]: 2024-05-20T16:27:43.873+0000 7f68a2750700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:56.630029+0000 front 2024-05-20T16:26:56.630059+0000 (oldest deadline 2024-05-20T16:27:18.930178+0000) 2024-05-20T16:27:44.062 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 20 16:27:43 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-7[54382]: 2024-05-20T16:27:43.873+0000 7f68a2750700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-20T16:27:04.032669+0000 front 2024-05-20T16:27:04.032493+0000 (oldest deadline 2024-05-20T16:27:29.332344+0000) 2024-05-20T16:27:44.062 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 20 16:27:43 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-7[54382]: 2024-05-20T16:27:43.873+0000 7f68a2750700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-05-20T16:27:13.234707+0000 front 2024-05-20T16:27:13.234674+0000 (oldest deadline 2024-05-20T16:27:39.134461+0000) 2024-05-20T16:27:44.062 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:27:43 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:27:43.715+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-05-20T16:26:47.437555+0000 front 2024-05-20T16:26:47.437468+0000 (oldest deadline 2024-05-20T16:27:15.038423+0000) 2024-05-20T16:27:44.062 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:27:43 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:27:43.715+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:55.038908+0000 front 2024-05-20T16:26:55.038825+0000 (oldest deadline 2024-05-20T16:27:18.439444+0000) 2024-05-20T16:27:44.062 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:27:43 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:27:43.715+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-20T16:27:04.241075+0000 front 2024-05-20T16:27:04.241297+0000 (oldest deadline 2024-05-20T16:27:27.141055+0000) 2024-05-20T16:27:44.062 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:27:43 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:27:43.715+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-05-20T16:27:12.943012+0000 front 2024-05-20T16:27:12.942981+0000 (oldest deadline 2024-05-20T16:27:36.442783+0000) 2024-05-20T16:27:44.588 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:27:44 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:27:44.331+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-05-20T16:26:47.217074+0000 front 2024-05-20T16:26:47.216883+0000 (oldest deadline 2024-05-20T16:27:10.716827+0000) 2024-05-20T16:27:44.589 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:27:44 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:27:44.331+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:54.817899+0000 front 2024-05-20T16:26:54.818008+0000 (oldest deadline 2024-05-20T16:27:20.018619+0000) 2024-05-20T16:27:44.589 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:27:44 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:27:44.331+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-20T16:27:01.719536+0000 front 2024-05-20T16:27:01.719675+0000 (oldest deadline 2024-05-20T16:27:27.019603+0000) 2024-05-20T16:27:44.859 INFO:journalctl@ceph.osd.9.smithi082.stdout:May 20 16:27:44 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-9[66777]: 2024-05-20T16:27:44.385+0000 7fa187fc1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-05-20T16:26:44.585296+0000 front 2024-05-20T16:26:44.585359+0000 (oldest deadline 2024-05-20T16:27:09.885493+0000) 2024-05-20T16:27:44.859 INFO:journalctl@ceph.osd.9.smithi082.stdout:May 20 16:27:44 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-9[66777]: 2024-05-20T16:27:44.385+0000 7fa187fc1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:55.786620+0000 front 2024-05-20T16:26:55.786679+0000 (oldest deadline 2024-05-20T16:27:19.286485+0000) 2024-05-20T16:27:44.859 INFO:journalctl@ceph.osd.9.smithi082.stdout:May 20 16:27:44 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-9[66777]: 2024-05-20T16:27:44.385+0000 7fa187fc1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-20T16:27:03.987755+0000 front 2024-05-20T16:27:03.987552+0000 (oldest deadline 2024-05-20T16:27:28.687562+0000) 2024-05-20T16:27:44.859 INFO:journalctl@ceph.osd.9.smithi082.stdout:May 20 16:27:44 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-9[66777]: 2024-05-20T16:27:44.385+0000 7fa187fc1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-05-20T16:27:13.388520+0000 front 2024-05-20T16:27:13.388549+0000 (oldest deadline 2024-05-20T16:27:38.688716+0000) 2024-05-20T16:27:44.885 INFO:journalctl@ceph.osd.8.smithi184.stdout:May 20 16:27:44 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-8[55074]: 2024-05-20T16:27:44.589+0000 7f4a3856b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:52.470105+0000 front 2024-05-20T16:26:52.470039+0000 (oldest deadline 2024-05-20T16:27:18.270525+0000) 2024-05-20T16:27:44.885 INFO:journalctl@ceph.osd.8.smithi184.stdout:May 20 16:27:44 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-8[55074]: 2024-05-20T16:27:44.589+0000 7f4a3856b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-20T16:27:05.271959+0000 front 2024-05-20T16:27:05.272026+0000 (oldest deadline 2024-05-20T16:27:28.772153+0000) 2024-05-20T16:27:44.885 INFO:journalctl@ceph.osd.8.smithi184.stdout:May 20 16:27:44 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-8[55074]: 2024-05-20T16:27:44.589+0000 7f4a3856b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-05-20T16:27:13.473547+0000 front 2024-05-20T16:27:13.473521+0000 (oldest deadline 2024-05-20T16:27:38.773369+0000) 2024-05-20T16:27:45.062 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 20 16:27:44 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-7[54382]: 2024-05-20T16:27:44.842+0000 7f68a2750700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:56.630029+0000 front 2024-05-20T16:26:56.630059+0000 (oldest deadline 2024-05-20T16:27:18.930178+0000) 2024-05-20T16:27:45.062 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 20 16:27:44 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-7[54382]: 2024-05-20T16:27:44.842+0000 7f68a2750700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-20T16:27:04.032669+0000 front 2024-05-20T16:27:04.032493+0000 (oldest deadline 2024-05-20T16:27:29.332344+0000) 2024-05-20T16:27:45.062 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 20 16:27:44 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-7[54382]: 2024-05-20T16:27:44.842+0000 7f68a2750700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-05-20T16:27:13.234707+0000 front 2024-05-20T16:27:13.234674+0000 (oldest deadline 2024-05-20T16:27:39.134461+0000) 2024-05-20T16:27:45.062 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 20 16:27:44 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-7[54382]: 2024-05-20T16:27:44.842+0000 7f68a2750700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-20T16:27:21.936147+0000 front 2024-05-20T16:27:21.936216+0000 (oldest deadline 2024-05-20T16:27:44.836201+0000) 2024-05-20T16:27:45.062 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:27:44 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:27:44.753+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-05-20T16:26:47.437555+0000 front 2024-05-20T16:26:47.437468+0000 (oldest deadline 2024-05-20T16:27:15.038423+0000) 2024-05-20T16:27:45.062 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:27:44 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:27:44.753+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:55.038908+0000 front 2024-05-20T16:26:55.038825+0000 (oldest deadline 2024-05-20T16:27:18.439444+0000) 2024-05-20T16:27:45.062 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:27:44 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:27:44.753+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-20T16:27:04.241075+0000 front 2024-05-20T16:27:04.241297+0000 (oldest deadline 2024-05-20T16:27:27.141055+0000) 2024-05-20T16:27:45.063 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:27:44 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:27:44.753+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-05-20T16:27:12.943012+0000 front 2024-05-20T16:27:12.942981+0000 (oldest deadline 2024-05-20T16:27:36.442783+0000) 2024-05-20T16:27:45.585 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:27:45 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:27:45.330+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-05-20T16:26:47.217074+0000 front 2024-05-20T16:26:47.216883+0000 (oldest deadline 2024-05-20T16:27:10.716827+0000) 2024-05-20T16:27:45.585 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:27:45 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:27:45.330+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:54.817899+0000 front 2024-05-20T16:26:54.818008+0000 (oldest deadline 2024-05-20T16:27:20.018619+0000) 2024-05-20T16:27:45.585 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:27:45 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:27:45.330+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-20T16:27:01.719536+0000 front 2024-05-20T16:27:01.719675+0000 (oldest deadline 2024-05-20T16:27:27.019603+0000) 2024-05-20T16:27:45.585 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:27:45 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:27:45.330+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-20T16:27:19.722847+0000 front 2024-05-20T16:27:19.722862+0000 (oldest deadline 2024-05-20T16:27:45.022529+0000) 2024-05-20T16:27:45.859 INFO:journalctl@ceph.osd.9.smithi082.stdout:May 20 16:27:45 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-9[66777]: 2024-05-20T16:27:45.416+0000 7fa187fc1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-05-20T16:26:44.585296+0000 front 2024-05-20T16:26:44.585359+0000 (oldest deadline 2024-05-20T16:27:09.885493+0000) 2024-05-20T16:27:45.859 INFO:journalctl@ceph.osd.9.smithi082.stdout:May 20 16:27:45 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-9[66777]: 2024-05-20T16:27:45.416+0000 7fa187fc1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:55.786620+0000 front 2024-05-20T16:26:55.786679+0000 (oldest deadline 2024-05-20T16:27:19.286485+0000) 2024-05-20T16:27:45.859 INFO:journalctl@ceph.osd.9.smithi082.stdout:May 20 16:27:45 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-9[66777]: 2024-05-20T16:27:45.416+0000 7fa187fc1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-20T16:27:03.987755+0000 front 2024-05-20T16:27:03.987552+0000 (oldest deadline 2024-05-20T16:27:28.687562+0000) 2024-05-20T16:27:45.859 INFO:journalctl@ceph.osd.9.smithi082.stdout:May 20 16:27:45 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-9[66777]: 2024-05-20T16:27:45.416+0000 7fa187fc1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-05-20T16:27:13.388520+0000 front 2024-05-20T16:27:13.388549+0000 (oldest deadline 2024-05-20T16:27:38.688716+0000) 2024-05-20T16:27:45.859 INFO:journalctl@ceph.osd.9.smithi082.stdout:May 20 16:27:45 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-9[66777]: 2024-05-20T16:27:45.416+0000 7fa187fc1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-20T16:27:19.189504+0000 front 2024-05-20T16:27:19.189629+0000 (oldest deadline 2024-05-20T16:27:44.489497+0000) 2024-05-20T16:27:45.885 INFO:journalctl@ceph.osd.8.smithi184.stdout:May 20 16:27:45 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-8[55074]: 2024-05-20T16:27:45.585+0000 7f4a3856b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:52.470105+0000 front 2024-05-20T16:26:52.470039+0000 (oldest deadline 2024-05-20T16:27:18.270525+0000) 2024-05-20T16:27:45.885 INFO:journalctl@ceph.osd.8.smithi184.stdout:May 20 16:27:45 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-8[55074]: 2024-05-20T16:27:45.585+0000 7f4a3856b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-20T16:27:05.271959+0000 front 2024-05-20T16:27:05.272026+0000 (oldest deadline 2024-05-20T16:27:28.772153+0000) 2024-05-20T16:27:45.885 INFO:journalctl@ceph.osd.8.smithi184.stdout:May 20 16:27:45 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-8[55074]: 2024-05-20T16:27:45.585+0000 7f4a3856b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-05-20T16:27:13.473547+0000 front 2024-05-20T16:27:13.473521+0000 (oldest deadline 2024-05-20T16:27:38.773369+0000) 2024-05-20T16:27:45.885 INFO:journalctl@ceph.osd.8.smithi184.stdout:May 20 16:27:45 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-8[55074]: 2024-05-20T16:27:45.585+0000 7f4a3856b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-20T16:27:22.174923+0000 front 2024-05-20T16:27:22.174955+0000 (oldest deadline 2024-05-20T16:27:45.074988+0000) 2024-05-20T16:27:46.062 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 20 16:27:45 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-7[54382]: 2024-05-20T16:27:45.808+0000 7f68a2750700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:56.630029+0000 front 2024-05-20T16:26:56.630059+0000 (oldest deadline 2024-05-20T16:27:18.930178+0000) 2024-05-20T16:27:46.062 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 20 16:27:45 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-7[54382]: 2024-05-20T16:27:45.808+0000 7f68a2750700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-20T16:27:04.032669+0000 front 2024-05-20T16:27:04.032493+0000 (oldest deadline 2024-05-20T16:27:29.332344+0000) 2024-05-20T16:27:46.062 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 20 16:27:45 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-7[54382]: 2024-05-20T16:27:45.808+0000 7f68a2750700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-05-20T16:27:13.234707+0000 front 2024-05-20T16:27:13.234674+0000 (oldest deadline 2024-05-20T16:27:39.134461+0000) 2024-05-20T16:27:46.062 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 20 16:27:45 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-7[54382]: 2024-05-20T16:27:45.808+0000 7f68a2750700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-20T16:27:21.936147+0000 front 2024-05-20T16:27:21.936216+0000 (oldest deadline 2024-05-20T16:27:44.836201+0000) 2024-05-20T16:27:46.063 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:27:45 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:27:45.738+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-05-20T16:26:47.437555+0000 front 2024-05-20T16:26:47.437468+0000 (oldest deadline 2024-05-20T16:27:15.038423+0000) 2024-05-20T16:27:46.063 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:27:45 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:27:45.738+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:55.038908+0000 front 2024-05-20T16:26:55.038825+0000 (oldest deadline 2024-05-20T16:27:18.439444+0000) 2024-05-20T16:27:46.063 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:27:45 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:27:45.738+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-20T16:27:04.241075+0000 front 2024-05-20T16:27:04.241297+0000 (oldest deadline 2024-05-20T16:27:27.141055+0000) 2024-05-20T16:27:46.063 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:27:45 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:27:45.738+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-05-20T16:27:12.943012+0000 front 2024-05-20T16:27:12.942981+0000 (oldest deadline 2024-05-20T16:27:36.442783+0000) 2024-05-20T16:27:46.635 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:27:46 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:27:46.334+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-05-20T16:26:47.217074+0000 front 2024-05-20T16:26:47.216883+0000 (oldest deadline 2024-05-20T16:27:10.716827+0000) 2024-05-20T16:27:46.635 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:27:46 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:27:46.334+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:54.817899+0000 front 2024-05-20T16:26:54.818008+0000 (oldest deadline 2024-05-20T16:27:20.018619+0000) 2024-05-20T16:27:46.635 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:27:46 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:27:46.334+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-20T16:27:01.719536+0000 front 2024-05-20T16:27:01.719675+0000 (oldest deadline 2024-05-20T16:27:27.019603+0000) 2024-05-20T16:27:46.635 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:27:46 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:27:46.334+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-20T16:27:19.722847+0000 front 2024-05-20T16:27:19.722862+0000 (oldest deadline 2024-05-20T16:27:45.022529+0000) 2024-05-20T16:27:46.636 INFO:journalctl@ceph.osd.8.smithi184.stdout:May 20 16:27:46 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-8[55074]: 2024-05-20T16:27:46.578+0000 7f4a3856b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:52.470105+0000 front 2024-05-20T16:26:52.470039+0000 (oldest deadline 2024-05-20T16:27:18.270525+0000) 2024-05-20T16:27:46.636 INFO:journalctl@ceph.osd.8.smithi184.stdout:May 20 16:27:46 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-8[55074]: 2024-05-20T16:27:46.578+0000 7f4a3856b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-20T16:27:05.271959+0000 front 2024-05-20T16:27:05.272026+0000 (oldest deadline 2024-05-20T16:27:28.772153+0000) 2024-05-20T16:27:46.636 INFO:journalctl@ceph.osd.8.smithi184.stdout:May 20 16:27:46 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-8[55074]: 2024-05-20T16:27:46.578+0000 7f4a3856b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-05-20T16:27:13.473547+0000 front 2024-05-20T16:27:13.473521+0000 (oldest deadline 2024-05-20T16:27:38.773369+0000) 2024-05-20T16:27:46.636 INFO:journalctl@ceph.osd.8.smithi184.stdout:May 20 16:27:46 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-8[55074]: 2024-05-20T16:27:46.578+0000 7f4a3856b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-20T16:27:22.174923+0000 front 2024-05-20T16:27:22.174955+0000 (oldest deadline 2024-05-20T16:27:45.074988+0000) 2024-05-20T16:27:46.859 INFO:journalctl@ceph.osd.9.smithi082.stdout:May 20 16:27:46 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-9[66777]: 2024-05-20T16:27:46.390+0000 7fa187fc1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-05-20T16:26:44.585296+0000 front 2024-05-20T16:26:44.585359+0000 (oldest deadline 2024-05-20T16:27:09.885493+0000) 2024-05-20T16:27:46.859 INFO:journalctl@ceph.osd.9.smithi082.stdout:May 20 16:27:46 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-9[66777]: 2024-05-20T16:27:46.390+0000 7fa187fc1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:55.786620+0000 front 2024-05-20T16:26:55.786679+0000 (oldest deadline 2024-05-20T16:27:19.286485+0000) 2024-05-20T16:27:46.859 INFO:journalctl@ceph.osd.9.smithi082.stdout:May 20 16:27:46 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-9[66777]: 2024-05-20T16:27:46.390+0000 7fa187fc1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-20T16:27:03.987755+0000 front 2024-05-20T16:27:03.987552+0000 (oldest deadline 2024-05-20T16:27:28.687562+0000) 2024-05-20T16:27:46.859 INFO:journalctl@ceph.osd.9.smithi082.stdout:May 20 16:27:46 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-9[66777]: 2024-05-20T16:27:46.390+0000 7fa187fc1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-05-20T16:27:13.388520+0000 front 2024-05-20T16:27:13.388549+0000 (oldest deadline 2024-05-20T16:27:38.688716+0000) 2024-05-20T16:27:46.859 INFO:journalctl@ceph.osd.9.smithi082.stdout:May 20 16:27:46 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-9[66777]: 2024-05-20T16:27:46.390+0000 7fa187fc1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-20T16:27:19.189504+0000 front 2024-05-20T16:27:19.189629+0000 (oldest deadline 2024-05-20T16:27:44.489497+0000) 2024-05-20T16:27:47.062 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 20 16:27:46 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-7[54382]: 2024-05-20T16:27:46.778+0000 7f68a2750700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:56.630029+0000 front 2024-05-20T16:26:56.630059+0000 (oldest deadline 2024-05-20T16:27:18.930178+0000) 2024-05-20T16:27:47.062 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 20 16:27:46 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-7[54382]: 2024-05-20T16:27:46.778+0000 7f68a2750700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-20T16:27:04.032669+0000 front 2024-05-20T16:27:04.032493+0000 (oldest deadline 2024-05-20T16:27:29.332344+0000) 2024-05-20T16:27:47.062 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 20 16:27:46 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-7[54382]: 2024-05-20T16:27:46.778+0000 7f68a2750700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-05-20T16:27:13.234707+0000 front 2024-05-20T16:27:13.234674+0000 (oldest deadline 2024-05-20T16:27:39.134461+0000) 2024-05-20T16:27:47.062 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 20 16:27:46 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-7[54382]: 2024-05-20T16:27:46.778+0000 7f68a2750700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-20T16:27:21.936147+0000 front 2024-05-20T16:27:21.936216+0000 (oldest deadline 2024-05-20T16:27:44.836201+0000) 2024-05-20T16:27:47.062 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:27:46 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:27:46.752+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-05-20T16:26:47.437555+0000 front 2024-05-20T16:26:47.437468+0000 (oldest deadline 2024-05-20T16:27:15.038423+0000) 2024-05-20T16:27:47.062 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:27:46 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:27:46.752+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:55.038908+0000 front 2024-05-20T16:26:55.038825+0000 (oldest deadline 2024-05-20T16:27:18.439444+0000) 2024-05-20T16:27:47.063 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:27:46 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:27:46.752+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-20T16:27:04.241075+0000 front 2024-05-20T16:27:04.241297+0000 (oldest deadline 2024-05-20T16:27:27.141055+0000) 2024-05-20T16:27:47.063 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:27:46 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:27:46.752+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-05-20T16:27:12.943012+0000 front 2024-05-20T16:27:12.942981+0000 (oldest deadline 2024-05-20T16:27:36.442783+0000) 2024-05-20T16:27:47.063 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:27:46 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:27:46.752+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-20T16:27:21.643969+0000 front 2024-05-20T16:27:21.644003+0000 (oldest deadline 2024-05-20T16:27:46.344106+0000) 2024-05-20T16:27:47.578 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:27:47 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:27:47.314+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-05-20T16:26:47.217074+0000 front 2024-05-20T16:26:47.216883+0000 (oldest deadline 2024-05-20T16:27:10.716827+0000) 2024-05-20T16:27:47.578 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:27:47 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:27:47.314+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:54.817899+0000 front 2024-05-20T16:26:54.818008+0000 (oldest deadline 2024-05-20T16:27:20.018619+0000) 2024-05-20T16:27:47.579 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:27:47 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:27:47.314+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-20T16:27:01.719536+0000 front 2024-05-20T16:27:01.719675+0000 (oldest deadline 2024-05-20T16:27:27.019603+0000) 2024-05-20T16:27:47.579 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:27:47 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:27:47.314+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-20T16:27:19.722847+0000 front 2024-05-20T16:27:19.722862+0000 (oldest deadline 2024-05-20T16:27:45.022529+0000) 2024-05-20T16:27:47.609 INFO:journalctl@ceph.osd.9.smithi082.stdout:May 20 16:27:47 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-9[66777]: 2024-05-20T16:27:47.348+0000 7fa187fc1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-05-20T16:26:44.585296+0000 front 2024-05-20T16:26:44.585359+0000 (oldest deadline 2024-05-20T16:27:09.885493+0000) 2024-05-20T16:27:47.609 INFO:journalctl@ceph.osd.9.smithi082.stdout:May 20 16:27:47 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-9[66777]: 2024-05-20T16:27:47.348+0000 7fa187fc1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:55.786620+0000 front 2024-05-20T16:26:55.786679+0000 (oldest deadline 2024-05-20T16:27:19.286485+0000) 2024-05-20T16:27:47.609 INFO:journalctl@ceph.osd.9.smithi082.stdout:May 20 16:27:47 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-9[66777]: 2024-05-20T16:27:47.348+0000 7fa187fc1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-20T16:27:03.987755+0000 front 2024-05-20T16:27:03.987552+0000 (oldest deadline 2024-05-20T16:27:28.687562+0000) 2024-05-20T16:27:47.609 INFO:journalctl@ceph.osd.9.smithi082.stdout:May 20 16:27:47 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-9[66777]: 2024-05-20T16:27:47.348+0000 7fa187fc1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-05-20T16:27:13.388520+0000 front 2024-05-20T16:27:13.388549+0000 (oldest deadline 2024-05-20T16:27:38.688716+0000) 2024-05-20T16:27:47.609 INFO:journalctl@ceph.osd.9.smithi082.stdout:May 20 16:27:47 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-9[66777]: 2024-05-20T16:27:47.348+0000 7fa187fc1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-20T16:27:19.189504+0000 front 2024-05-20T16:27:19.189629+0000 (oldest deadline 2024-05-20T16:27:44.489497+0000) 2024-05-20T16:27:47.885 INFO:journalctl@ceph.osd.8.smithi184.stdout:May 20 16:27:47 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-8[55074]: 2024-05-20T16:27:47.579+0000 7f4a3856b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:52.470105+0000 front 2024-05-20T16:26:52.470039+0000 (oldest deadline 2024-05-20T16:27:18.270525+0000) 2024-05-20T16:27:47.885 INFO:journalctl@ceph.osd.8.smithi184.stdout:May 20 16:27:47 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-8[55074]: 2024-05-20T16:27:47.579+0000 7f4a3856b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-20T16:27:05.271959+0000 front 2024-05-20T16:27:05.272026+0000 (oldest deadline 2024-05-20T16:27:28.772153+0000) 2024-05-20T16:27:47.885 INFO:journalctl@ceph.osd.8.smithi184.stdout:May 20 16:27:47 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-8[55074]: 2024-05-20T16:27:47.579+0000 7f4a3856b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-05-20T16:27:13.473547+0000 front 2024-05-20T16:27:13.473521+0000 (oldest deadline 2024-05-20T16:27:38.773369+0000) 2024-05-20T16:27:47.885 INFO:journalctl@ceph.osd.8.smithi184.stdout:May 20 16:27:47 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-8[55074]: 2024-05-20T16:27:47.579+0000 7f4a3856b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-20T16:27:22.174923+0000 front 2024-05-20T16:27:22.174955+0000 (oldest deadline 2024-05-20T16:27:45.074988+0000) 2024-05-20T16:27:48.062 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:27:47 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:27:47.773+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-05-20T16:26:47.437555+0000 front 2024-05-20T16:26:47.437468+0000 (oldest deadline 2024-05-20T16:27:15.038423+0000) 2024-05-20T16:27:48.062 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:27:47 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:27:47.773+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:55.038908+0000 front 2024-05-20T16:26:55.038825+0000 (oldest deadline 2024-05-20T16:27:18.439444+0000) 2024-05-20T16:27:48.062 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:27:47 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:27:47.773+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-20T16:27:04.241075+0000 front 2024-05-20T16:27:04.241297+0000 (oldest deadline 2024-05-20T16:27:27.141055+0000) 2024-05-20T16:27:48.062 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:27:47 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:27:47.773+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-05-20T16:27:12.943012+0000 front 2024-05-20T16:27:12.942981+0000 (oldest deadline 2024-05-20T16:27:36.442783+0000) 2024-05-20T16:27:48.062 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:27:47 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:27:47.773+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-20T16:27:21.643969+0000 front 2024-05-20T16:27:21.644003+0000 (oldest deadline 2024-05-20T16:27:46.344106+0000) 2024-05-20T16:27:48.553 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 20 16:27:48 smithi160 podman[72088]: 2024-05-20 16:27:48.234799086 +0000 UTC m=+5.793853596 container died d91279f7998d5de896103241ea0c0460bbd4ab3ae43bfc2d6741e461e2056fc0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-7, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, org.label-schema.build-date=20240514) 2024-05-20T16:27:48.609 INFO:journalctl@ceph.osd.9.smithi082.stdout:May 20 16:27:48 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-9[66777]: 2024-05-20T16:27:48.340+0000 7fa187fc1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-05-20T16:26:44.585296+0000 front 2024-05-20T16:26:44.585359+0000 (oldest deadline 2024-05-20T16:27:09.885493+0000) 2024-05-20T16:27:48.609 INFO:journalctl@ceph.osd.9.smithi082.stdout:May 20 16:27:48 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-9[66777]: 2024-05-20T16:27:48.340+0000 7fa187fc1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:55.786620+0000 front 2024-05-20T16:26:55.786679+0000 (oldest deadline 2024-05-20T16:27:19.286485+0000) 2024-05-20T16:27:48.609 INFO:journalctl@ceph.osd.9.smithi082.stdout:May 20 16:27:48 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-9[66777]: 2024-05-20T16:27:48.340+0000 7fa187fc1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-20T16:27:03.987755+0000 front 2024-05-20T16:27:03.987552+0000 (oldest deadline 2024-05-20T16:27:28.687562+0000) 2024-05-20T16:27:48.609 INFO:journalctl@ceph.osd.9.smithi082.stdout:May 20 16:27:48 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-9[66777]: 2024-05-20T16:27:48.340+0000 7fa187fc1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-05-20T16:27:13.388520+0000 front 2024-05-20T16:27:13.388549+0000 (oldest deadline 2024-05-20T16:27:38.688716+0000) 2024-05-20T16:27:48.609 INFO:journalctl@ceph.osd.9.smithi082.stdout:May 20 16:27:48 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-9[66777]: 2024-05-20T16:27:48.340+0000 7fa187fc1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-20T16:27:19.189504+0000 front 2024-05-20T16:27:19.189629+0000 (oldest deadline 2024-05-20T16:27:44.489497+0000) 2024-05-20T16:27:48.636 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:27:48 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:27:48.300+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-05-20T16:26:47.217074+0000 front 2024-05-20T16:26:47.216883+0000 (oldest deadline 2024-05-20T16:27:10.716827+0000) 2024-05-20T16:27:48.636 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:27:48 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:27:48.300+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:54.817899+0000 front 2024-05-20T16:26:54.818008+0000 (oldest deadline 2024-05-20T16:27:20.018619+0000) 2024-05-20T16:27:48.636 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:27:48 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:27:48.300+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-20T16:27:01.719536+0000 front 2024-05-20T16:27:01.719675+0000 (oldest deadline 2024-05-20T16:27:27.019603+0000) 2024-05-20T16:27:48.636 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:27:48 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:27:48.300+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-20T16:27:19.722847+0000 front 2024-05-20T16:27:19.722862+0000 (oldest deadline 2024-05-20T16:27:45.022529+0000) 2024-05-20T16:27:48.636 INFO:journalctl@ceph.osd.8.smithi184.stdout:May 20 16:27:48 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-8[55074]: 2024-05-20T16:27:48.547+0000 7f4a3856b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:52.470105+0000 front 2024-05-20T16:26:52.470039+0000 (oldest deadline 2024-05-20T16:27:18.270525+0000) 2024-05-20T16:27:48.636 INFO:journalctl@ceph.osd.8.smithi184.stdout:May 20 16:27:48 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-8[55074]: 2024-05-20T16:27:48.547+0000 7f4a3856b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-20T16:27:05.271959+0000 front 2024-05-20T16:27:05.272026+0000 (oldest deadline 2024-05-20T16:27:28.772153+0000) 2024-05-20T16:27:48.636 INFO:journalctl@ceph.osd.8.smithi184.stdout:May 20 16:27:48 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-8[55074]: 2024-05-20T16:27:48.547+0000 7f4a3856b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-05-20T16:27:13.473547+0000 front 2024-05-20T16:27:13.473521+0000 (oldest deadline 2024-05-20T16:27:38.773369+0000) 2024-05-20T16:27:48.636 INFO:journalctl@ceph.osd.8.smithi184.stdout:May 20 16:27:48 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-8[55074]: 2024-05-20T16:27:48.547+0000 7f4a3856b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-20T16:27:22.174923+0000 front 2024-05-20T16:27:22.174955+0000 (oldest deadline 2024-05-20T16:27:45.074988+0000) 2024-05-20T16:27:48.812 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 20 16:27:48 smithi160 podman[72088]: 2024-05-20 16:27:48.673284201 +0000 UTC m=+6.232338710 container cleanup d91279f7998d5de896103241ea0c0460bbd4ab3ae43bfc2d6741e461e2056fc0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-7, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240514, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-05-20T16:27:48.812 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 20 16:27:48 smithi160 bash[72088]: ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-7 2024-05-20T16:27:48.812 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:27:48 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:27:48.792+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-05-20T16:26:47.437555+0000 front 2024-05-20T16:26:47.437468+0000 (oldest deadline 2024-05-20T16:27:15.038423+0000) 2024-05-20T16:27:48.813 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:27:48 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:27:48.792+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:55.038908+0000 front 2024-05-20T16:26:55.038825+0000 (oldest deadline 2024-05-20T16:27:18.439444+0000) 2024-05-20T16:27:48.813 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:27:48 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:27:48.792+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-20T16:27:04.241075+0000 front 2024-05-20T16:27:04.241297+0000 (oldest deadline 2024-05-20T16:27:27.141055+0000) 2024-05-20T16:27:48.813 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:27:48 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:27:48.792+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-05-20T16:27:12.943012+0000 front 2024-05-20T16:27:12.942981+0000 (oldest deadline 2024-05-20T16:27:36.442783+0000) 2024-05-20T16:27:48.813 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:27:48 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:27:48.792+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-20T16:27:21.643969+0000 front 2024-05-20T16:27:21.644003+0000 (oldest deadline 2024-05-20T16:27:46.344106+0000) 2024-05-20T16:27:49.312 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 20 16:27:48 smithi160 podman[72109]: 2024-05-20 16:27:48.89972785 +0000 UTC m=+0.656484686 container remove d91279f7998d5de896103241ea0c0460bbd4ab3ae43bfc2d6741e461e2056fc0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, io.buildah.version=1.33.5, org.label-schema.build-date=20240514, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-05-20T16:27:49.532 INFO:journalctl@ceph.osd.8.smithi184.stdout:May 20 16:27:49 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-8[55074]: 2024-05-20T16:27:49.532+0000 7f4a3856b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:52.470105+0000 front 2024-05-20T16:26:52.470039+0000 (oldest deadline 2024-05-20T16:27:18.270525+0000) 2024-05-20T16:27:49.532 INFO:journalctl@ceph.osd.8.smithi184.stdout:May 20 16:27:49 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-8[55074]: 2024-05-20T16:27:49.532+0000 7f4a3856b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-20T16:27:05.271959+0000 front 2024-05-20T16:27:05.272026+0000 (oldest deadline 2024-05-20T16:27:28.772153+0000) 2024-05-20T16:27:49.532 INFO:journalctl@ceph.osd.8.smithi184.stdout:May 20 16:27:49 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-8[55074]: 2024-05-20T16:27:49.532+0000 7f4a3856b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-05-20T16:27:13.473547+0000 front 2024-05-20T16:27:13.473521+0000 (oldest deadline 2024-05-20T16:27:38.773369+0000) 2024-05-20T16:27:49.532 INFO:journalctl@ceph.osd.8.smithi184.stdout:May 20 16:27:49 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-8[55074]: 2024-05-20T16:27:49.532+0000 7f4a3856b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-20T16:27:22.174923+0000 front 2024-05-20T16:27:22.174955+0000 (oldest deadline 2024-05-20T16:27:45.074988+0000) 2024-05-20T16:27:49.533 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:27:49 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:27:49.282+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-05-20T16:26:47.217074+0000 front 2024-05-20T16:26:47.216883+0000 (oldest deadline 2024-05-20T16:27:10.716827+0000) 2024-05-20T16:27:49.533 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:27:49 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:27:49.282+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:54.817899+0000 front 2024-05-20T16:26:54.818008+0000 (oldest deadline 2024-05-20T16:27:20.018619+0000) 2024-05-20T16:27:49.533 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:27:49 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:27:49.282+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-20T16:27:01.719536+0000 front 2024-05-20T16:27:01.719675+0000 (oldest deadline 2024-05-20T16:27:27.019603+0000) 2024-05-20T16:27:49.533 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:27:49 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:27:49.282+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-20T16:27:19.722847+0000 front 2024-05-20T16:27:19.722862+0000 (oldest deadline 2024-05-20T16:27:45.022529+0000) 2024-05-20T16:27:49.594 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 20 16:27:49 smithi160 podman[72198]: 2024-05-20 16:27:49.239689881 +0000 UTC m=+0.017335551 image pull cfcc61b2d476f611479310eb042aa05f11dd5f60dbb65e3b0d7e0e11d024cdb3 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708 2024-05-20T16:27:49.594 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 20 16:27:49 smithi160 podman[72198]: 2024-05-20 16:27:49.367429198 +0000 UTC m=+0.145074860 container create b4eac4ebd0823677909872f566229e185110c0dfaaf76d8774f1eeaf16f0f012 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-7-deactivate, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240514) 2024-05-20T16:27:49.785 INFO:journalctl@ceph.osd.9.smithi082.stdout:May 20 16:27:49 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-9[66777]: 2024-05-20T16:27:49.375+0000 7fa187fc1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-05-20T16:26:44.585296+0000 front 2024-05-20T16:26:44.585359+0000 (oldest deadline 2024-05-20T16:27:09.885493+0000) 2024-05-20T16:27:49.786 INFO:journalctl@ceph.osd.9.smithi082.stdout:May 20 16:27:49 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-9[66777]: 2024-05-20T16:27:49.375+0000 7fa187fc1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:55.786620+0000 front 2024-05-20T16:26:55.786679+0000 (oldest deadline 2024-05-20T16:27:19.286485+0000) 2024-05-20T16:27:49.786 INFO:journalctl@ceph.osd.9.smithi082.stdout:May 20 16:27:49 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-9[66777]: 2024-05-20T16:27:49.375+0000 7fa187fc1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-20T16:27:03.987755+0000 front 2024-05-20T16:27:03.987552+0000 (oldest deadline 2024-05-20T16:27:28.687562+0000) 2024-05-20T16:27:49.786 INFO:journalctl@ceph.osd.9.smithi082.stdout:May 20 16:27:49 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-9[66777]: 2024-05-20T16:27:49.375+0000 7fa187fc1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-05-20T16:27:13.388520+0000 front 2024-05-20T16:27:13.388549+0000 (oldest deadline 2024-05-20T16:27:38.688716+0000) 2024-05-20T16:27:49.786 INFO:journalctl@ceph.osd.9.smithi082.stdout:May 20 16:27:49 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-9[66777]: 2024-05-20T16:27:49.375+0000 7fa187fc1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-20T16:27:19.189504+0000 front 2024-05-20T16:27:19.189629+0000 (oldest deadline 2024-05-20T16:27:44.489497+0000) 2024-05-20T16:27:49.896 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:27:49 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:27:49.821+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-05-20T16:26:47.437555+0000 front 2024-05-20T16:26:47.437468+0000 (oldest deadline 2024-05-20T16:27:15.038423+0000) 2024-05-20T16:27:49.896 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:27:49 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:27:49.821+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:55.038908+0000 front 2024-05-20T16:26:55.038825+0000 (oldest deadline 2024-05-20T16:27:18.439444+0000) 2024-05-20T16:27:49.897 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:27:49 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:27:49.821+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-20T16:27:04.241075+0000 front 2024-05-20T16:27:04.241297+0000 (oldest deadline 2024-05-20T16:27:27.141055+0000) 2024-05-20T16:27:49.897 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:27:49 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:27:49.821+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-05-20T16:27:12.943012+0000 front 2024-05-20T16:27:12.942981+0000 (oldest deadline 2024-05-20T16:27:36.442783+0000) 2024-05-20T16:27:49.897 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:27:49 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:27:49.821+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-20T16:27:21.643969+0000 front 2024-05-20T16:27:21.644003+0000 (oldest deadline 2024-05-20T16:27:46.344106+0000) 2024-05-20T16:27:49.897 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 20 16:27:49 smithi160 podman[72198]: 2024-05-20 16:27:49.595089917 +0000 UTC m=+0.372735590 container init b4eac4ebd0823677909872f566229e185110c0dfaaf76d8774f1eeaf16f0f012 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-7-deactivate, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240514, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-05-20T16:27:49.897 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 20 16:27:49 smithi160 podman[72198]: 2024-05-20 16:27:49.598568709 +0000 UTC m=+0.376214379 container start b4eac4ebd0823677909872f566229e185110c0dfaaf76d8774f1eeaf16f0f012 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-7-deactivate, org.label-schema.build-date=20240514, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, io.buildah.version=1.33.5, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-20T16:27:49.897 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 20 16:27:49 smithi160 podman[72198]: 2024-05-20 16:27:49.703592383 +0000 UTC m=+0.481238063 container attach b4eac4ebd0823677909872f566229e185110c0dfaaf76d8774f1eeaf16f0f012 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-7-deactivate, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240514, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-20T16:27:50.239 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 20 16:27:49 smithi160 podman[72198]: 2024-05-20 16:27:49.898107575 +0000 UTC m=+0.675753254 container died b4eac4ebd0823677909872f566229e185110c0dfaaf76d8774f1eeaf16f0f012 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-7-deactivate, org.label-schema.license=GPLv2, org.label-schema.build-date=20240514, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-05-20T16:27:50.572 DEBUG:teuthology.orchestra.run.smithi160:> sudo pkill -f 'journalctl -f -n 0 -u ceph-59d1600a-16c2-11ef-bc9a-c7b262605968@osd.7.service' 2024-05-20T16:27:50.578 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:27:50 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:27:50.283+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-05-20T16:26:47.217074+0000 front 2024-05-20T16:26:47.216883+0000 (oldest deadline 2024-05-20T16:27:10.716827+0000) 2024-05-20T16:27:50.578 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:27:50 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:27:50.283+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:54.817899+0000 front 2024-05-20T16:26:54.818008+0000 (oldest deadline 2024-05-20T16:27:20.018619+0000) 2024-05-20T16:27:50.579 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:27:50 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:27:50.283+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-20T16:27:01.719536+0000 front 2024-05-20T16:27:01.719675+0000 (oldest deadline 2024-05-20T16:27:27.019603+0000) 2024-05-20T16:27:50.579 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:27:50 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:27:50.283+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-20T16:27:19.722847+0000 front 2024-05-20T16:27:19.722862+0000 (oldest deadline 2024-05-20T16:27:45.022529+0000) 2024-05-20T16:27:50.609 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-20T16:27:50.609 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-05-20T16:27:50.609 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-05-20T16:27:50.609 DEBUG:teuthology.orchestra.run.smithi184:> sudo systemctl stop ceph-59d1600a-16c2-11ef-bc9a-c7b262605968@osd.8 2024-05-20T16:27:50.714 INFO:journalctl@ceph.osd.9.smithi082.stdout:May 20 16:27:50 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-9[66777]: 2024-05-20T16:27:50.406+0000 7fa187fc1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-05-20T16:26:44.585296+0000 front 2024-05-20T16:26:44.585359+0000 (oldest deadline 2024-05-20T16:27:09.885493+0000) 2024-05-20T16:27:50.714 INFO:journalctl@ceph.osd.9.smithi082.stdout:May 20 16:27:50 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-9[66777]: 2024-05-20T16:27:50.406+0000 7fa187fc1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:55.786620+0000 front 2024-05-20T16:26:55.786679+0000 (oldest deadline 2024-05-20T16:27:19.286485+0000) 2024-05-20T16:27:50.714 INFO:journalctl@ceph.osd.9.smithi082.stdout:May 20 16:27:50 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-9[66777]: 2024-05-20T16:27:50.406+0000 7fa187fc1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-20T16:27:03.987755+0000 front 2024-05-20T16:27:03.987552+0000 (oldest deadline 2024-05-20T16:27:28.687562+0000) 2024-05-20T16:27:50.714 INFO:journalctl@ceph.osd.9.smithi082.stdout:May 20 16:27:50 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-9[66777]: 2024-05-20T16:27:50.406+0000 7fa187fc1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-05-20T16:27:13.388520+0000 front 2024-05-20T16:27:13.388549+0000 (oldest deadline 2024-05-20T16:27:38.688716+0000) 2024-05-20T16:27:50.714 INFO:journalctl@ceph.osd.9.smithi082.stdout:May 20 16:27:50 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-9[66777]: 2024-05-20T16:27:50.406+0000 7fa187fc1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-20T16:27:19.189504+0000 front 2024-05-20T16:27:19.189629+0000 (oldest deadline 2024-05-20T16:27:44.489497+0000) 2024-05-20T16:27:50.885 INFO:journalctl@ceph.osd.8.smithi184.stdout:May 20 16:27:50 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-8[55074]: 2024-05-20T16:27:50.579+0000 7f4a3856b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:52.470105+0000 front 2024-05-20T16:26:52.470039+0000 (oldest deadline 2024-05-20T16:27:18.270525+0000) 2024-05-20T16:27:50.885 INFO:journalctl@ceph.osd.8.smithi184.stdout:May 20 16:27:50 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-8[55074]: 2024-05-20T16:27:50.579+0000 7f4a3856b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-20T16:27:05.271959+0000 front 2024-05-20T16:27:05.272026+0000 (oldest deadline 2024-05-20T16:27:28.772153+0000) 2024-05-20T16:27:50.885 INFO:journalctl@ceph.osd.8.smithi184.stdout:May 20 16:27:50 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-8[55074]: 2024-05-20T16:27:50.579+0000 7f4a3856b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-05-20T16:27:13.473547+0000 front 2024-05-20T16:27:13.473521+0000 (oldest deadline 2024-05-20T16:27:38.773369+0000) 2024-05-20T16:27:50.886 INFO:journalctl@ceph.osd.8.smithi184.stdout:May 20 16:27:50 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-8[55074]: 2024-05-20T16:27:50.579+0000 7f4a3856b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-20T16:27:22.174923+0000 front 2024-05-20T16:27:22.174955+0000 (oldest deadline 2024-05-20T16:27:45.074988+0000) 2024-05-20T16:27:50.886 INFO:journalctl@ceph.osd.8.smithi184.stdout:May 20 16:27:50 smithi184 systemd[1]: Stopping Ceph osd.8 for 59d1600a-16c2-11ef-bc9a-c7b262605968... 2024-05-20T16:27:51.291 INFO:journalctl@ceph.osd.8.smithi184.stdout:May 20 16:27:50 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-8[55074]: 2024-05-20T16:27:50.935+0000 7f4a3ffbd700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-20T16:27:51.291 INFO:journalctl@ceph.osd.8.smithi184.stdout:May 20 16:27:50 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-8[55074]: 2024-05-20T16:27:50.935+0000 7f4a3ffbd700 -1 osd.8 102 *** Got signal Terminated *** 2024-05-20T16:27:51.291 INFO:journalctl@ceph.osd.8.smithi184.stdout:May 20 16:27:50 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-8[55074]: 2024-05-20T16:27:50.935+0000 7f4a3ffbd700 -1 osd.8 102 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-20T16:27:51.312 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:27:50 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:27:50.829+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-05-20T16:26:47.437555+0000 front 2024-05-20T16:26:47.437468+0000 (oldest deadline 2024-05-20T16:27:15.038423+0000) 2024-05-20T16:27:51.312 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:27:50 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:27:50.829+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:55.038908+0000 front 2024-05-20T16:26:55.038825+0000 (oldest deadline 2024-05-20T16:27:18.439444+0000) 2024-05-20T16:27:51.312 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:27:50 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:27:50.829+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-20T16:27:04.241075+0000 front 2024-05-20T16:27:04.241297+0000 (oldest deadline 2024-05-20T16:27:27.141055+0000) 2024-05-20T16:27:51.312 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:27:50 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:27:50.829+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-05-20T16:27:12.943012+0000 front 2024-05-20T16:27:12.942981+0000 (oldest deadline 2024-05-20T16:27:36.442783+0000) 2024-05-20T16:27:51.312 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:27:50 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:27:50.829+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-20T16:27:21.643969+0000 front 2024-05-20T16:27:21.644003+0000 (oldest deadline 2024-05-20T16:27:46.344106+0000) 2024-05-20T16:27:51.550 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:27:51 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:27:51.292+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-05-20T16:26:47.217074+0000 front 2024-05-20T16:26:47.216883+0000 (oldest deadline 2024-05-20T16:27:10.716827+0000) 2024-05-20T16:27:51.550 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:27:51 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:27:51.292+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:54.817899+0000 front 2024-05-20T16:26:54.818008+0000 (oldest deadline 2024-05-20T16:27:20.018619+0000) 2024-05-20T16:27:51.550 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:27:51 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:27:51.292+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-20T16:27:01.719536+0000 front 2024-05-20T16:27:01.719675+0000 (oldest deadline 2024-05-20T16:27:27.019603+0000) 2024-05-20T16:27:51.550 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:27:51 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:27:51.292+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-20T16:27:19.722847+0000 front 2024-05-20T16:27:19.722862+0000 (oldest deadline 2024-05-20T16:27:45.022529+0000) 2024-05-20T16:27:51.859 INFO:journalctl@ceph.osd.9.smithi082.stdout:May 20 16:27:51 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-9[66777]: 2024-05-20T16:27:51.451+0000 7fa187fc1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-05-20T16:26:44.585296+0000 front 2024-05-20T16:26:44.585359+0000 (oldest deadline 2024-05-20T16:27:09.885493+0000) 2024-05-20T16:27:51.859 INFO:journalctl@ceph.osd.9.smithi082.stdout:May 20 16:27:51 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-9[66777]: 2024-05-20T16:27:51.451+0000 7fa187fc1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:55.786620+0000 front 2024-05-20T16:26:55.786679+0000 (oldest deadline 2024-05-20T16:27:19.286485+0000) 2024-05-20T16:27:51.859 INFO:journalctl@ceph.osd.9.smithi082.stdout:May 20 16:27:51 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-9[66777]: 2024-05-20T16:27:51.451+0000 7fa187fc1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-20T16:27:03.987755+0000 front 2024-05-20T16:27:03.987552+0000 (oldest deadline 2024-05-20T16:27:28.687562+0000) 2024-05-20T16:27:51.859 INFO:journalctl@ceph.osd.9.smithi082.stdout:May 20 16:27:51 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-9[66777]: 2024-05-20T16:27:51.451+0000 7fa187fc1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-05-20T16:27:13.388520+0000 front 2024-05-20T16:27:13.388549+0000 (oldest deadline 2024-05-20T16:27:38.688716+0000) 2024-05-20T16:27:51.859 INFO:journalctl@ceph.osd.9.smithi082.stdout:May 20 16:27:51 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-9[66777]: 2024-05-20T16:27:51.451+0000 7fa187fc1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-20T16:27:19.189504+0000 front 2024-05-20T16:27:19.189629+0000 (oldest deadline 2024-05-20T16:27:44.489497+0000) 2024-05-20T16:27:51.885 INFO:journalctl@ceph.osd.8.smithi184.stdout:May 20 16:27:51 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-8[55074]: 2024-05-20T16:27:51.551+0000 7f4a3856b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:52.470105+0000 front 2024-05-20T16:26:52.470039+0000 (oldest deadline 2024-05-20T16:27:18.270525+0000) 2024-05-20T16:27:51.885 INFO:journalctl@ceph.osd.8.smithi184.stdout:May 20 16:27:51 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-8[55074]: 2024-05-20T16:27:51.551+0000 7f4a3856b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-20T16:27:05.271959+0000 front 2024-05-20T16:27:05.272026+0000 (oldest deadline 2024-05-20T16:27:28.772153+0000) 2024-05-20T16:27:51.885 INFO:journalctl@ceph.osd.8.smithi184.stdout:May 20 16:27:51 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-8[55074]: 2024-05-20T16:27:51.551+0000 7f4a3856b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-05-20T16:27:13.473547+0000 front 2024-05-20T16:27:13.473521+0000 (oldest deadline 2024-05-20T16:27:38.773369+0000) 2024-05-20T16:27:51.885 INFO:journalctl@ceph.osd.8.smithi184.stdout:May 20 16:27:51 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-8[55074]: 2024-05-20T16:27:51.551+0000 7f4a3856b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-20T16:27:22.174923+0000 front 2024-05-20T16:27:22.174955+0000 (oldest deadline 2024-05-20T16:27:45.074988+0000) 2024-05-20T16:27:52.312 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:27:51 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:27:51.825+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-05-20T16:26:47.437555+0000 front 2024-05-20T16:26:47.437468+0000 (oldest deadline 2024-05-20T16:27:15.038423+0000) 2024-05-20T16:27:52.312 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:27:51 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:27:51.825+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:55.038908+0000 front 2024-05-20T16:26:55.038825+0000 (oldest deadline 2024-05-20T16:27:18.439444+0000) 2024-05-20T16:27:52.312 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:27:51 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:27:51.825+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-20T16:27:04.241075+0000 front 2024-05-20T16:27:04.241297+0000 (oldest deadline 2024-05-20T16:27:27.141055+0000) 2024-05-20T16:27:52.312 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:27:51 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:27:51.825+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-05-20T16:27:12.943012+0000 front 2024-05-20T16:27:12.942981+0000 (oldest deadline 2024-05-20T16:27:36.442783+0000) 2024-05-20T16:27:52.312 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:27:51 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:27:51.825+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-20T16:27:21.643969+0000 front 2024-05-20T16:27:21.644003+0000 (oldest deadline 2024-05-20T16:27:46.344106+0000) 2024-05-20T16:27:52.635 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:27:52 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:27:52.337+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-05-20T16:26:47.217074+0000 front 2024-05-20T16:26:47.216883+0000 (oldest deadline 2024-05-20T16:27:10.716827+0000) 2024-05-20T16:27:52.636 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:27:52 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:27:52.337+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:54.817899+0000 front 2024-05-20T16:26:54.818008+0000 (oldest deadline 2024-05-20T16:27:20.018619+0000) 2024-05-20T16:27:52.636 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:27:52 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:27:52.337+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-20T16:27:01.719536+0000 front 2024-05-20T16:27:01.719675+0000 (oldest deadline 2024-05-20T16:27:27.019603+0000) 2024-05-20T16:27:52.636 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:27:52 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:27:52.337+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-20T16:27:19.722847+0000 front 2024-05-20T16:27:19.722862+0000 (oldest deadline 2024-05-20T16:27:45.022529+0000) 2024-05-20T16:27:52.636 INFO:journalctl@ceph.osd.8.smithi184.stdout:May 20 16:27:52 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-8[55074]: 2024-05-20T16:27:52.546+0000 7f4a3856b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:52.470105+0000 front 2024-05-20T16:26:52.470039+0000 (oldest deadline 2024-05-20T16:27:18.270525+0000) 2024-05-20T16:27:52.636 INFO:journalctl@ceph.osd.8.smithi184.stdout:May 20 16:27:52 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-8[55074]: 2024-05-20T16:27:52.546+0000 7f4a3856b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-20T16:27:05.271959+0000 front 2024-05-20T16:27:05.272026+0000 (oldest deadline 2024-05-20T16:27:28.772153+0000) 2024-05-20T16:27:52.636 INFO:journalctl@ceph.osd.8.smithi184.stdout:May 20 16:27:52 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-8[55074]: 2024-05-20T16:27:52.546+0000 7f4a3856b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-05-20T16:27:13.473547+0000 front 2024-05-20T16:27:13.473521+0000 (oldest deadline 2024-05-20T16:27:38.773369+0000) 2024-05-20T16:27:52.636 INFO:journalctl@ceph.osd.8.smithi184.stdout:May 20 16:27:52 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-8[55074]: 2024-05-20T16:27:52.546+0000 7f4a3856b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-20T16:27:22.174923+0000 front 2024-05-20T16:27:22.174955+0000 (oldest deadline 2024-05-20T16:27:45.074988+0000) 2024-05-20T16:27:52.859 INFO:journalctl@ceph.osd.9.smithi082.stdout:May 20 16:27:52 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-9[66777]: 2024-05-20T16:27:52.413+0000 7fa187fc1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-05-20T16:26:44.585296+0000 front 2024-05-20T16:26:44.585359+0000 (oldest deadline 2024-05-20T16:27:09.885493+0000) 2024-05-20T16:27:52.859 INFO:journalctl@ceph.osd.9.smithi082.stdout:May 20 16:27:52 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-9[66777]: 2024-05-20T16:27:52.413+0000 7fa187fc1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:55.786620+0000 front 2024-05-20T16:26:55.786679+0000 (oldest deadline 2024-05-20T16:27:19.286485+0000) 2024-05-20T16:27:52.859 INFO:journalctl@ceph.osd.9.smithi082.stdout:May 20 16:27:52 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-9[66777]: 2024-05-20T16:27:52.413+0000 7fa187fc1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-20T16:27:03.987755+0000 front 2024-05-20T16:27:03.987552+0000 (oldest deadline 2024-05-20T16:27:28.687562+0000) 2024-05-20T16:27:52.859 INFO:journalctl@ceph.osd.9.smithi082.stdout:May 20 16:27:52 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-9[66777]: 2024-05-20T16:27:52.413+0000 7fa187fc1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-05-20T16:27:13.388520+0000 front 2024-05-20T16:27:13.388549+0000 (oldest deadline 2024-05-20T16:27:38.688716+0000) 2024-05-20T16:27:52.859 INFO:journalctl@ceph.osd.9.smithi082.stdout:May 20 16:27:52 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-9[66777]: 2024-05-20T16:27:52.413+0000 7fa187fc1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-20T16:27:19.189504+0000 front 2024-05-20T16:27:19.189629+0000 (oldest deadline 2024-05-20T16:27:44.489497+0000) 2024-05-20T16:27:53.312 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:27:52 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:27:52.861+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-05-20T16:26:47.437555+0000 front 2024-05-20T16:26:47.437468+0000 (oldest deadline 2024-05-20T16:27:15.038423+0000) 2024-05-20T16:27:53.312 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:27:52 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:27:52.861+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:55.038908+0000 front 2024-05-20T16:26:55.038825+0000 (oldest deadline 2024-05-20T16:27:18.439444+0000) 2024-05-20T16:27:53.312 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:27:52 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:27:52.861+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-20T16:27:04.241075+0000 front 2024-05-20T16:27:04.241297+0000 (oldest deadline 2024-05-20T16:27:27.141055+0000) 2024-05-20T16:27:53.312 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:27:52 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:27:52.861+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-05-20T16:27:12.943012+0000 front 2024-05-20T16:27:12.942981+0000 (oldest deadline 2024-05-20T16:27:36.442783+0000) 2024-05-20T16:27:53.312 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:27:52 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:27:52.861+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-20T16:27:21.643969+0000 front 2024-05-20T16:27:21.644003+0000 (oldest deadline 2024-05-20T16:27:46.344106+0000) 2024-05-20T16:27:53.635 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:27:53 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:27:53.349+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-05-20T16:26:47.217074+0000 front 2024-05-20T16:26:47.216883+0000 (oldest deadline 2024-05-20T16:27:10.716827+0000) 2024-05-20T16:27:53.635 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:27:53 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:27:53.349+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:54.817899+0000 front 2024-05-20T16:26:54.818008+0000 (oldest deadline 2024-05-20T16:27:20.018619+0000) 2024-05-20T16:27:53.635 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:27:53 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:27:53.349+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-20T16:27:01.719536+0000 front 2024-05-20T16:27:01.719675+0000 (oldest deadline 2024-05-20T16:27:27.019603+0000) 2024-05-20T16:27:53.635 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:27:53 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:27:53.349+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-20T16:27:19.722847+0000 front 2024-05-20T16:27:19.722862+0000 (oldest deadline 2024-05-20T16:27:45.022529+0000) 2024-05-20T16:27:53.636 INFO:journalctl@ceph.osd.8.smithi184.stdout:May 20 16:27:53 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-8[55074]: 2024-05-20T16:27:53.503+0000 7f4a3856b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:52.470105+0000 front 2024-05-20T16:26:52.470039+0000 (oldest deadline 2024-05-20T16:27:18.270525+0000) 2024-05-20T16:27:53.636 INFO:journalctl@ceph.osd.8.smithi184.stdout:May 20 16:27:53 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-8[55074]: 2024-05-20T16:27:53.503+0000 7f4a3856b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-20T16:27:05.271959+0000 front 2024-05-20T16:27:05.272026+0000 (oldest deadline 2024-05-20T16:27:28.772153+0000) 2024-05-20T16:27:53.636 INFO:journalctl@ceph.osd.8.smithi184.stdout:May 20 16:27:53 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-8[55074]: 2024-05-20T16:27:53.503+0000 7f4a3856b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-05-20T16:27:13.473547+0000 front 2024-05-20T16:27:13.473521+0000 (oldest deadline 2024-05-20T16:27:38.773369+0000) 2024-05-20T16:27:53.636 INFO:journalctl@ceph.osd.8.smithi184.stdout:May 20 16:27:53 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-8[55074]: 2024-05-20T16:27:53.503+0000 7f4a3856b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-20T16:27:22.174923+0000 front 2024-05-20T16:27:22.174955+0000 (oldest deadline 2024-05-20T16:27:45.074988+0000) 2024-05-20T16:27:53.859 INFO:journalctl@ceph.osd.9.smithi082.stdout:May 20 16:27:53 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-9[66777]: 2024-05-20T16:27:53.379+0000 7fa187fc1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-05-20T16:26:44.585296+0000 front 2024-05-20T16:26:44.585359+0000 (oldest deadline 2024-05-20T16:27:09.885493+0000) 2024-05-20T16:27:53.859 INFO:journalctl@ceph.osd.9.smithi082.stdout:May 20 16:27:53 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-9[66777]: 2024-05-20T16:27:53.379+0000 7fa187fc1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:55.786620+0000 front 2024-05-20T16:26:55.786679+0000 (oldest deadline 2024-05-20T16:27:19.286485+0000) 2024-05-20T16:27:53.859 INFO:journalctl@ceph.osd.9.smithi082.stdout:May 20 16:27:53 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-9[66777]: 2024-05-20T16:27:53.379+0000 7fa187fc1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-20T16:27:03.987755+0000 front 2024-05-20T16:27:03.987552+0000 (oldest deadline 2024-05-20T16:27:28.687562+0000) 2024-05-20T16:27:53.859 INFO:journalctl@ceph.osd.9.smithi082.stdout:May 20 16:27:53 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-9[66777]: 2024-05-20T16:27:53.379+0000 7fa187fc1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-05-20T16:27:13.388520+0000 front 2024-05-20T16:27:13.388549+0000 (oldest deadline 2024-05-20T16:27:38.688716+0000) 2024-05-20T16:27:53.859 INFO:journalctl@ceph.osd.9.smithi082.stdout:May 20 16:27:53 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-9[66777]: 2024-05-20T16:27:53.379+0000 7fa187fc1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-20T16:27:19.189504+0000 front 2024-05-20T16:27:19.189629+0000 (oldest deadline 2024-05-20T16:27:44.489497+0000) 2024-05-20T16:27:54.312 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:27:53 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:27:53.815+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-05-20T16:26:47.437555+0000 front 2024-05-20T16:26:47.437468+0000 (oldest deadline 2024-05-20T16:27:15.038423+0000) 2024-05-20T16:27:54.312 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:27:53 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:27:53.815+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:55.038908+0000 front 2024-05-20T16:26:55.038825+0000 (oldest deadline 2024-05-20T16:27:18.439444+0000) 2024-05-20T16:27:54.312 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:27:53 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:27:53.815+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-20T16:27:04.241075+0000 front 2024-05-20T16:27:04.241297+0000 (oldest deadline 2024-05-20T16:27:27.141055+0000) 2024-05-20T16:27:54.312 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:27:53 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:27:53.815+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-05-20T16:27:12.943012+0000 front 2024-05-20T16:27:12.942981+0000 (oldest deadline 2024-05-20T16:27:36.442783+0000) 2024-05-20T16:27:54.312 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:27:53 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:27:53.815+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-20T16:27:21.643969+0000 front 2024-05-20T16:27:21.644003+0000 (oldest deadline 2024-05-20T16:27:46.344106+0000) 2024-05-20T16:27:54.635 INFO:journalctl@ceph.osd.8.smithi184.stdout:May 20 16:27:54 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-8[55074]: 2024-05-20T16:27:54.460+0000 7f4a3856b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:52.470105+0000 front 2024-05-20T16:26:52.470039+0000 (oldest deadline 2024-05-20T16:27:18.270525+0000) 2024-05-20T16:27:54.635 INFO:journalctl@ceph.osd.8.smithi184.stdout:May 20 16:27:54 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-8[55074]: 2024-05-20T16:27:54.460+0000 7f4a3856b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-20T16:27:05.271959+0000 front 2024-05-20T16:27:05.272026+0000 (oldest deadline 2024-05-20T16:27:28.772153+0000) 2024-05-20T16:27:54.635 INFO:journalctl@ceph.osd.8.smithi184.stdout:May 20 16:27:54 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-8[55074]: 2024-05-20T16:27:54.460+0000 7f4a3856b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-05-20T16:27:13.473547+0000 front 2024-05-20T16:27:13.473521+0000 (oldest deadline 2024-05-20T16:27:38.773369+0000) 2024-05-20T16:27:54.636 INFO:journalctl@ceph.osd.8.smithi184.stdout:May 20 16:27:54 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-8[55074]: 2024-05-20T16:27:54.460+0000 7f4a3856b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-20T16:27:22.174923+0000 front 2024-05-20T16:27:22.174955+0000 (oldest deadline 2024-05-20T16:27:45.074988+0000) 2024-05-20T16:27:54.636 INFO:journalctl@ceph.osd.8.smithi184.stdout:May 20 16:27:54 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-8[55074]: 2024-05-20T16:27:54.460+0000 7f4a3856b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-05-20T16:27:28.575980+0000 front 2024-05-20T16:27:28.575917+0000 (oldest deadline 2024-05-20T16:27:53.876058+0000) 2024-05-20T16:27:54.636 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:27:54 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:27:54.355+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-05-20T16:26:47.217074+0000 front 2024-05-20T16:26:47.216883+0000 (oldest deadline 2024-05-20T16:27:10.716827+0000) 2024-05-20T16:27:54.636 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:27:54 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:27:54.355+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:54.817899+0000 front 2024-05-20T16:26:54.818008+0000 (oldest deadline 2024-05-20T16:27:20.018619+0000) 2024-05-20T16:27:54.636 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:27:54 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:27:54.355+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-20T16:27:01.719536+0000 front 2024-05-20T16:27:01.719675+0000 (oldest deadline 2024-05-20T16:27:27.019603+0000) 2024-05-20T16:27:54.636 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:27:54 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:27:54.355+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-20T16:27:19.722847+0000 front 2024-05-20T16:27:19.722862+0000 (oldest deadline 2024-05-20T16:27:45.022529+0000) 2024-05-20T16:27:54.636 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:27:54 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:27:54.355+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-05-20T16:27:27.323480+0000 front 2024-05-20T16:27:27.323448+0000 (oldest deadline 2024-05-20T16:27:53.723837+0000) 2024-05-20T16:27:54.859 INFO:journalctl@ceph.osd.9.smithi082.stdout:May 20 16:27:54 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-9[66777]: 2024-05-20T16:27:54.406+0000 7fa187fc1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-05-20T16:26:44.585296+0000 front 2024-05-20T16:26:44.585359+0000 (oldest deadline 2024-05-20T16:27:09.885493+0000) 2024-05-20T16:27:54.859 INFO:journalctl@ceph.osd.9.smithi082.stdout:May 20 16:27:54 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-9[66777]: 2024-05-20T16:27:54.406+0000 7fa187fc1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:55.786620+0000 front 2024-05-20T16:26:55.786679+0000 (oldest deadline 2024-05-20T16:27:19.286485+0000) 2024-05-20T16:27:54.859 INFO:journalctl@ceph.osd.9.smithi082.stdout:May 20 16:27:54 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-9[66777]: 2024-05-20T16:27:54.406+0000 7fa187fc1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-20T16:27:03.987755+0000 front 2024-05-20T16:27:03.987552+0000 (oldest deadline 2024-05-20T16:27:28.687562+0000) 2024-05-20T16:27:54.859 INFO:journalctl@ceph.osd.9.smithi082.stdout:May 20 16:27:54 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-9[66777]: 2024-05-20T16:27:54.406+0000 7fa187fc1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-05-20T16:27:13.388520+0000 front 2024-05-20T16:27:13.388549+0000 (oldest deadline 2024-05-20T16:27:38.688716+0000) 2024-05-20T16:27:54.859 INFO:journalctl@ceph.osd.9.smithi082.stdout:May 20 16:27:54 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-9[66777]: 2024-05-20T16:27:54.406+0000 7fa187fc1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-20T16:27:19.189504+0000 front 2024-05-20T16:27:19.189629+0000 (oldest deadline 2024-05-20T16:27:44.489497+0000) 2024-05-20T16:27:54.859 INFO:journalctl@ceph.osd.9.smithi082.stdout:May 20 16:27:54 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-9[66777]: 2024-05-20T16:27:54.406+0000 7fa187fc1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-05-20T16:27:26.790816+0000 front 2024-05-20T16:27:26.790873+0000 (oldest deadline 2024-05-20T16:27:53.791395+0000) 2024-05-20T16:27:55.062 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:27:54 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:27:54.808+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-05-20T16:26:47.437555+0000 front 2024-05-20T16:26:47.437468+0000 (oldest deadline 2024-05-20T16:27:15.038423+0000) 2024-05-20T16:27:55.062 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:27:54 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:27:54.809+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:55.038908+0000 front 2024-05-20T16:26:55.038825+0000 (oldest deadline 2024-05-20T16:27:18.439444+0000) 2024-05-20T16:27:55.062 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:27:54 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:27:54.809+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-20T16:27:04.241075+0000 front 2024-05-20T16:27:04.241297+0000 (oldest deadline 2024-05-20T16:27:27.141055+0000) 2024-05-20T16:27:55.062 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:27:54 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:27:54.809+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-05-20T16:27:12.943012+0000 front 2024-05-20T16:27:12.942981+0000 (oldest deadline 2024-05-20T16:27:36.442783+0000) 2024-05-20T16:27:55.062 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:27:54 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:27:54.809+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-20T16:27:21.643969+0000 front 2024-05-20T16:27:21.644003+0000 (oldest deadline 2024-05-20T16:27:46.344106+0000) 2024-05-20T16:27:55.635 INFO:journalctl@ceph.osd.8.smithi184.stdout:May 20 16:27:55 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-8[55074]: 2024-05-20T16:27:55.444+0000 7f4a3856b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:52.470105+0000 front 2024-05-20T16:26:52.470039+0000 (oldest deadline 2024-05-20T16:27:18.270525+0000) 2024-05-20T16:27:55.636 INFO:journalctl@ceph.osd.8.smithi184.stdout:May 20 16:27:55 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-8[55074]: 2024-05-20T16:27:55.444+0000 7f4a3856b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-20T16:27:05.271959+0000 front 2024-05-20T16:27:05.272026+0000 (oldest deadline 2024-05-20T16:27:28.772153+0000) 2024-05-20T16:27:55.636 INFO:journalctl@ceph.osd.8.smithi184.stdout:May 20 16:27:55 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-8[55074]: 2024-05-20T16:27:55.444+0000 7f4a3856b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-05-20T16:27:13.473547+0000 front 2024-05-20T16:27:13.473521+0000 (oldest deadline 2024-05-20T16:27:38.773369+0000) 2024-05-20T16:27:55.636 INFO:journalctl@ceph.osd.8.smithi184.stdout:May 20 16:27:55 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-8[55074]: 2024-05-20T16:27:55.444+0000 7f4a3856b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-20T16:27:22.174923+0000 front 2024-05-20T16:27:22.174955+0000 (oldest deadline 2024-05-20T16:27:45.074988+0000) 2024-05-20T16:27:55.636 INFO:journalctl@ceph.osd.8.smithi184.stdout:May 20 16:27:55 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-8[55074]: 2024-05-20T16:27:55.444+0000 7f4a3856b700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-05-20T16:27:28.575980+0000 front 2024-05-20T16:27:28.575917+0000 (oldest deadline 2024-05-20T16:27:53.876058+0000) 2024-05-20T16:27:55.636 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:27:55 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:27:55.364+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-05-20T16:26:47.217074+0000 front 2024-05-20T16:26:47.216883+0000 (oldest deadline 2024-05-20T16:27:10.716827+0000) 2024-05-20T16:27:55.636 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:27:55 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:27:55.364+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:54.817899+0000 front 2024-05-20T16:26:54.818008+0000 (oldest deadline 2024-05-20T16:27:20.018619+0000) 2024-05-20T16:27:55.636 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:27:55 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:27:55.364+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-20T16:27:01.719536+0000 front 2024-05-20T16:27:01.719675+0000 (oldest deadline 2024-05-20T16:27:27.019603+0000) 2024-05-20T16:27:55.636 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:27:55 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:27:55.364+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-20T16:27:19.722847+0000 front 2024-05-20T16:27:19.722862+0000 (oldest deadline 2024-05-20T16:27:45.022529+0000) 2024-05-20T16:27:55.636 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:27:55 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:27:55.364+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-05-20T16:27:27.323480+0000 front 2024-05-20T16:27:27.323448+0000 (oldest deadline 2024-05-20T16:27:53.723837+0000) 2024-05-20T16:27:55.859 INFO:journalctl@ceph.osd.9.smithi082.stdout:May 20 16:27:55 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-9[66777]: 2024-05-20T16:27:55.419+0000 7fa187fc1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-05-20T16:26:44.585296+0000 front 2024-05-20T16:26:44.585359+0000 (oldest deadline 2024-05-20T16:27:09.885493+0000) 2024-05-20T16:27:55.859 INFO:journalctl@ceph.osd.9.smithi082.stdout:May 20 16:27:55 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-9[66777]: 2024-05-20T16:27:55.419+0000 7fa187fc1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:55.786620+0000 front 2024-05-20T16:26:55.786679+0000 (oldest deadline 2024-05-20T16:27:19.286485+0000) 2024-05-20T16:27:55.859 INFO:journalctl@ceph.osd.9.smithi082.stdout:May 20 16:27:55 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-9[66777]: 2024-05-20T16:27:55.419+0000 7fa187fc1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-20T16:27:03.987755+0000 front 2024-05-20T16:27:03.987552+0000 (oldest deadline 2024-05-20T16:27:28.687562+0000) 2024-05-20T16:27:55.859 INFO:journalctl@ceph.osd.9.smithi082.stdout:May 20 16:27:55 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-9[66777]: 2024-05-20T16:27:55.419+0000 7fa187fc1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-05-20T16:27:13.388520+0000 front 2024-05-20T16:27:13.388549+0000 (oldest deadline 2024-05-20T16:27:38.688716+0000) 2024-05-20T16:27:55.859 INFO:journalctl@ceph.osd.9.smithi082.stdout:May 20 16:27:55 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-9[66777]: 2024-05-20T16:27:55.419+0000 7fa187fc1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-20T16:27:19.189504+0000 front 2024-05-20T16:27:19.189629+0000 (oldest deadline 2024-05-20T16:27:44.489497+0000) 2024-05-20T16:27:55.859 INFO:journalctl@ceph.osd.9.smithi082.stdout:May 20 16:27:55 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-9[66777]: 2024-05-20T16:27:55.419+0000 7fa187fc1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-05-20T16:27:26.790816+0000 front 2024-05-20T16:27:26.790873+0000 (oldest deadline 2024-05-20T16:27:53.791395+0000) 2024-05-20T16:27:56.312 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:27:55 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:27:55.817+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-05-20T16:26:47.437555+0000 front 2024-05-20T16:26:47.437468+0000 (oldest deadline 2024-05-20T16:27:15.038423+0000) 2024-05-20T16:27:56.312 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:27:55 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:27:55.817+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:55.038908+0000 front 2024-05-20T16:26:55.038825+0000 (oldest deadline 2024-05-20T16:27:18.439444+0000) 2024-05-20T16:27:56.312 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:27:55 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:27:55.817+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-20T16:27:04.241075+0000 front 2024-05-20T16:27:04.241297+0000 (oldest deadline 2024-05-20T16:27:27.141055+0000) 2024-05-20T16:27:56.312 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:27:55 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:27:55.817+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-05-20T16:27:12.943012+0000 front 2024-05-20T16:27:12.942981+0000 (oldest deadline 2024-05-20T16:27:36.442783+0000) 2024-05-20T16:27:56.312 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:27:55 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:27:55.817+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-20T16:27:21.643969+0000 front 2024-05-20T16:27:21.644003+0000 (oldest deadline 2024-05-20T16:27:46.344106+0000) 2024-05-20T16:27:56.312 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:27:55 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:27:55.817+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-05-20T16:27:30.846286+0000 front 2024-05-20T16:27:30.846116+0000 (oldest deadline 2024-05-20T16:27:55.546056+0000) 2024-05-20T16:27:56.635 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:27:56 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:27:56.379+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-05-20T16:26:47.217074+0000 front 2024-05-20T16:26:47.216883+0000 (oldest deadline 2024-05-20T16:27:10.716827+0000) 2024-05-20T16:27:56.636 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:27:56 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:27:56.379+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:54.817899+0000 front 2024-05-20T16:26:54.818008+0000 (oldest deadline 2024-05-20T16:27:20.018619+0000) 2024-05-20T16:27:56.636 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:27:56 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:27:56.379+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-20T16:27:01.719536+0000 front 2024-05-20T16:27:01.719675+0000 (oldest deadline 2024-05-20T16:27:27.019603+0000) 2024-05-20T16:27:56.636 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:27:56 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:27:56.379+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-20T16:27:19.722847+0000 front 2024-05-20T16:27:19.722862+0000 (oldest deadline 2024-05-20T16:27:45.022529+0000) 2024-05-20T16:27:56.636 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:27:56 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:27:56.379+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-05-20T16:27:27.323480+0000 front 2024-05-20T16:27:27.323448+0000 (oldest deadline 2024-05-20T16:27:53.723837+0000) 2024-05-20T16:27:56.859 INFO:journalctl@ceph.osd.9.smithi082.stdout:May 20 16:27:56 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-9[66777]: 2024-05-20T16:27:56.412+0000 7fa187fc1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-05-20T16:26:44.585296+0000 front 2024-05-20T16:26:44.585359+0000 (oldest deadline 2024-05-20T16:27:09.885493+0000) 2024-05-20T16:27:56.859 INFO:journalctl@ceph.osd.9.smithi082.stdout:May 20 16:27:56 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-9[66777]: 2024-05-20T16:27:56.412+0000 7fa187fc1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:55.786620+0000 front 2024-05-20T16:26:55.786679+0000 (oldest deadline 2024-05-20T16:27:19.286485+0000) 2024-05-20T16:27:56.859 INFO:journalctl@ceph.osd.9.smithi082.stdout:May 20 16:27:56 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-9[66777]: 2024-05-20T16:27:56.412+0000 7fa187fc1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-20T16:27:03.987755+0000 front 2024-05-20T16:27:03.987552+0000 (oldest deadline 2024-05-20T16:27:28.687562+0000) 2024-05-20T16:27:56.859 INFO:journalctl@ceph.osd.9.smithi082.stdout:May 20 16:27:56 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-9[66777]: 2024-05-20T16:27:56.412+0000 7fa187fc1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-05-20T16:27:13.388520+0000 front 2024-05-20T16:27:13.388549+0000 (oldest deadline 2024-05-20T16:27:38.688716+0000) 2024-05-20T16:27:56.859 INFO:journalctl@ceph.osd.9.smithi082.stdout:May 20 16:27:56 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-9[66777]: 2024-05-20T16:27:56.412+0000 7fa187fc1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-20T16:27:19.189504+0000 front 2024-05-20T16:27:19.189629+0000 (oldest deadline 2024-05-20T16:27:44.489497+0000) 2024-05-20T16:27:56.859 INFO:journalctl@ceph.osd.9.smithi082.stdout:May 20 16:27:56 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-9[66777]: 2024-05-20T16:27:56.412+0000 7fa187fc1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-05-20T16:27:26.790816+0000 front 2024-05-20T16:27:26.790873+0000 (oldest deadline 2024-05-20T16:27:53.791395+0000) 2024-05-20T16:27:57.103 INFO:journalctl@ceph.osd.8.smithi184.stdout:May 20 16:27:56 smithi184 podman[73417]: 2024-05-20 16:27:56.748194929 +0000 UTC m=+5.939066958 container died 4b0db37e2c628eb67ac83dd9257814aa159583cc874c5c4e574e48ec2672517f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-8, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240514, org.label-schema.license=GPLv2, ceph=True) 2024-05-20T16:27:57.312 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:27:56 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:27:56.861+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-05-20T16:26:47.437555+0000 front 2024-05-20T16:26:47.437468+0000 (oldest deadline 2024-05-20T16:27:15.038423+0000) 2024-05-20T16:27:57.312 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:27:56 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:27:56.861+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:55.038908+0000 front 2024-05-20T16:26:55.038825+0000 (oldest deadline 2024-05-20T16:27:18.439444+0000) 2024-05-20T16:27:57.312 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:27:56 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:27:56.861+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-20T16:27:04.241075+0000 front 2024-05-20T16:27:04.241297+0000 (oldest deadline 2024-05-20T16:27:27.141055+0000) 2024-05-20T16:27:57.312 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:27:56 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:27:56.861+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-05-20T16:27:12.943012+0000 front 2024-05-20T16:27:12.942981+0000 (oldest deadline 2024-05-20T16:27:36.442783+0000) 2024-05-20T16:27:57.312 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:27:56 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:27:56.861+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-20T16:27:21.643969+0000 front 2024-05-20T16:27:21.644003+0000 (oldest deadline 2024-05-20T16:27:46.344106+0000) 2024-05-20T16:27:57.312 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:27:56 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:27:56.861+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-05-20T16:27:30.846286+0000 front 2024-05-20T16:27:30.846116+0000 (oldest deadline 2024-05-20T16:27:55.546056+0000) 2024-05-20T16:27:57.385 INFO:journalctl@ceph.osd.8.smithi184.stdout:May 20 16:27:57 smithi184 podman[73417]: 2024-05-20 16:27:57.104782192 +0000 UTC m=+6.295654223 container cleanup 4b0db37e2c628eb67ac83dd9257814aa159583cc874c5c4e574e48ec2672517f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-8, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240514, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c) 2024-05-20T16:27:57.385 INFO:journalctl@ceph.osd.8.smithi184.stdout:May 20 16:27:57 smithi184 bash[73417]: ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-8 2024-05-20T16:27:57.385 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:27:57 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:27:57.345+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-05-20T16:26:47.217074+0000 front 2024-05-20T16:26:47.216883+0000 (oldest deadline 2024-05-20T16:27:10.716827+0000) 2024-05-20T16:27:57.386 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:27:57 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:27:57.345+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:54.817899+0000 front 2024-05-20T16:26:54.818008+0000 (oldest deadline 2024-05-20T16:27:20.018619+0000) 2024-05-20T16:27:57.386 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:27:57 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:27:57.345+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-20T16:27:01.719536+0000 front 2024-05-20T16:27:01.719675+0000 (oldest deadline 2024-05-20T16:27:27.019603+0000) 2024-05-20T16:27:57.386 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:27:57 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:27:57.345+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-20T16:27:19.722847+0000 front 2024-05-20T16:27:19.722862+0000 (oldest deadline 2024-05-20T16:27:45.022529+0000) 2024-05-20T16:27:57.386 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:27:57 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:27:57.345+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-05-20T16:27:27.323480+0000 front 2024-05-20T16:27:27.323448+0000 (oldest deadline 2024-05-20T16:27:53.723837+0000) 2024-05-20T16:27:57.771 INFO:journalctl@ceph.osd.8.smithi184.stdout:May 20 16:27:57 smithi184 podman[73432]: 2024-05-20 16:27:57.414258867 +0000 UTC m=+0.667282305 container remove 4b0db37e2c628eb67ac83dd9257814aa159583cc874c5c4e574e48ec2672517f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-8, org.label-schema.build-date=20240514, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, io.buildah.version=1.33.5) 2024-05-20T16:27:57.859 INFO:journalctl@ceph.osd.9.smithi082.stdout:May 20 16:27:57 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-9[66777]: 2024-05-20T16:27:57.388+0000 7fa187fc1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-05-20T16:26:44.585296+0000 front 2024-05-20T16:26:44.585359+0000 (oldest deadline 2024-05-20T16:27:09.885493+0000) 2024-05-20T16:27:57.859 INFO:journalctl@ceph.osd.9.smithi082.stdout:May 20 16:27:57 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-9[66777]: 2024-05-20T16:27:57.388+0000 7fa187fc1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:55.786620+0000 front 2024-05-20T16:26:55.786679+0000 (oldest deadline 2024-05-20T16:27:19.286485+0000) 2024-05-20T16:27:57.859 INFO:journalctl@ceph.osd.9.smithi082.stdout:May 20 16:27:57 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-9[66777]: 2024-05-20T16:27:57.388+0000 7fa187fc1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-20T16:27:03.987755+0000 front 2024-05-20T16:27:03.987552+0000 (oldest deadline 2024-05-20T16:27:28.687562+0000) 2024-05-20T16:27:57.859 INFO:journalctl@ceph.osd.9.smithi082.stdout:May 20 16:27:57 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-9[66777]: 2024-05-20T16:27:57.388+0000 7fa187fc1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-05-20T16:27:13.388520+0000 front 2024-05-20T16:27:13.388549+0000 (oldest deadline 2024-05-20T16:27:38.688716+0000) 2024-05-20T16:27:57.859 INFO:journalctl@ceph.osd.9.smithi082.stdout:May 20 16:27:57 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-9[66777]: 2024-05-20T16:27:57.388+0000 7fa187fc1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-20T16:27:19.189504+0000 front 2024-05-20T16:27:19.189629+0000 (oldest deadline 2024-05-20T16:27:44.489497+0000) 2024-05-20T16:27:57.859 INFO:journalctl@ceph.osd.9.smithi082.stdout:May 20 16:27:57 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-9[66777]: 2024-05-20T16:27:57.388+0000 7fa187fc1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-05-20T16:27:26.790816+0000 front 2024-05-20T16:27:26.790873+0000 (oldest deadline 2024-05-20T16:27:53.791395+0000) 2024-05-20T16:27:58.033 INFO:journalctl@ceph.osd.8.smithi184.stdout:May 20 16:27:57 smithi184 podman[73523]: 2024-05-20 16:27:57.675442398 +0000 UTC m=+0.017725477 image pull cfcc61b2d476f611479310eb042aa05f11dd5f60dbb65e3b0d7e0e11d024cdb3 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708 2024-05-20T16:27:58.033 INFO:journalctl@ceph.osd.8.smithi184.stdout:May 20 16:27:57 smithi184 podman[73523]: 2024-05-20 16:27:57.790371031 +0000 UTC m=+0.132654105 container create 7dd4a54ccd08e3a5189794e079a3a9a3066d6b30de0a09f1ce36d6b9d7061cd3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-8-deactivate, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240514, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-05-20T16:27:58.033 INFO:journalctl@ceph.osd.8.smithi184.stdout:May 20 16:27:58 smithi184 podman[73523]: 2024-05-20 16:27:58.034511652 +0000 UTC m=+0.376794728 container init 7dd4a54ccd08e3a5189794e079a3a9a3066d6b30de0a09f1ce36d6b9d7061cd3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-8-deactivate, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240514, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-05-20T16:27:58.312 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:27:57 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:27:57.855+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-05-20T16:26:47.437555+0000 front 2024-05-20T16:26:47.437468+0000 (oldest deadline 2024-05-20T16:27:15.038423+0000) 2024-05-20T16:27:58.312 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:27:57 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:27:57.855+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:55.038908+0000 front 2024-05-20T16:26:55.038825+0000 (oldest deadline 2024-05-20T16:27:18.439444+0000) 2024-05-20T16:27:58.312 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:27:57 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:27:57.855+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-20T16:27:04.241075+0000 front 2024-05-20T16:27:04.241297+0000 (oldest deadline 2024-05-20T16:27:27.141055+0000) 2024-05-20T16:27:58.312 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:27:57 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:27:57.855+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-05-20T16:27:12.943012+0000 front 2024-05-20T16:27:12.942981+0000 (oldest deadline 2024-05-20T16:27:36.442783+0000) 2024-05-20T16:27:58.312 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:27:57 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:27:57.855+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-20T16:27:21.643969+0000 front 2024-05-20T16:27:21.644003+0000 (oldest deadline 2024-05-20T16:27:46.344106+0000) 2024-05-20T16:27:58.312 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:27:57 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:27:57.855+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-05-20T16:27:30.846286+0000 front 2024-05-20T16:27:30.846116+0000 (oldest deadline 2024-05-20T16:27:55.546056+0000) 2024-05-20T16:27:58.327 INFO:journalctl@ceph.osd.8.smithi184.stdout:May 20 16:27:58 smithi184 podman[73523]: 2024-05-20 16:27:58.037953493 +0000 UTC m=+0.380236574 container start 7dd4a54ccd08e3a5189794e079a3a9a3066d6b30de0a09f1ce36d6b9d7061cd3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-8-deactivate, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240514, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-05-20T16:27:58.327 INFO:journalctl@ceph.osd.8.smithi184.stdout:May 20 16:27:58 smithi184 podman[73523]: 2024-05-20 16:27:58.1683407 +0000 UTC m=+0.510623790 container attach 7dd4a54ccd08e3a5189794e079a3a9a3066d6b30de0a09f1ce36d6b9d7061cd3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-8-deactivate, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240514, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-05-20T16:27:58.607 INFO:journalctl@ceph.osd.8.smithi184.stdout:May 20 16:27:58 smithi184 podman[73523]: 2024-05-20 16:27:58.329053219 +0000 UTC m=+0.671336300 container died 7dd4a54ccd08e3a5189794e079a3a9a3066d6b30de0a09f1ce36d6b9d7061cd3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-8-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240514, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS) 2024-05-20T16:27:58.608 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:27:58 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:27:58.363+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-05-20T16:26:47.217074+0000 front 2024-05-20T16:26:47.216883+0000 (oldest deadline 2024-05-20T16:27:10.716827+0000) 2024-05-20T16:27:58.608 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:27:58 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:27:58.363+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:54.817899+0000 front 2024-05-20T16:26:54.818008+0000 (oldest deadline 2024-05-20T16:27:20.018619+0000) 2024-05-20T16:27:58.608 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:27:58 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:27:58.363+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-20T16:27:01.719536+0000 front 2024-05-20T16:27:01.719675+0000 (oldest deadline 2024-05-20T16:27:27.019603+0000) 2024-05-20T16:27:58.608 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:27:58 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:27:58.363+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-20T16:27:19.722847+0000 front 2024-05-20T16:27:19.722862+0000 (oldest deadline 2024-05-20T16:27:45.022529+0000) 2024-05-20T16:27:58.608 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:27:58 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:27:58.363+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-05-20T16:27:27.323480+0000 front 2024-05-20T16:27:27.323448+0000 (oldest deadline 2024-05-20T16:27:53.723837+0000) 2024-05-20T16:27:58.859 INFO:journalctl@ceph.osd.9.smithi082.stdout:May 20 16:27:58 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-9[66777]: 2024-05-20T16:27:58.405+0000 7fa187fc1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-05-20T16:26:44.585296+0000 front 2024-05-20T16:26:44.585359+0000 (oldest deadline 2024-05-20T16:27:09.885493+0000) 2024-05-20T16:27:58.859 INFO:journalctl@ceph.osd.9.smithi082.stdout:May 20 16:27:58 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-9[66777]: 2024-05-20T16:27:58.405+0000 7fa187fc1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:55.786620+0000 front 2024-05-20T16:26:55.786679+0000 (oldest deadline 2024-05-20T16:27:19.286485+0000) 2024-05-20T16:27:58.859 INFO:journalctl@ceph.osd.9.smithi082.stdout:May 20 16:27:58 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-9[66777]: 2024-05-20T16:27:58.405+0000 7fa187fc1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-20T16:27:03.987755+0000 front 2024-05-20T16:27:03.987552+0000 (oldest deadline 2024-05-20T16:27:28.687562+0000) 2024-05-20T16:27:58.859 INFO:journalctl@ceph.osd.9.smithi082.stdout:May 20 16:27:58 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-9[66777]: 2024-05-20T16:27:58.405+0000 7fa187fc1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-05-20T16:27:13.388520+0000 front 2024-05-20T16:27:13.388549+0000 (oldest deadline 2024-05-20T16:27:38.688716+0000) 2024-05-20T16:27:58.859 INFO:journalctl@ceph.osd.9.smithi082.stdout:May 20 16:27:58 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-9[66777]: 2024-05-20T16:27:58.405+0000 7fa187fc1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-20T16:27:19.189504+0000 front 2024-05-20T16:27:19.189629+0000 (oldest deadline 2024-05-20T16:27:44.489497+0000) 2024-05-20T16:27:58.859 INFO:journalctl@ceph.osd.9.smithi082.stdout:May 20 16:27:58 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-9[66777]: 2024-05-20T16:27:58.405+0000 7fa187fc1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-05-20T16:27:26.790816+0000 front 2024-05-20T16:27:26.790873+0000 (oldest deadline 2024-05-20T16:27:53.791395+0000) 2024-05-20T16:27:58.944 DEBUG:teuthology.orchestra.run.smithi184:> sudo pkill -f 'journalctl -f -n 0 -u ceph-59d1600a-16c2-11ef-bc9a-c7b262605968@osd.8.service' 2024-05-20T16:27:58.981 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-20T16:27:58.982 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-05-20T16:27:58.982 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-05-20T16:27:58.982 DEBUG:teuthology.orchestra.run.smithi082:> sudo systemctl stop ceph-59d1600a-16c2-11ef-bc9a-c7b262605968@osd.9 2024-05-20T16:27:59.279 INFO:journalctl@ceph.osd.9.smithi082.stdout:May 20 16:27:59 smithi082 systemd[1]: Stopping Ceph osd.9 for 59d1600a-16c2-11ef-bc9a-c7b262605968... 2024-05-20T16:27:59.312 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:27:58 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:27:58.839+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-05-20T16:26:47.437555+0000 front 2024-05-20T16:26:47.437468+0000 (oldest deadline 2024-05-20T16:27:15.038423+0000) 2024-05-20T16:27:59.312 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:27:58 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:27:58.839+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:55.038908+0000 front 2024-05-20T16:26:55.038825+0000 (oldest deadline 2024-05-20T16:27:18.439444+0000) 2024-05-20T16:27:59.312 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:27:58 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:27:58.839+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-20T16:27:04.241075+0000 front 2024-05-20T16:27:04.241297+0000 (oldest deadline 2024-05-20T16:27:27.141055+0000) 2024-05-20T16:27:59.312 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:27:58 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:27:58.839+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-05-20T16:27:12.943012+0000 front 2024-05-20T16:27:12.942981+0000 (oldest deadline 2024-05-20T16:27:36.442783+0000) 2024-05-20T16:27:59.312 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:27:58 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:27:58.839+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-20T16:27:21.643969+0000 front 2024-05-20T16:27:21.644003+0000 (oldest deadline 2024-05-20T16:27:46.344106+0000) 2024-05-20T16:27:59.312 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:27:58 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:27:58.839+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-05-20T16:27:30.846286+0000 front 2024-05-20T16:27:30.846116+0000 (oldest deadline 2024-05-20T16:27:55.546056+0000) 2024-05-20T16:27:59.609 INFO:journalctl@ceph.osd.9.smithi082.stdout:May 20 16:27:59 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-9[66777]: 2024-05-20T16:27:59.279+0000 7fa18ba0b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-20T16:27:59.609 INFO:journalctl@ceph.osd.9.smithi082.stdout:May 20 16:27:59 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-9[66777]: 2024-05-20T16:27:59.279+0000 7fa18ba0b700 -1 osd.9 102 *** Got signal Terminated *** 2024-05-20T16:27:59.609 INFO:journalctl@ceph.osd.9.smithi082.stdout:May 20 16:27:59 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-9[66777]: 2024-05-20T16:27:59.279+0000 7fa18ba0b700 -1 osd.9 102 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-20T16:27:59.609 INFO:journalctl@ceph.osd.9.smithi082.stdout:May 20 16:27:59 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-9[66777]: 2024-05-20T16:27:59.433+0000 7fa187fc1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-05-20T16:26:44.585296+0000 front 2024-05-20T16:26:44.585359+0000 (oldest deadline 2024-05-20T16:27:09.885493+0000) 2024-05-20T16:27:59.609 INFO:journalctl@ceph.osd.9.smithi082.stdout:May 20 16:27:59 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-9[66777]: 2024-05-20T16:27:59.433+0000 7fa187fc1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:55.786620+0000 front 2024-05-20T16:26:55.786679+0000 (oldest deadline 2024-05-20T16:27:19.286485+0000) 2024-05-20T16:27:59.609 INFO:journalctl@ceph.osd.9.smithi082.stdout:May 20 16:27:59 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-9[66777]: 2024-05-20T16:27:59.433+0000 7fa187fc1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-20T16:27:03.987755+0000 front 2024-05-20T16:27:03.987552+0000 (oldest deadline 2024-05-20T16:27:28.687562+0000) 2024-05-20T16:27:59.609 INFO:journalctl@ceph.osd.9.smithi082.stdout:May 20 16:27:59 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-9[66777]: 2024-05-20T16:27:59.433+0000 7fa187fc1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-05-20T16:27:13.388520+0000 front 2024-05-20T16:27:13.388549+0000 (oldest deadline 2024-05-20T16:27:38.688716+0000) 2024-05-20T16:27:59.609 INFO:journalctl@ceph.osd.9.smithi082.stdout:May 20 16:27:59 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-9[66777]: 2024-05-20T16:27:59.433+0000 7fa187fc1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-20T16:27:19.189504+0000 front 2024-05-20T16:27:19.189629+0000 (oldest deadline 2024-05-20T16:27:44.489497+0000) 2024-05-20T16:27:59.610 INFO:journalctl@ceph.osd.9.smithi082.stdout:May 20 16:27:59 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-9[66777]: 2024-05-20T16:27:59.433+0000 7fa187fc1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-05-20T16:27:26.790816+0000 front 2024-05-20T16:27:26.790873+0000 (oldest deadline 2024-05-20T16:27:53.791395+0000) 2024-05-20T16:27:59.635 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:27:59 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:27:59.328+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-05-20T16:26:47.217074+0000 front 2024-05-20T16:26:47.216883+0000 (oldest deadline 2024-05-20T16:27:10.716827+0000) 2024-05-20T16:27:59.636 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:27:59 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:27:59.329+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:54.817899+0000 front 2024-05-20T16:26:54.818008+0000 (oldest deadline 2024-05-20T16:27:20.018619+0000) 2024-05-20T16:27:59.636 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:27:59 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:27:59.329+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-20T16:27:01.719536+0000 front 2024-05-20T16:27:01.719675+0000 (oldest deadline 2024-05-20T16:27:27.019603+0000) 2024-05-20T16:27:59.636 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:27:59 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:27:59.329+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-20T16:27:19.722847+0000 front 2024-05-20T16:27:19.722862+0000 (oldest deadline 2024-05-20T16:27:45.022529+0000) 2024-05-20T16:27:59.636 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:27:59 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:27:59.329+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-05-20T16:27:27.323480+0000 front 2024-05-20T16:27:27.323448+0000 (oldest deadline 2024-05-20T16:27:53.723837+0000) 2024-05-20T16:28:00.312 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:27:59 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:27:59.878+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-05-20T16:26:47.437555+0000 front 2024-05-20T16:26:47.437468+0000 (oldest deadline 2024-05-20T16:27:15.038423+0000) 2024-05-20T16:28:00.312 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:27:59 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:27:59.878+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:55.038908+0000 front 2024-05-20T16:26:55.038825+0000 (oldest deadline 2024-05-20T16:27:18.439444+0000) 2024-05-20T16:28:00.312 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:27:59 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:27:59.878+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-20T16:27:04.241075+0000 front 2024-05-20T16:27:04.241297+0000 (oldest deadline 2024-05-20T16:27:27.141055+0000) 2024-05-20T16:28:00.312 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:27:59 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:27:59.878+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-05-20T16:27:12.943012+0000 front 2024-05-20T16:27:12.942981+0000 (oldest deadline 2024-05-20T16:27:36.442783+0000) 2024-05-20T16:28:00.312 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:27:59 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:27:59.878+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-20T16:27:21.643969+0000 front 2024-05-20T16:27:21.644003+0000 (oldest deadline 2024-05-20T16:27:46.344106+0000) 2024-05-20T16:28:00.312 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:27:59 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:27:59.878+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-05-20T16:27:30.846286+0000 front 2024-05-20T16:27:30.846116+0000 (oldest deadline 2024-05-20T16:27:55.546056+0000) 2024-05-20T16:28:00.609 INFO:journalctl@ceph.osd.9.smithi082.stdout:May 20 16:28:00 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-9[66777]: 2024-05-20T16:28:00.429+0000 7fa187fc1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-05-20T16:26:44.585296+0000 front 2024-05-20T16:26:44.585359+0000 (oldest deadline 2024-05-20T16:27:09.885493+0000) 2024-05-20T16:28:00.609 INFO:journalctl@ceph.osd.9.smithi082.stdout:May 20 16:28:00 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-9[66777]: 2024-05-20T16:28:00.429+0000 7fa187fc1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:55.786620+0000 front 2024-05-20T16:26:55.786679+0000 (oldest deadline 2024-05-20T16:27:19.286485+0000) 2024-05-20T16:28:00.609 INFO:journalctl@ceph.osd.9.smithi082.stdout:May 20 16:28:00 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-9[66777]: 2024-05-20T16:28:00.429+0000 7fa187fc1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-20T16:27:03.987755+0000 front 2024-05-20T16:27:03.987552+0000 (oldest deadline 2024-05-20T16:27:28.687562+0000) 2024-05-20T16:28:00.609 INFO:journalctl@ceph.osd.9.smithi082.stdout:May 20 16:28:00 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-9[66777]: 2024-05-20T16:28:00.429+0000 7fa187fc1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-05-20T16:27:13.388520+0000 front 2024-05-20T16:27:13.388549+0000 (oldest deadline 2024-05-20T16:27:38.688716+0000) 2024-05-20T16:28:00.609 INFO:journalctl@ceph.osd.9.smithi082.stdout:May 20 16:28:00 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-9[66777]: 2024-05-20T16:28:00.429+0000 7fa187fc1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-20T16:27:19.189504+0000 front 2024-05-20T16:27:19.189629+0000 (oldest deadline 2024-05-20T16:27:44.489497+0000) 2024-05-20T16:28:00.609 INFO:journalctl@ceph.osd.9.smithi082.stdout:May 20 16:28:00 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-9[66777]: 2024-05-20T16:28:00.429+0000 7fa187fc1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-05-20T16:27:26.790816+0000 front 2024-05-20T16:27:26.790873+0000 (oldest deadline 2024-05-20T16:27:53.791395+0000) 2024-05-20T16:28:00.635 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:28:00 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:28:00.348+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-05-20T16:26:47.217074+0000 front 2024-05-20T16:26:47.216883+0000 (oldest deadline 2024-05-20T16:27:10.716827+0000) 2024-05-20T16:28:00.635 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:28:00 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:28:00.348+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:54.817899+0000 front 2024-05-20T16:26:54.818008+0000 (oldest deadline 2024-05-20T16:27:20.018619+0000) 2024-05-20T16:28:00.636 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:28:00 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:28:00.348+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-20T16:27:01.719536+0000 front 2024-05-20T16:27:01.719675+0000 (oldest deadline 2024-05-20T16:27:27.019603+0000) 2024-05-20T16:28:00.636 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:28:00 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:28:00.348+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-20T16:27:19.722847+0000 front 2024-05-20T16:27:19.722862+0000 (oldest deadline 2024-05-20T16:27:45.022529+0000) 2024-05-20T16:28:00.636 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:28:00 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:28:00.348+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-05-20T16:27:27.323480+0000 front 2024-05-20T16:27:27.323448+0000 (oldest deadline 2024-05-20T16:27:53.723837+0000) 2024-05-20T16:28:01.062 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:28:00 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:28:00.870+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-05-20T16:26:47.437555+0000 front 2024-05-20T16:26:47.437468+0000 (oldest deadline 2024-05-20T16:27:15.038423+0000) 2024-05-20T16:28:01.062 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:28:00 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:28:00.870+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:55.038908+0000 front 2024-05-20T16:26:55.038825+0000 (oldest deadline 2024-05-20T16:27:18.439444+0000) 2024-05-20T16:28:01.062 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:28:00 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:28:00.870+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-20T16:27:04.241075+0000 front 2024-05-20T16:27:04.241297+0000 (oldest deadline 2024-05-20T16:27:27.141055+0000) 2024-05-20T16:28:01.062 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:28:00 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:28:00.870+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-05-20T16:27:12.943012+0000 front 2024-05-20T16:27:12.942981+0000 (oldest deadline 2024-05-20T16:27:36.442783+0000) 2024-05-20T16:28:01.062 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:28:00 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:28:00.870+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-20T16:27:21.643969+0000 front 2024-05-20T16:27:21.644003+0000 (oldest deadline 2024-05-20T16:27:46.344106+0000) 2024-05-20T16:28:01.062 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:28:00 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:28:00.870+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-05-20T16:27:30.846286+0000 front 2024-05-20T16:27:30.846116+0000 (oldest deadline 2024-05-20T16:27:55.546056+0000) 2024-05-20T16:28:01.062 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:28:00 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:28:00.870+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.82:6822 osd.6 since back 2024-05-20T16:27:35.546850+0000 front 2024-05-20T16:27:35.546827+0000 (oldest deadline 2024-05-20T16:28:00.246640+0000) 2024-05-20T16:28:01.635 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:28:01 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:28:01.358+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-05-20T16:26:47.217074+0000 front 2024-05-20T16:26:47.216883+0000 (oldest deadline 2024-05-20T16:27:10.716827+0000) 2024-05-20T16:28:01.636 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:28:01 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:28:01.358+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:54.817899+0000 front 2024-05-20T16:26:54.818008+0000 (oldest deadline 2024-05-20T16:27:20.018619+0000) 2024-05-20T16:28:01.636 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:28:01 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:28:01.358+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-20T16:27:01.719536+0000 front 2024-05-20T16:27:01.719675+0000 (oldest deadline 2024-05-20T16:27:27.019603+0000) 2024-05-20T16:28:01.636 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:28:01 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:28:01.358+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-20T16:27:19.722847+0000 front 2024-05-20T16:27:19.722862+0000 (oldest deadline 2024-05-20T16:27:45.022529+0000) 2024-05-20T16:28:01.636 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:28:01 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:28:01.358+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-05-20T16:27:27.323480+0000 front 2024-05-20T16:27:27.323448+0000 (oldest deadline 2024-05-20T16:27:53.723837+0000) 2024-05-20T16:28:01.859 INFO:journalctl@ceph.osd.9.smithi082.stdout:May 20 16:28:01 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-9[66777]: 2024-05-20T16:28:01.465+0000 7fa187fc1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-05-20T16:26:44.585296+0000 front 2024-05-20T16:26:44.585359+0000 (oldest deadline 2024-05-20T16:27:09.885493+0000) 2024-05-20T16:28:01.859 INFO:journalctl@ceph.osd.9.smithi082.stdout:May 20 16:28:01 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-9[66777]: 2024-05-20T16:28:01.465+0000 7fa187fc1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:55.786620+0000 front 2024-05-20T16:26:55.786679+0000 (oldest deadline 2024-05-20T16:27:19.286485+0000) 2024-05-20T16:28:01.859 INFO:journalctl@ceph.osd.9.smithi082.stdout:May 20 16:28:01 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-9[66777]: 2024-05-20T16:28:01.465+0000 7fa187fc1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-20T16:27:03.987755+0000 front 2024-05-20T16:27:03.987552+0000 (oldest deadline 2024-05-20T16:27:28.687562+0000) 2024-05-20T16:28:01.859 INFO:journalctl@ceph.osd.9.smithi082.stdout:May 20 16:28:01 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-9[66777]: 2024-05-20T16:28:01.465+0000 7fa187fc1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-05-20T16:27:13.388520+0000 front 2024-05-20T16:27:13.388549+0000 (oldest deadline 2024-05-20T16:27:38.688716+0000) 2024-05-20T16:28:01.859 INFO:journalctl@ceph.osd.9.smithi082.stdout:May 20 16:28:01 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-9[66777]: 2024-05-20T16:28:01.465+0000 7fa187fc1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-20T16:27:19.189504+0000 front 2024-05-20T16:27:19.189629+0000 (oldest deadline 2024-05-20T16:27:44.489497+0000) 2024-05-20T16:28:01.859 INFO:journalctl@ceph.osd.9.smithi082.stdout:May 20 16:28:01 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-9[66777]: 2024-05-20T16:28:01.465+0000 7fa187fc1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-05-20T16:27:26.790816+0000 front 2024-05-20T16:27:26.790873+0000 (oldest deadline 2024-05-20T16:27:53.791395+0000) 2024-05-20T16:28:02.312 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:28:01 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:28:01.873+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-05-20T16:26:47.437555+0000 front 2024-05-20T16:26:47.437468+0000 (oldest deadline 2024-05-20T16:27:15.038423+0000) 2024-05-20T16:28:02.312 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:28:01 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:28:01.873+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:55.038908+0000 front 2024-05-20T16:26:55.038825+0000 (oldest deadline 2024-05-20T16:27:18.439444+0000) 2024-05-20T16:28:02.312 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:28:01 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:28:01.873+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-20T16:27:04.241075+0000 front 2024-05-20T16:27:04.241297+0000 (oldest deadline 2024-05-20T16:27:27.141055+0000) 2024-05-20T16:28:02.312 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:28:01 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:28:01.873+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-05-20T16:27:12.943012+0000 front 2024-05-20T16:27:12.942981+0000 (oldest deadline 2024-05-20T16:27:36.442783+0000) 2024-05-20T16:28:02.312 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:28:01 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:28:01.873+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-20T16:27:21.643969+0000 front 2024-05-20T16:27:21.644003+0000 (oldest deadline 2024-05-20T16:27:46.344106+0000) 2024-05-20T16:28:02.312 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:28:01 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:28:01.873+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-05-20T16:27:30.846286+0000 front 2024-05-20T16:27:30.846116+0000 (oldest deadline 2024-05-20T16:27:55.546056+0000) 2024-05-20T16:28:02.312 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:28:01 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:28:01.873+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.82:6822 osd.6 since back 2024-05-20T16:27:35.546850+0000 front 2024-05-20T16:27:35.546827+0000 (oldest deadline 2024-05-20T16:28:00.246640+0000) 2024-05-20T16:28:02.635 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:28:02 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:28:02.346+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-05-20T16:26:47.217074+0000 front 2024-05-20T16:26:47.216883+0000 (oldest deadline 2024-05-20T16:27:10.716827+0000) 2024-05-20T16:28:02.635 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:28:02 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:28:02.346+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:54.817899+0000 front 2024-05-20T16:26:54.818008+0000 (oldest deadline 2024-05-20T16:27:20.018619+0000) 2024-05-20T16:28:02.636 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:28:02 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:28:02.346+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-20T16:27:01.719536+0000 front 2024-05-20T16:27:01.719675+0000 (oldest deadline 2024-05-20T16:27:27.019603+0000) 2024-05-20T16:28:02.636 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:28:02 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:28:02.346+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-20T16:27:19.722847+0000 front 2024-05-20T16:27:19.722862+0000 (oldest deadline 2024-05-20T16:27:45.022529+0000) 2024-05-20T16:28:02.636 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:28:02 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:28:02.346+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-05-20T16:27:27.323480+0000 front 2024-05-20T16:27:27.323448+0000 (oldest deadline 2024-05-20T16:27:53.723837+0000) 2024-05-20T16:28:02.636 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:28:02 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:28:02.346+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.82:6822 osd.6 since back 2024-05-20T16:27:38.726211+0000 front 2024-05-20T16:27:38.726240+0000 (oldest deadline 2024-05-20T16:28:01.626037+0000) 2024-05-20T16:28:02.859 INFO:journalctl@ceph.osd.9.smithi082.stdout:May 20 16:28:02 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-9[66777]: 2024-05-20T16:28:02.512+0000 7fa187fc1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-05-20T16:26:44.585296+0000 front 2024-05-20T16:26:44.585359+0000 (oldest deadline 2024-05-20T16:27:09.885493+0000) 2024-05-20T16:28:02.859 INFO:journalctl@ceph.osd.9.smithi082.stdout:May 20 16:28:02 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-9[66777]: 2024-05-20T16:28:02.512+0000 7fa187fc1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:55.786620+0000 front 2024-05-20T16:26:55.786679+0000 (oldest deadline 2024-05-20T16:27:19.286485+0000) 2024-05-20T16:28:02.859 INFO:journalctl@ceph.osd.9.smithi082.stdout:May 20 16:28:02 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-9[66777]: 2024-05-20T16:28:02.512+0000 7fa187fc1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-20T16:27:03.987755+0000 front 2024-05-20T16:27:03.987552+0000 (oldest deadline 2024-05-20T16:27:28.687562+0000) 2024-05-20T16:28:02.859 INFO:journalctl@ceph.osd.9.smithi082.stdout:May 20 16:28:02 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-9[66777]: 2024-05-20T16:28:02.512+0000 7fa187fc1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-05-20T16:27:13.388520+0000 front 2024-05-20T16:27:13.388549+0000 (oldest deadline 2024-05-20T16:27:38.688716+0000) 2024-05-20T16:28:02.859 INFO:journalctl@ceph.osd.9.smithi082.stdout:May 20 16:28:02 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-9[66777]: 2024-05-20T16:28:02.512+0000 7fa187fc1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-20T16:27:19.189504+0000 front 2024-05-20T16:27:19.189629+0000 (oldest deadline 2024-05-20T16:27:44.489497+0000) 2024-05-20T16:28:02.859 INFO:journalctl@ceph.osd.9.smithi082.stdout:May 20 16:28:02 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-9[66777]: 2024-05-20T16:28:02.512+0000 7fa187fc1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-05-20T16:27:26.790816+0000 front 2024-05-20T16:27:26.790873+0000 (oldest deadline 2024-05-20T16:27:53.791395+0000) 2024-05-20T16:28:03.312 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:28:02 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:28:02.826+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-05-20T16:26:47.437555+0000 front 2024-05-20T16:26:47.437468+0000 (oldest deadline 2024-05-20T16:27:15.038423+0000) 2024-05-20T16:28:03.312 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:28:02 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:28:02.826+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:55.038908+0000 front 2024-05-20T16:26:55.038825+0000 (oldest deadline 2024-05-20T16:27:18.439444+0000) 2024-05-20T16:28:03.312 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:28:02 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:28:02.826+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-20T16:27:04.241075+0000 front 2024-05-20T16:27:04.241297+0000 (oldest deadline 2024-05-20T16:27:27.141055+0000) 2024-05-20T16:28:03.312 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:28:02 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:28:02.826+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-05-20T16:27:12.943012+0000 front 2024-05-20T16:27:12.942981+0000 (oldest deadline 2024-05-20T16:27:36.442783+0000) 2024-05-20T16:28:03.312 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:28:02 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:28:02.826+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-20T16:27:21.643969+0000 front 2024-05-20T16:27:21.644003+0000 (oldest deadline 2024-05-20T16:27:46.344106+0000) 2024-05-20T16:28:03.312 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:28:02 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:28:02.826+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-05-20T16:27:30.846286+0000 front 2024-05-20T16:27:30.846116+0000 (oldest deadline 2024-05-20T16:27:55.546056+0000) 2024-05-20T16:28:03.313 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:28:02 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:28:02.826+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.82:6822 osd.6 since back 2024-05-20T16:27:35.546850+0000 front 2024-05-20T16:27:35.546827+0000 (oldest deadline 2024-05-20T16:28:00.246640+0000) 2024-05-20T16:28:03.635 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:28:03 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:28:03.329+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-05-20T16:26:47.217074+0000 front 2024-05-20T16:26:47.216883+0000 (oldest deadline 2024-05-20T16:27:10.716827+0000) 2024-05-20T16:28:03.636 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:28:03 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:28:03.329+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:54.817899+0000 front 2024-05-20T16:26:54.818008+0000 (oldest deadline 2024-05-20T16:27:20.018619+0000) 2024-05-20T16:28:03.636 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:28:03 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:28:03.329+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-20T16:27:01.719536+0000 front 2024-05-20T16:27:01.719675+0000 (oldest deadline 2024-05-20T16:27:27.019603+0000) 2024-05-20T16:28:03.636 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:28:03 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:28:03.329+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-20T16:27:19.722847+0000 front 2024-05-20T16:27:19.722862+0000 (oldest deadline 2024-05-20T16:27:45.022529+0000) 2024-05-20T16:28:03.636 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:28:03 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:28:03.329+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-05-20T16:27:27.323480+0000 front 2024-05-20T16:27:27.323448+0000 (oldest deadline 2024-05-20T16:27:53.723837+0000) 2024-05-20T16:28:03.636 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:28:03 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:28:03.329+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.82:6822 osd.6 since back 2024-05-20T16:27:38.726211+0000 front 2024-05-20T16:27:38.726240+0000 (oldest deadline 2024-05-20T16:28:01.626037+0000) 2024-05-20T16:28:03.859 INFO:journalctl@ceph.osd.9.smithi082.stdout:May 20 16:28:03 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-9[66777]: 2024-05-20T16:28:03.512+0000 7fa187fc1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-05-20T16:26:44.585296+0000 front 2024-05-20T16:26:44.585359+0000 (oldest deadline 2024-05-20T16:27:09.885493+0000) 2024-05-20T16:28:03.859 INFO:journalctl@ceph.osd.9.smithi082.stdout:May 20 16:28:03 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-9[66777]: 2024-05-20T16:28:03.512+0000 7fa187fc1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:55.786620+0000 front 2024-05-20T16:26:55.786679+0000 (oldest deadline 2024-05-20T16:27:19.286485+0000) 2024-05-20T16:28:03.859 INFO:journalctl@ceph.osd.9.smithi082.stdout:May 20 16:28:03 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-9[66777]: 2024-05-20T16:28:03.512+0000 7fa187fc1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-20T16:27:03.987755+0000 front 2024-05-20T16:27:03.987552+0000 (oldest deadline 2024-05-20T16:27:28.687562+0000) 2024-05-20T16:28:03.859 INFO:journalctl@ceph.osd.9.smithi082.stdout:May 20 16:28:03 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-9[66777]: 2024-05-20T16:28:03.512+0000 7fa187fc1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-05-20T16:27:13.388520+0000 front 2024-05-20T16:27:13.388549+0000 (oldest deadline 2024-05-20T16:27:38.688716+0000) 2024-05-20T16:28:03.859 INFO:journalctl@ceph.osd.9.smithi082.stdout:May 20 16:28:03 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-9[66777]: 2024-05-20T16:28:03.512+0000 7fa187fc1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-20T16:27:19.189504+0000 front 2024-05-20T16:27:19.189629+0000 (oldest deadline 2024-05-20T16:27:44.489497+0000) 2024-05-20T16:28:03.859 INFO:journalctl@ceph.osd.9.smithi082.stdout:May 20 16:28:03 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-9[66777]: 2024-05-20T16:28:03.512+0000 7fa187fc1700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-05-20T16:27:26.790816+0000 front 2024-05-20T16:27:26.790873+0000 (oldest deadline 2024-05-20T16:27:53.791395+0000) 2024-05-20T16:28:04.312 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:28:03 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:28:03.845+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-05-20T16:26:47.437555+0000 front 2024-05-20T16:26:47.437468+0000 (oldest deadline 2024-05-20T16:27:15.038423+0000) 2024-05-20T16:28:04.312 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:28:03 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:28:03.845+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:55.038908+0000 front 2024-05-20T16:26:55.038825+0000 (oldest deadline 2024-05-20T16:27:18.439444+0000) 2024-05-20T16:28:04.312 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:28:03 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:28:03.845+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-20T16:27:04.241075+0000 front 2024-05-20T16:27:04.241297+0000 (oldest deadline 2024-05-20T16:27:27.141055+0000) 2024-05-20T16:28:04.312 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:28:03 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:28:03.845+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-05-20T16:27:12.943012+0000 front 2024-05-20T16:27:12.942981+0000 (oldest deadline 2024-05-20T16:27:36.442783+0000) 2024-05-20T16:28:04.312 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:28:03 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:28:03.845+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-20T16:27:21.643969+0000 front 2024-05-20T16:27:21.644003+0000 (oldest deadline 2024-05-20T16:27:46.344106+0000) 2024-05-20T16:28:04.312 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:28:03 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:28:03.845+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-05-20T16:27:30.846286+0000 front 2024-05-20T16:27:30.846116+0000 (oldest deadline 2024-05-20T16:27:55.546056+0000) 2024-05-20T16:28:04.312 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:28:03 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:28:03.845+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.82:6822 osd.6 since back 2024-05-20T16:27:35.546850+0000 front 2024-05-20T16:27:35.546827+0000 (oldest deadline 2024-05-20T16:28:00.246640+0000) 2024-05-20T16:28:04.635 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:28:04 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:28:04.287+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-05-20T16:26:47.217074+0000 front 2024-05-20T16:26:47.216883+0000 (oldest deadline 2024-05-20T16:27:10.716827+0000) 2024-05-20T16:28:04.636 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:28:04 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:28:04.287+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:54.817899+0000 front 2024-05-20T16:26:54.818008+0000 (oldest deadline 2024-05-20T16:27:20.018619+0000) 2024-05-20T16:28:04.636 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:28:04 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:28:04.287+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-20T16:27:01.719536+0000 front 2024-05-20T16:27:01.719675+0000 (oldest deadline 2024-05-20T16:27:27.019603+0000) 2024-05-20T16:28:04.636 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:28:04 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:28:04.287+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-20T16:27:19.722847+0000 front 2024-05-20T16:27:19.722862+0000 (oldest deadline 2024-05-20T16:27:45.022529+0000) 2024-05-20T16:28:04.636 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:28:04 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:28:04.287+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-05-20T16:27:27.323480+0000 front 2024-05-20T16:27:27.323448+0000 (oldest deadline 2024-05-20T16:27:53.723837+0000) 2024-05-20T16:28:04.636 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:28:04 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:28:04.287+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.82:6822 osd.6 since back 2024-05-20T16:27:38.726211+0000 front 2024-05-20T16:27:38.726240+0000 (oldest deadline 2024-05-20T16:28:01.626037+0000) 2024-05-20T16:28:05.062 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:28:04 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:28:04.810+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-05-20T16:26:47.437555+0000 front 2024-05-20T16:26:47.437468+0000 (oldest deadline 2024-05-20T16:27:15.038423+0000) 2024-05-20T16:28:05.062 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:28:04 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:28:04.810+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:55.038908+0000 front 2024-05-20T16:26:55.038825+0000 (oldest deadline 2024-05-20T16:27:18.439444+0000) 2024-05-20T16:28:05.062 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:28:04 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:28:04.810+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-20T16:27:04.241075+0000 front 2024-05-20T16:27:04.241297+0000 (oldest deadline 2024-05-20T16:27:27.141055+0000) 2024-05-20T16:28:05.062 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:28:04 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:28:04.810+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-05-20T16:27:12.943012+0000 front 2024-05-20T16:27:12.942981+0000 (oldest deadline 2024-05-20T16:27:36.442783+0000) 2024-05-20T16:28:05.062 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:28:04 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:28:04.810+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-20T16:27:21.643969+0000 front 2024-05-20T16:27:21.644003+0000 (oldest deadline 2024-05-20T16:27:46.344106+0000) 2024-05-20T16:28:05.062 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:28:04 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:28:04.810+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-05-20T16:27:30.846286+0000 front 2024-05-20T16:27:30.846116+0000 (oldest deadline 2024-05-20T16:27:55.546056+0000) 2024-05-20T16:28:05.062 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:28:04 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:28:04.810+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.82:6822 osd.6 since back 2024-05-20T16:27:35.546850+0000 front 2024-05-20T16:27:35.546827+0000 (oldest deadline 2024-05-20T16:28:00.246640+0000) 2024-05-20T16:28:05.379 INFO:journalctl@ceph.osd.9.smithi082.stdout:May 20 16:28:05 smithi082 podman[92412]: 2024-05-20 16:28:05.128953013 +0000 UTC m=+5.946864341 container died 1e12de28d19e6ed3eb34fd49e54d2bbfcd48d8f3e281609a13f0baff2e3ec29d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-9, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240514, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux ) 2024-05-20T16:28:05.379 INFO:journalctl@ceph.osd.9.smithi082.stdout:May 20 16:28:05 smithi082 podman[92412]: 2024-05-20 16:28:05.378799763 +0000 UTC m=+6.196711085 container cleanup 1e12de28d19e6ed3eb34fd49e54d2bbfcd48d8f3e281609a13f0baff2e3ec29d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-9, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, org.label-schema.build-date=20240514) 2024-05-20T16:28:05.379 INFO:journalctl@ceph.osd.9.smithi082.stdout:May 20 16:28:05 smithi082 bash[92412]: ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-9 2024-05-20T16:28:05.635 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:28:05 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:28:05.244+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-05-20T16:26:47.217074+0000 front 2024-05-20T16:26:47.216883+0000 (oldest deadline 2024-05-20T16:27:10.716827+0000) 2024-05-20T16:28:05.636 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:28:05 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:28:05.244+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:54.817899+0000 front 2024-05-20T16:26:54.818008+0000 (oldest deadline 2024-05-20T16:27:20.018619+0000) 2024-05-20T16:28:05.636 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:28:05 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:28:05.244+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-20T16:27:01.719536+0000 front 2024-05-20T16:27:01.719675+0000 (oldest deadline 2024-05-20T16:27:27.019603+0000) 2024-05-20T16:28:05.636 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:28:05 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:28:05.244+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-20T16:27:19.722847+0000 front 2024-05-20T16:27:19.722862+0000 (oldest deadline 2024-05-20T16:27:45.022529+0000) 2024-05-20T16:28:05.636 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:28:05 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:28:05.244+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-05-20T16:27:27.323480+0000 front 2024-05-20T16:27:27.323448+0000 (oldest deadline 2024-05-20T16:27:53.723837+0000) 2024-05-20T16:28:05.636 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:28:05 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:28:05.244+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.82:6822 osd.6 since back 2024-05-20T16:27:38.726211+0000 front 2024-05-20T16:27:38.726240+0000 (oldest deadline 2024-05-20T16:28:01.626037+0000) 2024-05-20T16:28:05.940 INFO:journalctl@ceph.osd.9.smithi082.stdout:May 20 16:28:05 smithi082 podman[92429]: 2024-05-20 16:28:05.621843035 +0000 UTC m=+0.495829298 container remove 1e12de28d19e6ed3eb34fd49e54d2bbfcd48d8f3e281609a13f0baff2e3ec29d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-9, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, org.label-schema.license=GPLv2, org.label-schema.build-date=20240514, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-20T16:28:06.232 INFO:journalctl@ceph.osd.9.smithi082.stdout:May 20 16:28:05 smithi082 podman[92518]: 2024-05-20 16:28:05.843812216 +0000 UTC m=+0.017167067 image pull cfcc61b2d476f611479310eb042aa05f11dd5f60dbb65e3b0d7e0e11d024cdb3 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708 2024-05-20T16:28:06.232 INFO:journalctl@ceph.osd.9.smithi082.stdout:May 20 16:28:05 smithi082 podman[92518]: 2024-05-20 16:28:05.947516156 +0000 UTC m=+0.120870995 container create 0866e3a9b17d30fea55ccd02984a15b448d57d7b9c3bc0c6972b3b5af814c55f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-9-deactivate, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, org.label-schema.build-date=20240514, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-05-20T16:28:06.232 INFO:journalctl@ceph.osd.9.smithi082.stdout:May 20 16:28:06 smithi082 podman[92518]: 2024-05-20 16:28:06.158249487 +0000 UTC m=+0.331604325 container init 0866e3a9b17d30fea55ccd02984a15b448d57d7b9c3bc0c6972b3b5af814c55f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-9-deactivate, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240514) 2024-05-20T16:28:06.232 INFO:journalctl@ceph.osd.9.smithi082.stdout:May 20 16:28:06 smithi082 podman[92518]: 2024-05-20 16:28:06.16152539 +0000 UTC m=+0.334880236 container start 0866e3a9b17d30fea55ccd02984a15b448d57d7b9c3bc0c6972b3b5af814c55f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-9-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240514, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=) 2024-05-20T16:28:06.232 INFO:journalctl@ceph.osd.9.smithi082.stdout:May 20 16:28:06 smithi082 podman[92518]: 2024-05-20 16:28:06.233151788 +0000 UTC m=+0.406506628 container attach 0866e3a9b17d30fea55ccd02984a15b448d57d7b9c3bc0c6972b3b5af814c55f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-9-deactivate, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240514, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-05-20T16:28:06.312 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:28:05 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:28:05.852+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-05-20T16:26:47.437555+0000 front 2024-05-20T16:26:47.437468+0000 (oldest deadline 2024-05-20T16:27:15.038423+0000) 2024-05-20T16:28:06.312 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:28:05 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:28:05.852+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:55.038908+0000 front 2024-05-20T16:26:55.038825+0000 (oldest deadline 2024-05-20T16:27:18.439444+0000) 2024-05-20T16:28:06.312 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:28:05 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:28:05.852+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-20T16:27:04.241075+0000 front 2024-05-20T16:27:04.241297+0000 (oldest deadline 2024-05-20T16:27:27.141055+0000) 2024-05-20T16:28:06.312 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:28:05 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:28:05.852+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-05-20T16:27:12.943012+0000 front 2024-05-20T16:27:12.942981+0000 (oldest deadline 2024-05-20T16:27:36.442783+0000) 2024-05-20T16:28:06.312 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:28:05 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:28:05.852+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-20T16:27:21.643969+0000 front 2024-05-20T16:27:21.644003+0000 (oldest deadline 2024-05-20T16:27:46.344106+0000) 2024-05-20T16:28:06.312 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:28:05 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:28:05.852+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-05-20T16:27:30.846286+0000 front 2024-05-20T16:27:30.846116+0000 (oldest deadline 2024-05-20T16:27:55.546056+0000) 2024-05-20T16:28:06.312 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:28:05 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:28:05.852+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.82:6822 osd.6 since back 2024-05-20T16:27:35.546850+0000 front 2024-05-20T16:27:35.546827+0000 (oldest deadline 2024-05-20T16:28:00.246640+0000) 2024-05-20T16:28:06.609 INFO:journalctl@ceph.osd.9.smithi082.stdout:May 20 16:28:06 smithi082 podman[92518]: 2024-05-20 16:28:06.456943613 +0000 UTC m=+0.630298467 container died 0866e3a9b17d30fea55ccd02984a15b448d57d7b9c3bc0c6972b3b5af814c55f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-9-deactivate, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, org.label-schema.license=GPLv2, org.label-schema.build-date=20240514, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True) 2024-05-20T16:28:06.635 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:28:06 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:28:06.205+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-05-20T16:26:47.217074+0000 front 2024-05-20T16:26:47.216883+0000 (oldest deadline 2024-05-20T16:27:10.716827+0000) 2024-05-20T16:28:06.635 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:28:06 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:28:06.205+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:54.817899+0000 front 2024-05-20T16:26:54.818008+0000 (oldest deadline 2024-05-20T16:27:20.018619+0000) 2024-05-20T16:28:06.635 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:28:06 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:28:06.205+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-20T16:27:01.719536+0000 front 2024-05-20T16:27:01.719675+0000 (oldest deadline 2024-05-20T16:27:27.019603+0000) 2024-05-20T16:28:06.635 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:28:06 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:28:06.205+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-20T16:27:19.722847+0000 front 2024-05-20T16:27:19.722862+0000 (oldest deadline 2024-05-20T16:27:45.022529+0000) 2024-05-20T16:28:06.635 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:28:06 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:28:06.205+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-05-20T16:27:27.323480+0000 front 2024-05-20T16:27:27.323448+0000 (oldest deadline 2024-05-20T16:27:53.723837+0000) 2024-05-20T16:28:06.635 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:28:06 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:28:06.205+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.82:6822 osd.6 since back 2024-05-20T16:27:38.726211+0000 front 2024-05-20T16:27:38.726240+0000 (oldest deadline 2024-05-20T16:28:01.626037+0000) 2024-05-20T16:28:06.930 INFO:journalctl@ceph.osd.9.smithi082.stdout:May 20 16:28:06 smithi082 podman[92544]: 2024-05-20 16:28:06.914043336 +0000 UTC m=+0.451399002 container remove 0866e3a9b17d30fea55ccd02984a15b448d57d7b9c3bc0c6972b3b5af814c55f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-9-deactivate, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240514, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-20T16:28:06.935 DEBUG:teuthology.orchestra.run.smithi082:> sudo pkill -f 'journalctl -f -n 0 -u ceph-59d1600a-16c2-11ef-bc9a-c7b262605968@osd.9.service' 2024-05-20T16:28:06.973 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-20T16:28:06.974 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-05-20T16:28:06.974 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-05-20T16:28:06.974 DEBUG:teuthology.orchestra.run.smithi160:> sudo systemctl stop ceph-59d1600a-16c2-11ef-bc9a-c7b262605968@osd.10 2024-05-20T16:28:07.312 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:28:06 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:28:06.828+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-05-20T16:26:47.437555+0000 front 2024-05-20T16:26:47.437468+0000 (oldest deadline 2024-05-20T16:27:15.038423+0000) 2024-05-20T16:28:07.312 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:28:06 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:28:06.828+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:55.038908+0000 front 2024-05-20T16:26:55.038825+0000 (oldest deadline 2024-05-20T16:27:18.439444+0000) 2024-05-20T16:28:07.312 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:28:06 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:28:06.828+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-20T16:27:04.241075+0000 front 2024-05-20T16:27:04.241297+0000 (oldest deadline 2024-05-20T16:27:27.141055+0000) 2024-05-20T16:28:07.312 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:28:06 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:28:06.828+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-05-20T16:27:12.943012+0000 front 2024-05-20T16:27:12.942981+0000 (oldest deadline 2024-05-20T16:27:36.442783+0000) 2024-05-20T16:28:07.312 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:28:06 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:28:06.828+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-20T16:27:21.643969+0000 front 2024-05-20T16:27:21.644003+0000 (oldest deadline 2024-05-20T16:27:46.344106+0000) 2024-05-20T16:28:07.312 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:28:06 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:28:06.828+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-05-20T16:27:30.846286+0000 front 2024-05-20T16:27:30.846116+0000 (oldest deadline 2024-05-20T16:27:55.546056+0000) 2024-05-20T16:28:07.312 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:28:06 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:28:06.828+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.82:6822 osd.6 since back 2024-05-20T16:27:35.546850+0000 front 2024-05-20T16:27:35.546827+0000 (oldest deadline 2024-05-20T16:28:00.246640+0000) 2024-05-20T16:28:07.313 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:28:07 smithi160 systemd[1]: Stopping Ceph osd.10 for 59d1600a-16c2-11ef-bc9a-c7b262605968... 2024-05-20T16:28:07.635 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:28:07 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:28:07.245+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-05-20T16:26:47.217074+0000 front 2024-05-20T16:26:47.216883+0000 (oldest deadline 2024-05-20T16:27:10.716827+0000) 2024-05-20T16:28:07.635 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:28:07 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:28:07.245+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:54.817899+0000 front 2024-05-20T16:26:54.818008+0000 (oldest deadline 2024-05-20T16:27:20.018619+0000) 2024-05-20T16:28:07.635 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:28:07 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:28:07.245+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-20T16:27:01.719536+0000 front 2024-05-20T16:27:01.719675+0000 (oldest deadline 2024-05-20T16:27:27.019603+0000) 2024-05-20T16:28:07.635 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:28:07 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:28:07.245+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-20T16:27:19.722847+0000 front 2024-05-20T16:27:19.722862+0000 (oldest deadline 2024-05-20T16:27:45.022529+0000) 2024-05-20T16:28:07.636 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:28:07 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:28:07.245+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-05-20T16:27:27.323480+0000 front 2024-05-20T16:27:27.323448+0000 (oldest deadline 2024-05-20T16:27:53.723837+0000) 2024-05-20T16:28:07.636 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:28:07 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:28:07.245+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.82:6822 osd.6 since back 2024-05-20T16:27:38.726211+0000 front 2024-05-20T16:27:38.726240+0000 (oldest deadline 2024-05-20T16:28:01.626037+0000) 2024-05-20T16:28:07.812 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:28:07 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:28:07.317+0000 7f332f90a700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-20T16:28:07.812 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:28:07 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:28:07.317+0000 7f332f90a700 -1 osd.10 102 *** Got signal Terminated *** 2024-05-20T16:28:07.812 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:28:07 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:28:07.317+0000 7f332f90a700 -1 osd.10 102 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-20T16:28:08.312 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:28:07 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:28:07.848+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-05-20T16:26:47.437555+0000 front 2024-05-20T16:26:47.437468+0000 (oldest deadline 2024-05-20T16:27:15.038423+0000) 2024-05-20T16:28:08.312 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:28:07 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:28:07.848+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:55.038908+0000 front 2024-05-20T16:26:55.038825+0000 (oldest deadline 2024-05-20T16:27:18.439444+0000) 2024-05-20T16:28:08.312 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:28:07 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:28:07.848+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-20T16:27:04.241075+0000 front 2024-05-20T16:27:04.241297+0000 (oldest deadline 2024-05-20T16:27:27.141055+0000) 2024-05-20T16:28:08.312 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:28:07 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:28:07.848+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-05-20T16:27:12.943012+0000 front 2024-05-20T16:27:12.942981+0000 (oldest deadline 2024-05-20T16:27:36.442783+0000) 2024-05-20T16:28:08.312 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:28:07 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:28:07.848+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-20T16:27:21.643969+0000 front 2024-05-20T16:27:21.644003+0000 (oldest deadline 2024-05-20T16:27:46.344106+0000) 2024-05-20T16:28:08.313 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:28:07 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:28:07.848+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-05-20T16:27:30.846286+0000 front 2024-05-20T16:27:30.846116+0000 (oldest deadline 2024-05-20T16:27:55.546056+0000) 2024-05-20T16:28:08.313 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:28:07 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:28:07.848+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.82:6822 osd.6 since back 2024-05-20T16:27:35.546850+0000 front 2024-05-20T16:27:35.546827+0000 (oldest deadline 2024-05-20T16:28:00.246640+0000) 2024-05-20T16:28:08.635 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:28:08 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:28:08.244+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-05-20T16:26:47.217074+0000 front 2024-05-20T16:26:47.216883+0000 (oldest deadline 2024-05-20T16:27:10.716827+0000) 2024-05-20T16:28:08.636 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:28:08 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:28:08.244+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:54.817899+0000 front 2024-05-20T16:26:54.818008+0000 (oldest deadline 2024-05-20T16:27:20.018619+0000) 2024-05-20T16:28:08.636 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:28:08 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:28:08.244+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-20T16:27:01.719536+0000 front 2024-05-20T16:27:01.719675+0000 (oldest deadline 2024-05-20T16:27:27.019603+0000) 2024-05-20T16:28:08.636 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:28:08 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:28:08.244+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-20T16:27:19.722847+0000 front 2024-05-20T16:27:19.722862+0000 (oldest deadline 2024-05-20T16:27:45.022529+0000) 2024-05-20T16:28:08.636 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:28:08 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:28:08.244+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-05-20T16:27:27.323480+0000 front 2024-05-20T16:27:27.323448+0000 (oldest deadline 2024-05-20T16:27:53.723837+0000) 2024-05-20T16:28:08.636 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:28:08 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:28:08.244+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.82:6822 osd.6 since back 2024-05-20T16:27:38.726211+0000 front 2024-05-20T16:27:38.726240+0000 (oldest deadline 2024-05-20T16:28:01.626037+0000) 2024-05-20T16:28:09.312 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:28:08 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:28:08.814+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-05-20T16:26:47.437555+0000 front 2024-05-20T16:26:47.437468+0000 (oldest deadline 2024-05-20T16:27:15.038423+0000) 2024-05-20T16:28:09.312 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:28:08 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:28:08.814+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:55.038908+0000 front 2024-05-20T16:26:55.038825+0000 (oldest deadline 2024-05-20T16:27:18.439444+0000) 2024-05-20T16:28:09.312 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:28:08 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:28:08.814+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-20T16:27:04.241075+0000 front 2024-05-20T16:27:04.241297+0000 (oldest deadline 2024-05-20T16:27:27.141055+0000) 2024-05-20T16:28:09.312 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:28:08 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:28:08.814+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-05-20T16:27:12.943012+0000 front 2024-05-20T16:27:12.942981+0000 (oldest deadline 2024-05-20T16:27:36.442783+0000) 2024-05-20T16:28:09.312 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:28:08 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:28:08.814+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-20T16:27:21.643969+0000 front 2024-05-20T16:27:21.644003+0000 (oldest deadline 2024-05-20T16:27:46.344106+0000) 2024-05-20T16:28:09.312 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:28:08 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:28:08.814+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-05-20T16:27:30.846286+0000 front 2024-05-20T16:27:30.846116+0000 (oldest deadline 2024-05-20T16:27:55.546056+0000) 2024-05-20T16:28:09.312 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:28:08 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:28:08.814+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.82:6822 osd.6 since back 2024-05-20T16:27:35.546850+0000 front 2024-05-20T16:27:35.546827+0000 (oldest deadline 2024-05-20T16:28:00.246640+0000) 2024-05-20T16:28:09.386 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:28:09 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:28:09.218+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-05-20T16:26:47.217074+0000 front 2024-05-20T16:26:47.216883+0000 (oldest deadline 2024-05-20T16:27:10.716827+0000) 2024-05-20T16:28:09.386 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:28:09 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:28:09.218+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:54.817899+0000 front 2024-05-20T16:26:54.818008+0000 (oldest deadline 2024-05-20T16:27:20.018619+0000) 2024-05-20T16:28:09.386 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:28:09 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:28:09.218+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-20T16:27:01.719536+0000 front 2024-05-20T16:27:01.719675+0000 (oldest deadline 2024-05-20T16:27:27.019603+0000) 2024-05-20T16:28:09.386 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:28:09 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:28:09.218+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-20T16:27:19.722847+0000 front 2024-05-20T16:27:19.722862+0000 (oldest deadline 2024-05-20T16:27:45.022529+0000) 2024-05-20T16:28:09.386 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:28:09 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:28:09.218+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-05-20T16:27:27.323480+0000 front 2024-05-20T16:27:27.323448+0000 (oldest deadline 2024-05-20T16:27:53.723837+0000) 2024-05-20T16:28:09.386 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:28:09 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:28:09.218+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.82:6822 osd.6 since back 2024-05-20T16:27:38.726211+0000 front 2024-05-20T16:27:38.726240+0000 (oldest deadline 2024-05-20T16:28:01.626037+0000) 2024-05-20T16:28:10.312 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:28:09 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:28:09.852+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-05-20T16:26:47.437555+0000 front 2024-05-20T16:26:47.437468+0000 (oldest deadline 2024-05-20T16:27:15.038423+0000) 2024-05-20T16:28:10.312 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:28:09 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:28:09.852+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:55.038908+0000 front 2024-05-20T16:26:55.038825+0000 (oldest deadline 2024-05-20T16:27:18.439444+0000) 2024-05-20T16:28:10.312 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:28:09 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:28:09.852+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-20T16:27:04.241075+0000 front 2024-05-20T16:27:04.241297+0000 (oldest deadline 2024-05-20T16:27:27.141055+0000) 2024-05-20T16:28:10.312 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:28:09 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:28:09.852+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-05-20T16:27:12.943012+0000 front 2024-05-20T16:27:12.942981+0000 (oldest deadline 2024-05-20T16:27:36.442783+0000) 2024-05-20T16:28:10.312 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:28:09 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:28:09.852+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-20T16:27:21.643969+0000 front 2024-05-20T16:27:21.644003+0000 (oldest deadline 2024-05-20T16:27:46.344106+0000) 2024-05-20T16:28:10.312 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:28:09 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:28:09.852+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-05-20T16:27:30.846286+0000 front 2024-05-20T16:27:30.846116+0000 (oldest deadline 2024-05-20T16:27:55.546056+0000) 2024-05-20T16:28:10.312 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:28:09 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:28:09.852+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.82:6822 osd.6 since back 2024-05-20T16:27:35.546850+0000 front 2024-05-20T16:27:35.546827+0000 (oldest deadline 2024-05-20T16:28:00.246640+0000) 2024-05-20T16:28:10.636 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:28:10 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:28:10.178+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-05-20T16:26:47.217074+0000 front 2024-05-20T16:26:47.216883+0000 (oldest deadline 2024-05-20T16:27:10.716827+0000) 2024-05-20T16:28:10.636 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:28:10 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:28:10.178+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:54.817899+0000 front 2024-05-20T16:26:54.818008+0000 (oldest deadline 2024-05-20T16:27:20.018619+0000) 2024-05-20T16:28:10.636 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:28:10 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:28:10.178+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-20T16:27:01.719536+0000 front 2024-05-20T16:27:01.719675+0000 (oldest deadline 2024-05-20T16:27:27.019603+0000) 2024-05-20T16:28:10.636 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:28:10 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:28:10.178+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-20T16:27:19.722847+0000 front 2024-05-20T16:27:19.722862+0000 (oldest deadline 2024-05-20T16:27:45.022529+0000) 2024-05-20T16:28:10.636 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:28:10 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:28:10.178+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-05-20T16:27:27.323480+0000 front 2024-05-20T16:27:27.323448+0000 (oldest deadline 2024-05-20T16:27:53.723837+0000) 2024-05-20T16:28:10.636 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:28:10 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:28:10.178+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.82:6822 osd.6 since back 2024-05-20T16:27:38.726211+0000 front 2024-05-20T16:27:38.726240+0000 (oldest deadline 2024-05-20T16:28:01.626037+0000) 2024-05-20T16:28:10.636 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:28:10 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:28:10.178+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.160:6820 osd.7 since back 2024-05-20T16:27:44.527310+0000 front 2024-05-20T16:27:44.527267+0000 (oldest deadline 2024-05-20T16:28:09.227181+0000) 2024-05-20T16:28:11.213 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:28:10 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:28:10.869+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-05-20T16:26:47.437555+0000 front 2024-05-20T16:26:47.437468+0000 (oldest deadline 2024-05-20T16:27:15.038423+0000) 2024-05-20T16:28:11.213 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:28:10 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:28:10.869+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:55.038908+0000 front 2024-05-20T16:26:55.038825+0000 (oldest deadline 2024-05-20T16:27:18.439444+0000) 2024-05-20T16:28:11.213 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:28:10 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:28:10.869+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-20T16:27:04.241075+0000 front 2024-05-20T16:27:04.241297+0000 (oldest deadline 2024-05-20T16:27:27.141055+0000) 2024-05-20T16:28:11.213 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:28:10 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:28:10.869+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-05-20T16:27:12.943012+0000 front 2024-05-20T16:27:12.942981+0000 (oldest deadline 2024-05-20T16:27:36.442783+0000) 2024-05-20T16:28:11.214 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:28:10 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:28:10.870+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-20T16:27:21.643969+0000 front 2024-05-20T16:27:21.644003+0000 (oldest deadline 2024-05-20T16:27:46.344106+0000) 2024-05-20T16:28:11.214 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:28:10 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:28:10.870+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-05-20T16:27:30.846286+0000 front 2024-05-20T16:27:30.846116+0000 (oldest deadline 2024-05-20T16:27:55.546056+0000) 2024-05-20T16:28:11.214 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:28:10 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:28:10.870+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.82:6822 osd.6 since back 2024-05-20T16:27:35.546850+0000 front 2024-05-20T16:27:35.546827+0000 (oldest deadline 2024-05-20T16:28:00.246640+0000) 2024-05-20T16:28:11.635 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:28:11 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:28:11.203+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-05-20T16:26:47.217074+0000 front 2024-05-20T16:26:47.216883+0000 (oldest deadline 2024-05-20T16:27:10.716827+0000) 2024-05-20T16:28:11.635 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:28:11 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:28:11.203+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:54.817899+0000 front 2024-05-20T16:26:54.818008+0000 (oldest deadline 2024-05-20T16:27:20.018619+0000) 2024-05-20T16:28:11.635 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:28:11 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:28:11.203+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-20T16:27:01.719536+0000 front 2024-05-20T16:27:01.719675+0000 (oldest deadline 2024-05-20T16:27:27.019603+0000) 2024-05-20T16:28:11.636 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:28:11 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:28:11.203+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-20T16:27:19.722847+0000 front 2024-05-20T16:27:19.722862+0000 (oldest deadline 2024-05-20T16:27:45.022529+0000) 2024-05-20T16:28:11.636 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:28:11 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:28:11.203+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-05-20T16:27:27.323480+0000 front 2024-05-20T16:27:27.323448+0000 (oldest deadline 2024-05-20T16:27:53.723837+0000) 2024-05-20T16:28:11.636 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:28:11 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:28:11.203+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.82:6822 osd.6 since back 2024-05-20T16:27:38.726211+0000 front 2024-05-20T16:27:38.726240+0000 (oldest deadline 2024-05-20T16:28:01.626037+0000) 2024-05-20T16:28:11.636 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:28:11 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:28:11.203+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.160:6820 osd.7 since back 2024-05-20T16:27:44.527310+0000 front 2024-05-20T16:27:44.527267+0000 (oldest deadline 2024-05-20T16:28:09.227181+0000) 2024-05-20T16:28:12.312 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:28:11 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:28:11.825+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-05-20T16:26:47.437555+0000 front 2024-05-20T16:26:47.437468+0000 (oldest deadline 2024-05-20T16:27:15.038423+0000) 2024-05-20T16:28:12.312 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:28:11 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:28:11.825+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:55.038908+0000 front 2024-05-20T16:26:55.038825+0000 (oldest deadline 2024-05-20T16:27:18.439444+0000) 2024-05-20T16:28:12.312 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:28:11 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:28:11.825+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-20T16:27:04.241075+0000 front 2024-05-20T16:27:04.241297+0000 (oldest deadline 2024-05-20T16:27:27.141055+0000) 2024-05-20T16:28:12.313 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:28:11 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:28:11.825+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-05-20T16:27:12.943012+0000 front 2024-05-20T16:27:12.942981+0000 (oldest deadline 2024-05-20T16:27:36.442783+0000) 2024-05-20T16:28:12.313 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:28:11 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:28:11.825+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-20T16:27:21.643969+0000 front 2024-05-20T16:27:21.644003+0000 (oldest deadline 2024-05-20T16:27:46.344106+0000) 2024-05-20T16:28:12.313 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:28:11 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:28:11.825+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-05-20T16:27:30.846286+0000 front 2024-05-20T16:27:30.846116+0000 (oldest deadline 2024-05-20T16:27:55.546056+0000) 2024-05-20T16:28:12.313 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:28:11 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10[61284]: 2024-05-20T16:28:11.825+0000 7f332bec0700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.82:6822 osd.6 since back 2024-05-20T16:27:35.546850+0000 front 2024-05-20T16:27:35.546827+0000 (oldest deadline 2024-05-20T16:28:00.246640+0000) 2024-05-20T16:28:12.635 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:28:12 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:28:12.184+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-05-20T16:26:47.217074+0000 front 2024-05-20T16:26:47.216883+0000 (oldest deadline 2024-05-20T16:27:10.716827+0000) 2024-05-20T16:28:12.636 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:28:12 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:28:12.184+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:54.817899+0000 front 2024-05-20T16:26:54.818008+0000 (oldest deadline 2024-05-20T16:27:20.018619+0000) 2024-05-20T16:28:12.636 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:28:12 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:28:12.184+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-20T16:27:01.719536+0000 front 2024-05-20T16:27:01.719675+0000 (oldest deadline 2024-05-20T16:27:27.019603+0000) 2024-05-20T16:28:12.636 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:28:12 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:28:12.184+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-20T16:27:19.722847+0000 front 2024-05-20T16:27:19.722862+0000 (oldest deadline 2024-05-20T16:27:45.022529+0000) 2024-05-20T16:28:12.636 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:28:12 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:28:12.184+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-05-20T16:27:27.323480+0000 front 2024-05-20T16:27:27.323448+0000 (oldest deadline 2024-05-20T16:27:53.723837+0000) 2024-05-20T16:28:12.637 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:28:12 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:28:12.184+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.82:6822 osd.6 since back 2024-05-20T16:27:38.726211+0000 front 2024-05-20T16:27:38.726240+0000 (oldest deadline 2024-05-20T16:28:01.626037+0000) 2024-05-20T16:28:12.637 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:28:12 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:28:12.184+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.160:6820 osd.7 since back 2024-05-20T16:27:44.527310+0000 front 2024-05-20T16:27:44.527267+0000 (oldest deadline 2024-05-20T16:28:09.227181+0000) 2024-05-20T16:28:13.312 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:28:12 smithi160 podman[72303]: 2024-05-20 16:28:12.945736098 +0000 UTC m=+5.771503879 container died 14aa686b3d1ec177ead5ca61c783fc11bb693cff1a9deb10a15334f0eee806c7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240514, org.label-schema.license=GPLv2) 2024-05-20T16:28:13.635 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:28:13 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:28:13.162+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-05-20T16:26:47.217074+0000 front 2024-05-20T16:26:47.216883+0000 (oldest deadline 2024-05-20T16:27:10.716827+0000) 2024-05-20T16:28:13.635 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:28:13 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:28:13.162+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:54.817899+0000 front 2024-05-20T16:26:54.818008+0000 (oldest deadline 2024-05-20T16:27:20.018619+0000) 2024-05-20T16:28:13.636 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:28:13 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:28:13.162+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-20T16:27:01.719536+0000 front 2024-05-20T16:27:01.719675+0000 (oldest deadline 2024-05-20T16:27:27.019603+0000) 2024-05-20T16:28:13.636 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:28:13 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:28:13.162+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-20T16:27:19.722847+0000 front 2024-05-20T16:27:19.722862+0000 (oldest deadline 2024-05-20T16:27:45.022529+0000) 2024-05-20T16:28:13.636 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:28:13 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:28:13.162+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-05-20T16:27:27.323480+0000 front 2024-05-20T16:27:27.323448+0000 (oldest deadline 2024-05-20T16:27:53.723837+0000) 2024-05-20T16:28:13.636 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:28:13 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:28:13.162+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.82:6822 osd.6 since back 2024-05-20T16:27:38.726211+0000 front 2024-05-20T16:27:38.726240+0000 (oldest deadline 2024-05-20T16:28:01.626037+0000) 2024-05-20T16:28:13.636 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:28:13 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:28:13.162+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.160:6820 osd.7 since back 2024-05-20T16:27:44.527310+0000 front 2024-05-20T16:27:44.527267+0000 (oldest deadline 2024-05-20T16:28:09.227181+0000) 2024-05-20T16:28:13.684 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:28:13 smithi160 podman[72303]: 2024-05-20 16:28:13.442394827 +0000 UTC m=+6.268162596 container cleanup 14aa686b3d1ec177ead5ca61c783fc11bb693cff1a9deb10a15334f0eee806c7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240514, ceph=True, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-05-20T16:28:13.684 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:28:13 smithi160 bash[72303]: ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10 2024-05-20T16:28:13.998 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:28:13 smithi160 podman[72321]: 2024-05-20 16:28:13.685047164 +0000 UTC m=+0.737346321 container remove 14aa686b3d1ec177ead5ca61c783fc11bb693cff1a9deb10a15334f0eee806c7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, io.buildah.version=1.33.5, org.label-schema.build-date=20240514, org.label-schema.schema-version=1.0) 2024-05-20T16:28:14.295 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:28:14 smithi160 podman[72407]: 2024-05-20 16:28:13.902625673 +0000 UTC m=+0.017645910 image pull cfcc61b2d476f611479310eb042aa05f11dd5f60dbb65e3b0d7e0e11d024cdb3 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708 2024-05-20T16:28:14.295 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:28:14 smithi160 podman[72407]: 2024-05-20 16:28:14.002329643 +0000 UTC m=+0.117349853 container create a9cd03b6ff859942ed83c8bfc7db88818122e993e5e4b91d823ed7fc827446dd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10-deactivate, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240514) 2024-05-20T16:28:14.295 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:28:14 smithi160 podman[72407]: 2024-05-20 16:28:14.221348834 +0000 UTC m=+0.336369050 container init a9cd03b6ff859942ed83c8bfc7db88818122e993e5e4b91d823ed7fc827446dd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10-deactivate, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240514, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-05-20T16:28:14.296 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:28:14 smithi160 podman[72407]: 2024-05-20 16:28:14.224503575 +0000 UTC m=+0.339523795 container start a9cd03b6ff859942ed83c8bfc7db88818122e993e5e4b91d823ed7fc827446dd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10-deactivate, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240514) 2024-05-20T16:28:14.296 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:28:14 smithi160 podman[72407]: 2024-05-20 16:28:14.296207035 +0000 UTC m=+0.411227276 container attach a9cd03b6ff859942ed83c8bfc7db88818122e993e5e4b91d823ed7fc827446dd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10-deactivate, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240514, GIT_BRANCH=HEAD) 2024-05-20T16:28:14.386 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:28:14 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:28:14.133+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-05-20T16:26:47.217074+0000 front 2024-05-20T16:26:47.216883+0000 (oldest deadline 2024-05-20T16:27:10.716827+0000) 2024-05-20T16:28:14.386 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:28:14 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:28:14.133+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:54.817899+0000 front 2024-05-20T16:26:54.818008+0000 (oldest deadline 2024-05-20T16:27:20.018619+0000) 2024-05-20T16:28:14.386 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:28:14 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:28:14.133+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-20T16:27:01.719536+0000 front 2024-05-20T16:27:01.719675+0000 (oldest deadline 2024-05-20T16:27:27.019603+0000) 2024-05-20T16:28:14.386 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:28:14 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:28:14.133+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-20T16:27:19.722847+0000 front 2024-05-20T16:27:19.722862+0000 (oldest deadline 2024-05-20T16:27:45.022529+0000) 2024-05-20T16:28:14.387 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:28:14 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:28:14.133+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-05-20T16:27:27.323480+0000 front 2024-05-20T16:27:27.323448+0000 (oldest deadline 2024-05-20T16:27:53.723837+0000) 2024-05-20T16:28:14.387 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:28:14 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:28:14.133+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.82:6822 osd.6 since back 2024-05-20T16:27:38.726211+0000 front 2024-05-20T16:27:38.726240+0000 (oldest deadline 2024-05-20T16:28:01.626037+0000) 2024-05-20T16:28:14.387 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:28:14 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:28:14.133+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.160:6820 osd.7 since back 2024-05-20T16:27:44.527310+0000 front 2024-05-20T16:27:44.527267+0000 (oldest deadline 2024-05-20T16:28:09.227181+0000) 2024-05-20T16:28:14.562 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 20 16:28:14 smithi160 podman[72407]: 2024-05-20 16:28:14.516509821 +0000 UTC m=+0.631530039 container died a9cd03b6ff859942ed83c8bfc7db88818122e993e5e4b91d823ed7fc827446dd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-10-deactivate, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240514, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-20T16:28:15.099 DEBUG:teuthology.orchestra.run.smithi160:> sudo pkill -f 'journalctl -f -n 0 -u ceph-59d1600a-16c2-11ef-bc9a-c7b262605968@osd.10.service' 2024-05-20T16:28:15.137 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-20T16:28:15.137 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-05-20T16:28:15.137 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-05-20T16:28:15.138 DEBUG:teuthology.orchestra.run.smithi184:> sudo systemctl stop ceph-59d1600a-16c2-11ef-bc9a-c7b262605968@osd.11 2024-05-20T16:28:15.386 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:28:15 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:28:15.124+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-05-20T16:26:47.217074+0000 front 2024-05-20T16:26:47.216883+0000 (oldest deadline 2024-05-20T16:27:10.716827+0000) 2024-05-20T16:28:15.386 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:28:15 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:28:15.124+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:54.817899+0000 front 2024-05-20T16:26:54.818008+0000 (oldest deadline 2024-05-20T16:27:20.018619+0000) 2024-05-20T16:28:15.386 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:28:15 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:28:15.124+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-20T16:27:01.719536+0000 front 2024-05-20T16:27:01.719675+0000 (oldest deadline 2024-05-20T16:27:27.019603+0000) 2024-05-20T16:28:15.386 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:28:15 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:28:15.124+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-20T16:27:19.722847+0000 front 2024-05-20T16:27:19.722862+0000 (oldest deadline 2024-05-20T16:27:45.022529+0000) 2024-05-20T16:28:15.386 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:28:15 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:28:15.124+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-05-20T16:27:27.323480+0000 front 2024-05-20T16:27:27.323448+0000 (oldest deadline 2024-05-20T16:27:53.723837+0000) 2024-05-20T16:28:15.386 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:28:15 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:28:15.124+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.82:6822 osd.6 since back 2024-05-20T16:27:38.726211+0000 front 2024-05-20T16:27:38.726240+0000 (oldest deadline 2024-05-20T16:28:01.626037+0000) 2024-05-20T16:28:15.386 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:28:15 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:28:15.124+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.160:6820 osd.7 since back 2024-05-20T16:27:44.527310+0000 front 2024-05-20T16:27:44.527267+0000 (oldest deadline 2024-05-20T16:28:09.227181+0000) 2024-05-20T16:28:15.386 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:28:15 smithi184 systemd[1]: Stopping Ceph osd.11 for 59d1600a-16c2-11ef-bc9a-c7b262605968... 2024-05-20T16:28:15.886 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:28:15 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:28:15.474+0000 7f99d20ee700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-20T16:28:15.886 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:28:15 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:28:15.474+0000 7f99d20ee700 -1 osd.11 102 *** Got signal Terminated *** 2024-05-20T16:28:15.886 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:28:15 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:28:15.474+0000 7f99d20ee700 -1 osd.11 102 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-20T16:28:16.386 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:28:16 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:28:16.122+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-05-20T16:26:47.217074+0000 front 2024-05-20T16:26:47.216883+0000 (oldest deadline 2024-05-20T16:27:10.716827+0000) 2024-05-20T16:28:16.386 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:28:16 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:28:16.122+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:54.817899+0000 front 2024-05-20T16:26:54.818008+0000 (oldest deadline 2024-05-20T16:27:20.018619+0000) 2024-05-20T16:28:16.386 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:28:16 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:28:16.122+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-20T16:27:01.719536+0000 front 2024-05-20T16:27:01.719675+0000 (oldest deadline 2024-05-20T16:27:27.019603+0000) 2024-05-20T16:28:16.386 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:28:16 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:28:16.122+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-20T16:27:19.722847+0000 front 2024-05-20T16:27:19.722862+0000 (oldest deadline 2024-05-20T16:27:45.022529+0000) 2024-05-20T16:28:16.386 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:28:16 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:28:16.122+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-05-20T16:27:27.323480+0000 front 2024-05-20T16:27:27.323448+0000 (oldest deadline 2024-05-20T16:27:53.723837+0000) 2024-05-20T16:28:16.386 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:28:16 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:28:16.122+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.82:6822 osd.6 since back 2024-05-20T16:27:38.726211+0000 front 2024-05-20T16:27:38.726240+0000 (oldest deadline 2024-05-20T16:28:01.626037+0000) 2024-05-20T16:28:16.386 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:28:16 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:28:16.122+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.160:6820 osd.7 since back 2024-05-20T16:27:44.527310+0000 front 2024-05-20T16:27:44.527267+0000 (oldest deadline 2024-05-20T16:28:09.227181+0000) 2024-05-20T16:28:17.635 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:28:17 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:28:17.141+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-05-20T16:26:47.217074+0000 front 2024-05-20T16:26:47.216883+0000 (oldest deadline 2024-05-20T16:27:10.716827+0000) 2024-05-20T16:28:17.635 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:28:17 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:28:17.141+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:54.817899+0000 front 2024-05-20T16:26:54.818008+0000 (oldest deadline 2024-05-20T16:27:20.018619+0000) 2024-05-20T16:28:17.636 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:28:17 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:28:17.141+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-20T16:27:01.719536+0000 front 2024-05-20T16:27:01.719675+0000 (oldest deadline 2024-05-20T16:27:27.019603+0000) 2024-05-20T16:28:17.636 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:28:17 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:28:17.141+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-20T16:27:19.722847+0000 front 2024-05-20T16:27:19.722862+0000 (oldest deadline 2024-05-20T16:27:45.022529+0000) 2024-05-20T16:28:17.636 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:28:17 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:28:17.141+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-05-20T16:27:27.323480+0000 front 2024-05-20T16:27:27.323448+0000 (oldest deadline 2024-05-20T16:27:53.723837+0000) 2024-05-20T16:28:17.636 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:28:17 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:28:17.141+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.82:6822 osd.6 since back 2024-05-20T16:27:38.726211+0000 front 2024-05-20T16:27:38.726240+0000 (oldest deadline 2024-05-20T16:28:01.626037+0000) 2024-05-20T16:28:17.636 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:28:17 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:28:17.141+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.160:6820 osd.7 since back 2024-05-20T16:27:44.527310+0000 front 2024-05-20T16:27:44.527267+0000 (oldest deadline 2024-05-20T16:28:09.227181+0000) 2024-05-20T16:28:18.635 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:28:18 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:28:18.167+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-05-20T16:26:47.217074+0000 front 2024-05-20T16:26:47.216883+0000 (oldest deadline 2024-05-20T16:27:10.716827+0000) 2024-05-20T16:28:18.635 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:28:18 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:28:18.167+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:54.817899+0000 front 2024-05-20T16:26:54.818008+0000 (oldest deadline 2024-05-20T16:27:20.018619+0000) 2024-05-20T16:28:18.636 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:28:18 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:28:18.167+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-20T16:27:01.719536+0000 front 2024-05-20T16:27:01.719675+0000 (oldest deadline 2024-05-20T16:27:27.019603+0000) 2024-05-20T16:28:18.636 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:28:18 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:28:18.167+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-20T16:27:19.722847+0000 front 2024-05-20T16:27:19.722862+0000 (oldest deadline 2024-05-20T16:27:45.022529+0000) 2024-05-20T16:28:18.636 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:28:18 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:28:18.167+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-05-20T16:27:27.323480+0000 front 2024-05-20T16:27:27.323448+0000 (oldest deadline 2024-05-20T16:27:53.723837+0000) 2024-05-20T16:28:18.636 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:28:18 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:28:18.167+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.82:6822 osd.6 since back 2024-05-20T16:27:38.726211+0000 front 2024-05-20T16:27:38.726240+0000 (oldest deadline 2024-05-20T16:28:01.626037+0000) 2024-05-20T16:28:18.636 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:28:18 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:28:18.167+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.160:6820 osd.7 since back 2024-05-20T16:27:44.527310+0000 front 2024-05-20T16:27:44.527267+0000 (oldest deadline 2024-05-20T16:28:09.227181+0000) 2024-05-20T16:28:18.636 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:28:18 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:28:18.167+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.184:6820 osd.8 since back 2024-05-20T16:27:53.327978+0000 front 2024-05-20T16:27:53.328034+0000 (oldest deadline 2024-05-20T16:28:18.028253+0000) 2024-05-20T16:28:19.635 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:28:19 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:28:19.162+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-05-20T16:26:47.217074+0000 front 2024-05-20T16:26:47.216883+0000 (oldest deadline 2024-05-20T16:27:10.716827+0000) 2024-05-20T16:28:19.635 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:28:19 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:28:19.162+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:54.817899+0000 front 2024-05-20T16:26:54.818008+0000 (oldest deadline 2024-05-20T16:27:20.018619+0000) 2024-05-20T16:28:19.636 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:28:19 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:28:19.162+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-20T16:27:01.719536+0000 front 2024-05-20T16:27:01.719675+0000 (oldest deadline 2024-05-20T16:27:27.019603+0000) 2024-05-20T16:28:19.636 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:28:19 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:28:19.162+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-20T16:27:19.722847+0000 front 2024-05-20T16:27:19.722862+0000 (oldest deadline 2024-05-20T16:27:45.022529+0000) 2024-05-20T16:28:19.636 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:28:19 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:28:19.162+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-05-20T16:27:27.323480+0000 front 2024-05-20T16:27:27.323448+0000 (oldest deadline 2024-05-20T16:27:53.723837+0000) 2024-05-20T16:28:19.636 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:28:19 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:28:19.162+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.82:6822 osd.6 since back 2024-05-20T16:27:38.726211+0000 front 2024-05-20T16:27:38.726240+0000 (oldest deadline 2024-05-20T16:28:01.626037+0000) 2024-05-20T16:28:19.636 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:28:19 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:28:19.162+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.160:6820 osd.7 since back 2024-05-20T16:27:44.527310+0000 front 2024-05-20T16:27:44.527267+0000 (oldest deadline 2024-05-20T16:28:09.227181+0000) 2024-05-20T16:28:19.636 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:28:19 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:28:19.162+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.184:6820 osd.8 since back 2024-05-20T16:27:53.327978+0000 front 2024-05-20T16:27:53.328034+0000 (oldest deadline 2024-05-20T16:28:18.028253+0000) 2024-05-20T16:28:20.385 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:28:20 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:28:20.125+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-05-20T16:26:47.217074+0000 front 2024-05-20T16:26:47.216883+0000 (oldest deadline 2024-05-20T16:27:10.716827+0000) 2024-05-20T16:28:20.386 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:28:20 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:28:20.125+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-20T16:26:54.817899+0000 front 2024-05-20T16:26:54.818008+0000 (oldest deadline 2024-05-20T16:27:20.018619+0000) 2024-05-20T16:28:20.386 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:28:20 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:28:20.125+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-20T16:27:01.719536+0000 front 2024-05-20T16:27:01.719675+0000 (oldest deadline 2024-05-20T16:27:27.019603+0000) 2024-05-20T16:28:20.386 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:28:20 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:28:20.125+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-20T16:27:19.722847+0000 front 2024-05-20T16:27:19.722862+0000 (oldest deadline 2024-05-20T16:27:45.022529+0000) 2024-05-20T16:28:20.386 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:28:20 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:28:20.125+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-05-20T16:27:27.323480+0000 front 2024-05-20T16:27:27.323448+0000 (oldest deadline 2024-05-20T16:27:53.723837+0000) 2024-05-20T16:28:20.386 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:28:20 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:28:20.125+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.82:6822 osd.6 since back 2024-05-20T16:27:38.726211+0000 front 2024-05-20T16:27:38.726240+0000 (oldest deadline 2024-05-20T16:28:01.626037+0000) 2024-05-20T16:28:20.386 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:28:20 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:28:20.125+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.160:6820 osd.7 since back 2024-05-20T16:27:44.527310+0000 front 2024-05-20T16:27:44.527267+0000 (oldest deadline 2024-05-20T16:28:09.227181+0000) 2024-05-20T16:28:20.386 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:28:20 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11[62113]: 2024-05-20T16:28:20.125+0000 7f99ce6a4700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.184:6820 osd.8 since back 2024-05-20T16:27:53.327978+0000 front 2024-05-20T16:27:53.328034+0000 (oldest deadline 2024-05-20T16:28:18.028253+0000) 2024-05-20T16:28:21.494 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:28:21 smithi184 podman[73642]: 2024-05-20 16:28:21.214993977 +0000 UTC m=+5.862976880 container died 69f1b6e3fbc5b85f2cfd9dcf1b4eee66e4a91ccce6d7f6364144a0518ff5f6f8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240514, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-05-20T16:28:21.774 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:28:21 smithi184 podman[73642]: 2024-05-20 16:28:21.557441635 +0000 UTC m=+6.205424538 container cleanup 69f1b6e3fbc5b85f2cfd9dcf1b4eee66e4a91ccce6d7f6364144a0518ff5f6f8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240514, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-20T16:28:21.774 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:28:21 smithi184 bash[73642]: ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11 2024-05-20T16:28:22.135 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:28:21 smithi184 podman[73657]: 2024-05-20 16:28:21.775486189 +0000 UTC m=+0.551737984 container remove 69f1b6e3fbc5b85f2cfd9dcf1b4eee66e4a91ccce6d7f6364144a0518ff5f6f8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240514, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-20T16:28:22.469 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:28:22 smithi184 podman[73747]: 2024-05-20 16:28:22.083491184 +0000 UTC m=+0.017681235 image pull cfcc61b2d476f611479310eb042aa05f11dd5f60dbb65e3b0d7e0e11d024cdb3 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708 2024-05-20T16:28:22.470 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:28:22 smithi184 podman[73747]: 2024-05-20 16:28:22.193262275 +0000 UTC m=+0.127452314 container create 1fb99258b5656a46cb80d5ab866abec7a8a75e02cb6c77010bee4169a56dc098 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11-deactivate, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240514, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, ceph=True) 2024-05-20T16:28:22.470 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:28:22 smithi184 podman[73747]: 2024-05-20 16:28:22.387165404 +0000 UTC m=+0.321355443 container init 1fb99258b5656a46cb80d5ab866abec7a8a75e02cb6c77010bee4169a56dc098 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11-deactivate, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240514, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True) 2024-05-20T16:28:22.470 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:28:22 smithi184 podman[73747]: 2024-05-20 16:28:22.390080866 +0000 UTC m=+0.324270907 container start 1fb99258b5656a46cb80d5ab866abec7a8a75e02cb6c77010bee4169a56dc098 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11-deactivate, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240514, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-05-20T16:28:22.885 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:28:22 smithi184 podman[73747]: 2024-05-20 16:28:22.470475461 +0000 UTC m=+0.404665500 container attach 1fb99258b5656a46cb80d5ab866abec7a8a75e02cb6c77010bee4169a56dc098 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240514, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS) 2024-05-20T16:28:22.885 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:28:22 smithi184 podman[73747]: 2024-05-20 16:28:22.677805766 +0000 UTC m=+0.611995807 container died 1fb99258b5656a46cb80d5ab866abec7a8a75e02cb6c77010bee4169a56dc098 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11-deactivate, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240514, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-05-20T16:28:23.225 DEBUG:teuthology.orchestra.run.smithi184:> sudo pkill -f 'journalctl -f -n 0 -u ceph-59d1600a-16c2-11ef-bc9a-c7b262605968@osd.11.service' 2024-05-20T16:28:23.246 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:28:23 smithi184 podman[73774]: 2024-05-20 16:28:23.207237993 +0000 UTC m=+0.523064802 container remove 1fb99258b5656a46cb80d5ab866abec7a8a75e02cb6c77010bee4169a56dc098 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-osd-11-deactivate, ceph=True, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240514, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-05-20T16:28:23.246 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:28:23 smithi184 systemd[1]: ceph-59d1600a-16c2-11ef-bc9a-c7b262605968@osd.11.service: Deactivated successfully. 2024-05-20T16:28:23.247 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:28:23 smithi184 systemd[1]: Stopped Ceph osd.11 for 59d1600a-16c2-11ef-bc9a-c7b262605968. 2024-05-20T16:28:23.247 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 20 16:28:23 smithi184 systemd[1]: ceph-59d1600a-16c2-11ef-bc9a-c7b262605968@osd.11.service: Consumed 9min 1.319s CPU time. 2024-05-20T16:28:23.261 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-20T16:28:23.261 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-05-20T16:28:23.261 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-05-20T16:28:23.261 DEBUG:teuthology.orchestra.run.smithi082:> sudo systemctl stop ceph-59d1600a-16c2-11ef-bc9a-c7b262605968@mds.a 2024-05-20T16:28:23.539 INFO:journalctl@ceph.mds.a.smithi082.stdout:May 20 16:28:23 smithi082 systemd[1]: Stopping Ceph mds.a for 59d1600a-16c2-11ef-bc9a-c7b262605968... 2024-05-20T16:28:23.810 INFO:journalctl@ceph.mds.a.smithi082.stdout:May 20 16:28:23 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-a[73412]: 2024-05-20T16:28:23.539+0000 7f2626bcd700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-20T16:28:23.810 INFO:journalctl@ceph.mds.a.smithi082.stdout:May 20 16:28:23 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-a[73412]: 2024-05-20T16:28:23.539+0000 7f2626bcd700 -1 mds.a *** got signal Terminated *** 2024-05-20T16:28:27.821 INFO:journalctl@ceph.mds.a.smithi082.stdout:May 20 16:28:27 smithi082 podman[92635]: 2024-05-20 16:28:27.567266533 +0000 UTC m=+4.106488677 container died d27456f6239e54ef86124f6ffd278195f9b281d45c270422b0a0a61d4bf716ae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-a, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240514, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, CEPH_POINT_RELEASE=) 2024-05-20T16:28:28.088 INFO:journalctl@ceph.mds.a.smithi082.stdout:May 20 16:28:27 smithi082 podman[92635]: 2024-05-20 16:28:27.905174806 +0000 UTC m=+4.444396941 container cleanup d27456f6239e54ef86124f6ffd278195f9b281d45c270422b0a0a61d4bf716ae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-a, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240514, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c) 2024-05-20T16:28:28.088 INFO:journalctl@ceph.mds.a.smithi082.stdout:May 20 16:28:27 smithi082 bash[92635]: ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-a 2024-05-20T16:28:28.309 DEBUG:teuthology.orchestra.run.smithi082:> sudo pkill -f 'journalctl -f -n 0 -u ceph-59d1600a-16c2-11ef-bc9a-c7b262605968@mds.a.service' 2024-05-20T16:28:28.346 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-20T16:28:28.346 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-05-20T16:28:28.346 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-05-20T16:28:28.346 DEBUG:teuthology.orchestra.run.smithi082:> sudo systemctl stop ceph-59d1600a-16c2-11ef-bc9a-c7b262605968@mds.d 2024-05-20T16:28:28.608 INFO:journalctl@ceph.mds.d.smithi082.stdout:May 20 16:28:28 smithi082 systemd[1]: Stopping Ceph mds.d for 59d1600a-16c2-11ef-bc9a-c7b262605968... 2024-05-20T16:28:29.109 INFO:journalctl@ceph.mds.d.smithi082.stdout:May 20 16:28:28 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-d[72463]: 2024-05-20T16:28:28.650+0000 7f30f0bc9700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-20T16:28:29.109 INFO:journalctl@ceph.mds.d.smithi082.stdout:May 20 16:28:28 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-d[72463]: 2024-05-20T16:28:28.650+0000 7f30f0bc9700 -1 mds.d *** got signal Terminated *** 2024-05-20T16:28:32.815 INFO:journalctl@ceph.mds.d.smithi082.stdout:May 20 16:28:32 smithi082 podman[92770]: 2024-05-20 16:28:32.52291029 +0000 UTC m=+3.953329025 container died 98a34b38a26205de4d53a8c54e1f2151826bffe593bbe7dac9d2edb99fb48b9f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-d, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, org.label-schema.build-date=20240514, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-05-20T16:28:33.109 INFO:journalctl@ceph.mds.d.smithi082.stdout:May 20 16:28:32 smithi082 podman[92770]: 2024-05-20 16:28:32.816984936 +0000 UTC m=+4.247403664 container cleanup 98a34b38a26205de4d53a8c54e1f2151826bffe593bbe7dac9d2edb99fb48b9f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-d, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240514, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-05-20T16:28:33.109 INFO:journalctl@ceph.mds.d.smithi082.stdout:May 20 16:28:32 smithi082 bash[92770]: ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-d 2024-05-20T16:28:33.109 INFO:journalctl@ceph.mds.d.smithi082.stdout:May 20 16:28:33 smithi082 podman[92781]: 2024-05-20 16:28:33.043109227 +0000 UTC m=+0.510835922 container remove 98a34b38a26205de4d53a8c54e1f2151826bffe593bbe7dac9d2edb99fb48b9f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-d, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240514, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS) 2024-05-20T16:28:33.175 DEBUG:teuthology.orchestra.run.smithi082:> sudo pkill -f 'journalctl -f -n 0 -u ceph-59d1600a-16c2-11ef-bc9a-c7b262605968@mds.d.service' 2024-05-20T16:28:33.208 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-20T16:28:33.209 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-05-20T16:28:33.209 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-05-20T16:28:33.209 DEBUG:teuthology.orchestra.run.smithi082:> sudo systemctl stop ceph-59d1600a-16c2-11ef-bc9a-c7b262605968@mds.g 2024-05-20T16:28:33.528 INFO:journalctl@ceph.mds.g.smithi082.stdout:May 20 16:28:33 smithi082 systemd[1]: Stopping Ceph mds.g for 59d1600a-16c2-11ef-bc9a-c7b262605968... 2024-05-20T16:28:33.858 INFO:journalctl@ceph.mds.g.smithi082.stdout:May 20 16:28:33 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-g[73054]: 2024-05-20T16:28:33.528+0000 7f5e204f6700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-20T16:28:33.859 INFO:journalctl@ceph.mds.g.smithi082.stdout:May 20 16:28:33 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-g[73054]: 2024-05-20T16:28:33.528+0000 7f5e204f6700 -1 mds.g *** got signal Terminated *** 2024-05-20T16:28:37.786 INFO:journalctl@ceph.mds.g.smithi082.stdout:May 20 16:28:37 smithi082 podman[92893]: 2024-05-20 16:28:37.536679755 +0000 UTC m=+4.101785968 container died e65fc67c67e41761d3346ac331f0d1cc1292972c6f7458ce3838f10a24efb837 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-g, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240514, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-20T16:28:38.109 INFO:journalctl@ceph.mds.g.smithi082.stdout:May 20 16:28:37 smithi082 podman[92893]: 2024-05-20 16:28:37.87853552 +0000 UTC m=+4.443641723 container cleanup e65fc67c67e41761d3346ac331f0d1cc1292972c6f7458ce3838f10a24efb837 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-g, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240514, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-20T16:28:38.109 INFO:journalctl@ceph.mds.g.smithi082.stdout:May 20 16:28:37 smithi082 bash[92893]: ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-g 2024-05-20T16:28:38.358 DEBUG:teuthology.orchestra.run.smithi082:> sudo pkill -f 'journalctl -f -n 0 -u ceph-59d1600a-16c2-11ef-bc9a-c7b262605968@mds.g.service' 2024-05-20T16:28:38.380 INFO:journalctl@ceph.mds.g.smithi082.stdout:May 20 16:28:38 smithi082 podman[92908]: 2024-05-20 16:28:38.129999938 +0000 UTC m=+0.583863468 container remove e65fc67c67e41761d3346ac331f0d1cc1292972c6f7458ce3838f10a24efb837 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-g, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240514, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-20T16:28:38.380 INFO:journalctl@ceph.mds.g.smithi082.stdout:May 20 16:28:38 smithi082 systemd[1]: ceph-59d1600a-16c2-11ef-bc9a-c7b262605968@mds.g.service: Deactivated successfully. 2024-05-20T16:28:38.380 INFO:journalctl@ceph.mds.g.smithi082.stdout:May 20 16:28:38 smithi082 systemd[1]: Stopped Ceph mds.g for 59d1600a-16c2-11ef-bc9a-c7b262605968. 2024-05-20T16:28:38.380 INFO:journalctl@ceph.mds.g.smithi082.stdout:May 20 16:28:38 smithi082 systemd[1]: ceph-59d1600a-16c2-11ef-bc9a-c7b262605968@mds.g.service: Consumed 1.101s CPU time. 2024-05-20T16:28:38.391 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-20T16:28:38.391 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-05-20T16:28:38.391 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-05-20T16:28:38.391 DEBUG:teuthology.orchestra.run.smithi082:> sudo systemctl stop ceph-59d1600a-16c2-11ef-bc9a-c7b262605968@mds.j 2024-05-20T16:28:38.723 INFO:journalctl@ceph.mds.j.smithi082.stdout:May 20 16:28:38 smithi082 systemd[1]: Stopping Ceph mds.j for 59d1600a-16c2-11ef-bc9a-c7b262605968... 2024-05-20T16:28:39.109 INFO:journalctl@ceph.mds.j.smithi082.stdout:May 20 16:28:38 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-j[72118]: 2024-05-20T16:28:38.724+0000 7f38e697c700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-20T16:28:39.109 INFO:journalctl@ceph.mds.j.smithi082.stdout:May 20 16:28:38 smithi082 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-j[72118]: 2024-05-20T16:28:38.724+0000 7f38e697c700 -1 mds.j *** got signal Terminated *** 2024-05-20T16:28:41.924 INFO:journalctl@ceph.mds.j.smithi082.stdout:May 20 16:28:41 smithi082 podman[93014]: 2024-05-20 16:28:41.666579844 +0000 UTC m=+3.051291517 container died 94754e0ff0ac95fadded8ca48f6eb594edaeb91b742dd1f23680815fd3ee5cea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-j, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240514, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-05-20T16:28:42.207 INFO:journalctl@ceph.mds.j.smithi082.stdout:May 20 16:28:41 smithi082 podman[93014]: 2024-05-20 16:28:41.982206269 +0000 UTC m=+3.366917935 container cleanup 94754e0ff0ac95fadded8ca48f6eb594edaeb91b742dd1f23680815fd3ee5cea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-j, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240514) 2024-05-20T16:28:42.207 INFO:journalctl@ceph.mds.j.smithi082.stdout:May 20 16:28:41 smithi082 bash[93014]: ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-j 2024-05-20T16:28:42.343 DEBUG:teuthology.orchestra.run.smithi082:> sudo pkill -f 'journalctl -f -n 0 -u ceph-59d1600a-16c2-11ef-bc9a-c7b262605968@mds.j.service' 2024-05-20T16:28:42.375 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-20T16:28:42.375 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-05-20T16:28:42.375 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-05-20T16:28:42.375 DEBUG:teuthology.orchestra.run.smithi160:> sudo systemctl stop ceph-59d1600a-16c2-11ef-bc9a-c7b262605968@mds.b 2024-05-20T16:28:42.708 INFO:journalctl@ceph.mds.b.smithi160.stdout:May 20 16:28:42 smithi160 systemd[1]: Stopping Ceph mds.b for 59d1600a-16c2-11ef-bc9a-c7b262605968... 2024-05-20T16:28:43.028 INFO:journalctl@ceph.mds.b.smithi160.stdout:May 20 16:28:42 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-b[66944]: 2024-05-20T16:28:42.707+0000 7f8d62005700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-20T16:28:43.029 INFO:journalctl@ceph.mds.b.smithi160.stdout:May 20 16:28:42 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-b[66944]: 2024-05-20T16:28:42.707+0000 7f8d62005700 -1 mds.b *** got signal Terminated *** 2024-05-20T16:28:46.308 INFO:journalctl@ceph.mds.b.smithi160.stdout:May 20 16:28:46 smithi160 podman[72512]: 2024-05-20 16:28:46.049740639 +0000 UTC m=+3.481363522 container died adc717c8bfc17087d9177d0c6e21018a2d2162505b682db9843921ede41dd960 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-b, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, org.label-schema.build-date=20240514, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-05-20T16:28:46.562 INFO:journalctl@ceph.mds.b.smithi160.stdout:May 20 16:28:46 smithi160 podman[72512]: 2024-05-20 16:28:46.391158038 +0000 UTC m=+3.822780913 container cleanup adc717c8bfc17087d9177d0c6e21018a2d2162505b682db9843921ede41dd960 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-b, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240514, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c) 2024-05-20T16:28:46.562 INFO:journalctl@ceph.mds.b.smithi160.stdout:May 20 16:28:46 smithi160 bash[72512]: ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-b 2024-05-20T16:28:46.822 DEBUG:teuthology.orchestra.run.smithi160:> sudo pkill -f 'journalctl -f -n 0 -u ceph-59d1600a-16c2-11ef-bc9a-c7b262605968@mds.b.service' 2024-05-20T16:28:46.857 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-20T16:28:46.857 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-05-20T16:28:46.858 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-05-20T16:28:46.858 DEBUG:teuthology.orchestra.run.smithi160:> sudo systemctl stop ceph-59d1600a-16c2-11ef-bc9a-c7b262605968@mds.e 2024-05-20T16:28:47.202 INFO:journalctl@ceph.mds.e.smithi160.stdout:May 20 16:28:46 smithi160 systemd[1]: Stopping Ceph mds.e for 59d1600a-16c2-11ef-bc9a-c7b262605968... 2024-05-20T16:28:47.562 INFO:journalctl@ceph.mds.e.smithi160.stdout:May 20 16:28:47 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-e[66323]: 2024-05-20T16:28:47.201+0000 7f9bc4871700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-20T16:28:47.562 INFO:journalctl@ceph.mds.e.smithi160.stdout:May 20 16:28:47 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-e[66323]: 2024-05-20T16:28:47.201+0000 7f9bc4871700 -1 mds.e *** got signal Terminated *** 2024-05-20T16:28:50.311 INFO:journalctl@ceph.mds.e.smithi160.stdout:May 20 16:28:50 smithi160 podman[72647]: 2024-05-20 16:28:50.050150981 +0000 UTC m=+2.967676746 container died cc8764d81953378b45630be4d1aef6d729048f6ad7342e09a40857ddbeeeac0a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-e, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240514, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, org.label-schema.vendor=CentOS) 2024-05-20T16:28:50.562 INFO:journalctl@ceph.mds.e.smithi160.stdout:May 20 16:28:50 smithi160 podman[72647]: 2024-05-20 16:28:50.395029685 +0000 UTC m=+3.312555445 container cleanup cc8764d81953378b45630be4d1aef6d729048f6ad7342e09a40857ddbeeeac0a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-e, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240514, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, org.label-schema.vendor=CentOS) 2024-05-20T16:28:50.562 INFO:journalctl@ceph.mds.e.smithi160.stdout:May 20 16:28:50 smithi160 bash[72647]: ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-e 2024-05-20T16:28:50.957 INFO:journalctl@ceph.mds.e.smithi160.stdout:May 20 16:28:50 smithi160 podman[72661]: 2024-05-20 16:28:50.687813612 +0000 UTC m=+0.630280686 container remove cc8764d81953378b45630be4d1aef6d729048f6ad7342e09a40857ddbeeeac0a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-e, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, org.label-schema.build-date=20240514, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-05-20T16:28:50.983 DEBUG:teuthology.orchestra.run.smithi160:> sudo pkill -f 'journalctl -f -n 0 -u ceph-59d1600a-16c2-11ef-bc9a-c7b262605968@mds.e.service' 2024-05-20T16:28:51.016 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-20T16:28:51.017 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-05-20T16:28:51.017 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-05-20T16:28:51.018 DEBUG:teuthology.orchestra.run.smithi160:> sudo systemctl stop ceph-59d1600a-16c2-11ef-bc9a-c7b262605968@mds.h 2024-05-20T16:28:51.312 INFO:journalctl@ceph.mds.h.smithi160.stdout:May 20 16:28:51 smithi160 systemd[1]: Stopping Ceph mds.h for 59d1600a-16c2-11ef-bc9a-c7b262605968... 2024-05-20T16:28:51.811 INFO:journalctl@ceph.mds.h.smithi160.stdout:May 20 16:28:51 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-h[66640]: 2024-05-20T16:28:51.347+0000 7f42aa97b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-20T16:28:51.812 INFO:journalctl@ceph.mds.h.smithi160.stdout:May 20 16:28:51 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-h[66640]: 2024-05-20T16:28:51.347+0000 7f42aa97b700 -1 mds.h *** got signal Terminated *** 2024-05-20T16:28:54.306 INFO:journalctl@ceph.mds.h.smithi160.stdout:May 20 16:28:54 smithi160 podman[72768]: 2024-05-20 16:28:54.04167334 +0000 UTC m=+2.805505006 container died 0e2acd3f66e0f0b71d50460f761fbd57505a06c0c7e29aefe2b3431a89436d99 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-h, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240514, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True) 2024-05-20T16:28:54.561 INFO:journalctl@ceph.mds.h.smithi160.stdout:May 20 16:28:54 smithi160 podman[72768]: 2024-05-20 16:28:54.406888341 +0000 UTC m=+3.170719998 container cleanup 0e2acd3f66e0f0b71d50460f761fbd57505a06c0c7e29aefe2b3431a89436d99 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-h, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240514, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, maintainer=Guillaume Abrioux ) 2024-05-20T16:28:54.562 INFO:journalctl@ceph.mds.h.smithi160.stdout:May 20 16:28:54 smithi160 bash[72768]: ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-h 2024-05-20T16:28:54.829 DEBUG:teuthology.orchestra.run.smithi160:> sudo pkill -f 'journalctl -f -n 0 -u ceph-59d1600a-16c2-11ef-bc9a-c7b262605968@mds.h.service' 2024-05-20T16:28:54.860 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-20T16:28:54.860 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-05-20T16:28:54.860 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-05-20T16:28:54.860 DEBUG:teuthology.orchestra.run.smithi160:> sudo systemctl stop ceph-59d1600a-16c2-11ef-bc9a-c7b262605968@mds.k 2024-05-20T16:28:55.201 INFO:journalctl@ceph.mds.k.smithi160.stdout:May 20 16:28:54 smithi160 systemd[1]: Stopping Ceph mds.k for 59d1600a-16c2-11ef-bc9a-c7b262605968... 2024-05-20T16:28:55.562 INFO:journalctl@ceph.mds.k.smithi160.stdout:May 20 16:28:55 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-k[66009]: 2024-05-20T16:28:55.201+0000 7fd336b2d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-20T16:28:55.562 INFO:journalctl@ceph.mds.k.smithi160.stdout:May 20 16:28:55 smithi160 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-k[66009]: 2024-05-20T16:28:55.201+0000 7fd336b2d700 -1 mds.k *** got signal Terminated *** 2024-05-20T16:28:58.959 INFO:journalctl@ceph.mds.k.smithi160.stdout:May 20 16:28:58 smithi160 podman[72888]: 2024-05-20 16:28:58.683463291 +0000 UTC m=+3.607178654 container died d601d02073592d37fbf7da1fb7daeb0799a3c21be670fc353d723ad8100a1a49 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-k, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240514, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-20T16:28:59.285 INFO:journalctl@ceph.mds.k.smithi160.stdout:May 20 16:28:59 smithi160 podman[72888]: 2024-05-20 16:28:59.060245549 +0000 UTC m=+3.983960916 container cleanup d601d02073592d37fbf7da1fb7daeb0799a3c21be670fc353d723ad8100a1a49 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-k, org.label-schema.build-date=20240514, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-20T16:28:59.285 INFO:journalctl@ceph.mds.k.smithi160.stdout:May 20 16:28:59 smithi160 bash[72888]: ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-k 2024-05-20T16:28:59.398 DEBUG:teuthology.orchestra.run.smithi160:> sudo pkill -f 'journalctl -f -n 0 -u ceph-59d1600a-16c2-11ef-bc9a-c7b262605968@mds.k.service' 2024-05-20T16:28:59.428 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-20T16:28:59.428 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-05-20T16:28:59.429 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-05-20T16:28:59.429 DEBUG:teuthology.orchestra.run.smithi184:> sudo systemctl stop ceph-59d1600a-16c2-11ef-bc9a-c7b262605968@mds.c 2024-05-20T16:28:59.722 INFO:journalctl@ceph.mds.c.smithi184.stdout:May 20 16:28:59 smithi184 systemd[1]: Stopping Ceph mds.c for 59d1600a-16c2-11ef-bc9a-c7b262605968... 2024-05-20T16:29:00.096 INFO:journalctl@ceph.mds.c.smithi184.stdout:May 20 16:28:59 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-c[71243]: 2024-05-20T16:28:59.721+0000 7f21ed4e3700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-20T16:29:00.096 INFO:journalctl@ceph.mds.c.smithi184.stdout:May 20 16:28:59 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-c[71243]: 2024-05-20T16:28:59.721+0000 7f21ed4e3700 -1 mds.c *** got signal Terminated *** 2024-05-20T16:29:02.314 INFO:journalctl@ceph.mds.c.smithi184.stdout:May 20 16:29:02 smithi184 podman[73853]: 2024-05-20 16:29:02.037980773 +0000 UTC m=+2.417601016 container died 9f97ccd0ffc6c1a5a8a965ef498af031b2f17bcf10c3df5f194fb140c6eb8a14 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-c, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240514, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, org.label-schema.schema-version=1.0) 2024-05-20T16:29:02.635 INFO:journalctl@ceph.mds.c.smithi184.stdout:May 20 16:29:02 smithi184 podman[73853]: 2024-05-20 16:29:02.413745208 +0000 UTC m=+2.793365450 container cleanup 9f97ccd0ffc6c1a5a8a965ef498af031b2f17bcf10c3df5f194fb140c6eb8a14 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-c, org.label-schema.build-date=20240514, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-05-20T16:29:02.635 INFO:journalctl@ceph.mds.c.smithi184.stdout:May 20 16:29:02 smithi184 bash[73853]: ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-c 2024-05-20T16:29:02.803 DEBUG:teuthology.orchestra.run.smithi184:> sudo pkill -f 'journalctl -f -n 0 -u ceph-59d1600a-16c2-11ef-bc9a-c7b262605968@mds.c.service' 2024-05-20T16:29:02.838 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-20T16:29:02.839 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-05-20T16:29:02.839 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-05-20T16:29:02.839 DEBUG:teuthology.orchestra.run.smithi184:> sudo systemctl stop ceph-59d1600a-16c2-11ef-bc9a-c7b262605968@mds.f 2024-05-20T16:29:03.385 INFO:journalctl@ceph.mds.f.smithi184.stdout:May 20 16:29:02 smithi184 systemd[1]: Stopping Ceph mds.f for 59d1600a-16c2-11ef-bc9a-c7b262605968... 2024-05-20T16:29:03.386 INFO:journalctl@ceph.mds.f.smithi184.stdout:May 20 16:29:03 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-f[67090]: 2024-05-20T16:29:03.158+0000 7f455b1c3700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-20T16:29:03.386 INFO:journalctl@ceph.mds.f.smithi184.stdout:May 20 16:29:03 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-f[67090]: 2024-05-20T16:29:03.158+0000 7f455b1c3700 -1 mds.f *** got signal Terminated *** 2024-05-20T16:29:06.292 INFO:journalctl@ceph.mds.f.smithi184.stdout:May 20 16:29:06 smithi184 podman[73986]: 2024-05-20 16:29:06.025372731 +0000 UTC m=+2.961555451 container died 70d70110f88e5e8cf7ccfbbff95d95264f2c1dac9ad285f40b3cdb7dbce6e49c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-f, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240514, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-05-20T16:29:06.635 INFO:journalctl@ceph.mds.f.smithi184.stdout:May 20 16:29:06 smithi184 podman[73986]: 2024-05-20 16:29:06.442965631 +0000 UTC m=+3.379148340 container cleanup 70d70110f88e5e8cf7ccfbbff95d95264f2c1dac9ad285f40b3cdb7dbce6e49c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-f, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240514, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-20T16:29:06.635 INFO:journalctl@ceph.mds.f.smithi184.stdout:May 20 16:29:06 smithi184 bash[73986]: ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-f 2024-05-20T16:29:06.931 DEBUG:teuthology.orchestra.run.smithi184:> sudo pkill -f 'journalctl -f -n 0 -u ceph-59d1600a-16c2-11ef-bc9a-c7b262605968@mds.f.service' 2024-05-20T16:29:06.951 INFO:journalctl@ceph.mds.f.smithi184.stdout:May 20 16:29:06 smithi184 podman[73998]: 2024-05-20 16:29:06.694382406 +0000 UTC m=+0.665426866 container remove 70d70110f88e5e8cf7ccfbbff95d95264f2c1dac9ad285f40b3cdb7dbce6e49c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-f, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240514, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-05-20T16:29:06.952 INFO:journalctl@ceph.mds.f.smithi184.stdout:May 20 16:29:06 smithi184 systemd[1]: ceph-59d1600a-16c2-11ef-bc9a-c7b262605968@mds.f.service: Deactivated successfully. 2024-05-20T16:29:06.952 INFO:journalctl@ceph.mds.f.smithi184.stdout:May 20 16:29:06 smithi184 systemd[1]: Stopped Ceph mds.f for 59d1600a-16c2-11ef-bc9a-c7b262605968. 2024-05-20T16:29:06.952 INFO:journalctl@ceph.mds.f.smithi184.stdout:May 20 16:29:06 smithi184 systemd[1]: ceph-59d1600a-16c2-11ef-bc9a-c7b262605968@mds.f.service: Consumed 2.335s CPU time. 2024-05-20T16:29:06.963 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-20T16:29:06.964 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-05-20T16:29:06.964 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-05-20T16:29:06.964 DEBUG:teuthology.orchestra.run.smithi184:> sudo systemctl stop ceph-59d1600a-16c2-11ef-bc9a-c7b262605968@mds.i 2024-05-20T16:29:07.385 INFO:journalctl@ceph.mds.i.smithi184.stdout:May 20 16:29:07 smithi184 systemd[1]: Stopping Ceph mds.i for 59d1600a-16c2-11ef-bc9a-c7b262605968... 2024-05-20T16:29:07.885 INFO:journalctl@ceph.mds.i.smithi184.stdout:May 20 16:29:07 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-i[67705]: 2024-05-20T16:29:07.438+0000 7f4ddeaaa700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-20T16:29:07.885 INFO:journalctl@ceph.mds.i.smithi184.stdout:May 20 16:29:07 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-i[67705]: 2024-05-20T16:29:07.438+0000 7f4ddeaaa700 -1 mds.i *** got signal Terminated *** 2024-05-20T16:29:10.313 INFO:journalctl@ceph.mds.i.smithi184.stdout:May 20 16:29:10 smithi184 podman[74109]: 2024-05-20 16:29:10.030252997 +0000 UTC m=+2.839329079 container died a27a46359d78869b0f838831347de84fe8bf0ccc74e75cee82130b40a2ad089c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-i, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240514, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-20T16:29:10.635 INFO:journalctl@ceph.mds.i.smithi184.stdout:May 20 16:29:10 smithi184 podman[74109]: 2024-05-20 16:29:10.405202693 +0000 UTC m=+3.214278765 container cleanup a27a46359d78869b0f838831347de84fe8bf0ccc74e75cee82130b40a2ad089c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-i, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240514, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-20T16:29:10.635 INFO:journalctl@ceph.mds.i.smithi184.stdout:May 20 16:29:10 smithi184 bash[74109]: ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-i 2024-05-20T16:29:10.780 DEBUG:teuthology.orchestra.run.smithi184:> sudo pkill -f 'journalctl -f -n 0 -u ceph-59d1600a-16c2-11ef-bc9a-c7b262605968@mds.i.service' 2024-05-20T16:29:10.811 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-20T16:29:10.812 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-05-20T16:29:10.812 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-05-20T16:29:10.812 DEBUG:teuthology.orchestra.run.smithi184:> sudo systemctl stop ceph-59d1600a-16c2-11ef-bc9a-c7b262605968@mds.l 2024-05-20T16:29:10.919 INFO:journalctl@ceph.mds.l.smithi184.stdout:May 20 16:29:10 smithi184 systemd[1]: Stopping Ceph mds.l for 59d1600a-16c2-11ef-bc9a-c7b262605968... 2024-05-20T16:29:11.385 INFO:journalctl@ceph.mds.l.smithi184.stdout:May 20 16:29:11 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-l[66787]: 2024-05-20T16:29:11.132+0000 7fa8f423c700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-20T16:29:11.385 INFO:journalctl@ceph.mds.l.smithi184.stdout:May 20 16:29:11 smithi184 ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-l[66787]: 2024-05-20T16:29:11.132+0000 7fa8f423c700 -1 mds.l *** got signal Terminated *** 2024-05-20T16:29:15.450 INFO:journalctl@ceph.mds.l.smithi184.stdout:May 20 16:29:15 smithi184 podman[74232]: 2024-05-20 16:29:15.138443648 +0000 UTC m=+4.103242870 container died e6e63d62472b9a95f4f78cbe4c2e809a26c82deaeb05d529bb3fe28c860b3b9a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-l, org.label-schema.build-date=20240514, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux ) 2024-05-20T16:29:15.766 INFO:journalctl@ceph.mds.l.smithi184.stdout:May 20 16:29:15 smithi184 podman[74232]: 2024-05-20 16:29:15.550236713 +0000 UTC m=+4.515035911 container cleanup e6e63d62472b9a95f4f78cbe4c2e809a26c82deaeb05d529bb3fe28c860b3b9a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-l, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240514, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-05-20T16:29:15.767 INFO:journalctl@ceph.mds.l.smithi184.stdout:May 20 16:29:15 smithi184 bash[74232]: ceph-59d1600a-16c2-11ef-bc9a-c7b262605968-mds-l 2024-05-20T16:29:15.897 DEBUG:teuthology.orchestra.run.smithi184:> sudo pkill -f 'journalctl -f -n 0 -u ceph-59d1600a-16c2-11ef-bc9a-c7b262605968@mds.l.service' 2024-05-20T16:29:15.928 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-20T16:29:15.928 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-05-20T16:29:15.928 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 59d1600a-16c2-11ef-bc9a-c7b262605968 --force --keep-logs 2024-05-20T16:29:16.120 INFO:teuthology.orchestra.run.smithi082.stdout:Deleting cluster with fsid: 59d1600a-16c2-11ef-bc9a-c7b262605968 2024-05-20T16:29:18.713 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 59d1600a-16c2-11ef-bc9a-c7b262605968 --force --keep-logs 2024-05-20T16:29:18.902 INFO:teuthology.orchestra.run.smithi160.stdout:Deleting cluster with fsid: 59d1600a-16c2-11ef-bc9a-c7b262605968 2024-05-20T16:29:21.388 DEBUG:teuthology.orchestra.run.smithi184:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 59d1600a-16c2-11ef-bc9a-c7b262605968 --force --keep-logs 2024-05-20T16:29:21.576 INFO:teuthology.orchestra.run.smithi184.stdout:Deleting cluster with fsid: 59d1600a-16c2-11ef-bc9a-c7b262605968 2024-05-20T16:29:23.977 DEBUG:teuthology.orchestra.run.smithi082:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-20T16:29:24.010 DEBUG:teuthology.orchestra.run.smithi160:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-20T16:29:24.041 DEBUG:teuthology.orchestra.run.smithi184:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-20T16:29:24.067 INFO:tasks.cephadm:Archiving crash dumps... 2024-05-20T16:29:24.068 DEBUG:teuthology.misc:Transferring archived files from smithi082:/var/lib/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/crash to /home/teuthworker/archive/pdonnell-2024-05-20_15:44:59-fs-wip-pdonnell-testing-20240517.203521-squid-debug-distro-default-smithi/7715545/remote/smithi082/crash 2024-05-20T16:29:24.069 DEBUG:teuthology.orchestra.run.smithi082:> sudo tar c -f - -C /var/lib/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/crash -- . 2024-05-20T16:29:24.101 INFO:teuthology.orchestra.run.smithi082.stderr:tar: /var/lib/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/crash: Cannot open: No such file or directory 2024-05-20T16:29:24.102 INFO:teuthology.orchestra.run.smithi082.stderr:tar: Error is not recoverable: exiting now 2024-05-20T16:29:24.104 DEBUG:teuthology.misc:Transferring archived files from smithi160:/var/lib/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/crash to /home/teuthworker/archive/pdonnell-2024-05-20_15:44:59-fs-wip-pdonnell-testing-20240517.203521-squid-debug-distro-default-smithi/7715545/remote/smithi160/crash 2024-05-20T16:29:24.105 DEBUG:teuthology.orchestra.run.smithi160:> sudo tar c -f - -C /var/lib/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/crash -- . 2024-05-20T16:29:24.147 INFO:teuthology.orchestra.run.smithi160.stderr:tar: /var/lib/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/crash: Cannot open: No such file or directory 2024-05-20T16:29:24.147 INFO:teuthology.orchestra.run.smithi160.stderr:tar: Error is not recoverable: exiting now 2024-05-20T16:29:24.149 DEBUG:teuthology.misc:Transferring archived files from smithi184:/var/lib/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/crash to /home/teuthworker/archive/pdonnell-2024-05-20_15:44:59-fs-wip-pdonnell-testing-20240517.203521-squid-debug-distro-default-smithi/7715545/remote/smithi184/crash 2024-05-20T16:29:24.150 DEBUG:teuthology.orchestra.run.smithi184:> sudo tar c -f - -C /var/lib/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/crash -- . 2024-05-20T16:29:24.186 INFO:teuthology.orchestra.run.smithi184.stderr:tar: /var/lib/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/crash: Cannot open: No such file or directory 2024-05-20T16:29:24.186 INFO:teuthology.orchestra.run.smithi184.stderr:tar: Error is not recoverable: exiting now 2024-05-20T16:29:24.187 INFO:tasks.cephadm:Checking cluster log for badness... 2024-05-20T16:29:24.188 DEBUG:teuthology.orchestra.run.smithi082:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | head -n 1 2024-05-20T16:29:24.279 INFO:tasks.cephadm:Compressing logs... 2024-05-20T16:29:24.279 DEBUG:teuthology.orchestra.run.smithi082:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-20T16:29:24.281 DEBUG:teuthology.orchestra.run.smithi160:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-20T16:29:24.284 DEBUG:teuthology.orchestra.run.smithi184:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-20T16:29:24.310 INFO:teuthology.orchestra.run.smithi082.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-05-20T16:29:24.311 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/ceph.log 2024-05-20T16:29:24.311 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/ceph.audit.log 2024-05-20T16:29:24.312 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/ceph-mds.g.log 2024-05-20T16:29:24.312 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/ceph-mds.j.log 2024-05-20T16:29:24.312 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/ceph-osd.3.log 2024-05-20T16:29:24.315 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/ceph-osd.0.log 2024-05-20T16:29:24.315 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/ceph-mgr.x.log 2024-05-20T16:29:24.315 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/ceph-mon.a.log 2024-05-20T16:29:24.315 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/ceph-osd.6.log 2024-05-20T16:29:24.316 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/ceph.cephadm.log 2024-05-20T16:29:24.316 INFO:teuthology.orchestra.run.smithi082.stderr: 86.7% -- replaced with /var/log/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/ceph.log.gz 2024-05-20T16:29:24.316 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/ceph-mds.a.log 2024-05-20T16:29:24.316 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/ceph-volume.log 2024-05-20T16:29:24.316 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/ceph-mds.d.log 2024-05-20T16:29:24.316 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/ceph-osd.9.log 2024-05-20T16:29:24.316 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-05-20T16:29:24.316 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/ceph-osd.9.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.log 2024-05-20T16:29:24.317 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/cephadm.log: 94.5% -- replaced with /var/log/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/ceph-mds.j.log.gz 2024-05-20T16:29:24.317 INFO:teuthology.orchestra.run.smithi160.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-05-20T16:29:24.318 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.log: 94.6% -- replaced with /var/log/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/ceph-mds.a.log.gz 2024-05-20T16:29:24.318 INFO:teuthology.orchestra.run.smithi160.stderr:gzip -5 --verbose -- /var/log/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/ceph.log 2024-05-20T16:29:24.323 INFO:teuthology.orchestra.run.smithi160.stderr:gzip -5 --verbose -- /var/log/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/ceph.audit.log 2024-05-20T16:29:24.323 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/ceph-osd.7.log 2024-05-20T16:29:24.323 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/ceph-mds.b.log 2024-05-20T16:29:24.323 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/ceph-osd.4.log 2024-05-20T16:29:24.323 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/ceph-osd.1.log 2024-05-20T16:29:24.323 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/ceph-mgr.y.log 2024-05-20T16:29:24.323 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/ceph-mds.h.log 2024-05-20T16:29:24.323 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/ceph-osd.10.log 2024-05-20T16:29:24.323 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/ceph.cephadm.log 2024-05-20T16:29:24.323 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/ceph-mds.e.log 2024-05-20T16:29:24.323 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/ceph-volume.log 2024-05-20T16:29:24.324 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/ceph-mds.e.log: gzip -5 --verbose -- /var/log/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/ceph-mon.b.log 2024-05-20T16:29:24.324 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/ceph-mds.k.log 2024-05-20T16:29:24.324 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-05-20T16:29:24.325 INFO:teuthology.orchestra.run.smithi184.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-05-20T16:29:24.325 INFO:teuthology.orchestra.run.smithi184.stderr:gzip -5 --verbose -- /var/log/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/ceph.log 2024-05-20T16:29:24.325 INFO:teuthology.orchestra.run.smithi184.stderr:gzip -5 --verbose -- /var/log/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/ceph-osd.2.log 2024-05-20T16:29:24.325 INFO:teuthology.orchestra.run.smithi184.stderr:gzip -5 --verbose -- /var/log/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/ceph-mds.f.log 2024-05-20T16:29:24.325 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/ceph.log: /var/log/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/ceph.audit.log 2024-05-20T16:29:24.328 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/ceph-mds.f.log: gzip -5 --verbose -- /var/log/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/ceph-osd.5.log 2024-05-20T16:29:24.328 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/ceph-mds.c.log 2024-05-20T16:29:24.328 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/ceph-mon.c.log 2024-05-20T16:29:24.328 INFO:teuthology.orchestra.run.smithi184.stderr:gzip/var/log/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/ceph-mds.c.log: -5 --verbose -- /var/log/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/ceph-mgr.z.log 2024-05-20T16:29:24.328 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/ceph-mds.i.log 2024-05-20T16:29:24.328 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/ceph-mgr.z.log: gzip -5 --verbose -- /var/log/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/ceph.cephadm.log 2024-05-20T16:29:24.328 INFO:teuthology.orchestra.run.smithi184.stderr:gzip -5 --verbose -- /var/log/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/ceph-osd.11.log 2024-05-20T16:29:24.328 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/ceph-mds.i.log: /var/log/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/ceph-osd.8.log 2024-05-20T16:29:24.328 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/ceph-volume.log 2024-05-20T16:29:24.328 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/ceph-mds.l.log 2024-05-20T16:29:24.328 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-05-20T16:29:24.328 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/ceph-mds.l.log: /var/log/ceph/cephadm.log: 94.5% -- replaced with /var/log/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/ceph-mds.l.log.gz 2024-05-20T16:29:24.388 INFO:teuthology.orchestra.run.smithi082.stderr: 92.6% -- replaced with /var/log/ceph/ceph-client.admin.log.gz 2024-05-20T16:29:24.404 INFO:teuthology.orchestra.run.smithi082.stderr: 90.0% -- replaced with /var/log/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/ceph.audit.log.gz 2024-05-20T16:29:24.434 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/ceph-mds.k.log: /var/log/ceph/cephadm.log: 92.9% -- replaced with /var/log/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/ceph-mgr.y.log.gz 2024-05-20T16:29:24.443 INFO:teuthology.orchestra.run.smithi082.stderr: 94.5% -- replaced with /var/log/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/ceph-mds.g.log.gz 2024-05-20T16:29:24.494 INFO:teuthology.orchestra.run.smithi160.stderr: 86.7% -- replaced with /var/log/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/ceph.log.gz 2024-05-20T16:29:24.506 INFO:teuthology.orchestra.run.smithi082.stderr: 93.3% -- replaced with /var/log/ceph/cephadm.log.gz 2024-05-20T16:29:24.512 INFO:teuthology.orchestra.run.smithi184.stderr: 82.5% -- replaced with /var/log/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/ceph.cephadm.log.gz 2024-05-20T16:29:24.514 INFO:teuthology.orchestra.run.smithi082.stderr: 91.1% -- replaced with /var/log/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/ceph-volume.log.gz 2024-05-20T16:29:24.548 INFO:teuthology.orchestra.run.smithi082.stderr: 82.9% -- replaced with /var/log/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/ceph.cephadm.log.gz 2024-05-20T16:29:24.570 INFO:teuthology.orchestra.run.smithi160.stderr: 91.1% -- replaced with /var/log/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/ceph-volume.log.gz 2024-05-20T16:29:24.589 INFO:teuthology.orchestra.run.smithi160.stderr: 90.1% -- replaced with /var/log/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/ceph.audit.log.gz 2024-05-20T16:29:24.684 INFO:teuthology.orchestra.run.smithi184.stderr: 91.0% -- replaced with /var/log/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/ceph-volume.log.gz 2024-05-20T16:29:24.714 INFO:teuthology.orchestra.run.smithi184.stderr: 94.6% -- replaced with /var/log/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/ceph-mds.f.log.gz 2024-05-20T16:29:24.764 INFO:teuthology.orchestra.run.smithi184.stderr: 86.7% 90.2% 92.4% -- replaced with /var/log/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/ceph.audit.log.gz -- replaced with /var/log/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/ceph.log.gz -- replaced with /var/log/ceph/cephadm.log.gz 2024-05-20T16:29:24.764 INFO:teuthology.orchestra.run.smithi184.stderr: 2024-05-20T16:29:24.764 INFO:teuthology.orchestra.run.smithi184.stderr: 2024-05-20T16:29:24.780 INFO:teuthology.orchestra.run.smithi160.stderr: 92.1% -- replaced with /var/log/ceph/cephadm.log.gz 2024-05-20T16:29:24.781 INFO:teuthology.orchestra.run.smithi184.stderr: 93.0% -- replaced with /var/log/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/ceph-mgr.z.log.gz 2024-05-20T16:29:25.021 INFO:teuthology.orchestra.run.smithi160.stderr: 82.5% -- replaced with /var/log/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/ceph.cephadm.log.gz 2024-05-20T16:29:25.062 INFO:teuthology.orchestra.run.smithi082.stderr: 94.6% -- replaced with /var/log/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/ceph-mds.d.log.gz 2024-05-20T16:29:25.103 INFO:teuthology.orchestra.run.smithi184.stderr: 93.7% -- replaced with /var/log/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/ceph-mds.i.log.gz 2024-05-20T16:29:25.532 INFO:teuthology.orchestra.run.smithi160.stderr: 93.2% -- replaced with /var/log/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/ceph-mds.h.log.gz 2024-05-20T16:29:25.704 INFO:teuthology.orchestra.run.smithi160.stderr: 94.5% -- replaced with /var/log/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/ceph-mds.k.log.gz 2024-05-20T16:29:26.466 INFO:teuthology.orchestra.run.smithi082.stderr: 90.3% -- replaced with /var/log/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/ceph-mgr.x.log.gz 2024-05-20T16:29:27.284 INFO:teuthology.orchestra.run.smithi160.stderr: 94.3% -- replaced with /var/log/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/ceph-mds.e.log.gz 2024-05-20T16:29:30.467 INFO:teuthology.orchestra.run.smithi160.stderr: 91.8% -- replaced with /var/log/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/ceph-mon.b.log.gz 2024-05-20T16:29:33.938 INFO:teuthology.orchestra.run.smithi184.stderr: 91.8% -- replaced with /var/log/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/ceph-mon.c.log.gz 2024-05-20T16:29:34.408 INFO:teuthology.orchestra.run.smithi082.stderr: 90.4% -- replaced with /var/log/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/ceph-mon.a.log.gz 2024-05-20T16:36:44.089 INFO:teuthology.orchestra.run.smithi184.stderr: 97.3% -- replaced with /var/log/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/ceph-mds.c.log.gz 2024-05-20T16:44:54.789 INFO:teuthology.orchestra.run.smithi184.stderr: 92.9% -- replaced with /var/log/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/ceph-osd.8.log.gz 2024-05-20T16:47:42.198 INFO:teuthology.orchestra.run.smithi184.stderr: 92.7% -- replaced with /var/log/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/ceph-osd.11.log.gz 2024-05-20T16:48:51.368 INFO:teuthology.orchestra.run.smithi160.stderr: 92.8% -- replaced with /var/log/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/ceph-osd.10.log.gz 2024-05-20T16:50:21.949 INFO:teuthology.orchestra.run.smithi082.stderr: 92.8% -- replaced with /var/log/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/ceph-osd.0.log.gz 2024-05-20T16:50:30.297 INFO:teuthology.orchestra.run.smithi184.stderr: 92.7% -- replaced with /var/log/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/ceph-osd.2.log.gz 2024-05-20T16:51:46.219 INFO:teuthology.orchestra.run.smithi184.stderr: 92.6% -- replaced with /var/log/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/ceph-osd.5.log.gz 2024-05-20T16:51:46.273 INFO:teuthology.orchestra.run.smithi184.stderr: 2024-05-20T16:51:46.273 INFO:teuthology.orchestra.run.smithi184.stderr:real 22m21.975s 2024-05-20T16:51:46.273 INFO:teuthology.orchestra.run.smithi184.stderr:user 7m27.931s 2024-05-20T16:51:46.274 INFO:teuthology.orchestra.run.smithi184.stderr:sys 0m23.215s 2024-05-20T16:52:29.002 INFO:teuthology.orchestra.run.smithi082.stderr: 92.6% -- replaced with /var/log/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/ceph-osd.3.log.gz 2024-05-20T16:54:21.048 INFO:teuthology.orchestra.run.smithi082.stderr: 92.6% -- replaced with /var/log/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/ceph-osd.6.log.gz 2024-05-20T16:54:53.066 INFO:teuthology.orchestra.run.smithi082.stderr: 92.4% -- replaced with /var/log/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/ceph-osd.9.log.gz 2024-05-20T16:54:53.068 INFO:teuthology.orchestra.run.smithi082.stderr: 2024-05-20T16:54:53.068 INFO:teuthology.orchestra.run.smithi082.stderr:real 25m28.773s 2024-05-20T16:54:53.068 INFO:teuthology.orchestra.run.smithi082.stderr:user 8m49.892s 2024-05-20T16:54:53.068 INFO:teuthology.orchestra.run.smithi082.stderr:sys 0m26.430s 2024-05-20T16:56:02.235 INFO:teuthology.orchestra.run.smithi160.stderr: 92.7% -- replaced with /var/log/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/ceph-osd.1.log.gz 2024-05-20T16:58:51.023 INFO:teuthology.orchestra.run.smithi160.stderr: 92.7% -- replaced with /var/log/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/ceph-osd.4.log.gz 2024-05-20T17:00:43.085 INFO:teuthology.orchestra.run.smithi160.stderr: 92.7% -- replaced with /var/log/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/ceph-osd.7.log.gz 2024-05-20T17:02:01.110 INFO:teuthology.orchestra.run.smithi160.stderr: 94.4% -- replaced with /var/log/ceph/59d1600a-16c2-11ef-bc9a-c7b262605968/ceph-mds.b.log.gz 2024-05-20T17:02:01.112 INFO:teuthology.orchestra.run.smithi160.stderr: 2024-05-20T17:02:01.112 INFO:teuthology.orchestra.run.smithi160.stderr:real 32m36.814s 2024-05-20T17:02:01.112 INFO:teuthology.orchestra.run.smithi160.stderr:user 10m20.276s 2024-05-20T17:02:01.112 INFO:teuthology.orchestra.run.smithi160.stderr:sys 0m32.282s 2024-05-20T17:02:01.113 INFO:tasks.cephadm:Archiving logs... 2024-05-20T17:02:01.114 DEBUG:teuthology.misc:Transferring archived files from smithi082:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-05-20_15:44:59-fs-wip-pdonnell-testing-20240517.203521-squid-debug-distro-default-smithi/7715545/remote/smithi082/log 2024-05-20T17:02:01.115 DEBUG:teuthology.orchestra.run.smithi082:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-20T17:03:25.058 DEBUG:teuthology.misc:Transferring archived files from smithi160:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-05-20_15:44:59-fs-wip-pdonnell-testing-20240517.203521-squid-debug-distro-default-smithi/7715545/remote/smithi160/log 2024-05-20T17:03:25.059 DEBUG:teuthology.orchestra.run.smithi160:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-20T17:04:57.714 DEBUG:teuthology.misc:Transferring archived files from smithi184:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-05-20_15:44:59-fs-wip-pdonnell-testing-20240517.203521-squid-debug-distro-default-smithi/7715545/remote/smithi184/log 2024-05-20T17:04:57.716 DEBUG:teuthology.orchestra.run.smithi184:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-20T17:06:09.211 INFO:tasks.cephadm:Removing cluster... 2024-05-20T17:06:09.212 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 59d1600a-16c2-11ef-bc9a-c7b262605968 --force 2024-05-20T17:06:10.512 INFO:teuthology.orchestra.run.smithi082.stdout:Deleting cluster with fsid: 59d1600a-16c2-11ef-bc9a-c7b262605968 2024-05-20T17:06:11.323 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 59d1600a-16c2-11ef-bc9a-c7b262605968 --force 2024-05-20T17:06:11.882 INFO:teuthology.orchestra.run.smithi160.stdout:Deleting cluster with fsid: 59d1600a-16c2-11ef-bc9a-c7b262605968 2024-05-20T17:06:12.742 DEBUG:teuthology.orchestra.run.smithi184:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 59d1600a-16c2-11ef-bc9a-c7b262605968 --force 2024-05-20T17:06:14.118 INFO:teuthology.orchestra.run.smithi184.stdout:Deleting cluster with fsid: 59d1600a-16c2-11ef-bc9a-c7b262605968 2024-05-20T17:06:14.980 INFO:tasks.cephadm:Removing cephadm ... 2024-05-20T17:06:14.980 DEBUG:teuthology.orchestra.run.smithi082:> rm -rf /home/ubuntu/cephtest/cephadm 2024-05-20T17:06:15.010 DEBUG:teuthology.orchestra.run.smithi160:> rm -rf /home/ubuntu/cephtest/cephadm 2024-05-20T17:06:15.029 DEBUG:teuthology.orchestra.run.smithi184:> rm -rf /home/ubuntu/cephtest/cephadm 2024-05-20T17:06:15.045 INFO:tasks.cephadm:Teardown complete 2024-05-20T17:06:15.045 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-05-20T17:06:15.057 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-05-20T17:06:15.057 DEBUG:teuthology.orchestra.run.smithi082:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-20T17:06:15.060 DEBUG:teuthology.orchestra.run.smithi160:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-20T17:06:15.072 DEBUG:teuthology.orchestra.run.smithi184:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-20T17:06:15.233 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-05-20T17:06:15.243 INFO:teuthology.task.clock:Checking final clock skew... 2024-05-20T17:06:15.244 DEBUG:teuthology.orchestra.run.smithi082:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-20T17:06:15.247 DEBUG:teuthology.orchestra.run.smithi160:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-20T17:06:15.249 DEBUG:teuthology.orchestra.run.smithi184:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-20T17:06:15.261 INFO:teuthology.orchestra.run.smithi082.stderr:bash: line 1: ntpq: command not found 2024-05-20T17:06:15.263 INFO:teuthology.orchestra.run.smithi184.stderr:bash: line 1: ntpq: command not found 2024-05-20T17:06:15.264 INFO:teuthology.orchestra.run.smithi160.stderr:bash: line 1: ntpq: command not found 2024-05-20T17:06:15.306 INFO:teuthology.orchestra.run.smithi160.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-20T17:06:15.307 INFO:teuthology.orchestra.run.smithi160.stdout:=============================================================================== 2024-05-20T17:06:15.307 INFO:teuthology.orchestra.run.smithi160.stdout:^* hv01.front.sepia.ceph.com 3 6 377 17 +2582us[+2568us] +/- 64ms 2024-05-20T17:06:15.307 INFO:teuthology.orchestra.run.smithi160.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 14 -1147us[-1147us] +/- 76ms 2024-05-20T17:06:15.307 INFO:teuthology.orchestra.run.smithi160.stdout:^+ hv03.front.sepia.ceph.com 3 7 377 71 -2006us[-2018us] +/- 89ms 2024-05-20T17:06:15.307 INFO:teuthology.orchestra.run.smithi160.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2024-05-20T17:06:15.337 INFO:teuthology.orchestra.run.smithi184.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-20T17:06:15.337 INFO:teuthology.orchestra.run.smithi184.stdout:=============================================================================== 2024-05-20T17:06:15.337 INFO:teuthology.orchestra.run.smithi184.stdout:^* hv01.front.sepia.ceph.com 3 6 377 12 +2478us[+2476us] +/- 64ms 2024-05-20T17:06:15.337 INFO:teuthology.orchestra.run.smithi184.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 14 -1241us[-1243us] +/- 76ms 2024-05-20T17:06:15.338 INFO:teuthology.orchestra.run.smithi184.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 12 -2154us[-2155us] +/- 90ms 2024-05-20T17:06:15.338 INFO:teuthology.orchestra.run.smithi184.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2024-05-20T17:06:15.340 INFO:teuthology.orchestra.run.smithi082.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-20T17:06:15.340 INFO:teuthology.orchestra.run.smithi082.stdout:=============================================================================== 2024-05-20T17:06:15.340 INFO:teuthology.orchestra.run.smithi082.stdout:^* hv01.front.sepia.ceph.com 3 6 377 11 +2456us[+2453us] +/- 64ms 2024-05-20T17:06:15.340 INFO:teuthology.orchestra.run.smithi082.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 10 -1257us[-1257us] +/- 76ms 2024-05-20T17:06:15.340 INFO:teuthology.orchestra.run.smithi082.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 8 -2166us[-2166us] +/- 90ms 2024-05-20T17:06:15.340 INFO:teuthology.orchestra.run.smithi082.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2024-05-20T17:06:15.350 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-05-20T17:06:15.360 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-05-20T17:06:15.361 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-05-20T17:06:15.370 DEBUG:teuthology.orchestra.run.smithi082:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-20T17:06:15.561 DEBUG:teuthology.orchestra.run.smithi160:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-20T17:06:15.643 DEBUG:teuthology.orchestra.run.smithi184:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-20T17:06:15.765 DEBUG:teuthology.orchestra.run.smithi082:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-20T17:06:15.797 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-20T17:06:15.797 DEBUG:teuthology.orchestra.run.smithi160:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-20T17:06:15.829 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-20T17:06:15.829 DEBUG:teuthology.orchestra.run.smithi184:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-20T17:06:15.855 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-20T17:06:15.856 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-05-20T17:06:15.867 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-05-20T17:06:15.877 INFO:teuthology.task.internal:Duration was 4238.544281 seconds 2024-05-20T17:06:15.877 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-05-20T17:06:15.888 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-05-20T17:06:15.888 DEBUG:teuthology.orchestra.run.smithi082:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-20T17:06:15.891 DEBUG:teuthology.orchestra.run.smithi160:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-20T17:06:15.893 DEBUG:teuthology.orchestra.run.smithi184:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-20T17:06:15.963 INFO:teuthology.orchestra.run.smithi082.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-20T17:06:15.968 INFO:teuthology.orchestra.run.smithi160.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-20T17:06:15.983 INFO:teuthology.orchestra.run.smithi184.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-20T17:06:16.463 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-05-20T17:06:16.463 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi082.front.sepia.ceph.com 2024-05-20T17:06:16.463 DEBUG:teuthology.orchestra.run.smithi082:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-20T17:06:16.491 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi160.front.sepia.ceph.com 2024-05-20T17:06:16.491 DEBUG:teuthology.orchestra.run.smithi160:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-20T17:06:16.521 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi184.front.sepia.ceph.com 2024-05-20T17:06:16.522 DEBUG:teuthology.orchestra.run.smithi184:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-20T17:06:16.556 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-05-20T17:06:16.556 DEBUG:teuthology.orchestra.run.smithi082:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-20T17:06:16.559 DEBUG:teuthology.orchestra.run.smithi160:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-20T17:06:16.564 DEBUG:teuthology.orchestra.run.smithi184:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-20T17:06:16.641 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-05-20T17:06:16.641 DEBUG:teuthology.orchestra.run.smithi082:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-20T17:06:16.644 DEBUG:teuthology.orchestra.run.smithi160:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-20T17:06:16.647 DEBUG:teuthology.orchestra.run.smithi184:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-20T17:06:17.036 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-05-20T17:06:17.055 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-05-20T17:06:17.055 DEBUG:teuthology.orchestra.run.smithi082:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-20T17:06:17.081 DEBUG:teuthology.orchestra.run.smithi160:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-20T17:06:17.110 DEBUG:teuthology.orchestra.run.smithi184:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-20T17:06:17.150 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-05-20T17:06:17.163 DEBUG:teuthology.orchestra.run.smithi082:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-20T17:06:17.165 DEBUG:teuthology.orchestra.run.smithi160:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-20T17:06:17.168 DEBUG:teuthology.orchestra.run.smithi184:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-20T17:06:17.195 INFO:teuthology.orchestra.run.smithi082.stdout:kernel.core_pattern = core 2024-05-20T17:06:17.200 INFO:teuthology.orchestra.run.smithi160.stdout:kernel.core_pattern = core 2024-05-20T17:06:17.218 INFO:teuthology.orchestra.run.smithi184.stdout:kernel.core_pattern = core 2024-05-20T17:06:17.274 DEBUG:teuthology.orchestra.run.smithi082:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-20T17:06:17.289 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-20T17:06:17.289 DEBUG:teuthology.orchestra.run.smithi160:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-20T17:06:17.303 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-20T17:06:17.303 DEBUG:teuthology.orchestra.run.smithi184:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-20T17:06:17.328 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-20T17:06:17.329 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-05-20T17:06:17.340 INFO:teuthology.task.internal:Transferring archived files... 2024-05-20T17:06:17.341 DEBUG:teuthology.misc:Transferring archived files from smithi082:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-05-20_15:44:59-fs-wip-pdonnell-testing-20240517.203521-squid-debug-distro-default-smithi/7715545/remote/smithi082 2024-05-20T17:06:17.343 DEBUG:teuthology.orchestra.run.smithi082:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-20T17:06:17.386 DEBUG:teuthology.misc:Transferring archived files from smithi160:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-05-20_15:44:59-fs-wip-pdonnell-testing-20240517.203521-squid-debug-distro-default-smithi/7715545/remote/smithi160 2024-05-20T17:06:17.387 DEBUG:teuthology.orchestra.run.smithi160:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-20T17:06:17.428 DEBUG:teuthology.misc:Transferring archived files from smithi184:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-05-20_15:44:59-fs-wip-pdonnell-testing-20240517.203521-squid-debug-distro-default-smithi/7715545/remote/smithi184 2024-05-20T17:06:17.429 DEBUG:teuthology.orchestra.run.smithi184:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-20T17:06:17.466 INFO:teuthology.task.internal:Removing archive directory... 2024-05-20T17:06:17.466 DEBUG:teuthology.orchestra.run.smithi082:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-20T17:06:17.470 DEBUG:teuthology.orchestra.run.smithi160:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-20T17:06:17.472 DEBUG:teuthology.orchestra.run.smithi184:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-20T17:06:17.512 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-05-20T17:06:17.523 INFO:teuthology.task.internal:Not uploading archives. 2024-05-20T17:06:17.524 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-05-20T17:06:17.534 INFO:teuthology.task.internal:Tidying up after the test... 2024-05-20T17:06:17.535 DEBUG:teuthology.orchestra.run.smithi082:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-20T17:06:17.537 DEBUG:teuthology.orchestra.run.smithi160:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-20T17:06:17.540 DEBUG:teuthology.orchestra.run.smithi184:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-20T17:06:17.551 INFO:teuthology.orchestra.run.smithi082.stdout: 265295 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 20 17:06 /home/ubuntu/cephtest 2024-05-20T17:06:17.552 INFO:teuthology.orchestra.run.smithi160.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 20 17:06 /home/ubuntu/cephtest 2024-05-20T17:06:17.567 INFO:teuthology.orchestra.run.smithi184.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 20 17:06 /home/ubuntu/cephtest 2024-05-20T17:06:17.568 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-05-20T17:06:17.580 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-05-20T17:06:17.605 INFO:teuthology.run:Summary data: description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/no 3-snaps/no 4-flush/yes 5-workunit/suites/iogen}} duration: 4238.544281005859 flavor: default owner: scheduled_pdonnell@teuthology success: true 2024-05-20T17:06:17.605 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-05-20T17:06:17.682 INFO:teuthology.run:pass